We respect attorney-client confidentiality. No tracking pixels in our emails.
Seven essential questions to ask any legal AI vendor before signing a contract. Covers data training, SOC 2, hallucination rates, acquisition risk, and SLAs.
The sales rep has had three calls with your team, sent a 47-slide deck, and arranged a custom demo. You still do not know if their tool trains on your client data. The questions most firms forget to ask cost them later.
This is our due diligence guide for evaluating legal AI vendors in 2026, written for attorneys, legal ops managers, and IT security professionals responsible for vendor selection.
LawyerAI built this guide. We earn no affiliate revenue from these tools.
Here are the 4 rules we set for ourselves before writing this:
We re-review this list every quarter.
Short answer: The seven questions below address the issues that matter most for legal AI vendor selection: data training practices, SOC 2 certification, data residency, hallucination rate methodology, acquisition risk, audit log availability, and SLA terms. Most vendors answer the easy questions well in demos. These seven questions surface the answers that matter for professional responsibility, malpractice risk, and strategic commitment. Get answers in writing — verbal assurances from sales reps are not enforceable.
LawyerAI scores every tool across five dimensions at /methodology. Security — which includes data training practices, SOC 2 status, zero-retention policies, and DPA terms — is one of the five dimensions. Tools that do not publish their data handling practices or that lack SOC 2 Type II certification score lower on Security regardless of their other capabilities. See our data-processing-agreement glossary entry for what a legally adequate DPA for AI tools includes.
| Tool | SOC 2 Type II | No Training Commitment | Data Residency Published |
|---|---|---|---|
| Harvey AI | Yes | Yes (explicit DPA) | Yes |
| Lexis+ AI | Yes (RELX Group) | Yes | US and EU options |
| Spellbook | Yes | Yes | US |
| Evisort | Yes (Workday) | Yes | US; EU options |
| Ironclad | Yes | Yes | US; EU options |
All information above is vendor-reported and subject to change. Verify current DPA terms directly with each vendor.
This is the most important question in the evaluation. The answer has direct implications for client confidentiality under Model Rule 1.6 and attorney-client privilege.
Ask specifically: "Does your platform use customer data — including documents, queries, and outputs — to train, fine-tune, or improve your models?"
Four dodge patterns to watch for:
"We anonymize the data." Anonymization is a technical process with known failure modes. Anonymized legal documents often contain enough context to be re-identified. Anonymization is not the same as no-training. Follow up: "Is the anonymized data used to update the model?"
"We aggregate data across customers." Aggregation reduces privacy risk but does not eliminate training. The question is whether your data is in the training set at all.
"Training is opt-out by default." This means you are training by default unless you actively configure otherwise. Opt-out requires your organization to take action; opt-in does not. Know which one you are agreeing to.
"We use isolated fine-tuning." This is a more sophisticated approach where a firm's data is used to fine-tune a model version specific to that firm. It is not the same as a no-training commitment — your data is still used to train a model.
The acceptable answer: "We do not train our models on customer data. This commitment is stated explicitly in our DPA." Harvey AI and Ironclad both include explicit no-training commitments in their enterprise DPAs. Spellbook publishes this as part of its standard terms. Get it in writing, in the contract — not from the sales rep.
See our zero-data-retention-policy entry for the full set of DPA provisions to look for.
SOC 2 is a security audit framework for service organizations. Type I confirms that security controls are designed appropriately at a point in time. Type II confirms that those controls operated effectively over a period of time (typically 6-12 months). Type II is the meaningful standard.
Ask specifically: "Do you have a current SOC 2 Type II report, and can you share the executive summary or Trust Service Criteria mapping with our security team?"
A vendor with only SOC 2 Type I certification has not demonstrated that their controls work in practice — only that they were designed on paper. A vendor with no SOC 2 certification is asking you to rely on their self-reported security posture without external validation.
What SOC 2 Type II covers (and does not cover):
SOC 2 covers security controls in the vendor's environment: access controls, change management, risk management, monitoring. It does not certify the accuracy of AI outputs. It does not certify that the vendor is GDPR-compliant. It does not certify that client data is not used for training. It is a security audit, not a privacy audit. Check the scope carefully. See our audit-log-legal-ai entry for what audit logs a legal AI tool should maintain.
For enterprises, ISO 27001 certification is an additional standard worth requiring. Harvey AI holds both SOC 2 Type II and ISO 27001. Spellbook holds SOC 2 Type II without ISO 27001, which is appropriate for its SMB market positioning.
"The cloud" is not an answer. You need specificity.
Ask: "In which AWS regions, Azure regions, or data center locations is our data processed and stored? Does our data pass through CDN caches or intermediary nodes in other regions? Who are your subprocessors?"
Why this matters in practice:
For EU-based firms and in-house teams, GDPR requires that personal data not be transferred outside the EU/EEA without an adequate legal mechanism. If a vendor processes EU client data on US servers without a Standard Contractual Clause (SCC) or adequacy decision in the DPA, there is a potential GDPR violation. See our gdpr-compliance-ai entry for the current framework.
For US government and regulated-sector clients, some client engagements require data to remain in specific jurisdictions. A vendor that processes data in multiple AWS regions without explicit customer control may not be compatible with those requirements.
CDN cache reality: Many SaaS vendors use content delivery networks that cache data in edge nodes globally for performance reasons. Your "US-only" data may pass through CDN nodes in other jurisdictions during processing. Ask specifically whether CDN intermediaries are excluded from the data residency commitment or whether they are covered.
The acceptable answer includes: specific named AWS or Azure regions, a list of subprocessors with their functions and locations, and explicit statement of whether CDN caches are included or excluded from data residency guarantees.
This question separates vendors with credible accuracy data from those with marketing claims.
Ask: "What is your hallucination rate on legal research tasks, and was it measured by an independent third party or by your own team?"
The only independent benchmark we reference at LawyerAI is the Stanford RegLab 2024 study, which measured:
Harvey AI, Spellbook, Luminance, and most other tools are not in the Stanford RegLab benchmark. No independent hallucination rate exists for most of the market.
When a vendor cites an accuracy number:
A vendor that says "our tool is 95% accurate" without naming an independent study, describing the task, or sharing the methodology is giving you a marketing claim, not a data point. Accept only independent benchmark data as evidence. Treat vendor-reported accuracy claims as unverified.
Legal AI is a consolidating market. Two significant acquisitions in the past three years illustrate the risk:
Casetext → Thomson Reuters (June 2023, $650 million): The standalone Casetext product, which was accessible to small firms and solos without a Westlaw subscription, was folded into the Westlaw ecosystem as CoCounsel. Customers who valued Casetext as a subscription-independent tool lost that option.
Evisort → Workday (2023): Evisort was acquired by Workday. For legal teams that are not Workday customers and have no interest in integrating their contract repository with HCM systems, the strategic question of where Evisort sits in Workday's product priorities is unresolved.
Ask vendors specifically: "If your company is acquired, what are our data rights? Does our data transfer to the acquiring entity? Can we exit the contract without penalty in the event of an acquisition? What happens to our data if you shut down the product?"
The acceptable answer includes a data export right (you can retrieve your data in a usable format), a contract exit right triggered by acquisition or material product changes, and a data deletion commitment following exit. Verify these provisions are in the contract, not the vendor's marketing materials.
Bar associations and courts are increasingly requiring lawyers to document AI use. The ABA Model Rule 1.1 competence requirement, malpractice defense, and court-specific AI disclosure rules all create practical demand for AI audit logs.
Ask: "Does your platform maintain logs of queries, documents submitted, and AI outputs? Can we access those logs? How long are they retained? Are the logs exportable?"
The bar association requirements:
Several state bars have issued guidance requiring attorneys to document their AI use in the matter file. For litigation specifically, court rules requiring AI disclosure presuppose that the attorney knows which AI tool generated which output in which filing. Without logs, this documentation depends on attorney memory.
The malpractice protection rationale:
If a client later claims that AI-generated advice was negligent, your ability to demonstrate that you applied verification protocols depends on documentation of what the AI produced and how you reviewed it. An exportable audit log is stronger evidence than a contemporaneous note.
Acceptable platform behavior: session logs retained for at least 12 months, accessible by firm administrators, exportable in a readable format. See our audit-log-legal-ai entry for the specific log elements that matter for legal compliance.
Service level agreements (SLAs) for legal AI tools matter for two reasons: uptime for time-sensitive work, and remedies that actually compensate for failure.
Ask: "What is your uptime SLA? What is your data recovery time objective (RTO) and recovery point objective (RPO)? What are our remedies if you miss the SLA?"
The uptime question: Most enterprise vendors promise 99.9% uptime, which translates to approximately 8.7 hours of downtime per year. For a firm dependent on an AI research tool for a brief due at 9 AM, an outage at 2 AM is a professional risk regardless of the annual uptime average. Ask specifically about maintenance windows and historical uptime data.
The remedies question: Many SLA "guarantees" provide service credits — a percentage discount on the next month's invoice. A service credit of 10% on a $3,000 monthly invoice is $300. That does not compensate a client for a missed filing deadline. Evaluate whether the SLA remedies are commercially meaningful for your practice, not just technically present.
The exit terms question: What happens to your data if you decide to leave? Most enterprise contracts include a data export window (typically 30-90 days after termination) followed by deletion. Verify the export window is sufficient, that export is in a usable format, and that deletion is certified. Some vendors charge for data export after termination — this is unusual but not unheard of.
Find your closest match for vendor evaluation:
How do I know if a vendor is training on my data?
Read the DPA (data processing agreement), not the privacy policy. The DPA is the legally binding document. Look for explicit language: "We do not use customer data to train, fine-tune, or improve our models." If the DPA says "we may use anonymized data for service improvement," that is not a no-training commitment. If the vendor says verbally that they do not train on your data but the DPA does not say so, the verbal statement is not enforceable. Get it in the contract.
What is the difference between SOC 2 and ISO 27001?
SOC 2 is a US standard from the American Institute of CPAs (AICPA). It evaluates whether a vendor's security controls are designed and operating effectively. ISO 27001 is an international standard that certifies an information security management system (ISMS). ISO 27001 is broader — it covers risk management processes, not just technical controls — and is required by more European enterprise procurement processes. For US law firms, SOC 2 Type II is the baseline. For firms with European clients or EU data processing requirements, ISO 27001 is the additional standard to require.
What should be in a legal AI DPA?
The minimum: explicit no-training commitment, data processing purposes limited to providing the service, subprocessor list with the right to object, data residency specification, data retention and deletion terms, security incident notification timeline (72 hours is the GDPR standard), data export right, and data subject rights procedures (GDPR-applicable entities). See our data-processing-agreement glossary entry for a detailed DPA review checklist.
How do I run a security review of an AI vendor?
Request: current SOC 2 Type II report (executive summary at minimum), penetration test summary (if SOC 2 is not available), DPA for legal review, subprocessor list, and data residency documentation. Your IT security team reviews the SOC 2 report against your firm's security baseline. Legal reviews the DPA. If the vendor declines to share any of these documents, that is itself a risk signal. Enterprise vendors — Harvey AI, Ironclad, Lexis+ AI — are accustomed to providing these documents in legal sector procurements.
What is a reasonable SLA for a legal AI tool?
99.9% uptime (approximately 8.7 hours of downtime per year) is the market standard for enterprise SaaS tools. Below 99.5% is a concern for any tool used in time-sensitive legal work. Meaningful remedies should be in the contract — not just service credits, but ideally a right to terminate without penalty for extended or repeated outages. For research tools used in brief preparation, ask specifically about maintenance windows — a tool that goes offline for maintenance on Tuesday nights is less disruptive than one that goes offline unpredictably.
LawyerAI evaluations are independent. We do not accept payment that influences our editorial scores. Featured placements are clearly labeled and do not affect our 5-dimension methodology (Accuracy / Speed / Usability / Value / Security). We re-review tools every 6 months.
If you believe any information is inaccurate, contact editor@lawyerai.directory.