Most advice about AI in legal work stops at “verify the output.” That’s correct and nearly useless, says Samir Ali. Here are five specific AI verification checks that ensure you can explain and defend every sentence.
Verification is a set of specific motions, and if you’ve never been shown them, “verify” just means reading the thing again more suspiciously — which catches almost nothing, because the AI’s output is fluent and fluency is what fools you.
How Lawyers Verify AI Output: 5 Critical Checks
Here are five checks, in the order I’d run them.
1. Pull every citation yourself, in a legal database, not a search engine.
Copy the citation. Search it two ways in Westlaw, Lexis, vLex Fastcase, or CourtListener: once by reporter cite, once by party name. A general web search is not this check — it will happily surface a blog post repeating the same fabricated cite.
You’re sorting each citation into three bins: it doesn’t exist; it exists but isn’t what you were told; it exists, is what you were told, and is no longer good law. Give yourself 90 seconds. If you can’t find it in a database in 90 seconds, stop treating it as real.
Then run the citator. A real case cited for a real proposition that was overruled four years ago would still pass every other check on this list.
2. Check the proposition, not just the existence.
An invented case is embarrassing but easy to catch. The more common failure is a real case, correctly cited, offered for something it does not hold. This is where the actual risk is.
For every proposition attributed to a case, open the opinion and find the sentence that supports it. Not the headnote — headnotes are editorial summaries, and AI output often reproduces their “shape” without their substance. Find the actual sentence, on the actual page, with a pinpoint you could put in a brief.
If you can’t point to a sentence, the citation does not support the claim, however reasonable the claim sounds.
Treat quotation marks with particular suspicion. AI output frequently presents paraphrases within it. Verify quoted language character by character, or don’t quote it.
Here’s a familiar hypothetical. You’ve asked for authority on when the duty to preserve documents attaches, and the output hands you a case — call it Harlan v. Meridian Freight — with a reporter cite, a pinpoint, and a clean one-sentence summary.
Search the reporter cite. Nothing. Search the party names. You get a case with one of those names, a different opposing party, a different circuit, a different year. That mismatch is the tell: the model assembled something plausible out of real-sounding parts.
Ninety seconds, done.
Now the harder version. The cite resolves. Case, parties and year are all correct. You open it and search the opinion for “preserve.” It appears twice, both times in a recitation of the parties’ arguments — not in the holding, which turned on something else. The citation is real, and the proposition is not. Nothing short of opening the opinion catches that, which is why this type of failure ends up in filings.
3. Run the client confidentiality check before you paste in the prompt, not after.
Once you’ve added text in an AI prompt box, your confidentiality check is over, and you’ve failed it. The client confidentiality check has to happen in the second before you hit paste.
The instinct is to sanitize: strip the names, round the dates, describe the situation generically. That instinct is where most well-intentioned failures happen, because the identification risk doesn’t come from any single element. A matter can be identifiable from jurisdiction, posture, claim type and a date range with no names in it at all. Small bar, distinctive facts, and you’ve described your client to anyone who recognizes the case.
So the question isn’t how to de-identify. It’s whether the task can be done with no client information in it at all. Plenty can: research on settled law, formatting a template carrying no case facts, and general administrative work. Anything that carries the matter can’t, and no amount of rewording gets around that.
For the cases in between, use this one filter:
If your supervising attorney could see exactly what you’re about to submit, would they be comfortable? Would your client?
Anything short of an unqualified yes means ask first. And if your firm or business has no written AI policy (most don’t), that isn’t permission; it means the conservative reading applies until you have one.
4. Name what you did not delegate.
Pen and paper work best here. Read the output and bracket every sentence that asserts a legal conclusion. Then take the brackets one at a time.
For each, ask:
Is this authority controlling here or merely persuasive, how much weight does the contrary authority deserve, and do these facts map onto that holding or only resemble it?
Write one sentence answering it — in the margin is fine.
If you can write the sentence, the judgment is yours, and the tool got you there faster. If you can’t, the call has been made by something that cannot be asked to explain it, and your name is going on it anyway. That’s the line, and it has nothing to do with how many of the words you rewrote.
5. Keep an audit log and write down what you did while you’re doing it.
One line per use: the tool, the task, what you checked, what you changed. Keep this log in the matter file, not in your head.
It takes 15 seconds, which seems like too much bureaucracy right up until someone asks how a document was produced. Then it’s the difference between an answer and a reconstruction — and the person asking usually can’t tell a bad memory from a bad process.
The Biggest AI Verification Test for Lawyers
If you keep one check from this article, keep this question:
Could you explain and defend every sentence, in your own words, without referring back to the AI output?
If yes, the thinking is yours, whatever tools were involved. If no, you’re submitting someone else’s reasoning under your name. The five checks above are the specific places that gap tends to hide.
Image © iStockPhoto.com.
Sign up for Attorney at Work’s daily practice tips newsletter here and subscribe to our podcast, Attorney at Work Today.
