The first wave of on-device AI tools is arriving in app stores now. The question — after why, exactly, on-device AI for lawyers is a big deal — is how to recognize the real thing. Here’s developer Josh Langberg’s five-question, on-device AI checklist.
You already know not to paste a client’s documents into ChatGPT. That advice is everywhere, and it’s correct as far as it goes. The problem is that it stops there, while every piece of software on your machine grows an “AI” button. Your PDF reader wants to summarize. Your notes app wants to draft. Your practice management suite has a copilot. Some of these send client material to a server on every keystroke; some never let a byte leave your machine. From the outside, the buttons look identical.
The good news: You don’t need to trust marketing copy to tell them apart. The difference is architectural, and architecture can be checked.
On-Device AI for Lawyers vs. the Cloud: The Spectrum, in Plain Terms
Every AI feature runs a model somewhere. There are only three places it can be:
- In the vendor’s cloud. Your document is uploaded, processed on someone else’s computer, and the answer comes back. This is how most AI tools work, including the famous ones. Your confidentiality now depends on the vendor’s retention policy, their security, their subpoena posture, and whatever their terms of service say about training, all of which can change after you’ve signed up.
- In a private cloud. Same upload, but to infrastructure the vendor promises is isolated. Sometimes your cloud, usually their cloud. Better on paper, but still a transmission of client material to a third party — the exact scenario the ABA addressed directly in Formal Opinion 512 (July 2024), its first ethics guidance on generative AI, which walks through the Model Rule 1.6 duty of confidentiality as applied to these tools. (Note: Rule numbering and comment language vary by state, so check your jurisdiction’s version before relying on this framing.)
- On the device. The model itself, the actual AI, is installed on your Mac or PC and runs there, the way your word processor does. The document is processed where it already sits. There is no upload, which means there is no retention policy to read, because there is nothing retained anywhere to worry about.
That third category used to be exotic. It isn’t anymore: Current laptops can run capable language models locally, and a first wave of on-device AI tools is arriving in app stores now. The question is how to recognize the real thing.
Privacy Policies Change, Architecture Doesn’t
Here’s the shift in thinking that matters: A privacy policy is a promise, and promises have exceptions, updates and breaches. Architecture is a fact. The right question about any AI tool is not “Does the vendor promise not to keep my data?” It is “Can my data leave this machine at all?”
For a meaningful class of software, that question has a verifiable answer — the operating system itself will tell you.
The On-Device AI Checklist: Five Questions You Can Actually Run
- Does it require an account? An account means a server relationship. Even if the documents stay local, your usage may not, and a tool that makes you sign in has, at minimum, a channel out. Genuinely on-device tools have no reason to know who you are.
- Does it ask for an API key? An API key (from OpenAI, Anthropic, Google, or anyone else) is a cloud dependency wearing a disguise. If a tool asks you to paste one in, your documents go to whoever issued that key. That’s not necessarily disqualifying — but it means the “local app” is a remote control for someone else’s computer.
- Does it work in airplane mode? The simplest real-world test, and you can run it in thirty seconds: turn off Wi-Fi, then use the AI feature on a scrap document. If it works identically offline, the model is on your machine — physics doesn’t allow an alternative explanation. If it errors or hangs, your documents need a network connection to be processed. Ask the vendor why.
- On a Mac: Does the app even have network permission? This is the granular one, and it’s worth understanding because it turns confidentiality from policy into physics. Every Mac App Store app runs in Apple’s “sandbox” and must declare its capabilities up front, including whether it may open network connections at all. An app that ships without the network entitlement cannot transmit anything, ever; macOS refuses the connection at the operating-system level. The developer couldn’t leak your documents if they wanted to. You can check this yourself in Terminal:
codesign -d --entitlements - /Applications/AppName.app 2>&1 | grep -i networkNo network entitlement in the output means no network, period. (A caveat: This guarantee comes with App Store distribution, where the sandbox is mandatory. Software downloaded directly from a website may not be sandboxed at all — there, you’re back to trusting the vendor.)
- What does the App Store privacy label say? Look for “Data Not Collected” — the strictest label Apple offers. It’s a formal declaration the developer is accountable for, not ad copy. Anything else (“Data Not Linked to You,” “Data Linked to You,” “Data Used to Track You”) is telling you something is being gathered; read on before feeding it client files.
One follow-up worth asking about any sync feature: Whose cloud? Syncing through your own iCloud or OneDrive account keeps documents inside an account you control and a vendor relationship you’ve already evaluated. Syncing through the tool-maker’s servers is a new disclosure decision.
What You Give Up, and Why It’s Still Worth It
Honesty requires saying this: an on-device AI model is smaller than a frontier cloud model. It won’t write your brief. What local models do well today is the unglamorous work that happens to involve your most sensitive files: filing and organizing documents by subject, OCR on scanned discovery, searching by meaning rather than exact words, summarizing and answering questions with citations to the page, and flagging personal information — Social Security numbers, account numbers, addresses — before a redaction pass.
Match the tool to the sensitivity: Public research can go to the cloud model; the client’s documents can get the local one.
Related reading: “How Lawyers Can Ethically Integrate AI Into Their Practice,” and “5 Essential Questions Every Lawyer Should Ask Before Using AI Tools.”
Where This Is Heading
I’ll disclose my interest: I’m a software developer, and I built one of the tools in this new category — Bibliome, a Mac PDF library manager whose language model ships inside the app, with the network entitlement removed. But the checklist above is deliberately not about my app. It works on any tool with an AI button, including the ones that don’t exist yet, because it tests the only thing that doesn’t change with the marketing: where the model runs, and whether your client’s documents can leave the building.
Five questions to verify on-device AI for lawyers and legal professionals. Two of them you can run in Terminal and airplane mode before lunch.
Image © iStockPhoto.com.
Sign up for Attorney at Work’s daily practice tips newsletter here and subscribe to our podcast, Attorney at Work Today.
