The ghost of sanctions yet to come.
What keeps me up at night.
Earlier today, I was at Bristol County Superior Court. New Bedford. The courthouse — which is, like many of its era, objectively gorgeous — was built at the height of the whaling era, when New Bedford, Massachusetts, was, per capita, the wealthiest town in the United States.
Whale was something everybody needed, it seemed. Industrialists used blubber for grease. Whale oil made the best candles — they burned without smoking, it’s said — and ointment, and soap. Whalebone was used for corsets, and trinkets, and furniture, and tools, and ambergris, in perfume.
It must have seemed like the era would never end. Every product. Every industry. Every conceivable use. No need to ask. Just cram it full of whale. Don’t ask where it came from. Or how many more there are out there. Or what the process was, by which living animal became soft glow from a hurricane lamp.
Anyway.
I have just about lost track of the latest and largest sanctions issued to lawyers for turning in briefs with what we’re presently referring to, euphemistically, as “hallucinated” citations, but it seems as if the rhythms have changed, slightly.
Where initially counsel would claim that they thought some new-to-them “tool” they’d used was something akin to a search engine, and the results as reliable as any other “research,” it seems like the current “mea culpa” is “I didn’t think it could do…this.”
“This” being fabricating snippets of records. Or false quotes, and holdings from real cases. Or citations that do, in fact, point to a single word — which appears — for a principle that does not.
Sanctions for use of hallucinated cases have become almost routine, and, unfortunately, are likely to remain so. Careful lawyers will return to the practice of cite-checking everything, personally. Other lawyers will roll the dice. And some percentage of them will get caught.
This does not keep me up at night. I don’t use LLMs for legal research. It would ruin all the fun.
What keeps me up at night is something else.
Hallucinated evidence.
Documents and information presented as if “retrieved” by some “AI-assisted search” rather than as something “generated” by AI.
Hallucinated cases are easy to prove false. Published legal cases have one, and usually several, unique identifiers, which follow a known system and can be checked against multiple outside sources.
They can even be checked without using a computer at all, by walking up to the right shelf in a law library, and taking the volume down from the shelf, and opening it to the indicated page.1
Not only is it easy to determine if a case is hallucinated, it’s not difficult to avoid being in a position to use hallucinated cases. No matter how much whale AI they keep cramming into everything, the legal research tools that existed before, continue to exist.
Documents are trickier. No citation. No universally-recognized unique identifier. How would a careful attorney know that the very-important-seeming-several-page-something bundled into a lengthy production existed prior to the prompt that “retrieved2” it?
That is what keeps me up at night.
Not hallucinated cases.
Inadvertent forgery.
Evidence that isn’t.
The ghost of sanctions-yet-to-come.
And sometimes whales.
A legal citation is, pretty much, instructions on how to do exactly what I described. Take: Goodridge v. Dept. of Public Health, 798 N.E.2d 941 (Mass. 2003),
“N.E 2d” is the name of the reporter, meaning, the series of books where the case is published. The “Northeastern Reporter, Second Series.”
798 is the volume the case appears in — so which book to take from the shelf. 941 is the page where the case starts in the volume. If you open to page 941, and you cannot find “Goodridge v. Dept. of Public Health.” then you know I am full of shit.
Consider: Most evidence that is eventually introduced at trial in a civil suit were obtained, or exchanged, in discovery. Attorneys will typically introduce, and rely on during preparation for trial, pdfs of marked, stamped documents exchanged in discovery, rather than originals.
The typical process is: Attorney A requests documents from Attorney B. Attorney B, to fulfill the requests, asks Client B for “all their relevant documents.” Client B locates the documents, sends them to Attorney B, who reviews them for relevance, privilege, and responsiveness, then (hopefully) marks the documents with a unique page number (A “Bates Stamp”), and sends to Attorney A.
If a client or attorney unknowingly uses a tool capable of “creating” documents rather than simply “retrieving” them, “hallucinated” documents could be introduced by the client while “searching,” the attorney while “marking” or preparing prior to production, or even opposing counsel while reviewing after production.