Law as Counter-Example: Extensions Exist But Were Bypassed

Source Note This page is compiled from web search results, the Charlotin hallucination database, Stanford CodeX findings, and practitioner accounts — not from a single ChatGPT Deep Research query. It synthesizes evidence from across the other reports where law appears as a comparison domain. Central Claim Law is NOT extension-poor. It has rich existing extensions: • Shepardizing (is this case still good law?) — a validation feedback loop • Partner review — human verification before filing • Opposing counsel — adversarial verification • Judicial review — institutional verification • Citation format requirements — structural constraints on output The Evidence The famous hallucination cases (Mata v. Avianca, 700+ tracked cases per Charlotin database) happened when lawyers used AI outside their normal extensions — filing briefs without Shepardizing, without partner review, without any of the normal scaffolding. Key Data Points • Leading legal AI tools hallucinate 17-33% of the time (Stanford CodeX, 2025) • General-purpose LLMs fabricate citations in 30-45% of legal responses • 700+ court cases now involve AI hallucinations (Charlotin database: https://www.damiencharlotin.com/hallucinations/) • 79% of lawyers use AI tools (2025 ABA TechReport) • A lawyer can Shepardize an AI citation in seconds — the extension already existed New Extensions Being Built • CiteCheck AI (LawDroid, 2025) — automated citation verification • BriefCatch RealityCheck — authority verification tool • RAG-grounded legal AI — grounds output in closed database of primary law • Green/Yellow/Red verification labels — visual extension for reviewers • Two-layer verification: deterministic citation check (no AI) + AI-assisted analysis of quoted language The Reframing Willison's "if you're a lawyer, you're screwed" is the isolated-task mistake. A lawyer isn't screwed. A lawyer using AI without their normal extensions is screwed — same as a developer shipping AI code without running the compiler. A lawyer who Shepardizes an AI-generated citation takes seconds to verify it. The extension already existed. The Sharper Claim The variable isn't "does this domain have extensions?" — it's "does this use of AI engage the extensions that already exist in the domain?" Sources • Charlotin hallucination database: https://www.damiencharlotin.com/hallucinations/ • LawDroid CiteCheck AI: https://www.lawnext.com/2025/06/lawdroid-launches-citecheck-ai • New verification tool: https://abovethelaw.com/2026/03/new-tool-catches-ai-hallucinations-in-legal-briefs/ • Stanford AI hallucination rates: https://hai.stanford.edu/news/ai-trial-legal-models-hallucinate-1-out-6-or-more • JELS hallucination study: https://onlinelibrary.wiley.com/doi/full/10.1111/jels.12413 Significance for the Extended Frontier Law is the critical counter-example that sharpens the argument. The naive version of the extensions thesis would predict that law is a domain where AI fails because extensions are weak. The evidence shows the opposite: law has strong extensions, and AI fails when those extensions are bypassed. This shifts the claim from "some domains have extensions and some don't" to "the variable is whether AI use engages the extensions that already exist."