Will AI Change the Practice of Law?

Yes, changing the practice of law is hard. For most of us, “new” goes with “shiny”. For attorneys, new comes with risks, as it hasn’t yet been litigated or documented as precedent.

Then again, major changes in law and technology often coincide:

Those two events led to accelerated investments in the internet, providing large-scale access to new financial instruments. Over the last 50 years, the increase in wealth from those investments shifted our culture’s acceptance and celebration of quantitative reasoning and all things STEM.

Two recent US Supreme Court (SCOTUS) decisions could dramatically spur accelerated investment in AI and large language models (LLMs):

Those decisions will create a demand for legal research to revisit the reasoning behind past decisions. They may very well set in motion the use of LLMs by attorneys to predict or 2nd guess logical reasoning behind legal arguments in judicial proceedings, legislative debates, rule-making, arbitration, mediation, or any other event facilitated by attorneys.

Are we ready for that?

LLMs today provide recall of facts, which isn’t the same as logical reasoning. Consider Julia Roberts’ recall of facts in Erin Brockovich, who in this scene appears to have Wikipedia-like command of the case file.

Now consider Marissa Tomei’s logical reasoning in My Cousin Vinny. Her expert testimony goes well beyond recall of Wikipedia entries for the Pontiac Tempest or Buick Skylark.

What if attorneys had an example of how another well trained career field adapted to new technology?

The adoption of spreadsheets and pivot tables illustrate how to adopt the use of LLMs for logical reasoning. Spreadsheets, introduced on mainframes in 1969, were once limited to the “quants” – accountants and finance professionals. VisiCalc, Lotus, Apple, IBM, and Microsoft installed spreadsheets with personal computers and office productivity tools through the 1980s. Microsoft, Apple, and Lotus added pivot tables to spreadsheets starting in 1993.

Increasing adoption of those office productivity tools coincided with the rise of quantitative investments and STEM generated wealth:

Movies more easily illustrate our acceptance of STEM and quantitative reasoning in every day life. 1984’s Ghostbusters was the last movie that made easy jokes about quants (if you can’t guess by looking at the screen capture, the dog eats the quant):

Since then, movies have steadily increased recognition of quants, nerds, and scientists as heroes.

Movie makers now take for granted that the audience accepts STEM as a driver of the plot. Here is the critical scene about science from 2019’s Captain Marvel:

The guy who couldn’t figure out the science died in the next scene.

Entrepreneurs have already built LLMs that review contracts. Conceptually, you could build an LLM for any business situation just like a recently released experimental AITAh LLM. The only difference being that instead of asking for advice on your life situation, you’d ask for advice on legal risks.

• Am I The (AIT) Small Claims Court Judge?

• AIT Home Owners Association?

• AIT Person in corporate finance who approves expense reports?

That’s enough to kick-start the VisiCalc era of LLMs, but several obstacles remain before realizing something close to ETFs or another dot-com era.

The skills illustrated in that scene from My Cousin Vinny are assessed in the Law School Admission Test (LSAT) for logical reasoning – which looks like capabilities you’d want in every LLM:

• Recognizing the parts of an argument and their relationships

• Recognizing similarities and differences between patterns of reasoning

• Drawing well-supported conclusions

• Reasoning by analogy

• Recognizing misunderstandings or points of disagreement

• Determining how additional evidence affects an argument

• Detecting assumptions made by particular arguments

• Identifying and applying principles or rules

• Identifying flaws in arguments

• Identifying explanations

LLMs like ChatGPT would still not be accepted into any premier law school based on LSAT scores. Top candidates score in the 170s. (In Legally Blonde, Reese Witherspoon played Elle Woods who went from the 140s to a 179 on the LSAT, improving by over 30 points and getting into Harvard! )

While LLMs haven’t scored that high yet in real life, they seem to improve by 10-15 points every year. Since LLMs like ChatGPT most recently scored in the low 160s, they could be Harvard ready in the next 1-3 years.

1973’s The Paper Chase depicts “thinking like a lawyer” as a never ending self-induced Socratic discussion. I’m convinced anyone who has graduated from law school since then (including every SCOTUS judge) secretly imagines themselves as the Harvard professor in The Paper Chase.

The first person in law school using LLMs for Socratic study sessions won’t be seated as a federal judge – I would guess – until 2040 at the earliest. Until then, one can only imagine how many hours law school students will spend training LLMs to answer:

• Should LLMs be free to consume copyrighted material under fair use clauses?

• Can proprietary LLMs be subject to eDiscovery?

• Are you liable if the LLM said it was legal to do something?

• Are you guilty if the LLM remembers you asking how to erase records of you asking if it was legal to do something?

The more society accepts AI in positive roles in popular culture, the more likely society accepts AI. Too often for AI, it comes packaged as a people-killing robot. However, when we package it as a product of something heartwarming like fairy godmothers or shooting stars, we more easily accept AI even with its flaws.

• Pinocchio: an AI wooden doll prone to falsehoods becomes a real boy.

• Ted movie franchise: an AI teddy bear prone to vulgar language and drug use makes friends, gets married, and plans to adopt a real boy.

Ted 2 even gave us a Dred Scott reference in an opening statement (from about 1:50 to 3:40 in the clip below) on extending human rights to all forms of intelligence, even those in the form of foul mouthed, drug using, teddy bears. Apologies in advance for the vulgar language.

For what it’s worth, the above scene had the least amount of profanity in any scene I found clipped from the Ted franchise.

2040, which is 15 years away, seems like a long time to wait before a judge familiar with LLMs begins issuing decisions. However, there is ample precedent for Oscar winning court-room dramas that foretell how the courts ever increase their capacity to see humanity in every walk of life.

After a few prompts with ChatGPT, here’s a plot and storyboard for the first Oscar winning movie(see footnotes for full plot) that challenges the courts to accept AI and LLMs:

Just like spreadsheets revolutionized finance and quantitative reasoning, LLMs are poised to reshape logical reasoning in the practice of law – which can be very uncomfortable if not scary. Movies at least can give us a safe place to contemplate the impact of AI.

I don’t know if Ted 3 will ever happen, let alone win an Oscar, but at least Pinocchio and Ted aren’t killing the human race.

Copyright © 2024 Raghav Vajjhala. All Rights Reserved. Any republication shall include a link to offtopictech.substack.com.