Law's AI revolution is here

The craze isn't over. On March 14, OpenAI released GPT-4, the latest version of their generative artificial intelligence tool and the backbone of ChatGPT, the clever program that became a worldwide obsession. Competitors are scrambling to put out their own generative AI creations.

On March 15, PwC announced an alliance with startup Harvey, backed by OpenAI's Startup Fund, and which has an AI tool to answer legal questions, which will be available to all PwC employees effectively immediately. On March 16, Microsoft, which is investing more than $10 billion in OpenAI, announced that its tool, Copilot, will be incorporated into Microsoft 365. The free ChatGPT version still uses GPT-3.5, but users with the paid ChatGPT+ can access GPT-4. 

Legal tech entrepreneurs who've been quietly working in AI for years are finally having a moment. Casetext launched CoCounsel during last month's ABA Techshow. What people didn't know is Casetext had been quietly building CoCounsel with GPT-4 for months in an exclusive deal with OpenAI. 

Meanwhile, Daniel Martin Katz, professor at Illinois Tech-Chicago Kent Law, and a small group of researchers became a viral sensation once again when they announced ChatGPT had not only passed the bar exam but scored in the 90th percentile. 

On March 23, OpenAI made another big move by launching 11 plugins for ChatGPT. This means developers can create new programs and integrate ChatGPT into existing software.

Rarely, if ever, has technology in the legal sector moved so quickly as ChatGPT, which has over 200 million users. Lawyers, fascinated with the platform, are embracing it. The AI revolution foretold so many years ago may finally be here.

What GPT-4 can do 

When Pablo Arredondo, co-founder and chief innovation officer of Casetext, presented at ABA Techshow, it was one day after he and CEO Jake Heller had appeared on MSNBC's Morning Joe to discuss CoCounsel, an AI legal assistant that can write legal memos, prepare deposition outlines, help with legal research and extract data from contracts. 

In October, shortly before OpenAI launched ChatGPT, the company approached Casetext and they began to work on CoCounsel using GPT-4. Speaking via Zoom only 48 hours after the GPT-4 launch, Arredondo discussed Casetext's growing waitlist and their beta testers for CoCounsel, which includes Fortune 500 companies.

"The folks we're getting now are not the early adopters," says Arredondo. "The value and energy are palpable. Things are moving so quickly."

Arredondo and his team began looking at OpenAI in 2020 but decided the tech needed more time to be ready for the law. "We were building our own language models, and then we got a glimpse of GPT-4," says Arredondo. "We realized every single part of the law would be impacted by this technology."

Large language models — LLMs — are machine learning models that use large amounts of data to recognize, summarize, analyze and make predictions with data. GPT-4 is by far the best-known model, but others exist, such as Microsoft's Copilot and Google's Bard. 

The power and the pitfalls of LLMs, which can write sonnets and research papers on demand, were on full display when ChatGPT was launched to the public in November 2022. Topping the list of pitfalls are what are known as hallucinations — whereby AI fills in knowledge gaps with false information. 

GPT-4 is designed to reduce hallucinations. According to Arredondo, the best way is through retrieval augmented generation, whereby AI tools are anchored in "a point of truth," and the system learns from external documents deemed factual and accurate. Information used for this process includes case law and legislation.

GPT-4 has left entrepreneurs scrambling to update their platforms. As owner of LawDroid, an AI legal assistant tool, Tom Martin has spent the last seven years in the field. During a session about GPT at ABA Techshow, Martin shared his view that the shift to remote work during the pandemic has opened people to using new technology. "This is a sea change," he says. "[T] hey're able to see the practical side of using GPT. The mainstream base for using AI is widening."

Scott Stevenson, the co-founder and CEO of Rally, heard rumours about GPT-4 as early as November but didn't expect the March launch. His company built its AI contract drafting tool on the AI language model. The new version focuses on "more facts, less creativity" with more accurate, neutral responses. But he says the real game changer is the ability to upload entire documents into ChatGPT. The new version allows up to 52 pages of text. 

"A lot of us have been constrained by the six- to seven-page limit, trying to break up legal documents into small chunks," he says. Now it's possible to "digest entire contracts at once."

Another highlight is GPT -4's ability to read images and convert them to text. Stevenson is excited for the day GPT can sort and create files. 

"Right now, we're stuck with small tasks like search," says Stevenson. "We want models that are able to process workflows. It would be great if it can conduct actions like find and dig into documents, compare those documents with precedents in a legal research tool, and write a memo based on that." 

Nailing the bar exam

And why not? GPT-4 scored in the 88th percentile on the LSATs and practically aced the Uniform Bar Exam, scoring in the 90th percentile.

Daniel Martin Katz, a professor at the Illinois Institute of Technology's Chicago-Kent College, along with Pablo Arredondo, Shang Gas from Casetext and Michael James Bommarito from Stanford Center for Legal Informatics were granted access to GPT-4 to work on the bar exam research. The team was behind the original GPT-3.5 bar exam challenge in late 2022 when it scored in the 40th percentile. They only had a few short weeks to run their tests. This time the AI program completed the entire test, including the essay portion and the section, and even reviewed information from pre-selected material to identify hypothetical legal issues for a client. GPT-4 performed better than expected, finishing in the 90th percentile.

Because of the secrecy, Katz didn't have a large team of researchers to review the qualitative results. "With the qualitative portion, it's a judgment call," says Katz. "I feel confident in our research. People can look at GPT's responses side-by-side with actual human responses and see for themselves."

GPT-4 made some errors. The AI program did poorly in calculating matrimonial property, didn't cite the rule of perpetuities properly and cited an evidence rule incorrectly. "But these are mistakes students would make," says Katz. "They're not hallucinations. It's an improper reference but it's within the scope of the response. We need to classify those errors differently than just broadly calling them a hallucination."

How lawyers can use it ChatGPT

As impressively as GPT-4 performed on the bar exam, there isn't yet a consensus on whether GPT-4 is the big step forward it is touted to be. As Daniel Linna Jr. and I chatted about his upcoming April workshop on ChatGPT at Northwestern Pritzker School of Law, he noted that OpenAI hadn't released data on how GPT improved its functionality and what data was used to make it better. 

"These tools will transform the legal system," says Linna Jr., director of law and technology initiatives at the school. "We have to make sure that these tools are safe, not biased, fair, transparent and explainable."

OpenAI acknowledges that the work to reduce misinformation and harm is ongoing. According to its technical paper about GPT-4, "[T]he profusion of false information from LLMs - either because of intentional disinformation, societal biases, or hallucinations — has the potential to cast doubt on the whole information environment, threatening our ability to distinguish fact from fiction."

Lawyers will have to adapt quickly. ChatGPT is one of the few tech products that both clients and lawyers have access to. Researchers and access-to-justice advocates are digging into how ChatGPT can help with unmet legal needs. The public can now easily ask ChatGPT to write a will or ask about how to incorporate a business.

"There's a gap between tech and legal services delivery," says Linna Jr. "We have a variety of tech that's increasingly capable, but we are only using it for the same things we've always done. Clients don't want more memos and emails. There's a tremendous opportunity to rethink our processes."

Junior lawyers, paralegals and other legal professionals can use ChatGPT to create legal documents more efficiently. Colin Lachance, the founder of AI research company Jurisage AI, has been approached by law firms asking him for guidance on training summer students and junior staff on appropriate ways to use ChatGPT.

"Every law firm will need to develop a strategy around language models," says Lachance. "Whether they adopt them or not, their operations will be impacted so ignoring them isn't an option."

Currently, the focus is on ChatGPT, but as the market grows, there will be increased competition. 

"The technology behind large language models will become infrastructure in the way same as cloud computing," says Lachance. Choosing between the Google or Microsoft brand is rarely a factor in determining whether to use cloud technology, he adds. "The power is in the cloud computing functionality."

Lawyers should also question privacy, data security and information governance issues when entering client information. "The more people have a personal appetite for this technology, the more they will become discerning users," says Lachance.

No one wants to predict what will happen next. The last few months have shown how difficult it is to read where this conversation will go. What is undeniable is the tremendous opportunity to change the practice of law.