Solving AI Hallucinations in Litigation

Lawyers are an understandably skeptical bunch, especially trial lawyers who are constantly probing the evidence. Not surprisingly, the rise of generative AI tools like ChatGPT has quickly engendered skepticism – even hostility – within the trial bar because they can convincingly make stuff up, i.e., hallucinate.

AI hallucinations in the legal realm are a problem. The lawyers who were sanctioned for using ChatGPT to write a brief they filed – that was filled with fake cases – certainly come to mind. According to a recent Stanford University study, these tools were wrong up to 33% of the time when tested against a benchmark of more than 200 legal questions. The study included legal products offered by Westlaw and Lexis.

Generative AI systems operate by submitting a question or instructions, to which the system provides a response or answer. Hallucinations are plausible-sounding but factually or legally incorrect responses. For example, a generative AI might confidently cite a fake case or statute or falsely claim that a real case supports its answer. While some wags might say lawyers have been doing this since ancient times, generative AI does it much faster and with more zeal.

Recognizing the problem, the American Bar Association recently published Ethics Opinion 512 to provide guidance on the ethical use of generative AI. It follows the pattern of several state bars that were quick to address this new technology. All the guidance mandates a human review of any generative AI legal work product. (We have previously discussed these ethical obligations discussed in a post you can read here.)

While generative AI can speed up time-intensive legal tasks, it does not excel at all of them. Experts have noted that there is a “jagged frontier” that is still being explored to determine those tasks at which AI is more successful.

Almost daily there is an announcement from one law firm or another undertaking a pilot program (or crash course) in applying generative AI to their business. Leaders are calling the shots about what to try, but it is the day-to-day users – lawyers, paralegals, and law clerks – that are at the forefront of deciding which tasks are best suited for generative AI.

Some assume that because generative AI tools can answer questions like “How big is the sun”, with access to a comprehensive database of case law, they can accurately provide answers based on them. Unfortunately, no. The Stanford researchers highlight what is required for effective legal research: “to synthesize facts, holdings, and rules from different pieces of text while keeping the appropriate legal context in mind.” At least for now, legal research is beyond that frontier.

Fortunately, there are some time-consuming legal tasks that are being solved with generative AI. The earliest products were aimed at contract drafting and review tasks, but newer ones are coming online for litigation. At esumry, we are focused on using generative AI for trial work to improve accuracy and reduce the time required for deposition analysis and summarization.

Deposition analysis software is a useful case study to understand why AI can work today, but also what trial lawyers and their law firms must look for to ensure generative AI products are accurate and secure.

Why Generative AI for Deposition Analysis Works

Deposition analysis is a great use case for generative AI. As trial lawyers know, digests and summaries are essential for motions practice and trial preparation:

Grounded in the text: Generative AI can be effectively focused on the information developed in each deposition.
Human in the loop: Tools that keep a human in the loop eliminate the risk of hallucination and ensure compliance with state bar ethics rules.
Easier to verify: Unlike legal research which requires synthesizing a corpus of case law, deposition analysis focuses on a single document. This makes it easy to pinpoint specific testimony and validate the generative AI output.

The Advantages of Professional Grade Legal AI

While deposition analysis is a solid use case for generative AI, the best AI for this task must be purpose-built and guided by litigation experts so it is accurate and trustworthy:

AI tested by attorneys: Ensures results are accurate, reliable, and conform with best practices to meet the needs of trial lawyers.
Confidentiality and security: With stringent security protocols and testing, purpose-built generative AI products protect confidential data, which is never used to train language models.
Accuracy: By keeping human users in the loop during the process, it ensures both AI accuracy and compliance with ethical rules on supervision.
Speed: Designed specifically for the task of deposition analysis, the use of generative AI and automation enables getting this task done significantly faster than traditional methods.
Explainability: Provides specific references to testimony underlying summaries and digests to verify accuracy.

Here’s how these fit together to improve the task of deposition analysis:

Automated processing: The AI quickly analyzes, indexes, summarizes and structures deposition testimony for downstream tasks.
Faster workflows: By eliminating the need for heavy typing or multi-step copy/pasting, legal professionals can quickly create user-defined digests of deposition testimony.
Client reporting: Generative AI automatically produces narrative summaries of each deposition for use in client reporting regarding the details of witness testimony.
On-screen editing: Unlike many generative AI outputs that cannot be edited, purpose-built on-screen editing greatly simplifies the process of utilizing AI output to easily add context or annotate with additional user insights.
Exporting: Easily export completed summaries and digests as fully formatted Word documents for individual or team use.

The legal industry has generally been cautious about new technologies, and for good reason. By building generative AI products that are well-suited for tasks like deposition analysis and summaries, and knowing what to look for in these products, trial lawyers and law firm leaders can help themselves without compromising on accuracy or professional ethics.

AI isn’t replacing lawyers. It’s enabling creative tools that amplify legal expertise, allowing legal professionals to focus on the high-value, strategic aspects of their roles.

While many legal tasks are beyond the “jagged frontier” of what generative AI can do, there are big wins available right now in some areas, including deposition analysis. Trial lawyers and their teams can help themselves by identifying these opportunities, preventing legal AI hallucinations, and choosing generative AI products that are lawyer-led, consistent, and trustworthy.

Try It Out!

We invite you to see firsthand how AI and automation can revolutionize your deposition analysis.

Get a 14-day free trial of esumry today and start transforming your litigation practice for the future.

Learn more

Scott GarrisonAugust 8, 2024

Solving AI Hallucinations in Litigation

While generative AI can speed up time-intensive legal tasks, it does not excel at all of them. Experts have noted that there is a “jagged frontier” that is still being explored to determine those tasks at which AI is more successful.

Deposition analysis software is a useful case study to understand why AI can work today, but also what trial lawyers and their law firms must look for to ensure generative AI products are accurate and secure.

Why Generative AI for Deposition Analysis Works

Deposition analysis is a great use case for generative AI. As trial lawyers know, digests and summaries are essential for motions practice and trial preparation:

Grounded in the text: Generative AI can be effectively focused on the information developed in each deposition.

Human in the loop: Tools that keep a human in the loop eliminate the risk of hallucination and ensure compliance with state bar ethics rules.

Easier to verify: Unlike legal research which requires synthesizing a corpus of case law, deposition analysis focuses on a single document. This makes it easy to pinpoint specific testimony and validate the generative AI output.

The Advantages of Professional Grade Legal AI

While deposition analysis is a solid use case for generative AI, the best AI for this task must be purpose-built and guided by litigation experts so it is accurate and trustworthy:

AI tested by attorneys: Ensures results are accurate, reliable, and conform with best practices to meet the needs of trial lawyers.

Confidentiality and security: With stringent security protocols and testing, purpose-built generative AI products protect confidential data, which is never used to train language models.

Accuracy: By keeping human users in the loop during the process, it ensures both AI accuracy and compliance with ethical rules on supervision.

Speed: Designed specifically for the task of deposition analysis, the use of generative AI and automation enables getting this task done significantly faster than traditional methods.

Explainability: Provides specific references to testimony underlying summaries and digests to verify accuracy.

Here’s how these fit together to improve the task of deposition analysis:

Automated processing: The AI quickly analyzes, indexes, summarizes and structures deposition testimony for downstream tasks.

Faster workflows: By eliminating the need for heavy typing or multi-step copy/pasting, legal professionals can quickly create user-defined digests of deposition testimony.

Client reporting: Generative AI automatically produces narrative summaries of each deposition for use in client reporting regarding the details of witness testimony.

On-screen editing: Unlike many generative AI outputs that cannot be edited, purpose-built on-screen editing greatly simplifies the process of utilizing AI output to easily add context or annotate with additional user insights.

Exporting: Easily export completed summaries and digests as fully formatted Word documents for individual or team use.

AI isn’t replacing lawyers. It’s enabling creative tools that amplify legal expertise, allowing legal professionals to focus on the high-value, strategic aspects of their roles.

Try It Out!

We invite you to see firsthand how AI and automation can revolutionize your deposition analysis.

Get a 14-day free trial of esumry today and start transforming your litigation practice for the future.

Solving AI Hallucinations in Litigation

While generative AI can speed up time-intensive legal tasks, it does not excel at all of them. Experts have noted that there is a “jagged frontier” that is still being explored to determine those tasks at which AI is more successful.

Deposition analysis software is a useful case study to understand why AI can work today, but also what trial lawyers and their law firms must look for to ensure generative AI products are accurate and secure.

Why Generative AI for Deposition Analysis Works

Deposition analysis is a great use case for generative AI. As trial lawyers know, digests and summaries are essential for motions practice and trial preparation:

Grounded in the text: Generative AI can be effectively focused on the information developed in each deposition.

Human in the loop: Tools that keep a human in the loop eliminate the risk of hallucination and ensure compliance with state bar ethics rules.

Easier to verify: Unlike legal research which requires synthesizing a corpus of case law, deposition analysis focuses on a single document. This makes it easy to pinpoint specific testimony and validate the generative AI output.

The Advantages of Professional Grade Legal AI

While deposition analysis is a solid use case for generative AI, the best AI for this task must be purpose-built and guided by litigation experts so it is accurate and trustworthy:

AI tested by attorneys: Ensures results are accurate, reliable, and conform with best practices to meet the needs of trial lawyers.

Confidentiality and security: With stringent security protocols and testing, purpose-built generative AI products protect confidential data, which is never used to train language models.

Accuracy: By keeping human users in the loop during the process, it ensures both AI accuracy and compliance with ethical rules on supervision.

Speed: Designed specifically for the task of deposition analysis, the use of generative AI and automation enables getting this task done significantly faster than traditional methods.

Explainability: Provides specific references to testimony underlying summaries and digests to verify accuracy.

Here’s how these fit together to improve the task of deposition analysis:

Automated processing: The AI quickly analyzes, indexes, summarizes and structures deposition testimony for downstream tasks.

Faster workflows: By eliminating the need for heavy typing or multi-step copy/pasting, legal professionals can quickly create user-defined digests of deposition testimony.

Client reporting: Generative AI automatically produces narrative summaries of each deposition for use in client reporting regarding the details of witness testimony.

On-screen editing: Unlike many generative AI outputs that cannot be edited, purpose-built on-screen editing greatly simplifies the process of utilizing AI output to easily add context or annotate with additional user insights.

Exporting: Easily export completed summaries and digests as fully formatted Word documents for individual or team use.

AI isn’t replacing lawyers. It’s enabling creative tools that amplify legal expertise, allowing legal professionals to focus on the high-value, strategic aspects of their roles.

Try It Out!

We invite you to see firsthand how AI and automation can revolutionize your deposition analysis.Get a 14-day free trial of esumry today and start transforming your litigation practice for the future.

We invite you to see firsthand how AI and automation can revolutionize your deposition analysis.

Get a 14-day free trial of esumry today and start transforming your litigation practice for the future.