top of page

AI in Litigation: What Hath ChatGPT Wrought?

Credits: Sistine Chapel. With apologies to Samuel Morse (The Telegraph – What Hath God Wrought?)

In the ever-evolving world of technology, few tools have made as immediate and profound an impact on the legal landscape as ChatGPT. Designed to generate human-like text based on the input it receives, it has become a focal point of debate.

What does it mean for the future of litigation?

The Rise of ChatGPT

In the vast ecosystem of artificial intelligence, ChatGPT has emerged as a marvel. Our company began experimenting with GPT-3 in 2021 but it wasn’t until the release of ChatGPT in November 2022 that it really showed its prowess in generating human-like text. Now a year later, ChatGPT and other Large Language Models (LLMs) like it are not just technological feats; they are revolutionizing many industry sectors, including the legal domain. But like all powerful tools, LLMs come with challenges.

Earlier this year, a lawyer in New York, drawn by the allure of efficiency and innovation, relied on a legal brief written by ChatGPT, only to find himself at the center of controversy for citing made-up cases. (These are referred to as hallucinations.) The Manhattan Incident was a stark revelation of the potential pitfalls of unchecked AI in legal practice. This wasn't a mere oversight; it was a result of the blurred lines between ChatGPT-generated content – which sounded remarkably authoritative but was actually false – and authentic legal sources to which it had no access.

The incident raised concerns and prompted introspection within the legal community. How did we get here? The answer lies in the rapid advancements of LLMs and the pressure to use these tools for a competitive edge. While ChatGPT's capabilities are commendable, its unchecked use in legal filings could undermine the very foundation of the justice system.

The litigation world found itself grappling with a conundrum: How do we harness the power of such AI tools without compromising the integrity of legal proceedings? The Manhattan Incident was a wake-up call, signaling that the integration of AI with law isn't just about efficiency; it's about responsibility, diligence, and a deep understanding of the tools at our disposal.

The Disclosure Directives

In the aftermath of the Manhattan Incident, the legal community found itself at a pivotal juncture. The judiciary, recognizing the profound and problematic implications of unchecked use of these LLMs for case filings, started taking action.

Judge Brantley Starr's mandate from Texas was groundbreaking, requiring lawyers to certify their non-reliance on unchecked AI-generated content. He was sending a clear message: the legal system will not tolerate the blurring of lines between human expertise and AI output. Other courts have started to implement similar requirements. These directives have raised several questions. Are they isolated responses to the ChatGPT incident, or indicative of a broader trend? Will state courts (or legislatures) follow suit? And most importantly, is this the beginning of an era where every AI tool will come under scrutiny?

The Florida bar's contemplation of a client consent requirement added another layer to the debate. On the surface, it seems like a logical step—after all, shouldn't clients be informed if AI tools are shaping their legal strategies? But dig deeper, and complexities emerge. Would such a requirement deter lawyers from using AI altogether? Would it create an unnecessary burden on clients, many of whom might not fully grasp the nuances of AI?

These reactions, while rooted in genuine concerns, also hint at a broader issue: the legal profession's struggle to keep pace with technological advancements. The rapid advances of AI tools like ChatGPT have outstripped the legal framework's ability to regulate them. The disclosure mandates, in many ways, are symptomatic of a system grappling with the challenges of integrating cutting-edge technology while preserving the sanctity of legal processes.

Efficiency vs. Ethics

The allure of AI in the legal realm is undeniable. In an industry often criticized for its resistance to change, tools like ChatGPT represent a beacon of modernization. They promise faster research, streamlined processes, and a level of precision that's hard to achieve manually. But with great power comes great responsibility, and the legal profession is now grappling with the ethical implications of this newfound power.

At the heart of the debate is a fundamental question: Where does the line between human judgment and AI recommendation lie? AI tools, with their vast databases and lightning-fast computations, can provide insights and suggestions that might elude even the most seasoned attorney. But they lack the human touch—the intuition, experience, and ethical grounding that define the legal profession.

The Manhattan Incident with ChatGPT is a stark reminder of the pitfalls of over-reliance on AI. While the tool might have generated a convincing legal argument, it lacked the understanding and veracity that are the cornerstones of legal practice. This incident demonstrated the tension between efficiency and ethics.

Moreover, the ethical quandary extends beyond just the truthfulness of AI-generated content. Consider client confidentiality, a sacrosanct principle in law. Can lawyers ensure that AI tools, which often rely on vast amounts of data, respect and uphold this principle? And what about the duty of zealous representation? If an AI tool offers a strategy or argument that the lawyer believes is not in the client's best interest, who prevails?

The challenge for the legal profession is to strike a balance. Embracing AI tools for their efficiency while ensuring that ethical standards are not compromised requires continuous education, rigorous oversight, and a commitment to the core values that have always defined the legal profession.

Are Existing Safeguards Enough?

The legal profession, with its rich history and traditions, has always been anchored by a set of professional norms and ethical guidelines. These safeguards, established over centuries, are designed to ensure the integrity of legal practice. In the face of the AI revolution, one can't help but wonder: Do these existing safeguards provide a sufficient buffer against the potential pitfalls of AI?

Rule 11 of the Federal Rules of Civil Procedure and state court analogs stand as a bulwark against what is commonly referred to as “frivolous litigation.” These rules mandate that attorneys ensure the legal and factual support for claims made to the courts, effectively holding them accountable for. In the context of the ChatGPT incident, Rule 11 raises a pertinent question: Doesn't this rule already encompass the challenges posed by AI-generated content? If lawyers are already bound to verify the accuracy of their submissions, are additional mandates specifically targeting AI tools needed?

The ethical duty of lawyers extends beyond just the content of their submissions. They are also responsible for supervising the work of non-lawyer assistants. In the age of AI, some have argued that tools like ChatGPT can be viewed as new age "nonlawyer assistants" covered by Model Rule of Professional Responsibility 5.3. They assist, they aid, but they don't replace the human touch. The existing rules around supervision of non-lawyer work could very well apply to AI tools. After all, isn't the principle the same? Whether it's a paralegal or a sophisticated AI tool, the lawyer remains responsible for the final output.

However, while these existing safeguards provide a foundation, they were crafted in a world where AI's influence was minimal or non-existent. (The ABA’s Model Rule 5.3 still implies that nonlawyers are “persons” not bots.) Relying solely on them might be akin to fitting a square peg in a round hole. The rapid rise of AI tools demands a fresh look at these safeguards, ensuring they are robust enough to address the unique challenges posed by AI.

Beyond ChatGPT: The Expanding AI Horizon in Litigation

While ChatGPT has been at the epicenter of recent debates, it's essential to recognize that it is part of a broader wave of AI tools entering the legal domain. These tools, each with their unique capabilities, promise to revolutionize various facets of litigation.

Legal Writing Tools: AI tools promise to refine legal arguments, ensuring they are concise and clear. They can also offer grammar and style checks. But they can also hallucinate and offer falsehoods as witnessed in the Manhattan Incident. Legal narratives are complex, weaving together various elements. AI changes in phrasing or tone can influence the content of arguments. All of this raises important questions: How much influence should an AI tool have on legal arguments? And where do we draw the line between human judgment and AI recommendations? In the quest for brevity, is there a risk of losing depth or oversimplifying arguments?

Litigation Work Tools: The courtroom, traditionally seen as a bastion of human judgment and advocacy, is not immune to the AI wave. AI-driven tools are emerging as powerful allies for trial attorneys. These tools can analyze vast amounts of case law in mere seconds, identifying patterns that are not apparent even to experienced researchers. Others can assist in predicting case outcomes based on historical data, helping lawyers strategize more effectively. Some tools like e-discovery help with evidence analysis, sifting through mountains of documents to help litigators focus on manageable datasets. Others like CaseChat™ can perform important litigation tasks such as finding relevant testimony, summarizing evidence, and drafting witness outlines. While the potential benefits are immense, they come with their set of challenges. The reliance on AI for case strategy raises ethical questions: What if the AI's prediction is wrong? How much should lawyers rely on these tools versus their own judgment?

The use of AI in litigation brings both opportunities and challenges. The trial bar, rooted in human expertise and judgment, is at a crossroads. The influence of AI-driven recommendations is undeniable, but it is essential to integrate them in a manner that upholds the responsibilities to clients, the judiciary, and opposing counsel.

The Road Ahead: Navigating an AI-Infused Legal Landscape

The integration of AI tools like ChatGPT into the legal profession is not a fleeting trend; it's the dawn of a new era. As we stand at this technological crossroads, the path forward is filled with both promise and uncertainty. Here are a few thoughts as we travel together.

Embracing Change While Upholding Tradition: The legal profession is steeped in tradition, with centuries-old practices and principles. As AI tools offer unprecedented efficiency and innovation, there is a growing realization that change is not just inevitable; it is necessary. However, this shift must be tempered with caution. The core values of the legal profession—integrity, diligence, and the pursuit of justice—must remain at the forefront, even as we navigate the uncharted waters of AI integration.

Setting Clear Boundaries: While AI tools bring a host of benefits, unchecked reliance on them can lead to pitfalls, as evidenced by the Manhattan Incident. The legal community must establish clear boundaries for AI use. This includes setting guidelines on when and how to use AI tools, ensuring rigorous oversight, and emphasizing the importance of human judgment.

Continuous Education and Training: The rapid advancements in AI mean that today's cutting-edge tool could be tomorrow's obsolete technology. For legal professionals to harness the full potential of AI, continuous education and training are paramount. This not only involves understanding the functionalities of AI tools but also their ethical implications and potential biases.

Collaborative Efforts: The challenges posed by AI in the legal domain cannot be tackled in isolation. It requires a collaborative effort, involving legal professionals, technologists, ethicists, and policymakers. By fostering a dialogue between these stakeholders, we can ensure that the integration of AI is both responsible and beneficial.


The road ahead is undeniably complex. The fusion of AI and law promises to reshape the very fabric of society and it is clear the influence of AI tools extends far beyond just ChatGPT. The profession is on the brink of a broader AI transformation. The decisions made today will shape the contours of future legal practice.

As AI makes its presence increasingly felt in the courtroom, judges, trial lawyers, and other stakeholders must grapple with finding the right balance between human expertise and machine efficiency. With thoughtful navigation, clear guidelines, and an unwavering commitment to the core values of the profession, the future of litigation in the age of AI is bright.

Want to try CaseChat™ to see how it can transform your litigation? Join our waitlist.


bottom of page