top of page
Search
Sarah Anderson & Julian Mahfouz
Jul 17, 20233 min read

ChatGPT Fabricates Content


It was inevitability that ChatGPT would worm itself into the practice of law. However, the staggering incompetence (or indifference) of the lawyers involved in a recent tort case gave a bombastic introduction of the chat bot into U.S. Courts. Spoiler Alert: ChatGPT was unwelcomed.


In Robert Mata v. Avianca, Inc. (22-cv-1461 (PKC)), U.S. District Court Judge P. Kevin Castle issued sanctions and scathing remarks following a legal counsel’s multiple uses of ChatGPT, which included false citations and fabricated case law. A basic tort suit, the plaintiff, Roberto Mata filed suit after allegedly injuring his knee with a metal serving cart aboard an Avianca Inc. airline flight from El Salvador to New York on August 27, 2019.


Represented by the Levidow & Oberman P.C. law firm, specifically by Steven Shwartz, Mata initially filed his claim in New York State Court before being removed to the U.S. District Court for the Southern District of New York under federal question jurisdiction. With Schwartz unable to practice in federal court, Peter LoDuca, a separate member of the Levidow firm assumed public representation while Schwartz assisted with the legal research and writing.


On March 15, 2023, Avianca filed a reply memorandum[1] in support of its Motion to Dismiss, stating that 5 cases cited by Mr. Mata in an opposition brief could not be found. For those unfamiliar with the legal research process, two primary professional legal research databases, namely Lexis Nexis ® and Westlaw ®, provide attorneys with electronic access to all reported state and federal court decisions on a near-time basis. After the court was also unable to find the plaintiff’s cited authorities, the Court issued an April 12, 2023, order requiring LoDuca to produce one of the phantom cases.


Despite already failing to adhere to professional standards by citing non-existent cases, neither LoDuca nor any other member of the Levidow law firm (including Schwartz) elected to admit fault and “careless” error. Instead, the attorneys chose to double down, returning to ChatGPT, and asking the technology tool to produce the cases.


ChatGPT is a revolutionary technology; however, it is still a chat bot, designed to mimic human speech, not perform accurate legal research. It is well documented that ChatGPT will make up references. In one test[2], ChatGPT was asked to produce a simple argument and provide at least six references back its argument. Five of those six references were fake. Therefore, when the Mata’s lawyers asked ChatGPT to produce the cases it previously cited, it complied, instantly producing excerpts from all but one of the fake cases. A cursory examination of any one of these excerpts would reveal to any diligent lawyer that they were bogus. For example, the fake Martinez[3] case shifts fonts and contains grammatical errors. Still, Mata’s lawyers continued to barrel onwards towards destruction. On April 25, 2023, LoDuca submitted an affidavit containing all the fake excerpts that ChatGPT produced, representing them as legitimate.


The Court was horrified by the lawyers’ lack of due diligence after reviewing the affidavit. Judge Castle noted that the excerpts “shows stylistic and reasoning flaws that do not generally appear in decisions issued by United States Courts of Appeals. Its legal analysis is gibberish.”[4] He also pointed out that the excerpts were “difficult to follow” and “nonsensical.”


On May 26, 2023, Judge Castle Ordered the Levidow attorneys to appear and personally explain their malfeasance. Indirectly blaming the technology, Schwartz stated, “I falsely assumed [ChatGPT] was like a super search engine called ChatGPT, and that's what I used.” Wrongly assuming that ChatGPT was infallible. Schwartz trusted ChatGPT’s opinions more than the fact that neither the defense nor the Court could find any of the cases in any legal database. LoDuca claimed that he worked with Schwartz for over twenty years, and that he trusted his work product. Unfortunately for both lawyers, these excuses were not enough to escape sanctions.


Judge Castle imposed a $5,000.00 fine and ordered the lawyers to send letters to all the Judges listed as authors of the five fake opinions, as well as include the fake cases and the full transcript from the sanctions hearing. Judge Castle further instructed Levidow lawyers to send the same letter to their client, Mr. Mata, with the same transcript to fully inform him of their errors. In short, Judge Castle ordered the Levidow attorneys to admit malpractice to their client, to the Court, and the judges identified in the five fabricated cases. Severe ripple effects of the Levidow attorneys’ behavior and the sanctions with the New York Bar Association are expected.


Although chat bots are attractive and convenient, the Levidow attorneys represent a cautionary tale against blindly trusting such tools. Apart from the lack of confidentiality and privilege concerns, the Mata sanctions order adds “falsification of content” to the list of reasons why professionals should avoid ChatGPT.


50 views0 comments

Comments

Post: Blog2_Post
bottom of page