In short
- Colombia’s Supreme Courtroom rejected a cassation attraction after AI detectors flagged it as machine-generated.
- Attorneys ran the ruling via the identical instruments and located it additionally appeared AI-written.
- Consultants and research confirmed AI-detection software program produced unreliable and inconsistent outcomes.
The Supreme Courtroom of Colombia denied a cassation attraction, arguing that it was generated by AI. However the identical software the court docket used to find out the attraction’s purported AI origins stated that its personal ruling additionally acquired generative assist.
Is it a double normal by the court docket, or defective instruments at play?
“Faced with a well-founded suspicion that the brief submitted by the attorney had not been drafted by the legal professional himself, the court submitted the text to the Winston AI tool,” the court docket argued. “Its analysis indicated that the document contained only 7% human content, evidencing a marked influence of automated writing and leading to the conclusion that it had been produced using artificial intelligence.”
After operating the evaluation with different instruments that offered comparable outcomes, the court docket dominated that “since the filing cannot be regarded as a duly submitted pleading, its dismissal as inadmissible is required.”
However when the court docket’s ruling confronted comparable scrutiny from authorized consultants, it confirmed comparable outcomes.
“I submitted the text of Auto AP760/2026 from the Supreme Court to the same Winston AI software cited in the ruling,” lawyer Emmanuel Alessio Velasquez wrote on X on Tuesday. “The result: The document contains 93% AI-generated text.”
“If the very ruling that condemns the use of artificial intelligence scores that percentage, the methodological fragility of using these detectors as argumentative support becomes self-evident,” he argued in a subsequent tweet.
Inside hours of the court docket posting a thread concerning the resolution on X, legal professionals started operating their very own exams. Velasquez’s put up went viral in authorized circles, accumulating tens of hundreds of views.
We ran the take a look at on the court docket’s verdict, as effectively, and issues initially didn’t look nice. When GPTZero scanned solely the opening phrases of the court docket textual content, it returned a 100% AI end result.
When the identical software processed an extended model together with the factual background part, it reversed course solely: 100% human.
The software is just not dependable sufficient to be trusted in court docket or in conditions that will require a excessive diploma of certainty.
Colombian attorneys reacted rapidly with their very own experiments. Felony protection lawyer and lecturer Andres F. Arango G, submitted a court docket submitting from 2019, years earlier than the massive language fashions these instruments have been skilled to detect even existed, and it got here again claiming 95% AI technology.
“These tools then invite you to ‘humanize’ the article through their paid services,” he wrote on X, noting an apparent industrial incentive baked into the detection enterprise mannequin.
Nicolas Buelvas ran his 2020 undergraduate thesis on the precept of belief in felony legislation. The end result? 100% AI.
Dario Cabrera Montealegre, one other Colombian lawyer, identified the hypocrisy of counting on know-how to attempt to fight it.
“The court is using AI to determine if there was AI,” he said. “Something contradictory from my practical point of view.”
La Corte usa IA para determinar si hubo IA…!? Algo contradictorio desde mi punto de vista práctico…Si se rechaza debe ser porque como humanos lo detectamos
— Darío Cabrera Montealegre (@dalcamont_daro) March 2, 2026
Past authorized circles, additional tech-savvy people identified the risks of extreme reliance on AI flagging instruments.
“To date, there is no publicly accessible tool that can accurately define the percentage of AI use when drafting a text,” Carlos Alejandro Torres Pinedo argued. “What is worse: No one can publicly verify the source code behind these detection platforms. How can they be used to delegitimize someone’s right of access to justice?”
The technical causes for these failures are well-documented. AI detectors measure statistical patterns: sentence size, vocabulary predictability, and a top quality that researchers name “burstiness,” which refers back to the pure rhythm variation people introduce of their writing.
The issue is that formal authorized prose, tutorial writing, and texts produced by individuals who write in a second language share lots of those self same statistical signatures.
Research on AI detection
A 2023 examine revealed in Patterns discovered that greater than 61% of Check of English as a International Language (TOEFL) essays by non-native English audio system have been incorrectly flagged as AI-generated.
A scientific evaluate by Weber-Wulff that very same yr concluded no accessible software is both exact or dependable. Turnitin acknowledged in June 2023 that its personal detector produced greater false optimistic charges when the AI content material degree in a doc fell under 20%.
Even OpenAI needed to take down its personal AI detection software following fixed inaccuracies and an lack of ability to do its precise job.
Universities have been grappling with this for years. Vanderbilt disabled Turnitin’s AI detector in 2023 after estimating it could generate round 3,000 false positives yearly.
The College of Arizona dropped AI-detection options from its plagiarism software program after a scholar misplaced 20% of a grade on a false optimistic. A 2024 case at UC Davis noticed 17 linguistics college students flagged, 15 of them non-native English audio system.
The sample is constant. The instruments penalize the individuals who write most formally, most repetitively, or most fastidiously, precisely the profile that legal professionals, teachers, and second-language audio system match.
The cultural fallout has bordered on absurdity. Throughout writing and journalism circles, individuals have began avoiding em dashes of their work, not due to any fashion information, however as a result of AI language fashions use them ceaselessly and detection instruments (and folks) have taken discover.
Writers are self-editing pure punctuation out of worry of algorithmic suspicion. Past the written world, artists have suffered the wrath of moderators and colleagues for making artwork items that look AI
Colombia’s two rulings—AC739-2026, through which the Civil Chamber fined a lawyer for citing 10 nonexistent AI-generated precedents in February, and AP760-2026—are rising as among the area’s first judicial choices straight confronting the misuse of generative AI in authorized filings.
Colombia’s judicial department adopted formal tips in December 2024 that regulate how judges and court docket workers can use synthetic intelligence.
The principles enable AI for use freely for administrative and assist duties, reminiscent of drafting emails, organizing agendas, translating paperwork, or summarizing texts, whereas allowing extra delicate makes use of, like authorized analysis or drafting procedural paperwork, solely with cautious human evaluate.
The rules explicitly prohibit counting on AI to judge proof, interpret the legislation, or make judicial choices, emphasizing that human judges stay absolutely accountable for all rulings and should disclose when AI instruments have been utilized in making ready judicial supplies.
These tips, compiled within the “PCSJA24-12243” settlement, may very well be used to contest such a choice.
The Supreme Courtroom has not but issued any extra assertion in response to the backlash over its selection of detection instruments. The ruling didn’t have em dashes, both.
Day by day Debrief E-newsletter
Begin on daily basis with the highest information tales proper now, plus authentic options, a podcast, movies and extra.
