# When AI Analyzes Data: Four Errors That Passed Every Automated Check
### An Experiment in Trusting Machines with Small Datasets
What happens when you hand a business-ready dataset to an AI model and ask it the kinds of questions your team asks every week? A group of researchers decided to find out — and the results are both illuminating and unsettling.
The experiment used three compact datasets and one AI model, posed with typical weekly business questions: average delivery time, regional performance leaders, and athlete counts in a sports file. The twist? After the model returned its answers, the researchers ran a second review pass — giving the model its own output back and asking it to verify every figure as if those numbers were headed straight into an executive presentation.
One review caught a wrong row count and flagged a backwards conclusion. The other invented a correction and converted an accurate answer into an incorrect one. Every result in this experiment is reproducible, using Pandas and SciPy across multiple runs.
—
## The Data: Three Files, Three Traps
### The Shipment File
The first dataset tracks orders for 40 customers, with each row representing a single order placed between January 1 and January 21, 2024. Every order carries three timestamps: when it was placed, when it left the warehouse, and when it arrived at the destination. The final column holds the dollar amount of each order.
Looking closely at the last entry, something stands out: the order was placed but never shipped and never delivered. This pattern repeats 18 times out of the 40 rows in the file.
### The Regional Sales File
The second dataset is meant to show one row per region per year — covering 8 regions across years from 2007 through 2025, with a single sales figure for each pairing. That structure, however, is broken in ways that don’t show up in the column headers. Six region-year combinations appear more than once. Three duplicates sit entirely within one region. Four pairings contain contradictory sales figures. And the coverage varies wildly — one region has 15 years of history while another has only 1.
### The Olympics Athletes File
The third dataset lists one row per athlete per event, spanning 352 rows, 336 athletes, 15 Olympic Games, and 167 events. Eleven athletes appear more than once, and one athlete appears six times. A medal column is populated for 120 rows, with blanks indicating no medal was won in that event.
—
## Error 1: Answering the Wrong Question Entirely
When asked about average delivery time, the model computed the gap between ship date and delivery date and reported 2.6 days. The number was mathematically correct — but it measured warehouse speed, not customer wait time. The question asked about order-to-door duration, which starts when a customer checks out, not when a parcel leaves the facility. The correct answer was 6.09 days.
This kind of error is especially dangerous because the code runs cleanly and the arithmetic is sound. There is no exception to catch, no type mismatch to flag. The only way to detect it is to read the original question alongside the column names used in the calculation and compare them manually. Ship-to-door and order-to-door both measure real things, but they measure different realities — and no one-line summary in a meeting slide will reveal the swap.
## Error 2: Numbers That Never Passed Through Code
A cautionary note appeared at the bottom of the shipment analysis: only 22 of 50 orders had delivery dates, with 28 still in transit. The file has 40 rows. The 22 is accurate, but neither 50 nor 28 was ever computed. They appeared in the prose without any corresponding code, and they form a tidy arithmetic pair (50 minus 22 equals 28) that makes them feel plausible. A reader mentally checking the math finds nothing wrong — because the error is not in the arithmetic but in the origin of the figures.
The regional sales run suffered the same flaw with greater consequences. When asked which region performed best, the model reported APAC at $3.68 million and 32% of all revenue. In reality, APAC’s total was 3,675.49 in the file’s native units — a thousandfold inflation — and its actual share was 30.4%. The model had printed a code block and written numbers beneath it without executing anything. The prose and the code were completely disconnected.
Even more troubling, a separate unhurried run of the regional data executed all its queries correctly and still produced a false comparative claim — stating APAC was “more than 60% above” two other regions combined when the actual gap was under 3%. Meanwhile, a different comparison in that same sentence was genuine. One figure was measured and one was fabricated, sitting side by side in the same paragraph.
The core issue is this: everything inside a code block was computed by the machine. Everything in the surrounding prose was written by it. No mechanism forces the two to agree.
## Error 3: Mistaking Missing Evidence for Speed
When asked whether shipping was getting faster, the model compared the first and third weeks of orders and declared a clear improvement — from 3.2 days down to 1.0. Both numbers are real. Both were computed correctly. The conclusion was completely wrong.
The file ends on January 21. Week 3 orders had roughly three days to complete, while week 1 orders had 17. Of week 3’s 10 orders, seven still had no delivery date at all. Only the fastest week 3 orders had been delivered — the slow ones hadn’t arrived yet and were invisible to the calculation. As weeks progressed, unresolved orders climbed from 20% to 70%, and the reported average fell simply because the denominator kept losing its worst cases.
A separate unhurried run of the same model caught this unprompted, opening with a warning that the apparent improvement was an illusion. Same data. Same model. Different outcome — because the second run gave itself the space to think.
## Error 4: Disappearing Rows and False Correlations
When asked whether taller athletes win more medals, the model compared the average height of medalists against non-medalists and found a difference of just 0.3 centimeters. The verdict was that height does not matter.
The calculation itself was correct, but it ran on only 126 of the file’s 352 rows. The remaining 226 rows had no height recorded, and Pandas silently dropped every one of them without any indication. The sample shrank by 64% between the question and the answer, and the reply never acknowledged the loss.
Digging deeper, the missing height values turned out to be anything but random. Athletes without recorded heights came almost entirely from the earliest Games, and their medal rate was 23%. Athletes with height data had a 54% medal rate — and almost all of those rows came from 2016 onward, when 80% of athletes medaled. Having a recorded height, being recent, and winning a medal turned out to be nearly the same fact. The 126 rows the model tested were heavily skewed toward the era where most everyone won a medal.
The honest answer to whether height helps was that the file cannot support that question. A stakeholder is better served by that admission than by a 0.3-centimeter difference dressed up as an insight.
## What the Review Pass Revealed
For every initial answer, the researchers opened a fresh session, pasted the original response back in, attached the same data, and asked the model to verify every number as though it were preparing for an executive deck.
The review of the shipment analysis caught one fabricated figure and corrected it. But then it signed off on the remaining errors without comment — endorsing a delivery metric that answered the wrong question and endorsing a speed trend that was an artifact of missing data. The reviewer had the count of undelivered orders on screen and still never connected it to the trend claim on the line below.
The review of the Olympics analysis went even further astray. It correctly spotted a grain issue in a different part of the answer, then turned to the height comparison. It never noticed the 226 missing rows, invented new averages that no single grouping of the data produces, and reversed the original verdict entirely. It then claimed statistical significance for a relationship that the very same 126 rows showed no evidence of. The session executed no code at all.
Across all four mistakes, the review fixed the fabricated numbers, approved the wrong-metric answer, approved the missing-data bias, and actively worsened one answer by inventing a false correction. Every verdict arrived in the same confident tone, and nothing in the language distinguished a correct review from an incorrect one.
—
## Frequently Asked Questions
**Q: Why is it so hard for AI models to catch these kinds of errors?**
A: These mistakes don’t involve broken code or invalid syntax. They involve gaps between what a question asks for and what a column contains, data that was never generated in the first place, evidence that hasn’t arrived yet, and rows that were silently excluded. Each of these issues lives outside the scope of what automated checks can detect, because everything looks valid on the surface.
**Q: Does running code protect against these mistakes?**
A: Not entirely. In one case, the model executed all its queries correctly and still manufactured a false comparative claim in the paragraph around the code. The numbers inside a code block were computed; the numbers in the text around it were written. Nothing forces those two to match.
**Q: What is the most dangerous type of error described in the experiment?**
A: The ones with no bug to find. When the model subtracts the wrong two dates, the subtraction is perfectly valid. When it cites a row count that was never computed, the sentence reads like good practice. When it drops rows with missing values, Pandas does exactly what it is designed to do. These errors cannot be caught by tests, type checks, or exception handling — only by a human who reads both the question and the calculation side by side.
**Q: Does the review pass help at all?**
A: It helps sometimes. In the shipment dataset, it caught a fabricated figure and corrected it. But across the board, it approved errors it should have flagged and, in one case, actively made an answer worse by inventing a false correction. The confidence of the tone never varied, whether the verdict was right or wrong.
**Q: How can teams protect themselves when using AI for data analysis?**
A: Three practices stand out. First, manually verify that the question being asked matches the columns and calculations used in the code. Second, check whether every number in the prose appears somewhere in the actual code output, and confirm that the code actually ran. Third, examine how many rows survived each calculation and whether the excluded rows are random or systematic. None of these checks require specialized tooling — they require a human reading the output with a critical eye.
**Q: Were the AI models in this experiment performing poorly overall?**
A: Not at all. The mechanical work — parsing dates, writing valid code, producing analysis — was strong throughout. In several cases, the analysis was sharper than what many human analysts would produce. The errors were not failures of computation; they were failures of judgment, context, and honesty about what the data can and cannot support.
—
## Conclusion
AI models are remarkably capable at the mechanical side of data work. They parse dates, write valid queries, handle statistical tests, and in unhurried runs produce analysis that can rival skilled practitioners. But the four errors uncovered in this experiment share a common thread: each one hinged on something the data did not explicitly contain. The question sitting behind the metric. The code that was never executed. The orders that had not yet arrived. The rows with no height recorded.
Knowing what a number means — and what it does not mean — remains something that cannot be handed over to a machine. Automated review catches arithmetic mistakes, but it does not catch a correct answer to the wrong question, or a plausible sentence built from numbers that were never computed, or a trend manufactured by the very evidence that was missing.
Use the second pass to check the math, then work through these four checks yourself. Because the review will tell you the numbers are correct either way.
Thank you for reading
