# Can AI Detectors Tell Human Writing from Machine-Generated Text? A Hands-On Experiment
## The Growing Problem of AI-Modified Text
A pair of influential 2024 studies have drawn attention to how AI-generated text is spreading through scholarly and digital ecosystems. One study, published at a major machine learning conference, estimated that between roughly 6 and 17 percent of peer review text submitted to leading artificial intelligence conferences after ChatGPT’s launch showed signs of substantial AI authorship. These were not casual product reviews or forum comments — they were technical evaluations written by researchers with real consequences for paper acceptance and rejection.
Around the same time, a separate study published in *Nature* demonstrated a concerning failure mode: when language models are trained repeatedly on text that other models have generated, they gradually lose access to rare patterns from the original data and begin producing narrower, degraded outputs. This “model collapse” phenomenon suggests that the long-term health of AI systems depends on maintaining diverse, human-origin training material.
These findings raise a practical question for anyone working with text data: how much of the messy, unstructured text in real datasets is machine-generated, and can inexpensive tools reliably tell the difference?
## Why Standard Data Cleaning Falls Short
For years, the most common complaint about data science workflows has been that the bulk of effort goes toward cleaning messy data. Missing values, duplicate rows, and out-of-range entries are familiar enemies. But a newer kind of mess has emerged: fluent, well-structured text that happens to have been written by an artificial intelligence system rather than a human.
Traditional data validation checks — looking for null fields, duplicate records, or values that fall outside an expected range — do nothing to answer the question of authorship. A paragraph of AI-generated text can look perfectly well-formed while carrying none of the idiosyncrasies, imperfections, or personal voice that characterize human writing.
This matters because downstream machine learning models — sentiment classifiers, topic categorizers, recommendation engines — all depend on the quality and origin of their training data. If a significant fraction of that data was produced by another AI system, the models built on it may inherit hidden biases, narrower coverage, or distorted patterns.
## Designing an Experiment to Test Detection Methods
To explore whether simple, low-cost checks could flag AI-generated text, and whether removing flagged content would improve a sentiment classification model, a set of controlled experiments was carried out using a publicly available movie review dataset.
The dataset contained 1,000 movie reviews collected from a well-known public film database, each labeled as positive or negative. It was published in early 2019, well before the public release of ChatGPT in late 2022, so the original reviews in the collection were treated as human-written references for the purposes of the experiment. The dataset covers 10 different films and is distributed under a permissive open license that allows reuse and adaptation.
The experiment proceeded in three stages: first, scanning the full collection for text that looked machine-generated; second, testing three different detection methods on a curated subset of 600 reviews; and third, measuring how filtering affected the performance of a sentiment classification model.
## Building the Test Dataset
To create a mixed dataset with known labels, 400 new reviews were generated by two different language models. Each model was asked to invent a fictional movie and write a short review expressing either a positive or negative sentiment — 100 positive and 100 negative from each model. No source examples were provided, and the models were not asked to imitate any particular reviewer.
These 400 generated reviews were then combined with 200 reviews drawn from the original Mendeley collection as a “human reference” group. The training set therefore contained 600 rows total: 200 treated as human-written and 400 known to be machine-generated. An additional 200 reviews from five films not seen during training were held out as a separate test set to evaluate the downstream classifier.
## Three Approaches to Spotting AI-Generated Text
Each of the 600 reviews was scored using three different detection methods, plus an averaged combined score.
**Perplexity** measures how predictable a piece of text is to a language model. AI-generated text often uses familiar word sequences and patterns, which can result in lower perplexity scores. However, human writing can also be predictable — especially in formulaic or factual prose — so perplexity alone is an imperfect signal.
**Near Duplicate Similarity** converts each review into a numerical vector, or embedding, that captures some of its meaning and wording. It then compares a review to its closest match in that numerical space. A high similarity score indicates that another review shares a great deal of the same wording or meaning.
**Embedding Density** takes a slightly different approach. Instead of looking at the single closest match, it measures the average distance between a review and its five nearest neighbors in embedding space. The idea is that if AI-generated reviews tend to cluster together, they will have shorter distances to their neighbors than the more varied human-written reviews.
The **combined score** averages the three individual scores after normalizing each to a zero-to-one range. Higher values indicate a stronger flag for possible AI authorship. This combined score is a warning indicator, not a probability of machine authorship.
## Results: What the Detectors Found in the Full Archive
When the combined score was applied to the remaining 594 reviews in the full archive (those not used for training or testing), it flagged 444 of them — roughly 75 percent — as possibly AI-generated. The same score also flagged 158 of the 200 held-out source reviews that were assumed to be human-written. These are detector warnings, not confirmations of machine authorship.
Breaking down the individual methods at a cutoff designed to catch at least 80 percent of the known generated reviews:
| Detection Method | Source Reviews Flagged (Training Set) | Source Reviews Flagged (Held-Out Test Set) |
|—|—|—|
| Perplexity | 193 of 200 (96.5%) | 571 of 594 (96.1%) |
| Near Duplicate Similarity | 143 of 200 (71.5%) | 386 of 594 (65.0%) |
| Embedding Density | 153 of 200 (76.5%) | 420 of 594 (70.7%) |
| Combined Score | 158 of 200 (79.0%) | 444 of 594 (74.7%) |
None of the methods performed well when judged by how many assumed-human reviews they incorrectly flagged. Even the best single method — embedding density — still flagged nearly half of the source reviews at the chosen recall level. Averaging the three methods into a combined score actually made things worse, not better.
## The Highest-Scoring Human Review and the Lowest-Scoring AI Review
One of the most revealing findings was that the highest-scoring source review in the dataset was not a piece of flowing personal commentary. It was a short, factual entry about the film *Amadeus*, listing awards, plot details, and historical rankings. Its high score came from text patterns — compact, formulaic structure — not from any evidence about who actually wrote it.
Conversely, one of the machine-generated reviews successfully slipped past all three detectors. It was a negative review of a fictional film called *The Last Stop on Neptune*, featuring sharp criticism of a space station murder mystery. Its combined score fell below the filtering cutoff, meaning none of the checks raised an alert — despite being entirely generated by a language model.
These two examples highlight the fundamental difficulty of authorship detection based on statistical patterns alone. Formulaic human writing can look machine-like, and creative machine-generated writing can look human.
## Did Filtering Help or Hurt the Sentiment Model?
The central practical question was whether removing flagged reviews would improve a sentiment classifier. A logistic regression model was trained in four different configurations: on all 600 reviews without any filtering, on the 134 reviews that remained after removing all rows at or above the combined-score cutoff, on all 600 reviews with flagged rows given reduced weight, and on the 200 source-only reference reviews.
Accuracy was measured on the 200 held-out test reviews from films the model had never seen during training. The results were striking: the untouched model achieved 67.5 percent accuracy. The filtered model — after removing flagged content — dropped to 57.5 percent. The downweighted model fell in between at 63.5 percent. The source-only reference reached 68.5 percent.
The drop in accuracy occurred because the filtering cutoff removed many of the assumed-human source reviews along with the generated ones. The training set shrank from 600 rows to just 134, and the classifier lost access to diverse examples. This does not mean that all the filtered reviews were human-written — some were genuinely machine-generated — but it demonstrates that a detector score by itself is insufficient justification for removing data from a training set.
## Key Takeaways
The practical implications of these findings are clear. The best detection method tested in this experiment caught 80 percent of known AI-generated reviews while simultaneously flagging 47 percent of the assumed-human source reviews. When that combined flagging score was used to filter the training data, downstream model accuracy fell from 67.5 percent to 57.5 percent. In this case, removing flagged content made the model worse, not better.
The core lesson is that detector scores should not be treated as proof of authorship, nor should they be used as the sole basis for deleting or filtering data from a training set. Instead, anyone working with text data should test detection methods against examples from their own specific domain and with verified authorship labels when available, then measure the downstream effect on the task the model actually needs to perform.
—
## Frequently Asked Questions
**What is perplexity and how does it relate to detecting AI-generated text?**
Perplexity measures how surprised a language model is by a given piece of text. Text that uses common patterns and predictable word sequences receives lower perplexity scores, which is why AI-generated text — often trained on vast corpora of similar material — can score low. However, human writers also produce predictable text, especially in formulaic or factual genres, so perplexity alone cannot reliably distinguish between the two.
**Why did the combined score perform worse than embedding density alone?**
Averaging the three detection signals into a single combined score diluted the strongest signal (embedding density) with weaker ones (perplexity and near duplicate similarity), each of which had its own blind spots. In this experiment, the combined cutoff flagged a larger proportion of assumed-human source reviews than embedding density alone, which reduced the training data further and hurt model performance.
**What does “model collapse” mean in the context of AI-generated text?**
Model collapse refers to the phenomenon where a language model trained on output produced by other models gradually loses access to rare or unusual patterns from the original human-written data. Over successive generations of training on machine-generated text, the model’s outputs become narrower, more repetitive, and less diverse — a degradation that compounds over time.
**Can these findings be generalized to other types of text data?**
The results are specific to the movie review collection, the two language models used for generation, the prompts employed, and the particular train-test split used in the experiment. Different datasets, domains, or detection tools may behave differently. The key insight — that detector flags alone are insufficient grounds for data removal — should be tested in each new context before drawing broad conclusions.
**What should a data practitioner do if they suspect AI-generated text in their dataset?**
First, set aside a portion of examples with verified authorship to measure how often your chosen detector produces false alarms. Second, do not remove or filter data based solely on a detector score. Third, compare the performance of models trained with and without filtering to see whether the intervention actually helps the downstream task. Fourth, if possible, use human review of flagged examples before taking any action.
**How was the experiment reproduced?**
The full reproduction code, scripts, and saved results are available in a public repository. The experiment can be run on any machine with Python 3.11 or newer by cloning the repository, installing dependencies, and executing the provided scripts. The Mendeley-derived review subset is included for convenience, and all audit reports flag possible AI authorship based on statistical patterns — they do not verify the identity of any individual author.
—
## Conclusion
The spread of AI-generated text into real-world datasets is a genuine and growing concern. The experiments described here demonstrate that even relatively simple detection methods — perplexity scoring, similarity checks, and embedding density — struggle to reliably distinguish machine-generated text from human writing in a movie review collection. More importantly, they show that using these detectors to filter training data can actively harm downstream model performance rather than improve it.
The most responsible approach is to treat AI detection scores as one piece of evidence among many, to validate any detection method against domain-specific examples with known authorship, and to measure the actual impact of any filtering decision on the task at hand. As AI-generated text becomes more common across all kinds of digital content, the need for rigorous, evidence-based approaches to data quality will only grow more urgent.
Thank you for reading
