How AI Text Detectors Work

On January 31, 2023, OpenAI, the company behind ChatGPT, released a free tool to tell whether text was written by AI. By OpenAI's own numbers it caught 26 percent of AI-written text and wrongly flagged 9 percent of human writing. On July 20, 2023, the company took it down, adding a one-line note to the announcement: "the AI classifier is no longer available due to its low rate of accuracy." The company that built the model could not reliably spot the model's writing.

Detection has moved a long way since. Some methods now score very well in independent tests, and some still fail badly. The difference comes down to what each method measures, so that is where to start.

Every detector answers the same narrow question

Strip away the branding and every AI text detector does one thing. It takes a text and produces a number that estimates how much the text resembles machine writing. Then someone, the vendor or the user, picks a threshold. Above it, the text gets labeled AI. Below it, human.

That threshold is a policy choice, not a fact about the text. Set it high and you wrongly accuse fewer people but miss more AI text. Set it low and you catch more AI text and accuse more innocent writers. Every accuracy number a vendor publishes is tied to the threshold it chose.

The methods differ in where the number comes from. There are five main families:

Family	What it measures	What it needs	Examples	Known weak spots
Statistical scoring	How predictable the text is to a language model, and how much that varies	One scoring model	GLTR (2019), GPTZero's original method (2023)	Formulaic and second-language writing; edited text
Trained classifiers	Whatever patterns separate labeled human text from labeled AI text in the training set	Large labeled datasets, retraining as models change	Turnitin, Copyleaks, Originality.ai, Pangram, GPTZero today	New models, new genres, short text; opaque reasoning
Zero-shot probability methods	How a text's probability behaves under rewrites or under a second model	Access to one or two open models' probabilities	DetectGPT, Fast-DetectGPT, Binoculars	Paraphrasing; choice of scoring model; compute cost
Stylometry	Style features compared with a known writer or with machine norms	Samples of the person's own writing, for authorship comparison	Authorship verification research	Needs enough known writing; people's style changes
Retrieval	Whether the text matches something the AI system actually generated	A provider-kept log of outputs	Proposed by Krishna et al. 2023	Only the provider can run it; useless for open models

Watermarks, signals deliberately planted at generation time, are a sixth approach and get their own lesson, Watermarks and provenance.

Statistical scoring: perplexity and burstiness

The first widely used detectors measured the two quantities explained in lesson 2, How machines write. Perplexity: how surprised a model is by each word. Burstiness: how much that surprise swings from sentence to sentence.

The idea predates ChatGPT. GLTR, built by Sebastian Gehrmann, Hendrik Strobelt and Alexander Rush and presented in 2019, showed readers where each word ranked in GPT-2's list of predictions. Machine text was mostly top-ranked words. It was a visual aid, and in their study it raised people's detection rate from 54 to 72 percent.

GPTZero, launched in January 2023, made the approach famous. Its founder Edward Tian described a perplexity layer and a burstiness layer and offered a rough rule: "a perplexity above 85 is more likely than not from a human source." GPTZero's explainer now carries a note that as of autumn 2023 it "no longer uses perplexity and burstiness" for detection because it "migrated to a deep-learning based architecture."

The weakness is built in. Predictability is a property of writing style, and plenty of human writing is predictable on purpose. Stanford's 2023 study of TOEFL essays, covered in lessons 1 and 6, showed what happens when the tools meet writers with smaller vocabularies.

Trained classifiers: the commercial tools

A trained classifier learns from examples. You collect a large set of text known to be human, a large set known to be machine-generated, and train a neural network to tell them apart. It does not need a theory of why AI text differs. It finds whatever patterns separate the two piles.

That is both the strength and the problem. A classifier can learn subtle signals no person would name. It can also learn the wrong ones, like the topic of the essays in its training set, and it can only be as current as its examples. When a new model ships with a different style, the classifier has to be retrained.

Here is how the major tools describe themselves, as of October 2026. Everything in the third column is the vendor's own claim.

Tool	Method as the vendor describes it	Vendor's own claims	Status notes
OpenAI AI Text Classifier	Fine-tuned language model trained on human and AI text	26 percent of AI text caught; 9 percent false positive rate; "very unreliable" under 1,000 characters	Launched January 31, 2023; withdrawn July 20, 2023
Turnitin AI writing indicator	Classifier inside its plagiarism product; reports a share of the document's text it predicts is AI-written, with sentence highlights	Document false positive rate under 1 percent for documents above 20 percent AI; sentence-level false positive rate about 4 percent; English learners' false positive rate 0.014 versus 0.013 for native writers	Launched April 2023; minimum raised from 150 to 300 words and asterisk added for scores under 20 percent in May 2023; its page now says it also flags text run through "AI bypassers (also called humanizers)"
GPTZero	Originally perplexity and burstiness; now a deep-learning, multi-task model described in a February 2026 technical report	"99.9% Accuracy" on its home page	Released a model it calls GPTZero 4o on September 24, 2026
Copyleaks	"Linguistic modeling, deep learning, and our proprietary AI Logic"	"99% accuracy"; "an industry-low .03% false positive rate"; more than 30 languages	Its own per-language figures show lower AI detection in some languages, such as 93.08 percent for Portuguese
Originality.ai	Trained classifier; says its first version shipped in November 2022, before ChatGPT	"99%+ Accuracy"	Launched an "AI Allowance" setting in June 2026 and retired its older models in September 2026
Pangram	Transformer classifier trained with a method its 2024 technical report calls "hard negative mining with synthetic mirrors"	99.7 percent accuracy on AI samples from 26 models it benchmarks; says it catches "humanized" text	Lists results for models tested in July 2026

A claim of 99 percent accuracy or higher appears on four of those pages. Treat it as a headline, not a measurement you can use. Accuracy blends the two errors together, depends on the mix of human and AI text in the test, and depends on the threshold. A tool can be "99 percent accurate" on a test set that is 90 percent AI text while still flagging a meaningful share of the human essays. The questions to ask are in lesson 6, Accuracy and false positives.

What independent tests found

Independent tests have come out on both sides, and the dates matter, because tools changed fast.

• Weber-Wulff and colleagues, 2023. Eight researchers led by Debora Weber-Wulff tested 14 tools, 12 public ones plus Turnitin and PlagiarismCheck, and published in the International Journal for Educational Integrity. Their conclusion: the tools were "neither accurate nor reliable" and leaned toward calling AI text human. Obfuscation made them much worse.
• RAID, 2024. Liam Dugan and colleagues built a benchmark with more than 6 million generations from 11 models across 8 domains, with 11 kinds of adversarial attack and 4 decoding strategies. They found current detectors "easily fooled" by attacks, changes in sampling settings, repetition penalties and models they had not seen. RAID keeps a public leaderboard.
• Jabarian and Imas, 2025. Brian Jabarian and Alex Imas, in a working paper featured by the University of Chicago Booth School of Business, tested Pangram, Originality.ai, GPTZero and an open-source RoBERTa model on about 2,000 human passages in six genres, from restaurant reviews to novels, plus AI versions from four models. Pangram had false positives near zero and false negatives of 2 to 4 percent. GPTZero had false positives under 1 percent and false negatives of roughly 0 to 2 percent. Originality.ai had false positives under 1 percent but missed 10 to 40 percent of AI text, depending on the model. The open-source model did far worse, close to guessing in many cases. All the commercial tools lost accuracy on passages under 50 words. The paper is NBER Working Paper 34223.

So the best commercial classifiers in 2025 did much better than the tools of 2023. That does not settle any single case. These tests use clean samples, either fully human or fully AI. Real disputes are usually about mixed or edited text, a short assignment, or a writer whose style the tool has rarely seen.

Zero-shot methods: reading the model's own math

Researchers have also built detectors that need no training examples at all. They are called zero-shot methods. They use the probabilities an open language model computes, in clever ways.

DetectGPT (Eric Mitchell and colleagues, ICML 2023) starts from an observation. Text a model generated tends to sit at a peak of that model's probability, because the model picked high-probability words. So DetectGPT makes many small random rewrites of the passage using a second model (T5), then checks the original model's probability for each version. If nearly every rewrite scores lower than the original, the original was probably sitting on a peak: likely machine text. Human text sits in more irregular terrain, and rewrites go up about as often as down. On fake news articles from a 20 billion parameter model, DetectGPT raised the AUROC from 0.81 for the best earlier zero-shot method to 0.95.

Fast-DetectGPT (Guangsheng Bao and colleagues, ICLR 2024) replaced the slow rewriting step with a sampling step. The authors report about 75 percent relative improvement over DetectGPT and a speedup of 340 times.

Binoculars (Abhimanyu Hans and colleagues, 2024) uses two related models, which the authors call an "observer" and a "performer," both open Falcon models. It divides the text's perplexity under one model by a "cross-perplexity" measuring how surprising one model's predictions are to the other. The ratio is meant to correct for odd prompts, which can make even machine text look surprising to a single model. The authors report it catches over 90 percent of ChatGPT samples at a false positive rate of 0.01 percent, without training on any ChatGPT data.

Zero-shot methods matter for public trust because they are published, open and repeatable. Anyone with a computer and an open model can check the method. They also have the weakness of all probability methods: paraphrase the text and the probabilities change. Krishna and colleagues showed a paraphrasing model dropped DetectGPT's detection rate from 70.3 percent to 4.6 percent at a 1 percent false positive rate.

Stylometry: comparing a text with its supposed author

Stylometry is older than computers. It measures style: how often a writer uses particular function words like "upon" or "whilst," average sentence length, punctuation habits, favorite constructions.

Applied to AI, stylometry asks two kinds of question.

• Does this text look like machine style in general? The Carnegie Mellon findings in lesson 2, such as models using present participle clauses at 2 to 5 times the human rate, are stylometric. So is the work of Tharindu Kumarage and colleagues (2023), who used style signals to spot when an AI began writing posts inside a person's social media timeline. Short posts are hard for other detectors, and the style shift across a timeline gave them something to measure.
• Does this text look like this particular person? This is authorship verification. If a student has turned in ten essays in one voice and the eleventh reads differently, that is a stylometric signal. A 2024 survey by Baixiang Huang, Canyu Chen and Kai Shu sorts the field into four problems: attributing human text, detecting machine text, attributing machine text to a specific model, and untangling human-machine co-authored text. They name generalization across domains and explaining the decision as the open challenges.

Comparison with a person's own past work is the method teachers and editors already use by instinct. It has a fairness advantage, because it compares the writer with themselves instead of with a population. It also has limits. People's writing changes with practice, a tutor, a new subject or a hard week. And a writer whose earlier work was itself AI-assisted gives you a false baseline.

Retrieval: asking the source

The most direct question is whether the AI system produced this text. Only the company that ran the model can answer it, by keeping a record of what it generated and searching it.

Kalpesh Krishna and colleagues proposed this in a paper presented at NeurIPS 2023. They had built DIPPER, an 11 billion parameter paraphrasing model, and showed it could slip past watermarks, GPTZero, DetectGPT and OpenAI's classifier. Then they tested a defense. Store every generation, and when a suspect text arrives, search the store for passages with similar meaning. With a database of 15 million generations, retrieval caught 80 to 97 percent of paraphrased generations while flagging only 1 percent of human text.

The catch is in the design. The provider has to keep and search its users' outputs, which raises its own privacy questions. Each provider can vouch only for its own model. And open-weight models that people download and run on their own computers keep no central log at all. The commercial vendors in the table above rely on classifiers instead.

What a score means, and what it does not

Different tools report different kinds of numbers, and people misread all of them.

• A share of the text. Turnitin's indicator estimates how much of the document's text looks AI-written, with sentences highlighted. "34 percent" means the tool thinks about a third of the text matches machine patterns. It does not mean the tool is 34 percent sure, and it does not say which tool, if any, the writer used. Turnitin says scores under 20 percent are less reliable, which is why they get an asterisk, and that falsely flagged sentences usually sit next to AI-written ones. Its own figures put 54 percent of falsely flagged sentences directly beside AI text.
• A probability or confidence. Other tools report how likely the whole text is to be AI. A high-confidence wrong answer is still wrong, and lesson 1 describes false flags issued with near-total claimed certainty.
• A label. "Likely AI," "Mixed," "Human." Behind every label is a threshold someone chose.

What no score can tell you: who typed the words; whether AI was used for an outline, a translation or a grammar check rather than the writing; whether the use broke a rule; or anything about a text shorter than the tool's minimum. OpenAI warned that its classifier could mislabel human writing with confidence, and Turnitin's guidance says its indicator should not be the sole basis for action against a student.

A score is a reason to look harder, using the evidence in lesson 9, Judging a claim. It is not a finding.