Lesson 6 of 9 · about 19 minutes

Accuracy and False Positives: Who Gets Flagged and Why

You'll be able to

Define true positive, false positive and false negative rates and explain why the share of flagged texts that are really AI depends on how common AI use is
Work a base-rate calculation for a school or agency and explain why a small false positive rate matters across thousands of documents
Summarize the evidence that detectors misjudge non-native English writers, formulaic writers, short texts and mixed human and AI text
Tell vendor accuracy claims from independent evaluations and name institutions that disabled AI detection, with dates

Every AI detector makes mistakes in both directions. It misses machine text, and it flags human text. Vendors usually lead with the first number, how much AI writing they catch. The second number decides who gets called into the dean's office, and it is the one this lesson is mostly about.

Four possible outcomes

Run a detector on a pile of texts whose true origin you know, and every result falls into one of four boxes.

	Detector says AI	Detector says human
Really AI	True positive	False negative (a miss)
Really human	False positive (a false accusation)	True negative

From those boxes come the rates you will see quoted:

True positive rate: of the AI texts, how many got flagged.
False positive rate: of the human texts, how many got flagged anyway.
False negative rate: of the AI texts, how many slipped through.

These trade against each other. Most detectors produce a score, and someone picks a threshold above which the text counts as AI. Set the threshold low and you catch more AI text and accuse more people. Set it high and you accuse fewer people and miss more. A single "accuracy" figure hides that choice. So does an accuracy figure measured only on AI text, which tells you nothing about false accusations at all.

The research papers in this course often report results "at a 1 percent false positive rate." That is a fair way to compare detectors: fix how many innocent texts you are willing to flag, then see how much AI text each tool catches under that limit. When you read a claim, look for which rate it is, at what threshold, on what kind of text. (How detectors produce their scores is in How AI detectors work.)

Base rates: why 1 percent is not small

The question a teacher or editor actually faces is not "how often does this tool flag human text?" It is "this text was flagged; how likely is it that a person wrote it?" The answer depends on how common AI writing is in the group being tested. That is the base rate.

Example: one detector, two schools

Picture a detector that catches 90 percent of AI essays and wrongly flags 1 percent of human essays. Both numbers are better than most tools managed in the independent tests below. Each school in this example runs 10,000 essays through it. These are illustrative numbers, computed in Python.

School A: 10 percent of essays are AI-written.

1,000 AI essays. 90 percent flagged: 900 true flags. 100 missed.
9,000 human essays. 1 percent flagged: 90 false flags.
Total flagged: 990. Of those, 90 were written by students. About 1 in 11 flags, or 9.1 percent, lands on an innocent student.

School B: 2 percent of essays are AI-written.

200 AI essays. 180 true flags. 20 missed.
9,800 human essays. 98 false flags.
Total flagged: 278. Of those, 98 were human. That is 35.3 percent, about 1 in every 2.8 flags. Only 64.7 percent of flags are correct.

Same tool, same settings. In the school where cheating is rare, more than a third of the students it flags did nothing wrong. Raise the false positive rate to 4 percent, the figure Turnitin gives for individual sentences, and School B's flags become 180 true and 392 false: 68.5 percent of flagged essays would be human-written. Lower the false positive rate to 0.1 percent in School A and the false flags drop to 9 out of 909, about 1 percent.

The same student, many essays. A false positive rate applies to every submission. If a student who never uses AI turns in 10 essays a year and each has a 1 percent chance of a false flag, the chance of at least one flag is 1 minus 0.99 to the 10th power, or 9.6 percent. Over 40 essays in four years it is 33.1 percent. That assumes each essay's chance is independent. In practice a student whose style a detector dislikes may be flagged again and again, which is worse for that student.

Vanderbilt University did this arithmetic in public. When Turnitin switched on AI detection in April 2023, it claimed a 1 percent false positive rate. Vanderbilt noted that it had submitted 75,000 papers to Turnitin in 2022, so about 750 student papers could have been wrongly labeled as partly AI-written. That was one of the reasons it turned the feature off.

The calculation runs the other way too. If a vendor's real false positive rate is 0.1 percent and AI use is common, nearly every flag is right. That is why the base rate and the false positive rate have to be known together. A detector can be very good and still produce mostly false accusations in a group where almost nobody cheats.

What the published evaluations found

Independent tests are snapshots. Detectors and models both change every few months, so read each result with its date.

OpenAI's own classifier, 2023. OpenAI launched its AI Text Classifier on January 31, 2023. On its challenge set it correctly identified 26 percent of AI-written text and labeled human text as AI 9 percent of the time. OpenAI warned it was very unreliable below 1,000 characters and recommended it only for English. On July 20, 2023, OpenAI withdrew it "due to its low rate of accuracy."
Weber-Wulff and colleagues, 2023. A team led by Debora Weber-Wulff of HTW Berlin, with co-authors from universities in Latvia, Sweden, the Czech Republic, Mexico and the United Kingdom, tested 12 publicly available tools plus two commercial systems, Turnitin and PlagiarismCheck. Published in the International Journal for Educational Integrity, they concluded the tools were "neither accurate nor reliable," leaned toward calling AI text human, and got worse when text was machine-translated or obfuscated.
RAID benchmark, 2024. Liam Dugan, Chris Callison-Burch and colleagues built a test set of over 6 million generations from 11 models across 8 domains, with 11 kinds of adversarial attack and 4 decoding strategies, presented at ACL 2024. They noted that many detectors claim 99 percent accuracy or more, then found that the 12 detectors they tested were "easily fooled" by attacks, by different sampling settings, by repetition penalties and by models they had not seen.
Bloomberg Businessweek, 2024. Reporters ran 500 college application essays submitted to Texas A&M in summer 2022, before ChatGPT, through GPTZero and Copyleaks. The services falsely flagged 1 to 2 percent as likely AI-written, sometimes claiming near 100 percent certainty.
Jabarian and Imas, 2025. Brian Jabarian and Alex Imas of the University of Chicago tested Pangram, Originality.ai and GPTZero, plus the open-source RoBERTa detector, on a large set of passages across genres and lengths from roughly 1,000 words down to under 50. Their NBER working paper reports the commercial tools beat the open-source one, and that Pangram had near-zero false negative and false positive rates that held across models, short "stubs" and humanizer tools. Pangram was the only tool to meet a strict cap of a 0.5 percent false positive rate without giving up accuracy. Shorter passages were harder for the tools in general.

Detectors have improved since 2023, and the best commercial tools now score far better than OpenAI's withdrawn classifier did. But the strong 2025 result comes from one working paper with no replications yet, and it tested passages from known sources rather than the messy mix of edited, assisted and pasted text that shows up in real classrooms.

Who gets wrongly flagged

Non-native English writers. The best-known study is by Weixin Liang, Mert Yuksekgonul, Yining Mao, Eric Wu and James Zou, published in Patterns in July 2023. They ran seven widely used detectors on 91 essays written for the TOEFL English test by non-native speakers, taken from a Chinese forum, and 88 essays by US eighth graders. The detectors classified the eighth-grade essays nearly perfectly. On the TOEFL essays, every one of them written by a person, the average false positive rate was 61.3 percent. All seven detectors agreed on 19.8 percent of the essays, and at least one detector flagged 97.8 percent.

The cause was perplexity, the measure of how predictable text is. Writers working in a second language tend to use a narrower range of words and structures, which makes their text more predictable. When the researchers had ChatGPT "enhance the word choices to sound more like that of a native speaker," the average false positive rate on the TOEFL essays fell from 61.3 to 11.6 percent. When they simplified the eighth graders' words, those essays started getting flagged.

Formulaic and neurodivergent writers. The same mechanism catches anyone who writes in a regular, predictable way. Bloomberg Businessweek reported in 2024 on Moira Olmsted, a student at Central Methodist University who got a zero after an AI detector flagged a reading summary. She told her instructor she has autism spectrum disorder and writes in a formulaic way that can be mistaken for AI. The grade was changed, with a warning that a second flag would be treated like plagiarism. Montclair State University's guidance says studies are underway on whether detectors show a similar bias against neurodivergent writers. As of October 2026 this lesson found no peer-reviewed study measuring that bias directly, so treat it as a documented risk backed by case reports, with no measured rate yet.

Short texts. Less text means less evidence. OpenAI's classifier was "very unreliable" under 1,000 characters. Turnitin raised its minimum from 150 to 300 words in May 2023 to reduce false positives. Watermarks have the same problem, as Watermarks and provenance shows.

Mixed and edited text. Real documents are often part human, part machine, and edited by both. Turnitin said in June 2023 that its document-level false positive rate was under 1 percent only for documents where it detected 20 percent or more AI writing, and that its sentence-level false positive rate was about 4 percent. Those false sentence flags were more common in documents mixing human and AI text, and 54 percent of the time sat right next to actual AI writing. In May 2023, after testing 800,000 papers written before ChatGPT, Turnitin found more false positives when it detected under 20 percent AI, and began marking those scores with an asterisk as less reliable. A score of "15 percent AI" is exactly the zone the vendor itself says to distrust.

Evasion: paraphrasers and humanizers

The other failure is the miss, and it is easy to cause on purpose.

Kalpesh Krishna, Mohit Iyyer and colleagues built DIPPER, an 11-billion-parameter paraphrasing model, and published the results at NeurIPS 2023. Paraphrasing AI text with it defeated watermarking, GPTZero, DetectGPT and OpenAI's classifier. DetectGPT's detection rate fell from 70.3 percent to 4.6 percent, at a constant 1 percent false positive rate, without much change in meaning. Vinu Sankar Sadasivan, Soheil Feizi and colleagues showed that paraphrasing repeatedly, feeding the output back through the paraphraser, cut detection further across many detector types.

It does not take a research model. In the Liang study, simply prompting ChatGPT to rewrite its own college essays with more literary language, using the prompt "Elevate the provided text by employing literary language," sent detection rates on those essays to near zero.

Commercial "humanizer" tools sell this as a service. The Jabarian and Imas paper found Pangram held up against the humanizers it tested. Expect this to keep moving. Each side tunes against the other's latest version.

Krishna's team also proposed a defense that sidesteps style altogether: the model provider keeps a database of what its model generated and searches it for close matches. In their test with 15 million stored generations, it caught 80 to 97 percent of paraphrased text while flagging 1 percent of human text. Only the company that ran the model can do this, which is the same limit watermarks have.

Vendor claims versus independent tests

Read vendor numbers for what they measure.

Claim	Source	What it does and does not tell you
Document false positive rate under 1 percent, for documents with 20 percent or more AI writing	Turnitin, May and June 2023 (vendor claim)	Applies only above the 20 percent line; sentence rate about 4 percent; tested by Turnitin on its own sample
"99.9% Accuracy"	GPTZero home page, as of October 2026 (vendor claim)	One figure combining both error types, on the vendor's chosen test set
Accurate on 99.7 percent of AI-generated samples across 26 models	Pangram home page, as of October 2026 (vendor claim)	Measures catching AI text only; says nothing about false positives by itself
Near-zero false positive and false negative rates for Pangram	Jabarian and Imas, NBER working paper, 2025 (independent evaluation)	Independent and recent, on a large multi-genre set; one working paper, not a journal article
Detectors "neither accurate nor reliable"	Weber-Wulff et al., 2023 (independent evaluation)	Peer-reviewed but tested 2023 tools; many have changed since

When someone hands you an accuracy figure, ask:

Which rate is it: catching AI, avoiding false flags, or a blend?
On what texts: whose writing, what length, what language, which AI models, edited or not?
At what threshold, and is that the threshold your school or office actually uses?
Broken out by group: non-native writers, younger students, technical writing?
Who ran the test, and who paid for it?

Read the vendor's own cautions too. Turnitin wrote in May 2023 that it "cannot mitigate the risk of false positives completely" and that educators should use the AI score "to start a meaningful and impactful dialogue" with students. Watermark makers are just as careful about their checks: OpenAI's October 2026 notice says "the absence of a detected watermark does not prove human authorship."

Institutions that turned detectors off

Many schools still use AI detectors. A number of well-known ones stopped, and their notices explain why. Each of these was checked against the school's own statement or a news report.

Vanderbilt University disabled Turnitin's AI detector "for the foreseeable future" in a notice dated August 16, 2023. It cited the 750-paper calculation, the reported bias against non-native English writers, Turnitin's lack of detail on how the tool works, and privacy concerns about sending student work to a detector. It also noted the feature had been switched on for customers with less than 24 hours' notice and no option at first to disable it.
Michigan State University, Northwestern University and the University of Texas at Austin were among universities that decided not to use Turnitin's AI detector over fears of false accusations, The Register reported in September 2023, citing Bloomberg.
Montclair State University discontinued Turnitin's AI detection under a provost's memorandum dated November 14, 2023, citing reliability and bias, and says it has no plans to license another AI detection tool.
The University of Waterloo announced on May 5, 2025 that the AI detection function in Turnitin would no longer be available to its users as of September 2025.
Curtin University in Australia announced on September 4, 2025 that Turnitin's AI writing detection would be disabled across all campuses from January 1, 2026.

Turning off a detector is not the same as ignoring AI use. What schools, publishers and employers do instead, from draft histories to oral checks, is the subject of Detection in practice.

Common mix-ups

Claim: A 1 percent false positive rate means 1 percent of flagged students are innocent. What the evidence shows: It means 1 percent of innocent work gets flagged. The share of flags that are wrong depends on how common AI use is. In the example above it ranges from about 9 to 35 percent with the same 1 percent rate.
Claim: Detectors are equally fair to everyone. What the evidence shows: Seven detectors in 2023 wrongly flagged 61.3 percent of human-written TOEFL essays on average while nearly acing native eighth-grade essays.
Claim: AI detectors do not work at all. What the evidence shows: Early tools performed poorly, including OpenAI's own. A 2025 independent test found the best commercial tool had very low error rates on its test set. Performance varies by tool, length, language and editing.
Claim: A low AI score clears a text. What the evidence shows: Paraphrasing and humanizer tools can push AI text past many detectors, and short texts give little to measure.
Claim: If a vendor says 99 percent accurate, a flag is 99 percent likely to be right. What the evidence shows: That depends on which rate the 99 percent is, on what data, and on the base rate where you use it.

Reporting angle

Ask your school district, college or agency whether it runs an AI detector, which product and version, what threshold triggers action, and whether a score alone has ever been used for discipline.
Request the counts: how many submissions were checked, how many were flagged, how many flags led to findings of misconduct, and how many were overturned on appeal. Those numbers let you estimate the real false positive burden.
Ask whether flags are tracked by student group, such as English learners or students with accommodations.
Get the vendor's validation report and compare it to independent tests. Note the date of each.
Interview students who were cleared after a flag. Their records, drafts and appeal letters are the evidence.

Research angle

Liang et al., GPT detectors are biased against non-native English writers, Patterns 2023.
Krishna et al., Paraphrasing evades detectors, with the DIPPER model and the retrieval defense.
The RAID benchmark and its public leaderboard.
Jabarian and Imas, Artificial Writing and Automated Detection, NBER 2025.
Weber-Wulff et al., Testing of detection tools for AI-generated text.
Vanderbilt's notice disabling Turnitin's AI detector, with its 750-paper calculation.
For how to read benchmark and vendor claims in general, see Checking an AI Claim; for bias in AI systems more broadly, Bias and Errors.

Check yourself

1. A detector catches 90 percent of AI essays and wrongly flags 1 percent of human essays. In a set of 10,000 essays where 2 percent are AI-written, about what share of flagged essays were written by people?

2. In Liang and colleagues' 2023 Patterns study, what was the average false positive rate of seven detectors on 91 TOEFL essays written by people?

3. What did Krishna and colleagues' DIPPER paraphraser do to DetectGPT, holding the false positive rate at 1 percent?

4. Why did Vanderbilt University disable Turnitin's AI detector in August 2023?

5. Which statement about the OpenAI AI Text Classifier is accurate?

Terms in this lesson

True positive rate: The share of AI-written texts that a detector correctly flags. Also called sensitivity or recall. One minus this is the false negative rate.
False positive rate: The share of human-written texts that a detector wrongly flags as AI. The number that decides how many innocent people get accused.
False negative rate: The share of AI-written texts that a detector misses and labels human.
Base rate: How common the thing you are looking for is in the group you are testing, such as the share of essays in a class actually written by AI. It changes what a flag means.
Positive predictive value: Of all the texts a detector flags, the share that really are AI-written. Depends on the true positive rate, the false positive rate and the base rate together.
Decision threshold: The score above which a detector or its user calls a text AI. Raising it lowers false positives and raises misses; lowering it does the reverse.
Humanizer: A tool or service that rewrites AI-generated text to evade detectors, usually by paraphrasing, varying sentence length and swapping vocabulary.
Paraphrase attack: Rewording AI-generated text, often with a second model, so a detector or watermark no longer recognizes it while the meaning stays about the same.
Retrieval defense: A detection approach in which the model provider stores what its model generated and searches that store for close matches to a suspect text.

Full glossary