What AI Detection Is and Why People Care

In the fall of 2019, a Harvard senior named Max Weiss wrote a small program, pointed it at the federal Medicaid website, and filed 1,001 comments on Idaho's proposed Medicaid waiver. None were written by a person. They made up 55.3 percent of all 1,810 comments the agency received. When Weiss later showed a mix of his bot comments and real ones to survey takers, they sorted them correctly 49.6 percent of the time, about what a coin flip gets you. He told the Centers for Medicare and Medicaid Services after the comment period closed, and the bot comments were withdrawn. The model he used was GPT-2, a small 2019 system that looks primitive now.

Four years later, a college instructor in Texas pasted his students' final papers into ChatGPT, asked it whether it had written them, believed it when it said yes, and told the class their grades were on hold. Several of those students had written every word themselves.

Those two stories are this course in miniature. Machine text can pass for human, and people get hurt when someone wrongly decides their own writing came from a machine. AI detection is the attempt to tell the two apart, and both kinds of failure are on the record.

What "detecting AI text" actually means

AI text detection is any method for estimating whether a piece of writing was produced, wholly or in part, by a large language model such as ChatGPT, Claude, Gemini or Llama. The word "estimating" matters. No method in use today reads a text and returns a fact. They return a judgment, usually with a number attached.

The methods fall into a few families, and later lessons take each one apart:

• Software detectors that score a text, such as Turnitin's AI indicator, GPTZero, Copyleaks, Originality.ai and Pangram. Lesson 3, How detectors work, explains what they measure.
• Tells, the word choices, structures and factual slips that readers notice. Lesson 5, The tells, covers them as evidence.
• Watermarks and provenance, signals planted at the moment of generation or records attached to a file. Lesson 4, Watermarks and provenance.
• Process evidence: drafts, notes, version history, and the writer explaining their own work out loud. Lesson 9, Judging a claim, shows why this is usually the evidence that settles things.

AI detection is a different job from plagiarism checking. A plagiarism checker compares your text against a giant library of existing documents and reports matches. A language model does not copy a document. It writes new sentences one token at a time. So plagiarism software is close to blind to it. In a 2023 study in npj Digital Medicine, a team led by Catherine Gao of Northwestern University had ChatGPT write 50 medical research abstracts in the style of JAMA, NEJM, BMJ, The Lancet and Nature Medicine. A plagiarism checker gave the fake abstracts a median score of 0 percent plagiarized. The real abstracts, which were already published, scored a median of 62.5 percent. By the plagiarism measure, the fakes looked more original than the real thing.

Why it suddenly mattered

Text generators existed for years before most people noticed. What changed was access. OpenAI released ChatGPT to the public on November 30, 2022, free, in a chat box anyone could use. Detection tools followed within weeks. OpenAI, the company that made ChatGPT, released its own classifier for spotting AI writing on January 31, 2023, and pulled it less than six months later.

The use numbers kept climbing. A few figures that frame the stakes:

Measure	Figure	Who reported it
U.S. teens 13 to 17 who say they use ChatGPT for schoolwork	26 percent in fall 2024, up from 13 percent in 2023	Pew Research Center, survey of 1,391 teens, September 18 to October 10, 2024
Teens who say it is acceptable to use ChatGPT to write essays	18 percent; 42 percent say it is not	Same Pew survey
K-12 teachers who say they regularly use an AI detection tool	68 percent, school year 2023-24	Center for Democracy and Technology, March 2024
Teachers who say students at their school faced consequences for using, or being accused of using, AI	64 percent, up 16 points from the year before	Same CDT survey
Submissions Turnitin's AI detector scanned in its first six weeks, spring 2023	38.5 million; 9.6 percent flagged as over 20 percent AI writing, 3.5 percent as 80 to 100 percent	Turnitin chief product officer, May 23, 2023 (vendor figures; includes assignments where AI was allowed)

Read that Turnitin line carefully. Those are flags from the company's own tool, so they measure what the tool flagged, which is not the same as what students did. The company said its counts include assignments where teachers allowed AI, and it does not separate those out.

Who needs to know, and why

Authorship matters wherever a piece of writing stands in for a person's effort, knowledge, opinion or identity. The settings differ a lot in what they are protecting.

Setting	What machine text threatens	A documented case or rule
Schools and universities	A grade is supposed to certify that the student can do the work	The Pew, CDT and Turnitin figures above; the false accusation cases below
Scientific publishing	Reviewers trust that the data and methods are real	Gao et al. 2023: ChatGPT wrote believable abstracts with fabricated data
Journalism and magazines	Readers trust that a named reporter exists and checked the facts	Sports Illustrated product reviews under invented bylines, 2023; a syndicated summer reading list full of books that do not exist, 2025
Hiring	A cover letter is supposed to show how the applicant writes and thinks	Lesson 7 covers how employers handle applications they suspect were machine-written
Search engines	Mass-produced pages crowd out useful ones	Google's March 2024 spam policy on "scaled content abuse," which applies "no matter whether content is produced through automation, human efforts, or some combination"
Reviews and scams	Fake praise steers buyers; fluent text makes scams look legitimate	The FTC's 2024 rule (16 CFR Part 465, effective October 21, 2024) bans selling or buying reviews by someone who does not exist, naming AI-generated reviews as an example
Elections and foreign influence	Fake voices multiply a message cheaply and in many languages	OpenAI's May 2024 report on five covert operations from Russia, China, Iran and Israel that used its models to write posts and articles
Government comment dockets and legislators' inboxes	Volume of comments can be faked; officials may read bulk text as public opinion	The 2019 Medicaid comment test; the Cornell study of 32,000 letters to state legislators
Copyright	Protection covers only human authorship	Copyright Office guidance, March 16, 2023: applicants "have a duty to disclose the inclusion of AI-generated content"
Personal trust	A message of sympathy or apology means something because a person wrote it	Vanderbilt's Peabody College diversity office, February 2023 (below)

That last row is easy to skip and it shouldn't be. On February 16, 2023, after a mass shooting at Michigan State University, the equity office at Vanderbilt's Peabody College emailed students a message about community and care. The last line read "(Paraphrase from OpenAI's ChatGPT AI language model, personal communication, February 15, 2023)." The associate dean apologized for "poor judgment," and two administrators stepped back from their duties while the office reviewed what happened. Nobody was cheated out of a grade. People were hurt because they found out a message about grief had been produced by software, and nobody had told them.

The law side of this table (the FTC rule, copyright, state election laws and the European Union's labeling duties) is covered in lesson 8, Law and policy.

When machine text got through

The record of AI text fooling people is long, and it starts before ChatGPT.

Legislators. Sarah Kreps and Douglas Kriner of Cornell sent more than 32,000 advocacy emails to about 7,000 U.S. state legislators during 2020, some written by people and some by GPT-3, on issues from gun control to taxes. Legislators answered 17.3 percent of the human letters and 15.4 percent of the machine ones. That gap is small. On education policy, the machine letters drew more replies. The results appeared in New Media and Society in March 2023.

Scientists. In the Gao study, blinded human reviewers caught 68 percent of the ChatGPT abstracts. So about a third got past them. The same reviewers wrongly called 14 percent of the real abstracts fake. Both errors at once, from experts who knew some of what they were reading was fake.

Magazines. In November 2023, Futurism reported that Sports Illustrated had run product reviews under author names and headshots that appeared to be invented, with the text produced by AI. The Arena Group, SI's publisher at the time, said the content came from a third-party company, AdVon Commerce, which assured it the articles were written by humans. Arena ended the partnership and pulled the articles.

Newspapers. In May 2025, a "Heat Index" summer supplement ran in the Chicago Sun-Times and the Philadelphia Inquirer with a reading list that included "The Last Algorithm" by Andy Weir and "Nightshade Market" by Min Jin Lee. Neither book exists. The freelance writer, Marco Buscaglia, said he had used AI for research and failed to check it; more than half the books on his list were fake. The syndicator, King Features, said it fired him. Min Jin Lee posted: "I have not written and will not be writing a novel called 'Nightshade Market.'" Note what caught it. Readers who knew the authors' real books, not software.

Influence operations. In late May 2024, OpenAI said it had disrupted five covert operations using its models: two from Russia (one it named Bad Grammar, plus the Doppelganger network), China's Spamouflage, Iran's International Union of Virtual Media, and an operation from Israel called Zero Zeno. They generated posts and articles in several languages, in some cases with fewer language errors than human operators would have made. OpenAI also said they did not appear to gain much audience from it. That is the company grading its own platform.

When people were accused of using AI

The other failure is quieter in headlines and louder in people's lives.

Texas A&M University-Commerce, May 2023. Jared Mumm, an instructor in agricultural sciences, pasted his students' final assignments into ChatGPT and asked whether it had written them. It claimed every one. ChatGPT has no ability to recognize its own past output; it will answer the question either way. He told students they would get an "X" grade. The university said no student failed or was blocked from graduating, that several students were exonerated and got their grades, that one admitted to using ChatGPT, and that others chose to complete a new assignment. The sequence matters. The students had to clear themselves of something a chatbot made up.

Central Methodist University, fall 2023. Moira Olmsted, studying online to become a teacher, got a zero on a reading summary after a detector, which she says was Turnitin, judged it likely AI-written. She has autism spectrum disorder and told the school she writes in a formulaic way. The grade was changed, but she was warned that a future flag would be treated like plagiarism. Bloomberg Businessweek reported her case in October 2024.

Australian Catholic University, 2024. ABC News reported in October 2025, based on internal documents, that ACU recorded nearly 6,000 alleged academic misconduct cases across its nine campuses in 2024, and about 90 percent involved AI. Integrity officers asked students for handwritten notes and internet search histories to prove they had not used AI. A nursing student named Madeleine waited six months to be cleared, with "results withheld" on her transcript, and she believes it cost her a graduate job. A paramedicine student's case rested on a report flagging 84 percent of his essay. ACU's deputy vice-chancellor said the figures were "substantially overstated," that cases where the detector was the only evidence were dismissed, and that "We regret the impact this had on some students." ACU stopped using Turnitin's AI detector in March 2025. Turnitin's own guidance says the tool should not be the sole basis for action against a student.

Who gets flagged. These are not random accidents. In a 2023 Stanford study published in Patterns, Weixin Liang and colleagues ran 91 TOEFL essays written by non-native English speakers, all from 2020 or earlier, through seven detectors. The average false positive rate was 61.22 percent. All seven detectors agreed that 18 of the 91 human essays were AI-written, and 89 of 91 were flagged by at least one detector. Plain vocabulary and predictable phrasing, which are normal in a second language, look machine-like to tools that reward surprise. Lesson 6, Accuracy and false positives, goes through that research and the vendor responses to it.

Small rates, big numbers. Bloomberg ran 500 Texas A&M application essays from summer 2022, before ChatGPT existed, through GPTZero and Copyleaks. The detectors falsely flagged 1 to 2 percent of them as likely AI-written, sometimes with near 100 percent claimed certainty. One or two percent sounds small until you count the essays a large university grades in a semester.

Two errors, two sets of victims

Every detector can fail in two directions, and the costs land on different people.

	What happened	Who pays	How it usually comes to light
False negative	AI text is accepted as human	Honest students graded on the same curve; readers given invented facts; agencies weighing fake comments; the publication's reputation	Later, if ever: a fake book title, a fabricated statistic, a confession
False positive	A person's own writing is labeled AI	That one person: a grade, a transcript hold, a job offer, a reputation	Immediately, as an accusation the person must answer

The asymmetry is the point. A false negative spreads its cost thinly across many people, and often nobody ever finds out. A false positive concentrates the whole cost on one individual, who has to prove a negative. How do you prove you did not use a tool that leaves no trace on your computer? The ACU students were asked for search histories. That is the burden of proof turned upside down.

Vendors usually quote one number, accuracy, which blends both errors together. Turnitin, for example, has said its document-level false positive rate is under 1 percent for documents above its 20 percent threshold, and about 4 percent at the sentence level. OpenAI's own withdrawn classifier, by its own account, caught 26 percent of AI text and wrongly flagged 9 percent of human text. Neither number alone tells you how often an accused student is actually innocent. That depends on how many students in the pool used AI in the first place, which lesson 6 works through with real arithmetic.

What a detector result can and cannot tell you

A detector score is a statement about statistical resemblance. It says this text looks, by the tool's measures, more like the machine text it was trained on than the human text it was trained on. It does not say who typed it, whether the person used AI for an outline or a grammar pass, or whether a rule was broken. OpenAI's 2023 announcement said its classifier "should not be used as a primary decision-making tool." Turnitin's guidance says the same about its own indicator.

Some practical points follow from that.

• Asking a chatbot is not detection. ChatGPT and similar tools cannot look up whether they wrote a text. The Texas case shows what happens when someone assumes they can.
• Length matters. OpenAI said its classifier was "very unreliable" below 1,000 characters. Turnitin raised its minimum from 150 to 300 words in 2023. A tweet, a short comment or a two-line review gives any method little to work with.
• Mixed writing is the hard case. A draft polished by a grammar tool or one paragraph rewritten by a chatbot sits between the pure human and pure machine text detectors are trained on.
• People who use AI a lot are good at spotting it. In a 2025 study, Jenna Russell, Marzena Karpinska and Mohit Iyyer of the University of Maryland found that annotators who regularly use language models for writing were strong detectors. A majority vote of five such readers misclassified just 1 of 300 articles, better than most software tested. The Medicaid survey takers, who had no such practice, did no better than chance. Experience with the tools is a skill.
• Process evidence beats a score. Drafts, version history, notes and a conversation with the writer about their choices are what actually settle authorship. Lesson 7, Detection in practice, shows how schools and publishers use them, and lesson 9 shows how writers can keep them.

Detection is still worth doing. The Medicaid docket, the fake reading list and the invented bylines all show real harm from missed machine text. The goal of this course is to make you a careful user of the evidence: someone who knows what each signal measures, how often it fails, and who bears the cost when it does. Lesson 2, How machines write, starts with why machine text leaves any signal at all.