The Tells: Word Choice, Habits and Invented Facts

The idea that you can spot machine writing by its words is not folklore. It has been measured, across millions of documents, by researchers who count words for a living. The trouble starts when people take a pattern that is real across a mountain of text and apply it to one essay, one public comment or one cover letter. This lesson covers what the tells are, how strong the evidence behind them is, and which kind of tell actually settles something.

If you want the practical word and phrase lists for cleaning up your own drafts, they are in Recognizing AI Tells in Writing. Here the question is different: what does a tell prove about who wrote a text?

The vocabulary shift, measured

The biggest study so far comes from Dmitry Kobak, Rita Gonzalez-Marquez, Emoke-Agnes Horvat and Jan Lause, first posted in June 2024 and published in Science Advances in July 2025. They took more than 15 million biomedical abstracts indexed by PubMed from 2010 to 2024 and asked which words suddenly appeared far more often than the earlier trend predicted. They called these "excess words."

Excess words are not new. In 2021 there were up to 190 of them, mostly content words tied to the pandemic, such as "coronavirus," "masks" and "remdesivir." In 2024 the count jumped to 454, and nearly all of them were style words, the kind that carry no subject matter. Of the 379 excess style words in 2024, 66 percent were verbs and 14 percent adjectives. The verb form "delves" showed up 28 times as often as expected. "Underscores" ran 13.8 times expected and "showcasing" 10.7 times. Among common words, "potential," "findings" and "crucial" showed strong excess use. From the size of the jump, the authors estimated that at least 13.5 percent of 2024 abstracts were processed with a language model, reaching above 40 percent in some fields, countries and journals. That is a lower bound, because abstracts that used none of the marker words could not be counted.

Other teams found the same thing in other places:

• Andrew Gray, in a March 2024 preprint, tracked keywords across the 2023 scholarly literature and estimated that at least 60,000 papers, slightly over 1 percent of all articles, were LLM-assisted. Kobak's paper notes Gray found "intricate" and "meticulously" roughly doubled in 2023.
• Weixin Liang and colleagues, at ICML 2024, studied peer reviews for four AI conferences held after ChatGPT's release. They estimated that between 6.5 and 16.9 percent of review text could have been substantially modified by a language model, beyond spell-checking. In ICLR 2024 reviews, "commendable" became 9.8 times more likely to appear in a sentence, "meticulous" 34.7 times and "intricate" 11.2 times. The estimated share was higher in reviews submitted within three days of the deadline and in reviews whose authors reported lower confidence. Nature portfolio journal reviews showed no significant sign of the shift.
• A second Liang study covered 950,965 papers posted to arXiv, bioRxiv and Nature journals from January 2020 to February 2024 and found estimated LLM modification reaching 17.5 percent in computer science and up to 6.3 percent in mathematics and Nature journals.

Why these words? Tom Juzek and Zina Ward, at COLING 2025, identified 21 focal words whose rise in abstracts is likely due to LLM use, including "delve," "intricate" and "underscore." They found no evidence that the model architecture or training data explained the overuse. Their model comparisons were consistent with the human feedback stage of training playing a role. (That stage, and the house style it produces, is covered in How machines write.)

Why a real pattern is weak evidence about one text

Liang's team built their method to estimate the fraction of a corpus that was AI-modified, and they say outright that estimating at the corpus level is more reliable than judging individual documents. Kobak's numbers are shares of millions of abstracts. None of these papers claims to identify which abstracts or reviews were machine-written.

The reason is simple arithmetic. A word that is 28 times more common than expected is still a word people used before 2022. Every word on these lists has human uses, and scientists who never touched a chatbot still wrote "crucial" in 2024. If you flag every abstract containing one marker word, you catch some machine text and a great many people.

And the lists move. Wikipedia's editors, who keep the most detailed public guide to signs of AI writing, note that the overused words have changed over time. The guide says "delve" was famously overused by ChatGPT in 2023 and early 2024, became less frequent later in 2024 and dropped off sharply in 2025. A detector or a reader working from a 2024 list is looking for the habits of 2024 models.

People picked up the words too

The vocabulary did not stay in the machines. Hiromu Yakura, Iyad Rahwan and colleagues analyzed 737,083 hours of unscripted conversation from 824,634 podcast episodes. Words that ChatGPT favors, including "delve," "showcase," "boast," "intricacies" and "meticulous," rose abruptly in spontaneous human speech, and a synthetic-control analysis tied the change to ChatGPT's release. In a preregistered experiment with 496 people, a short chat with a bot led participants to start using its words themselves, and the change persisted after a distraction task.

So a tell that worked in 2023 can be a contaminated signal by 2026. A person who reads a lot of chatbot output, or works next to people who use it, may write that way without ever generating a sentence. Students who learn from AI study aids may absorb the style. And people who learned to write from the same textbooks, business memos and grant applications that the models learned from always shared some of these habits. The models got their favorite constructions from somewhere.

Structural, formatting and punctuation habits

Beyond single words, there are habits of structure. Wikipedia's guide groups many of them, and course 16 lists them in detail. The ones most often cited as signs of machine text:

• Negative parallelism. Setting up a framing only to correct it: "not just X, but Y" and its cousins. Anthropic, explaining in August 2026 how commercial detectors work, said AI models "appear to be fond of the construction" "this isn't [X], it's [Y]" and use the word "quietly" more than you might expect.
• Groups of three. Adjectives or short phrases in threes, used to make a thin point sound complete.
• Tidy wrap-ups and announced lists. A closing paragraph that restates the piece; an opening that tells you how many points are coming.
• Uniform rhythm. Sentences of similar length and shape, paragraph after paragraph. This is the human-readable version of the "burstiness" that detectors score, described in How machines write.
• Formatting carried over from a chat window. Bold phrases scattered through paragraphs, bulleted lists with a bolded label and a colon at the start of every item, headings in title case, emoji used as bullets.

Then there is the long dash, the most argued-about punctuation mark in this whole debate. Wikipedia's guide says chatbot output used it far more than non-professional human writing of the same genre, often in a formulaic, punched-up way. But as of October 2026 the guide files this sign under "historical indicators," covering 2022 to September 2026. In November 2025 OpenAI's chief executive Sam Altman announced that ChatGPT would finally stop using the long dash when users told it to in their custom instructions. The guide now says newer models increasingly avoid the mark altogether. Meanwhile, as the same guide notes, human editors and writers use it often. A punctuation mark that one company can switch off in an update is a poor foundation for an accusation.

Some formatting tells are much stronger, because they point to copy and paste from a specific tool.

• Interface residue. Wikipedia editors list internal reference codes that leak from chatbot interfaces into pasted text, such as ChatGPT strings like "oaicite" or "turn0search0," Gemini's "[cite: 1]" markers, and stray Markdown asterisks and pound signs where the destination does not use Markdown.
• Tracking tags in links. The guide notes links carrying "utm_source=chatgpt.com," a tag added when a link was copied out of a ChatGPT answer.

These still show only that some text passed through a chatbot. They do not show how much, or whether the person disclosed it where disclosure was allowed.

The same guide keeps a list of "ineffective indicators" that people wrongly rely on. Perfect grammar is on it, because many people are skilled writers. So is prose that mixes clinical and emotional registers, which the editors say may reflect a technical background, youth, playfulness or neurodivergence. "Fancy" or formal prose is on it too. The section opens with a warning that false accusations of AI use drive away new editors.

Factual tells: the ones you can check

Every tell above is a matter of probability. One kind is not. When a document cites a case, a study, a book or a quote that does not exist, you can prove it by looking. Language models produce these because they generate likely-sounding text rather than retrieving records; the mechanics are in Why Chatbots Make Things Up.

The best-documented record is the June 22, 2023 Opinion and Order on Sanctions in Mata v. Avianca, Inc., No. 22-cv-1461 (PKC), in the U.S. District Court for the Southern District of New York. Judge P. Kevin Castel's 43-page order found that two lawyers and their firm "submitted non-existent judicial opinions with fake quotes and citations created by the artificial intelligence tool ChatGPT," then kept standing by them after the court questioned whether they existed. The order names the invented decisions, including "Varghese," "Miller," "Petersen," "Shaboon," "Martinez" and "Durden," and describes the fake "Varghese" opinion's legal analysis as "gibberish." The order also says plainly that there is nothing inherently improper about using a reliable AI tool for help; the violation was filing citations no one had verified. The sanctions: a $5,000 penalty paid to the court, and letters to the client and to each real judge falsely named as an author of a fake opinion.

It was not the last. Damien Charlotin keeps a public database of decisions in which a court or tribunal found, or strongly implied, that a party relied on hallucinated material. As of October 2026 it lists 2,184 decisions.

Outside legal filings:

• A newspaper insert, May 2025. The Chicago Sun-Times and at least one edition of The Philadelphia Inquirer ran a syndicated summer reading list in which, NPR found, only 5 of the 15 titles were real. Percival Everett never wrote a novel called The Rainmakers. The list came from King Features, a Hearst unit, and writer Marco Buscaglia, who claimed responsibility, said it was partly generated by AI and told NPR it was a "huge mistake on my part."
• A government report, 2025. Deloitte Australia agreed to refund the final payment on an A$440,000 contract with the Department of Employment and Workplace Relations after Chris Rudge, a University of Sydney researcher, flagged its 237-page report for fabricated references, the Associated Press reported in October 2025. The original contained a fabricated quote from a federal court judgment and references to nonexistent research. The revised version disclosed that Azure OpenAI was used in writing it.

A fabricated citation proves the document is unreliable and that whoever submitted it did not check. It does not by itself prove a machine wrote the rest. People invented sources long before 2022. But for most practical purposes, an editor, a clerk or a teacher does not need to know which tool was used. A public comment citing a study that does not exist has a problem regardless of authorship, and the problem can be shown to anyone.

How good are people at spotting AI by reading?

Mixed, and it depends heavily on who is reading. In a 2021 study presented at ACL, Elizabeth Clark and colleagues found that untrained evaluators told GPT-3 text from human text at random chance level, and brief training raised accuracy only to about 55 percent. Their reasons were often contradictory.

Four years later, Jenna Russell, Marzena Karpinska and Mohit Iyyer, at ACL 2025, got a very different result with a different group. They hired people who frequently use language models for their own writing and asked them to label 300 nonfiction articles as human or AI, from models including GPT-4o and Claude. The majority vote of five such readers misclassified only 1 of the 300 articles, beating most of the commercial and open-source detectors they tested, even against paraphrasing and humanizer tools. The experts leaned on vocabulary, but also on qualities like originality and clarity that are hard for software to judge.

Before you trust your own ear, look at the conditions. Russell's experts read full articles, not a paragraph, and worked as a panel whose votes were combined. One person reading one short comment, already suspicious, is in a much weaker position. And the readers in that study knew they were looking at a mix of human and machine texts, while an accuser in the real world has decided in advance which one they hold.

Weighing a tell

Tell	What it can show	Why it falls short
Favored vocabulary	Real shift across large bodies of text	Humans use every listed word, now more than before; lists go stale as models change
Stock structures and uniform rhythm	A formulaic style	Formulaic human writers, including many students, non-native writers and some neurodivergent writers, share it
The long dash	Little, as of 2026	Widely used by professionals; newer chatbots suppress it
Chat-window formatting	Text may have been pasted from a chat interface	Some people format that way by habit; says nothing about who composed it
Interface codes, chatbot link tags	Some text passed through a specific tool	Does not show how much, or whether use was allowed
Invented citation, quote or case	The document is false on a checkable point	Does not prove a machine wrote it; proves no one verified it