How-ToGuide
How AI Writing Detectors Work, and How to Read a Score
AI writing detectors do not look for a hidden signature. They estimate how predictable a text is and compare that with patterns they learned from examples. That makes them useful as a prompt for a conversation and unreliable as proof, and the difference matters most when the text being judged is yours.
On this page
What a detector measures
Language models write by repeatedly choosing a likely next word. Text produced that way tends to be statistically smooth: each word is close to what a model would have predicted. Most detectors run the text through a model of their own and measure that predictability, often called perplexity, along with how much it varies from sentence to sentence, sometimes called burstiness. Human writing usually wanders more: an unexpected word here, a very short sentence there. A trained classifier then turns those measurements into a percentage.
The weakness is built in. Plenty of human writing is predictable by design: lab reports, legal summaries, product descriptions, and anything written carefully in a second language with a limited, safe vocabulary. Plenty of machine text is not, once someone has edited it or asked the model to vary its style. Detectors are therefore always trading false alarms against misses, and no setting removes both.
| Situation | How reliable is a score? | Why |
|---|---|---|
| Long, unedited machine text | Fairly reliable | Strong, consistent statistical signal |
| Machine text lightly edited by a person | Unreliable | Edits break up the pattern the tool looks for |
| Short text under about 250 words | Unreliable | Too little text to measure |
| Formulaic genres (reports, abstracts, instructions) | Unreliable | Human writing is predictable here too |
| Writing by non-native English speakers | Unreliable, biased toward false positives | Simpler, safer word choices look machine-like |
| Mixed text, part human and part machine | Partly reliable | Some tools highlight passages, with wide error margins |
What the research says
Independent testing has been consistent. A 2023 evaluation of fourteen detection tools published in the International Journal for Educational Integrity found none of them accurate or reliable enough for use as evidence, with performance falling further on edited or paraphrased text. A Stanford study the same year found that several detectors flagged a large share of essays written by non-native English speakers as machine-written while rarely flagging essays by native speakers. A research group at the University of Maryland showed that paraphrasing defeats most detectors. One major AI developer withdrew its own public text classifier in 2023, citing its low rate of accuracy, and several universities have switched off automated detection features for the same reason.
How to read a score
Check the length
Below a few hundred words, ignore the number. Detectors themselves usually warn about short inputs.
Read it as a probability
A score of 60 percent means the tool is unsure, not that 60 percent of the text is machine-written. Look for whether the tool reports which passages drove the score.
Try more than one tool
Different detectors often disagree sharply on the same text. Disagreement is itself a sign the signal is weak.
Consider the genre and the writer
Formulaic writing and second-language writing produce false positives. Weigh the score accordingly.
Look at the process, not just the product
The strongest evidence of authorship is a record of the writing happening. The guide to keeping version history of your drafts shows how to create one.
If your writing is flagged
Stay calm and gather the record. Open the document's version history and show how the draft developed. Bring notes, outlines and sources. Offer to talk through the argument or to write a short passage on the topic in person. If you used an assistant in a permitted way, say exactly where. Ask which tool produced the score and what its published error rate is. Most reviewers want a fair process, and a documented history gives them one. If you draft with a local tool, the guide to AI tools and laptop privacy covers what those tools store about your sessions.
This explainer is part of the practical how-to section. If you draft with help from a model on your own machine, the guide to an offline AI writing assistant keeps that help separate and private, and a quick screenshot of a detector result is worth saving alongside your version history if you are ever asked about a score.
Common questions
Can an AI detector be 100 percent accurate?
No. Human and machine text overlap statistically, so every detector makes both kinds of error. Published figures describe averages on test sets, not certainty on your text.
Why was my own essay flagged?
Common reasons are a short length, a formal or formulaic style, careful simple phrasing, or heavy use of grammar tools. None of them mean you did not write it.
Do grammar checkers trigger detectors?
They can. Tools that rewrite sentences make text smoother and more predictable, which pushes scores up.
Is there a hidden watermark in AI text?
Some developers have researched statistical watermarks, but most text in circulation carries none that a public tool can check, and editing weakens any that exist.
What is the best defence against a false positive?
A version history that shows the writing process, plus notes and sources. Start keeping one before you need it.
Sources
- arXiv: Testing of Detection Tools for AI-Generated Text: weber-wulff and colleagues on detector accuracy.
- arXiv: Liang et al., detector bias against non-native English writers: non-native writing flagged more often.
- arXiv: Can AI-Generated Text be Reliably Detected?: sadasivan and colleagues on the limits of detection.
- Vanderbilt University: Guidance on AI Detection and Why We're Disabling Turnitin's AI Detector: a university disabling a detection feature in august 2023.
Also consulted: Weber-Wulff and colleagues, Testing of detection tools for AI-generated text, International Journal for Educational Integrity, volume 19, 2023; Liang, Yuksekgonul, Mao, Wu and Zou, study of detector bias against non-native English writers, Patterns, volume 4, July 2023; Sadasivan and colleagues, Can AI-generated text be reliably detected?, University of Maryland, 2023; Vanderbilt University, guidance on disabling an AI detection feature, August 2023.