How accurate is DetZero?
The short version: strong, published, and imperfect. These are the numbers from our own validation of the current model (detector-v4, November 2026), measured on a benchmark built to resist the measurement flaw that fooled us last time — see “Why we replaced the previous model” below. They are not best-case marketing figures.
The numbers
Every figure comes from one locked benchmark (our “DGB v2” set: 900 texts — 370 human, 310 AI, plus edited, mixed and paraphrased AI texts). The human texts were published before AI writing tools existed, keep their natural paragraph formatting, and are matched to the AI texts for length. The benchmark's topics are completely separate from the ones the model was tuned on, and it was scored exactly once, after the model was frozen.
| Test | Result | What it means |
|---|---|---|
| Ranking quality on the locked benchmark (AUROC) | 0.983 | Given one human and one AI text, the detector ranks the AI one higher about 98 times in 100. |
| AI texts caught at a strict operating point (only ~1% of human texts flagged) | 82.3% | This is the number we optimise for: catching AI writing while keeping false alarms on human writing at about 1 in 100. |
| AI texts caught at a looser operating point (5% of human texts flagged) | 94.8% | If you tolerate more false alarms, more AI text is caught. Our live verdict bands use the strict end of this range. |
| AI texts caught, by generator (strict operating point) | 80–87% | OpenAI 81%, Anthropic 84%, DeepSeek 80%, Google 87% — no generator family collapses. (The Google figure rests on 30 texts, so treat it as approximate.) |
| Edited AI text / paraphrased AI text detected (strict operating point) | 90% / 68% | Lightly edited AI text is still usually caught. Rewriting AI text in different words is harder — about a third slips through at the strict setting. |
| What to expect on brand-new topics | ~58% caught | When we set the threshold using only our separate development set and then measured on unseen topics, recall was about 58% at the same ~1% false-flag rate. Real-world performance on topics unlike our benchmark will land closer to this figure than to 82%. |
How the model was built
detector-v4 is an ensemble. A neural classifier, a text-predictability signal and writing-rhythm features produce one score; two further models look at linguistic features of the writing itself (sentence-length patterns, punctuation, vocabulary structure and similar). A small calibrated combiner weighs the three against each other. Everything was trained and tuned on a development set whose topics are disjoint from the benchmark above, and every check runs on our own server — no third-party AI API sees your text.
Why we replaced the previous model
Plainly: our previous model (detector-v3.1) had learned a formatting cue instead of a writing cue. In its training data, AI texts kept their paragraphs while human texts had been flattened into plain blocks — so “has paragraphs” became one of its strongest signals for “AI”. Its early test results looked excellent because the test sets shared the same flaw. When we rebuilt the benchmark with naturally formatted, length-matched human texts, the same model caught only about 4% of AI text at a 1% false-flag rate — and it flagged ordinary paragraphed human writing at a rate we could not control. We replaced it with the ensemble above, which was validated on the corrected benchmark from the start. We would rather publish that story than a flattering number.
Honest limitations
- No detector is proof. Scores are probabilities, not verdicts about a person. Never use a detector score as the sole evidence in an academic or employment decision.
- Mixed human + AI text is our known blind spot. A document that splices human-written and AI-written passages together is detected only rarely at our strict operating point (about 7% of such texts in the benchmark). If that is the case you care about, treat a low score as weak evidence, not a clearance.
- Short texts are unreliable — that is why we require at least 40 words, and longer texts give steadier results.
- Heavily edited or “humanized” AI text scores lower and can slip into the mixed zone; paraphrased AI text is caught about two times in three at the strict setting.
- Non-native English writing can look more regular than average and score higher than it should. Every known detector has this weakness; ours is no exception.
- Other languages are not what the model was trained on; results outside English are experimental.
- Our benchmark is web prose (encyclopedic and essay-style writing). Other kinds of writing — student essays, business documents — will differ somewhat from the figures above, in either direction.
- AI writing keeps changing. We retrain on newer data over time, and the model version serving this site is updated when a new one passes the same tests.
Think a result is wrong? Use the “This result is wrong” button under any result — it sends us only the score you were shown (never your text), and those reports guide the next round of testing.