The internet is filling up with AI content and nobody can tell anymore
By mid-2025, a practical problem had emerged that the AI industry had discussed theoretically for years: the web was filling up with AI-generated content at a rate that was starting to affect the quality of information retrieval. Not just spam or low-effort articles, but genuine, detailed, well-formatted text that was plausible enough to pass casual inspection but that cited sources that did not exist, contained errors presented confidently, and was contributing to what researchers were calling model collapse in downstream training data.
The volume problem
Content farms have existed for decades. The difference in 2025 was the combination of quality and scale. A single person with access to a mid-range language model could generate hundreds of articles per day at a quality level that would have required a team of writers a year earlier. The marginal cost of content production approached zero.
Research published in mid-2025 estimated that a significant portion of new web content published in early 2025 was AI-generated or AI-assisted to a degree that made it effectively synthetic. Precise figures are hard to establish because detection is imperfect, but the direction was unambiguous. More content was being published than ever before, and less of it was grounded in direct human experience or original research.
- ~200M
- New web pages indexed weekly (est.)
- $0
- Marginal cost per AI article
- 92%
- AI detector false-positive rate on human writing
- 3x
- Growth in content volume 2023-2025
Why detection does not work well
AI detection tools had a fundamental credibility problem by 2025. Every study that evaluated them found high rates of both false positives (classifying human-written text as AI-generated) and false negatives (classifying AI text as human). The false positive problem was particularly damaging because students and writers were being accused of using AI tools for work they had written themselves.
The deeper issue is that detection is an asymmetric problem. An AI system trained to avoid detection can generate text that evades classifiers. A detection system trained on that text improves and catches the next generation of evasion. The cycle never resolves in favour of detection because the generator has an inherent advantage.
Turnitin, one of the largest academic plagiarism detection services, reported in 2025 that its AI detection feature had been used to flag over 22 million papers. It also acknowledged that students with demonstrated false positives had legitimate grounds to appeal, which its administrators had no reliable way to evaluate.
Watermarking: the technical alternative
The technically more promising approach is watermarking, where a statistical signal is embedded in AI-generated text at generation time. Google's SynthID technology, initially developed for AI-generated images, was being adapted for text. Several AI labs were working on similar approaches. The idea is that rather than trying to detect AI content after the fact, you embed a traceable signal during generation.
| Item | Value |
|---|---|
AI detectors How: Classify text using trained classifiers Limitation: High error rates, gameable | |
Statistical watermarking How: Embed token bias patterns at generation Limitation: Editing removes the signal | |
Cryptographic watermarks How: Embed hash in generation metadata Limitation: Requires API, not for open models | |
Provenance metadata How: C2PA content credentials on files Limitation: Easy to strip, voluntary adoption | |
Human attestation How: Signed verification of human authorship Limitation: Not scalable, trust problem |
The SEO and creator economy impact
Google's search quality team was clearly aware of the problem and was updating its ranking algorithms to deprioritise thin content that matched patterns associated with AI generation. This created a peculiar incentive structure: sites with AI content had a declining return on that content over time as Google devalued it, which pushed them to produce more content to compensate, which degraded quality further.
For human content creators, the situation was demoralising in a specific way. Writing a detailed, expert article that took a week to research and compose now competed in search rankings against a thousand AI-generated articles on the same topic. The market for undifferentiated informational content had effectively collapsed. The value of genuinely original reporting, primary research, and expert analysis remained, but it required producers to find direct audiences rather than relying on search discovery.
Model collapse: the long tail problem
Researchers had been warning since 2023 about a phenomenon called model collapse: if AI models are trained on data that increasingly consists of AI-generated content, performance degrades over generations because the training data loses the diversity and specificity of human-produced information. By 2025 this was no longer a theoretical concern. Several research groups had demonstrated degradation empirically on models trained with progressively higher proportions of synthetic data.
The uncomfortable conclusion is that the viability of the next generation of AI models depends partly on the continued existence of a web with substantial human-authored content. The AI industry was, in a sense, consuming the resource it depended on. The labs were aware of this and were exploring agreements with news organisations and publishers for access to verified human-authored text, though the economic terms of these agreements remained contentious.