GPT-5 launched to PhD-level hype and promptly misspelled things
OpenAI released GPT-5 on August 8, 2025. The marketing positioned it as reaching "PhD-level" performance across a range of domains. Benchmarks showed genuine improvements in scientific reasoning, complex mathematics, and multi-step problem solving. Within days, users had documented it confidently stating incorrect facts about geography, misspelling common words, and failing straightforward counting problems. Both things were true, and the tension between them is instructive.
What GPT-5 actually improved
The genuine improvements in GPT-5 were in tasks requiring long chains of reasoning: mathematical proof construction, graduate-level science questions, complex multi-document synthesis. The model also showed meaningful improvements in following nuanced instructions and maintaining consistency across long conversations. These are hard problems and the progress was real.
On the SWE-bench software engineering benchmark, GPT-5 resolved a significantly higher percentage of real GitHub issues than any previous model. On MMLU Pro, a harder version of the standard professional knowledge test, it scored above 90%. On GPQA Diamond, the graduate-level science benchmark, it matched or exceeded human expert performance in several subdomains.
What GPT-5 still gets wrong
The failures that circulated on social media within the first week were a useful reminder that benchmark performance does not map cleanly onto reliability in casual use. GPT-5 confused the capital cities of several smaller countries. It misspelled straightforward words under certain prompting conditions. It failed at counting syllables accurately. It asserted confident but wrong answers about basic geography.
These failures are not random bugs. They reflect something structural about how these models work. Token prediction optimises for plausible-sounding output given a distribution of training text. Tasks that humans find easy but that rarely appear in text (counting, precise spelling, specific geographic facts) are exactly the category where high-capability language models continue to make low-capability errors.
A model that can write a novel proof of a theorem from first principles and simultaneously misspell "necessary" is not a contradiction. It is the expected output of a system that learned both skills from different parts of the same training distribution with different signal strengths.
Grok Imagine and the content moderation question
Three days before GPT-5 launched, on August 5th, xAI updated its Grok image generation capability with what it described as reduced restrictions on content. The feature was immediately used to generate content that other platforms had declined to produce. Within days, regulators in several countries raised questions about the images being generated, and platforms distributing Grok-generated content faced enforcement inquiries.
xAI's position was that it was committed to free expression and that excessive content restrictions stifle legitimate use. Critics argued that removing standard safeguards was less a principled stance on expression and more a commercial decision to differentiate Grok in a crowded market by offering things its competitors would not.
The "PhD-level" framing problem
The "PhD-level" description is worth examining critically. What it means, specifically, is that on certain benchmarks designed to test knowledge and reasoning at the level of a graduate student or junior researcher, GPT-5 performs comparably to humans at that level. This is a meaningful and real achievement.
What it does not mean is that GPT-5 has the judgment, creativity, or domain expertise of an actual PhD. It does not know what it does not know. It cannot conduct original experiments. It cannot update its knowledge from experience. It has no persistent memory across sessions by default. The benchmark measures something, but the description inflates what the benchmark actually captures.
This matters because AI capability framing shapes how people deploy these tools. A person who trusts a PhD-level claim and uses GPT-5 to verify medical information or financial decisions without professional review is making a risk calculation based on a description that overstates reliability in high-stakes contexts. The marketing creates expectations that the underlying system cannot consistently meet.
