AI hallucinates less than humans now, says Anthropic CEO
Anthropic CEO Dario Amodei says modern AI models like Claude 3.5 hallucinate less often than humans in factual tasks, based on internal benchmarks.
At the VivaTech 2025 conference in Paris, Anthropic CEO Dario Amodei made a striking claim: the most advanced artificial intelligence models now “hallucinate” less frequently than humans—at least when evaluated on factual tasks.
In AI development, hallucination refers to a model generating false or fabricated information. It’s long been one of the biggest challenges for companies building large language models like Claude, GPT, or Gemini. But according to Amodei, when tested under structured conditions, the latest AI systems are already beating people in factual accuracy.
Human minds are not perfect either
Amodei pointed out that humans, too, frequently misremember facts, believe misinformation, or provide incorrect answers with confidence. “If you define hallucination as confidently saying something that's wrong, humans do that a lot,” he said.
He cited recent internal research where AI models like Claude 3.5 performed better than human participants on factual quizzes. The models were tested on tasks requiring correct answers to verifiable questions — a setting in which AI has apparently become more dependable.
Hallucinations haven't disappeared, just reduced
Despite the breakthrough, Amodei cautioned that hallucinations haven’t been eliminated. In less structured, open-ended conversations, models can still generate inaccurate or misleading content. The improvement, he said, is mainly visible when models are used in narrow, clearly defined factual settings.
The CEO also emphasised the importance of context: how a prompt is phrased can significantly influence the accuracy of a model’s response. In high-stakes use cases like legal advice or medical information, even a small error rate remains a critical concern.
The need for better metrics
Amodei urged the industry to develop more consistent definitions and benchmarks for hallucinations. Right now, there’s little agreement on what counts as a hallucination and how to reliably measure it. “You can’t fix what you don’t measure precisely,” he noted.