What Are AI Hallucinations?
AI hallucinations are answers in which an AI system confidently states something that isn’t true. OpenAI describes hallucinations as “plausible but false statements generated by language models”.1 The US National Institute of Standards and Technology (NIST) uses the term confabulation for the same thing: AI systems that “generate and confidently present erroneous or false content”.2
Hallucinations can be small, such as a wrong date, or serious, such as an invented legal case, a medical claim with no basis, or a quotation nobody ever said. What makes them dangerous is the delivery. The false answer usually looks exactly as polished and certain as a true one.
Some researchers dislike the word hallucination because it suggests the AI perceives something; the system is simply producing text. The term has stuck in everyday use anyway, and it is mostly used for generative AI: chatbots built on large language models, image generators and similar tools.
Why Do AI Hallucinations Happen?
Because a language model is built to produce likely text, not to check facts. NIST puts it plainly: confabulations are “a natural result of the way generative models are designed”.2 A language model learns statistical patterns from enormous amounts of text and generates its answer one word-piece at a time, each time predicting what is likely to come next.
Most of the time, the likeliest continuation is also correct, because true statements are common in the text the model learned from. But when a question touches something rare, recent, very specific or not in its training data at all, the model still produces a fluent, likely-sounding answer. It has no built-in sense of “I don’t know” unless it has been trained to express one.
Why Training Rewards Guessing
OpenAI’s 2025 research adds a second reason: the way models are trained and tested rewards guessing. In a paper published in September 2025, its researchers argue that hallucinations persist because “standard training and evaluation procedures reward guessing over acknowledging uncertainty”.1
They compare it to a multiple-choice exam. If you leave a question blank you get zero, but a guess might be right, so guessing scores better on average. OpenAI gives the example of a birthday: a model that guesses “September 10” has a 1-in-365 chance of being right, while answering “I don’t know” always scores zero.1 Benchmarks that only count correct answers push models towards confident guesses.
The data in the same post shows the trade-off. On one factual-question test, an older OpenAI model answered 24% correctly but gave wrong answers 75% of the time, rarely declining to answer.1
A newer model got 22% right, a similar score, but its error rate fell to 26% because it said it didn’t know 52% of the time.1 Admitting uncertainty barely changed accuracy but cut errors by roughly two thirds.
| Model (OpenAI test) | Correct | Wrong | Declined to answer |
|---|---|---|---|
| OpenAI o4-mini | 24% | 75% | 1% |
| gpt-5-thinking-mini | 22% | 26% | 52% |
OpenAI’s proposed fix is to change how tests are scored: penalise confident errors more than uncertainty, and give partial credit for appropriately saying “I’m not sure”.1
Why Hallucinations Are Hard to Notice
People tend to trust answers that are fluent, well formatted and confident, and AI answers are all three. A made-up statistic arrives with a decimal point, a fake source comes with an author, a year and a journal name, and an invented court case comes with a citation in the right legal format. The polish signals reliability even when there is nothing behind it.
The problem is worse when you don’t already know the answer, which is usually why you asked. An expert might spot an odd detail at once; a newcomer has no way to tell. That is why checking the original source, rather than judging how convincing an answer sounds, is the only dependable test.
A Quick Test You Can Try
The easiest way to see hallucination for yourself is to ask about something you know well but that few others do. Try a question about your own small business, a local event, an obscure book or a minor historical figure. Many chatbots will produce a detailed, confident answer that mixes true and invented details.
Then ask the same chatbot, “Are you sure? What is your source?” Some will correct themselves; others will repeat or even expand the error. Asking a tool to check its own answer is not the same as checking it against an independent source. Seeing this once on a topic you can judge makes it much easier to stay alert on topics you can’t.
Real Cases Where Hallucinations Caused Harm
The best-known case came from a New York court in 2023. In Mata v. Avianca, lawyers filed a brief containing court decisions that did not exist. They had used ChatGPT for legal research, and it had invented the cases, complete with fake quotes and citations.
On 22 June 2023, US District Judge P. Kevin Castel imposed a $5,000 penalty on the two lawyers and their firm, and ordered them to notify their client and each judge falsely named as the author of the fake opinions.3
The court also noted that the lawyers kept standing by the fake opinions after the judge questioned whether they existed.3 The lesson wasn’t that AI is useless. It was that an AI answer was treated as a verified source and never checked. The same pattern appears elsewhere: invented references in student essays, made-up statistics in business reports, and confident but wrong instructions in technical answers.
How Language Models Work
Hallucinations come from how chatbots generate text. See how large language models are built and trained.
Read: What Is an LLM? →The Most Common Types of AI Hallucinations
Knowing what AI hallucinations usually look like makes them easier to spot. These six patterns account for most of them.
| Type | What it looks like | How to check |
|---|---|---|
| Invented sources | Books, papers, court cases or links that don’t exist | Search for the exact title or open the link |
| Made-up numbers | Precise statistics with no traceable origin | Find the figure on an official or original site |
| Fake quotes | Words put in a real person’s mouth | Search the exact phrase in quotation marks |
| Wrong details | Dates, names or places slightly off | Check a reliable reference |
| Outdated facts | Rates, rules or prices that have since changed | Check the date on the official source |
| Self-contradiction | An answer that disagrees with what it said earlier | Ask the question again in a different way |
How to Catch AI Hallucinations
A few habits catch most errors before they cause harm. NIST’s guidance for organisations recommends reviewing and verifying sources and citations in AI outputs and fact-checking content, especially when its origin is unclear.2 The same advice works for everyone:
- Ask for sources, then open them. Check that each link exists and actually says what the answer claims.
- Verify every number that matters against an official source, such as a government statistics office, regulator or the company itself.
- Be most suspicious of precise details. Exact quotes, page numbers, case names and statistics are where invention most often hides.
- Ask again differently. If the answer changes when you rephrase the question, treat it as uncertain.
- Use tools that search or cite. Answers grounded in live sources tend to be more accurate, though they still need checking.
- Know the tool’s cut-off. Many models don’t know recent events unless they can search the web.
Can AI Hallucinations Be Fixed?
They can be reduced a lot, but probably not removed entirely. Because generating likely text is how language models work, some risk of confident error comes with the technology. Several approaches cut it down:
- Grounding: connecting the model to search results or a trusted document library, so it answers from real sources rather than memory.
- Training to say “I don’t know”: as OpenAI’s results show, models that decline more often make far fewer errors.1
- Better testing: scoring that penalises confident mistakes rather than rewarding lucky guesses.1
- Human review: a person checking the output before anything important depends on it.
Security experts take the risk seriously too. The OWASP Top 10 for large language model applications, a widely used security checklist, lists misinformation from AI models as one of the ten main risks for organisations building with them.4
Hallucinations Beyond Chatbots
Text chatbots get the most attention, but other AI tools can invent things too. Image generators may add details that were never requested or get text and small objects wrong.
Summarising tools can insert a claim that wasn’t in the original document, or drop an important “not”. Transcription tools can mishear names and numbers. The same rule applies to all of them: compare the output with the source before relying on it.
Who Is Responsible When AI Gets It Wrong?
In practice, the person or organisation that uses the answer. In the Mata v. Avianca case, the court sanctioned the lawyers, not the software.3
Businesses that publish AI-written content, give AI-generated advice or let AI act on their behalf are generally answerable for the results, just as they would be for work done by a junior employee. That is the main reason human review is built into responsible uses of AI.
Hallucinations and AI Agents
Hallucinations matter even more when AI can take actions. A chatbot that invents a fact gives you bad information; an AI agent that invents a fact might act on it, by booking the wrong thing, sending a wrong figure or changing the wrong record. That is why well-designed agents check their own results, keep a person approving important steps, and work from verified data wherever possible.
Hallucinations in Search Results
AI answers now appear at the top of many search results, so the same caution applies there. Google’s AI Overviews link to the pages they draw on, which makes checking easier: open the linked source and confirm it says what the summary claims. Our guide to Google AI Overviews explains how those summaries are put together and how they affect what you see.
What Businesses Should Do About AI Hallucinations
Any organisation using AI to write, answer customers or make decisions needs a simple checking process. A practical minimum:
- Decide which tasks AI may draft and which always need a person to approve.
- Require sources for factual claims, and check them before anything is published or sent.
- Give customer-facing AI tools access to your own verified information, such as policies and price lists, rather than letting them answer from general knowledge.
- Log what the AI produced, so mistakes can be traced and corrected.
- Tell customers when they are dealing with AI, and give them an easy way to reach a person.
How We Handle This at WhyExplain
We use AI tools in our work, so we follow the same rules we recommend. AI helps with research and first drafts, but every figure, date and claim in our articles is checked by a person against the official or primary source listed at the end of the article. If a fact can’t be traced to a reliable source, it doesn’t go in. You can read the full process in our editorial standards.
Key Terms Explained
These are the words you will meet most often in discussions of AI accuracy.
| Term | What it means |
|---|---|
| Hallucination | A plausible but false statement made by an AI system. |
| Confabulation | NIST’s term for the same problem. |
| Grounding | Linking AI answers to real, checkable sources. |
| Training data | The text and other material a model learned from. |
| Knowledge cut-off | The date after which a model has no information unless it can search. |
| Abstention | When a model declines to answer instead of guessing. |
The Bottom Line
AI makes things up because it is designed to produce likely-sounding text, and because training has often rewarded confident guesses. The answers sound equally certain whether they are true or invented, so the responsibility for checking stays with the person using them. Ask for sources, open them, and verify anything that matters. To understand the technology behind the problem, read about large language models and machine learning, or browse more AI tools explainers.
One of the most effective fixes is retrieval-augmented generation, which lets AI check trusted documents before answering.
Clear instructions and good context reduce errors. Our explainer on prompt engineering shows how.
Frequently Asked Questions
It means the AI states something false as if it were true, such as an invented fact, quote or source. The answer usually sounds just as confident as a correct one.
Language models generate text that looks like what usually appears in similar contexts. A reference list looks like titles, authors and dates, so the model can produce convincing ones that don’t exist.
It varies widely by model, task and topic, so there is no single rate. Rare, recent and very specific facts are where errors are most common.
Probably not entirely, because predicting likely text is how these models work. Grounding answers in sources, training models to admit uncertainty and human checking reduce them a lot.
It is useful for finding leads and summaries. Treat its answers as a starting point, and confirm every fact you rely on in the original source.
In everyday use they overlap. Hallucination usually means content the AI invented, such as a source that doesn’t exist, rather than a simple slip in a calculation.
Sources
- 1OpenAI: Why Language Models Hallucinate (September 2025)
- 2NIST: Artificial Intelligence Risk Management Framework, Generative AI Profile (NIST AI 600-1, July 2024)
- 3LawSites: Court Imposes Sanctions on Lawyers Who Filed Bogus Cases From ChatGPT (June 2023)
- 4OWASP: Top 10 for Large Language Model Applications (2025)
Every figure in this article is checked against the sources above. How we research and fact-check →
How Language Models Work
Hallucinations come from how chatbots generate text. See how large language models are built and trained.
100% free · No sign-up
