AI hallucinations

An AI hallucination is when an AI model generates information that sounds plausible but is factually incorrect, fabricated, or not supported by its source material. AI hallucinations are one of the most-discussed failure modes of large language models because the model often produces them with the same fluency and confidence it uses for correct answers — making them hard to spot without verification.

Hallucinations are sometimes called confabulations. They occur in both text and multimodal models, including chatbots, AI agents, image describers, and code assistants.

Why AI hallucinations happen

Large language models don't store facts the way a database does. They predict the next token in a sequence based on statistical patterns learned during training. When the model has weak or contradictory training signal for a particular question — or when the question pushes it into territory it has limited exposure to — the most probable next tokens can produce a fluent answer that simply isn't true.

The most common causes of hallucinations:

• Gaps in training data: The model never learned the correct fact, but the question pattern is familiar enough that it generates a confident guess.
• Outdated training data: The model's training cutoff predates the correct answer.
• Ambiguous prompts: Vague or under-specified questions encourage the model to fill in the blanks.
• Lack of grounding: The model is asked to answer from memory when it should be retrieving information from an authoritative source.
• Long contexts: In long conversations, models can confuse earlier turns or drift from constraints.

Real-world examples of AI hallucinations

Hallucinations have caused notable incidents across industries. In 2023, a New York attorney was sanctioned after submitting a legal brief that cited six court cases generated by ChatGPT — none of which existed. A major airline's customer-support chatbot invented a refund policy that didn't exist, and a Canadian tribunal later held the airline liable for honoring it. AI coding assistants regularly invent function names and API methods that look real but don't exist in any library. Customer-support AI agents have invented product features, fabricated discount codes, and quoted policies that no human at the company ever wrote.

The common thread: the output is fluent, confident, and looks correct to a non-expert reader.

Hallucinations vs. errors vs. confabulation

Not every wrong AI output is a hallucination in the technical sense. A simple error is the model misunderstanding the question or making a reasoning mistake on otherwise correct facts. A hallucination specifically refers to invented or unsupported information presented as fact. Confabulation is sometimes used interchangeably with hallucination, borrowed from cognitive psychology. The distinction matters for mitigation: errors are usually addressed with better prompting or reasoning frameworks, while hallucinations are addressed by changing where the model gets its information.

How to detect and prevent AI hallucinations

The most effective single technique is AI grounding: forcing the model to base its answer on retrieved source material rather than its own parametric memory. This is most often implemented through retrieval-augmented generation (RAG), where relevant passages from a vetted knowledge base are retrieved and supplied to the model alongside the user's question.

Other layered defenses include citation requirements (the model must point to the source of every claim), confidence thresholds (the agent declines to answer when uncertainty is high), human-in-the-loop review for high-stakes outputs, and adversarial testing during development. For production AI agents, AI observability tooling tracks hallucination-prone patterns over time so teams can target fixes where they matter most.

Hallucinations in customer support AI

In customer support, a hallucinated answer is more than an embarrassment — it can create legal liability, erode trust, and force a remediation that costs more than the original ticket. The discipline for production conversational AI is to make the AI agent retrieval-first: it only commits to an answer when grounded source material supports it, and it gracefully escalates to a human or asks a clarifying question when it doesn't. Research from Stanford HAI has found that even modern reasoning models continue to hallucinate, reinforcing the case for grounding rather than relying on model improvements alone.

Frequently asked questions

What is an AI hallucination? An AI hallucination is when a model produces information that sounds plausible but is fabricated, factually wrong, or not supported by its source material.

What causes AI hallucinations? Hallucinations stem from gaps or biases in training data, outdated information, vague prompts, lack of grounding to a source, and the model's tendency to produce fluent text even when it doesn't know the answer.

How do you prevent AI hallucinations? The strongest preventions are grounding the model in retrieved source material via RAG, requiring citations for factual claims, setting confidence thresholds for when the model should defer, and using human-in-the-loop review for high-stakes outputs.

Can AI hallucinations be eliminated? Not entirely. Hallucinations are a property of how language models generate text. But they can be reduced from common to rare, and made detectable when they do occur, with grounding, citation, and observability practices.

Are AI hallucinations dangerous? They can be. In legal, medical, financial, and customer-support contexts, a confident but incorrect AI answer can cause real harm — financial loss, legal liability, or damage to customer trust.

For a deeper dive, download Decagon's guide to agentic AI for customer experience.