LLM Hallucinations: Why AI Makes Things Up

LLM hallucinations: the practitioner's taxonomy of what's fixable vs what isn't

Hallucination coverage in the mainstream press tends to flatten a multi-failure-mode phenomenon into a single category. The useful working distinctions come from threads on r/MachineLearning, r/LocalLLaMA, and recent papers like the Anthropic research on model beliefs and uncertainty.

The operational taxonomy practitioners actually use:

• →Intrinsic hallucination. The model contradicts provided context. Most fixable — RAG + constrained generation + grounded-answer prompting.
• →Extrinsic hallucination. The model asserts something not in provided context, and it happens to be wrong. Harder to fix — requires external verification or retrieval of the actual fact.
• →Fabricated citations. The model invents a plausible-looking paper, URL, or quote. The GPT-4 technical report documented this explicitly; it's reduced in newer models but not gone.
• →Confident confabulation. The model states an incorrect answer with no hedging. Arguably the most dangerous failure mode because users can't detect it by tone.

What genuinely reduces each class:

• →For intrinsic hallucination: Strong retrieval + "answer only from the provided context" prompting + refusal when context is insufficient. Measurable improvements when implemented correctly.
• →For extrinsic hallucination: External tool calls (search, databases) where ground truth exists. For pure reasoning tasks, multi-step verification with self-consistency helps.
• →For fabricated citations: Require citations only when you can verify them. Models are better at "here's what I know" than "here's the source."
• →For confident confabulation: Prompt the model to express uncertainty explicitly. Use verifier models. Pipe-through-search for critical facts.

What Reddit practitioners consistently emphasise:

• →Temperature matters. Default 0.7 is wrong for factual tasks; 0.0-0.3 reduces hallucination on recall tasks.
• →Model choice matters. Claude Opus and GPT-5 hallucinate meaningfully less than smaller models on hard tasks. For easy tasks, smaller models are fine.
• →RAG without ranking is not RAG. Many "RAG isn't working" threads are really "my retrieval doesn't rank well."
• →Users are part of the system. Training users to verify important facts is as important as any technical mitigation.

What's mostly snake oil:

• →"Hallucination-free" claims. No product delivers this. Any vendor claiming zero hallucination is marketing.
• →Prompt-only solutions like "think step by step." These sometimes help, often don't, and occasionally make things worse by inventing plausible chains.
• →"Just use a bigger model." Helps on some tasks, hurts on others. Model selection should be task-specific.

The honest framing: hallucination is a probabilistic feature of LLMs, not a bug being patched. You mitigate it with architecture, not hope. The teams that ship reliable LLM applications have invested in RAG, verification, and uncertainty handling; the ones that treat hallucination as "will be fixed in the next model" keep getting surprised.

What Are Hallucinations?

In AI, a hallucination is when the model generates content that is:

• →Factually incorrect, made-up facts, wrong dates, false claims
• →Nonsensical, logically inconsistent or meaningless
• →Fabricated, invented sources, fake quotes, non-existent people

The term comes from the idea that the AI "sees" something that isn't there.

Why Do LLMs Hallucinate?

1. They're Pattern Matchers, Not Knowledge Bases

LLMs predict the next most likely word based on patterns learned during training. They don't "know" facts-they've learned statistical associations.

When asked something outside their patterns, they generate plausible-sounding text that may not be true.

2. No Real-Time Verification

LLMs can't check if what they're saying is accurate. They have no way to verify facts against the real world while generating responses.

3. Training Data Issues

If training data contains errors, outdated information, or biases, these get baked into the model.

4. Pressure to Respond

LLMs are trained to always generate a response. When they don't know something, they often generate an answer anyway rather than saying "I don't know."

5. Probability vs. Truth

Language models optimize for probability, not truth. The most likely next word isn't always the most accurate.

Examples of Hallucinations

Fabricated Citations

User: Cite sources about X

AI: According to Johnson et al. (2019) published in the 
Journal of AI Research...

Reality: This paper doesn't exist

Invented Facts

User: When was the Eiffel Tower built?

AI: The Eiffel Tower was completed in 1887...

Reality: It was completed in 1889

Non-Existent People

User: Tell me about the CEO of Company X

AI: John Smith has been CEO since 2015 and previously...

Reality: There is no John Smith, or the details are wrong

Why This Matters

For Individuals

If you trust AI output without verification, you may spread misinformation, make bad decisions, or embarrass yourself professionally.

For Businesses

AI hallucinations in customer-facing applications can damage trust, cause legal issues, or lead to costly mistakes.

For Society

Widespread AI use amplifies the spread of hallucinated information, making it harder to distinguish fact from fiction.

How to Reduce Hallucinations

While you can't eliminate hallucinations entirely, you can reduce them:

1. RAG (Retrieval-Augmented Generation)

Ground the AI's responses in actual documents. When the AI has real sources to reference, hallucinations decrease.

2. Ask for Citations

Request sources for claims. While the AI may still fabricate them, it creates accountability and something you can verify.

3. Narrow the Scope

More specific, constrained questions tend to produce more accurate answers than broad, open-ended ones.

4. Cross-Check

Never fully trust AI output for factual claims. Verify important information independently.

5. Self-Consistency

Ask the same question multiple ways. If the answers contradict each other, there's likely a hallucination.

What's Being Done

The AI industry is actively working on solutions:

• →Better training techniques to improve factual grounding
• →RAG systems becoming standard for knowledge-intensive applications
• →Confidence indicators to signal when the AI is uncertain
• →Citation requirements built into model outputs
• →Fact-checking layers in enterprise applications

Quick Summary

1. →Hallucinations are when AI generates false but plausible content
2. →They happen because LLMs predict patterns, not facts
3. →AI cannot verify its own claims in real-time
4. →RAG and verification are essential for accuracy
5. →Always cross-check important information from AI

Ready to Build Reliable AI Systems?

This article covered the what and why of AI hallucinations. But building systems that users can trust requires deeper techniques.

In our training modules, you'll learn:

• →Module 5: RAG, Ground AI in reliable sources
• →Module 8: Ethics & Security, Build responsible AI systems

→ Explore the Full Program