Over the weekend, I was using ChatGPT to get suggestions for summer break reading for my rising fourth grader. My dude loves airplanes. He eats, sleeps, and breathes commercial airliners. So I thought, I’ll get him more interested in reading by finding kids’ books on his favorite topic.
As it turns out, there aren’t that many books about commercial aviation written for elementary-age kids. I asked ChatGPT for some recommendations. It suggested a few books and series I was already familiar with, and then it offered “one series most parents miss”:
The old Airline Pilot Club books by aviation author Patrick Chiles are specifically written for elementary-aged kids who are fascinated by commercial flying. They follow modern airline operations, airports, and pilots. They’re not widely known outside aviation circles, but they’re among the few chapter books aimed directly at “future airline pilot” kids.
Hot diggity dog, I thought! I’ll get these! But the public library didn’t have any of them. Amazon and eBay didn’t, either. Maybe they’re out of print?
Then it dawned on me that ChatGPT might have hallucinated that whole paragraph.
So I asked, “Wait, are these books real?” And it responded:
Good catch. I don’t think those books are real.
ChatGPT completely made this up! But why? How did this happen? I had it look under its own hood for an explanation:
I recognized a real pattern:
- There are children’s chapter book series built around specific interests (horses, trains, sports, dogs, etc.).
- There are real aviation authors, including Patrick Chiles.
- There are real aviation-themed books for children, though not many.
Based on that pattern, it created a book series that sounded like it could exist but didn’t. AI hallucinations in low-stakes contexts like this are disappointing, but errors when the stakes are higher can lead to real problems.
What are AI hallucinations?
Finding that hallucination felt like catching AI in a lie. But it wasn’t trying to mislead me on purpose.
Hallucinations are a side effect of how generative AI works. It identifies patterns in large amounts of text and predicts what content is likely to fit a given context. It doesn’t independently verify every claim before presenting it. As a result, AI sometimes gives you information that sounds completely believable despite being completely inaccurate.
Hallucinations are a known limitation of chatbots like ChatGPT, which is why it helps to understand what they look like, how to spot them, and how to reduce the risk.
What do AI hallucinations look like?
Lack of trust in AI results is a major factor in whether and how people use this technology. A Pew Research Center survey recently found that 76% of respondents said they don’t use AI chatbots because they don’t trust them to provide accurate information.
If we’re going to talk about AI accuracy, it helps to be specific about what can go wrong. Let’s move beyond the vague idea that AI sometimes “makes things up” and understand how hallucinations show up in AI output. They aren’t always obvious, and they don’t all look the same:
Hallucinations are more likely in some contexts than others.
Users often assume hallucinations are random. They’re not. Risk increases when the topic is obscure or changing rapidly, the task requires specialized expertise, or the model is asked for exact numbers or citations.
Hallucinations often sound convincing.
When AI gets things wrong, it’s more often a subtle error than a whopper. AI presents incorrect information with the same confidence and detail as correct information. Because people are wired to associate confidence with authority, it can be easy to miss these errors.
Hallucinations aren’t all-or-nothing.
A lot of the time, AI hallucinates one part of a fact or explanation while getting the rest correct. Or it provides real facts presented in the wrong context, or conclusions that aren’t supported by the evidence.
Hallucinations aren’t always about what’s included.
Sometimes the problem is what’s left out. AI may omit important caveats, exceptions, or context, resulting in an answer that is technically accurate but misleading.
How to spot AI hallucinations when you can’t verify everything
In a perfect world, people would be able to catch every AI hallucination. But realistically, most of the time we’re doing more spot checking than full fact-checking. How can we find hallucinations if we can’t check everything?
Pay extra attention to specific details.
Hallucinations don’t happen randomly. If you have limited time to fact-check, focus on details like dates, names, statistics, citations, and product specifications.
Are your spidey senses tingling? Listen up.
Detecting hallucinations often starts with human intuition. Sometimes, something just seems off:
- The answer seems oddly specific.
- It sounds too certain.
- It provides statistics without sources.
- It cites studies you’ve never heard of.
- It contradicts what you already know.
- It answers a question you didn’t actually ask.
Prioritize which claims to verify.
If you can’t fact-check every sentence, focus on claims that are:
- New to you
- Central to the answer
- Highly specific
- Likely to influence a decision
- Something you’ll repeat to others
Check the original source material, not just the citation.
It’s not enough to check that a citation exists. Hallucinations sometimes involve real sources with incorrect details or sources that exist but don’t support the stated claim.
Apply more scrutiny when the stakes are higher.
Hallucinating a book suggestion is very different from making a mistake in a medical recommendation, financial analysis, or technical report. Consider how the information will be used and what could happen if it’s wrong. The higher the potential consequences, the more important it is to fully verify key claims.
Prompting AI to minimize the risk of hallucinations
A lot of guidance implies that if you’re careful enough, you can avoid all AI hallucinations. Unfortunately, they’re baked into how generative AI works. But there are ways to reduce the chances that a chatbot will invent stuff. Here are a few practical tips, with ideas for specific prompts to try:
Provide constraints: Limit what AI is allowed to use or do so it has less opportunity to invent information.
- “Use only the information provided below.”
- “Answer this question based solely on the attached report.”
- “If the information is not in the source material, say so.”
Ask for evidence: Require AI to show where its conclusions came from.
- “Support each conclusion with evidence from the source.”
- “For each recommendation, cite the relevant information from the document.”
- “What information in the source led you to this conclusion?”
Separate facts from interpretation: Ask AI to distinguish between what the source actually says and what it is inferring from the source.
- “List the facts first, then provide your analysis.”
- “Separate observations from conclusions.”
- “Identify which statements come directly from the source and which are interpretations.”
Allow uncertainty: Tell AI to acknowledge gaps rather than filling them with guesses.
- “If the answer is unclear, explain what information is missing.”
- “Do not speculate. If you don’t know, say so.”
- “Identify any questions that cannot be answered from the information provided.”
Ask for a confidence check: Prompt AI to flag parts of its response that may be less reliable.
- “Which claims in this response should be independently verified?”
- “How confident are you in each of these conclusions?”
- “What parts of this answer are most uncertain?”
Reduce the need for AI to “fill in the blanks”: Provide enough context that AI doesn’t have to make assumptions.
- “Before answering, ask me any questions needed to clarify the request.”
- “Here is the background information, audience, and purpose for this task.”
- “Use the definitions below when interpreting these terms.”
Break complex tasks into smaller steps: Have AI tackle one part of the problem at a time rather than trying to do everything at once.
- “First identify the key findings. Then summarize them.”
- “Start by extracting the relevant information before making recommendations.”
- “Complete this analysis in stages and explain each step separately.”
Using AI despite hallucinations
The sneakiest AI hallucinations aren’t bizarre or out-there. They’re the ones that fit neatly into the user’s expectations (like my expectation that such a chapter-book series probably existed). Because they’re plausible and presented confidently, they may slip by you unnoticed.
Experience helps. The more you use AI chatbots, the better you’ll get at spotting answers that don’t quite pass the smell test. But it’s easy to become complacent as you get better. When you’re moving quickly or working in a familiar subject area, it’s easy to accept an answer that sounds right without looking too closely.
That’s why healthy skepticism is one of the most valuable AI skills you can develop. You don’t need to fact-check every sentence or assume every response is wrong. But when the stakes are high, or an answer seems a little too perfect, it’s worth taking a closer look.