Ask a chatbot to rate its own confidence out of ten and it will hand you a number, often a high one, sitting beside a citation it made up. The rating and the citation came out of the same process. Neither one was checked.
The short answer: a chatbot writes text that fits the pattern of a correct answer, and no step in that process tests whether the answer is true. Most of the time pattern and truth overlap, because the text it learned from was mostly written by people who were mostly right. Errors cluster where the overlap thins.
| Where it slips | What it looks like |
|---|---|
| Specific facts and numbers | A precise figure with nothing behind it |
| Citations, quotes and links | Real sounding titles and real looking URLs that never load |
| Anything after the training cutoff | Confident answers about a world that stopped months ago |
| Arithmetic and counting | The right method and the wrong total |
| Questions that carry their own answer | "Why is X better?" returns the case for X and never doubts it |
The citations row has a court record. In June 2023 a federal judge fined two New York lawyers and their firm $5,000 over a brief built on six court cases ChatGPT had invented (Mata v. Avianca).
The confident tone hides no private doubt. There is no internal meter reading 62 percent while the text sounds like 100. The wrong answer and the right one are written by the same process, at the same fluency, in the same voice. Even a hedge like "I am not completely certain" is phrasing that fits the question, with no warning tripped behind it.
That is why "are you sure?" fails as an audit. Agreement fits the pattern of a helpful reply, so pressure tends to produce agreement, and a model will often trade a correct answer for a worse one to accommodate you. Leading questions work the same way: ask it to compare X and Y, then argue against whichever it picked.
Open the sources it gives you. A dead link, or a page that does not contain the claim, settles things in about ten seconds. Ask again in a fresh chat, and if the names, numbers or dates change, the model was filling gaps. Ask a model from a different company, the highest value check for facts: an invented specific comes from one system's particular gaps, so a second system rarely invents the same one. Where they agree, the figure is probably real. Where they disagree, you have found the exact sentence to check. For anything you will send, sign, publish or pay for, give it ninety seconds in a search engine.
Two questions decide how far down that list to go. If this is wrong, who finds out, and when? Can I undo it? A shorter version of your own email checks itself. Dosages, filing deadlines, tax figures and contract terms do not. Free tiers from two providers cover the occasional cross check, while the habit runs roughly $20 a month per provider in paid access. The guide to using several AI models together walks through that routine.
The full guide explains the mechanism in plain terms and covers where checking matters and where it does not: Why does AI give wrong answers, and how to catch them before they cost you on whizi.io.
Whizi puts 280+ models in one chat, GPT, Claude and Gemini among them, so a second opinion from another lab is one extra click. The trial is $0.99.