When AI Gets It Wrong - Your Best Examples of Confident Hallucination

[SpinorWave]

Not a pile-on. More a practical thread for calibrating when to trust AI coding assistants and language models and when to verify carefully.

The confident wrong answer is the dangerous failure mode, much more so than obvious errors or appropriate expressions of uncertainty. The cases where the model produces plausible-sounding code, citations, API calls, or explanations that are simply false are the ones worth documenting.

What are your best examples, and has it changed how you use these tools?

[Fam41]

Asked Claude to describe a function from a library I was using. It described a function that did not exist in that version of the library with complete confidence including the parameter names. Cost me an hour

[TheRizz]

The invented citations problem is well documented but I keep running into it. Academic papers with real-sounding authors and titles that simply do not exist. Always verify before including in anything

[GameChanger]

Generated a regex that looked correct, passed my test cases, and then failed on input I had not tested because the model had subtly misunderstood what I was trying to match. The code was wrong in a very specific and hard to spot way

[QueueJump58]

Asked about a specific SMF function and got a detailed explanation of a function with a slightly different name that worked slightly differently. Close enough to be misleading, wrong enough to break things

[Foundry69]

The date problem. Models confidently stating current information that is actually from their training cutoff without flagging the uncertainty. Particularly bad for anything that changes frequently like API specifications or package versions

[Coder65]

I have started treating AI output the way I treat a confident junior developer. Probably right, worth checking, especially on anything that touches security or external systems

[ClientPilgrim]

The best calibration tool is to ask the model to explain its reasoning. Hallucinations often become visible when the model has to justify them step by step