the AI hallucination hall of fame. share your best ones.
I’ll start. asked an AI to give me a brief background on a colleague I was about to meet. it gave me a confident, detailed biography. the person had a common name. the biography was for a completely different person with that name who died in 1987.
asked it to find a quote from a named philosopher on a specific topic. it gave me a beautifully phrased, philosophically coherent quote. I could not find the quote anywhere. I asked the AI where it came from. it said “I may have synthesised this based on their known views.” I did not ask for a synthesised quote.
asked for three peer-reviewed sources on a topic. all three had real journal names, real-looking DOIs, plausible author names. none of them existed. one of the journals was real. the specific issue and page numbers were not.
mine told me that a specific law had been passed in a specific country in a specific year. confident. cited the date. the law did not exist. the country had debated a similar law but not passed it. when I pushed back it apologised and then told me the same false information again in different words.
PhD candidate. my hall of fame entry.
asked it to summarise a specific paper by a real researcher. gave it the title and author. it produced a detailed, structured summary. the methods section, the findings, the limitations, all plausible.
the paper I asked about was a working paper that had not been published. it did not exist in any database. the AI had generated an entire research paper’s worth of content and summarised it.
I told it the paper didn’t exist. it said it was possible it had confused it with a similar paper. it then offered to summarise that similar paper. I did not take it up on this.
the thing that gets me about all of these is that the confidence is calibrated to the wrongness. the more made-up the information, the more sure it sounds.