AI Makes Things Up. So Do You.

Ask a chatbot why it chose an answer and it will give you a neat, step-by-step reason. Sometimes that reason is not what decided the answer. People do the same thing: psychologists call it confabulation, giving a sincere reason that isn't the real one. This video puts the two side by side: four identical pairs of stockings (Nisbett & Wilson 1977), a magician's photo swap (Johansson et al. 2005), a trap that nudged AI models towards option A (Turpin et al. 2023), assistants that back down when you ask "are you sure?" (Sharma et al. 2024), why models guess instead of saying "I don't know" (Kalai et al. 2025, 2026), and what I'd do about it. The narration in this video is AI-generated (a synthetic voice, Maya1). The script was written with AI help and checked against the sources below. The limits, honestly: the stocking study is a short field report inside a 1977 review paper, with no table or full methods there. The face-swap study (Johansson et al. 2005) tested 120 people in a lab, with photos of women's faces: 13% of swaps were caught at once and at most 26% by any measure. The same team found the same blindness with jam and tea in a supermarket (Hall et al. 2010), but I found no recent direct replication of the face study itself, and the quotes in the video are the paper's sample sentences. The AI studies tested models from 2023 (GPT-3.5 and Claude 1.0; five assistants including GPT-4 and Claude 1.3), all since replaced, and these papers did not measure today's models. In the always-A test, GPT-3.5 lost 18.7 points of accuracy and Claude 1.0 only 4.7; the hand check that found no explanation mentioning the trick covered a sample (0 of 234, from the always-A and a hint test). The sycophancy authors say it is "likely driven in part" by human feedback, not proven to be the cause. The guessing argument is an argument about how models are scored, peer-reviewed in Nature (three of its four authors work or worked at OpenAI), not a measured cause; the thesis-title example is from their 2025 preprint, run on chatbots in May 2025. People are not always wrong about their own reasons: a 2025 study found people often accurate about how they chose. The checks I suggest at the end are my own advice built on these findings, not a tested method. CHAPTERS 0:00 The reasons we make up 0:39 Four identical pairs 1:32 The swapped photo 2:24 The answer is always A 3:43 Are you sure? 4:59 Guessing on the test 6:00 Check the answer, not the story 6:49 This voice, too SOURCES Nisbett, R. E. & Wilson, T. D. (1977). Telling more than we can know: Verbal reports on mental processes. Psychological Review, 84(3), 231-259. https://doi.org/10.1037/0033-295X.84.... Johansson, P., Hall, L., Sikström, S. & Olsson, A. (2005). Failure to detect mismatches between intention and outcome in a simple decision task. Science, 310(5745), 116-119. https://doi.org/10.1126/science.1111709 Hall, L., Johansson, P., Tärning, B., Sikström, S. & Deutgen, T. (2010). Magic at the marketplace: Choice blindness for the taste of jam and the smell of tea. Cognition, 117(1), 54-61. https://doi.org/10.1016/j.cognition.2... Turpin, M., Michael, J., Perez, E. & Bowman, S. R. (2023). Language models don't always say what they think: Unfaithful explanations in chain-of-thought prompting. NeurIPS 2023. https://arxiv.org/abs/2305.04388 Sharma, M. et al. (2024). Towards understanding sycophancy in language models. ICLR 2024. https://arxiv.org/abs/2310.13548 Kalai, A. T., Nachum, O., Vempala, S. S. & Zhang, E. (2025). Why language models hallucinate. arXiv preprint. https://arxiv.org/abs/2509.04664 Kalai, A. T., Nachum, O., Vempala, S. S. & Zhang, E. (2026). Evaluating large language models for accuracy incentivizes hallucinations. Nature, 653, 1047-1051. https://doi.org/10.1038/s41586-026-10... Introspective access to value-based multi-attribute choice processes (2025). Nature Communications. https://doi.org/10.1038/s41467-025-59... (the counterweight: people often know how they chose) Next: The Johari Window, the blind spot everyone else can see:    • The Johari Window: What Everyone Sees Abou...   #psychology #ai #chatgpt