Can an AI Know When It's Wrong? | Calibration, Explained Visually

A network says it is 90% sure. Over ten predictions, it is right six times. The gap between the number a model reports and how often it is actually right is called calibration — and closing that gap is not the same as teaching a model to know when it is wrong. We build the picture from the last layer up. What softmax actually computes, and why the base of the exponential is a temperature dial that changes every stated confidence without moving a single answer. How a reliability diagram is made out of a thousand predictions, and how the area between it and the diagonal becomes one number. Why deeper, more accurate, longer-trained networks came out less honest than the shallower ones before them. And the one scalar, fitted on held-out data, that straightens the curve while leaving every classification exactly where it was. Then the part that is not bookkeeping. Feed a calibrated classifier something it has never seen — a rotated seven, a letter, static — and it is still sure, because it has no bin for "this is not a digit". Train five copies from five random starts: they agree on familiar ground and scatter on strange ground. That spread is a number a single network never had, and it is a cheap way of carrying a distribution over weights instead of one point. Last: whether any of this survives in the models you actually use, before and after training on human preference. CHAPTERS 00:00 the same answer, a different confidence 01:00 where the number comes from 03:55 measuring honesty 06:26 why training makes it sure 09:02 one dial 10:52 a stranger 12:53 five random starts 15:17 the glass atlas 19:18 when a model says it is sure SOURCES Guo, Pleiss, Sun, Weinberger — "On Calibration of Modern Neural Networks", ICML 2017. https://arxiv.org/abs/1706.04599 The measured claim is for ResNets and DenseNets on CIFAR and ImageNet; the paper attributes the effect to several causes (depth, width, weight decay, batch norm), not to the loss alone. Lakshminarayanan, Pritzel, Blundell — "Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles", NeurIPS 2017. https://arxiv.org/abs/1612.01474 Gal, Ghahramani — "Dropout as a Bayesian Approximation", ICML 2016. https://arxiv.org/abs/1506.02142 OpenAI — GPT-4 Technical Report (2023), Figure 8. https://cdn.openai.com/papers/gpt-4.pdf One model, one benchmark (MMLU), pre-training vs post-RLHF — not a universal claim about preference training. Lin, Hilton, Evans — "Teaching Models to Express Their Uncertainty in Words". https://arxiv.org/abs/2305.14975 What was simplified: one classification head, one classifier, one dataset. The language-model counts shown are a toy example. Animated with Manim Community Edition.