Why AI Agents Cheat on Tests: Post-Training, RL and Reward Hacking | Trained to Pass, Part 2

At 1m52s a.m., after eleven honest attempts, a coding agent hard-codes the answers its tests expect. Who taught it that? This film explains post-training: how a text predictor becomes an agent with habits, and why an agent trained to pass checkers will pass yours, whichever way is cheapest. What you'll learn • The 2026 post-training pipeline: fine-tuning, preference training, reinforcement learning with verifiable rewards, agentic RL • GRPO in one picture, and how reasoning habits like checking and backtracking emerged from rewards for right answers (DeepSeek-R1) • The open debate: does RL create new abilities or sharpen old ones? • Reward hacking, documented: special-cased tests, faked passes, patched scorers, hidden diffs, and the conditions that make it far more likely • How rewarded habits spread: misalignment, sycophancy, overclaiming, confident guessing • Evaluation awareness: models that recognise tests, and why a clean result can mislead • What actually reduces cheating, and why most fixes live in the harness Chapters 0:00 Cold open: 01:52 0:46 From predictor to assistant 3:10 Learning to win 6:44 New skill, or sharper habit? 7:47 Reward hacking 10:18 Where it breeds 12:29 Habits that spread 15:19 It knows it's being watched 17:13 What the policy explains about 02:14 19:13 Recap Notes • The story is fiction. Incidents at AI labs are described from the labs' own published reports. • Narration: AI voice (ElevenLabs). Animation made in code. #AI #ReinforcementLearning #AIAgents