40 Lines of Code vs a Neural Network: Who Plays Krakout Better?

I tried to teach an AI to play Krakout, the 1987 Commodore 64 breakout game, from nothing but the screen and the score. Four ways to learn a game: trial and error, search, copying, and rules. And 40 lines of hand-written code embarrassed all of them, until the neural network finally learned to dodge the aliens. Final results, the same start states and the same randomness for both: Levels 1-4 with three bats: the AI cleared 550 of 570 tries, the 40-line rule 497 Levels 3-4 with one bat: 84 of 180 against 24 10 full games: the AI got past level 4 in 8 of them; its best game reached level 35 with 219,100 points So can an AI beat Krakout, all 100 levels? Not yet. Well, not on a Mac with an M3 Max processor and about seven hours of practice. Next game, and the next method: you choose. The poll is in the pinned comment. CHAPTERS 0:00 Two clips, one question 0:25 Why Krakout 1:09 How an AI plays a game 1:48 Inside the neural network 3:01 Finding the score in memory 4:38 Four ways to learn a game 5:21 Trial and error (PPO) 7:10 Search (Go-Explore) 9:00 A 40-line rule, and a network that copies it 10:25 The whole game: stuck on levels 3 and 4 12:43 The showdown 13:13 Keep training: the zap byte 13:52 My bug 14:21 Don't drop the bat 14:55 The student outgrows its teacher 15:35 Final results: level 35 16:36 So, can an AI beat Krakout? 17:31 What's next PAPERS DQN / Nature CNN: Mnih et al., "Human-level control through deep reinforcement learning", Nature 2015 PPO: Schulman et al., "Proximal Policy Optimization Algorithms", 2017 Go-Explore: Ecoffet et al., 2019; "First return, then explore", Nature 2021 Backward curriculum: Salimans & Chen, "Learning Montezuma's Revenge from a Single Demonstration", 2018 DAgger: Ross, Gordon & Bagnell, "A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning", AISTATS 2011 Agent57: Badia et al., "Agent57: Outperforming the Atari Human Benchmark", 2020 #Commodore64 #C64 #ReinforcementLearning #MachineLearning #RetroGaming