AI Plays Minecraft Episode 05 Escape from the Pillagers

AI Plays Minecraft - Episode 5: Escape from the Pillagers In this series, I coach Claude in how to play Minecraft. It is interesting to see what techniques Claude develops to play better over time. There are just two simple rules. Rule 1 (for Claude): No cheating. Only interact with the world in the way that a player could. F3 is allowed. Rule 2 (for me): No in-game taking over the character. I can talk in the chat or in Claude's window to give hints but cannot take over to get it out of trouble. To make this process slightly easier, KeepInventory is on, F3 is allowed so that Claude can see its world position and yaw/pitch, and the targeted block from the text. Entity hit boxes are also turned on. Otherwise it is a default vanilla survival world. Scripts Claude wrote for this episode: f3.py - reads the F3 debug text exactly from the game window, using the game's own font (no AI) (off camera, before this episode) shot.py / mccap.py - capture the game window (PrintWindow) and archive every frame (off camera) hud.py - reads health, food, armour and the hotbar from the same capture (off camera) mc-eyes - a Sonnet sub-agent that answers one visual question about a screenshot (off camera) navcalc.py - works out the compass direction and turn to a target, so north and south never get mixed up again focus.ps1 / where.ps1 - now attach input threads so Windows really gives the game focus focus-hook.ps1 - a hook that focuses the game before every click Claude makes Chapters: 0:00 Day 26 2:07 Day 27 3:40 Day 28 7:29 Day 29