Embedded AI Podcast E05: Context Management

In this episode, we tackle one of the most practical challenges in AI-assisted development: managing context windows. Ryan and Luca explain what tokens are, why context windows matter, and share hard-won lessons about keeping LLMs productive without overwhelming them. Turns out LLMs behave surprisingly like humans - remembering the beginning and end of conversations better than the middle, getting confused with too much information, and sometimes defaulting to outdated solutions. We discuss practical techniques like context compaction, using agents for side tasks, pulling in just the right files, and leveraging tools like RAG and Model Context Protocol. The key takeaway: be a better product manager. Break problems into manageable chunks, stay deliberate about what context you provide, and don't hesitate to start fresh when things get messy. 0:00 Welcome to the Podcast 1:00 What Are Tokens 4:04 Context Windows Are Working Memory 5:42 Too Much Context Causes Hallucinations 9:24 Beginning and End Matter Most 13:27 Compacting to Free Up Context 18:53 Using Agents for Side Tasks 23:18 Pulling in Files and Documents 26:47 Temporary Files for Context Management 32:41 RAG and Model Context Protocol 36:18 Knowledge Cutoff and Outdated APIs 39:59 Log Files and Visual Inputs 43:01 Context Management Best Practices