An agent with subagents, from a GPT we built | Minimum Viable GPT #21

The series: https://mvgpt.nenuadrian.com The code: https://github.com/nenuadrian/mvGPT Episode 21 of a series that builds mvGPT, a minimum viable GPT, from scratch and takes it through the training pipeline of a language model. Last episode, our model learned to call a calculator. This time it becomes an agent: one model, fine-tuned once into three roles. The agent writes and lists files and hands the rest of a task to subagents, the math role with the calculator and the files role, which reads, writes and adds up files. Each subagent is the same model in a fresh context, and only its answer goes back. The files live in a sandbox whose limits the program enforces, whatever the model asks. We generate sixty thousand traces, fine-tune for thirty minutes on the laptop, measure each role on held-out tasks, and compare the agent with subagents against one context on a long task: right more often, with a far smaller context, for more tokens in all. Then we put it in a Gradio chat that shows each step. Chapters: 0:00 Intro 2:13 Agents and subagents 4:38 One model, three roles 9:18 Sixty thousand traces 11:12 Fine-tuning 13:47 Measuring the agent 16:40 What subagents buy 17:59 The agent in a chat 20:53 Wrap-up To train on GPUs you rent by the hour, cheaply, I recommend Vast.ai (referral link): https://cloud.vast.ai/?ref_id=746308 Music: John Tramp (https://pixabay.com/users/tramp963-35...)