How AI Actually Learns: Gradient Descent

Every AI model you have used, ChatGPT included, learned by repeating one simple rule millions of times: take a small step downhill. In 12 minutes, this lesson builds gradient descent from scratch, shows why the learning rate decides everything, trains a classifier and a language model by hand, and watches a quantum computer measure its own slope. CHAPTERS 0:00 How every AI model learns 0:18 Last lesson's slopes, put to work 1:12 The loss is a landscape 2:04 The update rule, and why the minus sign 3:19 Worked example: rolling into a bowl 4:10 The learning rate decides everything 5:04 AI: one training step on the cat picture 6:09 AI: learning from small batches (SGD) 6:54 LLMs: one step on "the cat sat on the…" 7:53 LLMs: momentum, Adam and learning-rate schedules 8:57 Quantum: training a quantum circuit 10:22 One idea: follow the slope downhill 11:13 Go deeper PREVIOUS LESSONS Lesson 1, The dot product: [paste the Lesson 1 link] Lesson 2, Softmax: [paste the Lesson 2 link] Lesson 3, Cross-entropy: [paste the Lesson 3 link] GO DEEPER: the Mathematics for Everything series Maths for Optimization (From Gradients to Solvers): https://www.amazon.com/dp/B0HLYFNG99 Maths for AI: https://www.amazon.com/dp/B0H7QM2BB6 Maths for LLM: https://www.amazon.com/dp/B0HFF1RKFW Maths for Quantum Computing: https://www.amazon.com/dp/B0H7XVWYXN Readers in the UK: https://www.amazon.co.uk/dp/B0HLYFNG99 (Optimization), https://www.amazon.co.uk/dp/B0H7QM2BB6 (AI), https://www.amazon.co.uk/dp/B0HFF1RKFW (LLM), https://www.amazon.co.uk/dp/B0H7XVWYXN (Quantum) The whole series: https://www.amazon.com/dp/B0H9G22MHF Printable poster packs from the books: https://biddutheiitian.gumroad.com NEXT LESSON: Backpropagation, how to get the slope for millions of knobs at once. I am Avishek Nag, Associate Professor of Computer Science at University College Dublin. https://avisheknag.one #MachineLearning #GradientDescent #ChatGPT