Does Aadam Optimizer still need a learning rate schedule?

Every deep learning model faces the same decision at every single step: how big a step to take. Too large, and training explodes. Too small, and it stalls before reaching the answer. This is the learning rate problem — and even Adam, the optimizer many assume "handles it automatically," doesn't fully solve it. In this video, we trace the learning rate from its 1951 origins (Robbins-Monro) to the schedules powering models like GPT-3 today: 00:00 Why the size of each step matters more than you think 01:55 The 70-year-old formula quietly running modern AI (Cauchy to Robbins-Monro) 04:18 The Adam myth: why it still needs a learning rate schedule 09:34 The real enemy in high-dimensional training: saddle points 14:13 How schedules evolved - Step Decay, Cosine Annealing, Warmup, and GPT-3's approach 19:34 The hard limits: what no learning rate schedule can fix 22:41 Putting it all together - and what comes next with learned optimizers If you've ever wondered why training a model still takes careful tuning even with modern optimizers, this breaks down exactly what's happening under the hood. Watch list:    • Gradient descent, Mini-batches & The noise...   Algorithms, computer science, and AI - explained visually, without the fluff. New videos every week. A null pointer points to nothing. This channel points to everything worth understanding about how code actually works. Nullpoint breaks down algorithms, data structures, computer science, and AI into clear, visual explanations you'll actually remember. From sorting and Big-O to neural networks and the ideas behind modern AI - we skip the buzzwords and get to how things really work under the hood. #DeepLearning #MachineLearning #NeuralNetworks #Adam #AI #nullpoint