Loss Functions and Gradient Descent Explained Visually, With Numbers

How does a machine actually learn? It rolls downhill. In this video we build loss functions and gradient descent from scratch, with every number worked out by hand. We compare MSE and MAE on a tiny dataset, derive the gradient descent update rule step by step, and animate four learning rates on contour plots and 3D surfaces: convergence, slow crawling, overshooting and divergence. Then we look at convex and non-convex landscapes, local minima and saddle points, stochastic and mini-batch gradient descent, momentum, and an intro to Adam, and finish by implementing gradient descent in NumPy. This is episode 11 of the Machine Learning from Zero to Advanced series. Subscribe so you do not miss the next episode on Python and data foundations. 0:00 Intro 0:40 Objective, Loss and Cost 1:24 Parameters vs Hyperparameters 2:04 MSE vs MAE by Hand 4:48 Deriving Gradient Descent 6:49 The Learning Rate 8:52 Landscapes: Convex and Non-Convex 10:26 Stochastic and Mini-Batch Descent 11:37 Momentum and Adam 13:26 Implementing in NumPy 15:43 Common Mistakes and Debugging 16:34 Recap and Next Episode #MachineLearning #GradientDescent #LossFunction #DeepLearning #Optimization