What is Gradient Descent | Machine Learning Basics

Gradient Descent (GD) is one of the most important optimisation techniques in machine learning, and it plays a central role in making model training efficient. While it is rarely used in its simplest, original form, its core idea underpins many of the methods used to train frontier models across the field. In this video, we take a look at GD and why it works so well when other optimisation methods become intractable. We start with the basic idea, then explore some examples, the considerations involved, and the method's limitations. 📌 Timestamps: 0:00 Introduction 0:46 Problem Setting 1:56 Difficulty of Optimisation 4:22 Basic Idea of GD 8:23 The Update Rule 10:20 Applying GD 12:40 Different Learning Rates 15:16 Limitations of GD