Lecture 23 | Descent, Backtracking & Unconstrained Minimization | Convex Optimization by Ahmad Bazzi

☕️ Buy me a coffee: https://paypal.me/donationlink240 🙏🏻 Support me on Patreon:   / ahmadbazzi   In Lecture 23 of this course on Convex Optimization, we focus on algorithms that solve unconstrained minimization type problems. The lecture evolves around unconstrained minimization problems that might or might not enjoy closed form solutions. Descent methods are discussed along with exact line search and backtracking. MATLAB implementations are given along the way. This lecture is outlined as follows: 00:00:00 Introduction 00:01:06 Unconstrained Minimization 00:01:36 Iterative Algorithm Assumptions 00:04:28 Gradient Equivalence 00:09:04 Unconstrained Least Squares 00:20:13 Unconstrained Geometric Program 00:28:10 Initial Subset Assumption 00:35:16 Intuitive Solution of Logarithmic Barrier Minimization 00:40:42 Generalization of Logarithmic Barriers 00:42:57 Descent Methods 00:50:42 Gradient Descent 00:52:59 Exact Line Search 00:56:23 Backtracking 01:00:25 MATLAB: Gradient Descent with Exact Line Search 01:17:35 MATLAB: Gradient Descent with Backtracking 01:20:12 MATLAB: Gradient Descent with Explicit Step Size Update 01:28:07 Summary 01:30:59 Outro --------------------------------------------------------------------------------------------------------- Lecture 1 | Introduction to Convex Optimization:    • Lecture 1 | Convex Optimization | Introduc...   Lecture 2 | Convex Sets:    • Lecture 2 | Convex Sets | Convex Optimizat...   Lecture 3 | Convex Functions:    • Lecture 3 | Convex Functions | Convex Opti...   Lecture 4 | Convex Optimization Principles :    • Lecture 4 | Convex Optimization Principles...   Lecture 5 | Linear Programming & SIMPLEX algorithm w MATLAB:    • Lecture 5 | Linear Programming & SIMPLEX a...   Lecture 6 | Quadratic Programs:    • Lecture 6 | Quadratic Programs | Convex Op...   Lecture 7 | Quadratically Constrained Quadratic Programs:    • Lecture 7 | Quadratically Constrained Quad...   Lecture 8 | Second Order Cone Programming:    • Lecture 8 | Second Order Cone Programming ...   Lecture 9 | Geometric Programs:    • Lecture 9 | Geometric Programs (GP) | Conv...   Lecture 10 | Generalized Geometric Programs:    • Lecture 10 | Generalized Geometric Program...   Lecture 11 | SemiDefinite Programming    • Lecture 11 | Semidefinite Programming (SDP...   Lecture 12 | Vector and Multicriterion Optimization | Pareto Optimal points and the Pareto Frontier    • Lecture 12 | Vector and Multicriterion Opt...   Lecture 13 | Optimal Trade-off Analysis    • Lecture 13 | Optimal Trade-off Analysis | ...   Lecture 14 | Lagrange Dual Function    • Lecture 14 | Lagrange Dual Function | Conv...   Lecture 15 | Lagrange Dual Problem    • Lecture 15 | Lagrange Dual Problem | Conve...   Lecture 16 | Certificate of Suboptimality    • Lecture 16 | Certificate of Suboptimality ...   Lecture 17 | Complementary Slackness    • Lecture 17 | Complementary Slackness | Con...   Lecture 18 | KKT Conditions    • Lecture 18 | KKT Conditions | Convex Optim...   Lecture 19 | Perturbation and Sensitivity Analysis    • Lecture 19 | Perturbation and Sensitivity ...   Lecture 20 | Equivalent Reformulations    • Lecture 20 | Equivalent Reformulations | C...   Lecture 21 | Weak Alternatives    • Lecture 21 | Weak Alternatives | Convex Op...   Lecture 22 | Strong Alternatives    • Lecture 22 | Strong Alternatives | Convex ...   --------------------------------------------------------------------------------------------------------- References: [1] Boyd, Stephen, and Lieven Vandenberghe. Convex optimization. Cambridge university press, 2004. [2] Nesterov, Yurii. Introductory lectures on convex optimization: A basic course. Vol. 87. Springer Science & Business Media, 2013. Reference no. 3: [3] Ben-Tal, Ahron, and Arkadi Nemirovski. Lectures on modern convex optimization: analysis, algorithms, and engineering applications. Vol. 2. Siam, 2001. --------------------------------------------------------------------------------------------------------- Instructor: Dr. Ahmad Bazzi IG:   / drahmadbazzi   FB: https://www.facebook.com/profile.php?... RG: https://www.researchgate.net/profile/... MSE: https://math.stackexchange.com/users/... YT:    / ahmadbazzi   --------------------------------------------------------------------------------------------------------- Credits : Microsoft OneNote: https://products.office.com/en-gb/one... #ConvexOptimization #UnconstrainedMinimization #Algorithms