10-425 Introduction to Convex Optimization
12 units, Carnegie Mellon University
As machine learning grows in prominence, so also has optimization become a mainstay for machine learning, particularly techniques for convex optimization. Most learning problems are formulated as optimization of some objective function, sometimes subject to constraints. This course explores the optimization algorithms used to solve these machine learning problems. We characterize the properties of the optimization problems that enable these techniques to be efficient e.g. convexity, smoothness, linearity, separability as well as properties that inhibit efficient optimization e.g. nonconvexity. Core topics include first order methods gradient descent, subgradient methods, proximal and stochastic gradient descent, duality and linear programming, and secondorderquasiNewton methods. We also consider advanced techniques ranging from those that have spurred the growth of deep learning e.g. adaptive gradient methods, momentum and those that enable largescale distributed optimization. The course will focus both on theory and practical applications, frequently drawing motivation from examples in machine learning. The course is designed so that a machine learning ML course could be taken after or before this one ML is not a prerequisite. Students will gain the tools to both implement and analyze modern optimization techniques.
Prerequisites: (21-240 (min C) or 21-241 (min C)) and (21-325 (min C) or 36-219 (min C) or 36-217 (min C) or 15-359 (min C)) and (21-254 (min C) or 11-485 (min C) or 10-315 (min C) or 10-301 (min C) or 21-256 (min C) or 21-259 (min C)) and (15-151 (min C) or 21-127 (min C)) and 15-122 (min C)
Courseweave needs JavaScript to show sections, workloads and planning tools.