10-741 Representation Learning
12 units, Carnegie Mellon University
Representation Learning is a course that will cover both the theory and practice of representation learning. Till a decade ago, machine learning ML approaches fell into two broad categories predictive ML where we predict some output given input e.g. predicting cat or dog given an image, and generativedescriptive ML where we fit some model to the inputs themselves e.g. an image generative model that can generate new images. But over the past decade there has been a paradigm shift a new intermediate stage of representation learning, where we go from inputs to features that are vector representations of the input. The importance of this third and equally if not more important class of ML approaches can be seen from the name of one of the topthree conferences in ML the international conference on learning representations. The course will cover the theory as well as practice of modern representation learning how do we characterize good features in theory as well as practice How do modern deep neural model architectures, including recent ones such as transformers, perform representation learning What is selfsupervised learning, and how has it revolutionized representation learning How can we formally analyze existing approaches to learning features The course will be structured as a partial seminar course. Of the two classes each week, one class will be a traditional lecture on the emerging science of representation learning. The other class will be a studentled readingseminar on key papers, centering around the topic of that weeks lecture.
Prerequisites: 10-301 (min C) or 10-701 (min C) or 10-715 (min C) or 10-315 (min C) or 10-601 (min C)
Courseweave needs JavaScript to show sections, workloads and planning tools.