10-753 Deep Learning and GenAI Theory
12 units, Carnegie Mellon University
This course studies the theoretical foundations of modern deep learning and generative artificial intelligence. While deep neural networks and generative models have achieved remarkable empirical success across many domains, many of the theoretical principles underlying their behavior are still being actively developed. The goal of this course is to provide a rigorous mathematical framework for understanding these methods and analyzing their theoretical properties. The course will cover theoretical perspectives on deep neural networks, including expressivity, approximation theory, optimization landscapes, and generalization in overparameterized models. We will study the role of gradient-based optimization, stochastic dynamics, and convergence properties of large neural networks. We will also examine modern phenomena observed in deep learning systems, such as double descent, implicit regularization, and grokking. A central focus of the course will be the theory of generative models. Topics will include generative adversarial networks, score-based generative models, diffusion models, and flow-based models. We will analyze these models from probabilistic, optimization, and stochastic-process perspectives, connecting them to concepts such as optimal transport, Langevin dynamics, stochastic differential equations, Fokker-Planck equation, Schrödinger bridges, and energy-based models. The course will also examine emerging theoretical perspectives on foundation models and large-scale generative systems, including representation learning, self-supervised learning, and scaling laws. The course is intended for PhD students and advanced master's students interested in the mathematical and algorithmic foundations of modern deep learning and generative AI.
Prerequisites: none listed
Offered: Fall 2026
Courseweave needs JavaScript to show sections, workloads and planning tools.