10-808 Language Grounding to Vision & Control
12 units, Carnegie Mellon University
The course will study the question how can we understand what language really means, how not to form realistically looking sentences but rather deliver the right meaning needed for a task. Thus language will be studied as a means of 1 communication 2 weak supervision for learning execution graphs of vision or control routines and the inverse, how other modalities can provide supervision towards language grounding. We will cover topics on visual question answering, language pragmatics, program induction, automatic theorem proving, language models for parsing, translation, grounded conversation models. The format of the course is the following that at each lecture there will be some part which is formal presentation of background and preliminaries and then paper discussion. There will not be homeworks, but grading will be based on a participation and b the completion of a relevant final project. Prerequisites Familiarity with Computer Vision, Deep learning and basic reinforcement learning is assumed. Familiarity with computational linguistics is not necessary.
Prerequisites: none listed
Courseweave needs JavaScript to show sections, workloads and planning tools.