11-783 Rich Interaction in Virtual World Lab: Self Paced
6 units, Carnegie Mellon University
Massively Multiplayer Online RolePlaying Games have evolved into Virtual Worlds VWs, and are creating ever richer environments for experimentation on all aspects of human to human, or human to machine communication, as well as for information discovery and access. So far, interaction has been constrained by the limited capabilities of keyboards, joysticks, or computer mice. This creates an exciting opportunity for explorative research on speech input and output, speechtospeech translation, or any aspect of language technology. Of particular interest will be a combination with other novel real world RW input, or output devices, such as mobile phones or portable games consoles, because they can be used to control the VW, or make it accessible everywhere in RW. Language technologies in particular profit from context awareness, because domain adaptation can be performed. For scientific experimentation in that area, Virtual Worlds offer the opportunity to concentrate on algorithms, because context sensors can be written with a few lines of code, without the need for extra hardware sensors. Algorithms can also run continuously, without the need for specific data collection times or places, because the VW is always on. In this lab, we will enhance existing clients to virtual worlds so that they can connect to various speech and language related research systems developed at LTI and CMUs Silicon Valley campus. The lab will be held jointly at the CMUs Pittsburgh and Silicon Valley Campuses. We will eat our own dog food, so the goal will be to hold the last session entirely in a virtual class room, which will by that time include speech control of virtual equipment, speechtospeech translation, and some devices that can be controlled using nonPC type equipment, like mobile phones. Eventually, this virtual room or the knowhow gathered
Prerequisites: 11-751 and 18-799 and 18-781
Courseweave needs JavaScript to show sections, workloads and planning tools.