Trisk is a humanoid robot that integrates speech input, visual perception, and active touch in order to interact with humans and its environment. It can understand and obey natural language commands, and will soon be able to answer questions. The robot is a platform for designing new algorithms and multimodal knowledge representations for sensory-motor grounded language use.
The Human Speechome Project is an effort to observe and computationally model the longitudinal language development of a single child at an unprecedented scale. To achieve this, we are recording, storing, visualizing, and analyzing communication and behavior patterns in several hundred thousand hours of home video and speech recordings.
The quantity and availability of video content is soaring due to the combination of television networks and the Internet. The aim of this project is to develop more effective means to manage, search, and translate video content. We are developing algorithms that interpret language in video (speech and closed caption text) by exploiting aspects of the non-linguistic context, or situation, conveyed by the accompanying video.
