Concepts, Language, Embodiment, and Learning

MAS.964

Units: 0-12-0

Wednesdays 1-3pm

E15-054

This seminar will explore the role of embodiment and perception in the acquisition of concepts and language by studying and building computational models. Recent trends in cognitive science point towards human physiology as a key to understanding how humans develop conceptual knowledge. These trends have been complemented in the artificial intelligence community with a growing focus on the role of robotics and perceptual computing in developing knowledge representations. This seminar will be structured around (1) a set of case studies of computational/robotic efforts to build embodied communication machines, and (2) a group project aimed at building an embodied language learning system.

Instructor: Prof. Deb Roy (dkroy@media.mit.edu)

Administrative Assistant: Amy Sargent (asargent@media.mit.edu)

Format

Each week’s meeting will begin with a student presentation of a case study. Relevant papers for that study will have been distributed to the class in advance, and all members of the class will be expected to submit a one page summary of the model and its limitations on the Tuesday before each class. During class, one student will present the model and a second student will present a critique of the model. In the second part of each class, we will transition to discussions/progress reports on group projects. The final class(es) will be reserved for project presentations.

Grading

Case Study summaries 25%

Case Study presentation 25%

Project 50%

Some Suggested Case Studies

Winograd (1972)

Terry Winograd. Cognitive Psychology Volume 3 No 1, 1972, pp. 1-191.

Steven Harnad (1991)

Harnad, S., Hanson, S.J., & Lubin, J. (1991) Categorical Perception and the Evolution of Supervised Learning in Neural Nets. In: Working Papers of the AAAI Spring Symposium on Machine Learning of Natural Language and Ontology (DW Powers & L Reeker, Eds.) pp. 65-74.

Siskind (1992/2000)

Naive Physics, Event Perception, Lexical Semantics and Language Acquisition, Ph.D. thesis, Massachusetts Institute of Technology, January 1992.

Grounding the Lexical Semantics of Verbs in Visual Perception Using Force Dynamics and Event Logic, Technical Report 2000-105, NEC Research Institute, Inc., July 2000.

Lammens (1994)

Johan M. Lammens. A Computational Model of Color Perception and Color Naming. PhD thesis, Technical Report 94-26, Department of Computer Science, State University of New York at Buffalo, Buffalo, NY, June 1994. 253 pages.

Regier (1996)

T. Regier (1996). The Human Semantic Potential: Spatial Language and Constrained Connectionism, Cambridge, MA: MIT Press.

Bailey (1997)

D. Bailey (1997). When Push Comes to Shove: A Computational Model of the Role of Motor Control in the Acquisition of Action Verbs. Ph.D. Dissertation, Computer Science Division, University of California Berkeley, 1997.

Steels & Vogt (1997)

Steels, L. and P. Vogt (1997) Grounding adaptive language games in robotic agents. Physical implementation of language games and meaning creation. Submitted to ECAL 97. (http://arti.vub.ac.be/steels/publications.html).

Kronenberg & Kummert (2000)

Susanne Kronenberg and Franz Kummert. Generation of utterances based on visual context information.

In International Conference on Spoken Language Processing, volume 3, pages 1037-1040, Beijing, China, 2000.

Students may suggest other cases with consent of instructor.