push singh Push Singh

In Memory
(1972 - 2006)

Honoring Push

Background | Research | Publications | Activities | Random


Until recently, I was a Ph.D. student in the department of EECS at MIT. I am presently a Postdoctoral Associate at the MIT Media Lab, and I'll be joining the Media Lab faculty in 2007. My research is focused on finding ways to give computers human-like common sense, the ability to think about the everyday world like people do. I believe this will enable a new generation of computing systems that will be much more powerful and friendly than those based on present-day technologies.

I am actively pushing a project at the Media Lab to develop programs capable of commonsense thinking. This is a very hands-on effort to build a suite of commonsense knowledge bases, inference engines, and architectural elements for linking these together, as well as new kinds of applications built on these technologies. These systems use multiple representations including semantic networks, propositional and first-order probabilistic graphical models, case bases of story scripts, rule based systems, logical axioms, shape descriptions, and even English sentences. For more details about this effort please visit the Media Lab's Commonsense Computing web page.

My long-term goal is to understand how minds work, so that I can construct a machine that thinks. No small task, but I do have the advantage of an amazing mentor, the redoubtable Marvin Minsky.


I am always working on many things. Here are some projects that are particularly far along in their development. Many of these are collaborations with others, both inside and outside the Media Lab, including Barbara Barry, Walter Bender, Tim Chklovski, Nathan Eagle, Ian Eslick, Jose Espinosa, Henry Lieberman, Hugo Liu, Ashwani Kumar, Erik Mueller, Marvin Minsky, Bo Morgan, Alex Pentland, Dustin Smith, and Ryan Williams.

Commonsense AI Architecture.
scenario involving physical, social,
and mental reasoning

(click image to watch movie)
I am building a layered cognitive architecture that supports commonsense reasoning about scenarios involving the physical, social, and mental domains. I am developing and testing this architecture in a physically realistic virtual world consisting a pair of simulated 'people' that collaborate together on concrete tasks like building chairs and tables from their components. This project is funded under NSF IIS grant #0450770.

Open Mind. I built the Open Mind Common Sense web site to acquire a variety of commonsense knowledge from the general public. It has received a large amount of attention, having been mentioned in the Boston Globe, CNN, Discover, Time, Wired, the New York Times, and many other places. This article describes the philosophy behind the project. A recent snapshot of the corpus of commonsense facts we collected is available for download. Several new Open Mind web sites have been developed since, but so far these have all been research prototypes that we have not publically launched (e.g. Open Mind Experiences, and the LifeNet and StoryNet acquisition systems.)

ConceptNet. ConceptNet is a large-scale semantic network (over 1.6 million links) relating a wide variety of ordinary objects, events, places, actions, and goals by 20 different link types, mined from the OMCS corpus. ConceptNet is freely available for download here. It comes with a part-of-speech tagger and chunker, and a spreading activation inference engine. It has been been used by a number of projects at MIT and elsewhere.

LifeNet. LifeNet is the first large-scale probabilistic graphical model (400,000 links) focused on commonsense knowledge. LifeNet is based on a "first-person" propositional representation over which inference is performed by probabilistic belief propagation. The goal of LifeNet is to provide a substrate for commonsense reasoning about a person's context, by providing an ontology and knowledge base focused on aspects of their situation such as where they are, what they are doing, what they just did and what they might do next. More information about LifeNet is available on Bo Morgan's web page.

StoryNet. StoryNet is a database of story scripts we are presently developing. We are using several representations, based on (a) natural language story templates composed from plot units, (b) chains of first person propositions, and (c) scenarios in terms of a Cyc-like upper level ontology. I am presently developing a case-based reasoning engine to support reasoning with the StoryNet database.

ShapeNet. ShapeNet is a database of 50,000 object shapes that are linked to the concept nodes in ConceptNet. ShapeNet includes a matching system that supports shape retrieval by several different similarity metrics against a query shape. We are using ShapeNet to study how to use commonsense knowledge in vision tasks.

Experiential Knowledge Acquisition. We are developing algorithms for extending and evaluating our commonsense knowledge bases (in particular, LifeNet) using location and other sensor data about people's real-world experience. We have also developed tools to sense a person's context using noisy speech data. See Nathan Eagle's reality mining page for more information about this project.

Terascale Knowledge Acquisition. We are developing web mining frameworks to acquire large quantities of commonsense and other types of knowledge from the web. More information about this effort is available on Ian Eslick's web page.

Commonsense-Enabled Applications. We are developing a large number of application prototypes using these core reasoning and knowledge technologies. The conventional wisdom is that the commonsense reasoning problem needs to be fully solved before we can begin to incorporate common sense into applications -- but we have begun to show that even with partial knowledge bases and imperfect inference, there are many useful roles common sense can play. More information about many of these applications is available on Henry Lieberman's web page.

Theories of Commonsense Thinking. In addition to these and other concrete systems, I also spend time on more theoretical questions whose answers would help us build systems with common sense. I tend to divide these problems along the lines of knowledge, representation, reasoning, and architecture. I am concerned ways to acquire large-scale commonsense knowledge bases, in ways to represent important but relatively little studied types of commonsense knowledge such as negative expertise, in ways to reason with vague, ambiguous, and otherwise problematic units of commonsense knowledge, and in ways to arrange these kinds of resources into larger architectures that do more than any simple method possibly could.


New! My PhD Thesis (June 2005): EM-ONE: An Architecture for Reflective Commonsense Thinking (html) (pdf)

Journals, Magazines, Conferences, Workshops

Push Singh and Marvin Minsky (2005). An architecture for cognitive diversity. Visions of Mind, Darryl Davis (ed.), London: Idea Group Inc. (html)

Alex Pentland, Tanzeem Choudhury, Nathan Eagle, and Push Singh (2005). Human dynamics: computation for organizations. Pattern Recognition Letters, 26:503-511.

Ryan Williams, Barbara Barry, and Push Singh (2005). ComicKit: acquiring story scripts using commonsense feedback. Proceedings of the ACM International Conference on Intelligent User Interfaces (IUI 2005). San Diego, CA.

Henry Lieberman, Hugo Liu, Push Singh, and Barbara Barry (2004). Beating common sense into interactive applications. AI Magazine, Winter 2004, 25(4):63-76. AAAI Press. Argues that it is now possible to build commonsense-reasoning-enabled applications, with a number of examples.

Push Singh, Marvin Minsky, and Ian Eslick (2004). Computing commonsense. BT Technology Journal, 22(4):201-210.

Push Singh, Barbara Barry, and Hugo Liu (2004). Teaching machines about everyday life. BT Technology Journal, 22(4):227-240. Reviews several commonsense reasoning systems we are building at the lab--ConceptNet, LifeNet, and StoryNet.

Hugo Liu and Push Singh (2004). ConceptNet: a practical commonsense reasoning toolkit. BT Technology Journal, 22(4):211-226. Elaborates on ConceptNet.

Hugo Liu and Push Singh (2004). Commonsense reasoning in and over natural language. Proceedings of the 8th International Conference on Knowledge-Based Intelligent Information & Engineering Systems (KES-2004). Describes ConceptNet, a semantic network we mined out of the Open Mind Common Sense corpus, and its associated toolkit for making inferences.

Nathan Eagle and Push Singh (2004). Context sensing using speech and common sense. To appear in Proceedings of the NAACL/HLT 2004 workshop on Higher-Level Linguistic and Other Knowledge for Automatic Speech Processing. Describes two systems, GISTER and OVERHEAR, that infer aspects of a person's context from noisy transcriptions of their speech by using the ConceptNet and LifeNet commonsense knowledge bases.

Marvin Minsky, Push Singh, and Aaron Sloman (2004). The St. Thomas common sense symposium: designing architectures for human-level intelligence. AI Magazine, Summer 2004, 25(2):113-124. Summarizes the results of a meeting that Marvin Minsky, Aaron Sloman, and I organized to get feedback on a proposal we are developing to start a new large-scale common sense project.

Push Singh (2003). Examining the Society of Mind. Computing and Informatics, 22(5):521-543. An article that examines Marvin Minsky's Society of Mind theory. It briefly describes the history of the theory, explains some of its essential components, and relates it to recent developments in Artificial Intelligence. (html)

Push Singh and William Williams (2003). LifeNet: a propositional model of ordinary human activity. Proceedings of the Workshop on Distributed and Collaborative Knowledge Capture (DC-KCAP) at K-CAP 2003. Sanibel Island, FL. Describes a new common sense knowledge base we built from the Open Mind Common Sense corpus, one based on a "first-person" propositional representation over which statistical inference is performed.

Push Singh and Marvin Minsky (2003). An architecture for combining ways to think. Proceedings of the International Conference on Knowledge Intensive Multi-Agent Systems. Cambridge, MA. Describes a cognitive architecture supporting multiple "ways to think" each suited for a particular type of problem.

Push Singh and Barbara Barry (2003). Collecting commonsense experiences. Proceedings of the Second International Conference on Knowledge Capture (K-CAP 2003). Sanibel Island, FL. Describes our most recent Open Mind project, "Open Mind Experiences", a web site designed to collect stories and story explanations from the general public. (This site has not yet been launched.)

Nathan Eagle, Push Singh, and Alex (Sandy) Pentland (2003). Common sense conversations: understanding casual conversation using a common sense database. Proceedings of the Artificial Intelligence, Information Access, and Mobile Computing Workshop (IJCAI 2003). Acapulco, Mexico. Describes a system we built to guess the fine-grained topic of a spoken conversation given noisy speech transcriptions, location information, and common-sense.

Push Singh (2003). A preliminary collection of reflective critics for layered agent architectures. Proceedings of the Safe Agents Workshop (AAMAS 2003). Melbourne, Australia. Describes a class of agents concerned with noticing recent problems in the deliberations of a layered commonsense reasoning system. (html)

John McCarthy, Marvin Minsky, Aaron Sloman, Leiguang Gong, Tessa Lau, Leora Morgenstern, Erik T. Mueller, Doug Riecken, and Moninder Singh, and Push Singh (2002). An architecture of diversity for commonsense reasoning. IBM Systems Journal, 41(3):530-539. Summarizes the results of a meeting we held at IBM in March 2002 to promote interest in 'commonsense computing' at IBM Research.

Push Singh, Thomas Lin, Erik T. Mueller, Grace Lim, Travell Perkins and Wan Li Zhu (2002). Open Mind Common Sense: Knowledge acquisition from the general public. Proceedings of the First International Conference on Ontologies, Databases, and Applications of Semantics for Large Scale Information Systems. Irvine, CA. Describes the data we collected in the original Open Mind Common Sense web site, and describes the design of a second-generation version focusing more on templates and a more structured way of doing knowledge acquisition.

Push Singh (2002). The public acquisition of commonsense knowledge.Proceedings of AAAI Spring Symposium on Acquiring (and Using) Linguistic (and World) Knowledge for Information Access.Palo Alto, CA. Describes the Open Mind Common Sense project, and argues that English itself could be used as an underlying representation for commonsense reasoning. It also describes a simple web search engine application that uses commonsense knowledge to improve search queries.

Push Singh (2001). The Open Mind Common Sense project. KurzweilAI.net. A high level discussion of the goals and philosophy behind the Open Mind Common Sense project.

Henry Lieberman, Elizabeth Rosenzweig, and Push Singh (2001). Aria: An agent for annotating and retrieving images. IEEE Computer, July 2001, pp. 57-61. Describes a new kind of photo annotation and retrieval application. It was later extended by Hugo Liu to incorporate some commonsense reasoning into the retrieval process.

Push Singh (1998). How to build a human-level intelligence. Proceedings of Symposium on Abstraction, Reformulation, and Approximation (SARA-98). Pacific Grove, CA. Short position paper. Argues that an architecture for commonsense reasoning should be designed to use a multiplicity of representations and not just one.

MIT related

Push Singh (2003). Reaching for dexterous manipulation. EECS Area Exam Paper. An "area exam" paper that examines three papers on dexterous manipulation by robots, and suggests some new avenues we might go down.

Push Singh (2002). The Panalogy Architecture for Commonsense Computing. Ph.D. Thesis Proposal.

Push Singh (1998). Failure-directed reformulation.M. Eng thesis. My masters thesis.I suggested it was useful to precompile knowledge about how to change representations in order to quickly response to impasses during problem solving. It was during this time I first began thinking about how to make commonsense reasoning systems that could use many kinds of representations and methods to solve problems.

Push Singh (1997). M. Eng. Thesis Proposal. My masters thesis proposal. I didn't quite end up doing quite what I proposed to do! But it's still interesting to look at.


Push Singh (2003). The Panalogy architecture for commonsense computing - Brief Description. Report for the Institute for Defense Analysis. A brief summary of the commonsense computing architecture we are developing.

Push Singh (2003). Structural critics for commonsense knowledge bases. Describes a few types of critics useful for noticing the kinds of bugs that show up in commonsense knowledge bases like Open Mind. (html)

Push Singh and Marvin Minsky (2002). A ten year roadmap to machines with common sense. Report for the National Science Foundation. A brief document written for a friend at the National Science Foundation, sketching a large-scale plan to build machines with common sense.

Erik Mueller and Push Singh (2002). A commonsense bibliography. A growing bibliography of important papers in the area of commonsense computing.We put this together for the commonsense reading group we established in Spring 2002.

Push Singh (1999). Acquiring common sense over the Internet. Some of my thoughts as I was beginning the Open Mind project, anticipating certain types of objections one might have to such an approach.

Push Singh (1999). Mental agents for commonsense thinking. A list of commonsense mental processes that I made in order to emphasize that no simple algorithm or architecture would be adequate for the procedural side of commonsense thinking.

Push Singh (1999). A unified theory of non-unified theories of cognition. Some arguments for building heterogeneous cognitive architectures, followed by some criticisms of the Soar unified cognitive architecture.


Teaching. I have assisted Marvin Minsky with his course The Society of Mind for many years now. This has meant reading, grading, and occasionally lecturing. I have also lectured in other courses at MIT including courses on commonsense reasoning and knowledge based systems (some slides I used recently are available here.)

Symposia. I organized and co-chaired the Distributed and Collaborative Knowledge Capture Workshop with Tim Chklovski at this year's K-CAP meeting. Last year I organized the St. Thomas Commonsense Symposium, held in the U.S. Virgin Islands.

Commonsense Reading Group. I co-founded (with Stefan Marti and Barbara Barry) the MIT Commonsense Reading Group, in which students from the MIT Media Lab and the MIT AI Lab present and discuss papers relating to the topic of commonsense. We ran the group in the spring and fall of 2002.

Past work

As an undergraduate I worked on a variety of electronics and AI related projects. I worked with Mitchel Resnick on designs for the 6.270 control board and built a radio tranceiver so that two Lego bricks could talk to each other at a distance; with Pattie Maes on connectionist algorithms for planning in the blocks world with propositional strips operators and learning those operators through experience, and also on new types of genetic algorithms that learned over time better methods of mutation and crossover; with Rod Brooks on building motor control boards for the Cog humanoid robot; and with Marvin Minsky on various perceptual problems including sound recognition based on qualitative features of the spectrogram and object recognition based on recognizing their contours, on clustering algorithms, and on a natural language semantic parser.


Thoughts about the past 10 years in AI (written in 1996), with a comment from Bill Gates.

Comments on the idea that we each have a Self, for the students in The Society of Mind course.

A theory of beauty.