We are establishing an informal weekly reading group to discuss techniques for giving machines the capacity for human-like commonsense reasoning and ways to incorporate common sense into real-world applications. Topics will include:
The meetings will be held Tuesdays from 4-5:30 in the Garden Conference Room on the 3rd floor of the Media Lab. The room might be a bit hard to find for those of you who have never been to Media Lab before, so here is a link with a map (red circle).
We have collected a large bibliography of papers on topics related to the
problem of giving machines common sense. It is available at:
http://www.media.mit.edu/~push/CommonsenseBibliography.html
For the readings we will draw from this bibliography.
If you have any other questions about this reading group, please contact Push Sing, or Barbara Barry, or Stefan Marti.
Push: To begin with we are planning to read these two papers. The first is a classic by John McCarthy that was the first paper on logical AI, a popular way of representing commonsense knowledge. The second is a recent paper by Marvin Minsky that summarizes some of his current ideas about how to tackle the common sense problem, such as by developing architectures that can exploit many different kinds of representations and methods of reasoning.
McCarthy, John (1959). Programs with common sense.
http://www-formal.stanford.edu/jmc/mcc59.html
Minsky, Marvin (2000). Commonsense-based interfaces. Communications of the ACM, 43(8), 67-73.
http://www.acm.org/pubs/citations/journals/cacm/2000-43-8/p66-minsky/
http://commonsense.media.mit.edu/minsky.pdf
Push: Thanks to everyone who came! I thought it was a great discussion, with many different and interesting perspectives. Here are some of the questions people raised:
Let's take a more focused look at Cyc. This first article is a decade old but still a good review of what is under the hood of that project.
Guha, Ramanathan, & Lenat, Douglas (1990). Cyc: A midterm report. AI Magazine, 11(3), 32-59.
http://www.media.mit.edu/~push/cyc-midterm.pdf
This second article is a brief criticism of Cyc, which reflects some problems in Cyc that were there seven years ago. The article provides a valuable outsider perspective and some suggestions for how to evaluate Cyc:
http://boole.stanford.edu/cyc.html
If you want to play with a live version of Cyc, circa 1998, I set up a server at:
http://taffy.media.mit.edu/cgi-bin/cyccgi/cg?menu
There is also extensive on-line documentation at:
http://taffy.media.mit.edu/cycdoc/toc.html
This version has only a fraction of the knowledge of the full system but it has the full upper level ontology and enough specific knowledge to get a feel for the system. This is a kind of preview of the upcoming OpenCyc distribution.
http://www.opencyc.org
Push: We only scratched the surface today of what's under the hood of Cyc and why it does things the way it does, but I hope it was enough to give people the general idea. Keep your eyes open for OpenCyc (http://www.opencyc.org).
A few things about OpenCyc:
I ran across this CNN article today about Cyc. Looks like they are
definitely moving towards an Open Mind approach:
http://www.cnn.com/2002/TECH/industry/04/11/memome.project.idg/index.html
I'll be seeing Lenat this weekend so I'll have a chance to query him in person about it.
Peter Gorniak: Some people on the (potential) speaker list for the Harvard seminar I mentioned:
Susan Carey; Ray S Jackendoff; Marc Hauser; Lera Boroditsky; Josh Tenenbaum; Jesse Snedeker; James Pustejovsky; Elizabeth Spelke; Deborah Kelemen; Deb Roy; Daniel Simons; Alec Marantz; Fei Xu; Paul Harris
The page for this course is http://www.courses.fas.harvard.edu/~psy2210/ and contains info about location and times.
I mentioned Lera Boroditsky's work. She's at http://web.mit.edu/bcs/boroditsky.html.
Push: This week I'll talk about Open Mind Common Sense. There is a general overview of the project (and some high-level observations about common sense) at:
Singh, Push (2002). The Open Mind Common Sense project.
http://openmind.media.mit.edu/Kurzweil.htm
This paper describes the first version of the system:
Singh, Push (2002). The public acquisition of commonsense knowledge. Proceedings of AAAI Spring Symposium on Acquiring (and Using) Linguistic (and World) Knowledge for Information Access. Palo Alto, CA: AAAI.
http://www.media.mit.edu/~push/OMCS-1.pdf
This second paper describe the newer system that is under development:
Singh, Push, et al. (in submission). Open Mind Common Sense: Knowledge acquisition from the general public.
http://www.media.mit.edu/~push/OMCS-2.pdf
The system itself is running at:
http://openmind.media.mit.edu
Push: We are meeting today to discuss "commonsense in applications". Henry Lieberman and Barbara Barry will talk about their recent work in this area.
Henry Lieberman: For this week's Common Sense group, I will lead discussion on Applications. Here's an abstract of what I'll present:
Beating Some Common Sense Into Interactive Applications. Henry Lieberman, Kim Waters, Hugo Liu.
What kinds of interactive applications are the best candidates for applying "common sense" knowledge? Applications like question answering and general story understanding are tough, because if the system doesn't do as good a job as a human, the users will not find the application successful. However, there are many other situations in interactive interfaces where the user interface is vastly underconstrained, and almost any kind of common sense help from the system would be welcome. The key is to cast the machine in a fail-soft role of giving suggestions or assistance rather than try to completely replace human judgment.
I'll illustrate this with a description of Aria (Annotation & Retrieval Integration Agent), an agent for helping users tell stories with digital photographs. Aria proactively looks for opportunities for annotating and retrieving photographs in the context of a user's ordinary work, such as composing e-mail messages. Common sense knowledge about picture-taking situations such as family events and vacation travel can substantially improve Aria's performance without the danger of raising user expectations to unreasonable levels.
Please read:
An Agent for Integrated Annotation and Retrieval of Images. Henry Lieberman, Elizabeth Rozenzweig, Push Singh.
http://lieber.www.media.mit.edu/people/lieber/Lieberary/Aria/Aria-Intro.html
A Calendar with Common Sense. Erik Mueller.
http://lieber.www.media.mit.edu/people/lieber/IUI/Mueller/Mueller.html
Push: Thanks to Henry and Barbara for a very interesting presentation! They are really pioneering this area of putting some commonsense reasoning into applications. As far as I know there has not been much prior work besides Erik Mueller's and some small demos by Cycorp. It's just very hard to do this kind of work without some sort of commonsense knowledgebase to draw on.
This week Martin has offered to lead a discussion on critiquing what has come to be called the "classical AI" approach to natural language processing and commonsense reasoning. The reading will be chapters 5 and 9 from Winograd and Flores:
Terry Winograd and Fernando Flores: Understanding Computers and Cognition: A New Foundation for Design Addison-Wesley, 1987 (I'm afraid we don't have those readings on-line.)
Push: Thanks Martin! I thought that was a great discussion.
I think the conclusion is that common sense is still possible :-), but we need to clarify and develop this idea that all knowledge is contextualized in many ways, and it is rarely true that a small item of knowledge can totally stand alone. Henry has offer to lead a discussion later in the semester on this subject, for which there is a actually quite a vast literature (see http://www.eecs.umich.edu/~rthomaso/bibs/context.bib.txt).
Push: This week's topic will be: Symbolic vs Connectionist and other architectural issues in common sense. Jake has offered to lead the discussion, at least for the first hour. The papers will include the following one, and pieces of two others that Jake will send more information about later.
Minsky, Marvin (1991). Logical vs. analogical or symbolic vs. connectionist or neat vs. scruffy. AI Magazine, Summer 1991.
http://web.media.mit.edu/~minsky/papers/SymbolicVs.Connectionist.html
This is a beautiful discussion of the advantages and disadvantages of different types of representations, by a central figure in the debate—I guess you could say Marvin started off as a connectionist (he built the first neural network, and his thesis was the first on the topic), then he became a powerful developer of symbolic methods (his students built some the first symbolic and analogical heuristic reasoning systems) and critic of simple connectionist methods (including writing Perceptrons), and later he began to develop theories that tried to combine the advantages of both and other approaches (the Society of Mind).
Jake Beal: Push has asked me to lead our discussion this week: the theme will be architectural approaches to common sense.
Idea: Common sense is often treated as an abstract entity—a bloc of explicit knowledge to be added to a system to give it common sense. The architectural approach holds that we should instead build systems in which common sense is implicit in the computational structures we build to represent the external world.
Background Reading: I've included three papers for you to look at: a background paper by Minsky from 1990, and papers from two current projects at the AI lab taking an architectural approach to the problem.
Minsky's paper on Symbolic vs. Connectionist AI
http://web.media.mit.edu/~minsky/papers/SymbolicVs.Connectionist.html
Bob Hearn's master's thesis Building Grounded Abstractions for Artificial Intelligence Programming
http://www.swiss.ai.mit.edu/~bob/grounded_abst.pdf
"20 Ways to Contribute to the Bridge Project"—a working
document from the Genesis Group
http://www.ai.mit.edu/projects/genesis/projects.html
Bob's thesis is rather large, so just make sure you've read the Introduction and the sections on Polynemes and Frames.
Push: Thanks Jake for leading last meeting! I encourage everyone to take a look at the genesis home page which describes those ideas in greater detail—of finding ways to build common sense on top of more specific cognitive machinery evolved for visual inference, motor control, social reasoning, and many of the other abilities we share with the other mammals.
Jimmy Lin: Commonsense: field trip this coming week
When: This coming Tuesday, 4:00-5:30pm (usual Commonsense Seminar meeting time)
Where: Harvard Coop (basement, children's book department)
What: We will be reading and discussing children's stories, with the following guiding questions:
The literature that I would like to target is books for barely-literature children (beyond picture books, but just a little.) The reason is that I would like to isolate commonsense problems from NLP issues, i.e., I would like to examine sentences that our parsers would have no problem parsing, so as to not get bogged down in NL issues like alternations, complex parallel constructions, etc.
Suggest Readings:
Erik Mueller. Prospects for in-depth story understanding by computer.
http://web.media.mit.edu/~mueller/papers/storyund.html
Wendy G. Lenhert (1981). A Computational Theory of Human Question Answering. In Aravind K. Joshi and Bonnie L. Webber and Ivan A. Sag, eds., Elements of Discourse Understanding. Cambridge University Press. pp. 145-176 (I don't have an electronic copy... those of you in the AI Lab, please stop by my office 810. Those of you in the Media Lab, I'll try to get Push some physical copies.)
Lynette Hirschman, Marc Light, and Eric Breck and John Burger (1999).
Deep Read: A Reading Comprehension System. Proceedings of the 37th
Annual Meeting of the Association for Computational Linguistics (ACL'99).
http://www.ai.mit.edu/people/jimmylin/papers/Hirschman99.pdf
Come one, come all - if not anything, it'll be great fun reading children's stories!
Push: Thanks Jimmy for leading the discussion today! It's very illuminating looking at actual children's stories, as it makes you realize how sophisticated is even a first graders mind.
Henry Lieberman: Letter with subject: How I spent my Tuesday vacation
Dear Girls and Boys,
Once upon a time, there was a group of girls and boys called Common Sense. They went to spend an afternoon at the Harvard Coop Bookstore's Children's Basement.
Here are some of the things they learned:
We discovered that in many cases understanding the stories was not so simple as in the "Would Mary like a kite?" examples.
Tracy noted that understanding the pictures was often a key. Sometimes the text was not even understandable without the pictures. In stories for young children, where the story forms are simpler, the idea is to get children to associate the text with the pictures so they need to see the pictures to see what it is about.
Several members noted that the purpose of various stories differed. Some were meant to teach common sense.
Others had a "moral" and were meant to teach ethics and social situations.
The story goes on to show what the tree gave the girl throughout her life as she grows up—apples, branches, shade. Some common sense knowledge about plot, morals of stories, social situations is necessary to understand such stories. Does CYC have the requisite knowledge?
Some stories are meant to be funny, so intentionally show things that are incongruous.
Some stories are meant to be fantasy, so show things like talking animals. How do you get the idea that animals can talk in stories, but not in real life?
Push found a question-and-answer flip-card pack that asked questions about stories like this.
This is at the 2nd grade level. Such questions always ask about exactly what is in the story. By 4th grade, they ask for the implicit knowledge not mentioned in the story.
We analyzed what knowledge is necessary for a simple story.
Animals sleep. When an animal has not been sleeping for a long time, it becomes sleepy.
To sleep, you need to find a place that is quiet and comfortable.
Henry suggested that when knowledge is put into a CS KB in order to enable understanding of a particular story, the system ought to keep the association between the knowledge and the particular stories it was learned from. CYC does not do this. That way, when you have to use the piece of knowledge in the future, you could map it back to the story and try to do an analogy between the stories.
The Common Sense boys and girls did not get milk and cookies, but they did get crepes. And they lived happily ever after.
Your friend,
Henry
Mary Hegarty (2001). Capacity Limits in Mechanical Reasoning. Fifteenth International Workshop on Qualitative Reasoning AAAI. San Antonio, Texas. May 17-19, 2001.
http://www.vuse.vanderbilt.edu/~qr-01/Hegarty.htm
Aaron Sloman - Sloman, Aaron (2001). Diagrams in the mind. In M. Anderson, B. Meyer, & P. Olivier (Eds.), Diagrammatic Representation and Reasoning. London: Springer-Verlag.
http://www.cs.bham.ac.uk/research/cogaff/sloman.diagbook.ps
http://www.cs.bham.ac.uk/research/cogaff/sloman.diagbook.pdf
Push: Thanks Tracy and Olya for presenting today! Too bad we didn't get to the Sloman paper but I recommend you all read it if you haven't done so, since it's filled with nice observations about spatial reasoning.
Today Peter will talk about evidence for the Lakoff-ian idea that our understanding of time is built by analogy to our understanding of space.
Peter Gorniak: At this Commonsense AI meeting, we'll talk about mapping between different domains of reasoning. I believe this to be an absolutely essential issue for any commonsense reasoning system with the flexibility and generativity of human reasoning.
Let's talk about:
I'll initiate with a short overview of the papers below and some of my current thoughts on these questions.
Boroditsky, L. (2000). Metaphoric Structuring: Understanding time through spatial metaphors. Cognition, 75(1), 1-28
http://www.mit.edu/~lera/papers/metaphors.pdf
Jackendoff, R. (2002) Foundations of Language. The link below points to the second half of chapter 11. Just look at 11.7 and 11.8.
http://www.courses.fas.harvard.edu/~psy2210/Papers/Week-05/chap11_part2.pdf
Minsky, M. (1988) Society of Mind. I'm assuming people are familiar with this one.
Some of my thoughts on generativity in reasoning systems:
http://www.media.mit.edu/~pgorniak/generating_reasoning_systems.txt
Push: Thanks to Peter giving a great talk! We had a good discussion about the nature of "cognitive domains" and the correspondences that exist between them.
Stefan Marti will talk about the growing connection between Common Sense and the Semantic Web, as both require representing and reasoning about a wide range of things about the everyday world. Stefan will send out the readings later today or tomorrow.
Stefan Marti: Here are the suggested readings for next week (and some comments):
Berners-Lee, Tim (1998). Semantic Web Road map. Short (ca. 8 pages), early paper/draft from the "inventor" of the Semantic Web.
http://www.w3.org/DesignIssues/Semantic.html
Berners-Lee, Tim (1998). What the Semantic Web can represent. Even shorter (ca. 6 pages), still old, Q&A like, rather unpolished, and sometimes unnecessary details, but interesting because of its inverse approach: it's about what the Semantic Web is NOT.
http://www.w3.org/DesignIssues/RDFnot.html
Berners-Lee, Tim, Hendler, James, & Lassila, Ora (2001). The Semantic Web. Scientific American, 284(5), 34-43. A bit more polished. Not too deep.
http://www.sciam.com/print_version.cfm?articleID=00048144-10D2-1C70-84A9809EC588EF21
Koivunen, M.-R. & Miller, E. (2001). W3C Semantic Web Activity. Finally, a non-Berners-Lee paper; about the six main Principles of the Semantic Web. A rather fast read, I would guess.
http://www.w3.org/2001/12/semweb-fin/w3csw
If you have more time to read:
Fensel, Dieter, & Musen, Mark A. (Eds.). (2001). The semantic web. IEEE Intelligent Systems, 16(2), 24-79. A nice collection of very recent articles, some of them very in-depth. Worth browsing through!
Zipped file of separate articles: http://computer.org/intelligent/Is.zip
Pages 24-79 in one go: http://web.media.mit.edu/~stefanm/commonsense/SemanticWeb_IEEE2001.pdf
W3C website. And finally: all the gory details... Probably just interesting for the aficionado.
http://www.w3.org/2001/sw/
Stefan's presentation (powerpoint slides).
Push: Thanks to Stefan for a stimulating discussion!
Here are some pointers to work that was brought up:
The AI/LCS Haystack project home page. Includes pointers to information on
RDF, Notation3, and the Dublin Core Metadata Initiative.
http://haystack.lcs.mit.edu/
This year's KR conference.
http://www.irit.fr/kr02/
A dynamic prototype-based language with an XML syntax.
http://www.waterlang.org
Also, here is an announcement for a course being taught next fall by Benjamin Grosof about knowledge representation and the Semantic Web:
Announcing a new advanced info technology course in fall 2002 taught by Prof. Benjamin Grosof. Majority of the focus will be on AI knowledge representation, including the Semantic Web, and how to use them for e-business. Students will do a term paper which can be original research or literature survey.
Tim Chklovski: For the meeting on Tuesday, please take a look at:
My thesis proposal, off of http://www.media.mit.edu/~timc (the background and the chapter on the basic mechanisms are the most relevant parts)
For a closer look at using natural language ontologies to reason, take a look at: Enriching WordNet with Qualia Information, by Mendes and Chaves
http://citeseer.nj.nec.com/450623.html
For the discussion, I'd like to focus on alternatives to the full-blown logic approach.
There should also be a live demo of my "Learner" system that reasons over parsed natural language assertions collected by Push's Open Mind Common Sense, makes guesses about what else may be true, formulates questions about it, and asks you.
Push: Thanks Tim! I encourage people to visit Tim's learner site as soon as it is launched, to support his thesis work.
Push: This will be the final meeting of the semester. I thought I would take the opportunity to describe a project Marvin and I have been working to get started, to build an architecture for a commonsense thinking machine. We
arranged a remarkable symposium a few weeks ago, to discuss the idea with a number of other AI researchers who have worked in the commonsense area. A brief report about the symposium is available at:
http://www.media.mit.edu/~push/ResultsOfStThomas-Draft.pdf
You might also look at these slides by Aaron Sloman:
http://www.cs.bham.ac.uk/research/cogaff/manip/
Stefan: Thank you Push! This approach looks very promising. The symposium seems to reflect the increased interest in commonsense reasoning.
Push: I was thinking we should continue the commonsense reading group this semester, with more of a focus on specific methods of commonsense reasoning. The spring term was a good introduction to the topic for people, but this fall it would be nice to look more deeply at known techniques. Here's an initial list of topics:
