MIT Media Lab Course - Fall 2004 Common Sense Reasoning for Interactive Applications MAS 960(H) 0-9-0     Henry Lieberman, Research Scientist TA: Ashwani Kumar Mondays, 1-3 PM Room 335, MIT Media Lab, 20 Ames St., Cambridge

Things fall down, not up. Weddings have a bride and a groom (sometimes). If people yell at you, they're probably angry. One of the reasons that computers seem dumber than humans is that they don't have common sense -- a myriad of simple facts about everyday life and the ability to make use of that knowledge easily when appropriate.

A long-standing dream of artificial intelligence has been to put that kind of knowledge into computers, but it has proven slow and difficult. But considerable progress has been made over the last few years. There are now large knowledge bases of common sense knowledge and better ways of using it then we have had before. We may have gotten too used to putting common sense in that category of "impossible" problems and overlooked opportunities to actually put this kind of knowledge to work. We need to explore new interface designs that don't require complete solutions to the common sense problem, but can make good use of partial knowledge and human-computer collaboration.

As the complexity of computer applications grows, it may be that the only way to make applications more helpful and avoid stupid mistakes and annoying interruptions is to make use of common sense knowledge. Cell phones should know enough not to ring during the concert. Calendars should warn you if you try to schedule a meeting at 2 AM or plan to take a vegetarian to a steak house. Cameras should realize that if you took a group of pictures within a span of two hours, they are probably all of the same event.

This course will explore the state of the art in common sense knowledge, and class projects will design and build interfaces that can exploit this knowledge to make more usable and helpful interfaces.

Course requirements will consist of critiques of class readings [about 2 papers/week], and a final project [paper or computer implementation project]. Grades will be based primarily on the projects, as well as a small component for class and online participation. Class limited to 25 students; permission of instructor required if demand exceeds that.

---

Course Links:

• Web links for Resources about Common Sense.
• The ConceptNet Common Sense Semantic Network

Course News:

4 October 2004

This week's readings are:

ConceptNet: A Practical Commonsense Reasoning Toolkit.

The Emotion Machine, Chapter 6

27 September 2004

This week's readings are:

ThoughtTreasure , by Eric Mueller

A Calendar with Common Sense , by Eric Mueller

20 September 2004

This week's readings are:

Beating Some Common Sense into Interactive Applications, Henry Lieberman, Hugo Liu, Push Singh and Barbara Barry
(also, if you have time, check out the other articles on this page).

CYC: A Large-Scale Investment in Knowledge Infrastructure, Douglas B. Lenat, Communications of the ACM (pdf).

Upcoming schedule items:

4 October: Guest lecture from Eric Mueller (creator of ThoughtTreasure).

18 October: No class because of Media Lab Hell Week

25 October: Guest lecture TBA

13 September 2004

Welcome, everybody! Thanks for attending today's session.

If you are interested in taking the course, but did not attend the first session, please send mail to ashwani@media.mit.edu and attend the next session.

The course mailing list is commonsense-course@media.mit.edu. You will be automatically signed up if you signed the list at the first session. Let us know if you drop the course or wish to be removed from the list.

This week's readings are

Marvin Minsky, Commonsense Based Interfaces, Communications of the ACM (pdf)

Push Singh, et, al., Open Mind Common Sense: Knowledge Acquisition from the General Public, AAAI-2002 (pdf).

This week's assignment is:

Register for the Open Mind site (http://openmind.media.mit.edu/), and play around with it a bit after reading the Open Mind paper.

Pick a simple consumer electronics device that you own (e.g. cellphone, MP3 player, toaster, blender, television; best if you just bought it recently and don't know it well).

Write down a simple episode of your use of the device, step by step. Include steps where you make mistakes and what you do to correct them.

Example: I want to check my messages on my answering machine. I pressed PLAY. I listened to the first message. It was a long message asking me to support a candidate in the election, so I didn't listen to the whole thing and pressed ERASE. It played the next message. The next message was from a friend about going to see a movie. I listened to the whole thing, and the machine saved the message.

Imagine you are trying to explain the whole process to someone who isn't familiar with one (a small child, an alien who just arrived from Mars). Avoid the kind of details specific to the particular device, such as those you might find in the user manual. Write down some of the Common Sense background knowledge you would need to understand what is going on, in the style of Open Mind. Example:

I want to check the messages on my answering machine	An answering machine saves messages from phone calls when you are not able to answer them. You can ask an answering machine to play back messages that arrived when you were gone.
I pressed PLAY. I listened...	An answering machine will play back messages in the order they arrived and you can listen to each one and choose to save or delete it. Spam is when people who want to sell you something will send you e-mail or call you, even if they don't know you personally

Enter this knowledge into Open Mind. How much of it was already there? How much of what Open Mind knows about the situation is wrong or irrelevant?

How hard was it for you to come up with this knowledge?