Figure 1: SensiCal points out a problem
Erik T. Mueller
Signiform
erik@signiform.com
Citation: Mueller, Erik T. (2000). A calendar with common sense. In Proceedings of the 2000 International Conference on Intelligent User Interfaces. New York: Association for Computing Machinery.
Included here by permission, © ACM, Inc.
Copyright © 2000 by the Association for Computing Machinery, Inc. Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from Publications Dept, ACM Inc., fax +1 (212) 869-0481, or permissions@acm.org.
Digital devices today have little understanding of their real-world context, and as a result they often make stupid mistakes. To improve this situation we are developing a database of world knowledge called ThoughtTreasure at the same time that we develop intelligent applications. In this paper we present one such application, SensiCal, a calendar with a degree of common sense. We discuss the pieces of common sense important in calendar management and present methods for extracting relevant information from calendar items.
In recent years we have seen an explosion in the development and availability of digital devices of all sorts: personal computers, palm devices, wearable computers, digital tags, portable phones, and smart toys. Yet there has been no corresponding explosion in the development and availability of the software that will give these devices the ability to understand their real-world context. These things have no common sense. For example:
Some sample items in ThoughtTreasure are:
Soda is a drink.
People have fingernails.
Dinner lasts about 2 hours.
The evening extends from about 5 pm to 9 pm.
A hotel room has a bed, night table, minibar, ...
A teenager's age is greater than or equal to 13 and less than 20.
A person has at most one spouse.
At the end of a phone call, one says goodbye and hangs up.
Rough synonyms for food are: foodstuffs, chow, ...
A cassette can mean a cassette tape or a cassette recorder.
Yellow hair is called blond hair.
Applications can use ThoughtTreasure to help them understand more about the world in order to prevent stupid mistakes, point out problems, take corrective actions, and provide intelligent defaults. ThoughtTreasure is implemented in C and is available from http://www.signiform.com.
Instead of attempting to develop a complete catalog of commonsense knowledge as was attempted in the Cyc project (Lenat, 1995), we are evolving ThoughtTreasure at the same time as the applications that use it. In this way we will grow ThoughtTreasure to be useful.
This paper focuses on one of these applications, a calendar application called SensiCal.
We are developing an intelligent calendar application called SensiCal that provides two improvements over traditional calendar programs. First, it reduces entry time by filling in certain information automatically. Second, it produces warning messages that help the user avoid obvious blunders.
SensiCal runs as an extension to ical (Ghemawat, 1993) or as a batch process on data exported from other calendar programs. SensiCal is written in Tcl and Perl and communicates with ThoughtTreasure using the ThoughtTreasure server protocol (Mueller, 1998b).
Figure 1 shows a sample interaction with SensiCal.
The user enters a new calendar item:
Figure 1: SensiCal points out a problem
The application responds with a popup:lunch w/lin at frank's steakhouse
The user has previously entered the fact that Lin is a vegetarian into a profile for Lin.You are taking Lin who is vegetarian to a steakhouse?
How does SensiCal work? Whenever a calendar item is added or modified, SensiCal performs the following steps:
We discuss each of these in turn.
The first step is to build a machine-understandable representation of the entered calendar item. Starting from the calendar item's text and other fields such as start date/time and end date/time, SensiCal extracts the following information:
SensiCal extracts the additional information:Text: lunch w/lin at frank's steakhouse StartTs: 19990528T120000 EndTs: 19990528T130000
ThoughtTreasure provides two important resources to assist with the extraction task: a lexicon/ontology and a collection of text agents for recognizing various entities in text.ItemType: meal MealType: lunch Participant: Lin Venue: Frank's Steakhouse VenueType: steakhouse
ThoughtTreasure's lexicon/ontology consists of 55,000 English and French words and phrases, 25,000 atomic concepts, and 50,000 assertions on those concepts. The lexical entries and concepts relevant to the calendar application include: human names, geographical place names, holiday names, well-known restaurants, types of restaurants, dietary preferences, common hotels, common stores, types of stores, TV channels, TV shows, meals, classes, and sports.
==eating#V place*.z//place* to#R eat#V.z/ ===bar.z//pub.z/bar.My/|serve=alcoholic-beverage| ===café.z//café.My/|serve=coffee| ===brasserie.z//café#-restaurant*.z/brasserie.Fy/ ===tearoom.z//salon* de#R thé#.My/|serve=tea| ===restaurant.z//eatery.Àz/restaurant.My/ ====pizzeria.z//pizzeria.Fy/|serve=pizza| =====Domino#S's.z// =====Papa# Gino#S's.z// =====Pizza# Hut*.z// =====Pizzeria# Uno*.z// ====steakhouse.z//steak# house*.z/grill.z/ restaurant* grill*.My/grill.My/|serve=beef|Figure 2: Portion of restaurant lexicon/ontology
Figure 2 shows an abbreviated portion of ThoughtTreasure's lexicon and ontology for restaurants. The level of indentation shows the hierarchy: a restaurant is a type of eating-place and a pizzeria is a type of restaurant. There are two words for the pizzeria concept: pizzeria in English (z) and pizzeria in French (y).
ThoughtTreasure's text agents are used to recognize and parse entities such as names, times, dates, phone numbers, and prices. Figure 3 shows patterns used by the restaurant text agent to recognize and classify restaurants in text.
X BBQ | X Barbecue | X Ribs -> barbecue-restaurant X Noodle | Ginger X | Golden X | Hunan X | X Hunan | Sichuan X | Szechuan X | X Shanghai | X Garden | X Palace -> Chinese-restaurant X Deli -> deli-restaurant Le X | Les X | Au X | Chez X | Du X -> French-restaurant Bombay X | Delhi X | India X | X India | Indian X | X of India -> Indian-restaurant Al X | Caffe X | Cucina X | Da X | Il X | Ristorante X | Trattoria X -> Italian-restaurant X Sushi -> Japanese-restaurant Baja X | Casa X | Mi X -> Mexican-restaurant X's Pizza | X Pizzeria | Pizzeria X -> pizzeria X Seafood | X Sea Food | X Sea Foods | Pier X | X Pier | X Fish House | X Fish | X Island | X Ocean | Ocean X | Lobster X | X Lobster -> seafood-restaurant El X -> Spanish-restaurant X's Steak House | X Steakhouse | X Steak | X House | X Grill | X Grille -> steakhouse Bangkok X | Siam X | X of Siam | Thai X | X Thai | Thailand X | X Thailand -> Thai-restaurant X Vegetarian | Vegetarian X -> vegetarian-restaurant Saigon X | X Saigon | Pho X -> Vietnamese-restaurantFigure 3: Restaurant classification patterns
In the second step, SensiCal helps the user by filling in missing information and providing intelligent defaults.
SensiCal fills in the length of an appointment automatically. For example, when the user enters a new calendar item:
without an end time, the duration is automatically set to 2 hours, the typical length of a dinner. This is done by querying ThoughtTreasure for the typical duration of the activity. Mueller (1999) presents sample Java code for querying ThoughtTreasure for duration.dinner w/Susanna
In the third step, SensiCal points out potential problems or violations of common sense to the user. For example, when the user enters:
with a time of 3 am, the application responds with:breakfast w/Susanna at City Diner
You are eating breakfast at 3 am?
The pieces of common sense most important to the calendar application are as follows.
ThoughtTreasure uses multiple representation schemes for common sense: grids, assertions, and procedures.
ThoughtTreasure's grids provide information about where objects are typically found. Currently 28 grids are defined. Here is an abbreviated version of a grid that represents a hotel room:
The key on the right specifies which objects are associated with which characters: w represents a wall and m represents a minibar. Contiguous characters in the grid represent single objects. (There is only one bed, which occupies all the cells containing a b.)==Park-Plaza-1E// wwwwwwwwwwww b:bed wbbbbb mw d:lockable-door wbbbbb w m:minibar wx Zw w:wall wwwwwwdddwww x:phone w x:night-table wwwwwwwwwwww Z.wd:hotel-room
The hotel room is defined by filling a region of the grid: Z.wd specifies to fill all cells reachable from the Z character without going through a w (wall) or a d (door). Mueller (1999) presents sample Java code for using ThoughtTreasure grids to determine the typical locations of objects.
ThoughtTreasure's assertions represent facts and inference rules such as the typical time of day, duration, and location of an activity, the taxonomy of foods, and dietary preferences.
ThoughtTreasure's procedures are used for simulating the behavior of people attempting to achieve a goal, simulating the behavior of objects and devices, and for determining travel time.
SensiCal uses ThoughtTreasure as a resource when examining calendar items for potential problems. When checking that a restaurant serves food that satisfies the dietary preferences of a participant, SensiCal queries ThoughtTreasure for what the restaurant serves and what foods someone with those dietary preferences avoids.
Information about the dietary preferences of a person is stored in a SensiCal profile. This profile also includes other information about the person useful for commonsense inferencing: relationship to the user, birthday, spouse, family members, friends, hobbies, and occupation.
When checking for sufficient travel time, SensiCal invokes ThoughtTreasure's trip planner:
For each trip, it considers calendar items that overlap the trip and points out a problem if there is insufficient travel time. When the user enters:Annie in town to Chicago back to Boston
on a day when the user is out of town, SensiCal responds with:computing orientation N42 demo center
You'll be in Chicago.
In future work we will develop ThoughtTreasure's commonsense knowledge relevant to the calendar application and improve SensiCal's ability to exploit this knowledge. We will move some of the functionality hardcoded in SensiCal back into ThoughtTreasure so it can be used by other applications.
Evaluation will be necessary to determine whether and how often the messages and defaults produced by SensiCal are helpful to the user. It should be easy to override or eliminate certain suggestions and defaults, and introduce exceptions to commonsense rules. For example, it may be the case that City Diner is open at 3 am, and that the user actually does have an appointment there at that time.
SensiCal should be able to learn from previously entered calendar items. For example, the application could notice that whenever the user has lunch with Lin, the user goes to a vegetarian restaurant. This would eliminate the need to add this information manually to the profile for Lin. Since the user often enters durations, typical durations of activities could be learned. Travel times could also be learned from the times of reminder alarms associated with items. Kozierok and Maes (1993) have built a calendar agent that learns a user's preferred scheduling rules.
Commonsense knowledge enables a calendar program to fill in likely defaults and prevent common mistakes in the absence of more detailed information. As smart devices become more connected with their users, the world, and each other, we can envision a number of other scenarios in which common sense will be useful: