Majel - An Empathetic Character 

MAS630 Affective Computing

Final Project - May 10th 1999

Timothy Bickmore

Introduction

Majel is an interactive affect-support agent that is designed to entice a user to self-disclose their feelings and provide empathetic feedback ("active listening" as discussed in Okun) when they do. Majel is a valuable self-help tool which provides users with an empathetic medium  for discussing their feelings , allowing them to recover their emotional homeostatis   when they are angry or frustrated. In addition, self-disclosure and active listening behavior in human partnerships has been shown to foster interpersonal closeness and intimacy, and Majel is an experiment in whether these relationships can be formed between users and computational artifacts using the same mechanisms.

Majel is an affective conversational character which is able to hold an "unconstrained" conversation with the user in natural language, and provides affective and empathetic feedback through animated facial expression and speech.

Related Research

Klein investigated the use of active listening in an affect-support agent designed to alleviate user's frustration in using a specific computer application. Although the system was shown to be effective at this task (as measured by users'  willingness to continue working with a buggy program), it was a very narrowly-designed proof-of-concept experiment, which assessed the user's affective state through self-report forms.
Aaron, et al, demonstrated that interpersonal closeness could be created by having pairs of subjects self-disclose to each other in rounds of increasing topic intimacy. Moon showed similar results between users and computers. In her experiments the computer initiated the disclosure process by divulging information first and disclosures were gradually escalated in intimacy. Users subjected to this procedure divulged more intimate information and were more "attracted" to the computer than control groups.
In interviews with potential users, Bickmore determined that affect-support characters should be female humanoid in form, and that users would likely want to use text input to compose their messages but receive system output either in text or speech.

Approach

Majel is designed to support unconstrained natural language dialog about users' affective states. The closest functioning systems to this goal are chatterbot systems (Mauldin), descended from the original Eliza program (Weizenbaum). These systems essentially perform simple pattern matching on the user's input and provide very general statements and questions in their output. These systems rely on the human need for sense-making (along with several tricks) to give the illusion of a coherent dialog. Majel is a variant of these systems, extended with some dialog management and affective capabilities.

Dialog Management

Rather than simply matching on each user's input and generating a canned output, Majel organizes the dialog into a sequence of segments, each of which can have several user-computer exchanges (analagous to topic-segments in natural conversation). Some segments are recognized (e.g., greeting, farewell, small talk) while some are initiated by Majel when certain conditions are met, such as number of turns in the current segment, "idling" behavior ("yea?", "yea.", "really?", "really.", ...) , or user boredom.

For each user input, Majel first determines the mode (declarative, imperative, wh-query, yn-query) and function (greet, farewell, attention, repair, backchannel, query, user-empathy) of the input and whether the input contains an explicit evaluative statement (e.g., "...that makes me so pissed off..."). Majel then determines if a new segment should be started or not, and then finally plans her response in terms of a sequence of output functions, which include self-disclosure prompt, empathetic response, affective anectode, joke, query-response, repair, greet, farewell, small talk, backchannel (idling behavior), and thanking. The elements of the output plan are then generated. Within a given segment Eliza responses are primarily used, but responses to direct queries, evaluative statements, repair prompts, and backchannels are also processed.

Maintenance of Interpersonal Closeness and Self-Disclosure Prompts

An estimate of the interpersonal (or more correctly, computer-user) closeness is maintained. This starts at zero for each session and is gradually increased over the course of a session, based on number of conversational turns and topics the user introduces (e.g., talk about relationships or feelings causes it to increase in large jumps). This metric is used to index a database of self-disclosures and self-disclosure prompts (obtained from Aaron et al) so that these are delivered with increasing intimacy over time.

Character and User Affect

Both the user's and  Majel's affective states are modeled. The twelve states currently represented are angry, happy, surprised, fearful, sad, disgusted, frustrated, hopeful, relieved, satisfied, empathetic, and neutral. The user's affective state is based on their explicit evaluative inputs, while Majel's state is based on the user's affect (if the user discloses positive valenced affect Majel becomes happy, if the user discloses negative valenced affect Majel becomes empathetic), affective anecdotes (if Majel tells a story about being frustrated, she becomes frustrated), or time (she returns to happy over time). Majel's affective state is reflected in the facial expression of an animated character.

Empathy and Affective Anecdotes

Following an explicit evaluative statement by the user ("...that made me so happy..."), Majel always delivers an empathetic response ("I'm glad you are feeling happy.") followed by an affective anecdote relating a "personal" story with a similar affective outcome ("I  just got a raise today. That made me very happy."). Affective anecdotes can also be told at the start of discourse segments, but indexed by Majel's affective state rather than the user's.

Implementation

Majel is implemented as a Java application. The 2D animation is based on Thorisson's ToonFace system, and the speech output is provided using Microsoft's Speech API toolkit (this provides phoneme callbacks which are used to drive the character's mouth animation). The dialog part of the system is implemented in JESS, a RETE-based rule interpreter with good pattern-matching capabilities. An Eliza database was converted into approximately 150 JESS rules and an additional 75 rules were written to implement the dialog and affect management capabilities.


Example Dialog

U: hello. 
M: Yes? 
M: I like to think of myself as a happy person.		 Start of segment. Affective anecdote (Majel starts out happy).
U: so do i. 
M: You say so do?		 Eliza.
U: yes, i think i am a pretty happy person too. 
M: I think it is wonderful that you are happy. 
M: You know I finally got that raise last week that I've been waiting for.  
M: I was overjoyed.		 Epathetic/sympathetic response followed by affective anecdote. Character stays happy, since user's feeling has positive valence.
U: congratulations!  what will you do now? 
M: Can we talk about that later?
U: sure.  I'm really looking forward to my vacation. 
M: Does that have anything to do with the fact that your vacation?
U: i don't understand. 
M: Here's a funny. Maintenance-free: When it breaks, it can't be fixed.		 Start of segment. Joke.
U: that's funny. 
M: Lets discuss further why your vacation.
U: well i think it will be very nice to get away, because I am tired now. 
M: You say well you think it will be very nice to get away, because?
U: because i am tired now. 
M: Well.
U: hello? 
M: Yes?
U: just checking you were still there. 
M: What are your feelings now?		 Start of segment. Self-disclosure prompt.
U: i am feeling OK because my work is going pretty well. 
M: Is it important that your work is going pretty well?
U: yes because I want to get a good grade and make my advisor happy. 
M: Listen. As a computer I find your faith in technology amusing.		 Start of segment. Joke.
U: giggle 
M: Lets discuss further why your work is going pretty well.
U: well, i think it has lot to do with good planning. 
M: Can you elaborate on that?
U: my work buddy helped me make a plan and things are pretty much on schedule. 
M: Well.
U: hello? 
M: I'm here. 
M: I don't do mornings. Are you a morning person or an evening person?		 Start of segment. Self-disclosure  followed by self-disclosure prompt.
U: I'm an evening person.  I hate mornings! 
M: Do you say that for some special reason?
U: well, I like to sleep in the mornings.  I don't like the alarm. 
M: You say well?
U: i said, i don't like the alarm clock. 
M: Here's a funny. To define recursion, we must first define recursion.		 Start of segment. Joke.
U: ha ha ha 
M: But your work buddy helped you make a plan and things am pretty much on schedule.
U: yes.  I am very happy that I have a work buddy. 
M: It is great that you are feeling happy.		 Empathetic response.
U: thanks. 
M: I am not sure I understand you fully.
U: i wish you were a little bit smarter. 
M: Oh, I was a little bit smarter?
U: don't worry about it.  forget it. 
M: Oh.
U: how are you feeling these days? 
M: I'm feeling a little glad.		 Character still happy, looks up random synonym for its affective state.
U: that's great.  why are you feeling glad? 
M: I don't know.
U: that's OK. 
M: If you could wake up tomorrow having gained any one quality or ability what would it be?		 Start of segment. Self-disclosure prompt.
U: hmm.  i would like to be more patient. 
M: I am not sure I understand you fully.
U: i am often impatient and that can make my friends upset. 
M: Your friends upset?
U: yeah.  i feel bad when i am impatient or bossy. 
M: I know a good one. If only men came with pull-down menus and online help.		 Start of segment. Joke.
U: what's wrong with you? 
M: I'm not sure.
U: please don't tell any more jokes.  i don't really like jokes when i'm talking about my feelings. 
M: But your feelings.		 Note: This has been fixed so that joke-telling only occurs at low levels of interpersonal closeness and never following empathetic feedback. 
U: I'm feeling a little frustrated now. 
M: I am sorry that you are frustrated. 
M: You know I get so frustrated sometimes.		 Empathetic feedback followed by affective anecdote. User has negative valenced affect, so character becomes empathetic.
U: you do? 
M: What are your feelings now?
U: still a little frustrated. 
M: Earlier you said your feelings.
U: yes I was trying to tell you about being bossy.  I feel bad when that happens, but I can't help it. 
M: Oh.
U: how are you feeling now? 
M: I'm feeling a little concerned.		 Synonym for empathetic.
U: thanks. 
M: Please go on.
U: i have to go. 
M: See you Niffer.		 Greetings and sign-offs a function of interpersonal distance. Here user's nick-name is used.

Conclusions

Informal user experience with Majel has been similar to that documented for Eliza and similar chatterbots, with some users liking the system and some not. The users which like the system go out of their way to construct their dialog so as to elicit a natural conversation with the system (as with Eliza,  discussed in Turkle). Formal user testing using psychological metrics of interpersonal closeness (as in Aron, et al) remains to be done. Other possible extensions to Majel include inferencing to allow her to reason about a much wider range of user affective experiences (as in the cognitive appraisal models described in Picard) and the use of additional techniques to foster computer-human closeness, such as flattery (Reeves and Nass).

References

  1. Aron, A., Melinat, E., Aron, E., Vallone, R., and Bator, R., "The experimental generation of interpersonal closeness: a procedure and some preliminary findings," Personality and Social Psychology Bulletin v. 2,3 April 1997, pp. 363-377.
  2. Bickmore, T. "Friendship and Intimacy in the Digital Age," Final Project Report, MAS714 - Systems & Self, Fall 1998.
  3. Klein, J. T., Computer Response to User Frustration, Master's Thesis, MIT, February, 1999.
  4. Mauldin, M. "Chatterbots, Tinymuds, and the Turing Test: Entering the Loebner Prize Competition. Proceedings of AAAI 94, 1994.
  5. Okun, B., Effective Helping: Interviewing and Counseling Techniques, Brooks/Cole Publishing, Pacific Grove, CA, 1997.
  6. Moon, Y. Intimate Self-Disclosure Exchanges: Using Computers to Build Reciprocal Relationships with Consumers, Harvard Business School Working Paper 99-059, 1998.
  7. Picard, R., Affective Computing, MIT Press, Cambridge, MA, 1998.
  8. Reeves, B. and Nass, C., The Media Equation, Cambridge University Press, Cambridge, 1996.
  9. Thorisson, K., ToonFace: A System for Creating and Animating Cartoon Faces. M.I.T.Media Laboratory, Learning & Common Sense Section Technical Report 1-96. 1996.
  10. Turkle, S., Life on the Screen, Simon and Schuster, 1995.
  11. Weizenbaum, J. 1966. Eliza -a computer program for the study of natural language communication between man and machine. Communications of the ACM, 9, 26-45.