@media screen\n{\n\nbody {\n font: 13px/125% "Lucida Grande", "Trebuchet MS", "Bitstream Vera Sans", Verdana, Helvetica, sans-serif;\n}\n\n}\n\n@media print\n{\n\nbody {\n font-size: 6.2pt;\n font-family: "Lucida Grande", "Bitstream Vera Sans", Helvetica, Verdana, Arial, sans-serif;\n}\n\n}

Rana el Kaliouby | MIT Media Lab

Rana el Kaliouby, Ph.D. is co-founder and Chief Technology Officer of MIT start up Affectiva, and Research Scientist at the MIT Media Lab. El Kaliouby co-founded Affectiva with Professor Rosalind W. Picard to bring emotion measurement and communication technologies to the masses, including the facial expression recognition technology (Affdex, FaceSense), which she invented. The New York Times rated her research as one of the top 100 innovations of 2006, and her work has been featured in Reuters, Wired, The Boston Globe and more. Rana is the 2006 recipient of a Global Women and Inventors and Innovators Network award. She holds a BSc and MSc in computer science from the American University in Cairo and a Ph.D. from the computer laboratory, University of Cambridge. \n As Affectiva’s CTO, el Kaliouby leads the company’s science team and oversees its technology roadmap. She is Principal Investigator on NSF’s highly competitive Small Business Innovation Research (SBIR) grant to bring Affdex to market. Affectiva won its NSF SBIR Phase I grant of $180,000 in January 2011 to develop Affdex, the world’s first cloud-based facial analysis system. Affdex powered several public projects including one on Forbes.com, where anyone with a webcam and Internet connection could share their facial expressions as they watched videos online. Affectiva recently won an SBIR Phase II grant for $500,000 to scale Affdex for advertising testing. El Kaliouby also oversees Affectiva’s repository of face videos, the largest in the world, which is used to continuously train Affdex to recognize a wide range of emotional expressions from the face. \n At MIT Media Lab, el Kaliouby’s research focuses on applications of affective technologies in autism spectrum disorders. El Kaliouby was co-Principal Investigator on a $1M NSF grant with the Groden Center to prototype wearable glasses that enrich the social interactions of individuals on the autism spectrum. This research was rated among the top 100 innovations of the year 2006 by New York Times and was featured on the front page of New Scientist in 2011. El Kaliouby is currently co-PI on a 5 year NSF expedition grant with Picard and Goodwin at MIT and a multi-disciplinary team from Georgia Tech, CMU and BU. \n [[Cirriculum Vitae|cv.html]] or in [[pdf|Rana_el_Kaliouby_cv.pdf]]

This site was developed by [[Rana el Kaliouby]] using a ~JavaScript application called ~TiddlyWiki, and is primarily based on Andres Monroy Hernandez's webpage.

For as long as I remember, I have been fascinated by people’s ability to sense and make sense of themselves and others, and the important role this plays in how we connect to and communicate with others (across and inspite of distance, cultural and religious differences, and even in the absence of language). I am also interested in how the social and affective aspects of our life experiences influences our memories, how and what we learn, and eventually how the dots in our life connect. \n\n Thus, my research is about inventing new computational models and technologies that sense and have a commonsense understanding of people’s mental states, behaviors and experiences. My goal? (1) empower people to capture, learn from and share their social, affective and memorable experiences and ulimately connect in a fundamentally better way; (2) address open research questions regarding how our people sensing abilities shape our behaviour, memories and connections with others.

My current research projects: \n# [[Facial Analysis API|API.html]] for real-time analysis of affective and cognitive states \n# Detection and Analytsis of Hand over face gestures \n# [[Social-emotional toolkit for autism spectrum disorders|http://affect.media.mit.edu/projectpages/esp]] \n# Devise new computational models of people’s experiences (e.g., customer delight, memorable experiences) \n# Analysis of customer preferences \n# Sensor-toys for children, including those on the autism spectrum, that will help them navigate the world around them in richer and more memorable ways.

\n* Identification of people's nonverbal behaviour \n* Identification of people's actions and gestures \n* Inference and attribution of complex mental states \n* Mental state reasoning \n* Incorporation of context cues for mental state attribution \n* Inference of intention \n* Mind-reading and Decision-making \n* Applications of mind-reading to Autism Spectrum Disorders ([[ESP project|http://affect.media.mit.edu/projectpages/esp]]) \n* Prosthetics \n* Detection of driver mental states (in collaboration with University of Cambridge and Toyota Corporation)

My current research involves developing real-time vision and sensor models/systems for sensing people. Sensing channels include observable ones (e.g., face, hand-over-face gestures) as well as ones that are up-close (e.g., skin conductance, accelerometers, EEG). People sensors can also be embedded in everyday objects, such as toys or pens. I have used Dynamic Bayesian Networks to model affective (e.g., like and dislike) as well as cognitive (e.g., confusion) states of people. I would like to extend this model to include creativity (can we predict or influence people’s level of creativity) and memory (can we predict and influence what experiences people will find memorable). I am also working on extending the models to sense a variety of “people” including children and large audiences. \n # Real-Time Inference of Affective and Cognitive Mental states from Video \n # Detection and Localization of Hand-over-Face Gestures from Video (with Marwa Mahmoud)

Social communication and emotion regulation difficulties, lie at the core of Autism Spectrum Disorders, making interpersonal interactions overwhelming, frustrating and stressful. Often, these difficulties portray that persons on the autism spectrum are "choosing" to be disengaged from social interactions due to a lack of interest or desire, even when that is not the case. To the contrary, many persons on the autism spectrum who are now able to communicate, write about their persistent attempts to seek interaction with others using unconventional nonverbal cues that were either misinterpreted or simply unnoticed by family members and others. Communication difficulties combined with atypical visual and auditory perception in ASD makes traditional learning challenging, and suggests that independent, spontaneous and sensory-based learning comes more naturally to persons with ASD. \n\n In collaboration with the Groden Center, we are developing novel wearable, in situ social-emotional technology that helps individuals with ASD acquire an affinity for the social domain and improve their overall social communication abilities. We are also developing technologies that build on the nonverbal communication that individuals on the autism spectrum are already using to express themselves socially and emotionally, to help families, educators and other persons who deal with autism spectrum disorders to better understand these alternative means of nonverbal communication. Our work leverages the advances in affect sensing and perception to (1) develop technologies that are sensitive to people's affective-cognitive states; (2) advance autism research and (3) create new technologies that enhance the social-emotional intelligence of people diagnosed with autism, as well as those who are not. Several other projects are also underway, including autism and emotion regulation, sensor and toy technologies for monitoring children.

\n* Co-taught MAS.962: Autism Theory and Technology course during Spring 2006 with Rosalind Picard, Cynthia Breazeal and Sherry Turkle. \n* As an Adjunct Faculty member for Fall 2005, I taught Introduction to Computer Science using C++ at the [[Computer Science Department|http:\\www.cs.aucegypt.edu/index.jsp]] at the [[American University in Cairo (AUC)|http:\\www.aucegypt.edu]], where I've also worked as a Teaching Assistant between 1995 and 2001. At the Computer Laboratory in Cambridge I supervised several courses, graduation projects and internships. At the [[Academic Computing Services|http:\\www.aucegypt.edu/acs]] at AUC, I was responsible for training faculty, staff and students. I introduced and designed the content of two news tracks of courses and delivered the first online course at AUC. I have also collaborated with faculty members on integrating technology in their curricula.

\n!Book Chapters \n# Rana el Kaliouby and Peter Robinson. ~Real-Time Vision for HCI, chapter ~Real-time Inference of Complex Mental States from Facial Expressions and Head Gestures, pages 181200. ~Spring-Verlag, 2005. \n# Rana el Kaliouby and Peter Robinson. Designing a More Inclusive World, chapter The Emotional Hearing Aid: An Assistive Tool for Children with Asperger Syndrome, pages 163172. London: ~Springer-Verlag, 2004. \n\n \n!Journal Articles \n# Rana el Kaliouby and Peter Robinson. Prosthetic versus Therapeutic Assistive Technologies: The Case of Autism. Assistive Technology. Under review. \n# Rana el Kaliouby and Peter Robinson. The Emotional Hearing Aid: An Assistive Tool for Children with Asperger Syndrome. Universal Access in the Information Society 4(2), 2005. \n\n \n!Conference Proceedings \n# McDuff, D., Kaliouby, R., Picard, R., "Crowdsourced data collection of facial responses," 13th IEEE International Conference on Multimodal Interaction (ICMI'11), Alicante, Spain, November 14-18, 2011. \n# Sano, A., Picard, R.W., Kaliouby R., Malow, B., Goldman, S. "Autonomic Sleep Patterns in Children with Autism Spectrum Disorders," in the Extended Abstract of IMFAR 2011, San Diego, CA, USA, May 12-14. 2011. \n# McDuff, D., Kaliouby, R., Kassam, K., Picard, R., "Acume: A new visualization tool for understanding facial expression and gesture data," 9th IEEE International Conference on Automatic Face and Gesture Recognition (FG'11), Santa Barbara, CA, USA, March 21-25, 2011. PDF \n# Baltrusaitis, T., McDuff, D., Banda, N., Mahmoud, M., Kaliouby, R., Robinson, P., Picard, R., "Real-time inference of mental states from facial expressions and upper body gestures," 9th IEEE International Conference on Automatic Face and Gesture Recognition and Workshops (FG'11), Santa Barbara, CA, USA, March 21-25, 2011. PDF \n# Picard, R.W., Sano, A., Kaliouby, R.el, "Palmar vs. Forearm EDA during Natural Sleep at Home," 50th annual meeting of Society for Psychophysiological Research, Portland, OR, USA, Sep.29 - Oct.3, 2010 \n# McDuff, D., Kaliouby, R., Kassam, K., Picard, R., "Affect valence inference from facial action unit spectrograms," 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW'10), San Francisco, CA, USA, June 2012. \n# M. Madsen, R. el Kaliouby, M. Eckhardt, M. Hoque, M. Goodwin, and R.W. Picard, "Lessons from Participatory Design with Adolescents on the Autism Spectrum" Proc. Computer Human Interaction, CHI 2009. \n# Madsen, M., el Kaliouby, R., Goodwin, M., and Picard, R.W., "Technology for Just-In-Time In-Situ Learning of Facial Affect for Persons Diagnosed with an Autism Spectrum Disorder," Proceedings of the 10th ACM Conference on Computers and Accessibility (ASSETS) , October 13-15, 2008, Halifax, Canada. \n# Eckhardt, M., Madsen, M., Kashef, Y., Nasser, A.R. , Hoque, M.E., el Kaliouby, R., Goodwin, M., Picard, R.W., "User-Centered Design of Technology for Just-In-Time, In-Situ Exploration of Facial Affect for Persons on the Autism Spectrum" in the Extended Abstract of IMFAR 2009, Chicago, Illinois, USA, May 7-9, 2009. \n# D'Mello, S., Jackson, T., Craig, S., Morgan, B., Chipman, P., White,H., Person, N., Kort, B., el Kaliouby, R., Picard., R.W. and Graesser, A., "AutoTutor Detects and Responds to Learners Affective and Cognitive States," Workshop on Emotional and Cognitive Issues at the International Conference of Intelligent Tutoring Systems, June 23-27, 2008, Montreal, Canada. \n# Picard, R.W., Goodwin, M., Fletcher, R., Eydgahi, H., Williams, C., Marecki, A., Lee, C.H., Morris, R., Kim, K., Mota, S., and el Kaliouby, R. "Development of a New Toolkit Enabling Wearable Wireless Autonomic Nervous System Communication for Persons on the Autism Spectrum," Poster at International Meeting for Autism Research (IMFAR), May 15-17, 2008, London, UK. \n# Teeters, A., el Kaliouby, R., Goodwin, M., Shandell, M., and Picard, R.W. "Novel Wearable Apparatus for Quantifying and Reliably Measuring Social-Emotional Expression Recognition in Natural Face-to-Face Interaction," Poster at International Meeting for Autism Research (IMFAR), May 15-17, 2008, London, UK. \n# Rana el Kaliouby and Alea Teeters (2007). Eliciting, Capturing and Tagging Spontaneous Facial Affect in Autism Spectrum Disorder, International Conference on Multimodal Interfaces. \n# Ron Reisman and Rana el Kaliouby (2007). Facial Expression Affective State Recognition for Air Traffic Control Automation Concept Exploration, Proceedings of the ACM SIGGRAPH’07. \n# Rana el Kaliouby, Rosalind Picard, Alea Teeters, and Matthew Goodwin (2007). Social-Emotional technologies for ASD, International Meeting for Autism Research. \n# Alea Teeters, Rana el Kaliouby, and Rosalind W. Picard (2006). Self-Cam: Feedback from what would be your social partner, Proceedings of the ACM SIGGRAPH'06. \n# Rana el Kaliouby, Alea Teeters and Rosalind Picard (2006). An Exploratory Social-Emotional Prosthetic for Autism Spectrum Disorders, International Workshop on Wearable and Implantable Body Sensor Networks, April 3-5, MIT Media Lab, pages 3-4. \n# Rana el Kaliouby and Peter Robinson. Generalization of a Computational Model of ~Mind-Reading. In Proceedings of First International Conference on Affective Computing and Intelligent Interfaces, pp 582-589, 2005. \n# Rana el Kaliouby and Peter Robinson. Mind Reading Machines: Automated Inference of Cognitive Mental States from Video. In Proceedings of The IEEE International Conference on Systems, Man and Cybernetics, 2004. \n# Rana el Kaliouby and Peter Robinson. ~Real-Time Inference of Complex Mental States from Facial Expressions and Head Gestures. In the IEEE International Workshop on Real Time Computer Vision for Human Computer Interaction at el Kaliouby, CVPR, 2004. \n# Tal Sobol Shikler, Rana el Kaliouby and Peter Robinson. Design Challenges in Multi-modal Inference Systems for ~Human-Computer Interaction. In Proceedings of International Workshop on Universal Access and Assistive Technology, 2004. \n# Rana el Kaliouby and Peter Robinson. FAIM: Integrating Automated Facial Affect Analysis in Instant Messaging. In Proceedings of ACM International Conference on Intelligent User Interfaces (IUI), pages 244-246, 2004. \n# Rana el Kaliouby and Peter Robinson. ~Real-Time Inference of Complex Mental States from Facial Expressions and Head Gestures. In IEEE Workshop on ~Real-Time Vision for ~Human-Computer Interaction at the CVPR Conference, 2004. Won Best Paper Award by the Cambridge Computer Lab Ring. \n# Rana el Kaliouby and Peter Robinson. ~Real-Time Head Gesture Recognition in Affective Interfaces. In Proceedings of the 9th IFIP International Conference on ~Human-Computer Interaction (INTERACT), pages 950953, 2003. \n# Rana el Kaliouby, Peter Robinson and Simeon Keates. Temporal Context and the Recognition of Emotion from Facial Expression. In Proceedings of the 10th International Conference on ~Human-Computer Interaction (HCII): ~Human-Computer Interaction, Theory and Practice, volume 2, pages 631-635. Lawrence Erlbaum Associates, 2003. \n# Rana el Kaliouby and Peter Robinson. The Emotional Hearing Aid: An Assistive Tool for Autism. In Proceedings of the 10th International Conference on ~Human-Computer Interaction (HCII): Universal Access in HCI, volume 4, pages 68-72. Lawrence Erlbaum Associates, 2003. \n# Philipp Michel and Rana el Kaliouby. Facial Expression Recognition using Support Vector Machines. In Poster Proceedings of HCI International Conference, 2003. \n# Philipp Michel and Rana el Kaliouby. Emotion Recognition using Support Vector Machines. In Proceedings of International Conference on Multimodal Interfaces, 2003. \n# Amr Goneid and Rana el Kaliouby. Facial Feature Analysis of Spontaneous Facial Expression. In Proceedings of the 10th International AI Applications Conference, 2002. \n# Amr Goneid and Rana el Kaliouby. Enhanced Facial Feature Tracking of Spontaneous and Continuous Tracking. In Proceedings of HCI International Conference, 2002. \n\n \n!Dissertations \n# Rana el Kaliouby. Mind-reading Machines: the automated inference of complex mental states from video. Ph.D. Dissertation, Computer Laboratory, University of Cambridge. 2005. \n# Rana el Kaliouby. Enhanced Facial Feature Tracking of Spontaneous Facial Expression. M.Sc. Dissertation, Computer Science Department, American University in Cairo. 2000.

\n* IIT Techfast, Bombay, Jan 2007 \n* Royal Society Summer Science Exhibition, London, July 2006 \n* Royal Society Science Exhibition, Glasgow, September 2006 \n* IEEE International Conference on Computer Vision and Pattern Recognition (CVPR) in Washington DC, July 2004 \n* The Royal Institution of Great Britain, Feb. 2004 \n* Kettle's Yard, Cambridge UK Oct 19th - Nov 3rd 2002

kaliouby AT media.mit.edu

\n\n[[MIT|http://www.mit.edu/]] [[Media Lab|http://www.media.mit.edu/]]

[[Biography]] [[Research Interests]] [[Research Projects]] [[Teaching]] [[Contact]]

[img[Rana el Kaliouby| images/RanaelKaliouby.jpg]] [[Biography]] [[Research Interests]] [[Research Projects]] [[Publications]] [[Exhibitions]] [[Teaching]] [[Contact]]

ESP: Social Emotional Intelligence at the MIT Media Lab