Please visit my Media Lab webpage for latest projects:
https://www.media.mit.edu/people/haewon/overview/
Previous Projects
Theme 1: Robots Learning from Children
Reciprocal peer tutoring (RPT) focuses on the benefit students receive from tutoring one another. I am interested in developing robots that can learn from children as their peer. The first objective is to encourage social interaction by giving children an active role as a teacher. For instance, our study showed that children with autism spectrum disorder (ASD), characterized by their lack of social skills, initiated a very close number of social behaviors towards the robot compared to their typically developing peers when teaching a robot to play a game of Angry Birds.
When children demonstrate a task to a robot, they make a lot of mistakes! Sometimes, these errors have a pattern and we can reversely infer about the child’s task skill deficits by analyzing this pattern. Our second objective is to detect and computationally model these error patterns while the robot learns new tasks from a child's demonstration. Using this information, the robot can change its interaction strategy or present tasks that can maximize the intervention objectives.
Lastly, we want to measure the effect of robot intervention by conducting studies whether the interaction modalities that children with social impariments projected towards the robot can continue to be practiced when interacting with other people. (Obtained IRB consent for video sharing)
Below are my specific research topics:
1.1 Interactive Instance-based Learning (IIBL)
Robot needs to learn and act quickly even with a very small number of demonstrations from a child. We aim to solve the task learning problem using an experience-based learning framework that stores human demonstrations as task instances. These instances are retrieved when confronted with a similar task in which the system generates predictions of task behaviors based on prior actions. In order to automate the processes of instance encoding, acquisition, and retrieval, we have developed a framework that gathers task knowledge through interaction with human teachers. This approach, further referred to as interactive instance-based learning (IIBL), uses Bayesian case-based reasoning with neural nets that utilizes limited information available to the robot to generate similarity metrics for retrieving instances.
1.2 Robot Prompting
By studying how professional therapists utilize verbal, gaze, and gestural cues to engage/re-engage children, a robot prompting methodology can be developed. We are specifically interested in understanding the spatio-temporal features of the therapists’ gestures in correlation to the child’s engagement status. We also aim to create an RGB-D database of therapeutic prompting behaviors.
(Figure: Robot prompting methodology can be developed by analyzing the therapist's promoting strategies. Obtained IRB consent for picture sharing)
1.3 Reverse LfD Error Modeling
Most of the time, learning from demonstration (LfD) algorithms regard human demonstration errors as outliers or disregard them in the training data. However, in a scenario in which we would like to learn more about the user, these demonstration errors might give us an insight about the user’s task skill level. Within the IIBL framework, we can group instances that produce similar results and find a basis function for each cluster. By analyzing the basis function that corresponds to the instance cluster that produces the least desirable results, we can infer about the task features the user lacks understanding. We are interested in combining this information to develop continuously engaging robot interaction model as well as task progression strategies.
(Figure Top: Ratio of demonstration errors made by children while teaching a robot. Down: Instance clustering result from the Angry Birds)
1.4 Extending Robot Intervention Effects to Improve Interaction with Others for Children with Social Impairments
Our previous study showed an increase in initiated social behaviors when children with social impairments interact with robots. In the follow up study, we aim to validate our hypothesis that robot intervention can further benefit these children’s interaction with others. The study specifically aims at motivating children to share a task workspace and take turns with others. Our 6-week case study shows that after the robot intervention, the child with ASD required 58% less prompting to share turns, and voluntarily offered to share turns 3 times more often. (Obtained IRB consent for video sharing)
Theme 2: Building Intelligent Robots
I am very passionate about delpoying autonomous robots with social intelligence in our everyday lives. At this time, I am particularly interested in the long-term effects of robots in children’s therapeutic intervention. This goal is difficult to achieve without a commercialized product. Zyrobotics, LLC is helping me achieve that dream. We are working on a socially intelligent therapy robot as well as assistive devices that will help children with disabilities access their computer and robot friends.
2.1 Zyro Robot Therapy System: a Personalized Robot Coach that Gamifies Therapy for Kids with Disabilities
A social robot that reasons about children's skill deficits through interaction and presents therapeutic tasks necessary for successful interventions. Task progresses stored in the cloud provide guidelines for designing new educational contents. Awarded Best US Startup and Robohub Reader’s Pick Award and nominated as Best Startup Company at RoboLaunch 2015. Funded by NSF Accelerating Innovative Research - Technology Translation (AIR-TT) grant to seek lab-to-market opportunity of my PhD work. (More details to be posted on Zyrobotics website: http://zyrobotics.com/wpcproduct/zyro/)
2.2 BiPi: Bipedal Intelligent Robot
BiPi is a 23-DOF humanoid robot (23-DOF) that I made as a senior design project in 2006. I conducted research on bipedal gait, object tracking, sign reading, and obstacle avoidance algorithms.
Theme 3: Accessible Technology
3.1 Customizable Touch-sensitive Accessibility Plate
A customizable touch-sensor plate that transforms objects like plush toys into accessibility devices. Once the number of sensor placements are decided, lobes are created around each sensor for tension distribution and to minimize interruption between the sensors. After copper trace paths are created for the given sensor/plate configuration, the whole plate assembly is cut with a vinyl cutter (die cutter). Integrated into Zyrobotics products such as Zumo Learning System (http://zyrobotics.com/zumolearning/) that enables children to engage in interactive hands-on STEM learning irrespective of their motor, cognitive, and learning abilities.
3.2 TabAccess: A Wireless Adapter for Tablet Accessbility
A bluetooth adapter for connecting alternative switch-input devices to smartphones and tablets for users with motor impairments. This project began in the Fall of 2009 when we encountered students who could not access a touchscreen interface using tap, pinch zoom, and swipe during our outreach robot-programming activity. Since then, the device evolved through interviewing 100+ customers, competing in design competitions, and winning several grants. TabAccess can be purchased here: http://zyrobotics.com/wpcproduct/tabaccess/