Activity Recognition in the Home Setting using Simple and Ubiquitous Sensors
 |
Abstract
In this work, a system for recognizing activities in the home setting that uses a set of small and simple state-change sensors, machine learning algorithms, and electronic experience sampling is introduced. The sensors are designed to be “tape on and forget” devices that can be quickly and ubiquitously installed in home environments. The proposed sensing system presents an alternative to sensors that are sometimes perceived as invasive, such as cameras and microphones. Since temporal information is an important component of activities, a new algorithm for recognizing activities that extends the naive Bayes classifier to incorporate low-order temporal relationships was created. Unlike prior work, the system was deployed in multiple residential environments with non-researcher occupants. Preliminary results show that it is possible to recognize activities of interest to medical professionals such as toileting, bathing, and grooming with detection accuracies ranging from 25% to 89% depending on the evaluation criteria used. Although these preliminary results were based on small datasets collected over a two-week period of time, techniques have been developed that could be applied in future studies and at special facilities to study human behavior such as the MIT Placelab. The system can be easily retrofitted in existing home environments with no major modifications or damage and can be used to enable IT and health researchers to study behavior in the home. Activity recognition is increasingly applied not only in home-based proactive and preventive healthcare applications, but also in learning environments, security systems, and a variety of human-computer interfaces.
Overview
The proposed system consists of three major components. (1) The environmental state-change sensors used to collect information about use of objects in the environment, (2) the context-aware experience sampling tool (ESM) used for labelling the activities, and (3) the pattern recognition and classification algorithms for recognizing activities.
|
Data Collection
Two studies were run in two homes of people not afliated with our research group to collect data in order to develop and test the activity recognition algorithms. The first subject was a professional 30-year-old woman who spent free time at home, and the second was an 80-year-old woman who spent most of her time at home. Both subjects lived alone in one-bedroom apartments. 77 state-change sensors were installed in the first subject’s apartment and 84 in the second subject’s apartment. The sensors were left unattended, collecting data for 14 days in each apartment. During the study, the subjects used the context-aware ESM to create a detailed record of their activities.
 |
Algorithms
When designing algorithms for recognizing activities in real home environments, it is important to consider factors such as the ease of setup and training the system, how privacy concerns are addressed, and real-time reliability performance. The following design goals motivated the activity recognition algorithms developed in this work.
| Supervised learning | Useful for customizing the system to the users since homes and their furnishings have highly variable layouts, and individuals perform activities in many different ways. |
| Probabilistic classification | Offers a way to deal with ambiguous and noisy information from multiple sensors. Moreover, such probabilistic classification output can then be interpreted as needed by other systems that use the activity information. |
| Model-based learning. | To reduce users privacy concerns, sincel the raw sensor data could be eliminated as soon as the user model has been learned. |
| Sensor location and type independent | Ideally, robust activity recognition would not require this information since specifying the installation location and object type is a time consuming task. |
| Real-time performance. | trade-off between model, feature, and computational complexity to achieve real-time performance |
| Online learni | To be able to adjust the internal model in real-time as new examples of activities are available. |
The naive Bayes classifiers was used in this work since it meets most of the design goals. Two versions of the activity recognition classifier were implemented: (1) a multiclass naive classifier in which the class node represents all the activities to recognize and (2) multiple binary naive Bayes classifiers, each of them representing an activity to recognize. "Exist" and "before" binary temporal features were extracted from the sensor activation to incorporate activities temporal information.
Results
The following image shows an example of the output of the system. The X axis represent the time of the day, and the Y axis represent the probabilistic output for each activity analized. The blue boxes represent the user's activities as reported by the experience sampling software (15min resolution). Three different evaluation methods were used to evaluate the algorithms. Which evaluation method is most informative depends upon the type of application that would be using the activity recognition data. The methods used where "is the activity detected at least once?", "is the activity detected during the best detection interval? and "for how long is the activity correctly detected". The table also shows the recognition results for the multi-class naive Bayes classifier.
(Click on image to zoom-in)
(Click on image to zoom-in)
For more detailed information see
Master's Thesis Full Presentation