Quantifying the Relationship between Real Life Moment-to-Moment Activities and Physiological Signals
Josh Weaver and Elisabeth Sylvan
[ class web site ]

What is the relationship between where you are, what you feel, and what your body is doing? This project motivated to understand better the relationship between people's bodies, the spaces their bodies inhabit and how people feel. We built a wearable system with which we can gather data towards this goal, have gathered data, developed some methods for processing the data, and analyzed some data.

Ultimately the goal of this project is twofold. First as previously mentioned, we wish to contribute to the research on engineering psychophysiology. Second we wish to build interfaces that allow wearers to understand the relationship between both their moment-to-moment psychophysiology, but also what this daily data gathering can tell them about how their bodies react to what they are doing long term.

Because these are all difficult problems and because we were unsure of appropriate intervention before gathering data, we focused on building a platform on which to base these investigations.

IMPLEMENTATION
We implemented a system that records Galvanic Skin Response, temperature, EKG, movement and sound levels, along with a life event notation button. Data was stored on a Compact Flash Card. This system was based upon the Vadim Gerasimov's Hoarder Board and a daughter board that we built which has two channels for EKG, two for GSR and one for temperature. In addition we added a button for the purpose on annotation on one of the Hoarder board has (among other features) a PIC16F877 20-MHz microcontroller , a CompactFlash connector, a real time clock, a serial port, a programmer port. We also build database interfaces for annotating life events on our Palm Pilot and Ipaqs.

The goal of this data collection is to determine when people are engaged in a particular activity, how well they are engaged in the activity and how they feel at that time.

PRELIMINARY STUDY
Using the small, lightweight hardware (Gerasimov) that is easy to wear or carry, experimental participants recorded EKG, skin conductivity, and external body temperature and stored the information on a Compact Flash Card. (Future investigations will also incorporate movement identification through the use of accelerometers.) Since we were interested in the relationship between daily life events, feelings, and physiology, whenever they engage in a new activity, the wearers annotate in a handheld device my activity, location, emotion, intensity of emotion and how much in a flow state they feel. We defined a life event as any major change in location or activity or mood.

We then parsed the physiological data so that we have a number from each sensor every time the button is pushed. Here is some fairly controlled data in which we alternated walking, running, talking, resting, reading and going outside for a couple minutes each. These graphs show what Liz looks like when she is doing one of the most fearful things a person can do: talking in front of a large group. This data was recorded in a more naturalistic way and tracks running, walking, and getting a massage. Currently we are running a series of comparisons between the physiological data and the annotation.

AREAS OF APPLICATION
We have engaged in considerable discussion about the applications of this work. The following are brief discussion of these areas of applications.

1.) Heart disease. (Josh)
It is well known that stress and exercise levels and temperament affect the likelihood of having heart disease and its severity. Can we track exercise, mood and stress of those at risk and those not at risk to gather data on 1.) what precedes a heart disease event 2.) lifestyle differences of those who have and do not have heart disease and 3.) if we can inform people about how their current mood or activities affects their health long term and suggest methods of change, can we reduce the rate or severity of heart disease?

2.) Productivity, Creativity and Peak Performance (Elisabeth)
A specific opportunity for computational systems is to support the user’s concentration and focus on his work in order to facilitate a creative state of mind. Csikszenmihalyi (1990) introduced the concept of flow, a feeling of being highly focused, in control, as highly involved and effective, as we can be in a task, perhaps, even elated. To experience these rare moment when we are performing at our peak, we need to be engaged in our lives in an ongoing manner.

But how do we create the meta-awareness that helps us to maintain focus and achieve these states? One way of determining when people are in a flow state may be to observe their physiological signals. This information could then be portrayed back to them to encourage ongoing awareness of their involvement in their own work and play.

FUTURE WORK
We plan to continue to analyze the data we gathered after class is over and, in particular, we wish to use Roz's group's heart rate variability algorithm. We also plan to gather more data and see what we can find in terms of the relationship between activity, emotion, and flow.

In addition we would like the make several additions to the hardware. By the end of next week, We will incorporate the accelerometer from the MIThril project. We would also to have a more complete annotation system that was saved directly onto the same Flash Memory card as the sensor data, rather than saving annotation on a separate device.

To accomplish the goal of informing people about the relationships between their own bodies and their environment, two additional areas of work must be complete. Two long term goals are to develop software that allow people to visualize and analyze their daily rhythms on their computer and also display information about their current state on their body in real time. In particular, display modalities that indicate the current level of stress to the user. Design possibilities are the bracelet described in Elisabeth's initial proposal, a soothing, calming buzzer on the back of the neck, and possibly subliminal displays.

Last modified 20 May, 2002