6/28/01 Angela Chang

ComTouch Summer 2001 Research Plan

Abstract

This paper proposes the investigation of the effect of touch in remote communication. The investigation will primarily focus on ComTouch, a vibrotactile device that allows touch to be transmitted remotely in the form of vibration. The summer research will investigate the design of ComTouch, the development of the device, proposed tests to define the content of the touch-to-vibration channel, and interaction scenarios using ComTouch. The ComTouch augments existing audio communication by presenting users with new information using a separate modality. Combining and synchronizing touch with audio allow extra nonverbal information about the speaker to be transmitted. We hope to prove that this vibrotactile channel consists of nonverbal information (such as emotional state, simulated touch and pauses in conversational turn taking). The experiments proposed below focus on using the ComTouch device to evaluate the effect of touch in remote communication and further contribute research in the area of tangible interfaces for remote collaboration.

Keywords

human interface, Human Computer Interaction, tangible interfaces, tangible user interfaces, tactile communication, tangible telepresence

INTRODUCTION

Touch is a powerful interpersonal communication tool, used by many to communicate nonverbal information, emotions and affective state in addition to verbal conversation. The summer research for ComTouch proposes to explore the parameters of touch, when used in remote communication. A brief background history in the field of telehaptic communication and tangible telepresence gives us the motivation for this research. Advances in communication technology also make the possibility of including a tactile channel feasible. The following paragraphs describe research in one specific direction of tactile remote communication, defining the content of the information conveyed.

STATEMENT OF PURPOSE

The information content conveyed in remote communication is to be evaluated using experiments described below. By building a touch-to-vibration communication device that compliments audio communication, an extra channel of communication employing the modality of touch is available. This extra channel, called the touch channel, will be the main subject of investigation.

The specific non-verbal affective information ComTouch can convey are:

Flow of control: hesitance, agreement

Affective information: valence and arousal

Basic signaling information: goodbye, hello, okay

Experiments will determine the content of the touch channel conveyed in remote communication. Furthermore, evaluations of the device will determine if the tactile channel allows better transmission of nonverbal information. The hypothesis is that the channel of touch, when coupled with the audio channel, can transmit non-verbal affective information more accurately than when solely the audio channel is used. The tactile channel serves as a redundant sensory path for making affective communication more vivid.

RESEARCH QUESTIONS

What is the information content of touch in remote communication? Examination of a touch-to-vibration device will help evaluate the use of touch to communicate nonverbal information. The research question fall into two categories: 1) defining the parameters touch in remote communication and 2) evaluating the impact of the addition of a tactile channel on existing audio communication.

· What are some parameters of a touch-language? How can a touch-to-vibration channel be used to communicate nonverbal information? What information can we encode in touch and how to present the information tactually? How can be touch used to remotely communicate affective behavior?

· How much does the haptic quality add to communication? Furthermore, how good does haptic information have to be to be a significant impact over voice quality?

Similar to face-to-face interaction, we believe that touch can be used to remotely communicate nonverbal information. The proposed experiments aid in testing the information content conveyed using touch, and also in gauging whether the tactile channel improves existing remote communication methods.

PROPOSED EXPERIMENTS

1)Evaluating the effect of transmitting one digit of vibrotactile information

A ComTouch pad allowing one-finger of touch-to-squeeze is used. The one-finger of touch-to-vibration communication will also be compared to one finger of symmetric touch channel.

Hypothesis: One-finger of communication can improve emotional communication over straight guessing. The vibrotactile signal will be sufficient to convey emotional expressive word. However, the vibrotactile signal may not be sufficient to convey as much meaning as the unmodified touch channel.

Null Hypothesis: One-finger of communication does not improve emotional communication, and the vibrotactile channel is insufficient to encode emotional expression.

Methodology: This test involves viewing three sets of emotional words. The experiment is conducted on pairs of friends, one is the sender, and the other is the receiver. The sender and receiver have no visual contact with each other, only audio contact. The first set, the control, uses no tactile signal. The second set, the tactile control, allows the signal to be transmitted using direct touch. The third set, using the device, uses the ComTouch pad, which translates touch to vibration. For each set of tests, the sender receives a book of words, while the receiver has a contact sheet containing all the words. The sender uncovers each word one at a time and tries to communicate emotion using the modalities provided. When the receiver has an idea of which word is being viewed, she writes the order of the word on the contact sheet. No talking except the vocal cues of “next” or “repeat” are allowed. A pilot study was conducted at http://www.media.mit.edu/~anjchang/AC2001/finproj.html

Voice Only

To determine the baseline communication between the subjects, a control experiment is first conducted without the device. Only the vocal cues of “next” or “repeat” are used as the sender flips through words.

Symmetric Finger Touch

To determine control between subjects using direct touch, a hole through a wall is opened. The sender and the receiver can touch their fingertips together, and press against each other. The test is conducted with vocal cues in addition to the touch channel.

Asymmetric Finger Touch

In this set of experiments, a finger pressing down will cause the corresponding finger to move up.

Asymmetric Mapping using ComTouch

Brief explanations of the ComTouch pads are given, and 5-10 minutes are given for familiarizing with the device. The subjects will use the ComTouch pad to try to convey emotional feeling from sender to receiver.

Analysis: When the tasks are finished the subjects are asked about what method they used. Data is compared on how many correct pictures each subject received in each of the three cases. If people are more accurate when using the vibrotactile signal than guessing, then we can say the vibrotactile signal can give extra affective information that is useful. If people are more accurate with the one finger of touch, than with the vibrotactile signal than guessing.

2) Touch as a means of flow control in communication

Hypothesis: The tactile channel aids in the flow of control of audio communication.

Users can identify pauses in turn taking when the conversation includes the tactile channel. Also, because of the low requirement of attention needed by the tactile channel, users can send and receive tactile information simultaneously. Perhaps across cultures, having a means to request a change in dynamics during a meeting will allow smoother transition in audio communication. Furthermore, the ability to represent nonverbal signals such as handshaking, agreement and disagreement can be more directly applied in the tactile channel.

Methodology: Give each subject a competitve task. Each subject is told that there is a time limit in finding out information from the other subject, using a questionnaire. Record the conversation when there is no tactile channel, and see how many times a user is interrupted in mid-sentence. Record a similar task-oriented conversation when there is tactile channel present, and compare the correlation between tactile and audio conversation. For privacy, we will not log the conversation, but rather, just the presence of an audio signal. We log the tactile activity of two people using the ComTouch and determining if simultaneous transmission happens on each unit, or whether the touch channel is used as a turn-taking marker.

We hope to observe that the tactile channel is used to punctuate conversation, as an enhancement of the flow of control in audio communication. Contrary to tactile flow of control is to using audio as a wholly separate channel of information. A further test would be to observe if subjects could simultaneously send and receive signals at the same time.

3a) Nonverbal information from touch can be understood when matched with audio

Allow a user to compose a tactile and audio message (using Eric Gunther’s technology), test if companionship of signals affects the perception of the message.

Hypothesis: The tactile channel can enhance communication, or the perception of a message when it is in synchronization with the audio. The right messages affect perception, the wrong messages affect perception. Something gets transmission. Convey correct or incorrect information makes a difference,

Methodology:

Play an audio message with a variety of tactile accompaniments. Is there a general trend?

This test is conducted to determine how each audio track is located in the arousal-valence axis. A sample of pre-generated tactile messages will be played. Each subject will be asked which the tactile signal matches the feeling best. By finding the companion tactile signal to each audio input, we can test how the tactile channel affects perception of the voice channel.

The second portion of this test will allow subjects to record their own messages (audio + tactile) and play them back for others. We will note whether the content of the tactile message is used as basic signals for communication: hello, goodbye, or other examples.

Analysis: By looking at similar reports in how the touch signal is synchronized with audio signals, we can tell what the general trend for tactile signals are in each arousal-valence axis. This is a pretest for the second portion, which evaluates the effect of this tactile channel.

3b) Evaluating the degree of affective expression through touch

Hypothesis: The tactile channel can allow users to determine degrees of expressive communication such as arousal or valence. The presence of a tactile channel gives a better degree of accuracy in perceiving affective state. Subtleties in degrees of liking or arousal are better conveyed with the tactile channel. By comparing the reception of audio messages with and without tactile, we hope to prove that the perception of expressiveness is more specific.

Methodology:

The test consists of listening to prerecorded messages. Each subject will replay a series of messages and rate the degree of nonverbal information conveyed by the speaker. Each set will have 5 messages. The selected messages are distributed across the four quadrants of arousal-valence (mad, somber, bored, excited, sad). One set will contain tactile and audio information, while another set will contain only audio information. The audio messages will be the same in both sets of messages. The order of presentation between audio and audio+tactile messages will be random.

Analysis: Comparisons of how users rate the 2 sets of messages will be made using a valence-arousal chart. We will be looking for better consistency in the accuracy when combined tactile+audio messages are received, in terms of accuracy in valence (liking or disliking) or arousal (highly aroused or unaroused). When combined modality sets are assessed in comparison to audio-only sets, it can be determined whether the tactile signal has a significant effect over voice.

***8(Novelty Effect- I had talked to Scott Brave earlier about the need to account for the novelty effect.)

RESEARCH SCOPE (LIMITATIONS AND ASSUMPTIONS)

The content found in the tactile channel from these experiments is limited by the following conditions:

Audio in conjunction with tactile channel

This research assesses that the touch is used in conjunction with other modalities, particular speech, in remote communication. The ComTouch is designed for concurrent use with a phone. Furthermore, this study assumes that the information presented by both touch and audio channel are complementary. The tactile channel is evaluated as a complement to existing remote communication technology, not a substitution.

Misconceptions of asymmetric interaction of touch to vibration

Another challenge is anticipating the end-user misconceptions about touch-to-vibration.

Because touch is mapped asymmetrically to vibration, the mapping of different meanings of touch can be affected by the representation of touch as vibration. The change in modality from physical touch to vibration can potentially cause discomfort, as people will have to adjust to the new mapping. Special care will be taken to gauge discomfort, and time will be allotted for subjects to adapt to the device.

Ergonomics

Another challenge is the issue of ergonomics and human factors. Because touch is so personal and dependent on the physical dimensions of a person, the prototypes attempt only to cater to hands representative of adult, medium-sized hands. The researchers are aware that many people have different sized hands, and placing the sensors and actuators in the correct positions for use by all will be challenging. For simplicity, only one size of ComTouch is available, with the plan that in the future there may be more alternatives for different sized hands.

TACTUAL INTERACTION TECHNIQUES (augmenting Audio)***

By using ComTouch to explore the nature of the tactile remote communication channel, real-life scenarios can highlight the possible content of tactile information. Brief descriptions of possible tactile communication scenarios are listed here to highlight the benefits and intended application for ComTouch. Some of these scenarios can only be achieved if users are both adapted to the ComTouch. We hope to do field tests of the following scenarios to highlight the qualities that are desirable: privacy of expression, nonverbal expression, emotional expression, and low distraction communication.

Situations requiring privacy

Where audio communication is impossible, a touch-based device can provide a private channel for communication. For example, one might wish to remain connected even when inside a library. Touch based communication can allow discreet notification of personal messages without broadcasting an interruption to others.

Providing an emotional communication channel

Loved ones often want to communicate personal messages. If the modality of touch is available, being able to express higher-level emotions might be desired.

Multiplexing of information

In places where remote communication already takes place, touch devices can allow people to further their range of communication by multiplexing existing communication channels. The low attention requirement of ComTouch might allow users to have a spoken conversation over the phone that is entirely different from the tactile message they are sending.

TIMELINE

Week of:

June 4 -unpacking/urop introduction to project/set up schedule

June 11 -First prototype manufacturing phase

- building/ schedule research on what we are trying to prove

June 18 -Finished First Prototype (2 touch-to-vibration wraps for phones)

-begin building 8-10 phones for evaluation

-working draft of proposed paper/ experiment design done by Friday

June 25 – finishing first prototype manufacturing phase; second prototype manufacturing phase begin

July 2 - Finish Second Prototype phase (8-10 phones); Cellular port engineering; form factor engineering done; fabrication of first phase form factor

July 9 - preliminary testing and debugging of first phase; fabrication of second phase circuit board and form factor

July 16 –testing; begin running pilot studies; finish 2^nd phase fabrication

July 23 -Run experiments at lab (gather data)

July 30 -Run experiments at lab (gather data and preliminary evaluation)

(Wednesday) August 2 - Leave for Greece

August 6-19 (in Greece)

August 20- return from vacation/help Ars Electronica

(Wednesday) August 23-final data analysis and evaluation

August 24- in Austria

(Sunday) September 2- Return to Boston

Sept 3- Finish Evaluation; begin writing CHI paper

Sept 7- First draft of CHI paper out to readers

Sept 10- Major revisions of draft finished

Sept 14- Final draft sent to CHI