Proposal for Ph.D. General Examinations

Nitin Sawhney

MIT Media Laboratory

January 28, 2000

Introduction

People wish to maintain a level of awareness and communication with others in a variety of social settings at different times of the day. Awareness provides subtle cues about people's availability, rhythms, and regularity of activity (or lack thereof). One's expectations regarding the social and temporal structure of the environment, allows better coordination with others in a workgroup and enhances cognitive well being in a community.

Private/public space and time do not have well defined boundaries; they are highly malleable based on the context of situations. However our communication devices assume binary and absolute modes for interaction. Telephones can disrupt recipients and require a time-limited conversation between parties rather than an on-going awareness. Video and audio-conferencing permit open connections between participants but typically require complete engagement or attention. Communication technology can now be used to create more graceful means for peripheral awareness and background conversation.

The challenge is to develop means for extracting and representing everyday social/temporal patterns, designing lightweight communication techniques as well as shared 'community appliances'. One must consider approaches that permit on-going awareness of people's activity while allowing sufficient control to individuals to interact or retain a sense of privacy. Such methods can be evaluated in the context of casual workplace domains, distributed workgroups, infant/senior caretakers, and semi-public spaces.

This examination covers three areas related to analysis and representation of activity for enhancing community awareness and interaction. The main area, design for awareness and communication in community, is concerned with HCI approaches towards synchronous and asynchronous interaction within workgroups. The supporting technical area, recognizing human activity in audio/visual scenes, covers computational methods for representation and classification of human and environmental activity in visual and auditory scenes. The supporting context area, understanding socio-cultural behavior in public/private spaces, provides an overview of ethnographic and psychological studies within which to consider the implications of CSCW systems in social settings.

Design for awareness and communication in community

Main Area

Chris Schmandt

Principal Research Scientist, MIT Media Laboratory.

Recognizing human activity in audio/visual scenes

Supporting Technical Area

Trevor Darrell

Assistant Professor, Dept. of EECS and MIT A.I. Laboratory.

Understanding socio-cultural behavior in public/private spaces

Supporting Context Area

Mark S. Ackerman

Associate Professor, Information and Computer Science, University of California, Irvine.

Visiting Associate Professor, MIT Media Laboratory.

 

Main Area:

Design for awareness and communication in community

Examiner: Chris Schmandt, Principal Research Scientist, MIT Media Laboratory.

Description: The major area covers design of methods for communication and peripheral awareness in a distributed community. I wish to examine auditory and visual techniques that support a range of synchronous and asynchronous awareness and collaboration. The readings consider actual CSCW systems and evaluate their impact on distributed workgroups. We will examine notions such as space vs. place, intentional vs. impromptu interaction as well as support for varying temporal structures in a workgroup. The readings will consider trade-offs between providing rich information vs. issues of privacy and disruption. The second section of the reading list will consider techniques for representation of activity patterns using varying levels of visual/auditory scene abstraction, social visualization and auditory awareness.

Requirements: The requirement for this area will consist of a publishable-quality paper.

 

Signature and Date:

Reading List:

Communication & Awareness in CSCW

  1. Ackerman, M. S., Debby, H., Mainwaring, S. D. and Starr, B. 1997. Hanging on the ‘Wire: A Field Study of an Audio-Only Media Space. ACM Transactions on Computer-Human Interaction, Vol.4, No.1, 39-66.

  2. Ackerman, M.S., and Palen, L. 1996. The Zephyr Help Instance: Promoting ongoing activity in a CSCW system. In Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI'96). ACM, New York, 268-275.

  3. Agamanolis, S., Westner, A., and Bove, Jr. V. M. "Reflection of Presence: Toward More Natural and Responsive Telecollaboration," Proc. SPIE Multimedia Networks, 3228A, 1997.

  4. Cohen, J. 1994. Monitoring Background Activities. Auditory Display: Sonification, Audification, and Auditory Interfaces. Reading MA: Addison-Wesley.

  5. Dourish, P. Adler, A. Bellotti, V., and Henderson, A. 1996. "Your Place or Mine? Learning from Long-term Use of Audio-Video Communication." Computer-Supported Cooperative Work, 5(1), 33-62.

  6. Dourish, Paul and Bly, Sara. "Portholes: Supporting Awareness in a Distributed Work Group." Proceedings of CHI '92.

  7. Dourish, Paul. "Extending Awareness Beyond Synchronous Collaboration." Position Paper for CHI '97 Workshop on Awareness in Collaborative Systems, 1997.

  8. Egger, E., and Wagner, I. Time-management: a case for CSCW. Proceedings of CSCW '92. pp. 249-256.

  9. Harrison, Steve and Dourish, Paul. "Re-Place-ing Space: The Roles of Place and Space in Collaborative Systems." Proceedings of CSCW '96. pp. 67-76.

  10. Hindus, D. and Schmandt, C. Ubiquitous audio: capturing spontaneous collaboration. Proceedings of CSCW '92. pp. 210 - 217.

  11. Issacs, Ellen, Tang, J.C., Morris, T. "Piazza: A Desktop Environment Supporting Impromptu and Planned Interactions." Proceedings of CSCW '96. pp. 315-324.

  12. Lee, A. Schlueter, K. Girgensohn, A. "Sensing Activity in Video Images." Proceedings of CHI '97.

  13. Obata, A. and Sasaki, K. OfficeWalker: a virtual visiting system based on proxemics. Proceedings of CSCW '98. pp. 1-10.

  14. Pederson, E.R. and Sokoler, T. "Aroma: Abstract Representation of Presence Supporting Mutual Awareness." Proceedings of CHI '97.

  15. Smith, I. and Hudson, S. E. 1995. Low-disturbance audio for awareness and privacy in media space applications. In Proceedings of the ACM Conference on Multimedia. ACM, New York, 91-97.

  16. Watts, J. et. al. "Voice Loops as Cooperative Aids in Space Shuttle Mission Control." Proceedings of CSCW '96. pp. 48-56.

    17.

    Audio/Visual Representation of Human Activity

  17. Judith Donath. "Social Visualization". Media Lab Technical Report, 1998.

  18. Gaver, W. W., R. B. Smith, and T. O'Shea. "Effective Sounds in Complex Systems: The ARKola Simulation." In Proceedings of CHI '91, New Orleans, 1991.

  19. Gaver, W. W. "What in the World Do We Hear? An Ecological Approach to Auditory Source Perception." Ecol. Psych. (5) 1, 1993.

  20. Hudson, Scott E. and Smith, Ian. "Techniques for Addressing Fundamental Privacy and Disruption: Tradeoffs in Awareness Support Systems." Proceedings of CSCW '96. pp. 238-247.

  21. Mynatt, E.D., Back, M., Want, R., Baer, M., Ellis, J.B. 1998. Designing Audio Aura. Proceedings of CHI’98.

  22. Minar, N. and Donath, J. Visualizing the crowds at a web site. In Proceedings of CHI '99.

  23. Donath, J., Karahalios, K., and Viegas, F. Visualizing Conversations. Proceedings of HICSS-32, Maui, HI, January 5-8, 1999.

  24. Tufte, E. 1990. Envisioning Information. Cheshire, CT: Graphics Press.

  25. Zhao, Q. A. and Stasko, J. T. Evaluating image filtering based techniques in media space applications. Proceedings of CSCW '98.

 

 

Supporting Technical Area:

Recognizing human activity in audio/visual scenes

Examiner: Trevor Darrell, Associate Professor, Dept. of EECS and MIT A.I. Laboratory.

Description: This area will cover approaches to machine perception of human and environmental activity in visual and auditory scenes. The first section of readings examine pattern-recognition, statistical modeling and learning temporal sequences. The second section considers auditory scene analysis as a perceptual foundation for segmenting sounds in the environment, but also potentially informs the analysis of visual scenes. The third section surveys visual techniques for detecting and classifying human activity using motion templates and spatio-temporal models. The overall emphasis is on high-level representation of activity derived from both auditory and visual features of everyday environments.

Requirements: This area will be examined in the form of a 24-hour take home exam to be administered by Professor Darrell.

 

Signature and Date:

Reading List:

Pattern Recognition

  1. Bishop, C. M. Neural Networks for Pattern Recognition. Oxford University Press. 1995.

    2. Chapter 1: Statistical Pattern Recognition

    Chapter 2: Probability Density Estimation

    Chapter 3: Single Layer Networks

    Chapter 8: Pre-processing and Feature Extraction

    Chapter 9: Learning and Generalization

  2. Haykin, S. Neural Networks: A Comprehensive Foundation. Prentice-Hall, Inc. 1999.

    3. Chapter 6: Support Vector Machines

    Chapter 10: Information Theoretic Models

  3. Rabiner, L.R. and Juang, B.H. Ch. 6: Theory and Implementation of Hidden Markov Models. In Fundamentals of Speech Recognition. 1993.

    4. Recognizing Auditory Activity

  4. Bregman , A. 1990. Auditory Scene Analysis: The Perceptual Organization of Sound. The MIT Press.

    5. Chapter 1: The Auditory Scene

    Chapter 2: Sequential Integration

    Chapter 3: Integration of Simultaneous Auditory Components

    Chapter 4: Scheme-based Segregation and Integration

  5. Clarkson, B. and Pentland, A. "Unsupervised Clustering of Ambulatory Audio and Video." Proceedings of ICASSP, 1999.

  6. Clarkson, B., Sawhney, N., Pentland, A. 1995. "Auditory Context Awareness via Wearable Computing". 1998 Workshop on Perceptual User Interfaces (PUI'98). Nov 4-6, 1998. pp. 37-43.

  7. Roy, D., Schiele, B., and Pentland, A. "Learning Audio-Visual Associations using Mutual Information." International Conference on Computer Vision, Workshop on Integrating Speech and Image Understanding. Corfu, Greece, 1999.

  8. Saint-Arnaud, N. 1995. Classification of Sound Textures. M.S. Thesis, MIT.

  9. Schrier, E. and Slaney, M. "Construction and Evaluation of a Robust Multifeature Speech/Music Discriminator." Proceedings of the 1997 International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Munich, Germany, April 21-24, 1997.

    10. Recognizing Visual Activity

  10. Ballard, D. H., and Brown, C. M. Computer Vision. Prentice-Hall, Inc. 1982.

    11. Chapter 3: Early Processing

    Chapter 4: Boundary Detection

    Chapter 5: Region Growing

    Chapter 7: Motion

  11. Bobick, A. "Movement, Activity and Action: The Role of Knowledge in the Perception of Motion." Phil. Trans. R. Soc. Lond. B (1997) 352, 1257-1265.

  12. Cui, Yuntao. "Indoor monitoring via the collaboration between a peripheral sensor and a foveal sensor." IEEE Workshop on Visual Surveillance (in conjunction with ICCV '98), Mumbai, India, January 1998.

  13. Darrell, T., Gordon, G., Harville, M., Woodfill, J. "Integrated person tracking using stereo, color, and pattern detection." Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR'98), pp. 601-609, Santa Barbara, June 1998.

  14. Davis, J.W. "Recognizing movement using motion histograms." MIT Technical Report #487, 1999.

  15. Grimson, W.E.L., Stauffer, C., Romano, R. Lee, L. "Using adaptive tracking to classify and monitor activities in a site." Proceedings of CVVPR'98.

  16. Hogg, D.C. "Learning Spatiotemporal Models from Examples," Proceedings of the British Machine Vision Conference, Birmingham, Sept 95.

  17. Kettnaker, V., Brand, M. "Minimum-entropy models of scene activity." Proceedings of CVPR'99. pp. 281-286.

  18. Oliver, N., Rosario, B., Pentland, A. "A Bayesian Computer Vision System for Modeling Human Interactions." Proceedings of ICVS'99, Gran Canaria, Spain, January 99.

  19. Starner, T., Schiele, B., Pentland, A. 1995. "Visual Context Awareness via Wearable Computing". 1998 Workshop on Perceptual User Interfaces (PUI'98). Nov 4-6, 1998. pp. 59-62.

  20. Stauffer, C. "Automatic hierarchical classification using time-based co-occurrences." CVPR'99, Fort Colins, CO, June 1999.

  21. Wren, C., Azarbayejani, A., Darrell, T., Pentland, A. "Pfinder: Real-time Tracking of the Human Body." IEEE Trans. PAMI, 19(7): 780-785, July 1997.

 

Supporting Context Area:

Understanding socio-cultural behavior in public/private spaces

Examiner: Mark S. Ackerman, Associate Professor, Information and Computer Science, University of California, Irvine. Visiting Associate Professor, MIT Media Laboratory.

Description: The supporting context area examines behavior of people in public/private spaces and the social implications of intervention by communications technologies. The readings analyze individual expression and group behavior in different social settings, perception of activity sequences by people, privacy and peripheral awareness. An emphasis is on the role of long-term social awareness for cognitive support and negotiated order within 'collectives' such as in a casual workplace or elderly communities. The readings also consider methods of evaluation and ethnographic approaches to design of CSCW systems.

Requirements: This area will be examined in the form of a 24-hour take home exam to be administered by Professor Ackerman.

 

Signature and Date:

Reading List:

 

  1. Ackerman, M. and Starr, B. Social Activity Indicators: Interface Components for CSCW Systems. In Proceedings of the ACM Symposium on User Interface Software and Technology (UIST'95), 1995, pp. 159-168.

  2. Agre, Philip E. and Rotenberg, Marc (Ed.). 1998. Technology and Privacy: The New Landscape. The MIT Press.

    3. Chapter 1: Beyond the Mirror World: Privacy and the Representational Practices of Computing. (Philip E. Agre)

    Chapter 2: Design for Privacy in Multimedia Computing and Communications Environments. (Victoria Bellotti)

    Chapter 10: Interactivity as Though Privacy Mattered. (Rohan Samarajiva)

  3. Altman, Irwin, Lawtonm, M. P., Wohlwill, J.F. (Ed.) Elderly People and the Environment. Human Behavior and Environment, Advances in Theory and Research. Volume 7. Plenum Press. 1984.

    4. Chapter 5: Aging in Rural Environments (Graham D. Rowles)

    Chapter 8: The effects of Residential and Activity Behaviors on Old Peoples Environmental Experiences (Stephen M. Golant)

  4. Bentley, R., Hughes, J. A., Randall, D., Rodden, T., Sawyer, P., Shapiro, D. and Sommerville, I. Ethnographically-informed systems design for air traffic control. Proceedings of CSCW '92. pp. 123-129.

  5. Berger, P. 1963. Invitation to Sociology; a humanistic perspective. Anchor Books.

  6. Davis, Murray S. 1971. That's Interesting! Philosophy of the Social Sciences: 309-344.

  7. Dourish, Paul, and Victoria Bellotti. 1992. Awareness and Coordination in Shared Workspaces. Proceedings of the Conference on Computer-Supported Cooperative Work (CSCW'92): 107-114.

    8.  

  8. From, Franz. 1971. Perception of Other People. Columbia University Press.

    9. Chapter 2: The Varieties of Psychoid Entities

    Chapter 3: On the Perception of Action Sequences

  9. Gabarro, John J. 1990. The Development of Working Relationships. In Intellectual Teamwork: Social and Technological Foundations of Cooperative Work. Edited by J. Galegher and R. Kraut. 79-110. Hillsdale, NJ: Lawrence Erlbaum.

  10. Goffman, E. 1961. The Presentation of Self in Everyday Life. Anchor-Doubleday, New York.

  11. Grudin, Jonathan. Why groupware applications fail: problems in design and evaluation. CSCW'88.

  12. Hughes, J. A., Randall, D. and Shapiro, D. Faltering from ethnography to design. Proceedings of CSCW '92. pp. 115-122.

  13. Lofland, John, and Lyn H. Lofland. 1995. Analyzing Social Settings. New York: Wadsworth.

  14. Meyrowitz, J. 1985. No Sense of Place: The Impact of Electronic Media on Social Behavior. Oxford University Press, New York.

    15. Chapter 3: Media, Situations and Behavior

  15. Mynatt, E.D., Adler, A., Ito, M., Linde, C., O'Day, V.L. "The Network Communities of SeniorNet." Proceedings of ECSCW '99.

  16. O’Conaill, B. and Frohlich, D. 1995. Timespace in the Workplace: Dealing with Interruptions. Proceedings of CHI’95.

  17. Olson, Margrethe H., and Sara A. Bly. 1991. The Portland Experience: A Report on a Distributed Research Group. International Journal of Man-Machine Studies, 34 (2): 211-228.

  18. Strauss, A. 1991. Creating Sociological Awareness: Collective Images and Symbolic Representations. Transaction, New Brunswick, N.J.

    19. Chapter 2: Closed Awareness

    Chapter 8: Trajectory Framework for Management of Chronic Illness.

    Chapter 9: Illness Trajectories.

    Chapter 10: Negotiated Order and the Coordination of Work.

  19. Whittaker, S., Frohlich, D., and Daly-Iones, O. 1994. Informal Workplace Communication: What is it like and how might we support it? In Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI'93). ACM, New York. 131-137.

  20. Whyte, William H. 1988. City: Rediscovering the Center. New York: Doubleday.

  21. Zerubavel, Eviatar. 1981. Hidden Rhythms: Schedules and Calendars in Social Life. The University of Chicago Press.