Many different user interfaces become possible when a video camera is added to the wearable computer. While appropriate wearable video digitizers are just now becoming available, a wireless remote processing system is currently being used to experiment with this modality (see Section 5). A camera system allows the user to have an intuitive and convenient replacement for the mouse. Figure 4 shows tracking of the user's finger while he selects an item from a pull-down menu. Here, the computer tracks the color of the user's hand, though template matching techniques are also possible at higher computational expense. Thus, the finger can replace the mouse whenever a pointing device is desired. Such a pointing device itself takes no extra room, can not be lost, and can be used almost anywhere there is light.
Figure:
Using the finger as a mouse to control the system's pointer.
The systems described above augment the traditional computer interface. However, much more can be done when the context-sensing nature of the Remembrance Agent is extended to the physical world. Much of a user's life is spent away from a desk. Thus, the computer must be adapted to take advantage of user mobility.