Figure 1: Remote has laser for pointing and five simple buttons.
Lee W. Campbell
MAS 837
Room E15-383, The Media Laboratory
Massachusetts Institute of Technology
20 Ames St., Cambridge, MA 02139
People have an urge to control their environment. This shows up in the popularity of interior redecoration for the home and office, and in the wearing of personal sound systems for control of one's audio environment. Though some kinds of environmental control serve physical or emotional comfort, other kinds are important for safety. For example, control over access (e.g. door and window locks), winter heat, and summer air conditioning are essential.
Although easy fingertip controls such as remote controls are a luxury for the able-bodied (e.g. the couch potato), they can be very important to the dignity, security, and well-being of elderly or disabled people. One can imagine a situation where a person has lost some of his/her physical dexterity or mobility, and in the absence of suitable controls, would need a helper to manage her/his home, with the attendant expense and loss of independence and privacy. But with adequate automation this person might be able to stay independent.
Present day remote controls suffer from several problems: they have too many buttons and too few uses. Universal remotes are hard to learn and may require several button presses to achieve a simple function, while equipment specific remotes don't control enough devices so one must keep multiple remotes handy.
I propose a design of a universal remote with built-in laser pointer which greatly simplifies the control of multiple devices; the design philosophy is that it should take near zero time to learn. I also propose several possible technologies to implement the remote.
Figure 1: Remote has laser for pointing and five simple buttons.
The remote control should have only five buttons: left, right, up, down, and center (See Figure 1), plus a safety off switch. the devices to be controlled include: TV, VCR, cable box, loudspeakers, light fixture or switch, doorlock, thermostat, radiator, and air conditioner. The remote works as follows:
Some functions can be managed by dragging from one device to another; for example, to select what sound source should be selected, one can drag from tape deck or CD player to the speakers; like drawing a line with the laser from one to the other.
The simple commands probably cover 98% of the times you need to control something; the rest of the controls for devices are reached through shortcuts or onscreen menus. Onscreen menus are brought up by dragging from a device to the screen. Drawing the speakers to the screen would bring up all speaker controls: volume, balance, bass, treble, and loudness, perhaps as bar graphs. These controls could then be dragged up and down by the laser, holding the center button to drag, or they could be stepped up and down with the up/down button. Similarly, drawing the preamplifier to the screen would bring up all its controls; possibly on several pages. Again, the laser would act like a mouse, with the center button selecting and dragging. Left and right buttons would cycle through the pages if there were a multi page control. To bring up onscreen TV controls, wiggle the laser across the screen.
It is also possible to use the laser to draw symbols such as letters of the alphabet. These could serve as user-programmable shortcuts to frequently used commands. For example, a user could associate the letters B and T with bass and treble controls. Drawing M on the TV screen could bring up a master menu of all onscreen menus. The shortcuts would be programmed by the user in special onscreen menus.
The only hard problem is how the system figures out what device the laser is pointing at. Once this information is known, everything else is easy. There are three main ways to implement the remote: sensitive patches, position and attitude sensing, and machine vision. I shall discuss all three.
In this approach there would be a patch analogous to a bar code for each controllable device. Sweeping the laser across the patch would modulate the reflected laser light and indicate which device was to be controlled. This information would be picked up by the remote and included in the message it sends.
The problems with this approach are: the user must hit a particular patch, which is less intuitive than just hitting anywhere on the device; this method doesn't allow onscreen menus; and the presence of the patch would have a deleterious effect on the visual design of some devices, particularly light fixtures.
Nevertheless, bar code reading is a useful way to get data into the remote, and is good for teaching the remote about a new device to be controlled.
In this approach there would be Polhemus type sensors in the remote that would sense its position and attitude and thus determine where the laser is pointing. The disadvantage here is that the system would need a 3D model of each room, complete with locations of each device to be controlled. Although a 3D model could be built, it would make the training task too difficult.
One way to use machine vision would be to put a camera in the remote, looking down the laser beam and recognizing the devices being pointed at. However, the general case of recognizing objects in arbitrary orientations and lighting conditions in an unsolved problem. Instead we will use fixed wide angle video cameras in each room determining what the laser is pointing at.
Each device is represented as a rectangle or other simple shape in the 2D viewing field of a camera. When the laser is in or near the object shape, the camera decides that device is being controlled. Detailed imaging is unimportant; the cameras can have fisheye lenses so only one or two are needed to cover most rooms. To make the laser spots more visible, the camera lenses should be fitted with narrowband filters centered at the laser wavelength. finally, the laser should be modulated at half the frame rate of the cameras so that frame differencing will show the laser spot while removing all bright but unchanging images from the scene.
The only remaining task is to train the vision system to map locations in its camera images to devices to be controlled. This can be done in a one--time training process (as long as equipment is not moved) where the user takes the laser and draws the outline of each device to be controlled, starting with the TV screen, and enters an identifier for the device.
In this implementation, the remote is ``dumb;'' it points with the laser and tells the rest of the system what button is pressed by sending signals with its IR transmitters. The built-in system has the intelligence; it identifies the target and translates and re-sends commands to devices (like the ``cable mouse'' that comes with some VCRs).
Thus the equipment required include: 1 or 2 cameras per room with wide angle lenses and narrowband filters; 1 or 2 IR transceivers per room; System controller box wiring them all together, connected to TV.
If the remote can control essential functions like locks and HVAC, it must be secure from hacking and misuse. Otherwise, thieves might be able to shine a remote in through a window and cause doors to unlock or draw enough power to trip circuit breakers and shut down alarm systems. The way to protect against this is encryption: the remote must share a key with the receiving controller that identifies it uniquely. In addition, the messages from the remote should be ``salted'' with a time of day and date so that the signal from the remote cannot be recorded and played back at another time. Finally, the remote should identify the user holding it so that a stolen remote ceases to operate security devices such as door locks. At present I don't know a way to implement this. Instead, the remote can be made to only operate security devices when it is inside the house.
It is possible at present to pick up stray electromagnetic signals from TV monitors at a distance of a few hundred meters and reproduce what is displayed (unless the monitor meets the US government's TEMPEST shielding standards for electronic equipment); so one can imagine more sensitive devices picking up the stray emissions of cameras themselves. To protect against this kind of privacy invasion, the technique described above of narrowband filters, pulsed lasers, and a differencing camera are helpful because the images returned by the camera will show almost nothing in the room except the laser spot. Thus the same methods that remove clutter for the computer vision algorithms also protect privacy of the user.
This system could be built with present-day technology but the cost is high, and it still has problems with frequently moved devices (e.g. a TV on a rolling cart). Some movable devices could be fitted with IR LEDs that allow the vision system to determine position and orientation of the device. It is hoped that the large ``couch potato'' market will drive costs down so that the system can become a real benefit to elderly and disabled.
A More ``universal'' Remote Control
This document was generated using the LaTeX2HTML translator Version 95.1 (Fri Jan 20 1995) Copyright © 1993, 1994, Nikos Drakos, Computer Based Learning Unit, University of Leeds.
The command line arguments were:
latex2html -split 0 remote.tex.
The translation was initiated by Lee W Campbell on Fri May 23 15:23:59 EDT 1997