Virtual reality puts you in a
computer-generated world. Augmented reality allows for a mix of virtual
reality and the real world by placing computer-generated objects in a
Mitsubishi Electric Research Labs have brought dynamic, computer-generated
labels into the physical world with a combination of radio frequency
identification (RFID) tags and portable projectors.
Their Radio Frequency Identity and Geometry (RFIG)
system consists of a hand-held projector that shines dynamic images onto
physical objects of the user's preference, and radio frequency
identification tags augmented with photosensors, which identify objects
for the projector. Radio frequency identification tags contain tiny,
inexpensive chips that are read using radio waves. Photosensors detect
The system can be used to
find and track inventory, guide robots or precision handling systems on
assembly lines, locate small instruments and track movement of items in
health care settings, keep track of objects in homes, offices and
libraries, and enable games to integrate real and virtual objects, said
Ramesh Raskar, a research scientist at Mitsubishi Electric Research Labs.
The radio identification/photosensor tags can
be as small as a grain of rice, said Raskar.
To find an object, a user aims a radio frequency
reader in the general direction of a collection of tagged objects. Each
tag that is in range is activated by the radio frequency signal, prompting
its photosensor to take a reading of the existing light. Once this is
done, the projector embedded in the reader turns on, and each tag that
detects an increase in illumination sends a response indicating that it is
in the projector beam and is ready for interaction.
The projector then beams a sequence of about 20
images of horizontal or vertical bars of varying density, which form
unique codes indicating horizontal and vertical coordinates. Each tag
records the code, then transmits its identity plus the code back to the
radio frequency reader. This allows the reader to determine the location
of each tag in its range.
The projector then
marks the appropriate tags for the user. The projector image is dark where
there are no tags, and illuminates areas where there are tags, said
Raskar. The tags are accurate to within a millimeter, and can be used to
find objects and detect when objects have been moved, said Raskar.
If an object has several radio tags attached to
different surfaces, the system can be used to track the object's
orientation and shape, for example detecting when an object has shifted or
been deformed, said Raskar.
which the researchers developed previously, includes inertial sensors that
track its position, making it possible to project images from a moving
projector and compensate for the movements. The projector can also hold
one portion of a projection steady while allowing another portion to move,
making it possible to project a steady shot of a computer screen and at
the same time project a mouse pointer that the user can control by moving
the entire projector.
The projector can also
capture an image of an object or sets of objects including tag
information, then project the image on another surface. This cut-and-paste
capability is useful for interacting with tags that are in places that are
not suitable for projections, said Raskar.
radio frequency identity and geometry system solves three interface
problems, said Raskar: it locates tags precisely and can give visual
feedback, it allows the user to see a subset of tags among many tags, and
it allows the user to differentiate among multiple tags responding to a
A warehouse worker,
for instance, could use the system to view the tag information on a set of
objects and digitally mark the tags of selected objects with instructions
for coworkers, said Raskar. Or a librarian could use the system to
identify which books are upside-down or out of order, he said.
The system is the reverse of the more common idea of
putting beacons like light-emitting diodes on the objects themselves so
they can be located visually, said Raskar. "We turned this around and said
'what if the tiny indiscernible tag does not have a battery attached to
it... can it still be located and visually indicated?'"
The system can also be thought of as the reverse of
computer vision, said Raskar. Traditional computer vision uses a
centralized, dense, two-dimensional array of pixels to record information
about a visual field, and the cameras recording the field are outside the
field looking in. The researchers' tags, in contrast, are essentially
single-pixel cameras pasted on objects, and these cameras are looking out,
toward the projector.
Because radio frequency
identification tags can be tiny and require no power source, they could be
embedded in packaging to make automated tracking of even inexpensive
Most of the
researchers' prototype work was done with active, battery-powered radio
frequency ID tags because these can be programmed, but the system can be
adapted for passive ones as well, said Raskar. In addition, the
photosensors could be used to help power passive tags, he said.
The system's radio frequency tags will cost a little
more than ordinary radio frequency tags because they contain a
photosensor, and it will be two or three years before the passive sensing
tags are available at reasonable costs and in quantity, said Raskar.
Initially the tags will be useful in inventory
management, said Raskar. The system is really geared towards the first 10
meters and the last 10 meters of the tag's productive life -- initiation
of tags at the time of production, and locating objects in a warehouse or
human-interaction in cluttered environments, he said.
Within a decade most objects that are bar-coded
today will instead carry radio frequency tags, he said. "Adding a
geometric notion will be appealing for several applications," including
factory automation, health care, entertainment, and even surveillance,
The researchers' next step is to
put together a system that would allow more than one reader to work with a
set of tags at a time, said Raskar.
research colleagues were Paul Beardsley, Jeroen van Baar, Yao Wang, Paul
Deitz, Johnny Lee, Darren Leigh and Thomas Willwacher. The researchers are
scheduled to present the work at the Association of Computing Machinery
(ACM) Siggraph 2004 conference in Los Angeles on August 8 to 12. The
research was funded by Mitsubishi Electric Research Labs.
Timeline: 2-3 years, 10 years
TRN Categories: Applied Technology;
Story Type: News
Related Elements: Technical paper, "RFID Lamps:
Interacting with a Self-Describing World via Photosensing Wireless Tags
and Projectors," Association of Computing Machinery (ACM) Siggraph 2004
conference, Los Angeles, August 8-12