Contributions

Idea of Shader Lamps
We formally describe the idea of incorporation of three-dimensional computer graphics and animation directly into the real world all around us via explicit separation and then later merging of the physical geometry and visual attributes of a real scene.

The traditional approach for projector-based illumination has been to beam a pre-recorded two-dimensional image from a pre-determined location onto a physical object and replace only the visible texture.

Our idea introduces illumination as a basically perpspective 3D projection from a lamp enabling use of rendering with 3D computer graphics

We replace not just textures (i.e. diffuse component) but present techniques to reproduce virtually any BRDF appearance, i.e. view independent as well as view dependent effects
 

Making illumination practical
We present methods that make the process of illumination practical
We identify a radiance adjustment equation for guiding the rendering process and then present methods for the corresponding intensity correction.

We describe a method to align rendered images quickly using well-known computer vision technique

We address the issue of complete illumination by merging images from multiple projectors
 

Dealing with fragmented overlaps
We solve the problem of merging images from multiple devices in presence of depth discontinuities, so that the resultant illumination is seamless, using a new feathering algorithm

This is useful not just for Shader Lamps but also for traditional Image-Based Rendering (IBR) applications where multiple (depth-enhanced) images from different views need to be merged to render novel views. (The goal usually is to create seamless transitions from texture provided by one camera to the texture provided by the next camera.)
 

System
We demonstrate various table-top applications of varying complexity to create view-dependent reproduction of surface properties. We also display some limited animation such as movement of virtual light sources and rotation of a static symmetrical object.

We show simple extensions to walk-thru in a hybrid room-sized virtual and real environment and a 3D painting system with tracked illumination.