Detection and Highlighting using a Multi-Flash Technique, 2002
Imagine a camera, no
larger than existing digital cameras, that can directly find depth
edges or create stylized images. As we know, a flash to the left of a
camera creates a sliver of shadow to the right of each silhouette
(depth discontinuity) in the image. We add a flash on the right, which
creates a sliver of shadow to the left of each silhouette, a flash to
the top and bottom. By observing the shadows, one can robustly find all
the pixels corresponding to shape boundaries
(depth discontinuities). This is a strikingly simple way of calculating
edges. Below, we show one potential application in generating NPR
We present a
non-photorealistic rendering approach to capture and
convey shape features of real-world scenes. We use a camera with
multiple flashes that are strategically positioned to cast shadows
along depth discontinuities in the scene. The projective-geometric
relationship of the camera-flash setup is then exploited to detect
depth discontinuities and distinguish them from intensity edges due
to material discontinuities.
We introduce depiction methods that utilize the detected edge
features to generate stylized static and animated images. We can
highlight the detected features, suppress unnecessary details or
combine features from multiple images. The resulting images more
clearly convey the 3D structure of the imaged scenes.
We take a very different approach to capturing geometric features
of a scene than traditional approaches that require reconstructing
a 3D model. This results in a method that is both surprisingly
simple and computationally efficient. The entire hardware/software
setup can conceivably be packaged into a self-contained device no
larger than existing digital cameras.
Discontinuity Confidence Map created by Multi-Flash Camera
Notice the individual leaves now clearly visible
shape clarifying image
raw depth edge confidence map
Notice the four spark plugs, dip
stick, shape of engine next to dip stick, and the Honda sign now
visible in this shape clarifying image.
However, the disadvantage, common to NPR techniques, is that some
texture detail is lost.
Raw Result of Our Method for
Depth Edge Detection (with no post-processing)
One type of rendering
Comparison with Intensity Edge
detection (Canny) results
We were surprised by
the simplicity of our depth edge detection solution. We found it
difficult to believe that, while photometric stereo techniques solved
the more difficult problem of computing surface orientation (and
indirectly the depth values), procedures to directly find the depth
discontinuity were rarely explored.
Hence, after developing
this technique in 2002, we discussed with various researchers at CVPR
2003 (where it was also shown as a demo), ICCV 2003 and Siggraph 2003
(where the basic ideas of depth edge detected were presented as a
sketch) and asked for feedback. We concluded that, one possible reason
this type of strategic placement of light sources very close to the
camera was overlooked is that, in-fact for most photometric stereo or
shape-from-shadow or shape-from-shading technique, the flash
configuration is a failure case. However, beyond our search and
discussion with researchers, it is quite possible that a approach
to ours was explored in 60's or 70's. So we asked researchers who have
active since 70's who refered us to papers that analyzed intensity
in images but those techniques required widely varying
illumination on objects with uniform albedo.
For most techniques to
compare with, a
simple question to ask is : Will the method find a depth edge for a
white piece of paper in front of a white background ?
Here is what Stan Birchfield
(Ph.D. Thesis 'Depth and Motion Discontinuities', Stanford University,
1999) has to say about our work (printed with permission).
"Detecting depth discontinuities is fundamental to image understanding,
and important for many tasks. With a well-crafted hardware
modification, this work has moved this problem from theoretical
possibility to practical reality. The results are orders of
magnitude better than anything previous."
If you are familiar with
related techniques, we would love to hear from you.