Image Based Visual Hulls

PDF file

To Appear SIGGRAPH 2000

 

People

 

Goals

Develop fast machine vision techniques for extracting geometry from photographic input
Generate synthetic views of multiple-source data using image-based rendering methods

Visual Hull Overview

Perhaps one of the most robust ways of extracting geometry from photographic input is to use a space carving technique known as the visual hull. In this approach, one first finds the silhouette contours of a foreground object in the images. Each image region outside of the silhouette represents a region of space where the object cannot be. These regions can be carved away. The resulting volume, called the visual hull, is a conservative approximation to the actual geometry of the object. Object silhouettes are often easy to detect in images, and these methods can be quite robust.
 
A visual hull is the intersection of silhouette volumes seen from multiple points of view.

Algorithm

Out of a set of input images, fix one image as the reference image. Each pixel in the reference image represents the view along a single line of sight in 3D. This 3D line can be projected into one of the other images as a 2D line (see figure). This is the well-known epipolar construction from the computer vision literature. One can intersect this 2D line with the image regions occupied by the observed object. The result is a set of “possibly occupied” intervals on the 2D line. These intervals can be interpreted as “possibly occupied” intervals on the 3D line. The set of intervals can be refined by projecting the 3D line into each of the other images as well. The actual geometry can only reside in the intersection of the intervals computed using all of the views. When this procedure is applied to all of the pixels in the reference image, we have essentially constructed a visual hull. At each pixel, we have computed a set of depth intervals in which the volume of the object must lie. This image-based visual hull representation only requires a small quantity of floating point numbers to be stored with each pixel, and it allows us to construct a visual hull at the resolution of the input images.

 
The epipolar lines corresponding to the three points highlighted in the upper left image are drawn in the other five. 

System Design