Supplementary Material

Image Destabilization: Programmable Defocus using Lens and Sensor Motion


Ankit Mohan1    Douglas Lanman2    Shinsaku Hiura1,3    Ramesh Raskar1
1MIT Media Lab    2Brown University    3Osaka University


Abstract
We demonstrate and analyze the defocus effects produced by shifting a lens and a sensor relative to one another during an exposure. We combine this shift-based synthetic defocus with the physical defocus produced using a finite-sized aperture, yielding interesting optical effects that are difficult to replicate with other methods. We demonstrate how to simulate a larger effective aperture than that physically-supported by a given lens, achieving a shallow depth of field even using point-and-shoot and mobile phone cameras. We present methods for achieving exaggerated, programmable, and pleasing bokeh. We describe a novel method to produce a depth-independent defocus blur size over an extended range of scene distances. We analyze the case where the sensor is not parallel to the lens and obtain similar results to those produced using tilt-shift lenses. We conclude by presenting a detailed experimental analysis of a prototype implementation.


Image Destabilization Prototype

We constructed an image destabilization prototype, shown in several photographs below, containing a 12.2 megapixel Canon EOS Digital Rebel XSi camera, a Nikkor 50mm f/1.8D lens with manual aperture control, and a pair of linear translation stages driven by external stepper motors. We attached a second diverging lens behind the Nikkor lens to form a focused image on the sensor, increasing the effective focal length. The camera and lens were enclosed in a box to prevent stray light from reaching the sensor. The stepper motors and camera were computer-controlled and we ensured that the exposure occurred outside of the stepper motor ramp-up and ramp-down phases. The translation stages supported a total displacement of 4cm and typical exposures ranged from 5 to 30 seconds.

prototype image destabilization camera containing a lens and a sensor mounted on motorized linear translation stages


Qualitative Analysis of the Point Spread Function (PSF) for Various Configurations

In these examples the scene consists of an array of reflective spheres placed at increasing distances from the camera from the left to the right. A single bright point light source was located above the entrance aperture, creating an effective point source (and several reflections) as viewed from the camera. Individual PSF kernels for each configuration correspond to the blurred point sources in each image.

Qualitative Point Spread Function (PSF) Measurement using an Array of Reflective Spheres
static pinhole (approximated with an f/22 aperture) static lens with an f/2.8 aperture
pinhole laminography with a virtual focal plane in the center defocus enhancement using laminography and f/2.8 aperture
astigmatism created using laminography and a vertical slit aperture


Defocus using a Shifting Pinhole and Sensor

We demonstrate image destabilization using a pinhole aperture by stopping-down the lens to f/22. As shown below, we are able to change the effective focal length by adjusting the lens/sensor translation velocity ratio, allowing various scene planes to be brought into focus. Adjusting the total displacement, we are also able to control the effective f-number of the virtual lens. These images summarize the performance of our one-dimensional prototype for a scene containing an array of figurines at varying depths.


all-in-focus photograph with an f/22 aperture

laminography used to virtually focus in the front (10 second exposure)
laminography used to virtually focus in the middle (5 second exposure)
laminography used to virtually focus in the back (10 second exposure)

laminography used to virtually focus in the middle (5 second exposure) laminography used to virtually focus in the middle (30 second exposure)


Defocus Enhancement using a Shifting Lens and Sensor

We reduce the depth of field for a lens with an f/2.8 aperture using the method outlined in Section 3.1 in the paper. Our prototype seems to improve the aesthetic quality of the bokeh since the coordinated translation effectively applies a low-pass filter that removes high-frequency spherical aberrations artifacts. In the examples below, we once again consider an array of figurines located at increasing distances from the camera from left to right.

Preliminary Results for Defocus Enhancement using a Shifting Lens and Sensor
static f/2.8 aperture
synchronized motion applied to simulate a decreased f-number


Depth-invariant Blur Size using a Shifting Lens and Sensor

We achieve a depth-invariant blur by matching the physical blur kernel due to the lens aperture, and the synthetic blur kernel due to translating lens and sensor. We place a horizontal slit on the lens to make the PSF purely one dimensional. The lens is physically focused on the closest figure from the camera (on the left), and the virtual focal plane is at the farthest figure (on the right). As shown below, the resulting blur is approximately depth-invariant, allowing the application of non-blind image deconvolution to recover an all-in-focus image. In this example we apply Richardson-Lucy deconvolution to demonstrate preliminary results for recovering an all-in-focus image. Note that, in this example,
we used an array of figurines located at increasing distances from the camera from left to right.

Preliminary Results for Depth-invariant Blur Size using a Shifting Lens and Sensor
depth-invariant blur size
deconvolution result


Conclusion

In this paper we use motion blur (by intentionally shifting the sensor and lens) to produce effects similar to lens defocus. We combine two different types of defocus (motion-based and lens-based) to produce interesting optical effects that may be hard to achieve otherwise. While similar results might be achieved with light field cameras or camera arrays, capturing the complete 4D data is unnecessary, and to some degree excessive, for the applications we present. Furthermore, our setup captures high-resolution photos and does not suffer from discrete angular sampling artifacts. While our hardware setup might seem somewhat complicated, we believe it can be easily implemented using existing image stabilization technology already present in most cameras. We believe that greater control over the sensor and lens position during the exposure period offers enhanced flexibility useful for computational photography applications.