An Analysis of Focus Sweep for Improved 2D Motion Invariance

Yosuke Bando
TOSHIBA Corporation

Recent research on computational cameras has shown that it is possible to make motion blur nearly invariant to object speed and 2D (i.e., in-plane) motion direction, with a method called focus sweep that moves the plane of focus through a range of scene depth during exposure. Nevertheless, the focus sweep point-spread function (PSF) slightly changes its shape for different object speeds, deviating from perfect 2D motion invariance. In this paper we perform a time-varying light field analysis of the focus sweep PSF to derive a uniform frequency power assignment for varying motions, leading to a finding that perfect 2D motion invariance is possible in theory, in the limit of infinite exposure time, by designing a custom lens bokeh used for focus sweep. With simulation experiments, we verify that the use of the custom lens bokeh improves motion invariance also in practice, and show that it produces better worst-case performance than the conventional focus sweep method.

Paper (2.9MB, PDF):
Proceedings of IEEE International Workshop on Computational Cameras and Displays, 2013.


Normal camera image	Deblurred from conventional focus sweep image	Deblurred from improved focus sweep image

Focus sweep image	Error for conventional focus sweep image	Error for improved focus sweep image

Normal camera image suffers from both defocus and motion blur if the scene has moving objects at various depths (top left).
Blur can be made nearly identical irrespective of scene depth or motion by sweeping focus during exposure (bottom left).
Sharp image can be recovered by spatially-uniform non-blind deconvolution with a single, a priori known PSF (top center).
Quality of deblurred images can be improved by the proposed modification to the conventional focus sweep (top right).
Error (difference from the ground truth) is reduced, especially around the yellow fish's head (bottom center and right).

Proposed modification:

Conventional focus sweep uses a standard lens, whose defocus blur (bokeh) can be modeled as a disk (or pillbox). As this bokeh is radially symmetric, it can be concisely represented by its cross-section shown by the red line in the figure, where the vertical and horizontal axes represent intensity and radius, respectively. With this bokeh, focus sweep is only approximately motion-invariant.

In the paper, we derive a bokeh that makes focus sweep perfectly motion-invariant in theory. The derived bokeh is shown by the blue line. Because of its concave shape, we call it bowl bokeh.

Currently we have not found optics that realizes the bowl bokeh. An easy approximation is to scale it down and clip large values so it can be implemented as an attenuation mask placed at the aperture of a standard lens, at the cost of light efficiency. One example is shown by the green line, named clipped (bowl) bokeh.

Effect of the modification:

Consider a point light source moving downward during exposure.
By sweeping focus, the bokeh size changes along time while the bokeh center moves downward.
This time-varying bokeh gets integrated along time to produce a focus sweep PSF shown on the far right.

The plots show the vertical (red) and horizontal (green) profiles of the focus sweep PSFs with different bokehs.
With the standard disk bokeh, the two profiles are slightly different, indicating vertical elongation of the PSF (left).
With the proposed bowl bokeh, the two profiles are almost identical (center), which remains true for the clipped bokeh (right).
In summary:
With the bowl/clipped bokeh, motion-invariance of focus sweep improves, leading to better deblurring quality as shown above.

home