Eyeglasses-free Display: Towards Correcting Visual Aberrations with Computational Light Field Displays

Fu-Chung Huang1 Gordon Wetzstein2 Brian Barsky1 Ramesh Raskar2
1UC Berkeley - 2MIT Media Lab

SIGGRAPH 2014. ACM Transactions on Graphics 33(4).


Abstract

Millions of people worldwide need glasses or contact lenses to see or read properly. We introduce a computational display technology that predistorts the presented content for an observer, so that the target image is perceived without the need for eyewear. By designing optics in concert with prefiltering algorithms, the proposed display architecture achieves significantly higher resolution and contrast than prior approaches to vision-correcting image display. We demonstrate that inexpensive light field displays driven by efficient implementations of 4D prefiltering algorithms can produce the desired vision-corrected imagery, even for higher-order aberrations that are difficult to be corrected with glasses. The proposed computational display architecture is evaluated in simulation and with a low-cost prototype device.


In the News

MIT News - Vision-correcting displays: Technology could lead to e-readers, smartphones, and displays that let users dispense with glasses, July 2014

IEEE Spectrum - No Need for Reading Glasses With Vision-Correcting Display, July 2014

BBC NEWS - Display screen technology could correct vision problems, July 2014

Technology Review - Prototype Display Lets You Say Goodbye to Reading Glasses, July 2014

WIRED - MIT Develops a Phone Screen That Corrects Poor Vision, July 2014

WIRED UK - Future phone screens could correct defective vision, July 2014

FOX NEWS - Goodbye, glasses: Future smartphone screens could correct vision, August 2014

Discovery NEWS - Toss Your Glasses: Display Sharpens Blurry Images, August 2014

Engadget - Experimental display lets glasses wearers ditch the specs, August 2014

CNET - Could this vision-correcting display replace eyeglasses?, August 2014

Gizmag - Vision-correcting display lets users ditch their reading glasses, August 2014

...



Application Scenarios







Technical Video


Files

Citation

F. Huang, G. Wetzstein, B. Barsky, R. Raskar. Eyeglasses-free Display: Towards Correcting Visual Aberrations with Computational Light Field Displays. ACM Proc. of SIGGRAPH (Transactions on Graphics 33, 4), 2014.

BibTeX
@article{Huang:2014:VisionCorrectingDisplay,
author = {F. Huang and G. Wetzstein and B. Barsky and R. Raskar},
title = {{Eyeglasses-free Display: Towards Correcting Visual Aberrations with Computational Light Field Displays}},
journal = {ACM Trans. Graph. (Proc. SIGGRAPH)},
volume = {33},
number = {4},
year = {2014},
publisher = {ACM},
pages = {1--12},
address = {New York, NY, USA}
}

Presentation Slides



Additional Information

Vision correction with computational displays. On a conventional screen, people with optical aberrations see a blurred image (center left). Current approaches to aberration-correcting display use multilayer prefiltering (center) or light field displays (center right). While the former technology enhances perceived image sharpness, contrast is severely reduced. Existing light field-based solutions offer high contrast but require a very high angular sampling density, which significantly reduces image resolution. In this paper, we explore the convergence of light field display optics and computational prefiltering (right), which achieves high image resolution and contrast simultaneously.

Image credit: MIT Media Lab, Camera Culture Group

Illustration of vision-correcting displays. Observing a conventional 2D display outside the focal range of the eye results in a blurred image (top). A multilayer display with prefiltered image generation (second row) allows for improved image sharpness at the cost of reduced contrast. Image contrast can be preserved using a light field approach via lenslet arrays on the screen (third row); this approach severely reduces image resolution. Combining light field display and computational prefiltering, as proposed in this paper (bottom), allows for vision-correcting image display with significantly improved image resolution and contrast.

Image credit: MIT Media Lab, Camera Culture Group



Light field analysis for different displays. The light field emitted by a display is parameterized by its coordinates on the screen xd, on the pupil u, and on the retina x (a). This light field propagates through the pupil and is projected into a 2D image on the retina. For an in-focus display, the light field incident on the retina is a horizontal line in the frequency domain (b). For a displayed image outside the accommodation range of the observer, the corresponding light field is slanted and energy is lost at some spatial frequencies (c). Multilayer displays utilize an additional display layer to preserve all spatial frequencies (d). With light field displays, frequency loss is also avoided; the perceived image frequencies are a combination of all spatio-angular frequencies of the incident light field (e). The ray paths in (a) show two effects for a hyperopic eye observing a light field display. First, each photoreceptor on the retina averages over multiple neighboring pixels on the screen (green shaded regions). Second, each pixel on the screen (e.g., xd0) emits different intensities toward different regions on the pupil (u0,u1), allowing the same pixel to appear differently when observed from different locations (x0,x1) on the retina (red arrows).

Image credit: MIT Media Lab, Camera Culture Group



Light field prefiltering. The proposed prefiltering approach computes a light field (here with 3x3 views) that results in a desired 2D projection on the retina of an observer. The prefiltered light field for an example scene is shown in (a), its simulated projection on the retina in (c), and an image observed on a conventional screen in (b). Spatio-angular frequencies of the light field are amplified, resulting in the desired sharpening when integrated on the retina. Two sample "flatland" light fields with different angular sampling rates are shown in display (d,f) and in eye (e,g) coordinates. Here, the yellow boxes illustrate why 4D light field prefiltering is more powerful than 2D image prefiltering: a single region on the retina receives contributions from multiple different light field views (e,g). Wherever that is the case, the inverse problem of light field prefiltering is well-posed but in other regions the problem is the same as the ill-posed problem faced with conventional 2D displays (e).

Image credit: MIT Media Lab, Camera Culture Group



Conditioning analysis. The light field projection matrix corresponding to a defocused eye is ill-conditioned. With more angular resolution available in the emitted light field, more degrees of freedom are added to the system, resulting in lower condition numbers (lower is better). The condition number of the projection matrix is plotted for a varying defocus distance (kernel size) and angular resolution (number of light field views). We observe that even as few as 1.5 angular light field samples entering the pupil of an observer decrease the condition number.

Image credit: MIT Media Lab, Camera Culture Group



Tradeoff between angular light field resolution and image contrast. Top: we reconstruct a test image with different combinations of angular resolution and image contrast and plot the achieved PSNR. Bottom: using prefiltering with a conventional 2D display (b), we obtain either a low-quality but high-contrast image or a high-quality but low-contrast image. For a light field display with 1.5 or more prefiltered views entering the pupil (c), a similar trend is observed but overall reconstruction quality is significantly increased. (Snellen chart courtesy of Jeff Dahl)

Image credit: MIT Media Lab, Camera Culture Group



Compensating for a range of lateral viewpoints. Aberration-free image display is possible when the relative position of the eye with respect to the display is known. The green plot evaluates image degradation for viewpoints that deviate laterally from the sweetspot. Only slight ringing is visible and, due to periodic viewing zones of the employed parallax barrier display, image quality varies in a periodic manner (top, zoom-in). We can account for a range of perspectives in the compensation, ensuring high image quality for a wider viewing range (blue plot). The columns on the bottom right show on-axis and off-axis views with and without accounting for a range of lateral viewpoints in the optimization. (Source image by Picasso (Wikipedia))

Image credit: MIT Media Lab, Camera Culture Group



Accounting for a range of viewing distances. Top row: when considering a fixed viewing distance, defocus errors are compensated at that exact distance (top center) but image quality degrades when the observer moves forward or back (top left and right). The proposed method can account for a range of viewing distances (bottom row), which slightly degrades quality at the sweetspot but significantly improves all other distances. (Source image courtesy of KarHan Tan)

Image credit: MIT Media Lab, Camera Culture Group



Prototype display. We construct an aberration-correcting display using parallax barriers. The barrier mask contains a pinhole array (left) that is mounted at a slight offset in front of an Apple iPod touch 4 screen (lower right). The display emits a light field with a high-enough angular resolution so that at least two views enter the pupil of a human observer. This effect is illustrated on the top right: multiple Arabic numerals are emitted in different viewing directions; the finite pupil size then creates an average of multiple different views on the retina (here simulated with a camera).

Image credit: MIT Media Lab, Camera Culture Group



Photographs of prototype display. The hyperopic camera simulates a human pupil with a diameter of 6 mm at a distance of 25 cm to the screen. Focused at 38 cm, images shown on a conventional screen are blurred (first column). While previous methods theoretically facilitate increased image sharpness (second column), achievable resolution is fundamentally limited by the spatio-angular resolution tradeoff of the required light field display. Light field prefiltering, as proposed in this paper, allows for significantly increased resolutions (third column). The prototype closely resembles simulations (right column). (From top, source images by dfbphotos (flickr), van Gogh (Wikipedia), Houang Stephane (flickr), JFXie (flickr), Jameziecakes (flickr), Cezanne (Wikipedia), Picasso (Wikipedia), Matisse (Wikipedia))

Image credit: MIT Media Lab, Camera Culture Group



Evaluation and comparison to previous work. We compare simulations of conventional and vision-correcting image display qualitatively and quantitatively using contrast and quality-mean-opinion-square (QMOS) error metrics. A conventional out-of-focus display always appears blurred (second column). Multilayer displays with prefiltering improve image sharpness but at a much lower contrast (third column). Light field displays without prefiltering require high angular resolutions, hence provide a low spatial resolution (fourth column). The proposed method combines prefiltering and light field display to optimize image contrast and sharpness (right column). The QMOS error metric is a perceptually linear metric, predicting perceived quality for a human observer. We also plot maps that illustrate the probability of an observer detecting the difference of a displayed image to the target image (bottom row). Our method performs best in most cases. (Source images courtesy of flickr users Jameziecakes, KarHan Tan, Mostaque Chowdhury, Thomas Quine (from top))

Image credit: MIT Media Lab, Camera Culture Group



Correcting for higher-order aberrations. We simulate images observers with different types of higher-order aberrations perceive (top row) and show corresponding point spread functions (top row, insets), which exhibit a range of different shapes. Most of them are difficult to compensate with a conventional 2D display (bottom row, lower left parts), although the blur kernel associated with trefoil (lower right) is frequency preserving and therefore invertible. The proposed aberration-correcting display is successful in compensating all of these aberrations (bottom row, upper right parts). (Source image courtesy of flickr user Jameziecakes)

Image credit: MIT Media Lab, Camera Culture Group





Contact

Technical Details
Gordon Wetzstein, PhD
MIT Media Lab
gordonw (at) media.mit.edu




Press
Alexandra Kahn, Senior Press Liaison, MIT Media Lab
akahn (at) media.mit.edu or 617/253.0365


Related Projects

Compressive Light Field Photography Tensor Displays Layered 3D Polarization Fields Light Field Projection
Compressive Light Field Photography (SIG '13)
Tensor Displays (SIG '12)
Layered 3D (SIG '11)
Polarization Fields (SIG Asia '11)
Light Field Projection (SIG '14)

MIT Media Lab - All rights reserved.