[curriculum vitae]
[Light Field Archive]
[Display Blocks]

Gordon Wetzstein
Research Scientist
MIT Media Lab

75 Amherst St, E14-474C
Cambridge, MA 02139

phone: 617.715.4683
email: gordonw (at) media (dot) mit (dot) edu

Image Courtesy of The Boston Globe

I will be starting as an Assistant Professor at Stanford University (EE) in September 2014. I am looking for motivated and experienced students, postdocs, and interns. Send me an email if you are interested!

Short Biography: Gordon Wetzstein is a Research Scientist in the Camera Culture Group at the MIT Media Lab. His research focuses on computational imaging and display systems as well as computational light transport. At the intersection of computer graphics, machine vision, optics, scientific computing, and perception, this research has a wide range of applications in next-generation consumer electronics, scientific imaging, human-computer interaction, remote sensing, and many other areas. Gordon's cross-disciplinary approach to research has been funded by DARPA, NSF, Samsung, and other grants from industry sponsors and research councils. In 2006, Gordon graduated with Honors from the Bauhaus in Weimar, Germany, and he received a Ph.D. in Computer Science from the University of British Columbia in 2011. His doctoral dissertation focuses on computational light modulation for image acquisition and display and won the Alain Fournier Ph.D. Dissertation Annual Award. He organized the IEEE 2012 and 2013 International Workshops on Computational Cameras and Displays, founded displayblocks.org as a forum for sharing computational display design instructions with the DIY community, and presented a number of courses on Computational Displays and Computational Photography at ACM SIGGRAPH. Gordon won the best paper award for "Hand-Held Schlieren Photography with Light Field Probes" at ICCP 2011 and a Laval Virtual Award in 2005.

Please see my curriculum vitae for more details.

Research Highlights

The core goal of my research is the design and implementation of novel approaches that push the boundaries of conventional light capture and display technology through the co-design of optics and computational processing. Such computational imaging systems facilitate new capabilities such as super-resolution, high dynamic range, glasses-free 3D display, extended depth of field photography and projection, lensless imaging, optical computing and image processing, acquiring and reconstructing refractive phenomena, and seamless projection on complex surfaces. Practical application is an important factor in my research; much of it has been conducted in close collaboration with some of the top academic and industrial research institutes worldwide.

Several research projects are highlighted below, please see the research page for a detailed list.

Computational Displays
Eyeglasses-free Display (ACM SIGGRAPH 2014)

Millions of people worldwide need glasses or contact lenses to see or read properly. We introduce a computational display technology that predistorts the presented content for an observer, so that the target image is perceived without the need for eyewear. We demonstrate a low-cost prototype that can correct myopia, hyperopia, astigmatism, and even higher-order aberrations that are difficult to correct with glasses.

Compressive Light Field Projection (ACM SIGGRAPH 2014, Paper & ETech)

A compressive approach to light field synthesis with projection devices. We propose a novel, passive screen design that is combined with high-speed light field projection and nonnegative light field factorization. We demonstrate that the projector can alternatively achieve super-resolved and high dynamic range 2D image display when used with a conventional screen.

Focus 3D (ACM Trans. Graph. 2013 & SIGGRAPH 2014)

A new computational light field (glasses-free 3D) display that supports accommodative depth cues in addition to motion parallax and binocular cues. Building on multilayer attenuator and directional backlight architectures, the proposed design uses nonnegative light field tensor factorization to achieve compressive image display.

Tensor Displays (ACM SIGGRAPH 2012, Paper & ETech)

A new family of compressive light field displays comprising all architectures employing stacks of high-speed LCDs illuminated by uniform or directional backlighting (i.e., any low-resolution light field emitter). Supports wider fields of view. larger depths of field, and thinner form factors than previous layered displays. SIGGRAPH 2012 Paper Highlight, Emerging Technologies Highlight, and featured in New Scientist, The Boston Globe, engadget, slashdot, and many more (see below)!

Layered 3D (ACM SIGGRAPH 2011)

A compressive light field display using stacked, inkjet-printed transparencies in combination with computed tomographic light field synthesis. This design allows for the fabrication of inexpensive, high-resolution, and bright light field displays showing static content. Real-time OpenGL and offline Matlab code available. SIGGRAPH 2011 Paper Highlight, Proceedings Cover Feature, and featured in The Boston Globe!

Computational Imaging
Light Field Microscopy (Nature Methods 2014)

High-speed, large-scale three-dimensional (3D) imaging of neuronal activity poses a major challenge in neuroscience. Here we demonstrate simultaneous functional imaging of neuronal activity at single-neuron resolution in an entire Caenorhabditis elegans and in larval zebrafish brain. Our technique captures the dynamics of spiking neurons in volumes of ~700 um 700 um 200 um at 20 Hz. Its simplicity makes it an attractive tool for high-speed volumetric calcium imaging. Featured on MIT Homepage and by MIT News!

Compressive Light Field Photography (ACM SIGGRAPH 2013 and ICCP 2014)

A single-shot approach to high-resolution light field photography. We introduce light field atoms as a sparse representation for natural light fields and derive optimized optical systems to acquite them with commodity hardware in a single image. Best Paper Award! Featured on MIT Homepage and by MIT News!

A Theory of Plenoptic Multiplexing (IJCV 2013 and IEEE CVPR 2010)

A general theory of multiplexing the dimensions of the plenoptic function onto an image sensor enabling principled comparisons of plenopticc multiplexing schemes, including noise analysis, as well as the development of unifying reconstruction algorithms.

Ultra-fast Lensless Computational Imaging (IJCV 2013 and ECCV 2012)

A frequency analysis of light transport considering spatial, angular, and temporal light variation. Based on this analysis, a new computational camera design is introduced: lensless imaging using an ultra fast sensor and pulsed laser illumination.

Computational Light Transport
Transparent Object Reconstruction via Coded Transport of Intensity (IEEE CVPR 2014, Oral)

An intuitive formulation of light transport in refractive media using light fields and the transport of intensity equation. We show how coded illumination in combination with pairs of recorded images allow for robust computational reconstruction of dynamic two and three-dimensional refractive phenomena.

Adaptive Image Synthesis for Compressive Displays (ACM SIGGRAPH 2013)

An integrated approach to generate high-quality images and light fields for emerging compressive displays. The algorithm unifies sampling, rendering, and display-adaptive optimization to increase performance by orders of magnitude. We show applications for compressive light field displays, high dynamic range displays, and super-resolution displays. This work is the first step toward a graphics pipeline tailored for next-generation displays.

Schlieren Photograph with Light Field Probes (IJCV 2013, ICCV 2011, and ICCP 2011)

A new approach to capturing and reconstructing transparent, refractive media. High-dimensional light field probes optically encode refraction in variations of color and intensity in captured photographs, which allows dynamic phenomena to be reconstructed from a single shot. Best Paper Award!

Optical Image Processing (Computer Graphics Forum 2011)

The sunglasses, windshields, and motorcycle visors of the future: transparent, light modulating displays replace the glass and are computationally driven to optically enhance scene contrast, highlight objects of interest, or de-metamerize colors.

Courses Overview

Mathematical Methods in Imaging (MIT Media Lab, MAS 132/532, Spring 2013 and Spring 2014)

This course deals with both optical and computational aspects in imaging, starting with classical imaging modalities and image processing formulations and ending with innovative and state of the art techniques for computation imaging and object description. We will discuss various mathematical aspects, including fourier optics, sparse representations and variational and geometric models. We will forray as well to describe non-standard and computational photography techniques such as including light field imaging.

Computational Cameras and Photography (MIT Media Lab, MAS 131/531, Fall 2012)

In this couse we will study the emerging multi-disciplinary field of computational photography - one which is at the intersection of signal processing, applied optics, computer graphics and vision, electronics, art, and online sharing through social networks. We will examine whether innovative camera-like sensors can overcome the tough problems in scene understanding and generate insightful awareness. In addition, we will develop new algorithms to exploit unusual optics, programmable wavelength control, and femto-second accurate photon counting to decompose the sensed values into perceptually critical elements.

Future of Imaging (MIT Media Lab, MAS 132/532, Spring 2012)

In this course, we will survey the landscape of imaging techniques and learn how to conduct research in imaging. With more than a billion people with networked, mobile cameras in their hands, we are seeing a rapid evolution in activities based on visual exchange. People's daily activities are increasingly based on pervasive recording and eager consumption of images and video. We will look at the technical as well the social aspects of this rapidly evolving camera culture.

Computational Displays: Combining Optical Fabrication, Computational Processing, and Perceptual Tricks to Build the Displays of the Future (ACM SIGGRAPH 2012 Course, Eurographics 2013 Tutorial)

This course serves as an introduction to the emerging field of computational displays. The pedagogical goal of this course is to provide the audience with the tools necessary to expand their research endeavors by providing step-by-step instructions on all aspects of computational displays: display optics, mathematical analysis, efficient computational processing, computational perception, and, most importantly, the effective combination of all these aspects. Specifically, we discuss a wide variety of different applications and hardware setups of computational displays, including high dynamic range displays, advanced projection systems as well as glasses-free 3D display. SIGGRAPH 2012 Courses Highlight!

Computational Plenoptic Imaging (ACM SIGGRAPH 2012 Course, Eurographics 2011 STAR & Tutorial)

This course is intended to review the state of the art in joint optical light modulation and computational reconstruction of visual information transcending that captured by traditional photography. In contrast to prior courses on general computational photography this course gives a broad, well-structured, and intuitive overview of all aspects of plenoptic image acquisition and focuses on two recent developments: light field acquisition and ultra-fast cameras. We unveil the secrets behind capturing light at a trillion frames per second and the Lytro camera. This course serves as a resource for interested parties by providing a categorization of recent research and help in the identification of unexplored areas in the field.

Recent Talks

KEYNOTE: Compressive displays - combining optical fabrication, computational processing, and perceptual tricks to build the displays of the future (2014, SPIE Stereoscopic Displays & Applications)

In this talk, we explore modern approaches to glasses-free 3D display using compressive light field displays. In contrast to conventional technology, compressive displays aim for a joint-design of optics, electronics, and computational processing that together exploit compressibility of the presented data. For instance, multiview images or light fields show the same 3D scene from different perspectives - all these images are very similar and therefore compressible. By combining displays that use multilayer architectures or directional backlighting combined with optimal light field factorizations, limitations of existing devices, for instance resolution, depth of field, and field of view, can be overcome. In addition to light field display, we will discuss approaches to compressive super-resolution image display and compressive high dynamic range display. As with compressive light field displays, these technologies rely on multiplexing image content in time such that the visual system of a human observer combines presented patterns into a consistent 3D, high-resolution, or high-contrast image. With the invention of integral imaging and parallax barriers in the beginning of the 20th century, glasses-free 3D displays have become feasible. With rapid advances in optical fabrication, digital processing power, and computational perception, a new generation of display technology is emerging: compressive displays exploring the co-design of optical elements and computational processing while taking particular characteristics of the human visual system into account. We will review these techniques and also give an outlook on next-generation compressive light field camera technology. Slides are available here.

KEYNOTE: Compressive Light Field Displays (2013, CUSO Winter School on Computational Photography and Display, Lenk, Switzerland)

In this talk, we explore next-generation light field or glasses-free 3D display technology. Through the co-design of display optics and computational processing, compressive light field displays allow for thinner form factors, higher resolutions, better 3D effects, and higher contrast that previous 3D display technologies. We discuss how psycho-physiological aspects of human vision are important for display design and demonstrate how perceptually-driven 3D displays can enhance the capability of current technology even further.

Computational Light Field Displays (2012, Invited Talks)

With the invention of integral imaging and parallax barriers in the beginning of the 20th century, glasses-free 3D displays have become feasible. Only today - more than a century later - glasses-free 3D displays are finally emerging in the consumer market. The technologies being employed in current-generation devices, however, are fundamentally the same as what was invented 100 years ago. In this talk, we explore modern approaches to glasses-free 3D display using the co-design of display optics and computational processing.

Different versions of this talk were given at Carnegie Mellon University (Pittsburgh, USA), Microsoft Research Asia (Beijing, China), Tsinghua University (Beijing, China), HP Research Labs (USA), Telecom ParisTech (Paris, France), Max-Planck-Institut fuer Informatik (Saarbruecken, Germany), Eberhard Karls Universitaet (Tuebingen, Germany), Disney Research (Zuerich, Switzerland), Pixtronix, Inc. (Wilmington, USA), Rochester Institute of Technology (Rochester, USA), University of Toronto (Toronto, Canada), 3M Optical Systems Division (St. Paul, USA).

In the News

  MIT News - Illuminating neuron activity in 3-D, May 2014

  MIT News - Glasses-free 3-D projector, May 2014

  Polka Magazine - Les Experts a Boston (in French), Sept 2013

  MIT News - Multiview 3-D photography made simple, June 2013

  MIT Cover Page - Picture perfect, June 2013

  Physics World - 3D TV without the Glasses, April 2013

  MIT Technology Review - New 3-D Display Could Let Phones and Tablets Produce Holograms, March 2013

  Wired - MIT's Camera Culture group is working on a goggle-free 3D TV experience, November 2012

  Engadget - MIT Media Lab's Tensor Displays stack LCDs for low-cost glasses-free 3D, August 2012

  The Boston Globe - A new vision for 3-D TV, August 2012

  MIT Cover Page - Glasses-free 3-D TV looks nearer, July 2012

  Home Theater Geeks - Podcast 122, July 2012

  Huffington Post - Tensor Display 3D TV From MIT Media Lab May Be 'Window Into Another World', July 2012

  MIT News - Glasses-free 3-D TV looks nearer, July 2012

  Expert Reviews - MIT breakthrough promises realistic glasses-free 3D, July 2012

  Robaid - Tensor Display - a step closer to glasses-free 3-D TV, July 2012

  The Boston Globe - Holograms, 3-D said to be on verge of new era, June 2012

  NewScientist - Glasses-free 3D screens let you see the wider picture, July 2012

  3D Focus - More details emerge about MIT Labs glasses free 3D display, July 2012

  Slashdot - MIT Develops Holographic, Glasses-Free 3D TV, July 2012

  ExtremeTech - MIT develops holographic, glasses-free 3D TV, July 2012

  The Verge - Layered LCD panels could create more realistic glasses-free 3D, July 2012

  Photonics Spectra - Hitting Every Angle with Autostereoscopic 3-D Displays, May 2012

  SIGGRAPHITTI - SIGGRAPH Announces Papers and Courses Highlights, May 2012

  3D Artist - SIGGRAPH 2012 Emerging Technologies Highlights, May 2012

  Science News - Out of the Box, December 2011

  Display Daily - New Approach to 3D Shown at SIGGRAPH, August 2011

  Popular Science - MIT's Smarter Glasses-Free 3-D Tech Provides Realistic Multiple Perspectives, Wider Angle, May 2011

06/14 Job accepted: Stanford EE starting 09/2014
06/14 Paper accepted: Optics Express (Multi-model Superresolution Display)
05/14 Paper accepted: Nature Methods (Light Field Microscope), also featured on MIT Homepage
05/14 Our Light Field Projector Technology is featured on the MIT Homepage!
05/14 Best paper award at ICCP 2014
03/14 TEDx talk
03/14 Three papers to be presented at SIGGRAPH 2014
02/14 Invited talk at OSA Computational Optical Sensing and Imaging (COSI) 2014 conference
02/14 Five papers accepted: IEEE ICCP 2014,SID 2014, CVPR 2014, OSA CLEO 2014, and OSA Optics Letters
02/14 Serving on the ACM SIGGRAPH 2014 Technical Papers Committee and four other international conference PCs in 2014
01/14 Invited talk at German Photonics Congress 2014
01/14 Keynote at SPIE Stereoscopic Displays and Applications 2014 conference
11/13 Two papers accepted to IJCV special issue on computational photography