Gordon Wetzstein - MIT Media Lab

Gordon Wetzstein
MIT Media Lab

75 Amherst St, E14-474C
Cambridge, MA 02139
USA

phone: 617.715.4683
email: gordonw (at) media (dot) mit (dot) edu

Image Courtesy of The Boston Globe

Short Biography: Gordon Wetzstein is a Postdoctoral Researcher in the Camera Culture Group at the MIT Media Lab. His research interests are at the intersection of computer graphics, machine vision, optics, scientific computing, and perception. Gordon received a Diplom in Media System Science with Honors from the Bauhaus-University Weimar in 2006 and a Ph.D. in Computer Science at the University of British Columbia in 2011. His doctoral dissertation focuses on computational light modulation for image acquisition and display and won the Alain Fournier Ph.D. Dissertation Annual Award. He organized the IEEE CVPR 2012 Workshop on Computational Cameras and Displays, presented the "Computational Displays" and "Computational Plenoptic Imaging" courses at ACM SIGGRAPH 2012, and won a best paper award for "Hand-Held Schlieren Photography with Light Field Probes" at ICCP 2011, introducing light field probes as computational displays for computer vision and fluid mechanics applications.

Please see my curriculum vitae for more details.

Research Overview and Highlights

The core goal of my research is the design and implementation of novel approaches that push the boundaries of conventional light capture and display technology through the co-design of optics and computational processing. Such computational imaging systems facilitate new capabilities such as high-resolution, glasses-free 3D display, high dynamic range imaging, extended depth of field photography and projection, lensless imaging, optical image processing for computational sunglasses and windshields, acquiring and reconstructing transparent, refractive phenomena, and seamless projection on complex surfaces. Practical application is an important factor in my research; much of it has been conducted in close collaboration with some of the top academic and industrial research institutes worldwide, including MIT, UBC, Max-Planck-Institut fuer Informatik in Germany, Johannes Kepler University in Austria, NVIDIA Research, and Dolby Canada.

Several research projects are highlighted below, please see the research page for a detailed list.

Compressive Displays

Tensor Displays (ACM SIGGRAPH 2012, Paper & ETech)

A new family of compressive light field displays comprising all architectures employing stacks of high-speed LCDs illuminated by uniform or directional backlighting (i.e., any low-resolution light field emitter). Supports wider fields of view. larger depths of field, and thinner form factors than previous layered displays. SIGGRAPH 2012 Paper Highlight, Emerging Technologies Highlight, and featured in New Scientist, The Boston Globe, engadget, slashdot, and many more (see below)!

Polarization Fields (ACM SIGGRAPH Asia 2011)

An optically and computationally efficient compressive light field design. Multiple stacked LCDs, stripped off their polarizing films and color filters, act as programmable polarization rotators. Combined with efficient solvers implemented on the GPU, real-time framerates for dynamic content are achieved. Real-time OpenGL and offline Matlab code available. Featured in Wired and The Boston Globe!

Layered 3D (ACM SIGGRAPH 2011)

A compressive light field display using stacked, inkjet-printed transparencies in combination with computed tomographic light field synthesis. This design allows for the fabrication of inexpensive, high-resolution, and bright light field displays showing static content. Real-time OpenGL and offline Matlab code available. SIGGRAPH 2011 Paper Highlight, Proceedings Cover Feature, and featured in The Boston Globe!

Coded Aperture Projection (ACM Trans. Graph. & SIGGRAPH 2010, ICCP 2013)

A computational display approach to extended depth of field projection. An LCD in the aperture modulates light with content-adaptive optical codes; combined with efficient algorithms that exploit limitations of the human visual system, this display achieves a high light throughput and a large depth of field for real-time projections on non-planar surfaces.

Computational Photography

Compressive Light Field Photography (ACM SIGGRAPH 2013)

A single-shot approach to high-resolution light field photography. We introduce light field atoms as a sparse representation for natural light fields and derive optimized optical systems to acquite them with commodity hardware in a single image.

A Theory of Plenoptic Multiplexing (IJCV 2012 and IEEE CVPR 2010)

A general theory of multiplexing the dimensions of the plenoptic function onto an image sensor enabling principled comparisons of plenopticc multiplexing schemes, including noise analysis, as well as the development of unifying reconstruction algorithms.

Frequency Analysis of Transient Light Transport (ECCV 2012)

A frequency analysis of light transport considering spatial, angular, and temporal light variation. Based on this analysis, a new computational camera design is introduced: lensless imaging using an ultra fast sensor and pulsed laser illumination.

Single-Shot Computational High Dynamic Range Imaging (IEEE CVPR 2010)

A frequency analysis of sensor saturation and a novel computational camera design that suppresses saturation artifacts and allows for single-shot high dynamic range imaging.

Computational Light Transport

Adaptive Image Synthesis for Compressive Displays (ACM SIGGRAPH 2013)

An integrated approach to generate high-quality images and light fields for emerging compressive displays. The algorithm unifies sampling, rendering, and display-adaptive optimization to increase performance by orders of magnitude. We show applications for compressive light field displays, high dynamic range displays, and super-resolution displays. This work is the first step toward a graphics pipeline tailored for next-generation displays.

Schlieren Photograph with Light Field Probes (ICCV 2011 and ICCP 2011)

A new approach to capturing and reconstructing transparent, refractive media. High-dimensional light field probes optically encode refraction in variations of color and intensity in captured photographs, which allows dynamic phenomena to be reconstructed from a single shot. Best Paper Award!

Optical Image Processing (Computer Graphics Forum 2011)

The sunglasses, windshields, and motorcycle visors of the future: transparent, light modulating displays replace the glass and are computationally driven to optically enhance scene contrast, highlight objects of interest, or de-metamerize colors.

Radiometric Compensation through Inverse Light Transport (Pacific Graphics 2007)

A novel approach to inverse light transport that allows for seamless projections on complex surfaces, such as museum exhibits, historic sites, air-plane cabins, or stage performances. Many illumination aspects, such as interreflections, refractions, and defocus are compensated.

View-Dependent Stereo Projection in Real Environments (ACM SIGGRAPH ETech 2005 and ISMAR 2005)

View-dependent image-based and geometric warping, radiometric compensation, and multi-focal projection enable seamless visualizations on geometric complex, colored, and textured surfaces within everyday environments.

Courses Overview

Mathematical Methods in Imaging (MIT Media Lab, MAS 132/532, Spring 2013)

This course deals with both optical and computational aspects in imaging, starting with classical imaging modalities and image processing formulations and ending with innovative and state of the art techniques for computation imaging and object description. We will discuss various mathematical aspects, including fourier optics, sparse representations and variational and geometric models. We will forray as well to describe non-standard and computational photography techniques such as including light field imaging.

Computational Cameras and Photography (MIT Media Lab, MAS 131/531, Fall 2012)

In this couse we will study the emerging multi-disciplinary field of computational photography - one which is at the intersection of signal processing, applied optics, computer graphics and vision, electronics, art, and online sharing through social networks. We will examine whether innovative camera-like sensors can overcome the tough problems in scene understanding and generate insightful awareness. In addition, we will develop new algorithms to exploit unusual optics, programmable wavelength control, and femto-second accurate photon counting to decompose the sensed values into perceptually critical elements.

Future of Imaging (MIT Media Lab, MAS 132/532, Spring 2012)

In this course, we will survey the landscape of imaging techniques and learn how to conduct research in imaging. With more than a billion people with networked, mobile cameras in their hands, we are seeing a rapid evolution in activities based on visual exchange. People's daily activities are increasingly based on pervasive recording and eager consumption of images and video. We will look at the technical as well the social aspects of this rapidly evolving camera culture.

Computational Displays: Combining Optical Fabrication, Computational Processing, and Perceptual Tricks to Build the Displays of the Future (ACM SIGGRAPH 2012 Course, Eurographics 2013 Tutorial)

This course serves as an introduction to the emerging field of computational displays. The pedagogical goal of this course is to provide the audience with the tools necessary to expand their research endeavors by providing step-by-step instructions on all aspects of computational displays: display optics, mathematical analysis, efficient computational processing, computational perception, and, most importantly, the effective combination of all these aspects. Specifically, we discuss a wide variety of different applications and hardware setups of computational displays, including high dynamic range displays, advanced projection systems as well as glasses-free 3D display. SIGGRAPH 2012 Courses Highlight!

Computational Plenoptic Imaging (ACM SIGGRAPH 2012 Course, Eurographics 2011 STAR & Tutorial)

This course is intended to review the state of the art in joint optical light modulation and computational reconstruction of visual information transcending that captured by traditional photography. In contrast to prior courses on general computational photography this course gives a broad, well-structured, and intuitive overview of all aspects of plenoptic image acquisition and focuses on two recent developments: light field acquisition and ultra-fast cameras. We unveil the secrets behind capturing light at a trillion frames per second and the Lytro camera. This course serves as a resource for interested parties by providing a categorization of recent research and help in the identification of unexplored areas in the field.

Recent Talks

Compressive Light Field Displays (2013, Keynote, CUSO Winter School on Computational Photography and Display, Lenk, Switzerland)

In this talk, we explore next-generation light field or glasses-free 3D display technology. Through the co-design of display optics and computational processing, compressive light field displays allow for thinner form factors, higher resolutions, better 3D effects, and higher contrast that previous 3D display technologies. We discuss how psycho-physiological aspects of human vision are important for display design and demonstrate how perceptually-driven 3D displays can enhance the capability of current technology even further.

Computational Light Field Displays (2012, Invited Talks)

With the invention of integral imaging and parallax barriers in the beginning of the 20th century, glasses-free 3D displays have become feasible. Only today - more than a century later - glasses-free 3D displays are finally emerging in the consumer market. The technologies being employed in current-generation devices, however, are fundamentally the same as what was invented 100 years ago. In this talk, we explore modern approaches to glasses-free 3D display using the co-design of display optics and computational processing.

Different versions of this talk were given at Carnegie Mellon University (Pittsburgh, USA), Microsoft Research Asia (Beijing, China), Tsinghua University (Beijing, China), HP Research Labs (USA), Telecom ParisTech (Paris, France), Max-Planck-Institut fuer Informatik (Saarbruecken, Germany), Eberhard Karls Universitaet (Tuebingen, Germany), Disney Research (Zuerich, Switzerland), Pixtronix, Inc. (Wilmington, USA), Rochester Institute of Technology (Rochester, USA), University of Toronto (Toronto, Canada), 3M Optical Systems Division (St. Paul, USA).

Computational Light Modulation for Image Acquisition and Display (2010, Invited Talk, Max-Planck-Institut fuer Informatik, Saarbruecken, Germany)

In this talk, we explore a number of approaches to joint optical multiplexing and computational reconstruction of the dimensions of the plenoptic function. The combined design of optical light modulation and computational processing is not only useful for photography, displays benefit from similar ideas. Within this scope, three different approaches to computational displays are discussed: glasses-free 3D displays for entertainment and scientific visualization, optical see-through displays enhancing the capabilities of the human visual system, and computational probes-displays designed for computer vision applications rather than for direct view.

In the News

Physics World - 3D TV without the Glasses, April 2013