MIT Media Lab
75 Amherst St, E14-474C
Cambridge, MA 02139
email: gordonw (at) media (dot) mit (dot) edu
Image Courtesy of The Boston Globe
Gordon Wetzstein is a Research Scientist in the Camera Culture Group at the MIT Media Lab. His research focuses on computational imaging and display systems as well as computational light transport. At the intersection of computer graphics, machine vision, optics, scientific computing, and perception, this research has a wide range of applications in next-generation consumer electronics, scientific imaging, human-computer interaction, remote sensing, and many other areas. Gordon's cross-disciplinary approach to research has been funded by DARPA, NSF, Samsung, and other grants from industry sponsors and research councils. In 2006, Gordon graduated with Honors from the Bauhaus in Weimar, Germany, and he received a Ph.D. in Computer Science from the University of British Columbia in 2011. His doctoral dissertation focuses on computational light modulation for image acquisition and display and won the Alain Fournier Ph.D. Dissertation Annual Award. He organized the IEEE 2012 and 2013 International Workshops on Computational Cameras and Displays, founded displayblocks.org as a forum for sharing computational display design instructions with the DIY community, and presented a number of courses on Computational Displays and Computational Photography at ACM SIGGRAPH. Gordon won the best paper award for "Hand-Held Schlieren Photography with Light Field Probes" at ICCP 2011 and a Laval Virtual Award in 2005.
Please see my curriculum vitae for more details.
The core goal of my research is the design and implementation of novel approaches that push the boundaries of conventional light capture and display technology through the co-design of optics and computational processing. Such computational imaging systems facilitate new capabilities such as super-resolution, high dynamic range, glasses-free 3D display, extended depth of field photography and projection, lensless imaging, optical computing and image processing, acquiring and reconstructing refractive phenomena, and seamless projection on complex surfaces. Practical application is an important factor in my research; much of it has been conducted in close collaboration with some of the top academic and industrial research institutes worldwide.
Several research projects are highlighted below, please see the research page for a detailed list.
| Focus 3D (ACM Trans. Graph. 2013)
A new computational light field (glasses-free 3D) display that supports accommodative depth cues in addition to motion parallax and binocular cues. Building on multilayer attenuator and directional backlight architectures, the proposed design uses nonnegative light field tensor factorization to achieve compressive image display.
| Tensor Displays (ACM SIGGRAPH 2012, Paper & ETech)
A new family of compressive light field displays comprising all architectures employing stacks of high-speed LCDs illuminated by uniform or directional backlighting (i.e., any low-resolution light field emitter). Supports wider fields of view. larger depths of field, and thinner form factors than previous layered displays. SIGGRAPH 2012 Paper Highlight, Emerging Technologies Highlight, and featured in New Scientist,
The Boston Globe,
engadget, slashdot, and many more (see below)!
| Polarization Fields (ACM SIGGRAPH Asia 2011)
An optically and computationally efficient compressive light field design. Multiple stacked LCDs, stripped off their polarizing films and color filters, act as programmable polarization rotators. Combined with efficient solvers implemented on the GPU, real-time framerates for dynamic content are achieved. Real-time OpenGL and offline Matlab code available. Featured in Wired and The Boston Globe!
| Layered 3D (ACM SIGGRAPH 2011)
A compressive light field display using stacked, inkjet-printed transparencies in combination with computed tomographic light field synthesis. This design allows for the fabrication of inexpensive, high-resolution, and bright light field displays showing static content. Real-time OpenGL and offline Matlab code available. SIGGRAPH 2011 Paper Highlight, Proceedings Cover Feature, and featured in The Boston Globe!
| Coded Aperture Projection (ACM Trans. Graph. & SIGGRAPH 2010, ICCP 2013)
A computational display approach to extended depth of field projection. An LCD in the aperture modulates light with content-adaptive optical codes; combined with efficient algorithms that exploit limitations of the human visual system, this display achieves a high light throughput and a large depth of field for real-time projections on non-planar surfaces.
Computational Light Transport
| Adaptive Image Synthesis for Compressive Displays (ACM SIGGRAPH 2013)
An integrated approach to generate high-quality images and light fields for emerging compressive displays. The algorithm unifies sampling, rendering, and display-adaptive optimization to increase performance by orders of magnitude. We show applications for compressive light field displays, high dynamic range displays, and super-resolution displays. This work is the first step toward a graphics pipeline tailored for next-generation displays.
| Schlieren Photograph with Light Field Probes (IJCV 2013, ICCV 2011, and ICCP 2011)
A new approach to capturing and reconstructing transparent, refractive media. High-dimensional light field probes optically encode refraction in variations of color and intensity in captured photographs, which allows dynamic phenomena to be reconstructed from a single shot. Best Paper Award!
| Optical Image Processing (Computer Graphics Forum 2011)
The sunglasses, windshields, and motorcycle visors of the future: transparent, light modulating displays replace the glass and are computationally driven to optically enhance scene contrast, highlight objects of interest, or de-metamerize colors.
| Radiometric Compensation through Inverse Light Transport (Pacific Graphics 2007)
A novel approach to inverse light transport that allows for seamless projections on complex surfaces, such as museum exhibits, historic sites, air-plane cabins, or stage performances. Many illumination aspects, such as interreflections, refractions, and defocus are compensated.
| View-Dependent Stereo Projection in Real Environments (ACM SIGGRAPH ETech 2005 and ISMAR 2005)
View-dependent image-based and geometric warping, radiometric compensation, and multi-focal projection enable seamless visualizations on geometric complex, colored, and textured surfaces within everyday environments.
Mathematical Methods in Imaging (MIT Media Lab, MAS 132/532, Spring 2013 and Spring 2014)
This course deals with both optical and computational aspects in imaging, starting with classical imaging modalities and image processing formulations and ending with innovative and state of the art techniques for computation imaging and object description. We will discuss various mathematical aspects, including fourier optics, sparse representations and variational and geometric models. We will forray as well to describe non-standard and computational photography techniques such as including light field imaging.
Computational Cameras and Photography (MIT Media Lab, MAS 131/531, Fall 2012)
In this couse we will study the emerging multi-disciplinary field of computational photography - one which is at the intersection of signal processing, applied optics, computer graphics and vision, electronics, art, and online sharing through social networks. We will examine whether innovative camera-like sensors can overcome the tough problems in scene understanding and generate insightful awareness. In addition, we will develop new algorithms to exploit unusual optics, programmable wavelength control, and femto-second accurate photon counting to decompose the sensed values into perceptually critical elements.
Future of Imaging (MIT Media Lab, MAS 132/532, Spring 2012)
In this course, we will survey the landscape of imaging techniques and learn how to conduct research in imaging. With more than a billion people with networked, mobile cameras in their hands, we are seeing a rapid evolution in activities based on visual exchange. People's daily activities are increasingly based on pervasive recording and eager consumption of images and video. We will look at the technical as well the social aspects of this rapidly evolving camera culture.
Computational Displays: Combining Optical Fabrication, Computational Processing, and Perceptual Tricks to Build the Displays of the Future (ACM SIGGRAPH 2012 Course, Eurographics 2013 Tutorial)
This course serves as an introduction to the emerging field of computational displays. The pedagogical goal of this course is to provide the audience with the tools necessary to expand their research endeavors by providing step-by-step instructions on all aspects of computational displays: display optics, mathematical analysis, efficient computational processing, computational perception, and, most importantly, the effective combination of all these aspects. Specifically, we discuss a wide variety of different applications and hardware setups of computational displays, including high dynamic range displays, advanced projection systems as well as glasses-free 3D display. SIGGRAPH 2012 Courses Highlight!
Computational Plenoptic Imaging (ACM SIGGRAPH 2012 Course, Eurographics 2011 STAR & Tutorial)
This course is intended to review the state of the art in joint optical light modulation and computational reconstruction of visual information transcending that captured by traditional photography. In contrast to prior courses on general computational photography this course gives a broad, well-structured, and intuitive overview of all aspects of plenoptic image acquisition and focuses on two recent developments: light field acquisition and ultra-fast cameras. We unveil the secrets behind capturing light at a trillion frames per second and the Lytro camera. This course serves as a resource for interested parties by providing a categorization of recent research and help in the identification of unexplored areas in the field.
KEYNOTE: Compressive displays - combining optical fabrication, computational processing, and perceptual tricks to build the displays of the future (2014, SPIE Stereoscopic Displays & Applications)
In this talk, we explore modern approaches to glasses-free 3D display using compressive light field displays. In contrast to conventional technology, compressive displays aim for a joint-design of optics, electronics, and computational processing that together exploit compressibility of the presented data. For instance, multiview images or light fields show the same 3D scene from different perspectives - all these images are very similar and therefore compressible. By combining displays that use multilayer architectures or directional backlighting combined with optimal light field factorizations, limitations of existing devices, for instance resolution, depth of field, and field of view, can be overcome. In addition to light field display, we will discuss approaches to compressive super-resolution image display and compressive high dynamic range display. As with compressive light field displays, these technologies rely on multiplexing image content in time such that the visual system of a human observer combines presented patterns into a consistent 3D, high-resolution, or high-contrast image. With the invention of integral imaging and parallax barriers in the beginning of the 20th century, glasses-free 3D displays have become feasible. With rapid advances in optical fabrication, digital processing power, and computational perception, a new generation of display technology is emerging: compressive displays exploring the co-design of optical elements and computational processing while taking particular characteristics of the human visual system into account. We will review these techniques and also give an outlook on next-generation compressive light field camera technology. Slides are available here.
KEYNOTE: Compressive Light Field Displays (2013, CUSO Winter School on Computational Photography and Display, Lenk, Switzerland)
In this talk, we explore next-generation light field or glasses-free 3D display technology. Through the co-design of display optics and computational processing, compressive light field displays allow for thinner form factors, higher resolutions, better 3D effects, and higher contrast that previous 3D display technologies. We discuss how psycho-physiological aspects of human vision are important for display design and demonstrate how perceptually-driven 3D displays can enhance the capability of current technology even further.
Computational Light Field Displays (2012, Invited Talks)
With the invention of integral imaging and parallax barriers in the beginning of the 20th century, glasses-free 3D displays have become feasible. Only today - more than a century later - glasses-free 3D displays are finally emerging in the consumer market. The technologies being employed in current-generation devices, however, are fundamentally the same as what was invented 100 years ago. In this talk, we explore modern approaches to glasses-free 3D display using the co-design of display optics and computational processing.
Different versions of this talk were given at Carnegie Mellon University (Pittsburgh, USA), Microsoft Research Asia (Beijing, China), Tsinghua University (Beijing, China), HP Research Labs (USA), Telecom ParisTech (Paris, France), Max-Planck-Institut fuer Informatik (Saarbruecken, Germany), Eberhard Karls Universitaet (Tuebingen, Germany), Disney Research (Zuerich, Switzerland), Pixtronix, Inc. (Wilmington, USA), Rochester Institute of Technology (Rochester, USA), University of Toronto (Toronto, Canada), 3M Optical Systems Division (St. Paul, USA).
In the News
Polka Magazine -
Les Experts a Boston (in French), Sept 2013
MIT News -
Multiview 3-D photography made simple, June 2013
MIT Cover Page -
Picture perfect, June 2013
Physics World - 3D TV without the Glasses, April 2013
MIT Technology Review - New 3-D Display Could Let Phones and Tablets Produce Holograms, March 2013
Wired - MIT's Camera Culture group is working on a goggle-free 3D TV experience, November 2012
MIT Media Lab's Tensor Displays stack LCDs for low-cost glasses-free 3D, August 2012
The Boston Globe -
A new vision for 3-D TV, August 2012
MIT Cover Page -
Glasses-free 3-D TV looks nearer, July 2012
Home Theater Geeks - Podcast 122, July 2012
Huffington Post - Tensor Display 3D TV From MIT Media Lab May Be 'Window Into Another World', July 2012
MIT News -
Glasses-free 3-D TV looks nearer, July 2012
Expert Reviews -
MIT breakthrough promises realistic glasses-free 3D, July 2012
Tensor Display - a step closer to glasses-free 3-D TV, July 2012
The Boston Globe - Holograms, 3-D said to be on verge of new era, June 2012
NewScientist - Glasses-free 3D screens let you see the wider picture, July 2012
3D Focus - More details emerge about MIT Labs glasses free 3D display, July 2012
Slashdot - MIT Develops Holographic, Glasses-Free 3D TV, July 2012
ExtremeTech - MIT develops holographic, glasses-free 3D TV, July 2012
The Verge - Layered LCD panels could create more realistic glasses-free 3D, July 2012
Photonics Spectra - Hitting Every Angle with Autostereoscopic 3-D Displays, May 2012
SIGGRAPHITTI - SIGGRAPH Announces Papers and Courses Highlights, May 2012
3D Artist - SIGGRAPH 2012 Emerging Technologies Highlights, May 2012
Science News - Out of the Box, December 2011
Display Daily - New Approach to 3D Shown at SIGGRAPH, August 2011
Popular Science - MIT's Smarter Glasses-Free 3-D Tech Provides Realistic Multiple Perspectives, Wider Angle, May 2011