MIT Media Lab
75 Amherst St, E14-474C
Cambridge, MA 02139
email: gordonw (at) media (dot) mit (dot) edu
Image Courtesy of The Boston Globe
Gordon Wetzstein is a Postdoctoral Researcher in the Camera Culture Group at the MIT Media Lab. His research interests are at the intersection of computer graphics, machine vision, optics, scientific computing, and perception. Gordon received a Diplom in Media System Science with Honors from the Bauhaus-University Weimar in 2006 and a Ph.D. in Computer Science at the University of British Columbia in 2011. His doctoral dissertation focuses on computational light modulation for image acquisition and display and won the Alain Fournier Ph.D. Dissertation Annual Award. He organized the IEEE CVPR 2012 Workshop on Computational Cameras and Displays, presented the "Computational Displays" and "Computational Plenoptic Imaging" courses at ACM SIGGRAPH 2012, and won a best paper award for "Hand-Held Schlieren Photography with Light Field Probes" at ICCP 2011, introducing light field probes as computational displays for computer vision and fluid mechanics applications.
Please see my curriculum vitae for more details.
Research Overview and Highlights
The core goal of my research is the design and implementation of novel approaches that push the boundaries of conventional light capture and display technology through the co-design of optics and computational processing. Such computational imaging systems facilitate new capabilities such as high-resolution, glasses-free 3D display, high dynamic range imaging, extended depth of field photography and projection, lensless imaging, optical image processing for computational sunglasses and windshields, acquiring and reconstructing transparent, refractive phenomena, and seamless projection on complex surfaces. Practical application is an important factor in my research; much of it has been conducted in close collaboration with some of the top academic and industrial research institutes worldwide, including MIT, UBC, Max-Planck-Institut fuer Informatik in Germany, Johannes Kepler University in Austria, NVIDIA Research, and Dolby Canada.
Several research projects are highlighted below, please see the research page for a detailed list.
| Tensor Displays (ACM SIGGRAPH 2012, Paper & ETech)
A new family of compressive light field displays comprising all architectures employing stacks of high-speed LCDs illuminated by uniform or directional backlighting (i.e., any low-resolution light field emitter). Supports wider fields of view. larger depths of field, and thinner form factors than previous layered displays. SIGGRAPH 2012 Paper Highlight, Emerging Technologies Highlight, and featured in New Scientist,
The Boston Globe,
engadget, slashdot, and many more (see below)!
| Polarization Fields (ACM SIGGRAPH Asia 2011)
An optically and computationally efficient compressive light field design. Multiple stacked LCDs, stripped off their polarizing films and color filters, act as programmable polarization rotators. Combined with efficient solvers implemented on the GPU, real-time framerates for dynamic content are achieved. Real-time OpenGL and offline Matlab code available. Featured in Wired and The Boston Globe!
| Layered 3D (ACM SIGGRAPH 2011)
A compressive light field display using stacked, inkjet-printed transparencies in combination with computed tomographic light field synthesis. This design allows for the fabrication of inexpensive, high-resolution, and bright light field displays showing static content. Real-time OpenGL and offline Matlab code available. SIGGRAPH 2011 Paper Highlight, Proceedings Cover Feature, and featured in The Boston Globe!
| Coded Aperture Projection (ACM Trans. Graph. & SIGGRAPH 2010, ICCP 2013)
A computational display approach to extended depth of field projection. An LCD in the aperture modulates light with content-adaptive optical codes; combined with efficient algorithms that exploit limitations of the human visual system, this display achieves a high light throughput and a large depth of field for real-time projections on non-planar surfaces.
Computational Light Transport
| Adaptive Image Synthesis for Compressive Displays (ACM SIGGRAPH 2013)
An integrated approach to generate high-quality images and light fields for emerging compressive displays. The algorithm unifies sampling, rendering, and display-adaptive optimization to increase performance by orders of magnitude. We show applications for compressive light field displays, high dynamic range displays, and super-resolution displays. This work is the first step toward a graphics pipeline tailored for next-generation displays.
| Schlieren Photograph with Light Field Probes (ICCV 2011 and ICCP 2011)
A new approach to capturing and reconstructing transparent, refractive media. High-dimensional light field probes optically encode refraction in variations of color and intensity in captured photographs, which allows dynamic phenomena to be reconstructed from a single shot. Best Paper Award!
| Optical Image Processing (Computer Graphics Forum 2011)
The sunglasses, windshields, and motorcycle visors of the future: transparent, light modulating displays replace the glass and are computationally driven to optically enhance scene contrast, highlight objects of interest, or de-metamerize colors.
| Radiometric Compensation through Inverse Light Transport (Pacific Graphics 2007)
A novel approach to inverse light transport that allows for seamless projections on complex surfaces, such as museum exhibits, historic sites, air-plane cabins, or stage performances. Many illumination aspects, such as interreflections, refractions, and defocus are compensated.
| View-Dependent Stereo Projection in Real Environments (ACM SIGGRAPH ETech 2005 and ISMAR 2005)
View-dependent image-based and geometric warping, radiometric compensation, and multi-focal projection enable seamless visualizations on geometric complex, colored, and textured surfaces within everyday environments.
Mathematical Methods in Imaging (MIT Media Lab, MAS 132/532, Spring 2013)
This course deals with both optical and computational aspects in imaging, starting with classical imaging modalities and image processing formulations and ending with innovative and state of the art techniques for computation imaging and object description. We will discuss various mathematical aspects, including fourier optics, sparse representations and variational and geometric models. We will forray as well to describe non-standard and computational photography techniques such as including light field imaging.
Computational Cameras and Photography (MIT Media Lab, MAS 131/531, Fall 2012)
In this couse we will study the emerging multi-disciplinary field of computational photography - one which is at the intersection of signal processing, applied optics, computer graphics and vision, electronics, art, and online sharing through social networks. We will examine whether innovative camera-like sensors can overcome the tough problems in scene understanding and generate insightful awareness. In addition, we will develop new algorithms to exploit unusual optics, programmable wavelength control, and femto-second accurate photon counting to decompose the sensed values into perceptually critical elements.
Future of Imaging (MIT Media Lab, MAS 132/532, Spring 2012)
In this course, we will survey the landscape of imaging techniques and learn how to conduct research in imaging. With more than a billion people with networked, mobile cameras in their hands, we are seeing a rapid evolution in activities based on visual exchange. People's daily activities are increasingly based on pervasive recording and eager consumption of images and video. We will look at the technical as well the social aspects of this rapidly evolving camera culture.
Computational Displays: Combining Optical Fabrication, Computational Processing, and Perceptual Tricks to Build the Displays of the Future (ACM SIGGRAPH 2012 Course, Eurographics 2013 Tutorial)
This course serves as an introduction to the emerging field of computational displays. The pedagogical goal of this course is to provide the audience with the tools necessary to expand their research endeavors by providing step-by-step instructions on all aspects of computational displays: display optics, mathematical analysis, efficient computational processing, computational perception, and, most importantly, the effective combination of all these aspects. Specifically, we discuss a wide variety of different applications and hardware setups of computational displays, including high dynamic range displays, advanced projection systems as well as glasses-free 3D display. SIGGRAPH 2012 Courses Highlight!
Computational Plenoptic Imaging (ACM SIGGRAPH 2012 Course, Eurographics 2011 STAR & Tutorial)
This course is intended to review the state of the art in joint optical light modulation and computational reconstruction of visual information transcending that captured by traditional photography. In contrast to prior courses on general computational photography this course gives a broad, well-structured, and intuitive overview of all aspects of plenoptic image acquisition and focuses on two recent developments: light field acquisition and ultra-fast cameras. We unveil the secrets behind capturing light at a trillion frames per second and the Lytro camera. This course serves as a resource for interested parties by providing a categorization of recent research and help in the identification of unexplored areas in the field.
Compressive Light Field Displays (2013, Keynote, CUSO Winter School on Computational Photography and Display, Lenk, Switzerland)
In this talk, we explore next-generation light field or glasses-free 3D display technology. Through the co-design of display optics and computational processing, compressive light field displays allow for thinner form factors, higher resolutions, better 3D effects, and higher contrast that previous 3D display technologies. We discuss how psycho-physiological aspects of human vision are important for display design and demonstrate how perceptually-driven 3D displays can enhance the capability of current technology even further.
Computational Light Field Displays (2012, Invited Talks)
With the invention of integral imaging and parallax barriers in the beginning of the 20th century, glasses-free 3D displays have become feasible. Only today - more than a century later - glasses-free 3D displays are finally emerging in the consumer market. The technologies being employed in current-generation devices, however, are fundamentally the same as what was invented 100 years ago. In this talk, we explore modern approaches to glasses-free 3D display using the co-design of display optics and computational processing.
Different versions of this talk were given at Carnegie Mellon University (Pittsburgh, USA), Microsoft Research Asia (Beijing, China), Tsinghua University (Beijing, China), HP Research Labs (USA), Telecom ParisTech (Paris, France), Max-Planck-Institut fuer Informatik (Saarbruecken, Germany), Eberhard Karls Universitaet (Tuebingen, Germany), Disney Research (Zuerich, Switzerland), Pixtronix, Inc. (Wilmington, USA), Rochester Institute of Technology (Rochester, USA), University of Toronto (Toronto, Canada), 3M Optical Systems Division (St. Paul, USA).
Computational Light Modulation for Image Acquisition and Display (2010, Invited Talk, Max-Planck-Institut fuer Informatik, Saarbruecken, Germany)
In this talk, we explore a number of approaches to joint optical multiplexing and computational reconstruction of the dimensions of the plenoptic function. The combined design of optical light modulation and computational processing is not only useful for photography, displays benefit from similar ideas. Within this scope, three different approaches to computational displays are discussed: glasses-free 3D displays for entertainment and scientific visualization, optical see-through displays enhancing the capabilities of the human visual system, and computational probes-displays designed for computer vision applications rather than for direct view.
In the News
Physics World - 3D TV without the Glasses, April 2013
MIT Technology Review - New 3-D Display Could Let Phones and Tablets Produce Holograms, March 2013
Wired - MIT's Camera Culture group is working on a goggle-free 3D TV experience, November 2012
MIT Media Lab's Tensor Displays stack LCDs for low-cost glasses-free 3D, August 2012
The Boston Globe -
A new vision for 3-D TV, August 2012
Home Theater Geeks - Podcast 122, July 2012
Huffington Post - Tensor Display 3D TV From MIT Media Lab May Be 'Window Into Another World', July 2012
MIT News -
Glasses-free 3-D TV looks nearer, July 2012
Expert Reviews -
MIT breakthrough promises realistic glasses-free 3D, July 2012
Tensor Display - a step closer to glasses-free 3-D TV, July 2012
The Boston Globe - Holograms, 3-D said to be on verge of new era, June 2012
NewScientist - Glasses-free 3D screens let you see the wider picture, July 2012
3D Focus - More details emerge about MIT Labs glasses free 3D display, July 2012
Slashdot - MIT Develops Holographic, Glasses-Free 3D TV, July 2012
ExtremeTech - MIT develops holographic, glasses-free 3D TV, July 2012
The Verge - Layered LCD panels could create more realistic glasses-free 3D, July 2012
Photonics Spectra - Hitting Every Angle with Autostereoscopic 3-D Displays, May 2012
SIGGRAPHITTI - SIGGRAPH Announces Papers and Courses Highlights, May 2012
3D Artist - SIGGRAPH 2012 Emerging Technologies Highlights, May 2012
Science News - Out of the Box, December 2011
Display Daily - New Approach to 3D Shown at SIGGRAPH, August 2011
Popular Science - MIT's Smarter Glasses-Free 3-D Tech Provides Realistic Multiple Perspectives, Wider Angle, May 2011