slc.gif (139 bytes)

Ramesh  Raskar
Ramesh Raskar
| Publications | Courses | Talks | Patents | Press | Art&Demos | Personal |

MIT Media Lab , Associate Professor      
 
Camera Culture group, Leader   

Center for Future Storytelling, Co-director

Alfred P. Sloan Foundation Research Fellow
 

Short Bio

Past Affiliations: Ph.D., December 2002, UNC, Chapel Hill, Senior Research Scientist at MERL

raskaratmitedu, Room E14-474G,
75 Amherst St, Cambridge, MA 02139, 
Assistant: Maggie Churc,
mdchurch
atmitedu,
Room E14-474, Maggie: (617)324-0523
fb.com/rraskar, @raskarmit, LinkedIn
Fall course: MAS 131/531
Computational Camera and Photography
Spring course: MAS 532/132
Future of Imaging

Spring course: MAS 533
Imaging Ventures
Research Areas and Interests


Computational Light Transport Computational Photography Spatial Augmented Reality Visual Social Computing (ViSoCo)
Femto-Photography
Hardware App-Store Eye-Diagnostics Compressive (3D) Displays


News
CORNAR: Looking Around Corners, Nature Magazine Video, TED
Femto-Photography: Trillion Frames Per Second Imaging, NYTimes
Tensor Display: Compressive Display for Glasses-free 3D
CATRA: Cataract Mapping with Snap-on Eyepiece for Mobile Phone based Lightfield Display
HR3D: Glasses-free 3D display using algebraic rank constraints of lightfields

DARPA Young Faculty Award 2010
NETRA: Cellphone-based Optometry Solution using Inverse Shack Hartmann Technique, EyeNetra.com
Looking Around Corners using Transient Imaging (Marr Prize#2)
Coded Strobing Photography: Compressive Sensing of High Speed Periodic Events
Rank-constraints in 3D Displays: Relationship between Parallax Barriers and Holograms
BiDi Screen: A Thin, Depth-Sensing LCD for 3D Interaction using Light Field Heterodyning
Bokode: A new optical tag ('barcode') using angular encoding and defocus based decoding
Lightfield Ray-model upgraded to represent Wave Phenomena, Diffraction and Interference
Alfred P. Sloan Foundation Research Fellowship
Computational Photography: From Epsilon to Coded Photography, book chapter
Group newspage


** We have open MEngg, RA, UROP and Post-doc slots in Camera Culture group. Pl Apply.

slc.gif (139
                  bytes)

Projects


Visual Social Computing
          See Camera Culture group page for details

Computational Light Transport

The research goal is to create an entirely new class of imaging devices and a novel theoretical framework for inverse problems in space-time analysis of light transport. Two novel forms of imaging show great potential for research and practical applications: (i) time resolved transient imaging that exploits multi-path analysis and (ii) angle resolved imaging without lenses for displays, medical devices and phase analysis. Can we photograph objects that are not in the direct line of sight? Can we automatically identify materials (e.g., wood, metal or skin) using a single photo? Can we build portable CAT-scanning machines with no moving parts? Can we convert LCDs into large lensless cameras? Can exploit high resolution of displays to create unusual scientific and medical instruments? Underlying theoretical contributions include multi-path analysis, spatial heterodyning, augmented light fields and algebraic rank-constraints of 3D displays.



Femto-Photography
 
Multi-path
Analysis
 
Spatial Heterodyning
 
Augmented Light Field  
Rank-constraint of
3D Displays
 
Computational Probes
 
Randomized
Projections


Looking Around Corners, Trillion FPS imaging

CAT-Scan without
moving parts

BiDi Screen

Geometric
+ Wave optics

Holograms +
Parallax Barriers

Bokode
Coded Strobing


Looking Around the corner using
                                Transient Imaging

CAT Scan


BiDi Screen

Augmented Light Field



Hologram + Parallax Barrier

Bokode

Coded Strobing using Compressive
                            Sensing



Descattering
Shield Fields
Wave-BSDF
Content Adaptive

NETRA for Optometry
Sparsity Analysis



CAT Scan
                                2
Shield Fields
WBSDF
HR3D:
                                  Content-Adaptive Parallax Barriers
NETRA
Sparsity Analysis


2008 -  (ICCV 2009, CVPR 2011, SigA 2011,
Nature Communications 2012)

CVPR 2010
ECCV 2010

Siggraph Asia 2008
Siggraph Asia 2009

Eurographics 2009
ICCP 2011

Siggraph Asia 2010
Siggraph 2011
Siggraph 2012

Siggraph 2009
Siggraph 2010
Siggraph 2011

PAMI 2011



Computational Photography

Computational Photography is an emerging multi-disciplinary field that is at the intersection of optics, signal processing, computer graphics+vision, electronics, art, and online sharing in social networks. The field is evolving through three phases. The first phase was about building a super-camera that has enhanced performance in terms of the traditional parameters, such as dynamic range, field of view or depth of field. I call this Epsilon Photography. Due to limited capabilities of a camera, the scene is sampled via multiple photos, each captured by epsilon variation of the camera parameters. It corresponds to the low-level vision: estimating pixels and pixel features. The second phase is building tools that go beyond capabilities of this super-camera. I call this Coded Photography. The goal here is to reversibly encode information about the scene in a single photograph (or a very few photographs) so that the corresponding decoding allows powerful decomposition of the image into light fields, motion deblurred images, global/direct illumination components or distinction between geometric versus material discontinuities. This corresponds to the mid-level vision: segmentation, organization, inferring shapes, materials and edges. The third phase will be about going beyond the radiometric quantities and challenging the notion that a camera should mimic a single-chambered human eye. Instead of recovering physical parameters, the goal will be to capture the visual essence of the scene and analyze the perceptually critical components.  I call this Essence Photography and it may loosely resemble depiction of the world after high level vision processing. It will spawn new forms of visual artistic expression and communication. Related work


Coded Time (Exposure)
 
Coding in Space
 
Coded Illumination  
Coded Wavelength
Coded Sensing
Flutter Shutter Cam
for motion deblurring

Coded Aperture for
Extended DoF
Mask-based Optical
Heterodyning for
Light Field Capture

Multi-flash Camera
for Depth Edge
Detection

Agile Spectrum for
programmable
wavelength
 
Gradient Encoding
Sensor for HDR
Flutter Shutter Camera

Encoded Blur Camera for Digital
                                Refocussing LightField Camera

Multi Flash
                                      Camera
Agile Spectrum Imaging

Cliques for computing forward
                              difference
2006

2007
2007
2004
2008
2005


Computational Illumination: Projector as the dual of a camera
Traditionally, a projector is considered a display device that maps a 2D image onto a larger flat screen. In our research, we have questioned this restrictive consideration. By treating projector as the dual of a camera – that is, a 3D projective device that maps 2D pixels to 4D rays – we have shown that programmable illumination can be used in a versatile fashion for display, augmentation, optical communication and mobilityPrevious related work

Disclosed Product Impact:
Acute Angle Seamless Multiprojector Display
MPC Polgyonal 3D scene display
Multiprojector quadric curved immersive display
Six Projector 3D rendering on curved display (A detailed press release)

Intelligent Self-ID and Tagging

Computational probes that support the ability to identify and locate hundreds of tags in real time with milimeter precision. We bring RFID and wireless tags into the realm of computer vision, graphics and HCI by supporting precise geometric analysis of tagged objects and surfaces using a composite RF and optical communication. Related work


Office of the Future
Multi-Projector
Display

Spatial Augmented
Reality (SAR)

Pocket Projector


Optical
Communication

3D videoconferencing

Quadric Transfer

Shader Lamps

iLamps

Location Tracking

Office of the Future
Multi Projector Display

Shader Lamps

iLamps
                                      Siggraph 2003
RFIG and Motion Capture

1998

1999

1998

2000

2003
RFIG
Prakash

Bokode
Photosensing RFID
+Pocket Projector

Lighting-Aware
Motion Capture

Angular Encoding
RFIG

Prakash
                                                      Motion Capture

Bokode
2004

2007

2009

Non photorealistic Rendering

Features such as silhouette edges, ridges, valleys and intersections of geometric primitives are useful in conveying shape in illustrations and art. We have developed a proactive imaging method to capture such features in real world scenes of various scales. One area of focus is shape clarifying multi-flash imaging for endscopes. For polygonal scenes, our focus has been real-time rendering of these features without requiring the connectivity information. Previous related work

Intelligent user interfaces

We are developing user interaction methods with novels gadgets and sensors. We have also exploited variational implicit surfaces for free form sketching and manipulation. Previous related work


slc.gif (139
                    bytes)

Publications


For newer publications please see the Camera Culture group page.



Siggraph Papers


2012


Tensor Displays: Compressive Light Field Synthesis using Multilayer Displays with Directional Backlighting
G Wetzstein, D Lanman, M Hirsch, R Raskar, SIGGRAPH 2012
High resolution lightfield display using tensor factorization


Primal-Dual Coding to Probe Light Transport
M O'Toole, R Raskar, K Kutulakos, SIGGRAPH 2012
Single shot global illumination separation using high speed coding but slow speed camera/projector


Tailored Displays to Compensate for Visual Aberrations
V Pamplona, M Oliveira, D Aliaga, R Raskar, SIGGRAPH 2012
Displays that adapt to lack of eye-glasses using high resolution l


Reflectance Model for Diffraction
T Cuypers, SB Oh, T Haber, P Bekaert, R Raskar, ToG to be presented at SIGGRAPH 2012
Augmented Light Fields to model wave effects with ray representation



2011
Cataract
                  Screening

CATRA: Cataract Probe with a Lightfield Display and a Snap-on Eyepiece for Mobile Phones
V Pamplona, E Passos, J Zizka, M. Oliveira, E Lawson, E Clua, R Raskar, SIGGRAPH 2011
Interactive lightfield display that scans eye lens for cataracts and creates opacity and scattering maps



Thick 3D: Tomographic Image Generation for Attenuation-based Light Field and High Dynamic Range Displays
Wetzstein, Lanman, Heidrich, Raskar, SIGGRAPH 2011
Creating glasses-free 3D displays using multi-layer patterns



Switchable Primaries Using Shiftable Layers of Color Filter Arrays
Sajadi, Majumder, Hiwada, Maki, Raskar, SIGGRAPH 2011
Low light photography by adapting color filters on sensors



Highlighted Depth-of-Field Photography: Shining Light on Focus
Kim, Horstmeyer, Kim, Raskar, ACM Transactions on Graphics (TOG paper to be presented at Siggraph 2011)
Smart flash-camera combination that makes out of focus parts not only blurred but also dim



Single View Reflectance Capture using Multiplexed Scattering and Time-of-flight Imaging
Naik,Zhao,Velten, Raskar, Bala, SIGGRAPH 2011
Femto-photography to identify materials from a single viewpoint



2010
NETRA: Clip-on
                for eyetests
NETRA: Interactive Display for Estimating Refractive Errors and Focal Range
V Pamplona, A Mohan, M Oliveira, R Raskar, SIGGRAPH 2010
Shack Hartmann wavefront sensor via clip-on eyepice on cellphone

HR3D display
HR3D Content-Adaptive Parallax Barriers: Optimizing Dual-Layer 3D Displays using Low-Rank Light Field Factorization,
D Lanman, M Hirsch, Y Kim, R Raskar, SIGGRAPH Asia 2010
Exploiting algebraic rank of lightfield displays for brighter and higher resolution 3D display


2009
Bokode: Long Distance Barcodes


Paper: Bokode: Imperceptible Visual tags for Camera-based Interaction from a Distance

A Mohan, G Woo, S Hiura, Q Smithwick, R Raskar, SIGGRAPH 2009
Long distance barcodes of size 3x3mm captured via ordinary cameras at 3 meters for id, and 6DoF pose (angle and distance).
Invertible Motion Blur in
                Video


Paper: Invertible Motion Blur in Video

A Agrawal, Y Xi, R Raskar, SIGGRAPH 2009
Smarter image capture to make blur invertible in video frames and for producing the very first completely automatic motion deblurring system.
BiDi Screen A Thin, Depth-Sensing LCD for
                3D Interaction using Light Fields


Paper: BiDi Screen: A Thin, Depth-Sensing LCD for 3D Interaction using Light Fields

Matthew Hirsch, Douglas Lanman, Henry Holtzman, Ramesh Raskar, SIGGRAPH Asia 2009
A sensor layer, separated by a small distance from a normal LCD display captures lightfield when a mask image is displayed on the LCD to compute the distance to objects in front of the screen.


2008
Glare Reduction


Paper: Glare Aware Photography: 4D Ray Sampling for Reducing Glare Effects of Camera Lenses

R Raskar, A Agrawal, C Wilson, A
Veeraraghavan
SIGGRAPH 2008
First 'single-shot' approach to classify and reduce glare. We show that glare manifests as an outlier in 4D ray space and use a statistical approach to remove it by 4D sampling without reconstructing a 4D-lightfield.
6D Display


Paper: Towards 6D Displays
M Fuchs, Raskar, HP Seidel, H Lensch

SIGGRAPH 2008
A method for embedding 4D and 6D data into 2D films and employing lenslet arrays so that an observer experiences encoded objects as if they are lit by real-world incident illumination.
Shield Field 2008

Paper: Shield Fields: Heterodyned Shadows for Single-Shot Capture and Visual Hull Reconstruction of 3D Occluders
Lanman, Raskar, Agrawal, Taubin

SIGGRAPH ASIA 2008
A method for capturing shadows created due to multiple light sources using a mask and converting shadows into a 3D visual hull.


2007
Light Field and
                Coded Aperture Camera


Paper: Coded aperture and Optical Heterodyning : A mask-based approach for Digital Refocusing and Light Field Acquisition by Conventional Cameras

A Veeraraghavan, R Raskar, A Agrawal, A Mohan, J Tumblin
SIGGRAPH 2007
We exploit Fourier domain remapping of 4D ray-space to capture 4D light field on a 2D sensor in a conventional camera using a transmissive mask. No lens array is used.
Outdoor motion
                capture
Paper: LumiNetra: High Speed Scene Point Capture and Video Enhancement using Photosensing Markers and Multiplexed Illumination
R Raskar, H Nii, B de Decker, Y Hashimoto, J Summet, D Moore, Y Zhao,J Westhues, P Dietz, M Inami, S Nayar, J Barnwell, M Noland, P Bekaert, V Branzoi, E Bruns
SIGGRAPH 2007

A high speed optical motion capture also measures orientation and incident illumination via photosensing tags. Imperceptible tags work in natural lighting, to support on-set mocap or real-time broadcast of virtual_sets.


Course: Computational Photography, Course details and Slides
Course: Spatial Augmented Reality:Merging Real and Virtual Worlds
Emerging Technologies : Illumination Sensitive Dynamic Virtual Sets, [More Details]



2006

Motion deblurring of Taxi
Paper: Coded Exposure Photography: Motion Deblurring via Fluttered Shutter
R Raskar, A Agrawal, J Tumblin
SIGGRAPH 2006
This method flutters the camera shutter open and closed with a binary pseudo-random sequence during exposure. Fast-moving objects are blurred, but sequenced shuttering preserves all spatial frequencies, simplifying the deblurring process.



Paper: Fast Separation of Direct and Global Components of a Scene Using High Frequency Illumination
S Nayar, G Krishnan, M Grossberg, R Raskar
SIGGRAPH 2006
This work proposes a fast capture method for separating a scene into its direct and global illumination components. The method can handle a variety of global effects including inter-reflections, subsurface scattering, and volumetric scattering.



Course: Computational Photography, Course details and Slides
Course: Spatial Augmented Reality: A Modern Approach to Augmented Reality
Emerging Technologies Booth showing real-time demonstration of a new 500 Hz Motion Tracking which costs only a few hundred dollars, [More Details]


2005

Flash Articat Removal and Recovering the reflection
                layer
Paper: Removing Photography Artifacts Using Gradient Projection and Flash-Exposure Sampling
A Agrawal, R Raskar, S Nayar, Y Li

SIGGRAPH 2005
We address three problems: over or under illumination at a given flash brightness, unwanted reflections in glass, and attenuation over depth. And show a strikingly simple technique based on local gradient analysis.

Highlighted Course: Computational Photography, Course details and Slides
Course: Spatial Augmented Reality: A Modern Approach to Augmented Reality
Panel: The Ultimate Display(My position: Pocket Projector)


2004

Paper: Non-photorealistic Camera: Depth Edge Detection and Stylized Rendering using Multi-Flash Imaging
R Raskar, K Tan, R Feris, J Yu, M Turk

SIGGRAPH 2004 Flashlights cast annoying slivers of shadow around the subject. We locate shape contours by analyzing these cast shadows. We then use these silhouettes for artistic effects. But the contours can be used for many Computer and Machine vision applications.

Related Siggraph 2004 Activities and Publicity
Emerging Technologies Booth showing real-time demonstration
Siggraph 2004 cover
Highlighted Paper
Highlighted Emerging Technology activity, (More)
Electronic Theater (conference video highlight)
Student Poster (Specularities)
Paper: RFIG Lamps: Interacting with Self-describing World via Photosensing Wireless Tags and Projectors
R Raskar, P Beardsley, J vanBaar, Y Wang, P Dietz, J Lee, D Leigh, T Willwacher

SIGGRAPH 2004 It is difficult to precisely locate wireless tags using RF alone. But by adding a photosensor to these tags, we can locate using a hybrid RF+Optical scheme. We show this can be implemented in RFID tags and locate these tags using a pocket projector.


Related Siggraph 2004 Activities and Publicity
Emerging Technologies Booth showing real-time demonstration of handheld iLamps projectors
Highlighted Paper

Course: Advanced Projector Geometry topics



2003

Pocket
                Projector Based Interaction Paper: iLamps: Geometrically Aware and Self-Configuring Projectors
R Raskar, J vanBaar, P Beardsley, T Willwacher, S Rao, C Forlines

SIGGRAPH 2003
Related research on mobile phone-type with pocket projector.

Course: Projectors: Advanced Geometric Issues in Applications



2002


Course: Projector-Based Graphics



2000

Paper: Image Based Visual Hulls
Matusik, Buehler, Raskar, Gortler, McMillan 
SIGGRAPH 2000


Paper: The Office of the Future: Unification of Image-Based Modeling and Immersive Displays
R Raskar,G Welch, M Cutts, A Lake, L Stesin, H Fuchs,
SIGGRAPH 1998 


Other Journal and Conference Papers


For newer publications please see the Camera Culture group page.


Iterative aperture mask design in phase space using a rank constraint
R Horstmeyer, S B Oh, R Raskar, Optics Express 2010



Analysis of Light Transport in Scattering Media
Y.Mukaigawa, Y.Yagi, R.Raskar, CVPR 2010 (Oral)



Descattering Transmission via Angular Filtering
J Kim, D Lanman, Y Mukaigawa, R Raskar, ECCV 2010



Image Destabilization:Programmable Defocus using Lens and Sensor Motion
A Mohan, D Lanman, S Hiura, R Raskar, ICCP 2009



Looking Around the corner using Transient Imaging
A Kirmani, T Hutchison, J Davis, R Raskar.
ICCV 2009, (oral, Marr Prize honorable mention)




A Projector-Camera Setup for Geometry-Invariant Frequency Demultiplexing
D. A. Vaquero, R. Raskar, R. S. Feris, and M. Turk.
CVPR 2009




Optimal Single Image Capture for Motion Deblurring
Amit Agrawal and Ramesh Raskar
CVPR 2009



3D Pose Estimation and Segmentation using Specular Cues
Ju Yong Chang, Amit Agrawal, Ramesh Raskar
CVPR 2009




Multiflash Stereopsis: Depth-Edge-Preserving Stereo with Small Baseline Illumination
Rogerio Feris, Ramesh Raskar, Longbin Chen, Kar-Han Tan, Matthew Turk
IEEE Trans. Pattern Anal. Mach. Intell. 30(1): 147-159 (2008)


Agile Spectrum Imaging Agile Spectrum Imaging: Programmable Wavelength Modulation for Cameras and Projectors
A Mohan, R Raskar, J Tumblin 

Eurographics 2008



Sensing increased image resolution using aperture masks
Ankit Mohan, Xiang Huang, Jack Tumblin, Ramesh Raskar
CVPR 2008



Non-Refractive Modulators for Encoding and Capturing Scene Appearance and Depth
Ashok Veeraraghavan, Amit Agrawal, Ramesh Raskar, Ankit Mohan and Jack Tumblin
CVPR 2008



Characterizing the Shadow Space of Camera-Light Pairs
Daniel A. Vaquero, Rogerio S. Feris, Matthew Turk, Ramesh Raskar
CVPR 2008



Capturing Images with Sparse Informational Pixels using Projected 3D Tags
Li Zhang, Neesha Subramaniam, Robert Lin, Ramesh Raskar, Shree K. Nayar
IEEE-VR 2008: 11-18


Super-resolution from Motion
Resolving Objects at Higher Resolution from a Single Motion-Blurred Image
A Agrawal, R Raskar 

CVPR 2007 (Oral)


Retargetting
Automatic Image Retargeting
Setlur, V., Takagi, S., Raskar, Ramesh., Gleicher, M., and Gooch, B.
Mobile and Ubiquitous Multimedia (MUM) 2005, ACM Press 2005.
Related research on Computational Photography.

Related wonderful research on Seam Carving and Context Aware Resizing.



Videoshop: A new framework for spatio-temporal video editing in gradient domain 
Hongcheng Wang, Ning Xu, Ramesh Raskar and Narendra Ahuja 

Graphical Models Volume 69, Issue 1, Pages 1-88 (January 2007)



Detecting and Segmenting Un-occluded Items by Actively Casting Shadows
Tze Koh, Amit Agrawal, Ramesh Raskar, Steve Morgan, Nicholas Miles and Barrie Hayes-Gill
ACCV 2007 (oral presentation)



Edge
                Suppression
Edge Suppression by Gradient Field Transformation Using Cross-Projection Tensors 
A Agrawal, R Raskar, R Chellappa 

CVPR 2006, New York, NY



What is the Range of Surface Reconstructions from a Gradient Field?
A Agrawal, R Raskar, R Chellappa

ECCV 2006 (Oral), Graz, Austria



Discontinuity Preserving Stereo with Small Baseline Multi-Flash Illumination
R Feris, R Raskar, L Chen, K Tan, M Turk

ICCV 2005 (Oral), Beijing



An Algebraic Approach to Surface Reconstruction from Gradient Fields
A Agrawal, R Chellappa, R Raskar

ICCV 2005, Beijing



Photosensing Wireless Tags for Geometric Procedures
R Raskar, P Dietz, P Beardsley, J vanBaar

Communication of the ACM (Invited Article) Sept 05


Gradient Camera
Why I Want a Gradient Camera
J Tumblin, A Agrawal, R Raskar

CVPR 2005, San Diego, CA


Quadric Image Transfer for Curved
                Screens
Quadric Image Transfer for Immersive Curved Screen Displays
R Raskar, J vanBaar, T Willwacher, S Rao

Eurographics 2004, Grenoble, France



Automatic Projector Calibration with Embedded Light Sensors
Lee, J.C.; Dietz, P.H.; Maynes-Aminzade, D.; Raskar, R.; Hudson, S.E.

UIST 2004




Display Grid
R Raskar, J vanBaar, P Beardsley

SID (Society for Information Display) 2004
(Comparison of Compute Grid with Projector-based Display Grid)



Seamless Video Editing
H Wang, R Raskar, N Ahuja

ICPR 2004



Image Fusion for Enhancing Context in Images and Video
R Raskar, A Ilie, J Yu

ACM Nonphotorealistic Animation and Rendering (NPAR) 2004, Annecy, France      ( Results, Project)



The Large Display Dilemma
R Raskar

Invited Talk at Pro-Cams workshop, Nice, France, Oct 2003           (Slides)



Blending Multiple Views
R Raskar, K Low

Pacific Graphics 2002, Beijing, China


Merging Fossil Specimens with Computer-Generated Information
Bimber, Gatesy, Witmer, Raskar, Encarnacao

IEEE Computer, Sept 2002


Free Form Sketching Using Variational Implicit Surfaces
O Karpenko, John F Hughes, R Raskar

Eurographics 2002, Saarbruken, Germany


Cartoon Diorama in Motion
R Raskar, R Ziegler, T Willwacher

NPAR 2002, Annecy, France


A Self Correcting Projector
R Raskar, P Beardsley

IEEE CVPR, Lihue, 2001
Related research on pocket projector


Shader Lamps
R Raskar, G Welch, K Low, D Bandyopadhyay

Eurographics Workshop on Rendering, London, 2001
Hardware Support for Non-photorealistic Rendering
R Raskar

Siggraph/Eurographics Graphics Hardware, LA, 2001 Source Code


A Low Cost Projector Mosaic
R Raskar, J vanBaar, J Chai

ACCV 2002
Related research on projectors.



Interacting with Spatially Augmented Reality
R Raskar, K Low

ACM Afrigraph, 2001


Dynamic Shader Lamps: Painting on Real Objects
D Badyopadhyay, R Raskar, H Fuchs

IEEE ACM Int. Symp. on Augmented Reality (ISAR), New York, 2001
Related research on Projector Based Augmented Reality 



Projected Imagery in Your Office of the Future
G Welch, H Fuchs, R Raskar, M Brown, H Towles
IEEE Computer Graphics and Applications, November 2000 


Immersive Planar Displays using Roughly Aligned Projectors
R Raskar   
IEEE VR 2000, March 2000, NJ



Multi-Projector Displays using Camera-Based Registration
R Raskar, M Brown, R Yang, W Chen, G Welch, H Towles, B Seales, H Fuchs
IEEE Visualization 99, Oct 1999, San Francisco, CA



Tabletop Spatially Augmented Reality: Bringing Physical Models to Life using Projected Imagery
R Raskar, G Welch, W Chen
Second Int Workshop on Augmented Reality (IWAR'99), October 1999, San Francisco, CA  



Oblique Projector Rendering on Planar Surfaces
R Raskar
ACM Siggraph 99 Sketch 


Image Precision Silhouette Edges
R Raskar, Michael Cohen
Symposium on Interactive 3D Graphics, Atlanta GA, April 1999  Source Code



Efficient Image Generation for Multiprojector and Multisurface Displays
R Raskar, Matt Cutts, Greg Welch, Wolfgang Stuerzlinger
Ninth EuroGraphics Rendering Workshop,(Appeared in Drettakis, G., Max, N. (eds.), Rendering Techniques '98. Proceedings of the Eurographics Workshop in Vienna, Austria, June 1998



Seamless Projection Overlaps Using Image Warping and Intensity Blending
R Raskar, G Welch, Henry Fuchs
Fourth International Conference on Virtual Systems and Multimedia, Gifu, Japan. Nov 1998. 



Spatially Augmented Reality
R Raskar, G Welch, H Fuchs
First International Workshop on Augmented Reality, San Francisco CA, Nov 1998. 


Augmented Reality Visualization for Laparoscopic Surgery
H Fuchs, M Livingston, R Raskar and others
First International Conference on Medical Image Computing and Computer-Assisted Intervention 1998. 



Simulation, Animation and Analysis of Design Disassembly for Maintainability Analysis
R Vujosevic, R Raskar, N Yetukuri
International Journal of Production Research, November 1995 



Incremental Path Generation for 6-DOF Robots
Ninth International Congress of Cybernetics and Systems, New Delhi, January 1993.




slc.gif (139
                  bytes)

Courses


MIT Elsewhere
Spring courses

Camera Culture (0-9-0) F 1:00-3:00pm


Imaging Culture (0-6-0) W 7:00-9:00pm


Fall course

Computational Camera and Photography (6-3-3)

Next billion cameras
R Raskar, S Seitz, A Efros
Siggraph 2009
[ Course Notes ]
Augmented Light Fields: Present and Future
R Raskar, S B Oh, A Accardi, Z Zhang
CVPR 2009
Advanced Topics in Computational Photography
R Raskar, J Tumblin
Siggraph 2008 [ Course Notes ]
Gradient Domain Techniques
A Agrawal, R Raskar
ICCV 2007, CVPR 2008 [ Course Notes ]

Computational Photography
R Raskar, J Tumblin
Siggraph 2006 and 2007 [ Course Notes ]
Computational Photography
R Raskar, J Tumblin
Siggraph 2005 [ Course Notes ]
 
Spatial Augmented Reality
O Bimber, R Raskar
Siggraph 2006 [ Course Notes ]
Spatial Augmented Reality: A Modern Approach to Augmented Reality
O Bimber, R Raskar
Siggraph 2005 [ Course Notes ]

Projectors: Advanced Graphics and Vision Techniques
R Raskar
Siggraph 2004 [ Course Notes in PDF ]
Modern Approaches to Augmented Reality
O Bimber and R Raskar
Eurographics 2004 [ Course Notes in PDF ]
Alternative Augmented Reality Approaches: Concepts, Techniques, and Applications
O Bimber and R Raskar
Eurographics 2003 [ Course Notes in PDF ]
Projectors: Advanced Geometric Issues in Applications
R Raskar
Siggraph 2003 [ Course Notes in PDF ]
Projector-based Graphics
R Raskar, E Lantz
Siggraph 2002 [ Course Notes in PDF ]

slc.gif (139
                  bytes)

Patents and Applications



Issued (40+)

1 7,792,423 4D light field cameras
2 7,780,364 Apparatus and method for reducing glare in images
3 7,756,407 Method and apparatus for deblurring images
4 7,738,725 Stylized rendering using a multi-flash camera
5 7,639,289 Increasing object resolutions from a motion-blurred image
6 7,580,620 Method for deblurring images using optimized temporal coding patterns
7 7,574,069 Retargeting images for small displays
8 7,454,136 Method and apparatus for acquiring HDR flash images
9 7,443,443 Method and apparatus for enhancing flash and ambient images
10 7,421,111 Light pen system for pixel-based displays
11 7,403,707 Method for estimating camera settings adaptively
12 7,359,562 Enhancing low quality videos of illuminated scenes
13 7,295,720 Non-photorealistic camera
14 7,292,269 Context aware projector
15 7,252,387 System for adjusting projector pose with six DoF for image alignment
16 7,242,818 Position and orientation sensing with a projector
17 7,229,023 Radio and optical identification tags
18 7,227,592 Self-correcting rear projection television
19 7,218,792 Stylized imaging using variable controlled illumination
20 7,206,449 Detecting silhouette edges in images
21 7,154,395 Interactive wireless tag location and identification system
22 7,137,707 Projector-camera system with laser pointers
23 7,103,227 Enhancing low quality images of naturally illuminated scenes
24 7,102,638 Reducing texture details in images
25 7,068,274 System and method for animating real objects with projected images
26 7,038,185 Camera for directly generating a gradient image
27 7,019,748 Simulating motion of static objects in scenes
28 7,001,023 Method calibrating projectors to with discrete optical sensors at surfaces
29 6,930,681 System for registering multiple images with three-dimensional objects
30 6,834,965 Self-configurable ad-hoc projector cluster
31 6,811,264 Geometrically aware projector
32 6,795,069 Free-form modeling of objects with variational implicit surfaces
33 6,793,350 Projecting warped images onto curved surfaces
34 6,781,591 Blending multiple images using local and global information
35 6,764,185 Projector as an input and output device
36 6,755,537 Method for globally aligning multiple projected images
37 6,741,248 Rendering geometric features of scenes models by individual polygons
38 6,733,138 Multi-projector mosaic with automatic registration
39 6,729,733 Method for a rectangular imaging within projected quadrilaterals
40 6,715,888 Method and system for displaying images on curved surfaces
41 6,709,116 Shape-adaptive projector system
42 6,677,956 Method for cross-fading multiple images of a scene for seamlessness
43 6,527,395 Method for calibrating a projector with a camera
44 6,520,647 Automatic keystone correction for projectors with arbitrary orientation


Public (USPTO link for before last 18 months,    45+)
Pending (   12+)


slc.gif (139
                  bytes)

Press

CNN   BBC  NPR  New York Times
 
New
                    Scientist  The Economist   Discovery

Recent Media Coverage

Coverage on Camera Culture group

NETRA: Cellphone-based Optometry Solution using Inverse Shack Hartmann Technique CNN, BBC, NPR, NewScientist
Looking Around Corners using Transient Imaging (Marr Prize#2) NYTimes, BBC, Economist
BiDi Screen: A Thin, Depth-Sensing LCD for 3D Interaction using Light Field Heterodyning   CNN, NewScientist, BBC
Bokode: A new optical tag ('barcode') using angular encoding and defocus based decoding  CNN, BBC, NPR

slc.gif (139
                  bytes)

Books

  Spatial Augmented Reality


(Also available as Free Download, Please make a donation at http://www.krebshilfe.de, http://www.cancer.org or http://www.ashanet.org/ )


Computational Photography: Mastering New Techniques for Lenses, Lighting, and Sensors


slc.gif (139
                  bytes)

Presentations and Talks



Keynote Addresses

Stereoscopic Displays & Applications, Jan 2011, San Francisco, CA (Keynote)
Imaging Science Hall of Fame Induction, Rochester, NY, Dec 2010 (Keynote)
Eighteenth Color Imaging Conference, San Antonio, TX, Nov 2010 (Opening Keynote)
6Sight Future of Imaging, ‘Computational Photography: The Future Starts Now’, Nov 2009, Monterrey, CA (Opening Keynote) (Slides) (Video)
Spring Conference on Computer Graphics, April 2008, 'Can Computer Graphics Impact Our Daily Life', Bratislava, Slovakia (Keynote)
Asian Conference on Computer Vision (ACCV), November 2007, Tokyo, Japan (Keynote) 'Less is More: Coded Computational Photography', (Slides)
Vision Modeling and Visualization, November 2007, Saarbrucken, Germany (Keynote) 'Coded Imaging', (Slides)
International Symposium on Mixed and Augmented Reality (ISMAR) October 2006, San Diego, 'The Poor Man's Palace: Special Effects in the Real World' (Opening Keynote) (Slides)
Imagina February 2006, Monaco (Keynote) (Slides)
Ciencia and Arte April 2005, Caxias do Sul, Brazil, ‘Active Imaging and Display’ (OpeningKeynote) (Slides)
Graphics Interface May 2005, Victoria, Canada, 'Spatial Augmented Reality: New Problems in Human Interface' (Opening Keynote) (Slides [17MB])
Symposium on Virtual Reality October 2004, Sao Paulo, Brazil, ‘Spatial Augmented Reality’ (Keynote) (Slides)

Invited Presentations

National Academy of Engineering, Frontiers of Engineering, Sept 2010
Frontiers in Optics, ‘Shift Glass’, Oct 2010, Rochester NY
Launch.org by NASA and USAID, Oct 2010, Kennedy Space Center, FL
• Imaging Systems, June 2010, Tucson, AZ
• FMX, ‘Imaging for Interaction’, Stutgart, Germany, May 2010
• MIT Technology Review (EmTech India 2010), ‘Future of Imaging’, Bangalore, India
• Frontiers in Optics, COSI, ‘Computational Photography’, Oct 2009, San Jose, CA
• MIT Technology Review (EmTech India 2009), ‘Next Billion Cameras’, New Delhi, Indi
a
Emerging Trends in Visual Computing (ETVC'08), Ecole Polytechnique (LIX), Paris, France, November 2008, 'Computational Photography: Epsilon to Coded Imaging' (Slides) (Video)
NIPS Symposium on Computational Photography, Vancouver, Canada, December 2008, 'Computational Photography: From Epsilon to Coded Photography', (Slides)
Flat Information Display November 2005, San Francisco, CA, 'Pocket Projector: All-in-one Solution for Mobility, Augmentation, Communication' (Slides)
Active RFID Summit November 2005, San Antonio, TX, 'RFIG : RFID + Geometry' (Slides)
8th Brazilian Conference of Design Research and Development - P&D DESIGN 2008, São Paulo, 'Next in Augmented Reality', (Slides)
ProCams October 2003, Nice, France, 'The Large Display Dilemma: Projectors are Dead' (Slides)


Panels

Symposium on Computational Photography and Video at MIT, What Will a Camera Look Like in 20 Years ? May 2005 (Organizer)
Siggraph 2005, The Ultimate Display: What Will It Be?, August 2005


Professional Activities

Papers Committee: Siggraph 2007, Siggraph 2008, Siggraph Asia 2009, Siggraph Asia 2010, Siggraph 2011
Papers Committee: Eurographics ( EG) 2005, Eurographics 2008, Eurographics 2009
Area Chair: CVPR 2009, ACCV 2008, ECCV 2010

Program Committee: International Conference on Computer Vision (IEEE ICCV) 2005, 2007
Program Committee: Computer Vision and Pattern Recognition (IEEE CVPR) 2005, 2006, 2007, 2008
Program Committee: European Conference on Computer Vision (ECCV) 2006, 2008
Program Committee: Computer Graphics International ( CGI) 2005
Program Committee: Computer Vision and Pattern Recognition (IEEE CVPR) 2004, 2005, 2006
Program Committee: Non-photorealistic Animation and Rendering (ACM NPAR) 2004, 2006
Program Committee: High Performance Computing - Virtual Reality (HPC) 2004
Program Committee: Sketch-based Interfaces (with Eurographics) 2004

Judging panel member for Global Indus Technovator Awards at MIT, 2004

Reviewer: SIGGRAPH(1999-2004-), Eurographics(2001-2003), EGRW/EGRS, I3D(2001,2003), IEEE Visualization(1999,2000,2002,2003), ISMAR (Mixed and Augmented Reality, 2003,2004), ACM Multimedia, NPAR (2002,2004), UIST(2001,2002), ACM TOG, IEEE CG&A, IEEE TVCG


Awards and Honors
  • Vodafone Wireless Innovation Award (First place), 2011
  • LAUNCH Health Innovation Award, presented by NASA, USAID, US State Dept and NIKE, 2010,
  • DARPA Young Faculty Award, 2010
  • Deshpande Center Ignition Grant 2010 (Netra Eyetest Device)
  • Marr Prize Honorable Mention (Looking Around a Corner with Transient Imaging), 2009
  • MIRU Best Paper Award
  • Alfred P. Sloan Foundation Research Fellowship, 2009
  • Mitsubishi Electric Valuable Invention Award, 2006 (For Photosensing Tags, RFIG)
  • Mitsubishi Electric Valuable Invention Award, 2006 (For Dome Screen Projector Display)
  • Dome Screen Projector for prestigious 'Best 10 New Products' Prize in Japan given by Nikkan Kogyo Simbun
  • Technology Review's Top Young Innovators Under 35 (TR100), 2004 (More links)
  • Mitsubishi Electric Valuable Invention Award, 2004
  • MIT Global Indus Technovator Awards (Top 20 Indian Researcher), 2003
  • Mitsubishi Electric Information Technology R&D Award, 2003
  • Link Fellowship Award, 1998-99
  • The undergraduate degree project in robotics awarded "Best Project of the University" prize by IEEE chapter of Pune Institute of Computer Technology , 1991
  • Best student paper, "Incremental Curve Generation for Robot Path Planning" at state level technical paper contest, 1991
  • Ranked First among 500,000 students appearing for entrance exam (HSC Merit List) for the bachelor's program in the state of Maharashtra, India, 1987
  • Selected among TOP 20 to represent India at the International Mathematics Olympiad, 1987

Art/Tech Shows

2010
Slow Display, Emerging Tech, SIGGRAPH, Jul 2010, Los Angles
2009

Living Windows (6D Display), Ars Electronica, Sept 2009, Linz, Austria
2007
Prakash: Lighting Aware Motion Capture for Dynamic Virtual Sets, Emerging Tech, SIGGRAPH, Aug 2007, San Diego

2006
Instant Replay Air Hockey , Emerging Tech, SIGGRAPH, Aug 2006, Boston
2005
Contours with Multi-flash , CVPR 2005, June 2005, San Diego
2004
Camera Non-photo , COLLISION Senses Show, Nov 2004, Cambridge
Stylized Images with NPR Camera , Emerging Tech, SIGGRAPH, Aug 2004, Los Angeles
Interacting With Projections Using iLamp Projectors , Emerging Tech, SIGGRAPH, Aug 2004, Los Angeles

2001
Contra Inertia , superCOLLISION Show, Oct 2001, Cambridge