slc.gif (139 bytes)

Ramesh  Raskar
Ramesh Raskar
MIT Media Lab , Associate Professor
Tufts Medical CTSI, Adjunct    
   
 
Camera Culture group, Leader   | 
AI for Impact MIT Initiative  | 
REDX.io Innovating for Billions
  |   Center for Future Storytelling, Co-director
AI for Health, Botnar Foundation

Short Bio
Ramesh Raskar is an Associate Professor at MIT Media Lab and directs the Camera Culture research group. His focus is on AI and Imaging for health and sustainability. They span research in physical (e.g., sensors, health-tech), digital (e.g., automated and privacy-aware machine learning) and global (e.g., geomaps, autonomous mobility) domains. He received the Lemelson Award (2016), ACM SIGGRAPH Achievement Award (2017), DARPA Young Faculty Award (2009), Alfred P. Sloan Research Fellowship (2009), TR100 Award from MIT Technology Review (2004) and Global Indus Technovator Award (2003). He has worked on special research projects at Google [X] and Facebook and co-founded/advised several companies.

Research Areas and Interests
Contact Info
fb.com/rraskar, @raskarmit, LinkedIn



For Students

How to give a talk
Open Problems in Comp Photo
Quick paper outline
Preparing Masters Thesis
Why 'Media' in Media Lab Name

Innovation Resources
 How to invent: The Idea Hexagon
How to G.R.O.W. your research
Continuous Sustained Innovation
Innovation Choices
Imaging Startups

Apply for 'Pathfinder' MEngg, RA, UROP and
Post-doc slots in Camera Culture group

slc.gif (139
                  bytes)

Projects


See Camera Culture group page for latest

Videos and Key Presentations

Computational Health 2019
  Privacy Preserving Machine Learning for Health, Video1, Video2

  Augmented Surgeons: Population Heath with AI AR and IA
Lemelson Award 2016:

  Article, Patents
  https://www.youtube.com/watch?v=9c3PxR3s72k
  https://www.technologyreview.com/video/604685/emtech-2016-presented-by-lemelson-mit/

Siggraph Achievement Award 2017:

  Video, Article: https://www.media.mit.edu/posts/ramesh-raskar-to-receive-the-acm-siggraph-achievement-award/

TED.com: Trillion frames per second imaging    

   http://www.ted.com/talks/ramesh_raskar_a_camera_that_takes_one_trillion_frames_per_second.html

UIST Keynote 2015

   Innovating for Billions, UIST Keynote Presentation, Stanford HCI

DARPA Wait What: Extreme Computational Imaging  (Overview of our Imaging research projects)

   https://www.youtube.com/watch?v=ZTRWjSIqARk

TEDMED: Eye is a window to your health  (Retinal and refraction measurements for prediction of systemic health)

   http://www.tedmed.com/talks/show?id=103806

Wired  (How to use the ‘Idea Hexagon’ to invent new ideas)

   http://www.wired.co.uk/magazine/archive/2012/11/start/inventing-a-new-field-in-vision

   http://www.wired.co.uk/news/archive/2012-10/26/ramesh-raskar

Google SolveForX

  https://www.solveforx.com/moonshots/eyes-are-the-window-to-health-ramesh-raskar-at-tedxyouthbeaconstreet

TEDxBoston : Eye Tests on Mobile Phones

   http://tedxboston.org/speaker/raskar

TEDxBeaconstreet: Idea Hexagon

   http://tedxbeaconstreet.com/rameshraskar/

6Sight: Future of Imaging, Future of Photography. Open challenges presented in 2008 but still relevant a decade later.

  https://www.youtube.com/watch?v=1bGVKvXG4dU



Automated Machine Learning and Computational Health
         
Privacy Preserving Distributed Machine Learning for Health Data, Summary Video
         
Architecture Sweep (ICLR 2017)
          Neural Network Performance Prediction
          Data Exchanges (2017)
          See arXiv page for details


Tech, HCI and AI for Impact
AI for Impact, Emerging Worlds initiative,
Innovating for Billions, UIST Keynote Presentation
LVP-MITRA for Low-Vision Solutions, Video Engineering the Eye
REDX.io for Young Innovators via Lemelson Foundation
Kumbhathon, Pop-up City, (Article)

Computational Sustainability and Visual Social Computing via Large Scale Image Based Studies of Environments
            DeepGlobe paper and keynote
            Street Address for All, Robocodes: Towards Generative Street Addresses from Satellite Imagery
            Predictors of physical urban change, Proc. of the National Academy of Sciences, 2017, StreetChange Project
            Deep learning the City, Quantifying Urban Perception At A Global Scale, ECCV 2016
            Streetscore – Predicting the Perceived Safety of One Million Streetscapes

Geocoding
 
Geo Timelines  
GeoViz  
GeoSemantics
Angled Graphs
Street Address for All

StreetChange

Urban Perception

StreetScore
 
CrowdCam
Robocode for Geocoding

StreetChange

Urban Perception
StreetScore

crowdCam
2017

2017
2016
2014
2005

Computational Light Transport

The research goal is to create an entirely new class of imaging devices and a novel theoretical framework for inverse problems in space-time analysis of light transport. Two novel forms of imaging show great potential for research and practical applications: (i) time resolved transient imaging that exploits multi-path analysis and (ii) angle resolved imaging without lenses for displays, medical devices and phase analysis. Can we photograph objects that are not in the direct line of sight? Can we automatically identify materials (e.g., wood, metal or skin) using a single photo? Can we build portable CAT-scanning machines with no moving parts? Can we convert LCDs into large lensless cameras? Can exploit high resolution of displays to create unusual scientific and medical instruments? Underlying theoretical contributions include multi-path analysis, spatial heterodyning, augmented light fields and algebraic rank-constraints of 3D displays.



Femto-Photography
 
Multi-path
Analysis
 
Spatial Heterodyning
 
Augmented Light Field  
Rank-constraint of
3D Displays
 
Computational Probes
 
Randomized
Projections


Looking Around Corners, Trillion FPS imaging

CAT-Scan without
moving parts

BiDi Screen

Geometric
+ Wave optics

Holograms +
Parallax Barriers

Bokode
Coded Strobing


Looking Around the corner using
                                Transient Imaging

CAT Scan


BiDi Screen

Augmented Light Field



Hologram + Parallax Barrier

Bokode

Coded Strobing using Compressive
                            Sensing



Descattering
Shield Fields
Wave-BSDF
Content Adaptive

NETRA for Optometry
Sparsity Analysis



CAT Scan
                                2
Shield Fields
WBSDF
HR3D:
                                  Content-Adaptive Parallax Barriers
NETRA
Sparsity Analysis


2008 -  (ICCV 2009, CVPR 2011, SigA 2011,
Nature Communications 2012)

CVPR 2010
ECCV 2010

Siggraph Asia 2008
Siggraph Asia 2009

Eurographics 2009
ICCP 2011

Siggraph Asia 2010
Siggraph 2011
Siggraph 2012

Siggraph 2009
Siggraph 2010
Siggraph 2011

PAMI 2011



Computational Photography

Computational Photography is an emerging multi-disciplinary field that is at the intersection of optics, signal processing, computer graphics+vision, electronics, art, and online sharing in social networks. The field is evolving through three phases. The first phase was about building a super-camera that has enhanced performance in terms of the traditional parameters, such as dynamic range, field of view or depth of field. I call this Epsilon Photography. Due to limited capabilities of a camera, the scene is sampled via multiple photos, each captured by epsilon variation of the camera parameters. It corresponds to the low-level vision: estimating pixels and pixel features. The second phase is building tools that go beyond capabilities of this super-camera. I call this Coded Photography. The goal here is to reversibly encode information about the scene in a single photograph (or a very few photographs) so that the corresponding decoding allows powerful decomposition of the image into light fields, motion deblurred images, global/direct illumination components or distinction between geometric versus material discontinuities. This corresponds to the mid-level vision: segmentation, organization, inferring shapes, materials and edges. The third phase will be about going beyond the radiometric quantities and challenging the notion that a camera should mimic a single-chambered human eye. Instead of recovering physical parameters, the goal will be to capture the visual essence of the scene and analyze the perceptually critical components.  I call this Essence Photography and it may loosely resemble depiction of the world after high level vision processing. It will spawn new forms of visual artistic expression and communication. Related work


Coded Time (Exposure)
 
Coding in Space
 
Coded Illumination  
Coded Wavelength
Coded Sensing
Flutter Shutter Cam
for motion deblurring

Coded Aperture for
Extended DoF
Mask-based Optical
Heterodyning for
Light Field Capture

Multi-flash Camera
for Depth Edge
Detection

Agile Spectrum for
programmable
wavelength
 
Gradient Encoding
Sensor for HDR
Flutter Shutter Camera

Encoded Blur Camera for Digital
                                Refocussing LightField Camera

Multi Flash
                                      Camera
Agile Spectrum Imaging

Cliques for computing forward
                              difference
2006

2007
2007
2004
2008
2005


Computational Illumination: Projector as the dual of a camera
Traditionally, a projector is considered a display device that maps a 2D image onto a larger flat screen. In our research, we have questioned this restrictive consideration. By treating projector as the dual of a camera – that is, a 3D projective device that maps 2D pixels to 4D rays – we have shown that programmable illumination can be used in a versatile fashion for display, augmentation, optical communication and mobilityPrevious related work

Disclosed Product Impact:
Acute Angle Seamless Multiprojector Display
MPC Polgyonal 3D scene display
Multiprojector quadric curved immersive display
Six Projector 3D rendering on curved display (A detailed press release)

Intelligent Self-ID and Tagging

Computational probes that support the ability to identify and locate hundreds of tags in real time with milimeter precision. We bring RFID and wireless tags into the realm of computer vision, graphics and HCI by supporting precise geometric analysis of tagged objects and surfaces using a composite RF and optical communication. Related work


Office of the Future
Multi-Projector
Display

Spatial Augmented
Reality (SAR)

Pocket Projector


Optical
Communication

3D videoconferencing

Quadric Transfer

Shader Lamps

iLamps

Location Tracking

Office of the Future
Multi Projector Display

Shader Lamps

iLamps
                                      Siggraph 2003
RFIG and Motion Capture

1998

1999

1998

2000

2003
RFIG
Prakash

Bokode
Photosensing RFID
+Pocket Projector

Lighting-Aware
Motion Capture

Angular Encoding
RFIG

Prakash
                                                      Motion Capture

Bokode
2004

2007

2009

Non photorealistic Rendering

Features such as silhouette edges, ridges, valleys and intersections of geometric primitives are useful in conveying shape in illustrations and art. We have developed a proactive imaging method to capture such features in real world scenes of various scales. One area of focus is shape clarifying multi-flash imaging for endscopes. For polygonal scenes, our focus has been real-time rendering of these features without requiring the connectivity information. Previous related work

Intelligent user interfaces

We are developing user interaction methods with novels gadgets and sensors. We have also exploited variational implicit surfaces for free form sketching and manipulation. Previous related work


slc.gif (139
                    bytes)

Publications



For publications please see the Camera Culture group page or Google Scholar list


slc.gif (139
                  bytes)

Courses


MIT
Spring courses

Mathematical Methods for Imaging (0-9-0) F 1:00-3:00pm


Imaging Ventures (0-6-0) W 7:00-9:00pm


Fall course

Health Technologies (0-9-0)

Computational Camera and Photography (6-3-3)

slc.gif (139
                  bytes)

Patents and Applications



List of Issued Patents (80+)

Patent Analysis for Lemelson Award

Public (USPTO link for before last 18 months,    45+)
 

Pending (   12+)

Raskar 75 Patents for Lemelson
                                    Award 2016


slc.gif (139
                  bytes)

Press

 
  
     

Recent Media Coverage

Coverage on Camera Culture group

NETRA: Cellphone-based Optometry Solution using Inverse Shack Hartmann Technique CNN, BBC, NPR, NewScientist
Looking Around Corners using Transient Imaging (Marr Prize#2) NYTimes, BBC, Economist
BiDi Screen: A Thin, Depth-Sensing LCD for 3D Interaction using Light Field Heterodyning   CNN, NewScientist, BBC
Bokode: A new optical tag ('barcode') using angular encoding and defocus based decoding  CNN, BBC, NPR

slc.gif (139
                  bytes)

Books

Spatial Augmented Reality


(Also available as Free Download, Please make a donation at http://www.krebshilfe.de, http://www.cancer.org or http://www.ashanet.org/ )


Computational Photography: Mastering New Techniques for Lenses, Lighting, and Sensors


slc.gif (139
                  bytes)

Presentations and Talks



Keynote Addresses

Stereoscopic Displays & Applications, Jan 2011, San Francisco, CA (Keynote)
Imaging Science Hall of Fame Induction, Rochester, NY, Dec 2010 (Keynote)
Eighteenth Color Imaging Conference, San Antonio, TX, Nov 2010 (Opening Keynote)
6Sight Future of Imaging, ‘Computational Photography: The Future Starts Now’, Nov 2009, Monterrey, CA (Opening Keynote) (Slides) (Video)
Spring Conference on Computer Graphics, April 2008, 'Can Computer Graphics Impact Our Daily Life', Bratislava, Slovakia (Keynote)
Asian Conference on Computer Vision (ACCV), November 2007, Tokyo, Japan (Keynote) 'Less is More: Coded Computational Photography', (Slides)
Vision Modeling and Visualization, November 2007, Saarbrucken, Germany (Keynote) 'Coded Imaging', (Slides)
International Symposium on Mixed and Augmented Reality (ISMAR) October 2006, San Diego, 'The Poor Man's Palace: Special Effects in the Real World' (Opening Keynote) (Slides)
Imagina February 2006, Monaco (Keynote) (Slides)
Ciencia and Arte April 2005, Caxias do Sul, Brazil, ‘Active Imaging and Display’ (OpeningKeynote) (Slides)
Graphics Interface May 2005, Victoria, Canada, 'Spatial Augmented Reality: New Problems in Human Interface' (Opening Keynote) (Slides [17MB])
Symposium on Virtual Reality October 2004, Sao Paulo, Brazil, ‘Spatial Augmented Reality’ (Keynote) (Slides)

Invited Presentations

National Academy of Engineering, Frontiers of Engineering, Sept 2010
Frontiers in Optics, ‘Shift Glass’, Oct 2010, Rochester NY
Launch.org by NASA and USAID, Oct 2010, Kennedy Space Center, FL
• Imaging Systems, June 2010, Tucson, AZ
• FMX, ‘Imaging for Interaction’, Stutgart, Germany, May 2010
• MIT Technology Review (EmTech India 2010), ‘Future of Imaging’, Bangalore, India
• Frontiers in Optics, COSI, ‘Computational Photography’, Oct 2009, San Jose, CA
• MIT Technology Review (EmTech India 2009), ‘Next Billion Cameras’, New Delhi, Indi
a
Emerging Trends in Visual Computing (ETVC'08), Ecole Polytechnique (LIX), Paris, France, November 2008, 'Computational Photography: Epsilon to Coded Imaging' (Slides) (Video)
NIPS Symposium on Computational Photography, Vancouver, Canada, December 2008, 'Computational Photography: From Epsilon to Coded Photography', (Slides)
Flat Information Display November 2005, San Francisco, CA, 'Pocket Projector: All-in-one Solution for Mobility, Augmentation, Communication' (Slides)
Active RFID Summit November 2005, San Antonio, TX, 'RFIG : RFID + Geometry' (Slides)
8th Brazilian Conference of Design Research and Development - P&D DESIGN 2008, São Paulo, 'Next in Augmented Reality', (Slides)
ProCams October 2003, Nice, France, 'The Large Display Dilemma: Projectors are Dead' (Slides)


Awards and Honors
  • ACM SIGGRAPH Achievement Award, 2017
  • Lemelson Prize awarded to mid career impact inventors, 2016  (news),  Video, Citation Patents
  • 'PharmaVoice 100' awarded to 100 health sciends ce innovators, 2013
  • Edison Awards for Innovation and Innovators, (for EyeNetra), 2012
  • Popular Mechanics Breakthrough Award, 2012
  • Vodafone Wireless Innovation Award (First place), 2011
  • LAUNCH Health Innovation Award, presented by NASA, USAID, US State Dept and NIKE, 2010,
  • DARPA Young Faculty Award, 2010
  • Deshpande Center Ignition Grant 2010 (Netra Eyetest Device)
  • Marr Prize Honorable Mention (Looking Around a Corner with Transient Imaging), 2009
  • MIRU Best Paper Award
  • Alfred P. Sloan Foundation Research Fellowship, 2009
  • Mitsubishi Electric Valuable Invention Award, 2006 (For Photosensing Tags, RFIG)
  • Mitsubishi Electric Valuable Invention Award, 2006 (For Dome Screen Projector Display)
  • Dome Screen Projector for prestigious 'Best 10 New Products' Prize in Japan given by Nikkan Kogyo Simbun
  • Technology Review's Top Young Innovators Under 35 (TR100), 2004 (More links)
  • Mitsubishi Electric Valuable Invention Award, 2004
  • MIT Global Indus Technovator Awards (Top 20 Indian Researcher), 2003
  • Mitsubishi Electric Information Technology R&D Award, 2003
  • Link Fellowship Award, 1998-99
  • The undergraduate degree project in robotics awarded "Best Project of the University" prize by IEEE chapter of Pune Institute of Computer Technology , 1991
  • Best student paper, "Incremental Curve Generation for Robot Path Planning" at state level technical paper contest, 1991
  • Ranked First among 500,000 students appearing for entrance exam (HSC Merit List) for the bachelor's program in the state of Maharashtra, India, 1987
  • Selected among TOP 20 to represent India at the International Mathematics Olympiad, 1987

Art/Tech Shows

2010
Slow Display, Emerging Tech, SIGGRAPH, Jul 2010, Los Angles
2009

Living Windows (6D Display), Ars Electronica, Sept 2009, Linz, Austria
2007
Prakash: Lighting Aware Motion Capture for Dynamic Virtual Sets, Emerging Tech, SIGGRAPH, Aug 2007, San Diego

2006
Instant Replay Air Hockey , Emerging Tech, SIGGRAPH, Aug 2006, Boston
2005
Contours with Multi-flash , CVPR 2005, June 2005, San Diego
2004
Camera Non-photo , COLLISION Senses Show, Nov 2004, Cambridge
Stylized Images with NPR Camera , Emerging Tech, SIGGRAPH, Aug 2004, Los Angeles
Interacting With Projections Using iLamp Projectors , Emerging Tech, SIGGRAPH, Aug 2004, Los Angeles

2001
Contra Inertia , superCOLLISION Show, Oct 2001, Cambridge