Occluded Imaging with Time of Flight Sensors

Achuta Kadambi1   Hang Zhao2   Boxin Shi1,2   Ramesh Raskar1

1MIT Media Lab    2Singapore University of Technology and Design   

ACM Transactions on Graphics, Vol 35 Issue 2 (Presented at SIGGRAPH 2016)

Localizing a moving target in real-time. (a) The ping pong ball is outside the line of sight of the camera and moves horizontally. The camera measures backscattered reflections from the wall. (b) A conventional video of the object in motion. Here, three frames of the video are shown at times 0, 1 and 2 seconds where the ball position is 30, 40, and 50 cm. (c) Our technique is able to find the location of the target in real time. Please see supplementary video.

Tracking a Ping Pong Ball Around the Corner


We explore the question of whether phase based Time of Flight (ToF) range cameras can be used for looking around corners and through scattering dif- fusers. By connecting time of flight measurements with theory from array signal processing we conclude that performance depends on two primary factors: camera modulation frequency and the width of the specular lobe (“shininess”) of the wall. For purely Lambertian walls, commodity time of flight sensors achieve resolution on the order of meters between targets. For seemingly diffuse walls, such as posterboard, the resolution is drastically reduced, to the order of ten centimeters. In particular, we find that the re- lationship between reflectance and resolution is nonlinear—a slight amount of shininess can lead to a dramatic improvement in resolution. Since many realistic scenes exhibit a slight amount of shininess we believe that off-the- shelf ToF cameras can look around corners.

Technical Paper

Occluded Imaging with Time of Flight Sensors, ACM Transactions on Graphics 2016 [PDF] [PDF Low Res]



 author = {Kadambi, Achuta and Zhao, Hang and Shi, Boxin and Raskar, Ramesh},
 title = {Occluded Imaging with Time-of-Flight Sensors},
 journal = {ACM Trans. Graph.},
 issue_date = {May 2016},
 volume = {35},
 number = {2},
 publisher = {ACM},
 address = {New York, NY, USA},
 keywords = {Time of flight, computational photography, scattering},


Achuta Kadambi
MIT Media Lab