Home   |   News   |   Join Us   |   People   |   Projects   |   Publications   |   Talks   |   Courses


NETRA: Refractive Tests on a Mobile Phone


Inverse Shack-Hartmann Wavefront Sensor
using High Resolution Mobile Phone Display



Vitor F. Pamplona*        Ankit Mohan        Manuel M. Oliveira*      Ramesh Raskar
David Schafran           Erick Passos      Everett Lawson  

Camera Culture Group - MIT Media Lab
* Visiting from Instituto de Informática - UFRGS

See our latest works on Cataract Mapping and tailored displays

SIGGRAPH'10 Paper      Vitor's PhD. Thesis      Frequently Asked Questions     Hi-Res Free Images     Sign up

NETRA Concept
 
OneSight's Mission Trip - Nairoby, Kenya                   LVPEI's Clinical Trials - Hyderabad, India
Figure 1: Netra is a $2 clip-on eyepiece that goes on top of a cell phone. The user looks through this eye piece and interactively aligns the displayed patterns by clicking the buttons. The number of clicks required to bring the patterns into alignment indicates the refractive error. Our optometry solution combines inexpensive optical elements and interactive software components to create a new portable and inexpensive device that mimics (and replaces) the expensive laser-based Shack-Hartmann wavefront aberrometers. Ilustration: Tiago Allen

Abstract

We introduce a portable and inexpensive solution for estimating refractive errors in the human eye. While expensive optical devices for automatic estimation of refractive correction exist, our goal is to greatly simplify the mechanism by putting the human subject in the loop. A traditional Shack-Hartmann wavefront sensor uses lasers and highly sensitive digital sensor which makes those solutions expensive, bulky and requires trained professionals.

Our solution creates an inverse Shack-Hartmann sensor. It is based on a high-resolution programmable display and combines inexpensive optical elements, interactive GUI, and computational reconstruction. The key idea is to interface a lenticular view-dependent display with the human eye at close range - a few millimeters apart. Via this platform, we create a new range of interactivity that is extremely sensitive to parameters of the human eye, such as the refractive errors, focal range, focusing speed, lens opacity, etc. We propose several simple optical setups, verify their accuracy, precision, and validate them in a user study.

Awards

1st prize




LAUNCH Health Innovation Award

Lemelson-MIT Program award



Ignition Grant



Innovation Grant

Research Papers

  1. Vitor F. Pamplona
    Interactive Measurements and Tailored Displays for Optical Aberrations of the Human Eye
    PhD Thesis
    2012.
    Draft Version.


  2. Vitor F. Pamplona, Ankit Mohan, Manuel M. Oliveira, Ramesh Raskar.
    NETRA: Interactive Display for Estimating Refractive Errors and Focal Range.
    Proc. of SIGGRAPH 2010
    (ACM Transactions on Graphics 29, 4), 2010.
    Paper


  3. Vitor F. Pamplona, Ankit Mohan, Manuel M. Oliveira, Ramesh Raskar.
    Dual of Shack-Hartmann Optometry Using Mobile Phones.
    Proc. of Frontiers in Optics, Optical Society of America. Oct. 24, Rochester, New York. 2010.
    Paper

  4. Vitor F. Pamplona, Ankit Mohan, Manuel M. Oliveira, Ramesh Raskar.
    Low-cost and Portable tool for Measuring Eye Refractive Disorders using Active Participation.
    Proc. of 88th Annual Meeting of the American Academy of Optometry
    , San Francisco.
    Abstract

  5. Veerendranath Pesala, Sangeetha Srinivasan, Ethan Solomon, Vitor F. Pamplona, Manuel M. Oliviera, Ramesh Raskar, Shrikant Bharadwaj.
    Comparison of a Novel Cell Phone-Based Refraction Technique (NETRA) With Objective Clinical Retinoscopy.
    ARVO 2011
    Abstract

  6. Vitor F. Pamplona, Ankit Mohan, Manuel M. Oliveira, Ramesh Raskar.
    NETRA: A Novel Clip-on Eye-piece for Mobile Phone based Optometry Solution and Low Cost Eye Tests.
    mHealth Summit 2010
    .

  7. Martha P. Lang, Helena M. Pakter, Lisia B. Ferreira, Ankit Mohan, Ramesh Raskar, Vitor F. Pamplona, Manoel M. Oliveira
    Comparison of a Cell Phone-Based Refraction Technique (NETRA) With Auto-Refraction.
    ARVO 2012
    Abstract

  8. Bruce D. Moore, Nadine Solaka, Vitor F. Pamplona, David Schafran, Amy Canham, Ramesh Raskar, Hilary Gaiser.
    Comparison of a Novel Cell Phone-based Refraction Technique (NETRA-G) with Subjective Refraction. AAO 2012, Chicago, IL.
    Abstract Suplement
    









Worldwide Impact

    29 research teams in 14 countries already have our prototypes.
NETRA worldwide
Collaboration opportunities and clinical studies, please introduce yourself at our google group.
Please look at FAQ for more information. Press package, pictures and illustrations here.

Videos

TEDxBoston: EyeExams: there is an app for that.

MIT Media Lab LabCast: Credit, Paula Aguilera and Jonathan Williams

NETRA at NASA Launch

CNN - NETRA: Cheap, Portable Eye Exam System

Ramesh Raskar @ NASA Launch

1st Prize: Vodafone Wireless Innovation Project 2011



More videos here.

Slide Set

NETRA on SIGGRAPH 2010
View more presentations from Vitor Pamplona.

Brief Technical Description

The NETRA system uses the dual of a Shack-Hartman sensor, and replaces the laser with simple user interaction. Todayś methods using the Shack-Hartmann sensor shine a laser into the eye of the patient, and measure the reflected light with a wavefront sensor. Hence they are quite expensive, and require a trained professional operator.  A cell phone based solution significantly reduces the cost of the device and makes it appropriate for self-evaluation, while still providing comparable data.

The subject looks into this display at a very close range and aligns (overlaps) displayed patterns (Figure 1). Since the light rays from these patterns pass through different regions of the visual system, the alignment task gives a measure of the optical distortions of those regions. The subject repeats this procedure for a few meridians with appropriate variation in the patterns. The system computes the corresponding refractive error for myopia, hyperopia and astigmatism.

Prototype Nexus One Prototype
Figure 2: Current prototypes using the Samsung Behold II and the Nexus One. We place an optical phase plate to create virtual images and achieve 0.6 and 0.4 diopter resolution respectively.

Evaluation: We tested accuracy and precision of the technique in two experiments: (i) using lenses and a SLR camera and (ii) comparing our device against actual prescriptions in a user study. The resolution is 0.4 diopters using the Nexus One device (focal length 30mm). The Apple iPhone 4G, with the new Retina Display should achieve a resolution of approximately 0.28 diopters (focal length 30mm). For measuring eye correction, the average absolute errors from the known prescriptions were under 0.5 diopter (σ = 0.2) for both cylindrical and spherical powers. The average absolute error of our estimates of the cylindrical axis was under 6 degrees. Optometrists typically prescribe in multiples of 0.25 diopter, and 10 degrees axis.

In controlled user experiments with 16 subjects, the average absolute errors from the known prescriptions were under 0.5 diopter, with a standard deviation of 0.2 diopter for both cylindrical (astigmatism) and spherical powers (myopia and hyperopia). The average absolute error of the cylindrical axis is less than 6 degrees. We are able to achieve this without the use of cycloplegic eye drops for relaxing accommodation.

Limitations: Since our solution relies on subjective feedback, it cannot be used by individuals who cannot reliably perform the user-required tasks, such as very young children.

Existing Techniques

Existing systems to diagnose refractive eye conditions include Snellen charts (with a set of trial lenses), auto-refractometers, and wavefront aberrometers.  The NETRA solution offers some unique benefits over these existing techniques, which make it specially suited for deployment in developing countries:

Current solutions for analyzing refractive errors.
Figure 3: Current solutions for analyzing refractive errors. Subjective Methods (far left and center) rely upon the user's judgment of sharpness or blurriness of a test object. Objective Methods (far right) require a mechanically moving lens, a camera, a trained technician, and a large investment.

Refraction Services Requirement on Developing Countries [Vision 2020 Report]: The following table provides a comprehensive list of techniques and equipment for assessing refractive conditions of an eye.

Technique

Objectivity

Speed

Accuracy/ Reliability

Electricity Requirements

Mobility

Training

Equipment Requirements (Cost bracket)*

Cost Efficiency rank

Suitability for Children

Retinoscopy (Slit Lamp)

Objective = does not rely on patient responses

Fast

+/- 0.50D unless affected by media opacities or accommodation

Batteries

Good

High

Retinoscope, plus trial lens set and trial frame ($2000), OR phoropter ($1600), OR variable focus specs ($1600)

Economical – low up-front cost, high durability, low maintenance

Sometimes

Subjective refraction (Eye Charts)

Subjective = does rely on patient responses

Slow

+/- 0.25D but dependent on patient reliability

None

Good

High

Trial lens set and trial frame ($1400), OR phoropter ($1000), OR variable focus specs ($600)

Economical

Sometimes (only with experienced practitioners)

Auto Refraction

Objective

Fast

Relies on both equipment and patient factors

Mains

Low

Basic

Auto-refractor ($15K)

Expensive

No

Portable Auto Refraction

Objective

Fast

Relies on both equipment and patient factors

Mains or batteries

Good

Basic

Portable auto-refractor ($20K)

Expensive

No

NETRA

Subjective

Fast

Relies on both equipment and patient factors

< 0.50D unless affected by accommodation

Cell phone batteries

Excellent

Basic

Plastic Piece ($2) and a cell phone ($300).

Economical

Yes

* Costs were extracted from the Vision2020 report. Some cheaper options may exist - for example, we were able to acquire a set of trial lenses for $300. Note that simple reading charts can be expensive because they must be used under optimal lighting conditions and need a set of trial lenses.

Potential Impact

More than two billion people worldwide have refractive error. Very few have access to quality eye-care because existing solutions require a trained optometrist or expensive equipment [VISION2020 Report, Holden2007]. This impacts the developing world in a significant way:

* WHO definition for blindness: vision worse that 3/60 in the better eye.

Uncorrected refractive problems may lead to a significant loss in productivity, with estimates ranging from USD 88.74 to USD 133 billion. To put things in perspective, this productivity loss exceeds the annual GDP of 46 of the 52 African countries. Our technology can address all types of refractive errors.


Acknowledgments: Thanks to the volunteers who tested our device, Xiaoxi Wang and Andrew Song for prototype construction, Tiago Allen Oliveira for illustrations, Tyler Hutchison for the video voiceover, Taya Leary for her relentless support, the entire Camera Culture group for all the useful discussions; and the reviewers for their valuable feedback. Dr. James Kobler (Mass. General Hospital), Dr.  Dr. Joseph Ciolino (Mass. Eye and Ear Infirmary), and Dr. Fuensanta Vera Diaz (Schepens Eye Research Institute) provided valuable resources and insightful discussions about optometry and ophthalmology. We thank Dick Lyon and Google's open source office for Google Nexus One mobile phone, and Samsung Electronics for Samsung Behold II mobile phone. Vitor and Manuel acknowledge CNPq-Brazil fellowships 142563/2008-0, 200763/2009-1, 200284/2009-6, 476954/2008-8. Ramesh Raskar is supported by an Alfred P. Sloan Research Fellowship.

Media Coverage

Pictures

Group Picture
NETRA team: Ankit Mohan, Manuel M. Oliveira Neto, Vitor Pamplona and Ramesh Raskar at the MIT Media Lab.
Photo: Jonathan Williams
Astronaut Ron using NETRA
Astronaut Ron Garan testing NETRA at the Kennedy Space Center

© Mike Ritter / ritterbin.com

© Mike Ritter / ritterbin.com

© Mike Ritter / ritterbin.com

© Mike Ritter / ritterbin.com

© Mike Ritter / ritterbin.com

© Mike Ritter / ritterbin.com






Testing NETRA
Picture Credit: Anderson Maciel
Testing NETRA
Picture Credit: Anderson Maciel
Testing NETRA
Picture Credit: Anderson Maciel
UFRGS Vice Reitor Rui Oppermann checking NETRA out
Picture Credit: Anderson Maciel
NETRA at AAO
Image Credit:Dominick M Maino
NETRA Poster at AAO


(Credit: Ethan Solomon)

(Credit: Ethan Solomon)

(Credit: Ethan Solomon)

(Credit: Ethan Solomon)

(Credit: Ethan Solomon)



Andrew Bastawrous

Matthew Burton

Hannah Kuper



NETRA in Brazil
NETRA in Brazil
NETRA in Brazil





Dr. Vicki Chen at Tufts

Vitor Pamplona demonstrates NETRA
Vitor Pamplona demonstrates NETRA
Vitor Pamplona demonstrates NETRA.
Vitor Pamplona demonstrates NETRA
Manuel Oliveira demosntrates NETRA
Manuel M. Oliveira demonstrates NETRA
Manuel M. Oliveira and Vitor Pamplona
Manuel and Vitor
Ramesh Raskar demosntrates NETRA
Ramesh Raskar demonstrates NETRA
Ramesh Raskar and Ankit Mohan
Ramesh and Ankit
Ankit Mohan
Ankit Mohan demonstrates NETRA
Ankit Mohan demonstrates NETRA
Ankit Mohan demonstrates NETRA

Vitor Pamplona, Margaret McKenna, Ankit Mohan and Chika Ekeji.
Photo by Xun (Helen) Hou

Vitor Pamplona, Margaret McKenna, Ankit Mohan and Chika Ekeji.
Photo by Xun (Helen) Hou

Margaret McKenna, Chika Ekeji, Ankit Mohan and Vitor Pamplona.
Photo by Xun (Helen) Hou

Photo by Xun (Helen) Hou
People using the Samsung Behold setup
Photo: Michal Massey
PerfectSight: Prototypes ready for Siggraph
PerfectSight: Viewmaster Prototype
PerfectSight: Final Behold protytpe

PerfectSight: Using the Evaluation Setup PerfectSight: Camera Evaluation






PerfectSight: Initial Tests PerfectSight: Initial Tests

Please visit the Camera Culture group page to see more projects.

Home   |   News   |   Join Us   |   People   |   Projects   |   Publications   |   Talks   |   Courses

MIT Media Lab - All rights reserved.