NETRA on SIGGRAPH 2010

MIT media lab camera culture EyeNetra.com

1

NETRA: Interactive Display for Estimating Refractive

Errors and Focal RangeVitor Pamplona Ankit Mohan Manuel M. Oliveira Ramesh Raskar


2

NETRA: Near Eye Tool for Refractive Assessment

Vitor Pamplona Ankit Mohan Manuel M. Oliveira Ramesh Raskar


Challenge

6.5 Billion people

4.5B have a cell phone

2B haverefractive errors

0.6B have URE

NETRA at LVP Eye Institute


4

Computational Photography

Optometry/Opthalmalogy

Measure .. Overcome Limitations .. Extend Abilities


Phoropter

Snellen chart

Trial lenses

Reading Charts• Accuracy

– Sharpness Estimation is subjective– Brightness affects results– Pupil size variation and DoF

• Cost– Trial Lens Set > $150– Bulky

Retino scope w/

Lenses

Auto-refracto-

meter

Chart with

Lenses

In-Focus: Focometer Optiopia

Solo-health: EyeSite

NETRA

Technology Shining Light plus lenses

Fundus Camera

Moving lenses + target

Moving lenses + target

Reading chart on monitor

Cellphone + eyepiece

Cost to buy $2,000* ~$10,000 ~$100 ~$495 ~$200 -- $30

Cost per test ~$36 ~$36 ~$5 -- -- -- ~$1

Data capture No Comp. No No No Comp. Phone

Mobility <500g >10Kg 2kg 1kg <5kg >10Kg <100g

Speed Fast Fast Medium Medium -- Fast Fast

Scalability No No No Yes Probably No Yes

Accuracy 0.15 0.15 0.5 0.75 -- -- <0.5

Self evaluation No No Yes Yes Yes Yes Yes

Electricity Req No Yes No No -- Yes No

Astigmatism Yes Yes Yes/No No -- Yes Yes

Network No Yes No No No Yes Yes

Training High High High Medium Medium Low Low

* Phoropter-based: $5,000.00

Needs expert, Moving parts, Shining lasers


7

Shack-Hartmann Wavefront Sensor

Expensive; Bulky, Requires trained professionals

Wavefront aberrometer


8

Shack-Hartmann ~ Lightfields

Levoy et al 2009 Zhang and Levoy 2009: Observable Light Field

Oh, Raskar, Barbastathis 2009: Augmented Light Field


Laser

Sensor Microlens Array

Planar Wavefront

Shack & Platt 1971Liang et al 1994

David Williams et al, Rochester

Spot Diagram


9

Laser

Sensor

Displacement = Local Slope

of the Wavefront

Spot Diagram



10

NETRA = Inverse of Shack-Hartmann

Spot Diagram on LCD

Cell Phone Display

Eye Piece


11

Spot Diagram on LCD

Inverse of Shack-HartmannUser interactively creates the Spot Diagram

Displace 25 points but 3 parameters


12

Infinity

Optometry

Human Eye

Infinity

Subject can focusat infinity

10cm

Normal Vision

Accommodation Range


13

Myopia (nearsightedness)

Infinity

Subject cannot focusat far distances

10cm

Normal Vision

Myopia

Accommodation Range

Infinity

Shifted Accommodation Range


14

Myopia Correction

Infinity

Subject can focusat infinity

10cm

Normal Vision

Myopia

Accommodation Range

Infinity

Human Eye

Shifted Accommodation Range


16

Hyperopia (farsightedness)

Infinity 10cm

Normal Vision

Myopia

Hyperopia

Wrong focal point

Accommodation Range

Infinity

Human Eye


17Infinity

Accommodation Range

10cm

Normal Vision

Myopia

Hyperopia

‘Beyond’Infinity

Human Eye

Hyperopia (farsightedness)


19

Refractive Errors and Shifted Range

Perfect vision

Myopia

Hyperopia

10cmInfinity

Need to measure

1m 33cm Distance


20

Refractive Errors and Shifted Range

Perfect vision

Myopia

Hyperopia

10cmInfinity

10D0D

Diopter = 1/Distance

3D1D

1m 33cm

-1D-3D

Distance

Diopter


21

Eye

Relaxed Eye with Myopia

Blurred point

Red pointat infinity

perfect vision

myopia

hyperopia

~10cminfinity

Focusing Range


22

Eye


Distinctimage points

perfect vision

myopia

hyperopia

~10cminfinity

Focusing Range

Pinholes

Red pointat infinity

Scheiner’s Principle


23

Eye


Virtual red pointat infinity

perfect vision

myopia

hyperopia

~10cminfinity

Focusing Range

A

B


Display


24


A

B

perfect vision

myopia

hyperopia

~10cminfinity

Focusing Range

Virtual red pointat finite distance

Move spots towards each

other

Display

Eye



25

B

A

Points overlap


Display

perfect vision

myopia

hyperopia

~10cminfinity

Focusing Range

Eye



other


26

B

A

Points overlap


Display

perfect vision

myopia

hyperopia

~10cminfinity

Focusing Range

Eye



other

d


27

Points overlap


perfect vision

myopia

hyperopia

~10cminfinity

Focusing Range

Eye

d

d

1

Point at infinity


29


Relaxed Eye with Hyperopia

perfect vision

myopia

hyperopia

~10cminfinity

Focusing Range

A

B

Display

Virtual red pointat infinity

Eye


30

Points overlap


Virtual point“beyond” infinity

Move spots away from each other

perfect vision

myopia

hyperopia

~10cminfinity

Focusing Range

Display

A

B

Display


31

Points overlap


Virtual point“beyond” infinity

Move spots away from each other

perfect vision

myopia

hyperopia

~10cminfinity

Focusing Range


32

NETRA: Using pinholes

Patterns on an LCD

Pinhole array


33

f

Patterns on an LCD

Microlens array

NETRA: Using Lens to Increase Light

tda

fc

2

t

a

Pixel Pitch

Virtual Depth


34

Interactive Method

Farthest Focal Point(myopia, hyperopia)


35

Interactive Method



36

Interactive Method



37

Overview• Inverse of Shack Hartmann Wavefront Sensor

• Hi-res displays + interaction• Measuring Spherical Error

• No moving parts, lasers• Blur -> Alignment problem• ~ Lightfield Display for Single Eye

• Astigmatism• Novel Patterns• Focal Range• User Study

MIT media lab camera culture UFRGS EyeNetra.com

38

Astigmatism: angle-dependent refractive error

http://www.elizabethpope.co.uk/eyeinfo/astigmatism.html


39




40




41

Refractive Power as a Function of Angle

Cylinder SphereAxis Cyl.Unknowns:

SCP )(sin( 2


42

Astigmatism

Cross or points may never meet with a 1d search !


43

Astigmatism

Lines reduce the problem to a 1d search


44

Interactive Method

Farthest Focal Point(myopia, hyperopia, astigmatism)


45

Interactive Method



46

Interactive Method



47

Interactive Method



48

Interactive Method



49

Evaluation Prototype

LCD Display

Minification

Camera simulatesthe perfect eye

Trial lenses simulate lens

aberration


50

Subjective Validation: User Study


51

Measuring the Accommodation Range

Perfect vision

~10cmInfinity

Myopia

Hyperopia

Step 1: Far limit Step 2: Near limit


52


Perfect vision

~10cmInfinity

Myopia

Hyperopia



53


Perfect vision

~10cmInfinity

Myopia

Hyperopia



54

Points overlap

A

B

Display

Virtual Point at the far limit

Relaxed Eye


55

Points overlap

A

B

Display

Accommodated Eye

Virtual pointgetting closer

Move points towards each other

Subject Accommodates to fix the “blur”


56

Points overlap

A

B

Display

Accommodated Eye



Subject Accommodates to fix the “blur”


57

Points overlap

A

B

Display

Accommodated Eye



Subject cannot accommodate more than the previous point


58

Patterns for Alignment Task

Displayed

Subject view

Displayed

Subject view

Visual Cryptography

A B A B A B A B A B

A B A B A B A B A B

[NaorShamir94]


59

Patterns for Alignment Task

Displayed

Subject view

Displayed

Subject view

Visual Cryptography

A B A B A B A B

A B A B A B A B A B

A B

[NaorShamir94]


gh

ghh

Subject View as Convolution

Subject’ s View

Subject’ s View

Display

g h

S

S


Viewmaster prototype

+3D to -5D with accommodation61

Subject View

h - Jittered Pinholes

G(-5D) G(0D) G(+5D)

Scaled Patterns


62

Summary of Interaction

Farthest Point(myopia, hyperopia, astigmatism)

Nearest Point(presbyopia)

Accommodation Range


63

Limitations

• Children• Ability to align lines

• Single Eye test– Other eye for convergence-forced accommodation

• Resolution is a function of the display DPI– Samsung Behold II – 160 DPI – 0.35D– Google Nexus One – 250 DPI – 0.2D– Apple iPhone 4G – 326 DPI – 0.14D


64

Future Work• Clinical research for an optometry device

– Side-by-side validation tests– Field trials – Tests for cataract, lazy eye, etc

• Opportunity to monitor one’s eyesight– Diabetes/Glucose non-invasive meter– Unstable lens prescriptions

• Distribution in Developing Countries– Software app for free– Eyepiece blueprint to NGOs for <$1– More at EyeNetra.com


65

Future Work• Multi-focus display without moving parts

– [Akeley04][Rolland00][Hua09] [Barsky04]

• Personalized devices (clear without glasses) – Alarm Clock – Cell phones– eReaders

Inju Fernando Meyer


66

Acknowledgements

• Volunteers• Dr. Joseph Ciolino (MGH Mass Eye and Ear Inst.)• Dr. Fuentanta Vera Diaz (Schepens Eye Research Inst.)• Dr. James Kobler (MGH Mass Eye and Ear Inst.)• Dr. Shrikant Bhardwaj, (LV Prasad Eye Institute, India)

Sponsors• CNPq-Brazil• Alfred P. Sloan Research Fellowship• Google • Samsung


67

NETRA: Display for Eye Refraction Tests

• Inverse of Shack-Hartmann wavefront aberrometer– High-resolution displays and user interaction

• Focal Parameters– Myopia, Hyperopia, Astigmatism– Focal range

• Thermometer for the eye– Measurement not prescription– Promote Self Awareness

• Impact in Developing Countries– 600 Million without corrective glasses– $1 cost, easy to deploy


68

NETRA: Interactive Display for Estimating Refractive Errors and Focal Range

Vitor Pamplona Ankit Mohan Manuel M. Oliveira Ramesh Raskar