Unusual apps: NETRA and CATRA

1

Health Power to the People

2

NETRA: Interactive Display for Estimating Refractive Errors

and Focal RangeVitor Pamplona Ankit Mohan Manuel M. Oliveira Ramesh Raskar

3

Millions have poor vision, but are not getting corrected…

Kenya

India

6.5 Billion people

4.5B have a cell phone

2B haverefractive errors

0.6B have URE

Photography in 1960s

Photo: Illana TamirPhoto: Roboppy

Today, cameras are everywhere

Photo: Derek K. Miller

Photo: Tyler

Photo: John Kannenberg

Corneal TopographerWavefront Aberrometer

Phoropter

Optometry Today

Snellen chartTrial set of lenses Lasers

NETRA: low-cost easy-to-use clip on for cell phones

Do the test at home, by yourself, in 2 minutes

8

Shack-Hartmann Wavefront Sensor

Expensive; Bulky, Requires trained professionals

Wavefront aberrometer

Shack-Hartmann Wavefront Sensor

Laser

Sensor Microlens Array

Planar Wavefront

Spot Diagram

10

Laser

Sensor

Displacement = Local Slope

of the Wavefront

Spot Diagram

Shack-Hartmann Wavefront Sensor

11

NETRA = Inverse of Shack-Hartmann

Spot Diagram on LCD

Cell Phone Display

Eye Piece

12

Spot Diagram on LCD

Inverse of Shack-HartmannUser interactively creates the Spot Diagram

Displace 25 points but 3 parameters

13

Spot Diagram on LCD

Inverse of Shack-HartmannUser interactively creates the Spot Diagram

Displace 25 points but 3 parameters

14

Human Eye

Human EyeCornea(~40D) Crystalline lens

(10~20D)

Retina

15

Human Eye

Human EyeCornea(~40D) Crystalline lens

(10~20D)

RetinaAccommodation

16

Infinity

Perfect Vision System

Human Eye

Infinity

Subject can focusat infinity

10cm

Normal Vision

Accommodation Range

17

Myopia (nearsightedness)

Infinity

Subject cannot focusat far distances

10cm

Normal Vision

Myopia

Wrong focal point

Accommodation Range

Infinity

Human Eye

18

Myopia Correction

Infinity

Subject can focusat infinity

10cm

Normal Vision

Myopia

Divergent Lens

Corrected Myopia

Accommodation Range

Infinity

Human Eye

19

Hyperopia (farsightedness)

Infinity 10cm

Normal Vision

Myopia

Hyperopia

Wrong focal point

Accommodation Range

Infinity

Human Eye

20

Hyperopia Correction

Infinity

Accommodation Range

10cm

Normal Vision

Myopia

Hyperopia

Convergent Lens

Corrected Hyperopia

Infinity

Human Eye

22

Refractive Errors and Shifted Range

Perfect vision

Myopia

Hyperopia

10cmInfinity

Need to measure

1m 33cm Distance

23

Refractive Errors and Shifted Range

Perfect vision

Myopia

Hyperopia

10cmInfinity

-10D0D

Diopter = 1/Distance

-3D-1D

1m 33cm

+1D+3D

Distance

Diopter

24

Eye

Relaxed Eye with Myopia

Blurred point

Red pointat infinity

perfect vision

myopia

hyperopia

~10cminfinity

Focusing Range

25

Eye

Relaxed Eye with Myopia

Distinctimage points

perfect vision

myopia

hyperopia

~10cminfinity

Focusing Range

Pinholes

Red pointat infinity

Scheiner’s Principle

26

Eye

Relaxed Eye with Myopia

Virtual red pointat infinity

perfect vision

myopia

hyperopia

~10cminfinity

Focusing Range

A

B

Distinctimage points

Display

27

Relaxed Eye with Myopia

A

B

perfect vision

myopia

hyperopia

~10cminfinity

Focusing Range

Virtual red pointat finite distance

Move spots towards each

other

Display

Eye

Distinctimage points

28

B

A

Points overlap

Relaxed Eye with Myopia

Display

perfect vision

myopia

hyperopia

~10cminfinity

Focusing Range

Eye

Virtual red pointat finite distance

Move spots towards each

other

29

B

A

Points overlap

Relaxed Eye with Myopia

Display

perfect vision

myopia

hyperopia

~10cminfinity

Focusing Range

Eye

Virtual red pointat finite distance

Move spots towards each

other

d

30

Points overlap

Relaxed Eye with Myopia

perfect vision

myopia

hyperopia

~10cminfinity

Focusing Range

Eye

d

d

1

Point at infinity

32

Distinctimage points

Relaxed Eye with Hyperopia

perfect vision

myopia

hyperopia

~10cminfinity

Focusing Range

A

B

Display

Virtual red pointat infinity

Eye

33

Points overlap

Relaxed Eye with Hyperopia

Virtual point“beyond” infinity

Move spots away from each other

perfect vision

myopia

hyperopia

~10cminfinity

Focusing Range

Display

A

B

Display

34

Points overlap

Relaxed Eye with Hyperopia

Virtual point“beyond” infinity

Move spots away from each other

perfect vision

myopia

hyperopia

~10cminfinity

Focusing Range

35

NETRA: Using pinholes

Patterns on an LCD

Pinhole array

36

f

Patterns on an LCD

Microlens array

NETRA: Using Lens to Increase Light

tda

fc

2

t

a

Pixel Pitch

Virtual Depth

37

Interactive Method

Farthest Focal Point(myopia, hyperopia)

38

Interactive Method

Farthest Focal Point(myopia, hyperopia)

39

Interactive Method

Farthest Focal Point(myopia, hyperopia)

40

Overview• Inverse of Shack Hartmann Wavefront Sensor• Hi-res displays + interaction

• Measuring Spherical Error• No moving parts, lasers• Blur -> Alignment problem• ~ Lightfield Display for Single Eye

• Astigmatism• Novel Patterns• Focal Range• User Study

41

Astigmatism: angle-dependent refractive error

http://www.elizabethpope.co.uk/eyeinfo/astigmatism.html

42

Astigmatism: angle-dependent refractive error

http://www.elizabethpope.co.uk/eyeinfo/astigmatism.html

43

Astigmatism: angle-dependent refractive error

http://www.elizabethpope.co.uk/eyeinfo/astigmatism.html

44

Refractive Power as a Function of Angle

Cylinder SphereAxis Cyl.Unknowns:

SCP )(sin( 2

45

Astigmatism

Cross or points may never meet with a 1d search !

46

Astigmatism

Lines reduce the problem to a 1d search

47

Interactive Method

Farthest Focal Point(myopia, hyperopia, astigmatism)

48

Interactive Method

Farthest Focal Point(myopia, hyperopia, astigmatism)

49

Interactive Method

Farthest Focal Point(myopia, hyperopia, astigmatism)

50

Interactive Method

Farthest Focal Point(myopia, hyperopia, astigmatism)

51

Interactive Method

Farthest Focal Point(myopia, hyperopia, astigmatism)

52

Measuring the Accommodation Range

Perfect vision

~10cmInfinity

Myopia

Hyperopia

Step 1: Far limit Step 2: Near limit

53

Measuring the Accommodation Range

Perfect vision

~10cmInfinity

Myopia

Hyperopia

Step 1: Far limit Step 2: Near limit

54

Measuring the Accommodation Range

Perfect vision

~10cmInfinity

Myopia

Hyperopia

Step 1: Far limit Step 2: Near limit

55

Points overlap

A

B

Display

Virtual Point at the far limit

Relaxed Eye

56

Points overlap

A

B

Display

Accommodated Eye

Virtual pointgetting closer

Move points towards each other

Subject Accommodates to fix the “blur”

57

Points overlap

A

B

Display

Accommodated Eye

Virtual pointgetting closer

Move points towards each other

Subject Accommodates to fix the “blur”

58

Points overlap

A

B

Display

Accommodated Eye

Virtual pointgetting closer

Move points towards each other

Subject cannot accommodate more than the previous point

59

Patterns for Alignment Task

Displayed

Subject view

Displayed

Subject view

Visual Cryptography

A B A B A B A B A B

A B A B A B A B A B

[NaorShamir94]

60

Patterns for Alignment Task

Displayed

Subject view

Displayed

Subject view

Visual Cryptography

A B A B A B A B

A B A B A B A B A B

A B

[NaorShamir94]

61

Summary of Interaction

Farthest Point(myopia, hyperopia, astigmatism)

Nearest Point(presbyopia)

Accommodation Range

Phoropter

Snellen chart

Trial lenses

Reading Charts• Accuracy

– Sharpness Estimation is subjective– Brightness affects results– Pupil size variation and DoF

• Cost– Trial Lens Set > $150– Bulky

Retino scope w/

Lenses

Auto-refracto-

meter

Chart with

Lenses

In-Focus: Focometer Optiopia

Solo-health: EyeSite

NETRA

Technology Shining Light plus lenses

Fundus Camera

Moving lenses + target

Moving lenses + target

Reading chart on monitor

Cellphone + eyepiece

Cost to buy $2,000* ~$10,000 ~$100 ~$495 ~$200 -- $30

Cost per test ~$36 ~$36 ~$5 -- -- -- ~$1

Data capture No Comp. No No No Comp. Phone

Mobility <500g >10Kg 2kg 1kg <5kg >10Kg <100g

Speed Fast Fast Medium Medium -- Fast Fast

Scalability No No No Yes Probably No Yes

Accuracy 0.15 0.15 0.5 0.75 -- -- <0.5

Self evaluation No No Yes Yes Yes Yes Yes

Electricity Req No Yes No No -- Yes No

Astigmatism Yes Yes Yes/No No -- Yes Yes

Network No Yes No No No Yes Yes

Training High High High Medium Medium Low Low

* Phoropter-based: $5,000.00

Needs expert, Moving parts, Shining lasers

64

Limitations

• Children• Ability to align lines

• Single Eye test– Other eye for convergence-forced accommodation

• Resolution is a function of the display DPI– Samsung Behold II – 160 DPI – 0.35D– Google Nexus One – 250 DPI – 0.2D– Apple iPhone 4G – 326 DPI – 0.14D

BBC CNN NBC

MIT News

Gizmodo

O Estado de SP - Brazil

NY Times Time - Wellness

Media Coverage

Confidential 66

NETRA Prototypes Worldwide29 partners in 14 countries.

Clinical Testing Partners

NETRA Team at LVPEI, India

OneSight, Kenya

Tufts U. (Boston) NETRA vs Manifest Refraction

-5.00 -4.00 -3.00 -2.00 -1.00 0.00 1.00 2.00 3.00 4.00 5.00

-5.00

-4.00

-3.00

-2.00

-1.00

0.00

1.00

2.00

3.00

4.00

5.00

Retinoscopy (D)

Net

ra (D

)

13 adults

-7.00 -5.00 -3.00 -1.00 1.00 3.00 5.00 7.00

-7.00

-5.00

-3.00

-1.00

1.00

3.00

5.00

7.00

Retinoscopy (D)

Net

ra (D

)

29 adults

LVPEI (India)NETRA vs Retinoscopy

Validation: Side by Side Trials

Preventable Blindness

WHO 2004

Unique, quantitative lens mapping for size and density of eye opacities

CATRA: Cataract Screening Tool

Testing the Presence of Cataracts

73

Blin

king

patt

erns

on

Scre

en

Cell Phone Display

Pinhole

Lens

Point Spread Function Mapping

74

Blin

king

patt

erns

on

Scre

en

Cell Phone Display

Pinhole

Lens

Point Spread Function Mapping

75

Blin

king

patt

erns

on

Scre

en

Cell Phone Display

Pinhole

Lens

Traditional User Driven

Mass-use Devices Scientific Instruments

For the Future• Ophtalmology for Masses

– >.5 billion URE. > 2.5 billion RE. -> Devices for all

• Quality of phones (resolution) will increase exponentially

• New features (recently cataracts, next Retinal Netra)

• Smart phones will take over the market in developing world countries like India in next 5 years.

• Hardware store

EHREHR

DeliveryDeliveryDiagnosticDiagnostic EyecareProvidersEyecare

Providers

New Ecosystem

Asynchronous

Medical IT Systems

NETRAhttp://eyenetra.com