Proxy-Based Image Verification

Thomas Kotwaltkotwal@mit.edu

MIT Laboratory for Computer Science

January 9, 2002

System Layout

• He wishes to communicate with his proxy.

• Bob has access to an untrustworthy computer.

• He can trust what he sees on the screen, because his camera equipped K21 is monitoring the screen content.

Proxy-Based Approach

• Add an IR link between the K21 and untrustworthy computer

• This allows the K21 to send data (image) to the proxy

• Proxy can do large amounts of computation

• Proxy knows what the screen should display

Security Protocol

1. U.C. displays an image and the K21 takes a picture

2. K21 sends picture + nonce + MAC to proxy via U.C. using IR link

3. Proxy verifies the picture and sends nonce + MAC to K21 via U.C. using IR link

Performing a Secure Request

4. U.C. displays the image and the K21 takes a picture of it

5. K21 sends picture + nonce + MAC to the proxy using the IR link

6. Proxy verifies the picture and sends nonce + MAC to the K21 using the IR link

1. User enters a request

2. U.C. sends the request to the proxy

3. Proxy sends an image summarizing the request to the U.C.

Image Verification ProblemIs this… …the same as this?

Obstacles:• Linear and non-linear distortions• Decreased resolution• Noise

Steps to solution• Undo distortions• Compare content

Step 1: Undo Image Distortiona) Undo lens distortion

– Model as radially symmetric distortion– Non-linear process

b) Undo linear distortions– Corrects for affine (scaling, rotation, translation) and

projection distortion– Requires four known points in distorted image

c) Undo other non-linear distortions– Corrects for curvature of screen, etc.– Least-squares with orthogonal polynomials– May not be necessary

a) Radially Symmetric Distortion(x,y)

r(u,v)

2

2

1

1

kr

yv

kr

xu

+=

+= 22 yxr +=

k : constant

b) Linear Distortion

(x,y)(u,v)

[ ] [ ]

=

333231

232221

131211

11

aaa

aaa

aaa

vuyx

Step 2: Compare Content

• Assume content is text only

• Perform OCR on processed image– Advantage: proxy knows what the text should

say– To save computation time compare each

character with what it should be, not every possible character

– Constrain font to facilitate OCR routine

Next Step: Compare Images

• Generalize content to be anything, not just text

• Find areas of large difference between original and processed image

• Requires very accurate undistortion

Summary of Approach1. K21 takes a picture and uses its IR link to send the

picture to the proxy

2. Proxy corrects for image distortion and performs OCR

SELL: IBM300 SHARESTOTAL:$300

3. Proxy sends a confirmation back to the K21

Why Proxy-based Approach?• Advantages

• Proxy knows what the screen should display• No image processing on the K21• No calibration necessary• Camera does not have to be immobile during

session• Translates easily to color

• Disadvantages• Requires IR link between K21 and

untrustworthy computer• Possibly long verification time• Image verification allows for possibility of

error

Summary / Status

Q: How do you use a computer that you don’t trust?

Our Answer: Use visual authentication

• Pixel-Matching solution is running

• Proxy-based solution is under development

Questions?