Progress Presentation

IRALARBreanna Heidenburg -- Michael Lenisa -- Daniel Wentzel

Advisor: Dr. Malinowski

Project Overview Application Layout & Progress

◦Image Processing◦Calibration◦Clicking◦Communication◦User Interface

Overall Progress

What is our project?

Track a user’s eye◦ Small form factor camera

Use input to control cursor◦ Where you look is where the cursor is◦ Need to calibrate for every user

Create user interface

How will we achieve our goal?

Hardware◦ Camera

QuickCam Pro for Notebooks

◦ Polarizer Tiffen 25mm polarizing filter Removes glare from eye reflections

◦ Lighting Diffuse LEDs Not sure if they are necessary, will find out when we

have the display

New Image processing methods:

◦ Blob Recognition

◦ Classic Image Processing Techniques Contrast stretching Thresholding Adaptive Windowing

Sample Recognitions:

Difficult Low Light False Positive

Why do we need to calibrate?◦ Map the user’s eye to a point on the screen◦ Everyone’s eyes are different◦ Account for various positions of the camera

What does our data look like?◦ Somewhat unreliable◦ Problem:

Movements while taking data

X-Axis values

Y-Axis values

What does the data tell us?◦ A 2D mapping is not good enough◦ Horizontal eye movement is closer to linear than

vertical eye movement◦ Vertical eye movement depends on both

coordinates of data

How do we solve the problem?◦ Multiple Variable Linear Regression – Least

Squares Y = B0 + B1x1 + … + Bkxk

◦ Uses matrix algebra to obtain a coefficient matrixB[] = (X’X)-1X’Y

Results◦ pixelX = 224.9 + 4.8*eyeX – 2.9*eyeY◦ pixelY = 1612.5 + 2.0*eyeX – 5.6*eyeY

◦ Error = very high (need more than 4 random data points)

B[] B[] == ((XX’’XX))-1-1XX’’YY

How do we click?◦ Monitor eye movements◦ Identify pauses

◦ When eye position is within a small percentage for a certain amount of time

◦ Set to 5 frames (~200 mS)◦ Generally, it takes 230 mS for a hand to click a

mouse.

Why is it important?◦ Allows multiple processes to communicate◦ Allows relay of time sensitive information

UDP vs. TCP◦ UDP -> ‘Connectionless’◦ TCP -> ‘Connection oriented’

Multi-threading◦ Necessary for running multiple pieces of code in a

single process

Multi-threading◦ Issues with public variable usage◦ Solutions we are using:

Critical Section Raises thread priority (does not allow for interrupts)

Mutual Exclusion Creates a variable lock Using variable first requires ‘unlocking’ it

Achieved via a combination of Hardware and Software

Software:WPF & C#◦ XAML

(eXtensible Application Markup Language)◦ Very similar to XML◦ Uses ‘code-behind’ in a similar style to javascript

‘code-behind’ is in C#

Framework is in place for all pages

Navigation between pages is complete

Core operating portions are complete

Hardware◦ LitEye LE500 has been ordered

(will arrive early-mid March)◦ SVGA resolution (800x600)◦ Transparent

How far along are we?

HMD has arrived (today)

◦ Have not had much time to work with it

◦ We aren’t 100% sure how we will physically mount the camera to the display

◦ Aren’t sure if extra lighting will be necessary

Image Processing

◦ Still working on some recognition improvements Calibration

◦ Minor adjustments left to be made Communication

◦ Minor adjustments left to be made User Interface

◦ Only essential parts are included at the momentStill ironing out details on what will be included in final version

ID Task Name Start DurationDec 2007 Jan 2008 Feb 2008 Mar 2008 Apr 2008 May 2008

12/2 12/9 12/16 12/23 12/30 1/6 1/13 1/20 1/27 2/3 2/10 2/17 2/24 3/2 3/9 3/16 3/23 3/30 4/6 4/13 4/20 4/27 5/4 5/11

1 1.4w12/11/200712/3/2007Proof of Concept

2 10.8w2/14/200812/3/2007Develop Image Processing Applicaiton

3 1.4w12/11/200712/3/2007Develop System Test UI

6 2.2w2/7/20081/24/2008HMD testing

4.2w3/13/20082/14/2008Application level UI Development

Finish

4 10.6w2/21/200812/11/2007Develop interactive UI

3.2w3/6/20082/14/2008Basic User Testing (product research)

5

11

10

9

8

7

3.2w3/27/20083/6/2008Application User Testing

2.2w4/10/20083/27/2008Summarize findings

4.6w5/5/20084/3/2008Prepare for RESNA submission

2.4w4/25/20084/10/2008Presentations