Electrical & Computer Engineering Dept. University of Patras, Patras, Greece Evangelos Skodras Nikolaos Fakotakis

Electrical & Computer Engineering Dept.

University of Patras, Patras, Greece

Evangelos Skodras

Nikolaos Fakotakis

Eyes represent the most distinctive landmarks of the human face

The position and movements of eyes are a significant source of information about the cognitive and affective state of human beings: eyes hold a key role in expressing interest, intention and attention

Importance of eye localization

University of University of PatrasPatras 1/131/13

3

A wide range of applications…

University of University of PatrasPatras

Non-glasses type 3D technologies Monitoring of drivers’ attention and vigilance

Visual attention analysis Interactive gaze-based interfaces for disabled people

2/132/13

4

Why hasn’t it been solved yet?


Completely unobtrusive eye localization with remotely located passive sensors (cameras)

Low resolution, inexpensive imaging devices (e.g. webcams, pinhole cameras)

Accurate and robust detection for human-machine interaction applications even in outdoor environments

Requirements posed:

Active sensors or dedicated

hardware

3/133/13

5

Great variability in shape, color , eye state, head pose and ethnicity

Imaging conditions such as lighting and camera characteristics have a strong influence on how the eyes appear in the image

Low resolution or compressed images derived from inexpensive devices

Occlusions caused by hair, glasses, shadows and reflections

Challenges of remote passive sensors


What has been used1. Appearance based Methods

2. Feature based Methods

5/135/13

Incorporate eye knowledge implicitly They generally require a large amount of training data and powerful

non-linear algorithms in order to learn the high variability of eyes High accuracy in detecting the eye area Fail to provide an accurate detection of the eye center

Explicit use of the a priori eye knowledge in order to derive features such as shape, geometry, color and symmetry

• Geometrical information of edges• Parametric models (e.g. deformable templates)• Circular shape modeling (Hough, Starburst, Integrodifferential)• Symmetry operators• …

Proposed Method


o Based upon a synergy of color and radial symmetry

• Color distribution of the eye area is significantly different to its surroundings

• Iris and pupil are characterized by radially symmetric brightness patterns

Simple – no machine vision required

Efficient – 0.12 sec for a 240 x 320 color image (Matlab implementation)

Robust – occlusions, eye state, head pose, lighting changes

Accurate – for human computer interaction applications

Low-cost – using a standard webcam or pinhole camera

6/136/13


Eye Map Construction

Cb2 Cr2

Y Dilated

EyeMapC

Y

EyeMapC Eroded

EyeMapI

/

{ }2 21/

3 b r b rEyemapC C C C C= + + 𝐸𝑦𝑒𝑀𝑎𝑝𝐼= 𝑌⊕ 𝐵1𝐸𝑦𝑒𝑚𝑎𝑝𝐶 ⊝ 𝐵2

7/137/13

Gradient - based interest operator which detects points of high radial symmetry

Fast Radial Symmetry Transfrom


Determines the contribution each pixel makes to the symmetry of pixels around it

The contribution of every orientation is computed in a single pass over the image

8/138/13

Proposed System Overview


+

EyeMapI

Y Sluminance

SEyeMapI

9/139/13


Quantitative ResultsMethod

Accuracydn ≤ 0.05 dn ≤ 0.1 dn ≤ 0.25

Proposed Method

87.94 % 94.26 % 97.45 %

Yang et al. 68.72 % 81.56 % 96.45 %

Valenti et al.

56.54 % 84.69 % 97.92 %

MethodAccuracy

dn ≤ 0.05 dn ≤ 0.1 dn ≤ 0.25

Proposed Method

99.43 % 99.54 % 99.66 %

Yang et al. 98.16 % 98.39 % 99.08 %

Valenti et al.

89.81 % 98.59 % 99.88 %

GT

AV

Calt

ech

10/1310/13

Qualitative Results


Low computational complexity which allows real time performance with a proper C or hardware implementation

The use of color information (rarely investigated in literature for this purpose) contributes significantly for precise localization

High accuracy rates, outperforming existing methods, especially in low-resolution images

Future work Correlate the level of attention of a driver (gaze estimation) with

the danger of an impeding collision (vehicle detection and braking recognition).

Conclusions

12/1312/13


Thank you for your Thank you for your attention!attention!

[email protected]

Documents

Electrical & Computer Engineering Dept. University of Patras, Patras, Greece Evangelos Skodras Nikolaos Fakotakis