Criminal Court Case Facial Comparison and Facial Recognition Limits

2011© AFIS and Biometrics Consulting Inc

Ben Bavarian,

Mehrad Tavakoli

Principal Consultants

AFIS and Biometrics Consulting Inc.

96th IAI Educational Conference

Milwaukee, Wisconsin

August 10, 2011

Workshop Agenda

2

The Case Background

The data

The defense expert testimony

The FBI expert counter arguments

Open discussion

The Case

3

THE SUPERIOR COURT OF THE STATE OF CALIFORNIA

IN AND FOR THE CITY AND COUNTY OF SAN FRANCISCO

HONORABLE JEROME T. BENSON, JUDGE

DEPARTMENT NO. 21, SCN 210246 COURT NO. 2429070

PEOPLE OF THE STATE OF CALIFORNIA,(PLAINTIFF, )

VS.

CHARLES HEARD, (DEFENDANT. )

APPEARANCES OF COUNSEL:

J. MICHAEL SWART, ASSISTANT DISTRICT ATTORNEY

FOR THE DEFENDANT: ERIC M. SAFIRE, ATTORNEY AT LAW

LAW OFFICES OF ERIC M. SAFIRE

June 2010

The Assignment

4

Task Description

The objective is to carry out an independent scientific examination of

landmark features that are used in technical matching techniques for

face recognition biometric to compare the recently captured photos of

Mr. Charles Heard with the unknown person (the one with the black

hoodie), in the video from the security camera on Montgomery Street

showing two figures, Afro-American males, appear at 56:18:04, or

perhaps a few seconds sooner.

Conclusions

Based on the analysis detailed out in this report the comparison is

inconclusive. In other words it is not possible to assess that the figure in

the said video is that of Mr. Charles Head based on comparing the

biometric landmark features.

Methods for the one-to-one facial image comparison Facial Identification Scientific Working Group (FISWG)

5

Holistic comparison – the process of comparing faces by looking at the

face as a whole and not component parts;

Morphological analysis – applying the classification and description of the

form and structure of facial features;

Photo-anthropometric – the measurement of dimensions and angles of the

anthropologic landmarks in order to quantify characteristics and proportions

of a person in an image;

Superimposition – the process of creating a scaled overlay of one image

and aligning it with a second image.

The biometrics identification community has added the 5th method for the

comparison:

Biometric Facial Recognition – applies mathematical algorithms which

use principal abstract components of the digital image and calculates a

degree of likelihood or a probability match score between two images.

Component-based Face Matching; Facial Image Component

decomposition and Analysis (PCA, ICA).

Feature-based Face Recognition; 2D landmarks can be defined

and tracked on face images

A note on Features used in Face recognition

6

Biometric Data: 2D Facial Images : Challenges:

Sensitive to changes in Light & Make-Up Sensitive to Angle of Capture Inter-class and intra-class variability

7

www.marykateandashley.com news.bbc.co.uk/hi/english/in_depth/americ

as/2000/us_elections

Identical twins Almost identical father and son

Pose and make-up variations

Professor Massimo Testarelli, Biosecure Industry Committee Workshop, Feb. 2006, Computer Vision Laboratory University of Sassari – Ital

MONTGONERY – 56:16:74

The Data

8

MONTGOMERY – 56:17:83

The Data

9

MONTGONERY – 56:18:04

The Data

10

MONTGONERY – 56:18:26

The Data

11

MONTGONERY – 56:18:48

The Data

12

MONTGONERY – 56:18:70

The Data

13

The Data

14

15

The Data ANSI/NIST-ITL Standards Data Format for Interchange of Fingerprint , Facial & other Biometric Information Section 15.1.27 Field 10.029: Facial feature points. MPEG4 Standard; Annex C of ISO/IEC 14496-2

Methodology used for the Analysis

16

The following four steps will be used in our

scientific analysis study:

Step1: Data Gathering and Preparation

Step 2: Feature point measurements and tabulations

Step 3: Face feature calculations from facial feature points and

data normalization

Step 4: Data Analysis

The science applied Probability of complementary event

17

Note:

Our method is to answer the question of “non-matching”, “dissimilarity” between

two samples or solving the problem of “complementary event” in probability

theory.

The complementary probability, determining “non-match”, problem that we have

solved here is order of magnitude easier and straight forward to do than the

facial recognition finding the “match” problem.

A simple explanation.

Suppose one throws an ordinary six-sided die. The probability of number “2”, or

any number, coming out is (1/6)th. The probability of any other number coming

out is (5/6)th. That is five times more than for a single event. That is called the

complementary event or complementary probability of an event. In layman

terms it is much easier to get the complementary event that the event itself. Or

you will be better off betting on the complementary event with higher chances of

winning.

Biometric Analysis Approach

Step 1

Data Gathering and Preparation

Step 2

Feature point measurements and

tabulations Step 3

Face feature calculations from facial

feature points and data normalization

Step 4

Data Analysis ≠ ? 18

Biometric Analysis Approach Reference Photos Security Video Captures

MONTGOMERY – 56:16:74

MONTGOMERY – 56:17:83

MONTGOMERY – 56:18:04

Biometric Analysis Approach

20

Biometric Analysis Approach

Reference Photos

Security Video Capture

21

Biometric Analysis Approach

L4

Measurement

L0

Measurement

L3

Measurement

L1

Measurement

L2

Measurement

22

L3

L0

Biometric Analysis Approach - Normalization

Video Footage

1

2 .5

L3

L0

L3

L0

Reference Photo

10

20 .5

L3

L0

23

L3

L0

Video Footage

13

37 .35

L3

L0

Reference Photo

282

914 .31

L3

L0

L3

L0

Biometric Analysis Approach - Normalization

24

Video

Face is

Wider

Biometric Analysis Approach

L1-

forehead-to-nose length

L2-

nose-to-chin

length

L3-right eye-to-

left-eye length

L4-

Mouth

length

target jail photos Average Length 0.54 0.48 0.32 0.27

Standard Deviation 0.03 0.03 0.02 0.01

the video scene

pictures

Average Length 0.52 0.48 0.36 0.30

Standard Deviation 0.03 0.03 0.05 0.02

Video

Head is

Shorter

Video

Face is

Wider

Table 12- The final set of feature lengths and the

statistical variations for both data sets

25

26

The FBI expert counter arguments

The resolution of the video images are not sufficient for facial

recognition algorithms

The data elements are not marked correctly on the sample data

The lighting and subject position in the video lend itself to

photographic issues such as perspective distortions, angle of

pose issues, etc.

Statistical error analysis are missing in the expert witness work.

The defendant expert is not an “expert” in photography and

Photo-anthropometric area.

The data points used are not reliable points.

Open discussion

What is the scientific basis of facial recognition?

The features used;

The statistical characterization and uniqueness;

Reliability; permanence; and uniqueness characterization of the features;

Applications of facial recognition in law enforcement;

Evolutionary process for adoptation

27

Dr. Bavarian is one of the pioneers and industry leading authority in the field of Biometric

Identification with over 26 years of R&D and management experience in industry and academics.

He founded the AFIS and Biometrics Consulting Inc. in 2007, providing subject matter expert

consulting and strategic management services in Biometrics Identification Industry. The company

has successfully completed more than two dozen contracts in the last three years.

Prior to ABC Inc. Dr. Bavarian was the Vice President of Motorola Biometrics, where he led the

business turn around and four fold increase in Sales by directing the development of the industry

leading Automated Biometric Identification System products with over 100 large scale

deployments.

Before moving to the industry in 1992, Dr. Bavarian was a professor in the Department of

Electrical and Computer Engineering at the University of California, Irvine, where he conducted

original research in image processing, computer vision, intelligent systems and published over

120 technical papers and received several awards for outstanding research and distinguished

teaching.

Dr. Behnam (Ben) Bavarian received his Ph.D. in Electrical and Computer Engineering from The

Ohio State University, Columbus Ohio in 1984.

Author Biography - Dr. Ben Bavarian

28

Mr. Mehrad Tavakoli is PhD graduate student at University of California, Irvine California (UCI),

where he is pursuing his research in advancing Biometrics Identification science and its

application for Law Enforcement, Security and Commercial applications.

Mr. Tavakoli is the lead research scientist for the UCI project on Biometrics Identification on the

Move System™ (BIMS) where he has developed new techniques for using high resolution

imaging and the practical applications to capture of face, iris and fingerprint and palmprint friction

ridge details.

In his prior experience Mehrad has designed and build a Geo-Spatial Imaging system similar to

Google Streets, where massive amount of image database where captured and managed for

instant access. He has also led the R&D efforts for new projects in IT and Communications areas

such as new ad-hoc routing protocols, MAC-layer protocols, and underwater communication

which included the underwater surveillance security systems and solutions.

Mr. Tavakoli received his Masters of Science Degree from UCI, specializing in Computer

Architecture, Image processing, Signal and Systems and completing his thesis on Stand-Off

Biometrics Identification.

Author Biography - Mr. Mehrad Tavakoli

29