Anil K. Jain Michigan State University

http://biometrics.cse.msu.edu October 27, 2015

Forensics: The Next Frontier for Biometrics

Bertillon System (1882)

H.T. F. Rhodes, Alphonse Bertillon: Father of Scientific Detection, Harrap, 1956

Habitual Criminal Act (1869): Identify repeat offenders

Friction Ridge Patterns

Cummins and Midlo, Finger Prints, Palms and Soles, Dover, 1961 3

Fingerprint Matching: 1992 v. 2014

N. Ratha, D. Rover and A.K. Jain, "An FPGA-Based Point pattern Matching Processor with application to Fingerprint Matching", CAMP `95, Italy

Sun SPARCstation host ~100 MHz CPU; 512 MB RAM

On Splash 2 @1 MHz clock: ~ 6,300 matches/sec; on host 70 matches/sec.

16 Xilinx 4010s as PEs (512 KB memory)

CPU: 1.4 GHz RAM: 1GB

Apple iPhone6

Forensics & Biometrics: Shared Goals

Forensics Biometrics

• Latent prints • Fibers • Explosives • Paint chips • DNA • Tire marks • Shoe prints • Bite marks • SMT

• 2D Face • 3D Face • Fingerprint • Iris • Speech • Signature • Gait • Ear • Palmprint • Keystroke

Forensics: Identify suspects from crime scene evidence

Biometrics: Automated person recognition from body traits

Mugshots v. Body cam Images

6 http://www.abc15.com/news/region-northern-az/flagstaff/flagstaff-police-release-body-camera-video-showing-moments-before-officer-tyler-stewart-shot-killed

Constrained & cooperative

Unconstrained & uncooperative

Biometrics Forensics

Subject Cooperative Uncooperative

Data acquisition Constrained Unconstrained

Data noise (Illumination, background, distortion)

Low High

Repeat acquisition Possible Not possible

Data for training & test Moderate to large Small

Purpose User authentication Convince jury & judge

Statistical validation of identification

Not required Necessary for conviction

Biometrics v. Forensics

Meijer supermarket, Okemos

Time & Attendance

UAE border crossing

Cashless payment for lunch

Biometric Authentication

Coal mine safety HK smartcard ID

U.S. Visit (OBIM)

Apple Pay

• People can no longer be trusted based on credentials (pw, PIN, ID card)

• Apple Pay: mobile payment (reader, template & matcher stored in phone)

UAE border crossing

World’s Largest Biometric ID System

9

~900M unique 12-digit IDs issued (Oct. 2015) out of 1.2B residents

1: 900M slap & iris comparisons (de-dup) done before issuing new ID https://uidai.gov.in/

Reference database

Forensics: Latent Search

Latent with markup AFIS

1,290 71 70 48 44 Scores:

Top 5 candidates

• Suspect may not be in the database • Poor latent quality • Errors in markup

Forensics: Partial Face Search

Who is he?

One of them?

Automated Face Search

80M Face Database

Manual Forensic Examination

D. Wang, C. Otto and A. K. Jain, "Face Search at Scale: 80 Million Gallery", MSU Technical Report, MSU-CSE-15-11, July 24, 2015

State of the Art: Fingerprint Matching

Latent identification

Test Database & Evaluation Performance

FpVTE

2003

10K plain fingerprints; 1:1 comparison (Medium scale)

FRR = 0.6% @FAR=0.01%

FpVTE

2012

30K subjects (slap) v. 100K subjects (slaps);

1:N comparison (open set)

FNIR=1.9% @FPIR=0.1%

(right index finger)

ELFT-EFS

2011

1,114 latent prints against 100K subjects (Rolled + Plain)

Rank-1: 62.2%

ELFT-EFS

2012

1,066 latent prints v. 100K subjects (Rolled + Plain)

Rank-1: 67.2%

http://www.nist.gov/itl/iad/ig/biometric_evaluations.cfm

State of the Art: Face Verification LFW (2007) NIST IJB-A (2015) NIST FRGC v2.0 (2006) NIST MBGC (2010)

D. Wang, C. Otto and A. K. Jain, "Face Search at Scale: 80 Million Gallery", arXiv, July 28, 2015

Difficulties For Recognition Systems & Modeling

Intra-class variation Inter-class similarity

Unconstrained face images

VID VEO NV

Poor Latent quality

Biometrics to Forensics

15

• Improving latent performance

• Unconstrained face recognition

• Systems for tattoo matching, sketch (composite) to photo matching, altered fingerprint detection

• Addressing fundamental premise

• Distinctiveness (Pankanti, Prabhakar & Jain, 2002)

• Evidential value (Nagar, Choi & Jain, 2012)

• Persistence (Yoon & Jain, 2015)

Fingerprint database

(NIST 4)

32×32 patches

64×64 patches

Learning

coarse-level dictionary

(1,024 elements)

16 fine-level dictionaries

(64 elements)

Dictionary Learning

16

(0, π/16]

(π/16, 2π/16]

(15π/16, π]

Latent Enhancement & Cropping

Good

quality

Bad

quality

Ugly

quality

Latent Texture component Cropping Cropping & enhancement

Forensic Triage

Pool of examiners

Latent

AFIS Is markup needed?

Yes

No

Markups

Top-K candidates &

scores Rank-1

...

Rank-2

...

S. S. Arora, K. Cao, A. K. Jain and G. Michaud, "Crowd Powered Latent Fingerprint Identification: Fusing AFIS with Examiner Markups", ICB 2015

Sufficient quality?

Yes

No

Reject

Variability in Markups

Lights-out: No match Fusion rank: 2

Markup 1 (Rank 80)

Markup 2 (Failed to match)

Markup 3 (Rank 45)

Markup 4 (Rank 7)

Markup 5 (Rank 57)

Markup 6 (Rank 12,971) 19

258 latents 250K reference prints COTS AFIS 6 latent examiners

How Many Markups?

107 latents did not need any markup; hit rate saturates with 3 examiners

20

Unconstrained Face Recognition: Landmarks

Face detection and Landmarks (NIST IJB-A)

Unconstrained Face Recognition: Deep Learning

Input face image Learning network parameters Learned representation

# Parameters=5M

Networks need large training set: 500K faces of 10K persons (CASIA)

Scars, Mark & Tattoos

http://wtvr.com/2012/05/04/pictures-investigators-seek-shirtless-heavily-tattooed-suspect/ 23

J-E. Lee, W. Tong, R. Jin, and A. K. Jain, "Image Retrieval in Forensics: Tattoo Image Database Application", IEEE Multimedia, Vol. 19, 2012

Witness Description of Suspect

Tipsters told police the gunman appears to be a man in his 30s with close-cropped hair and stubble on his cheeks and chin

http://www.lansingstatejournal.com/viewart/20121023/NEWS01/310230020/Michigan-roadway-shootings-put-drivers-alert-prompt-school-lockdowns

FaceSketchID System

S. Klum, H. Han, B. Klare and A. K. Jain, "The FaceSketchID System: Matching Facial Composites to Mugshots", IEEE TIFS, 2014

Normal or Abnormal Ridge Flow?

27

Fingerprint of Gus Winkler (1933) before and after alteration

Fingerprint Capture of Newborn & Infants

September 21, 2015, Agra, India

Fingerprints of Newborns

6 hours (Subject 1) 1 week (Subject 2)

1270 ppi reader (2cm x 2cm x 8mm)

Forensics

Summary

Data collection

Image process.

Pattern recog.

Domain experts

Prob. models

Evidence

What About the Data?