BIOMETRIC DATA INDEXING FOR LARGE SCALE IDENTIFICATION SYSTEMScse.iitkgp.ac.in/~dsamanta/resources/thesis/Somnath-Dey... · 2017-06-20 · BIOMETRIC DATA INDEXING FOR LARGE SCALE

BIOMETRIC DATA INDEXING FOR LARGE SCALEIDENTIFICATION SYSTEMS

Somnath Dey

BIOMETRIC DATA INDEXING FOR LARGE SCALEIDENTIFICATION SYSTEMS

Thesis submitted to theIndian Institute of Technology Kharagpur

for award of the degree

of

Doctor of Philosophy (Ph.D.)

by

Somnath Dey

Under the guidance of

Dr. Debasis Samanta

School of Information TechnologyIndian Institute of Technology Kharagpur

Kharagpur - 721 302, IndiaJune 2013

c©2013 Somnath Dey. All rights reserved.

Approval of the Viva-Voce Board

26/06/2013

Certified that the thesis entitled Biometric Data Indexing for Large Scale Iden-tification Systems submitted by Somnath Dey to Indian Institute of Technology,Kharagpur, for the award of the degree Doctor of Philosophy, has been accepted bythe external examiners and that the student has successfully defended the thesis in theviva-voce examination held today.

Signature: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Name: Dr. Debasis Samanta, Information Technology, Supervisor

Signature: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Name: Prof. D. Roy Chowdhury, Computer Science & Engineering, Member

Signature: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Name: Dr. K. S. Rao, Information Technology, Member

Signature: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Name: Prof. M. K. Tiwari, Industrial Engineering & Management, Member

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

(External Examiner)

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

(Chairman of DSC)

CERTIFICATE

This is to certify that the thesis entitled Biometric Data Indexing for Large ScaleIdentification Systems, submitted by Somnath Dey to Indian Institute of Technol-ogy, Kharagpur, is a record of bona fide research work under my supervision and weconsider it worthy of consideration for the award of the degree of Doctor of Philosophyof the Institute.

Date: 26/06/2013

Dr. Debasis SamantaAssociate ProfessorSchool of Information TechnologyIndian Institute of Technology KharagpurKharagpur - 721302, India

DECLARATION

I certify that

a. The work contained in the thesis is original and has been done by myself under thegeneral supervision of my supervisor.

b. The work has not been submitted to any other Institute for any degree or diploma.

c. I have followed the guidelines provided by the Institute in writing the thesis.

d. I have conformed to the norms and guidelines given in the Ethical Code of Conductof the Institute.

e. Whenever I have used materials (data, theoretical analysis, and text) from othersources, I have given due credit to them by citing them in the text of the thesisand giving their details in the references.

f. Whenever I have quoted written materials from other sources, I have put themunder quotation marks and given due credit to the sources by citing them andgiving required details in the references.

Somnath Dey

BIOGRAPHY

Somnath Dey is currently a Ph.D. research scholar in the School of Information Technol-ogy at Indian Institute of Technology Kharagpur, India. In his research, he is focusingon biometric data indexing algorithms for different biometric traits, especially applicableto iris, fingerprint, face and multimodal traits. He has received the B.Tech. degree inInformation Technology from University of Kalyani, India in 2004 and the M.S. degree inInformation Technology from Indian Institute of Technology, Kharagpur, India in 2008.

His current research interests include image processing, computer vision, patternrecognition and biometric security. He is a student member of IEEE.

vii

Dedicated toMy parents and other family members

ACKNOWLEDGMENT

During this period of my doctoral study there are many people whose guidance, sup-port, encouragement and sacrifice has made me indebted for my whole life. I take thisopportunity to express my sincere thanks and gratitude to all these people.

First and foremost I would like to express my deepest gratitude to my revered supervi-sor Dr. Debasis Samanta for his invaluable guidance, and his encouragement throughoutmy work. His guidance and support is far beyond duty. His constant motivation, sup-port and infectious enthusiasm have guided me towards the successful completion of mydoctoral studies. My interactions with him have been of immense help in defining myresearch goals and in identifying ways to achieve them. His encouraging words have oftenpushed me to put in my best possible efforts. Above all, the complete belief that he hasentrusted upon me and has instilled a great sense of confidence and purpose in my mind,which I am sure, will stand me in good stead throughout my career.

It gives me immense pleasure to thank my doctoral scrutiny committee members Prof.D. Roy Chowdhury, Prof. M. K. Tiwari, Dr. K. S. Rao for their valuable suggestionsduring my research tenure. My sincere thanks to the heads of the department Prof. I.Sengupta and Prof. J. Mukhopadhyay for the world class infrastructure provided in thedepartment to the research students. I also thank all the faculty members of the School ofInformation Technology for their many helpful comments and constant encouragement.I owe my deepest gratitude to Dr. S. Misra and Dr. Monalisa Sarma for their continuoussupport and encouragement during my doctoral study. I sincerely remember the supportof office staffs Mithunda, Somadi, Malayda, Vinodda, Pratap and others.

I would like acknowledge the financial support of Department of Information Technol-ogy, New Delhi, India for my study. I am grateful to Tejas, Om Prakash and Jyotirmayfor their full support in the development of some algorithms in my research.

I wish to convey my special thanks to my old friends Chandan, Bodhi and Aparnafor their constant support and help during the various stages of my work. I am greatlyindebted to many of my friends for their constant inspiration. The support of my labmates namely Sushantada, Debasish, Rajkumar, Col. Ranjit Singh, Prasenjit, Shob-hana madam, Ashalata madam, Shankar, Prasenjit, Barsha, Narendra, Jaswasi, Jainath,Gaurang, Anant, Soumitri is invaluable. I would also like to express my thanks to

Soumyajit, Barikda, Sajalda, Ranjan, Pushpitadi, Aditi, Soumya, Arindamda, Nirnay,Sudhamay, Gautamda, Subarao, Kanchan, Partha, Saptarshi, Subhomoy, Soumyadipand many more. It is a great fun and source of ideas and energy to have friends likeSayan, Manoj, Pradipta, Soumalya, Santa, Jayeeta, Indira and many more during mystay at IIT Kharagpur. I also thanks all the family members of contemporary fellowneighbor, in particular Amit, Ashokeda, Kakali boudi and Anki for making the stay atIIT Kharagpur, ever memorable.

Nothing would have been possible without the moral support of my parents who havebeen the pillars of strength in all my endeavours. I am always deeply indebted to themfor all that they have given me. I also thank all the other members of my family includingmy brother and sisters for their love, affection and timely help. Finally, to thank my wifeSayani, I really have no words to express my gratitude for all her support, encouragement,understanding and sacrifice, without which it would have been impossible for me to finishthis work.

Somnath Dey

xii

List of Symbols and Abbreviations

List of Symbols

∠ angle

∂ Partial derivation

σ Standard deviation∑Summation

| · | Absolute value

Cb Blue chroma components

Cr Red chroma components

cdf Cumulative distribution function

CLTCNTRD Storing structure of cluster center

CMS Cumulative match score

CSET Candidate Set

ED Euclidean distance

EER Equal error rate

ER Indexing error

ERR Indexing error rate

FMR False Match Rate

FNIR False Negative Identification Rate

FNMR False Non Match Rate

FPIR False Positive Identification Rate

xiii


Gallery Gallery data set

GE Gabor energy

GI Gabor filter response

HR Hit rate

Id Identity of a subject

IDL List of identities

ImGI Imaginary part of Gabor filter response

indx Index key vector

INDX Index cube

KDINDX Kd-tree index space

LNINDX Linear storing structure

max Maximum

MDL List of median values

Median(·) A function returns median values

min Minimum

PR Penetration rate

Probe Probe data set

ReGI Real part of Gabor filter response

SCORE Score function

List of Abbreviations

2-D Two Dimensional

BATH Bath University Iris Database

BWT Burrows-Wheeler Transform

CANPASS Canadian Passenger Accelerated Service System

xiv


CASIA Chinese Academy of Sciences

CASIAV3I CASIA-IrisV3-Interval Iris Database

CASIAV4T CASIAIrisV4-Thousand Iris Database

CBSR Center for Biometrics and Security Research

CKS Clustered kd-tree Search

CMC Cumulative Match Characteristics

CN Crossing Number

CS Clustered Search

DARPA Defense Advanced Research Products Agency

DCT Discrete Cosine Transform

DHA Department of Home Affairs

DHS Department of Homeland Security

FBI Federal Bureau of Investigation

FERET Facial Recognition Technology

FRGC Face Recognition Grand Challenge

FVC Fingerprint Verification Competition

GB Gigabyte

GCC GNU Compiler Collection

HG Hash Generation

IAFIS Integrated Automated Fingerprint Identification System

IRIS Iris Recognition Immigration System

IrisCode Iris Code

KB Kilobyte

Kd-tree K-dimensional tree

KL Karhunen-Loeve

KM Karnaugh Map

LBP Local Binary Pattern

xv


LS Linear Search

LSH Locality-Sensitive Hashing

MB Megabyte

MCC Minutia Cylinder-Code

MMU2 Multimedia University Iris Database version-2

ms millisecond

NIST National Institute of Science and Technology

PCA Principle Component Analysis

PIN Personal Identification Number

RAM Random Access Memory

RGB Red Green Blue

SIFT Scale Invariant Feature Transform

SPLDH Signed Pixel Level Difference Histogram

ST Searching time

SURF Speed Up Robust Feature

SVM Support Vector Machine

TSA Transportation Security Administration

UIDAI Unique Identification Authority of India

VIP Verified Identity Pass

WVU West Virginia University Iris Database Release 1

xvi

Abstract

Biometric-based person identification is gaining its importance and now-a-days it is re-quired to process a large amount of biometric data in the order of millions. As a con-sequence, the traditional approaches where the identity of a query template is decidedby matching the query template with all stored templates are impractical. Further, theproblem is compounded when we have to deal with two or more biometric traits. Objec-tive of this thesis is to address these issues and investigate the indexing techniques forunimodal and multimodal biometric traits with large scale biometric data so that thematching process can be accomplished in a real time without compromising the accuracyof person identification.

In this work, we consider three biometric traits (iris, fingerprint and face). We proposean efficient indexing technique for each biometric trait and also propose the indexingmethod when these traits are used for a multimodal identification system. Our irisindexing mechanism uses Gabor energy features to generate a low dimensional index keyfrom an iris template. A new index space organization for iris biometric trait is proposedto retrieve similar iris templates from the database.

In fingerprint indexing, we consider local topology of minutiae using two closest pointstriangle for index key generation. The features are invariant to rotation and scaling andhence, the approach can deal with fingerprints from different devices and sensors. Weuse clustering and k-d tree-based indexing techniques for fingerprint identification.

Our proposed face indexing technique is based on the SURF key points and SURFdescriptors. We create a two level index space based on the key points and divide theindex space into a number of cells. We apply hash function on the key points to decidethe cell position of each SURF descriptors.

Our multimodal indexing method is based on the relative scores with respect to aset of reference subjects corresponding to each trait. We combine the scores using SVM-based score level fusion technique. These scores are used to generate index key for asubject. Based on the index code values, we store the subject identity into the database.We propose a new index space organization in the database and a technique to store andretrieve the subject identities into/from the database. We introduce a new rank levelfusion technique on the retrieved candidate sets using SVM rank.

The major contributions of the thesis are generating index key and proposing newindexing mechanism for different biometric traits. With our indexing approaches, it ispossible to retrieve a set of biometric templates similar to the query template in theorder of milliseconds, is independent of size of databases, and with higher hit rate andlower penetration rate. Thus, the index space organization proves to be effective for fastand accurate retrieval. Moreover, our proposed multimodal indexing approach can beextended for any number of biometric modalities.

Keywords: Iris data indexing, Fingerprint data indexing, Face data indexing, Multi-modal biometric data indexing, Identification systems

xvii

Contents

Approval i

Certificate iii

Declaration v

Biography vii

Dedication ix

Acknowledgment xi

List of Symbols and Abbreviations xiii

Abstract xvii

Contents xix

List of Figures xxv

List of Tables xxxiii

1 Introduction 1

1.1 Biometric Authentication Systems . . . . . . . . . . . . . . . . . . . . . . 11.1.1 Some Major Biometric Applications . . . . . . . . . . . . . . . . . 21.1.2 Mode of Operations . . . . . . . . . . . . . . . . . . . . . . . . . . 4

1.2 Biometric Data Indexing . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.3 Need and Urgency of Biometric Data Indexing . . . . . . . . . . . . . . . 61.4 Objectives and Scope of the Work . . . . . . . . . . . . . . . . . . . . . . 7

1.4.1 Iris Biometric Data Indexing . . . . . . . . . . . . . . . . . . . . . 7

xix

Contents

1.4.2 Fingerprint Biometric Data Indexing . . . . . . . . . . . . . . . . . 81.4.3 Face Biometric Data Indexing . . . . . . . . . . . . . . . . . . . . . 91.4.4 Multimodal Biometric Data Indexing . . . . . . . . . . . . . . . . . 9

1.5 Contributions of the Thesis . . . . . . . . . . . . . . . . . . . . . . . . . . 91.5.1 Iris Biometric Data Indexing . . . . . . . . . . . . . . . . . . . . . 91.5.2 Fingerprint Biometric Data Indexing . . . . . . . . . . . . . . . . . 101.5.3 Face Biometric Data Indexing . . . . . . . . . . . . . . . . . . . . . 111.5.4 Multimodal Biometric Data Indexing . . . . . . . . . . . . . . . . . 11

1.6 Organization of the Thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

2 Related Work 152.1 Survey on Iris Biometric Data Indexing . . . . . . . . . . . . . . . . . . . 15

2.1.1 Iris Texture-Based Indexing . . . . . . . . . . . . . . . . . . . . . . 152.1.2 Color Feature-Based Indexing . . . . . . . . . . . . . . . . . . . . . 17

2.2 Survey on Fingerprint Biometric Data Indexing . . . . . . . . . . . . . . . 182.2.1 Minutiae-Based Indexing . . . . . . . . . . . . . . . . . . . . . . . . 182.2.2 Ridge Orientation-Based Indexing . . . . . . . . . . . . . . . . . . 212.2.3 Other Feature-Based Indexing Techniques . . . . . . . . . . . . . . 24

2.3 Survey on Face Biometric Data Indexing . . . . . . . . . . . . . . . . . . . 262.4 Survey on Multimodal Biometric Data Indexing . . . . . . . . . . . . . . . 272.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

3 Iris Biometric Data Indexing 313.1 Preliminaries of Gabor Filter . . . . . . . . . . . . . . . . . . . . . . . . . 323.2 Preprocessing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343.3 Feature Extraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353.4 Index Key Generation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 363.5 Storing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

3.5.1 Index Space Creation . . . . . . . . . . . . . . . . . . . . . . . . . . 373.5.2 Storing Iris Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

3.6 Retrieving . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 413.7 Performance Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

3.7.1 Performance Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . 443.7.2 Databases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 453.7.3 Evaluation Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . 473.7.4 Validation of the Parameter Values . . . . . . . . . . . . . . . . . . 473.7.5 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

xx

Contents

3.8 Comparison with Existing Work . . . . . . . . . . . . . . . . . . . . . . . . 553.9 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

4 Fingerprint Biometric Data Indexing 574.1 Preprocessing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

4.1.1 Normalization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 584.1.2 Segmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 594.1.3 Local Orientation Estimation . . . . . . . . . . . . . . . . . . . . . 594.1.4 Local Frequency Image Representation . . . . . . . . . . . . . . . . 604.1.5 Ridge Filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 604.1.6 Binarization and Thinning . . . . . . . . . . . . . . . . . . . . . . . 604.1.7 Minutiae Point Extraction . . . . . . . . . . . . . . . . . . . . . . . 61

4.2 Feature Extraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 634.2.1 Two Closest Points Triangulation . . . . . . . . . . . . . . . . . . . 634.2.2 Triplet Generation . . . . . . . . . . . . . . . . . . . . . . . . . . . 64

4.3 Index Key Generation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 654.4 Storing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

4.4.1 Linear Index Space . . . . . . . . . . . . . . . . . . . . . . . . . . . 684.4.2 Clustered Index Space . . . . . . . . . . . . . . . . . . . . . . . . . 694.4.3 Clustered kd-tree Index Space . . . . . . . . . . . . . . . . . . . . . 72

4.5 Retrieving . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 764.5.1 Linear Search (LS) . . . . . . . . . . . . . . . . . . . . . . . . . . . 774.5.2 Clustered Search (CS) . . . . . . . . . . . . . . . . . . . . . . . . . 774.5.3 Clustered kd-tree Search (CKS) . . . . . . . . . . . . . . . . . . . . 79

4.6 Performance Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 804.6.1 Databases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 814.6.2 Evaluation Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . 824.6.3 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 824.6.4 Searching Time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 924.6.5 Memory Requirements . . . . . . . . . . . . . . . . . . . . . . . . . 94


5 Face Biometric Data Indexing 995.1 Preprocessing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

5.1.1 Geometric Normalization . . . . . . . . . . . . . . . . . . . . . . . 1005.1.2 Face Masking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102

xxi

Contents

5.1.3 Intensity Enhancement . . . . . . . . . . . . . . . . . . . . . . . . . 1025.2 Feature Extraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103

5.2.1 Key Point Detection . . . . . . . . . . . . . . . . . . . . . . . . . . 1035.2.2 Orientation Assignment . . . . . . . . . . . . . . . . . . . . . . . . 1075.2.3 Key Point Descriptor Extraction . . . . . . . . . . . . . . . . . . . 108


5.4.1 Index Space Creation . . . . . . . . . . . . . . . . . . . . . . . . . . 1105.4.2 Linear Storing Structure . . . . . . . . . . . . . . . . . . . . . . . . 1135.4.3 Kd-tree Storing Structure . . . . . . . . . . . . . . . . . . . . . . . 113

5.5 Retrieving . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1165.5.1 Linear Search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1175.5.2 Kd-tree Search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117

5.6 Performance Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1205.6.1 Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1205.6.2 Evaluation Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1215.6.3 Validation of the Parameter Value . . . . . . . . . . . . . . . . . . 1235.6.4 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124


6 Multimodal Biometric Data Indexing 1376.1 Feature Extraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1386.2 Score Calculation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1406.3 Reference Subject Selection . . . . . . . . . . . . . . . . . . . . . . . . . . 141

6.3.1 Sample Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1416.3.2 Subject Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142

6.4 Reference Score Calculation . . . . . . . . . . . . . . . . . . . . . . . . . . 1446.5 Score Level Fusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144

6.5.1 Score Normalization . . . . . . . . . . . . . . . . . . . . . . . . . . 1446.5.2 Score Fusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145


6.7.1 Index Space Creation . . . . . . . . . . . . . . . . . . . . . . . . . . 1486.7.2 Storing Multimodal Biometric Data . . . . . . . . . . . . . . . . . 149

6.8 Retrieving . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151

xxii

Contents

6.9 Rank Level Fusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1526.9.1 Creating Feature Vector for Ranking . . . . . . . . . . . . . . . . . 1526.9.2 SVM Ranking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153

6.10 Performance Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1566.10.1 Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1566.10.2 Evaluation Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1586.10.3 Training of SVM-based Score Fusion Module . . . . . . . . . . . . 1586.10.4 Training of SVM-based Ranking Module . . . . . . . . . . . . . . . 1596.10.5 Validation of the Parameter Values . . . . . . . . . . . . . . . . . . 1596.10.6 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162


7 Conclusions and Future Research 1697.1 Dimensionality of Index Key Vector . . . . . . . . . . . . . . . . . . . . . . 1697.2 Storing and Retrieving . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1717.3 Performance of Indexing Techniques . . . . . . . . . . . . . . . . . . . . . 1727.4 Threats to Validity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174

7.4.1 Internal Validity . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1747.4.2 External Validity . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1757.4.3 Construct Validity . . . . . . . . . . . . . . . . . . . . . . . . . . . 176

7.5 Future Scope of Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176

References 179

Publications 199

xxiii

List of Figures

1.1 Examples of biometric traits that can be used for authenticating an indi-vidual. Physical traits include fingerprint, iris, face and hand geometrywhile behavioral traits include signature, keystroke dynamics and gait [103]. 2

1.2 Different government and commercial applications which use biometricauthentication system [68,103]. . . . . . . . . . . . . . . . . . . . . . . . . 3(a) Iris recognition at Amsterdam Schiphol airports . . . . . . . . . . . . 3(b) Fingerprint verification system between local banks and the Depart-

ment of Home Affairs (DHA) . . . . . . . . . . . . . . . . . . . . . . 3(c) Iris recognition system in the e-airport at Tokyo Narita Airport . . . 3(d) Contactless palm-vein systems in ATMs in Japan. . . . . . . . . . . 3

1.3 Different modes of operations of a biometric authentication system. . . . . 5(a) Enrollment step of an authentication system. . . . . . . . . . . . . . 5(b) Steps of an authentication system in verification mode . . . . . . . . 5(c) Steps of an authentication system in identification mode . . . . . . . 5

1.4 Major steps in biometric identification system with indexing. . . . . . . . 5

3.1 An overview of our proposed iris biometric data indexing approach. . . . . 313.2 Preprocessing result of a sample iris image [71,73]. . . . . . . . . . . . . . 35

(a) Original eye image . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35(b) Pupil boundary detected eye image . . . . . . . . . . . . . . . . . . . 35(c) Iris boundary detected eye image . . . . . . . . . . . . . . . . . . . . 35(d) Localized iris image . . . . . . . . . . . . . . . . . . . . . . . . . . . 35(e) Normalized iris image . . . . . . . . . . . . . . . . . . . . . . . . . . 35(f) Enhanced iris image . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

3.3 Index space organization. . . . . . . . . . . . . . . . . . . . . . . . . . . . 383.4 Index keys of the 6th subject with 5 samples. . . . . . . . . . . . . . . . . 393.5 Retrieving a match for a query iris image from a database. . . . . . . . . . 43

xxv

List of Figures

3.6 HR and PR with different scales and orientations. . . . . . . . . . . . . . 49(a) HR with different scales and orientations . . . . . . . . . . . . . . . 49(b) PR with different scales and orientations . . . . . . . . . . . . . . . 49

3.7 HR and PR for different values of δ. . . . . . . . . . . . . . . . . . . . . . 50(a) HR for different values of δ . . . . . . . . . . . . . . . . . . . . . . . 50(b) PR for different values of δ . . . . . . . . . . . . . . . . . . . . . . . 50

3.8 CMC curves with reference to different databases. . . . . . . . . . . . . . 513.9 FPIR vs. FNIR curves with and without indexing for CASIA-V3-

Interval database. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 523.10 Average table size and memory requirements to enroll the iris data for

different iris image databases. . . . . . . . . . . . . . . . . . . . . . . . . . 54(a) Average table size to enroll iris data for different iris image databases 54(b) Memory requirements to enroll iris data for different iris image databases 54

4.1 An overview of our proposed fingerprint biometric data indexing approach. 574.2 Preprocessing of fingerprint image. . . . . . . . . . . . . . . . . . . . . . . 584.3 Pixel position and crossing number for two different types of minutiae. . . 61

(a) Pixel position in a 3× 3 window . . . . . . . . . . . . . . . . . . . . 61(b) CN=1 (ridge ending) . . . . . . . . . . . . . . . . . . . . . . . . . . . 61(c) CN=3 (bifurcation) . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

4.4 Minutiae position and minutiae orientation. . . . . . . . . . . . . . . . . . 62(a) Ridge ending . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62(b) Bifurcation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

4.5 Two closest point triangulation with sample fingerprint image. . . . . . . . 65(a) Two closest point triangulation . . . . . . . . . . . . . . . . . . . . . 65(b) Two closest point triangle with sample minutiae points of a finger-

print image . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65(c) Duplicate two closest point triangles . . . . . . . . . . . . . . . . . . 65(d) Duplicate triplets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

4.6 Components of a minutiae triplet. . . . . . . . . . . . . . . . . . . . . . . . 664.7 Linear index space. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 704.8 Clustered index space. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 714.9 Structure of a kd-tree and an example of kd-tree with three dimensional

points. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74(a) Components of a kd-tree node . . . . . . . . . . . . . . . . . . . . . 74(b) 3D points . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

xxvi

List of Figures

(c) Kd-tree example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 744.10 Clustered kd-tree index space. . . . . . . . . . . . . . . . . . . . . . . . . . 744.11 CMC curves with one enrolled sample per fingerprint in LS, CS and CKS

for different databases. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84(a) CMC curves with NIST DB4 database . . . . . . . . . . . . . . . . . 84(b) CMC curves with NIST DB4 Natural database . . . . . . . . . . . . 84(c) CMC curves with FVC 2004 DB1 database . . . . . . . . . . . . . . 84(d) CMC curves with FVC 2004 DB2 database . . . . . . . . . . . . . . 85(e) CMC curves with FVC 2004 DB3 database . . . . . . . . . . . . . . 85(f) CMC curves with FVC 2004 DB4 database . . . . . . . . . . . . . . 85

4.12 CMC curves of LS with different number of enrolled samples for differentdatabases. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87(a) CMC curves of LS with gallery G1, G3 and G5 for FVC 2004 DB1

database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87(b) CMC curves of LS with gallery G1, G3 and G5 for FVC 2004 DB2

database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87(c) CMC curves of LS with gallery G1, G3 and G5 for FVC 2004 DB3

database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87(d) CMC curves of LS with gallery G1, G3 and G5 for FVC 2004 DB4

database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 884.13 CMC curves of CS with different number of enrolled samples for different

databases. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88(a) CMC curves of CS with gallery G1, G3 and G5 for FVC 2004 DB1

database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88(b) CMC curves of CS with gallery G1, G3 and G5 for FVC 2004 DB2

database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89(c) CMC curves of CS with gallery G1, G3 and G5 for FVC 2004 DB3

database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89(d) CMC curves of CS with gallery G1, G3 and G5 for FVC 2004 DB4

database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 894.14 CMC curve of CKS with different number of enrolled samples for different

databases. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90(a) CMC curves of CKS with gallery G1, G3 and G5 for FVC 2004

DB1 database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90(b) CMC curves of CKS with gallery G1, G3 and G5 for FVC 2004

DB2 database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

xxvii

List of Figures

(c) CMC curves of CKS with gallery G1, G3 and G5 for FVC 2004DB3 database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

(d) CMC curves of CKS with gallery G1, G3 and G5 for FVC 2004DB4 database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

4.15 Comparison with existing work for NIST DB4 database. . . . . . . . . . . 95(a) NIST DB4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95(b) NIST DB4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95

4.16 Comparison with existing work for NIST DB4 Natural database. . . . . . 96(a) NIST DB4 Natural . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96(b) NIST DB4 Natural . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96

4.17 Comparison with existing work for FVC2004 DB1 database. . . . . . . . . 97

5.1 An overview of our proposed face biometric data indexing approach. . . . 995.2 Face images after different preprocessing tasks. . . . . . . . . . . . . . . . 101

(a) Face image before align . . . . . . . . . . . . . . . . . . . . . . . . . 101(b) Face image after align . . . . . . . . . . . . . . . . . . . . . . . . . . 101(c) Detected face boundary . . . . . . . . . . . . . . . . . . . . . . . . . 101(d) Fixed size face image . . . . . . . . . . . . . . . . . . . . . . . . . . . 101(e) Masked face image . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101(f) Enhanced face image . . . . . . . . . . . . . . . . . . . . . . . . . . . 101

5.3 Approaches to scale space creation. . . . . . . . . . . . . . . . . . . . . . . 104(a) Scale space creation by reducing image size . . . . . . . . . . . . . . 104(b) Scale space creation by increasing filter size . . . . . . . . . . . . . . 104

5.4 Gaussian partial 2nd order derivative and their approximation. . . . . . . 106(a) Derivative in x- direction (Lxx) . . . . . . . . . . . . . . . . . . . . . 106(b) Derivative in y- direction (Lyy) . . . . . . . . . . . . . . . . . . . . . 106(c) Derivative in xy- direction (Lxy) . . . . . . . . . . . . . . . . . . . . 106(d) Approximated derivative in x- direction (Dxx) . . . . . . . . . . . . . 106(e) Approximated derivative in y- direction (Dyy) . . . . . . . . . . . . . 106(f) Approximated derivative in xy- direction (Dxy) . . . . . . . . . . . . 106

5.5 Non-maximal suppression by checking the nearest neighbor in subsequentscale spaces. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107

5.6 Haar filters in x- and y- directions and orientation of a key point. . . . . . 108(a) Haar filter in x-direction . . . . . . . . . . . . . . . . . . . . . . . . . 108(b) Haar filter in y-direction . . . . . . . . . . . . . . . . . . . . . . . . . 108(c) Orientation assignment to a key point . . . . . . . . . . . . . . . . . 108

xxviii

List of Figures

5.7 SURF descriptor extraction at a key point. . . . . . . . . . . . . . . . . . 1095.8 Proposed index space to store all index keys of all face images. . . . . . . 1125.9 Linear index space to store all index keys of the ith cell. . . . . . . . . . . 1135.10 Kd-tree to store the index keys of the ith cell of the index space. . . . . . 1155.11 Sample images from FERET, FRGC and CalTech256 databases. . . . . . 121

(a) Frontal face image samples without expression from FERET database121(b) Frontal face image samples with expression from FERET database . 121(c) Frontal face image samples without expression from FRGC database 121(d) Frontal face image samples with expression from FRGC database . . 121(e) Sample face images in outdoor environment from FRGC database . 121(f) Sample face images from CalTech database . . . . . . . . . . . . . . 121

5.12 HR and PR with FERET, FRGC and CalTech256 databases for differentvalues of δ. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124(a) HR and PR with Gallery11 and Probe11 for different values of δ . 124(b) HR and PR with Gallery21 and Probe21 for different values of δ . 124(c) HR and PR with Gallery41 and Probe41 for different values of δ . 124

5.13 CMC curve with and without indexing for FERET, FRGC and CalTech256databases. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126(a) CMC curve with and without indexing for FERET Gallery11 and

Probe11 sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126(b) CMC curve with and without indexing for FRGC Gallery21 and

Probe21 sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126(c) CMC curve with and without indexing for CalTech256 Gallery41

and Probe41 sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1265.14 CMC curves for different Probe sets with single enrolment of a subject

with FERET, FRGC and CalTech256 databases. . . . . . . . . . . . . . . 127(a) CMC curves for different Probe sets with FERET Gallery11 . . . . 127(b) CMC curves for different Probe sets with FRGC Gallery21 . . . . . 127(c) CMC curves for different Probe sets with CalTech256 Gallery41 . . 127

5.15 CMC curves for different Probe sets with multiple enrolment of a subjectwith FERET, FRGC and CalTech256 databases. . . . . . . . . . . . . . . 129(a) CMC curves for different Probe sets with FERET Gallery12 . . . . 129(b) CMC curves for different Probe sets with FRGC Gallery22 . . . . . 129(c) CMC curves for different Probe sets with CalTech256 Gallery42 . . 129

5.16 FPIR vs. FNIR curve with and without indexing for FERET, FRGC andCalTech256 databases. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131

xxix

List of Figures

(a) FPIR vs. FNIR curve for FERET database . . . . . . . . . . . . . . 131(b) FPIR vs. FNIR curve for FRGC database . . . . . . . . . . . . . . . 131(c) FPIR vs. FNIR curve for FRGC database . . . . . . . . . . . . . . . 131

5.17 Average searching time with different sizes of databases for FERET, FRGCand CalTech256 databases. . . . . . . . . . . . . . . . . . . . . . . . . . . 132(a) Average searching time for FERET database . . . . . . . . . . . . . 132(b) Average searching time for FRGC database . . . . . . . . . . . . . . 132(c) Average searching time for CalTech256 database . . . . . . . . . . . 132

5.18 Average number of comparisons with different sizes of databases for FERET,FRGC and CalTech256 databases. . . . . . . . . . . . . . . . . . . . . . . 133(a) Average number of comparisons for FERET database . . . . . . . . 133(b) Average number of comparisons for FRGC database . . . . . . . . . 133(c) Average number of comparisons for CalTech256 database . . . . . . 133

5.19 Memory requirements with different sizes of databases for FERET, FRGCand CalTech256 databases. . . . . . . . . . . . . . . . . . . . . . . . . . . 135(a) Memory requirements for FERET database . . . . . . . . . . . . . . 135(b) Memory requirements for FRGC database . . . . . . . . . . . . . . . 135(c) Memory requirements for CalTech256 database . . . . . . . . . . . . 135

6.1 Overview of the proposed approach. . . . . . . . . . . . . . . . . . . . . . 138(a) Reference subject selection . . . . . . . . . . . . . . . . . . . . . . . 138(b) Identification system with indexing . . . . . . . . . . . . . . . . . . . 138

6.2 Representation of multimodal biometric samples. . . . . . . . . . . . . . . 1396.3 Index space for the mth reference subject. . . . . . . . . . . . . . . . . . . 1496.4 Example of storing the 3rd sample of the 5th subject into the 1st index

space of the database. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150(a) Index key of the 3rd sample of the 5th subject . . . . . . . . . . . . . 150(b) Database of the 1st index space . . . . . . . . . . . . . . . . . . . . . 150

6.5 Example of retrieving candidate sets for a given query. . . . . . . . . . . . 153(a) Tables of the biometric trait (B1) from all index spaces of the database153(b) Index keys for query subject . . . . . . . . . . . . . . . . . . . . . . . 153(c) Candidate sets for all biometric traits . . . . . . . . . . . . . . . . . 153

6.6 An example of SVM-based rank to each retrieved subject for a query. . . . 155(a) Candidate sets with retrieved data . . . . . . . . . . . . . . . . . . . 155(b) Feature vectors for all subjects in candidate sets . . . . . . . . . . . 155(c) Subject with rank after SVM . . . . . . . . . . . . . . . . . . . . . . 155

xxx

List of Figures

6.7 Sample images of CASIAV3I iris, WVU fingerprint and FRGC face databases.158(a) Sample iris images . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158(b) Sample fingerprint images . . . . . . . . . . . . . . . . . . . . . . . . 158(c) Sample face images . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158

6.8 HR and PR with different number of reference subjects. . . . . . . . . . . 1606.9 HR and PR with different number of enrolled samples and different table

sizes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161(a) HR with different number of enrolled samples and different table sizes161(b) PR with different number of enrolled samples and different table sizes161

6.10 HR and PR for different values of δ with the proposed indexing technique.1626.11 CMC curves of the proposed indexing technique with different combination

of biometric traits. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1636.12 FPIR versus FNIR curves of the proposed identification system without

any indexing and with iris, fingerprint, face and multimodal based indexing.1646.13 Memory requirements for different number of enrolled samples in the pro-

posed indexing technique. . . . . . . . . . . . . . . . . . . . . . . . . . . . 167

7.1 CMC curves of different indexing approaches. . . . . . . . . . . . . . . . . 173

xxxi

List of Tables

2.1 Summary of different iris data indexing techniques . . . . . . . . . . . . . 19

2.2 Summary of different fingerprint data indexing techniques . . . . . . . . . 25

2.3 Summary of different face data indexing techniques . . . . . . . . . . . . . 27

2.4 Summary of different multimodal biometric data indexing techniques . . . 28

3.1 Characteristics of databases used in our experiments . . . . . . . . . . . . 47

3.2 HR and PR for different iris image databases . . . . . . . . . . . . . . . . 50

3.3 Time required (in second) to retrieve best match using indexing and with-out indexing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

3.4 Comparison of HR and PR with existing work . . . . . . . . . . . . . . . 55

3.5 Retrieval time complexity and execution time of different indexing mech-anisms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

4.1 HR, PR and searching time (ST ) for different fingerprint databases withsingle enrollment (gallery G1) into the database . . . . . . . . . . . . . . 83

4.2 HR, PR and searching time (ST) in LS with different number of enroll-ments (Gallery G1, G3 and G5) into the database . . . . . . . . . . . . . 86

4.3 HR, PR and searching time (ST ) in CS with different number of enroll-ments (gallery G1, G3 and G5) into the database . . . . . . . . . . . . . . 88

4.4 HR, PR and searching time (ST ) in CKS with different number of enroll-ments (gallery G1, G3 and G5) into the database . . . . . . . . . . . . . 90

4.5 Execution time and average number of comparisons required in LS, CSand CKS with different number of enrolled samples (ES) . . . . . . . . . . 93

4.6 Memory requirements (in KB) of linear, cluster and clustered kd-tree indexspaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94

4.7 Comparison of search complexity and execution time of the existing work 97

xxxiii

List of Tables

5.1 Description of Gallery and Probe sets of FERET, FRGC and CalTech256face databases. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122

5.2 Performance of the proposed approach with and without indexing usinglinear and kd-tree search . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125

5.3 Performance of different Probe sets with single enrolment of a subject inlinear and kd-tree search . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128

5.4 Performance of different Probe sets for multiple enrolments of a subjectin linear and kd-tree search . . . . . . . . . . . . . . . . . . . . . . . . . . 130

5.5 Comparison of the proposed approach with existing approaches . . . . . . 136

6.1 Feature summary of different biometric traits . . . . . . . . . . . . . . . . 1406.2 Characteristics of the different score normalization methods . . . . . . . . 1456.3 Characteristics of the different score fusion methods . . . . . . . . . . . . 1466.4 Retrieved subjects and their votes in each candidate set . . . . . . . . . . 1546.5 Summary of biometric databases and virtual users . . . . . . . . . . . . . 1576.6 HR and PR of the proposed indexing technique with unimodal and mul-

timodal traits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1636.7 Average retrieving time for a query with different database sizes. . . . . . 1666.8 Comparison with existing multimodal indexing techniques [90] . . . . . . . 168

7.1 Feature representations of the different indexing approaches . . . . . . . . 1707.2 Performances of the different indexing approaches . . . . . . . . . . . . . . 173

xxxiv

Chapter 1

Introduction

In today’s security-concerned society, biometrics-based authentication systems are used inmany important applications [1,9–11,13,14,23,30,68,100,100]. Most of these applicationsneed to deal with a large amount of complex biometric data in the order of millions.Further, the biometric data do not follow the natural sorting order. As a consequence,the traditional indexing mechanisms are not suitable for biometric based identificationsystem to retrieve biometric data in faster way. In this dissertation, we investigate thebiometric data indexing techniques for three traits, namely iris, fingerprint and face. Wealso explore the indexing mechanism for multimodal biometric using these traits.

This chapter starts with an overview of biometric-based authentication system. Sec-tion 1.2 defines the data indexing of biometric. We discuss the need and urgency of thebiometric data indexing in an identification system in Section 1.3. Section 1.4 describesthe objectives and scope of the work. An overview of thesis contributions is given inSection 1.5. Finally, Section 1.6 gives the organization of the rest of the thesis.

1.1 Biometric Authentication Systems

Biometrics is the science of using physiological or behavioral characteristics of a humanto verify or identify the identity of a person. Fingerprint, iris, face, hand-geometry,palmprint and ear are the most commonly used physiological biometric traits whereasvoice, signature, key-stroke dynamics, gait are the some example of behavioral biometrictraits. Figure 1.1 shows different biometric traits for a person. When single biometrictrait is used for authentication purpose, then the authentication system is called unimodalauthentication system. Unimodal biometric systems have several limitations such as noisydata, intra-class variations, restricted degrees of freedom, non-universality, spoof attacks

1

1. Introduction

Physiological Biometric Trait Behavioral Biometric Trait

Iris Vein pattern

Palmprint

Hand

geometry

EarFaceFingerprint Voice Key-stroke

dynamics

Gait Signature

Figure 1.1: Examples of biometric traits that can be used for authenticating an individ-ual. Physical traits include fingerprint, iris, face and hand geometry while behavioral traitsinclude signature, keystroke dynamics and gait [103].

and unacceptable error rates. Some of these limitations can be overcome by using multiplebiometric traits or multiple source of information. Such system is called multimodalauthentication system. In different applications, biometrics are used in different modeswhich are described in the following.

1.1.1 Some Major Biometric Applications

Several commercial and government organizations use biometric based authentication inmany critical applications [1, 10, 30, 68, 100] like forensics science, surveillance, nationalborder controls, national identity card, defense security, attendance management, payroll management, driving licenses, secure financial transactions, ticket less travel, etc.In the US, Federal Bureau of Investigation (FBI) manages a fingerprint identificationsystem and database called the Integrated Automated Fingerprint Identification System(IAFIS) which currently consists of the fingerprints records of over 51 million criminalsand over 1.5 million civils (non-criminals). The US Transportation Security Administra-tion (TSA) and Department of Homeland Security (DHS) [27] deployed biometric-basedpassenger departure system to make security procedure more efficient at different air-ports [10, 68]. In UK, Iris Recognition Immigration System (IRIS) [17] is deployed toreplace passport presentation with biometric-based authentication system [11]. An auto-mated iris recognition-based immigration clearing system is used at Amsterdam SchipholAirport under Privium Program [18,26]. In Canada, biometric based Canadian PassengerAccelerated Service System (CANPASS) [5,6,68] operates in eight international airports.Though, the most of the current biometric based authentication systems rely on unimodalbiometric trait, there are several applications [1, 9–11, 24, 28, 30, 91] which are operated

2

1.1. Biometric Authentication Systems

with multimodal biometric in large-scale. For example, US Visit [30] has a repositoryof two fingerprints for more than 50 million people. In Orlando, LaGuardia, Newark,Dulles, Regan, Denver and San Francisco international airports, CLEAR program [9,91]

uses iris and fingerprint biometric for passenger departure. The program is operated byVerified Identity Pass (VIP) and it enrolls 1,75,000 members till July 2008 [68]. ThePrivium Program [18] has registered about 40,000 frequent travellers [68] with multiplebiometric in Schiphol Airport (Amsterdam NL). Unique Identification Authority of India(UIDAI) [1,29] has planned to register 600 million users with multiple biometric traits inIndia in next few years where number of accesses per day (in different public and privatedomains) are expected to be around 1 to 5 million. A snapshot of few biometric basedapplications are shown in Fig. 1.2.

(a) Iris recognition at AmsterdamSchiphol airports

(b) Fingerprint verification system be-tween local banks and the Department ofHome Affairs (DHA)

(c) Iris recognition system in the e-airport at Tokyo Narita Airport

(d) Contactless palm-vein systems inATMs in Japan.

Figure 1.2: Different government and commercial applications which use biometric au-thentication system [68,103].

3

1. Introduction

1.1.2 Mode of Operations

Depending on the context of applications, a biometric system may operate either inverification or identification mode. In both modes, users have to enroll via enrollmentprocess. The system validates an individual’s identity by comparing the captured bio-metric data with his/her own biometric template(s) stored in the system database in theverification mode. Here, an individual submits a claimed identity to the system, such aspersonal identification number (PIN), a user name or a smart card. The system performsone-to-one comparisons to determine whether the identity claimed by the individual isgenuine or false. In the identification mode, an individual is recognized by searching thetemplates of all users in the database. This system performs one-to-many comparison toestablish the identity of individual. In identification mode, there is no need to provideany identity by the individual. Figure 1.3(a) shows the enrollment process of a biometricauthentication system, and Figure 1.3(b) and (c) show the block diagrams of verificationand identification systems, respectively. In these block diagrams, sensor module capturesthe raw biometric information from human body and feature extraction module processesthe raw biometric information and extracts the features which is also referred as biomet-ric template. The matching module compares the templates of two persons and generatesa score value which represents the similarity (dissimilarity) of two persons. Finally, thedecision module determines the genuineness or identity of a person in verification andidentification mode, respectively.

Moreover, in identification mode, the matching module compares the biometric tem-plate of a person with all stored templates in the database. This process is computation-ally expensive when the number of stored templates in the database is huge. Identificationof a person can be done faster if we filter out some templates which are not similar tothe captured template.

1.2 Biometric Data Indexing

Biometric data indexing is a scheme to generate an index key from the biometric dataand assign the key to the corresponding template. Based on the index keys the bio-metric templates can be categorized into different groups in the database. The indexingtechnique is used to declare a person’s identity with lesser number of comparisons ratherthan searching the entire database. An identification system retrieves small set of similartemplates from the database based on the index key and performs detailed matching withthe retrieved templates to determine the identity of an individual. An overview of anidentification system with indexing is shown in Fig. 1.4.

4

1.2. Biometric Data Indexing

��

��

��

��

��

��

��

��

��

(a) Enrollment step of an authentication system.

��

��

��

��

��

��

��

��

� ��

��

��

��

��!�

��

��

��

"#"

(b) Steps of an authentication system in verification mode

��

��

��

��

��

��

��

� ��

��

��

��

��!�

$�%��

&��

��

"#'

(c) Steps of an authentication system in identification mode

Figure 1.3: Different modes of operations of a biometric authentication system.

��

��

��

��

��

��

��

��

��

��!�

$�%��

&��

��

��(��

��

��(��

Figure 1.4: Major steps in biometric identification system with indexing.

5

1. Introduction

1.3 Need and Urgency of Biometric Data Indexing

The applications like forensic science, surveillance, national border control, national iden-tity card, pay roll management, driving license, ticket less travel, etc. [1,10,29,30,68,100]

deal with large biometric databases, in the order of millions, and the size of databases isincreasing rapidly [1,10,18,29,68]. In large-scale applications, response time becomes im-portant in addition to accuracy. The high accuracy can be achieved by applying multiplebiometric traits in authentication systems [174,176]. Note that the biometric-based iden-tification systems are to deal with high dimensional features. Therefore, the exhaustivesearching with high dimensional features in a large-scale biometric system increases theresponse time of the system which may not be acceptable. The accuracy also deteriorateswhen the size of the database increases [156,157].

One way to improve the response time is to follow indexing techniques. It may benoted that the traditional database indexing techniques are not applicable to biometricdata because biometric data do not follow any natural order by which biometric datacan be sorted [156, 157]. Although, there exist several indexing techniques [62, 99, 130,140,141,146,149] for non-alpha numeric data such as images, video and audio. But theyhave the following issues.

• Image and video data are to be represented with metadata [140] such as annotatedtexts, symbols, tags etc.

• These data do not follow any traditional order such as numeric, alphanumeric etc.

• It is difficult to find semantic information [146] from multimedia data.

• In case of content based image or video indexing, high dimensional feature vectorsare used to represent the image or video. Searching with these high dimensionaldata is computationally intensive [149].

• Color features of images and videos may not give good classification because thesefeatures perform coarse level classification [130].

We cannot apply the existing image indexing techniques [62,99,130,140,141,146,149]

to the biometric data because the coarse level classification is not useful for biometricdata indexing where all biometric samples look almost similar. More specifically, iristexture, fingerprint ridge and minutiae pattern from different persons look very similarwith minute differences. For face biometric, distinction between the silhouette structuresfor different faces is less. These distinctions cannot be captured with standard image

6

1.4. Objectives and Scope of the Work

indexing techniques. Some image indexing techniques [130,141,146,149] use high dimen-sional feature vectors for finer level classification. But, these techniques are not suitablefor faster indexing in case of biometric data with high dimensional feature vector. Again,the color features based image indexing technique [58,62,99] cannot be applied to biomet-rics like iris, fingerprints where no color information is present. Though, face biometrichas color information, we could not apply color based image indexing technique becausecolor pattern remains almost same for the different persons after normalization. Hence,we need a low dimensional indexing technique for biometric data which can index thebiometric traits in finer level with less computation time.

Another way to improve the response time is by designing biometric template withlow dimensional features which allows for rapid matching [43,51,89,90,142,181,182,195].But, this method may not be applicable for a very large scale system because the templateneeds to compare with all stored templates to retrieve top matches though the size ofthe template is small. Hence, there is a need to develop a technique which can retrievethe biometric templates from the database in a faster and accurate manner for a largebiometric-based identification system.

1.4 Objectives and Scope of the Work

Objectives of this work is to explore indexing schemes for different biometric traits withlow dimensional features, which retrieves a small subset of the top best matches from alarge database and reduces the search space as well as the searching time. In this work,we consider three unimodal biometric traits namely iris, fingerprint and face. Anotherobjective of this work is to propose a new indexing method for multimodal biometricidentification system which uses iris, fingerprint and face biometric. The scope work tomeet the objectives with respect to different biometric traits are stated in the following.

1.4.1 Iris Biometric Data Indexing

Statistical independence of the iris pattern [66] of different eyes makes the iris to be avery reliable biometric modality for identification. Thus, iris-based identification is veryeffective in large databases. However, human iris consists of rich texture pattern. Dueto the rich texture it is observed that dimensionality of the extracted feature from iris isvery high. Hence, the matching process in identification mode can be computationallyexpensive. Therefore, there is a need to develop an indexing method to speed up theidentification. Indexing method reduces the search space of an identification systemby rapidly choosing a subset of iris images from the database before matching. For

7

1. Introduction

this purpose a number of work have been proposed. IrisCode [66] and Signed PixelLevel Difference Histogram (SPLDH)-based method has been proposed by Mukherjeeand Ross [164]. Mehrotra et al. [154] propose geometric hashing-based indexing schemefor iris images using Scale Invariant Feature Transform (SIFT) features. The limitationof these methods is that all of them deal with high dimensional features which make themethods computationally expensive and suspect to statistical unreliability [74]. Thus,there is a scope to generate low dimensional index key and develop efficient index spaceorganization technique into the database for iris biometric trait which can be performedin a faster manner.

1.4.2 Fingerprint Biometric Data Indexing

Fingerprint is another biometric trait which is also considered as highly reliable for au-thentication purpose due to its unique pattern [103, 104, 148]. A fingerprint containsdistinct ridge and minutiae pattern. Features extracted from ridge and minutiae of afingerprint are of high dimension [38, 43, 82, 175]. Again these features do not followany ordering. Therefore, exhaustive searching with these features in a large-scale systemmakes the identification process slow. Further, matching with high dimensional featuresis computationally expensive. This problem can be addressed by dividing the fingerprintsinto some predefined classes (arch, tented arch, left loop, right loop and whorl in Henry’sclassification) [148]. But the number of classes is small and fingerprints are unevenlydistributed among them (e.g. more than 90% of the fingerprints belong to only threeclasses (right loop, left loop, and whorl) [148]). To avoid the classification problem, con-tinuous classification scheme [43, 51, 52, 88, 129, 131, 132, 135, 196] is used for fingerprintindexing. In continuous classification, the fingerprint search is performed by either com-paring the query fingerprint with all database templates based on index key values orcomparing the query fingerprint with the templates until the match value satisfies somecriteria like less than some threshold value. The problem of the continuous classificationtechniques is that these are not able to sufficiently narrow down the search space [135].Further, majority of these approaches use singular points (i.e. core and delta points).Partial or distorted fingerprint and fingerprint scanned from small sensor may not havesingular points all the time [148]. Some minutiae-based approaches [38, 43, 82, 175] havebeen proposed to index the fingerprints. Nevertheless, these techniques suffer with thedimensionality problem of the index key. Hence, there is a scope to propose a new fea-ture set to generate low dimensional index key and develop a new indexing technique forfingerprint.

8

1.5. Contributions of the Thesis

1.4.3 Face Biometric Data Indexing

Face is another biometric which is commonly used in different applications for authenti-cation purpose. Each human face consists of some specific structure and contains someunique skin pattern. Generally, face recognition algorithms extract geometric featuresor statistical features from the face image. These extracted features are also with highdimension. For a large scale face-based identification system, same problems arise withhigh dimensional features as in other two biometric traits. The dimensionality reductiontechniques [39,142,142,158,159,181,182,194,195,197,199] have been proposed to addressthese problems. However, these methods use linear searching or nearest neighbor search-ing with the reduced low dimensional features which are not able to sufficiently narrowdown the search space. In this area, we would explore a new indexing mechanism withlow dimensional face feature.

1.4.4 Multimodal Biometric Data Indexing

Of late, authentication system uses multiple biometric traits to improve the performanceof the system. However, use of multiple biometric traits increases the number of featuresas well as the dimension of feature. This makes multi-biometric based identificationprocess more complex and computationally expensive. The tree-based structure [109–111,156, 157] is not suitable for faster retrieving with multimodal biometric traits. Becausethe dimensionality of the feature vector affects the searching time of tree structured-based index space [89, 90]. Further, cascading technique cannot take the advantagesof using multiple biometric traits. Moreover, linear search with a low dimension indexkey [89,90,95] cannot reduce the search space and may not be applicable when the size ofthe database is huge. In this area, we would propose to construct low dimensional indexby utilizing the advantages of multi-biometric traits and develop an efficient indexingscheme for multimodal biometric identification system using multiple biometric traits.

1.5 Contributions of the Thesis

In this work, we explore four indexing techniques for iris, fingerprint, face and multimodalbiometric traits. A brief description of each indexing technique is given in the following.

1.5.1 Iris Biometric Data Indexing

We propose an indexing technique for an iris-based biometric identification system usingthe iris texture features. The texture features are extracted from the iris image using a

9

1. Introduction

Gabor filter in different orientations and different scales followed by Gabor energy featuresare calculated from these texture features. We generate twelve dimensional index keyvector from these Gabor energy features to store and retrieve the iris data into and froma database. We create an index space based on the values of index keys of all subjects.The index space contains a table corresponding to each dimension of the index key. Westore the identities of the subjects and index key information into a table based on theindex key value. Each table stores the identities of all subjects. We retrieve similar iristemplates from the tables for a given query iris template depending on the index key valueof the query subject. Our proposed method retrieves a set of subject identities withoutsearching the entire database. Finally, we rank all the retrieved subjects according tothe number of occurrence in the retrieved set. We have experimented with different irisdatabases and measured the performance of the proposed approach with respect to hitrate, penetration rate and searching time. We achieve on an average 82.79% hit rate and13.78% penetration rate for all iris databases. The retrieving of subjects from the indexspace using iris indexing requires O(1) computation time and performs in the order ofmilliseconds irrespective of the sizes of databases. The average cumulative match scoreachieved by our proposed approach is 96.75% on the average at the 30th rank.

1.5.2 Fingerprint Biometric Data Indexing

In this indexing technique, our aim is to develop an efficient indexing technique forfingerprint-based identification system. Our fingerprint indexing technique uses the localtopology of the minutiae points. First, we detect the minutiae points from a fingerprintand find the unique two-closest point triangles for all minutiae points. Then, we computean eight dimensional feature vector for each triangle. The feature vector contains boththe geometric and texture features which are invariable to scale, rotation and translation.We generate a set of index keys from these feature vectors. We apply k-means clusteringon all index keys of all subjects to divide the index keys into different groups. We createa linear index space into the database to store the centroid of each cluster and create akd-tree index space for each cluster to store the identities and the index key informationof a subject. We retrieve a set of similar fingerprint templates for the index keys of aquery fingerprint. For each query index key, first, we select the cluster which containsthe fingerprint templates similar to the query index key. Then, we find the most similarfingerprint template for the query index key from the kd-tree index space of that cluster.We give a rank to each retrieved template according to their number of occurrencesin the retrieved set. We have tested our approach with different fingerprint databasesand measured hit rate, penetration rate, searching time and cumulative match score to

10

1.5. Contributions of the Thesis

substantiate the efficiency of our proposed approach. We achieve 83.83% hit rate and14.05% penetration rate, on an average. Our approach requires O(

√N) computation

time to retrieve a set of similar fingerprint templates and takes time in the order ofmilliseconds. We can achieve 99.83% average cumulative match score at the 30th rank.

1.5.3 Face Biometric Data Indexing

In our face indexing technique, we align and scale the face images before extractingthe features from the face images. We detect Speed Up Robust Feature (SURF) keypoints [36,37] in different scale spaces of the preprocessed face images and extract SURFfeature descriptors at each key point. We generate a set of sixty eight dimensional indexkeys from the key points, feature descriptors and identity of a face image. The firstfour dimensions of index keys are used to create a two-level index space. The first levelindex space divides all face images into two groups depending on the value of the firstdimension. In the second level index space, we create two index cubes based on the otherthree dimensions. Each cell of an index cube keeps the reference of a kd-tree where westore the feature descriptors and the identity of a set of face images. We apply hashfunctions on the index keys to find the cell positions for face images. At the time ofidentification, we apply the same hash functions on the extracted key points of the queryface image and search the kd-tree to retrieve a small set of similar identities from theindex space. Finally, we rank all retrieved identities according to their occurrences. Ourproposed face indexing method is tested with different face databases. We achieve onthe average 93.52% hit rate and 9.30% penetration rate. The searching complexity ofthe proposed face indexing is O(logN). Our approach is able to achieve on the average96.69% cumulative match score for all databases.

1.5.4 Multimodal Biometric Data Indexing

In our proposed multimodal biometric-based indexing approach, we use iris, fingerprintand face biometric traits. Our approach selects a small set of reference subjects for eachtrait based on intra sample and inter subject variance to ensure the discriminant indexkey generation. The discriminant index key helps to retrieve accurate subject from thedatabase. We generate a low dimensional index key from the relative scores of a subjectwith respect to a small set of reference subjects, as low dimensional index key performsfaster search in the database. We generate index key for a subject from the score valuesof each unimodal and their combined score values. This takes the advantages of each uni-modal trait as well as the multimodal traits in indexing. We propose a table-based index

11

1. Introduction

space organization technique for the database. Our database contains an index space fora reference subject and each index space accommodates a set of tables related to eachbiometric trait. We store the identity of a subject into the index spaces of the databaseaccording to index keys generated by the subject. The subjects with similar index keyswill be stored together into the index space. We retrieve a small set of candidates sim-ilar to the query subject for each key value of the query index key vector related to abiometric trait. We apply hash function to retrieve the candidate set for each modalityin a constant time. A subject in different candidate sets may have different ranks fordifferent traits [89,90]. We apply Support Vector Machine (SVM) based rank level fusiontechnique to combine the retrieved candidates. The rank level fusion using SVM givesthe ranking of candidates according to their likelihood. We define a new feature set forRank SVM model [117, 118]. The proposed multimodal indexing is tested on a virtualusers database. The virtual users are created from the different iris, fingerprint and facedatabases. We attain 96.11% hit rate and 13.86% penetration rate for the virtual usersdatabase. Our indexing approach retrieves a small set of similar candidates from thedatabase in O(1) computation time and performs in the order of milliseconds. We canachieve 99.25% cumulative match score for our proposed multimodal indexing approach.

We summarize the major contributions of our thesis in the following.

• Gabor energy-based twelve dimensional index key generation for iris biometric dataindexing is proposed.

• Based on the generated iris index keys a new indexing mechanism is proposed tostore and retrieve the identities of subjects for iris-based identification system.

• A new set of triplet features are extracted from a fingerprint by two-closest pointtriangulation method. These triplet features are used to index the fingerprint data.

• K-means clustering and kd-tree based storing and retrieving are explored for fin-gerprint data indexing.

• A two-level indexing mechanism is proposed to index face database. The SURFkey points of a face image are used to generate low dimensional index keys.

• A relative score-based index key generation method is proposed for multimodalbiometric indexing system.

• A new multi-biometric indexing mechanism is developed to store and retrieve thesubject identities.

• We introduce a new rank level fusion technique on the retrieved candidate setsusing SVM rank.

12

1.6. Organization of the Thesis

1.6 Organization of the Thesis

This thesis contains seven chapters including this introductory chapter. This chaptercontains brief description of biometric system, need and urgency of biometric data in-dexing, scope and objectives of the work and major contributions in this dissertation.The rest of the thesis is organized as follows.

Chapter 2 : Related WorkThis chapter includes state of the arts for different data indexing techniques for iris,fingerprint and face. We also discuss about existing multimodal biometric data indexingmethods.

Chapter 3 : Iris Biometric Data IndexingWe describe the proposed iris biometric data indexing technique based on Gabor energyfeatures and provide the experimental results on different iris databases in this chapter.

Chapter 4 : Fingerprint Biometric Data IndexingThe proposed fingerprint data indexing technique based on two-closest point triangula-tion is described in this chapter. The experimental results of the proposed fingerprintindexing method on different fingerprint databases are also given in this chapter.

Chapter 5 : Face Biometric Data IndexingSURF key point-based indexing technique for face images and experimental results ondifferent face databases are given in this chapter.

Chapter 6 : Multimodal Biometric Data IndexingThis chapter describes the indexing techniques using multi-biometric traits. We alsoprovide the experimental results of multi-biometric indexing method in this chapter.

Chapter 7 : Conclusions and Future ResearchThis chapter concludes our study in the domain of biometric data indexing and discussespotential future research directions in this field.

13

Chapter 2

Related Work

In this chapter, a survey of literature related to the contributions made in this dissertationis reported. The chapter is organized as follows. Various iris data indexing techniques arereviewed in Section 2.1. Section 2.2 covers an in-depth discussion about the fingerprintbiometric data indexing. Work related to face biometric data indexing is presented inSection 2.3. Section 2.4 reviews the work on multimodal biometric data indexing. Finally,the chapter is summarized in Section 2.5.

2.1 Survey on Iris Biometric Data Indexing

In the last few decades, iris recognition algorithms for verification [66, 67, 70, 73, 94, 124,127,144,145,177,198] have been extensively investigated. However, work on iris biometricdata indexing techniques have been scarcely reported. We can divide all the existingwork on iris data indexing techniques [81, 112, 153, 154, 163, 164, 169, 171, 185, 193] intotwo categories: iris texture-based indexing and iris color-based indexing. In iris texture-based indexing, the index keys are generated from the iris texture whereas color of iris isused in iris color-based indexing. Detailed descriptions of these techniques are given inthe following.

2.1.1 Iris Texture-Based Indexing

Yu et. al. [193] propose a coarse iris classification technique using fractals, which classifiesiris images into four categories. In this method, an iris image is segmented into sixteenblocks. Among these blocks, eight blocks belong to an upper part and remaining blocksto a lower part of the iris. Authors calculate fractal dimension value from the imageblocks and take the mean value of the fractal dimension of the upper and the lower part.

15

2. Related Work

Finally, all the iris images are classified into four categories in accordance with the fractaldimensions of the upper and the lower parts.

Mukherjee and Ross [163, 164] develop two methods for iris data indexing. In theirfirst method, they calculate IrisCode features [70] using Gabor wavelet. IrisCode featuresaccording to their method are as large as of 2048 dimensions. To reduce the dimension-ality of features they propose row/column/block averaging and Principle ComponentAnalysis (PCA). Then k-means clustering approach is used to partition the reduced di-mension iris code into multiple groups. They propose another technique called SignedPixel Level Difference Histogram (SPLDH) Analysis [163, 164]. In this technique, theydivide an iris texture into a number of blocks. For each block they calculate the his-togram of signed differences pixel intensities (of similar positioned pixels in the blockand adjacent blocks). Like the IrisCode method, this method also deals with features ofaround 256 dimensions.

Local Binary Pattern (LBP) [165] is used in another iris biometric data indexingmethod proposed by Mukherjee [163]. Author divides the normalized iris image into anumber of blocks and computes block-based LBP values. Finally, k-means clustering isapplied to group the iris data.

Mehrotra et al. [154] use Scale Invariant Feature Transform (SIFT) [139] method toextract texture features for iris data indexing. This method finds a number of key points.Then high dimensional (124 dimensions) SIFT features are extracted for each key point.The key points are indexed using geometric hashing [190]. This indexing method alsodeals with a large number of key points (≈ 100 key points) with high dimensional featurevector.

In another work, Mehrotra et al. [153] propose an iris data indexing technique basedon Discrete Cosine Transform (DCT). They divide the normalized iris image [153] intosub-bands using multi-resolution DCT transformation [191] and calculate the DCT en-ergy based histogram for each sub-band. Then, each histogram is divided into fixed sizebins to group the iris images. To generate a global key for each iris image, they obtainthe bin number for each sub-band and traverse all sub-bands in Morton order [161]. Theyuse B-tree structure to store the iris data. At the time of probing, they generate a keyfrom query iris image and traverse the tree using the query key to retrieve the iris data.Finally, they compare the retrieved data with the query to find the best match.

Gadde et al. [81] design an iris indexing mechanism based on the context clusteringproperty of the Burrows-Wheeler Transform (BWT) [49]. First, they convert a normal-ized gray scale iris image to a binary image and choose a horizontal n-bit pattern. Thenthe locations of these patterns are found in the iris image using BWT. They define an

16

2.1. Survey on Iris Biometric Data Indexing

indexing scheme based on the count of occurrence of the n-bit binary patterns in thenormalized binary iris image. For this purpose, the image is divided into K segmentsvertically. After that the iris image is assigned an index with the segment number wherethe segment number has the maximum number of occurrences for the n-bit pattern. Atthe time of probing, the test image is assigned four indices corresponding to the top foursegments with maximum counts of occurrence of n-bit pattern. Finally, the test imageis compared only against those images in the database that have these 4 indices as theirindex, starting with the first index.

In another work, Rathgeb and Uhl [171] utilize biometric hash generation (HG) [183]

for iris indexing. The basic idea of this method is to generate hash value from the differentparts of an iris image. In their approach, the normalized iris image [171] is divided intoseveral blocks and an n-bit binary hash is computed using HG [183]. They construct ann-bit Karnaugh Map (KM) [186] to store the iris data. A pointer to each iris template isstored at the according node of the KM. At the time of querying, the extracted hash ismapped onto the stored KM. If a node points at one or more iris templates, the extractediris-code is matched against those. This procedure is repeated recursively for neighboringnodes until identification is yielded or a certain search depth is reached.

Apart from the above texture-based iris data indexing techniques, Vatsa et al. [185]

develop another indexing scheme which consists of two phases. In the first phase, themethod uses Euler code [44, 184] to generate a small subset of possible matches. In thenext phase, it uses 2ν-SVM [60] match score fusion algorithm to find the best matchesfrom the list of possible matches obtained in first phase using textural and topologicalfeatures of the iris images.

2.1.2 Color Feature-Based Indexing

Few iris color-based indexing schemes have been proposed in the existing literature. Fuet al. [78] make first attempt to classify iris images using color. Their technique is basedon artificial color filtering. They divide all irises into nine classes and design an artificialcolor filter [57] corresponding to each class. For an iris image they apply all nine filtersto the image and assign the class corresponding to the filter which gives the maximumresponse. The artificial color filter trains with a discriminator that assigns a pixel eitherto the class of interest or to some other classes.

Another attempt to index iris images using color has been made by Puhan andSudha [169]. This method also refers group based color indexing scheme which relieson the natural iris color. In this technique, the iris color image is converted from RGBspace to YCbCr color space and two types of color indices, namely blue and red indices,

17

2. Related Work

are computed using Cb and Cr components, respectively. The range of values of red andblue color indices are partitioned into a number of groups. Depending upon the values ofred and blue color indices, an image is assigned to one of these groups. During searching,for a query, a few groups from blue and red color indices are selected based on the blueand red color indices of the query image. The nearest identities are declared based onthe intersection between these groups.

Jayaraman et al. [112] develop a hierarchical approach to efficiently retrieve the irisimages from a large iris database. This approach uses iris color to index the databaseand iris texture to retrieve the iris images from the indexed iris database. First, theyselect a set of iris images which are similar to the query image by applying indexingtechnique. The index value is computed by averaging the intensity values of all red andblue color pixels. They use kd-tree structure to store the iris data. Then, they use iristexture features of the selected images to determine the identity of the query image. Theiris texture features are extracted through Speeded Up Robust Features (SURF) [37]

algorithm.A summary of different iris biometric-based data indexing techniques is given in

Table 2.1.

2.2 Survey on Fingerprint Biometric Data Indexing

Fingerprint based biometric data indexing has been extensively studied. Several tech-niques [38, 43, 51, 52, 54, 55, 59, 61, 76, 82, 88, 105, 114, 122, 129, 131, 132, 135–138, 143, 150,163, 175, 188, 196] have been proposed in literature for fingerprint data indexing. Wecan divide the existing fingerprint indexing techniques into two main categories basedon the feature extraction methods. The first category is Minutiae-based indexing whichuses minutiae points of a fingerprint to extract the indexing features and the secondcategory is Ridge orientation-based indexing which use orientation of the ridges and val-leys of a fingerprint for index key generation. A detailed descriptions of some importantfingerprint data indexing methods are presented in the following.

2.2.1 Minutiae-Based Indexing

In this category of fingerprint indexing techniques, the index feature vectors are ex-tracted from either the minutiae description or the triplet structure form by the minutiaepoints. Germain et al. [82] present an algorithm to generate a list of candidate set usingminutiae triplets from the database. This method uses two structures, called map andmultimap [50], to compute all index score values. In enrollment time, they extract all

18

2.2. Survey on Fingerprint Biometric Data Indexing

Table 2.1: Summary of different iris data indexing techniques

Method Feature used Dimension Indexingtechnique

Iris

text

ure-

base

din

dexi

ng

Yu et al. [193] Fractal dimensions 16 Classificationtechnique

Mukherjee and Ross[164]

Colomn/Block Average andPCA of Iris Code 64 to 256 k-mean

clustering

Mukherjee and Ross[164]

Signed Pixel LevelDifference Histogram of non

overlapping blocks256 Tree structure

Mukherjee [163] Histogram of block-wiseLBP >128 k-mean

clustering

Mehrotra et al. [154] SIFT features at differentscale spaces 256 Geometric

hashing

Mehrotra et al. [153] DCT energy 10 B-tree

Gadde et al. [81] Burrows-WheelerTransform >50 Group based

method

Rathgeb and Uhl [171] Hash Code from block wisegray scale value 16 to 32 Karnaugh map

Vatsa et al. [185] Euler code 4 Linear

Iris

colo

r-ba

sed

inde

xing

Fu et al. [78] Artificial color filter >9Pattern

recognitionmethod

Puhan and Sudha[169] Color indices 2 Group based

method

Jayaraman et al. [112] Color indices 2 Kd-tree

possible minutiae triplets from a fingerprint and generate the set of index keys from thesetriplets. The length of each side, ridge counts between each pair of sides, angles measureswith respect to the sides for a triplet are used as index key features. In this method, alabel is given to each index key and an entry to the multimap structure is added for theindex key. In the querying process, a set of index keys is generated for a given query andused to retrieve the items from the multimap which are stored under the same index.Each retrieved item is represented by a hypothesized match between subsets of featuresin the query instance and the reference model instance that creates the item stored inthe multimap. This hypothesized match is labeled by the reference model identifier.Further, a map structure is used to calculate the votes for these hypothesized matchesand generate a candidate list.

19

2. Related Work

Bebis et al. [38] propose a fingerprint indexing technique based on the Delaunay tri-angulations [41]. This method, first, generates all Delaunay triangles from the extractedminutiae points. The ratios of largest side with the two smallest sides and cosine an-gle between the two smallest sides of the triangle are defined as index feature vector.The method uses fingerprint retrieving method similar to that described by Germain etal. [82] to generate candidate list.

Bhanu and Tan [43] propose fingerprint indexing technique based on minutiae triplets.They, first, create all possible triplets from the extracted minutiae points. Geometricfeatures like angles, handedness, type, direction and maximum side are used for indexingpurpose. In the querying process, they search for a number of matching triplets betweenquery and all stored templates by applying some constraints on the geometric featuresof the triplets. They select those templates which have more number of matched tripletsthan a threshold value. Further, they compute posterior probability of the query and theselected templates as index score and sort the index scores in descending order. Finally,candidate set is generated by putting a threshold on the index score.

Another Delaunay triangulations based fingerprint indexing approach is proposed byLiang et al. [132]. However, their approach use low-order Delaunay triangulation [33,86].In this approach, first, authors find all minutiae points from a fingerprint and generatea set of minutiae triplet using low-order Delaunay triangulation [33]. For each tripletthey derive minutia details of each vertex of the triangle, minimum and median anglesin the triangle, triangle handedness, length of the longest edge in the triangle and thedifference between angles of two edges which are used as indexing features. In this way, aset of indexing features are generated from the all triplets. At the time of querying, theyfirst find the matched triplets between the query and the stored templates by applyingindividual threshold on each feature. Finally, indexing scores are calculated betweenmatched triplets and query triplet and based on a threshold value a candidate list isgenerated.

Ross and Mukherjee [175] suggest ridge associated features for calculating index vec-tors. This approach is also based on Delaunay triangulation [33, 86]. They derive a setof triplets using Delaunay triangulation and generate a feature vector from each triplet.They compute features corresponds to the geometry of the triangles which are generatedby the triplets. Further, they estimate the shape of the ridges associated with the threeminutiae points constituting the triplet. The shape features are computed by fitting aquadratic curve to the ridges associated with each triplet. Once all feature vectors ofall triplets are generated, they partition these vectors into a number of clusters using k-means clustering algorithm for indexing purpose. Querying performs by k-mean clustered

20


search. The score value is computed from the correspondence between the fingerprinttemplates of the retrieved clusters and the query template.

Zhang et al. [196] introduce an indexing technique based on minutiae structure. Theyextract direction, coherence and curvature from each minutiae point as indexing features.In addition to this, an orientation vector is defined to characterize the each minutiae.They maintain an orderly sequence of these features to estimate the similarity betweentwo minutiae points. At the time of querying, they first find the similar minutiae pointsfrom two fingerprints by comparing the direction, coherence, curvature and orientationfeatures. An index score is computed which represents the possibility that two minutiaeare correspondent based on not only their similarity but the similarities to other minutiaepoints. Finally, they sort the fingerprint in descending order of the index scores andretrieve top fingerprints to generate candidate list.

In a recent work, Minutia Cylinder-Code (MCC) [53] based fingerprint indexing tech-nique has been proposed by Cappelli et al. [54]. They represent spatial and directionalrelationships between a minutiae and its neighborhood structure with a minutiae cylin-der [53]. Then each minutiae cylinder is encoded into a fixed-length binary feature vector.Hence, a fingerprint image is represented by a set of bit vectors. The bit vectors are in-dexed by means of Locality-Sensitive Hashing (LSH) [83, 101]. In LSH, each vector isprojected into a subspace with low dimension. For this purpose, they define a set ofhash functions. These hash functions map a binary vector to a natural number in lowdimension. They maintain a hash table corresponding to a hash function to store thefeature vectors into the database. The feature vectors are stored into the hash tablesbased on the hash values. To retrieve the candidates from the database, the same hashfunctions are applied to each binary vector of the query template and the number ofcollisions with database vectors is counted using an accumulator matrix. The candidatesare finally ranked according to the similarity distances of the query and the retrievedtemplates. The similarity between two MCC codes is measured by Hamming distance.

2.2.2 Ridge Orientation-Based Indexing

In this category, fingerprint indexing techniques use the ridge orientation to computethe index feature vectors. Lumini et al. [143] present an indexing scheme based ondirectional image which represents the local ridge and valley directions from a fingerprintimage. They generate a multidimensional feature vectors from the directional image.Further, they apply Karhunen-Loeve (KL) transform to reduce the dimensionality of thedirectional feature vectors. At the time of retrieving, they apply linear search with theselow-dimensional features and generate the candidate list based on a threshold value.

21

2. Related Work

Cappelli et al. [55] presents a fingerprint indexing method which uses dynamic masksfor directional image partitioning. Directional image represents local average directionsof a fingerprint ridge lines and is obtained by Chong et al. [61] method. Authors perform“guided” partitioning of the directional image according to the fingerprint topology. Forthis purpose they define a set of dynamic masks which are directly derived from the mostcommon fingerprint classes (Arch, Tented Arch, Left Loop, Right Loop, Whorl, etc.).Each mask is characterized by a set of vertices which defines the borders of the regions anduse to segment the directional image. Further, the application of these masks produces anumerical feature vector which represents each fingerprint as a multidimensional point.At the time of retrieval, the numerical feature vector is generated from a query imageand used as an access key for similarity searches. The searching is performed in themultidimensional space.

Lee et al. [129] aim for a fingerprint classification method based on a feature map.They use ridge features consisting of orientations and inter-ridge spacing within a rela-tively small fingerprint area to build the feature map. They divide the fingerprint imageinto a number of non-overlap blocks. The local ridge orientation and inter-ridge spacingare calculated for a block using the method proposed by Hong et al. [96] and each blockis represented by a vector. They define feature map as a matrix, where each elementrepresents a distribution of a feature vector. Each row and column of the matrix rep-resents orientation and inter-ridge spacing values, respectively. The dimension of thefeature map is reduced using well-known principle component analysis [79] dimensional-ity reduction method. Finally, they compute the distances between query and all storedtemplates and generate a candidate list by putting threshold on the distance.

Another fingerprint retrieval framework has been reported by Jiang et al. [114].Authors use orientation features as the main retrieval feature and the dominant ridgedistance as an auxiliary retrieval feature. To compute orientation features, first, theyestimate orientation field [96] of a fingerprint image which consists of local dominantorientations in different local neighborhoods of a fingerprint image. Then they select areference point and find its location and direction. Finally, they align the local orienta-tions with respect to the reference point. The orientation feature vectors are constructedby concatenating the aligned local orientations within a circle. Additional to these ori-entation feature vectors, they define the dominant ridge distance as the second meanof the local ridge distances. The local ridge distance is computed by measuring thedistance between the center points of two adjacent ridges along a line perpendicular tothe local ridge orientation using Hong et al. [96] method. They perform indexing usingboth continuous classification and clustering approach. In continuous classification, they

22


sort the fingerprints in the database according to the dominant ridge distance. At thetime of retrieval, first, they narrow the search space by selecting a subset of fingerprintswhose dominant ridge distances are within a range centered at the query dominant ridgedistance. Then, they use orientation vector to generate the final candidate list from thereduced search space. In clustering technique, they use k-mean clustering algorithm tocluster the fingerprints into different groups.

Liu et al. [136] introduce a fingerprint retrieval based on two complex filter responses.In their approach, first, they compute the orientation field of the fingerprint image andfind the location and orientation of some singular points (e.g. core or delta) in thefingerprint image. They apply two complex filters on the orientation field at the singularpoint and calculate the magnitude of the filter responses. Further, they align the filterresponse with the direction of the singular point and construct a numerical feature vectorby concatenating the aligned magnitudes of the two filter responses. This feature vector isused as the global feature for fingerprint indexing. For a query fingerprint, the retrieval isdone by measuring the Euclidean distances between the query and all stored fingerprints.Based on a threshold value they generate the final candidate list.

In another approach, Liu et al. [137] propose clustering of fingerprints using multi-scale orientation features and average ridge distance feature. First, they compute theorientation field by uniformly dividing the fingerprint into a number of blocks [115]. Tocompute the multi-scale orientation field, they construct a circular tessellation with non-uniform blocks and place the circular tessellation at some reference point (core or delta)of the fingerprint. They compute the orientation feature vectors from the each blockof circular tessellation. In addition to this, they compute the average ridge distance ofthe fingerprint. They partition all fingerprints by applying k-means clustering with theorientation feature vectors. Further, they divide each partition into a number of binsbased on the average ridge distance. At the time of retrieval, first they select the clusterusing orientation feature vectors of query fingerprint. Further, they use average ridgedistance to retrieve the fingerprint from the bins of the selected cluster.

In a recent work, Cappelli [51] proposes a fingerprint indexing approach based onvector and scalar fingerprint features. The both types of features are obtained from lo-cal ridge-line orientations and frequencies. To compute local orientations, they dividethe image into a number of non-overlapping blocks and use traditional gradient-basedtechnique [170] to estimate the local orientations. Once local orientations are avail-able, they estimate local ridge-line frequencies at the same location. They also find thefingerprint core points in order to correctly align the fingerprint [122, 172] using localorientations. Then both the orientation and frequency images are aligned with respect

23

2. Related Work

to each core point and are downsampled. They compute the downsampled local orienta-tions, corresponding strength of the orientations and downsampled local frequencies asvector features for indexing. In addition to these vector features, two additional scalarfeatures are used. These two scalar values are the average frequency and the averagevertical orientation difference of the non-downsampled orientation and frequency images.To compare two fingerprints, they calculate four score values. First score compares localorientations and strength, second score compares local frequencies, third score comparesaverage frequencies and fourth score compares average vertical orientation differences oftwo fingerprints. The weighted combination of these four scores is used to retrieve thecandidate set. At the time of retrieval, the combined scores are calculated between querytemplate and all stored templates. The candidates similar to the query fingerprint areretrieved from the database by putting a threshold on the combined score.

2.2.3 Other Feature-Based Indexing Techniques

Other than the above two categories, some approaches use different features for finger-print indexing. For example, Li et al. [131] use three symmetric filters to measure dif-ferent orientation structures of a fingerprint. Based on three symmetrical measurementsthey propose a fingerprint indexing approach. They design three kinds of symmetricalfilters, which are the core type filter, the delta type filter and the parallel type filter, onthe orientation image to map the different structures of fingerprints into three differentfeature spaces. They obtain three index vectors from these feature vectors. These vectorsare reduced to a fixed length and rotated to set the direction of the cores as a verticalline. Authors generate three lists of candidates at the time of querying. The Euclideandistances between the index vectors of the query template and all stored templates areused to measure the similarity. Finally, they combine these three resulting lists usinghighest rank method to generate the final candidate set.

In another work, Gyaourova and Ross [88] develop a method to index and retrievefingerprint images by utilizing match scores. Their method relies on comparing a finger-print with a small set of reference images. First, they select a set of reference subjectsand compute match scores against each reference subject. Then discretization functionis applied on the match scores to convert them to a discrete domain. These discretescore values are used to generate an index key for a fingerprint. At the time of retrieval,they compute Hamming distance between query index key and all stored index keys. Acandidate is retrieved by applying a threshold on the calculated Hamming distance.

We summarize the existing fingerprint data indexing techniques in Table 2.2.

24


Table 2.2: Summary of different fingerprint data indexing techniques

Method Feature used DimensionIndexingtechnique

Min

utia

e-ba

sed

inde

xing

Germain et al.[82]

Length of each side, ridge countsbetween each pair of sides and anglesof minutiae triplets

9 FLASH

Bebis et al. [38]

Ratio of the largest side withminimum and median sides and cosineangle between the two smallest sidesof the Delaunay triangles

3 FLASH

Bhanu and Tan[43]

Angles, handedness, type of minutiae,direction and length of the maximumside of all minutiae triplets

6Continuousclassification

Liang et al. [132]

Orientation, minimum and medianangles, length of the longest edge,type of minutiae of low-orderDelaunay triangles

>6Continuousclassification

Ross andMukherjee [175]

Geometry of the Delaunay triangleand shape of ridges associated withthe minutiae points of the triangle

9k-meansclustering

Zhang et al. [196]Direction, coherence, curvature andorientation vector of minutiae point


Cappelli et al.[54]

MCC of neighborhood structure ofminutiae points

>15 LSH

Rid

geor

ient

atio

n-ba

sed

inde

xing

Lumini et al.[143]

Reduced dimensional directionalimage


Cappelli et al.[55]

Average ridge directions and dynamicmasks

>30Continuousclassification

Lee et al. [129]Feature map of local orientations andinter-ridge spacing


Jiang et al. [114]Dominant ridge distance and localridge orientations

193

Continuousclassificationand k-meanclustering

Liu et al. [136] Gabor response at singular points 338Continuousclassification

Liu et al. [137]Average ridge distance and localorientations

157k-meansclustering

Continued to next page

25

2. Related Work

Table 2.2 – continued from previous page

Method Feature used DimensionIndexingtechnique

Cappelli [51]Downsampled local ridge orientationsand frequencies


Oth

ers

Li et al. [131]Core orientation pattern, deltaorientation pattern and parallelorientation pattern


Gyaourova andRoss [88]

Variance of query image and referencescore


2.3 Survey on Face Biometric Data Indexing

In last few decades, extensive studies have been done in the field of face recognitionalgorithms. Several feature extraction methods are known for face-based authenticationsystem. Nevertheless, retrieval of face biometric data with a large number of featuresstored in a huge pool of memory are scarcely reported. In the existing literature, veryfew work have been reported for face indexing to reduce the search space. We describethese techniques [123,133,159] in the following.

Lin et al. [133] propose an indexing structure to search the face from a large database.They compute a set of Eigenfaces based on the faces in the database. Then, they assigna rank to each face in the database according to its projection onto each of the Eigen-face [181]. Similarly, they compute the Eigenfaces for a query and rank a query face.Finally, they select a set of faces from the database corresponding to the nearest facesin the ranked position with respect to each Eigenface of the query face. These selectedfaces are used for recognition.

A linear subspace approximation method for face indexing has been developed byMohanty et al. [159]. They build a linear model to create a subspace-based on thematch scores. A linear transformation is applied to project face images into the linearsubspace. For this purpose, first, they apply a rigid transformation obtained throughprincipal component analysis and then a non-rigid affine transformation. An iterativestress minimization algorithm is used to obtain a distance matrix in a low-dimensionalspace and propose a linear out-of-sample projection scheme for test images. Any newface image is projected into this embedded space using an affine transformation.

Kaushik et al. [123] introduce a modified geometric hashing technique to index theface database. They extract features from a face image using SURF [36, 37] operator.

26

2.4. Survey on Multimodal Biometric Data Indexing

They apply mean centering, principal component analysis, rotation and normalization topreprocess the SURF features. Finally, they use geometric hashing to hash these featuresto index each facial image in the database.

Other than the above indexing techniques, several dimensionality reduction tech-niques [142, 142, 158, 159, 181, 194, 195, 197] have been proposed for face identificationsystem. But these techniques are not utilizing any indexing technique. Hence, we skipthe discussion of these techniques. We have given a summary of the above mentionedface biometric based indexing techniques in Table 2.3.

2.4 Survey on Multimodal Biometric Data Indexing

Multimodal biometric data indexing is newly explored area in the field of biometricidentification. Very few work have been reported in the literature. We give an overviewof these techniques in the following.

Jayaraman et al. [109, 111] propose an indexing technique for multimodal biometricsystem using iris, ear, face and signature. They used kd-tree to store the feature vectorof these traits. Before storing the data into kd-tree they normalized the features of eachbiometric trait and projected to a lower dimensional feature space. They applied PCAfor dimensionality reduction. In this technique, more than 100 dimension feature vectorsare used.

Cascading technique for multimodal biometric system has been developed by Honget. al. [95]. They use face and fingerprint in their multimodal system. In this technique,exhaustive searching is performed with a single biometric trait to reduce the search spaceand final matching is performed with multiple traits into the reduced search space.

Gyaourova and Ross [89, 90] developed an indexing technique to reduce the searchspace for multimodal biometric system. They select a fixed set of images of each biometrictrait from the database. They compute scores between an image and the fixed set ofimages for each biometric trait. Based on these scores they generate a fixed-length index

Table 2.3: Summary of different face data indexing techniques

Method Feature used Dimension Indexing technique

Lin et al. [133] PCA and Eigenface >83 Condenseddatabase

Mohanty et al. [159] Subspace approximation on pair-wisedistances of training face data >125 k-nearest neighbor

search

Kaushik et al. [123] Speeded-Up Robust Features (SURF) >128 Geometric hashing

27

2. Related Work

codes for each biometric trait and store these index code into the database. To retrieve aset of candidates, they compute similarity between the probe index code and all enrolledindex codes and put a threshold to select the candidates. They propose two fusiontechniques which use the information from multiple modalities. In one fusion technique,they concatenate the index codes of different modalities and retrieve the candidate set. Inanother technique [89,90], they retrieve the candidate set corresponding to the index codeof each modality and then take the union of all retrieved candidate set. We summarizethe above mentioned multimodal biometric indexing techniques in Table 2.4.

2.5 Summary

In this chapter, we present a survey on biometric data indexing techniques with iris,fingerprint, face and multimodal traits. From the survey of iris data indexing, we canobserve that the existing indexing techniques use either texture or color features for in-dexing. The texture feature-based techniques use high dimensions of features whereasthe color features are with very low dimension. Further, the iris color is not stable in alltimes. Several indexing techniques are reported for fingerprint data indexing. However,the majority of these approaches use continuous classification scheme. In continuous clas-sification, the fingerprint search is performed by either comparing the query fingerprintwith all database templates based on index key values or comparing the query finger-print with the templates until the match value satisfies some criteria like less than somethreshold value. Although, the comparison between the query fingerprint and templateis much faster than the fine matching. Moreover, the continuous classification only ranksthe database templates according to their similarities to the query fingerprint. Some

Table 2.4: Summary of different multimodal biometric data indexing techniques

Method Feature used Dimension Modality Fusion Indexingtechnique

Jayaramanet al. [111]

Harr features, PCA toreduce the dimensionality >100 Iris, ear, face

and signatureFL and

SL Kd-tree

Hong andJain [95]

PCA on Eigenface forretrieval. Minutiaefeatures of fingerprint formatching

>100 Face andfingerprint DL Cascading

technique

Gyaourovaand Ross [90]

Index code based oncombined score value 256 Face and

fingerprint SL Linearsearch

FL → Feature level fusion; SL → Score level fusion, DL → Decision level fusion

28

2.5. Summary

techniques use a large number of low dimensional index key for searching. Face-basedindexing techniques are not well studied in the literature. However, there are severalmethods for dimensionality reduction of face features are available in the literature. Butthese methods are suitable when the database size is very large. Further, reported workfor multimodal biometric data indexing deal with the high dimensional feature vectors ofdifferent unimodal traits. The dimensionality of the feature vector affects the searchingtime. In addition to this, cascading technique uses single trait for searching the databasewhich does not take the advantages of other biometric traits at the time of searching.Finally, the multimodal data indexing approaches use different level of fusions which, infact, affect the performance of a multimodal biometric identification system.

29

Chapter 3

Iris Biometric Data Indexing

Iris biometric-based identification system deals with high dimensional complex features.Hence, the searching with these features makes the identification process extremely slowas well as increases the false acceptance rate beyond an acceptable range. To improve theefficiency of an iris-based identification system, we propose an indexing mechanism. Inthis chapter, we describe the proposed indexing technique with low dimensional featurevector for an iris-based biometric identification system. Our propose method is based onthe iris texture features. We compute Gabor energy features from iris texture and createa twelve dimensional index key vector. A new storing structure and retrieving techniquehave been proposed to store and retrieve the iris data from the database. The blockdiagram of our proposed iris biometric data indexing method is given in Fig. 3.1. Thedifferent tasks involved in our approach are discussed in this chapter.

The rest of the chapter is organized as follows. An overview of Gabor filter is discussedin Section 3.1. In Section 3.2, we talk about the preprocessing technique for iris image.Section 3.3 describes the feature extraction technique for iris biometric. We introduce

)��(��

��

��

��

��

� � � ��

*��

� ��

+��

��

��

$��,��

��(��

��

��

�� )��

.��

+��,��

Figure 3.1: An overview of our proposed iris biometric data indexing approach.

31

3. Iris Biometric Data Indexing

the index key generation method for iris biometric data in Section 3.4. The storing andretrieving techniques of iris data are given in Section 3.5 and Section 3.6, respectively.We present the performance evaluation of the proposed method in Section 3.7. Thecomparison with existing work is reported in Section 3.8. Finally, Section 3.9 gives thesummary of this chapter.

3.1 Preliminaries of Gabor Filter

We apply a Gabor filter in our work. For better understanding of our work, we brieflydiscuss the concept of the Gabor filter in this section.

Gabor transform theory was first proposed by D. Gabor in 1946 [80]. Daugman [64]

proposed two-dimensional (2-D) Gabor transform theory in 1985. It has been observedthat the 2-D Gabor filter is an effective method for time-frequency analysis [64,69]. In thespatial domain, a 2-D Gabor filter consists of a sinusoidal wave modulated by a Gaussianenvelope. It performs a localized and oriented frequency analysis of a 2-D signal.

A 2-D Gabor function g(x, y) and its Fourier transform G(u, v) can be expressed asin Eq. (3.1) and Eq. (3.2) [92,149], respectively.

g(x, y) =

(1

2πσxσy

)exp

[−1

2

(x2

σ2x

+y2

σ2y

)+ 2πjWx

](3.1)

G(u, v) = exp

{−1

2

[(u−W )2

σ2u

+v2

σ2v

]}(3.2)

where, σu = 1/2πσx and σv = 1/2πσy. In Eq. (3.1), σx and σy are the standarddeviations of the Gaussian envelope along x and y directions, respectively and determinethe filter bandwidth. Here, (x, y) and (u, v) are the spatial domain and frequency domaincoordinates, respectively. W is the center frequency of the filter in frequency domain andj is called the imaginary unit, where j =

√−1.A multi-resolution Gabor filter (also called multi-channel Gabor wavelet) is a set of

filter banks with different scales (frequencies) and orientations. The Gabor filter formsa complete but nonorthogonal basis set. Expanding a signal using this basis providesa localized frequency description, thus capturing local features or energy of the signal.Texture features can then be extracted from this group of energy distributions. TheGabor wavelet can be represented with a mother wavelet and derived the appropriateGabor functions by different rotations and scales. Let g(x, y) be the mother wavelet ofthe Gabor filter family. The set of Gabor filter bank gm,n(x, y) can be generated fromg(x, y) as expressed in Eq. (3.3).

32

3.1. Preliminaries of Gabor Filter

gm,n(x, y) = a−2mg(x′, y′) (3.3)

where a is a constant such that a > 1, m,n are two integer variables, and x′ and y′ arerepresented in Eq. (3.4).

x′ = a−m(x cos θn + y sin θn) (3.4)

y′ = a−m(−x sin θn + y cos θn)

In Eq. (3.4), θn = nπ/K, m = 0, 1, . . . , S − 1, and n = 0, 1, . . . ,K − 1. The param-eters S and K are the total number of scales and orientations in the multi-resolutiondecomposition, respectively. The values of S and K are to be chosen to ensure that thehalf-peak magnitude support of the filter responses in the frequency spectrum touch eachother. The filter parameters a, σu and σv can be computed as follows:

a =

(Uh

Ul

) 1S−1

(3.5)

σu =(a− 1)Uh

(a+ 1)√2 ln 2

and σv = tan(π

2K)

[U2h

2 ln 2− σ2

u

]

where Ul and Uh denote the lower and upper center frequencies of interest, respectively.

The Gabor feature space consists of responses calculated with a multi-resolutionGabor filter at several different scales (frequencies) and orientations. The response ofa Gabor filter to an image is obtained by a 2-D convolution. Convolution of an imagewith the kernel gives a response that is proportional to how well the local features inthe image match the kernel. Let I(x, y) denotes an image and GIm,n(x, y) denotes theresponse of a Gabor filter at the mth scale and nth orientation to an image point (x, y).The Gabor filter response to an image is defined in Eq. (3.6).

GIm,n(x, y) =∑x1

∑y1

I(x1, y1)gm,n(x− x1, y − y1) (3.6)

The Gabor filtered image has both real and imaginary components, as the responseof the Gabor filter is complex. The magnitude of the Gabor filtered image is calculatedby using Eq. (3.7).

33


|GIm,n(x, y)| =√

ReGIm,n(x, y)2 + ImGIm,n(x, y)

2 (3.7)

where ReGIm,n(x, y) and ImGIm,n(x, y) are the real and imaginary parts of the Gaborfiltered image, respectively.

In this work, we use multi-resolution Gabor filter to extract features from iris images.

3.2 Preprocessing

We use iris localization and normalization as the preprocessing tasks for a captured eyeimage. First, in localization, the iris part is isolated from an eye image. To localize theiris part, we preprocess the eye image by applying downscaling and color level transform.The downscaling of the eye image is done to reduce the search area for pupil and irisboundaries. The original eye image is with the resolution 320×280 and the eye imageis downscaled to the resolution 160×140. Figure 3.2(a) shows a sample eye image. Thecolor level transform is applied to minimize the influence of irrelevant edges. In otherwords, the color level transform increases the intensity differences between the pupiland iris parts, and iris and sclera parts which in turn helps to detect the pupil and irisboundaries efficiently and accurately [71,72]. The next step is to find the pupil boundary.To detect the pupil boundary, we convert the color level transformed image to a binaryimage and find all connected components in the binary image. Then we remove thesmall irrelevant components which may occur due to eyelashes, eyelids and noise. Pupilcomponent is selected by counting the number of pixels within the calculated averageradius. The pupil boundary is found from the pupil component by edge detection andedge connection. The pupil centroid is determined by calculating the centroid of all pixelswithin the pupil boundary. Figure 3.2(b) shows an eye image with detected pupil.

Iris boundary detection is the next step in the iris localization. To detect the irisboundary, the eye image is divided into left and right images at the pupil centroid. Colorlevel transform is applied on both left and right images to enhance the contrast betweeniris and sclera boundaries. The image preprocessing technique, namely dilation [84], isfollowed to color level transformed image to reduce the noise effect in edge detection.The dilated image is then thresholded to create a binary image and to detect the verticaledges which mainly occur due to iris boundary. Irrelevant edges which occur due tonoise and eyelashes are removed after the edge detection. Iris boundary and eyelidboundary are detected by checking the pixel connectivity and drawing the small linesin particular directions. Figure 3.2(c) shows detected iris boundary in the eye image.Finally, resizing of pupil and iris boundary information is done. The detected iris part

34

3.3. Feature Extraction

(a) Original eye image (b) Pupil boundary de-tected eye image

(c) Iris boundary de-tected eye image

(d) Localized iris image (e) Normalized iris image (f) Enhanced iris image

Figure 3.2: Preprocessing result of a sample iris image [71,73].

is shown in Fig. 3.2(d). Detailed of the pupil and iris boundary detection have beenreported in [71,72].

After completion of iris localization, the iris part is wrapped into a fixed size rectan-gular block to make the iris sample scale and translation invariant. This process is callediris normalization. A detailed about iris image normalization technique can be foundin [73]. The wrapping process is done by transforming the iris texture from Cartesianto polar coordinate. The Daugman’s homogeneous rubber sheet model [70] is applied tonormalize the iris texture. The normalized iris image is shown in Fig. 3.2(e). Then thenormalized iris is enhanced to make the iris texture illumination invariant by applyingMa et al. [144] technique. This enhanced image is used in the feature extraction. Anexample of an enhanced normalized iris image is shown in Fig. 3.2(f).

3.3 Feature Extraction

We extract Gabor energy features in different scales and orientations from a normalizediris image as stated bellow. We know that the response of a Gabor filter to an image

35


is obtained by a 2-D convolution operation. Let I(x, y) be the normalized iris imageand GIm,n(x, y) denotes the response of Gabor filter in the mth scale and nth orientationto an image at point (x, y) on the image plane. The Gabor filtered image can then beobtained using Eq. (3.6) as stated in Section 3.1.

We apply the multi-resolution Gabor filter to extract the iris texture features. Forthis purpose, we apply the Gabor filter in each scale and orientation, and compute theresponse at each position of the image. These responses are called Gabor coefficientvalues which are complex. We calculate the magnitude values of the responded imageat each pixel. The Gabor energy is estimated by summing up the square values of themagnitude of Gabor responses at each pixel as expressed in Eq. (3.8).

GEm,n =∑x,y

[|GIm,n(x, y)|]2 (3.8)

It is observed that the range of values of Gabor energy features is very high. It isalso examined that intra-feature range distribution is higher for lower values of m and n.To make the intra-feature range distribution similar, we have studied that a non-linearfunction such as log can serve the purpose, that is, it effectively maps the high range ofGabor energy features’ values to a lower range of values. Thus, for a given iris image werepresent the Gabor energy features in the form of a matrix as shown in Eq. (3.9).

GF =

⎡⎢⎢⎢⎢⎣

log(GE0,0) log(GE0,1) . . . log(GE0,K−1)log(GE1,0) log(GE1,1) . . . log(GE1,K−1)

...... . . .

...log(GES−1,0) log(GES−1,1) . . . log(GES−1,K−1)

⎤⎥⎥⎥⎥⎦S×K

(3.9)

The rows and columns denote the number of scales (S) and number of orientations(K) of the multi-resolution Gabor filter. It may be noted that the feature vector in ourapproach consists of S ×K number of features.

3.4 Index Key Generation

The extracted Gabor energy features in different scales and orientations are used togenerate a key for iris database indexing. In our approach, a key represents a featurevector which consists of S×K number of features’ values. We denote the ith value of anindex key as f(i) in Eq. (3.10).

f(i) = GFm,n (3.10)

36

3.5. Storing

where GFm,n represents the logarithm value of the Gabor energy feature in the mth scaleand nth orientation. In Eq. (3.10), m = 0, 1, . . . , S − 1 and n = 0, 1, . . . ,K − 1, andi = n + (m × K) + 1. In other words, the ith feature of an index key represents thelogarithmic value of a Gabor energy feature in a particular scale and orientation. Notethat the length of the index key is S×K. Let us denote this length as FL. Suppose, thereare P number of subjects and for each subject there are Q number of samples. The indexkey for the qth sample of the pth subject is denoted by Eq. (3.11) where fp,q(i) representsthe ith feature of the qth sample of the pth subject, q = 1, 2, . . . , Q and p = 1, 2, . . . , P .

indxp,q =< fp,q(1) fp,q(2) · · · fp,q(i) · · · fp,q(FL) > (3.11)

3.5 Storing

To store the iris data, we first create an index space in the database, and then we storethe iris data into that index space of the database. The descriptions of creating indexspace and storing mechanism are discussed in the following.

3.5.1 Index Space Creation

For a given iris image, we create its index key as mentioned in Section 3.4. To storeany index key for any sample we propose an index organization as shown in Fig. 3.3.The organization consists of FL number of tables. Each table is corresponding to eachfeature of the index key. The length of a table for a feature depends on the minimumand maximum values for all samples of all subjects. Thus, the length of the table for theith feature is calculated using Eq. (3.12).

TLi = Fmaxi − Fmini + 1 (3.12)

where Fmaxi and Fmini are the maximum and minimum values for all samples of allsubjects of the ith feature, and i varies from 1 to FL. The Fmaxi and Fmini are thencalculated using Eq. (3.13). The index value of the table corresponding to the ith featureis started with 1 and end with TLi.

Fmaxi = max(fp,q(i)) ∀ q ∈ Q and ∀ p ∈ P (3.13)

Fmini = min(fp,q(i)) ∀ q ∈ Q and ∀ p ∈ P

Each entry of a table contains two lists (see Fig. 3.3). The list MDL contains a

37


� �

� �

��/

��/"

��/

��/0

��/

��/*/"

* ��"

� �

��/

��/"

��/

��/0

��/

��/*/�

* ��

� �

��/

��/"

��/

��/0

��/

��/*/�/

* ��/

Figure 3.3: Index space organization.

set of values which we call the median values of all samples of a subject for a featurerepresenting the table. The list IDL contains a set of identities which uniquely identifyan iris sample.

3.5.2 Storing Iris Data

To store the index keys in the database, we sort the values of a feature corresponding toall given samples of a subject. Then we find the center value of the feature values forthat subject. Note that median or mean value can give us the center value. The medianvalue gives the central position which minimizes the average of the absolute deviations.On the other hand, the mean value gives the center position biasing toward the extremevalue. For example, if any feature value is very low or very high for a particular sample,then the mean value is close to that feature value of the sample. Hence, we use themedian value in our approach. For any ith feature, we compute the median value givenQ number of samples for a subject say p using Eq. (3.14).

pMedi = Median(fp,q(i)) ∀ q ∈ Q (3.14)

Note that if there are P number of subjects and each subject has Q number ofsamples, then we store N = P ×Q number of median values and identities in each table.

Let us denote the unique identifier for the qth sample of the pth subject as Idpq and theith feature value is denoted as fp,q(i). We store the median value and Idpq correspondingto the fp,q(i) at the ti

th position in the ith table. Given the value of fp,q(i), the value ofindex say ti, in the ith table is calculated using Eq. (3.15).

ti = fp,q(i)− Fmini + 1 (3.15)

38

3.5. Storing

To enroll an additional samples of a subject which is already enrolled in the database,we recalculate the median value of all enrolled samples with the additional samples forthat subject. Then, we enroll the all samples of the subject into the database. If wewant to enroll a new subject, first we calculate the median value of each feature for allsamples of the new subject. Then we check the minimum and maximum values of eachfeature whether the values are within the range of the table or not. If feature values liewithin the range of the table then we enroll the identity and the median value of the newsubject. If the values do not fall within the range, then we increase the size of table withthe new minimum and maximum values and reorganize the enrolled samples. Finally, weenroll all samples of the new subject.

Illustration: We illustrate our approach to store and index iris biometric data with anexample. Suppose, we want to store the data of 10 subjects and each subject has 5samples. Thus, the total number of samples to be stored and indexed is 50. Further, weconsider 3 scales and 4 orientations for Gabor feature extraction. Hence, the length ofthe index key is 12. That is, we need 12 tables in the database to store the 12 featuresfor each index key. The index keys for 50 samples are created as discussed in Section 3.4.Figure 3.4 shows the index keys of 5 samples and all data pertaining to say, sixth subjectfor an instance. Let us assume that the minimum and maximum values of the firstfeature analyzing all fifty samples are found to be 100 and 350, respectively. Thus, thelength of the first table in the database is 251 (Fmax1 −Fmin1 +1). Similarly, the lengthof the other tables in the database can be calculated from the maximum and minimumvalues of features. Now, we want to enroll the first sample of the sixth subject into thedatabase. Among all the index keys of the sixth subject, the first sample is highlightedas shown in Fig. 3.4. Now, we are to find the median value for each feature of the sixthsubject. The values of the first feature of the sixth subject are shown as a rectangularblock in Fig 3.4. From Fig. 3.4, we can see that the median value of the first feature ofthe sixth subject is 318. The first feature value of the first sample of the sixth subject is316. We store the median value (318) of the sixth subject in the MDL and the identityof the first sample of the sixth subject (Id61) in the IDL at the 217th (316 - 100 + 1 )

1 1

" "

1 1

0 0

1 1

2 2

1

3

4"5 405 425 435 465 415 475 485 495 4":5 4""5 4"05

�� 2"1 091 086 072 068 037 02" 0"2 0:6 "87 "11 "67 �

�� 2"8 097 082 070 066 036 009 0"3 "99 "88 "19 "66 �

�� 2"7 091 086 076 067 031 020 0"" 0:0 "86 "7: "66 �

�� 2"8

� � � � � � � � � � � � ��

1

3

1 1

6 6

097 08" 072 063 036 02" 0"" 0:0 "83 "19 "61 �

�� 2"9 098 08: 073 066 031 022 0": 0:2 "86 "19 "63 �

Figure 3.4: Index keys of the 6th subject with 5 samples.

39


location of the first table of the database, respectively. We store other features of thefirst sample of the sixth subject likewise. We can enroll the rest of the samples in thesame way.

Algorithm 3.1 formally states the techniques of creating tables in the database. Thedifferent steps of enrollment of all samples for all subjects are summarized in Algo-rithm 3.2.

We use the following notations in our algorithms.

Notations used in our algorithmsP = Number of subjectsQ = Number of samples of each subjectFL = Number of features in an index keyfp,q(i) is the ith feature of the qth sample of the pth subjectIdpq is the identity of qth sample of the pth subjectMDL is the list of median valuesIDL is the list of iris sample identitiesFmini is the minimum value of the ith

Fmaxi is the maximum value of the ith

Tablei is the table for the ith featureSi is the set for retrieved iris template corresponding to the ith featureCSET is the candidate set

Algorithm 3.1 Creating tables for indexingInput: All index keys for all person’sOutput: Tables in index space1: for i = 1 to FL do2: for all p = 1, 2, . . . , P do3: for all q = 1, 2, . . . , Q do4: Fmaxi = max(fp,q(i)) // Finding the maximum of all samples of all subjects5: Fmini = min(fp,q(i)) // Finding the minimum of all samples of all subjects6: end for7: end for8: TLi = Fmaxi − Fmini + 1 // Calculate the length of the table for each feature9: Create Tablei of length TLi // Create the table for each feature

10: end for

40

3.6. Retrieving

Algorithm 3.2 Enrollment all samples for all subjects into the databaseInput: All index keys for all subjects with identitiesOutput: Tables in index space with enrolled samples1: for p = 1 to P do2: for i = 1 to FL do3: pFmedi = Median(fp,q(i)) ∀ q ∈ Q4: end for

// Store feature vector into database5: for all q = 1, 2, . . . , Q do6: for i = 1 to FL do7: ti = fp,q(i)− Fmini + 1 // Calculate the index for feature value fp,q(i)

8: Add pFmedi to Tablei[ti].MDL9: Add Idpq to Tablei[ri].IDL

10: end for11: end for12: end for

3.6 Retrieving

Once all subjects are enrolled into the database, we can use it to retrieve data for amatch. As a task of retrieving, we are to find the iris templates from the database whichare the most similar to the query iris template. To do this, first we generate the indexkey of length FL from the query iris as discussed in Section 3.4. The index of the queryiris is represented in Eq. (3.16).

indxt =< ft(1) ft(2) · · · ft(3) · · · ft(FL) > (3.16)

Note that, we have only a set of feature values for the given query iris image. Thereis no median value as we do not know the median value a prior. At the time of retrieval,we retrieve a set of median values and sample identities corresponding to each featurevalue. For the ith feature (with feature value say ft(i)) of the query index key, we retrievea particular position of the ith table in the database based on the value of ft(i). Theposition of the ith table is decided by Eq. (3.17).

TabIndexi = ft(i)− Fmini + 1 (3.17)

We retrieve a set of identities (IDL) at TabIndexi location of the ith table. Letthis set be Si. We also find the minimum and maximum median values (Medmax andMedmin) from the list MDL at TabIndexi location of the ith table. Next, we retrieve theIDL sets corresponding to the values Medmin − δ to Medmax + δ from Tablei and add

41


these identities to Si. Here, δ is a threshold value which would be decided empirically(see Section 3.7.4.2). In this way, we retrieve the candidate identities for the ith featureand store them in a temporary list Si. Similarly, we retrieve the identities for otherfeatures and store them in respective temporary lists.

In our next step, we find the most common identities among all sets Si’s and computeranking for each identity. First, we merge all Si’s to S, that is, S =

∑Si. To count

the number of occurrences of each subject appeared in S, we maintain a candidate set(CSET ) which contains to fields, namely id and vote. The id filed stores the uniquesubjects’ identities in S and vote filed stores the number of occurrences of that identitiesin S. We increase the vote corresponding to an identity when it occurs in the set S.Then we sort the CSET in descending order based on the values of vote field. We assignrank one to the subject correspond to the highest value of the vote, rank two to the nexthighest value of the vote and so on. It means that the maximum occurrence is assignedas rank one and other ranks are given in descending order of number of occurrences. Thehighest rank indicates that the retrieved identities is most similar to the query image.

Illustration: We illustrate how we can retrieve a match for a given query iris image. Weconsider that the index keys of 10 subjects and 5 samples for each subject are storedin our database. Thus, there are 50 index keys in the said database. Let us assumethe minimum and maximum values of the 1st feature (with respect to these 50 indexkeys) are 100 and 350, respectively. The length of the 1st table in the database is 251(Fmax1 − Fmin1 + 1). Figure 3.5(a) shows the 1st table for the 1st feature. Now, weconsider the index key indxt for a query iris. We generate the index key from the querytemplate. In this example, let us consider 3 scales and 4 orientations for Gabor featureextraction as in the storing time. Hence, the length of the index key is 12. The indexkey generated from the query iris template of length 12 is shown in Fig. 3.5(b).

Here, the 1st feature value (ft(1)) of the query iris template is 317 (see Fig. 3.5(b)).Knowing this, we retrieve all median values in the MDL at location 218 (317− 100+ 1)of the 1st table. We see that the median values in the MDL at the 218th location ofTable1’s table are 315 and 318 (see Fig. 3.5(a)). The minimum and maximum medianvalues in the MDL are 315 and 318, respectively. We also retrieve all identities in theIDL at location 218 (317−100+1) to S1. Next, we add the identities in the IDL from arange of locations based on the minimum and maximum median values and preassignedthreshold value δ. Let the value of δ is chosen as 1. Therefore, the range of locationsis 215 (315 − 100 − 1 + 1) to 220 (318 − 100 + 1 + 1). We add all identities in IDL atlocations 215 to 220 into S1. The set S1 is shown in Fig. 3.5(c).

Similarly, we create sets for the rest of the features. We avoid the showing of identities’

42

3.6. Retrieving

��/

��/

;2"6<�2"8=2 1

2 2;� <�� =

2 1

" ";� <� =

0

3;� =

" 1 1

0 0 3;� <�� <�� =

1

6;� =

;2""=

;2"6<�2"8=

;2"8<�2"9=

;2"8=

* ��"" 0 2 3 6 1 7 8 9 ": "" "0

>

� � � � � � � � � � � �

�� 2"7 096 083 073 067 036 020 0"0 0:0 "83 "19 "63

" " 0 0 2 1 1 1 1

" 0 2 3 6 2 " 0 3 6� ?;� <�� <�� <�� <�� <�� <�� <�� <�� =

�� "��

�� " ��"��

�� (��,��@��

0"6

0"1

0"7

0"8

0"9

00:

00"

06"

"

��/

��/

��/

��/

��/

��/

��/

��/

��/

��/

��/

��/

��/

��/

��/

��/

;2""=

;2"9<�202=

0

6;� =

" 6

2 ";� <� =

� �

� �

�� %��A�� (�

" " 2 3 1 1 7

0 2 6 3 6 " 2 2

" 3 3 1 1 1 7 7

2 3 2 3 " 0 2 0 3

" 2 2 2 1 1 1 1 7

3 0 2 3 6 " 2 3 6 3

0 3 3 1 1 1 1

6 6 2 3 0 2 3 6

� ?;� <�� <�� <�� <�� <�� <�� =

� ?;� <�� <�� <�� <�� <�� <�� <�� =

� ?;� <�� <�� <�� <�� <�� <�� <�� <�� =

� ?;� <�� <�� <�� <�� <�� <�� <�� 8

3

" " " 0 2 1 1 1 1 1 8 8

1 " 2 3 6 3 " 0 2 3 6 " 6

" " " 0 0 1 1 1 7

7 0 2 6 0 3 " 2 3 3

" 0 2 1 1 8

8 2 6 2 2 3 0

0 0 1

9 2 6 2 3

=

� ?;� <�� <�� <�� <�� <�� <�� <�� <�� <�� <�� <�� =

� ?;� <�� <�� <�� <�� <�� <�� <�� <�� =

� ?;� <�� <�� <�� <�� <�� =

� ?;� <�� <�� <��1 1

6

2 2 2 1 1 1 8 8

": 0 2 6 2 3 6 " 2

" 1 1 1 1 8 8

"" 6 0 2 3 6 2 6

2 2 1 1 1 1 7 8 8

"0 2 6 0 2 3 6 3 0 3

<�� =

� ?;� <�� <�� <�� <�� <�� <�� <�� =

� ?;� <�� <�� <�� <�� <�� <�� =

� ?;� <�� <�� <�� <�� <�� <�� <�� <�� =

�� C�� %��.*

�� ,��

�" "3

�0 9

�2

"0

�3

6

�1

3"

�7 1

�8 ":

� ,��

�1 "

�" 0

�2

2

�8

3

�0

6

�7 1

�3 7

� � �(

Figure 3.5: Retrieving a match for a query iris image from a database.

position in the other tables due to clarity of the figure and the limitation of space in thepaper. For continuity of our illustration, we assume the other eleven sets of retrievedidentities, as shown in Figure 3.5(d). We merge all such sets to S. Finally, we calculatethe number of occurrences of particular Idp in S and calculate the rank. To do this, weinsert each unique subjects of S into the CSET . We increment the vote corresponding toa subject when it occurs in the set S. Figure 3.5(e) shows the CSET which consists thevote values for each subject identity. Figure 3.5(f) shows the subjects with correspondingrank and the sixth subject being the rank 1 will be retrieved as the best match. Thesteps for retrieving iris template from a database are summarized in Algorithm 3.3.

43


Algorithm 3.3 Retrieving iris template from databaseInput: Index key of query iris (indxt)Output: Retrieved identity set with their ranks1: for i = 1 to FL do2: TabIndexi = ft(i)− Fmini + 1 // Calculate the table location3: Si = Tablei[TabIndexi].IDL4: Medmaxi = max(Tablei[TabIndexi].MDL)− Fmini + 15: Medmini = min(Tablei[TabIndexi].MDL)− Fmini + 16: for k = Medmini − δ to Medmaxi + δ do

// δ = offset value7: Si = Si

⋃Tablei[TabIndexi].IDL // Add iris identities to Si

8: end for9: S = S + Si // Merge all set Si to S

10: end for11: Add unique subject identity in S into the id field of CSET12: Initialize all vote of the CSET to zero

// Calculate the number of occurrence of each subject13: for Each Idpq in S do14: CSET [Idp] → vote = CSET [Idp] → vote+ 115: end for16: Sort all CSET in descending order based on the values of vote17: Assign rank based on the value of vote

3.7 Performance Evaluation

To study the efficacy of the proposed iris biometric data indexing approach, we haveconducted a number of experiments. In this section, we present the experiments carriedout and the experimental results observed.

3.7.1 Performance Metrics

Accuracy and efficiency are the two main criteria usually considered to measure the per-formance of biometric indexing techniques. The accuracy of biometric indexing approachis commonly evaluated by the Hit Rate (HR) and Penetration Rate (PR) [43,54,90,169].The HR is the percentage of probes for which the correct identity is retrieved for a galleryby the indexing mechanism [90]. The PR is the percentage of the database retrieved fora query to get a correct match [90]. Let Ng be the number of entries in the databaseand Np be the number of query in the probe set. For a query sample if Ni is the numberof entries retrieved for ith probe then the PR is defined as in (3.18). If Nc (Nc < Np)is the number of query sample for which successful matches are found within the top r

44

3.7. Performance Evaluation

retrieved candidates then the HR at rank r is defined as in Eq. (3.19).

PR =1

Np

Np∑i=1

Ni

Ng(3.18)

HR =Nc

Np(3.19)

Another metric called Cumulative Match Score (CMS) is also used to measure theperformance of a biometric indexing technique. The CMS gives the probability of atleast one correct identity present within a top rank which also represents the cumulativeHR at different ranks. In other words, the HR at different ranks is represented withCumulative Match Characteristics (CMC) curve.

False Positive Identification Rate (FPIR) and False Negative Identification Rate(FNIR) is also used to check the accuracy of an identification system. To calculateFNIR and FPIR of an identification system [148] without indexing and with indexingwe follow Eq. (3.20) and Eq. (3.21), respectively.

FNIR = FNMR

FPIR = 1− (1− FMR)N(3.20)

FNIR = ER+ (1− ER)× FNMR

FPIR = 1− (1− FMR)N×PR(3.21)

Here, FMR and FNMR are the False Match Rate and False Non Match Rate ofa system, respectively. The FMR and FNMR are calculated from the genuine andimposter score distribution. In Eq. (3.20) and (3.21), N is the number of samples inthe database, PR is the penetration rate of the indexing system and ER is the indexingerror. We calculate ER using Eq. (3.22) where HR represents the hit rate of the indexingtechnique.

ER =100−HR

100(3.22)

3.7.2 Databases

In our experiment, we have used five different iris image databases: Bath University IrisDatabase (BATH) [2], CASIA-IrisV3-Interval Iris Database (CASIAV3I) [7], CASIA-IrisV4-Thousand Iris Database (CASIAV4T) [8], Multimedia University Iris DatabaseVersion-2 (MMU2) [19] and West Virginia University Iris Database Release 1 (WVU) [31].A detailed description of each database is given in the following.

45


Bath University Iris Database (BATH): The BATH database [2] contains 1000 eyeimages of 25 persons. Each person has 20 images of left eyes and 20 images of right eyes.These images are high resolution (1280×960 pixels), and have been compressed withJPEG-2000 to 0.5 bits/pixel. The database is created in conjunction with the Universityof Bath and Smart Sensors Limited. It may be noted that the iris data of left and righteyes of a person are different [65], we therefore consider 50 unique subjects in BATHdatabase.

CASIA-IrisV3-Interval Iris Database (CASIAV3I): CASIAV3I database [7] is col-lected by the Institute of Automation at the Chinese Academy of Sciences (CASIA).This database contains 2639 eye images of 249 persons. The eye images in the databaseare captured from 395 eyes. The iris images are captured with the close-up iris cameradeveloped by Center for Biometrics and Security Research (CBSR). The most of theimages are taken in two sessions, with at least one month interval in indoor environment.The resolution of image is 640×480 pixels. From the CASIAV3I database we consider395 unique subjects in CASIAV3I database as the two eyes are different for a person.

CASIA-IrisV4-Thousand Iris Database (CASIAV4T): This database is also de-veloped Institute of Automation at the Chinese Academy of Sciences [8]. The CASIAV4Tdatabase contains 200000 eye images of 1000 persons and each person has 10 left and 10right eye images. All iris images are 640×480 pixels resolution and collected under nearinfrared illumination using IKEMB-100 camera. We consider 2000 unique subjects forCASIAV4T database.

Multimedia University Iris Database Version-2 (MMU2): The MMU2 databaseis collected at Multimedia University [19]. It contains 995 eye images from 100 persons.Five images are captured from each eye of a person. There are 5 left eye images whichare excluded from the database due to cataract disease. The images are captured usingPanasonic BM-ET100US camera and with 320×238 pixels resolution. The total numberof unique subjects considered for this database is 199.

West Virginia University Iris Database (WVU): This database is created by Cen-ter for Identification Technology Research, West Virginia University [31]. The databasecontains 3099 eye images from 244 persons. Left and right eye images are captured from241 and 236 persons, respectively. In the WVU database, one to twenty samples are

46


captured for each eye. The images are captured using OKI IRISPASS-h handheld devicewith 480×640 pixels resolution. We consider 477 unique subjects from WVU databasein our experiment.

A summary of all the databases is given in Table 3.1.

3.7.3 Evaluation Setup

To evaluate the performance of the proposed approach, we divide all samples of all virtualusers into two sets: Gallery and Probe. The samples in the Gallery are enrolled into theindex database and samples in the Probe are used as query to search the index database.We use 80% samples of each subject to create the Gallery and other 20% to create theProbe. We select samples of the Gallery Set randomly from each subject.

We have done our experiments with an Intel Core2Duo processor (2.00 GHz) and2.0-GB memory. We use GNU Compiler Collection (GCC) 4.3 compiler to develop ourprogram.

3.7.4 Validation of the Parameter Values

Our experiments are involved with different parameters and resources. Eventually, thereported experimental results are subjected to the validity of the above resources andassumptions on values of parameters. In the following, we validate the values of differentparameters.

We assess validity [126] on the basis of experimental evaluation of the variables. Inour experiment, three parameters namely number of scales (S), number of orientations(K) and δ are considered. First, we validate the assumed values of number of scales andnumber of orientations of Gabor filter. Next, we validate the value of δ.

Table 3.1: Characteristics of databases used in our experiments

Database Size # Persons # Samples # Uniquesubjects

BATH 1000 25 20 50

CASIAV3I 2639 249 2 to 10 395

CASIAV4T 20000 1000 10 2000

MMU2 995 100 5 199

WVU 3099 244 1 to 20 477

47


3.7.4.1 Values of S and K

In our experiment, we are to decide the number of scales (S) and orientations (K) forfeature extraction that should give better results. It may be noted that if the numberof scales and number of orientations increase, then the number of features in index keyincreases. On the other hand, the memory requirement to store the index keys alsoincreases when the number of features in index key increases. Further, HR increases asthe number of features increases. Hence, there is a trade-off between the number of scalesand orientations (and hence memory overhead) with the HR. We have experimented withdifferent scales and orientations, and the result on the CASIAV3I database is shown inFig. 3.6. In our experiment, we consider 10 different scales and 10 different orientations.Figure 3.6(a) and (b) show the HR and PR for different combination of scales andorientations, respectively. We see that a higher number of scales and orientations givesthe good HR but this also gives high PR which is not desirable. On the other hand, smallvalues of number of scales and orientations give good PR but it give poor HR which isalso not desirable. From Fig. 3.6, we see that the HR does not change significantly whenwe increase the number of scales and orientations beyond 3 and 4, respectively. Again,from Eq. (3.23), we see that the memory requirement to store the indexed features isdirectly proportional to the number of features used in indexing. In other words, withlarge values of number of scales and orientations memory requirement increases greatlywithout much improvement in HR. With this observation we have chosen the value ofscale as 3 and orientation as 4. In fact, this combination gives a better HR and PR asshown in Fig. 3.6.

3.7.4.2 Value of δ

We calculate the HR and PR for different values of δ. Figure 3.7 shows the resultfor different values of δ. From Fig. 3.7(a), we observe that the HR does not increasesignificantly beyond the value of δ = 5. On the other hand, from Fig. 3.7(b), we observethat PR is changing rapidly after δ = 5. Considering this trend it is appropriate to setthe value of δ as 5.

3.7.5 Evaluation

We judge the efficiency of our proposed iris indexing technique with respect to accuracy,searching time and memory requirement. We evaluate the experiments with the galleryand probe sets created in Section 3.7.3. In our experiment, we perform the indexingusing iris.

48


:0

31

8":

:

0

3

1

8

":2:

3:

6:

1:

7:

8:

9:

"::

�(

D$

(a) HR with different scales and orientations

:0

31

8":

:

0

3

1

8

"::

":

0:

2:

3:

6:

1:

7:

�(

)$

(b) PR with different scales and orientations

Figure 3.6: HR and PR with different scales and orientations.

3.7.5.1 Accuracy

We analyze the best match for each sample in the Probe set at the first rank. We calculatethe rank one HR, PR and retrieving time. The results are summarized in Table 3.2.In Table 3.2, we see that the HR for WVU is considerably less because the quality ofthe iris images in WVU database are poor than the other databases. We may note thatthe average PR of BATH, CASIAV3I, CASIAV4T, MMU2 and WVU iris databases are11.6%, 15.3%, 16.9%, 14.3% and 10.8%, respectively. From Table 3.2, we observe thatthe retrieving time in CASIAV4T is slightly higher than the other databases though thesize of database is significantly larger than others.

49

3. Iris Biometric Data Indexing�

�

�

�

�

�

1:

7:

8:

9:

"::

: 0 3 1 8 ":

D$

J

(a) HR for different values of δ

�

�

�

�

�

�

:

":

0:

2:

3:

: 0 3 1 8 ":

)$

J

(b) PR for different values of δ

Figure 3.7: HR and PR for different values of δ.

Table 3.2: HR and PR for different iris image databases

Database HR PR Retrievingtime (ms)

BATH 82.38 11.6 0.013

CASIAV3I 82.38 15.3 0.016

CASIAV4T 84.64 16.9 0.018

MMU2 83.92 14.3 0.011

WVU 80.64 10.8 0.016

We also substantiate our result in terms of CMS which gives the probability of atleast one correct identity is present within a top rank. How CMS varies with rank isshown in Fig. 3.8 as CMC curve. It is obvious that CMS increases with the increase ofrank. From Fig. 3.8, we see that 99.4%, 98.8%, 98,7, 98.9% and 98.8% CMSs are possiblewithin top 60 ranks for BATH, CASIAV3I, CASIAV4T, MMU2 and WVU databases,respectively. In the existing literature on iris indexing, the best reported result [154] sofar is 98.56% CMS within top 100 rank for CASIAV3I iris database. In other words, ifwe consider top 100 rank in our approach then the CMS will eventually increase.

50


�

�

�

�

�

�

�

�

76

8:

86

9:

96

"::

" 0 2 3 6 ": "6 0: 2: 3: 6: 1:

��

�

$ �(

�&*D

�&��&C2�

�&��&C3*

��0

LC�

Figure 3.8: CMC curves with reference to different databases.

Further, we analyze the performance of the proposed method with respect to FPIR

and FNIR on CASIAV3I database. To do this, first we calculate FMR and FNMR

as follows. We match each query template of the Probe set with each template in theGallery set using Daugman’s [70] iris recognition method. We choose 3,000 genuine pairsand 1,242,608 imposter pairs from the Gallery and Probe of CASIAV3I database. Wecalculate the genuine score and imposter score using Daugman’s method [70] for eachgenuine and imposter pairs, respectively. Finally, we calculate FNIR and FPIR foran identification system [148] without indexing and with indexing using Eq. (3.20) and(3.21), respectively. The trade-off between FPIR and FNIR for the identification sys-tem without indexing and with indexing is shown in Fig. 3.9 for CASIAV3I iris database.From our experimental results, it may be interpreted that we can achieve 0.72% FPIR

and 3.95% FNIR with indexing and 3.93% FPIR and 3.83% FNIR without indexing.From Fig. 3.9, we can observe that using our proposed indexing approach we can achievelow FPIR for an FNMR.

3.7.5.2 Searching Time

We analyze the run-time complexity with big-O notation for gallery match correspondingto a query sample. Let N be the total number of samples enrolled in the database (fromP number of subjects), and each subject has Q number of samples. With reference to theAlgorithm 3.3, given a query template, we retrieve the identities of subjects (IDL) andmedian values (MDL) stored at TabIndex position from the ith(i = 1, 2, . . . , 12) table(Tablei). TabIndex is calculated using Eq. (3.17). The time complexity of this processis O(1). Next, we retrieve the IDL sets corresponding to the values min(MDL) − δ to

51

3. Iris Biometric Data Indexing�

�

�

�

:

0

3

1

8

":

"0

"3

"1

: 0: 3: 1: 8: "::

�'

�$

�)�$

L��

L��

Figure 3.9: FPIR vs. FNIR curves with and without indexing for CASIA-V3-Intervaldatabase.

max(MDL)+δ from Tablei and add these names to Si for rank calculation. This processcan be accomplished in O(1) time. Let IL be the average size of the IDL list. The rankcalculation process can be done in O(IL) time. In the worst case, when all samples arestored in one position then IL is equal to N , which is very unlikely to occur.

We analyze the search efficiency by measuring the average time taken to retrieve iristemplates from the database for a given query iris. Let tp be the average time to per-form addition, subtraction and assignment operations. Our indexing approach requires6 comparisons to retrieve candidates corresponding to one feature and a candidate setof size IL is retrieved using 12 features (see Algorithm 3.3). Therefore, the time taken(Tr) to retrieve a candidate set of size IL is (tp × 6)× 12. Let td be the time to comparequery iris template with one stored iris template for matching. Hence, the search timeusing our proposed indexing approach (Ts) is Tr + td × IL. On the other hand, linearsearch requires td×N time. Thus, our indexing approach takes less time than the linearsearch approach because IL << N .

We summarize the result for each approach in Table 3.3. We observe that, on anaverage, 52% less time is required to search the database with the proposed indexingcompared to the non-indexed search.

52


Table 3.3: Time required (in second) to retrieve best match using indexing and withoutindexing

Database Withoutindexing (s)

Withindexing (s) Time reduction (%)

BATH 1.913 1.017 46.84

CASIAV3I 2.532 1.324 47.71

CASIAV4T 7.871 2.647 66.37

MMU2 1.891 0.896 52.62

WVU 2.672 1.416 47.01

3.7.5.3 Memory Requirement

Our proposed approach considers a number of tables to index iris data. We analyze theamount of memory overhead for the proposed indexing mechanism. In our database, eachtable stores a sample one time and each table has entries for all samples. Each entryrequires 4 bytes. Hence, the memory requirement to store all index keys is calculatedusing Eq. (3.23).

MemoryReq = (Tsize × 4 +N ×B)× L (3.23)

In Eq. (3.23), Tsize denotes the size of a table, L is the number of features used forindexing, N is the number of samples in the database and B is the memory required tostore a sample. Figure 3.10 shows the average table size and memory requirement to storethe different number of samples for different databases. From Fig. 3.10(a), we see that thetable size does not vary linearly when the number of enrolled samples increases. Further,Fig. 3.10(b) shows that the memory requirement to store samples is increased linearly.This result substantiates that the total memory is not increasing non-polynomially.

We have calculated the memory cost for 1,000,000 and 1,000,000,000 subjects accord-ing to Eq. (3.23). The size of a table depends on the range of values of Gabor energyfeatures. From Fig. 3.10(a) we observe that the size of a table is approximately 1.2 timesthe number of samples (according to our experiment with 1,000 enrollments). We maynote that beyond 1,000 enrollments the size of the table does not increase significantly.Considering for a large enrollments, we may reasonably assume the size of the table as1.5 times the number of subjects. Further, we assume that 4 bytes are required to storean identity of a sample for a subject. With this consideration, we estimate the memory

53


�

�

�

�

�

3::

6::

1::

7::

8::

9::

":::

""::

"0::

"2::

"3::

3:: 9:: "3:: "9:: 03::

&,

�M��

��

��N

�

O��

� ��

�&��&C2�

�&��&C3*

��0

LC�

(a) Average table size to enroll iris data for different iris image databases

�

�

�

�

�

3:

1:

8:

"::

"0:

"3:

"1:

"8:

3:: 9:: "3:: "9:: 03::

��

��

��@

��

��

��4�

��

�5

Q"

::

:

O��

� ��

�&��&C2�

�&��&C3*

��0

LC�

(b) Memory requirements to enroll iris data for different iris imagedatabases

Figure 3.10: Average table size and memory requirements to enroll the iris data for differentiris image databases.

requirement for 1,000,000 and 1,000,000,000 enrollments are as follows.

MemReq106 = ([1.5× 1000000]× 4 + 1000000× 4)× 12 bytes

≈ 115MB

MemReq109 = ([1.5× 1000000000]× 4 + 1000000000× 4)× 12 bytes

≈ 112GB

Hence, to store the above mention data proposed approach requires approximately115MB and 112GB storage, respectively.

54

3.8. Comparison with Existing Work

3.8 Comparison with Existing Work

We compare our work with some of the existing indexing techniques mentioned in [154,164]. To compare our approach we consider the top five matches for all databases. Notethat existing approaches do not follow rank-based evaluation. Hence, we compare theresult of the existing work with the same at fifth rank of our algorithm with respectto HR and PR. We consider the fifth rank in our approach because in all databasesminimum five samples of a subject are present. Table 3.4 shows the comparison results.From Table 3.4, we see that our approach gives better HR than the approaches proposedin [154, 164]. It may be noted that the penetration rate according to our approach iscomparable to the existing indexing techniques.

We also analyze the existing algorithms to compare the retrieval times. In indexingmethod, a set of templates is retrieved for a given query template and then searching ormatching is performed on the retrieved data. The efficiency of indexing system can bemeasured in terms of the retrieval time of the system. We have analyzed the retrievaltimes of existing indexing techniques which is stated as follows. The IrisCode basedmethod [163,164] stores iris data into a number of clusters using k-means clustering. For agiven query sample, the IrisCode method determines the target cluster for closest match.This indicates that the query template is needed to compare with all cluster centers tofind a matched cluster. This technique requires minimum K number of comparisons tofind the appropriate cluster where K is the number of clusters. Typical values of K =

5, 10, 15, 20 for 2000 samples and K may increase when the number of enrolled samplesincrease. Hence, IrisCode based technique requires K number of O(1) computations toretrieve similar iris templates. It also may be noted that the number of K varies withthe sizes of databases [163,164].

The SPLDH based approach [164] uses tree data structure to store iris data. Theminimum search time for tree data structure to find a match with query template is

Table 3.4: Comparison of HR and PR with existing work

MethodHR PR

BATH CASIAV3I MMU2 WVU BATH CASIAV3I MMU2 WVU

IrisCode1[164] 84 80 79 82 16 17 21 20

IrisCode2[164] 84 80 79 82 18 21 23 25

IrisCode3[164] 84 80 79 82 7 8 12 13

SPLDH [164] 88 84 80 86 28 30 39 39

SHIFT [154] 88.9 89.5 86.2 87.6 13 24 18 21

Proposed 98.2 91.1 85.2 96.0 11.6 15.3 14.3 10.8IrisCode1: PCA based, IrisCode2: Column based, IrisCode3: Block based

55


O(logN) which is the height of the tree with N number of enrollments. This is sobecause the height of the tree depends on the number of samples stored in the tree.

The SIFT based technique [154] uses geometric hashing for indexing. In SIFT basedindexing, minimum M number of comparisons are required to hash all key points whereM is the number of key points in query sample and the value of M is approximately 100.This time is independent of the sizes of databases.

On the other hand, in our approach, we retrieve the similar templates correspond-ing to twelve index key values of query template. This requires twelve comparisons inO(1) time. Further, the retrieval time of our algorithm does not depend on the sizesof the databases. The retrieval time complexities and execution times (specific to com-puting environment Intel Core 2, 2.0GHz processor, 2 GB Memory) of different indexingmechanisms are summarized in Table 3.5.

3.9 Summary

We propose a novel indexing technique using Gabor wavelet features. Our proposedtechnique requires a less number of features compared to the existing approaches. As aconsequence, we need less memory to store index data. Further, with less number of fea-tures, the index space is organized in such a way that we retrieve a set of identity similarto the query in a constant time. We achieve the memory and computation time advan-tages without compromising the accuracy. In our approach, we achieve approximately99% CMS.

Table 3.5: Retrieval time complexity and execution time of different indexing mechanisms

Indexing method Retrieval timecomplexity

Time(ms)

IrisCode (row/column/ block based averaging, PCA)

Minimum K comparisons are required where K is the number ofclusters (typically K = 5, 10, 15, 20 for 2000 samples and the valueof K increases as the number of enrollments increases).

O(1) 0.02

SPLDH

Execution time depends on the number of enrolled samples (N).O(logN) 64

SIFT

Minimum M comparisons are required where M is the number ofkey points in an iris image. Typical value of key points is ≈100.

O(1) 231

Proposed approach

Only FL comparisons are needed where FL is the dimension ofthe index key. In our approach FL = 12 and it is independent ofthe number of samples.

O(1) 0.014

56

Chapter 4

Fingerprint Biometric DataIndexing

In fingerprint identification system, when the size of database is very large, a one-to-onematching makes the system extremely slow. In this work, our objective is to achieve afaster retrieval of fingerprint match by indexing the database. We propose a fingerprintindexing technique based on multi-dimensional invariable set of feature vectors. Weintroduce a new scheme for generating index key from minutiae triplets and considerthe local topology of minutiae using two nearest points triangle. We investigate threedifferent indexing techniques (linear, clustered and clustered kd-tree) with invariableset of features for a fingerprint identification system. The major steps in our proposedmethod are shown in Fig. 4.1.

The rest of the chapter is divided into eight sections. Section 4.1 describes the pre-processing technique of fingerprint image. Feature extraction methodology is discussedin Section 4.2. We discuss index key generation method for fingerprint in Section 4.3.

)��(��

��

��

��

��

��

� � � ��

*��

� ��

+��

��

��

$��,��

��(��

��

��

�� )��

.��

+��,��

Figure 4.1: An overview of our proposed fingerprint biometric data indexing approach.

57

4. Fingerprint Biometric Data Indexing

The techniques of storing and retrieving the fingerprint data are described in Section 4.4and Section 4.5, respectively. The performance of our proposed method is presented inSection 4.6. We present the comparison with existing work in Section 4.7. Section 4.8summarizes the chapter.

4.1 Preprocessing

A captured fingerprint image does not necessarily be a good quality. Hence, it is necessaryto improve the image quality before extracting the features from a fingerprint image.Preprocessing makes the feature extraction task easy and ensure the quality of extractedfeatures. To preprocess the fingerprint image we follow the existing techniques [96,180].Basic steps of fingerprint image preprocessing are shown in Fig. 4.2. We briefly discusseach step of preprocessing in this section.

4.1.1 Normalization

Normalization is used to reduce the variations in gray level values along ridges and valleysby adjusting the range of gray level values [96, 180]. In normalization, we normalize aninput fingerprint image so that the image has a prespecified mean and variance. LetI(x, y) and N(x, y) denote the intensity value of input fingerprint image and normalizedimage at pixel (x, y). The normalization is done using Eq. (4.1) [96].

N(x, y) =

⎧⎪⎨⎪⎩

M0 +

√V0×(I(x,y)−M)2

V ifI(x, y) > M

M0 −√

V0×(I(x,y)−M)2

V otherwise

(4.1)

In Eq. (4.1), M and V are the estimated mean and variance of I(x, y), respectively,and M0 and V0 are the desired mean and variance values, respectively.

'�� N �� /��

��

�� N ��

��

$��

��

��

��

��

��

.��

��

/�� @��

��

Figure 4.2: Preprocessing of fingerprint image.

58

4.1. Preprocessing

4.1.2 Segmentation

In this step, we separate the foreground fingerprint region which contains the ridgesand valleys from the image. The foreground region has a very high gray scale variancevalue [180]. Hence, we use a variance based thresholding method [96] for segmentation.First, we divide the normalized fingerprint image into a number of non-overlapping blocksand calculate the gray scale variance for each block. Next, the foreground blocks andbackground blocks are decided based on a global threshold value. The threshold valueis fixed for all blocks. We compare the gray-scale variance value of each block withthe global threshold value. If the variance is less than the global threshold value, thenwe assign the block as background region, otherwise, we assign it as foreground region.Finally, the background region is removed from the image and the fingerprint region isidentified which is used in subsequent stages of preprocessing and feature extraction.

4.1.3 Local Orientation Estimation

The orientation field of a fingerprint image represents the local orientation of the ridgesand valleys in a local neighborhood. To compute the orientation image, we use Hong etal. [96] least mean square estimation method. In this method, block-wise orientations ofthe segmented image are calculated. First, the whole image is divided into a number ofblocks each of size 16×16 and the gradient ∂x and ∂y along x- and y- axis are calculated,respectively. The gradients are computed using Sobel operator [84]. Next, the localorientation θ of each block centered at pixel (x, y) is calculated using Eq. (4.2), (4.3) and(4.4) [96].

θ(x, y) =1

2tan−1

(Vy(x, y)

Vx(x, y)

)(4.2)

Vx(x, y) =

x+w2∑

u=x−w2

y+w2∑

v=y−w2

2∂x(u, v)∂y(u, v) (4.3)

Vy(x, y) =

x+w2∑

u=x−w2

y+w2∑

v=y−w2

∂2x(u, v)∂

2y(u, v) (4.4)

Finally, we perform smoothing of the orientation field in a local neighborhood toremove the noise present in the orientation image. For this purpose, the orientationimage is converted into a continuous vector field and then Gaussian filter [96, 180] isapplied on the orientation image for smoothing.

59


4.1.4 Local Frequency Image Representation

The frequency image represents the local ridge frequency which is defined as the frequencyof the ridge and valley structures in a local neighborhood along a direction normal to thelocal ridge orientation [96]. Local ridge frequency can be computed as the reciprocal of thenumber of pixels between two consecutive ridges in a region. We use method proposedby Hong et al. [96] to calculate the ridge frequency. In this method, the normalizedfingerprint image is divided into a number of non-overlapping blocks. An oriented windowis defined in the ridge coordinate system along a direction orthogonal to the local ridgeorientation at the center pixel for each block. The average number of pixels between twoconsecutive ridges are calculated. Let T (x, y) be the average number of pixels of blockcentered at (x, y). Then, the frequency F (x, y) of the block (x, y) is calculated usingEq. (4.5) [96].

F (x, y) =1

T (x, y)(4.5)

If no consecutive peaks are there in the block, then the frequency of the block isassigned a value of −1 to differentiate it from the valid frequency values.

4.1.5 Ridge Filtering

We use a 2-D Gabor filter to remove efficiently the undesired noise and enhance the ridgeand valley structures of the fingerprint image. The Gabor filter is employed because ithas both frequency and orientation selective properties [180] which adjust the filter togive maximal response to ridges at a specific orientation and frequency in the fingerprintimage. Therefore, the Gabor filter preserves the ridge and valley structures and alsoreduces noise. An even-symmetric Gabor filter tuned with local ridge frequency and ori-entation is used to filter the ridges in a fingerprint image. The orientation and frequencyparameters of the even-symmetric Gabor filter are determined by the ridge orientationand frequency information of the fingerprint. The detailed method can be found in [96].

4.1.6 Binarization and Thinning

We use binarization to improve the contrast between the ridges and valleys in a fingerprintimage by converting a gray level image into a binary image [180]. The binary image isuseful for the extraction of minutiae points. We check the gray level value of each pixelin the enhanced image. If the value is greater than a predefined threshold value, thenthe pixel value is set to one, otherwise, it is set to zero. The binary image contains twolevels of information, the foreground ridges and the background valleys.

60

4.1. Preprocessing

Finally, we perform thinning morphological operation prior to minutiae extraction.Thinning is performed by successively eroding the foreground pixels until they are onepixel wide. We apply a standard thinning algorithm [87] which performs the thinningoperation using two sub iterations. In each sub-iteration, we examine the neighborhoodof each pixel in the binary image and decide, based on a particular set of pixel-detectioncriteria, whether the pixel can be deleted or not. These processes continue until thereare no more pixels for deletion. It may be noted that the application of the thinningalgorithm to a fingerprint image preserves the connectivity of the ridge structures whileforming a thinning image of the binary image. This thinned image is then used in thesubsequent minutiae extraction process.

4.1.7 Minutiae Point Extraction

Two main minutiae features namely ridge ending and ridge bifurcation are consideredin this work. The Crossing Number (CN) [155, 180] method is followed to extract theseminutiae features. This method extracts the ridge endings and bifurcations from thethinned image by scanning the local neighborhood of each ridge pixel using a 3 × 3

window [155]. A 3× 3 window centered at point P and its eight neighbor points Pi (i =

1, 2, . . . , 8) are shown in Fig. 4.3(a). The CN at point P is defined as half the sum of thedifferences between pairs of adjacent pixels in the eight-neighborhood (see Eq. (4.6)) [155,180]. Note that value of each Pi (i = 1, 2, . . . , 8) is either 0 or 1, as the image is binarized(see Section 4.1.6).

CN =1

2

8∑i=1

|Pi − Pi+1| with P9 = P1 (4.6)

Using the properties of the CN, the ridge ending and bifurcation are detected forCN = 1 and CN = 3, respectively. Figure 4.3(b) shows a ridge ending with CN = 1

and Fig. 4.3(c) shows a bifurcation minutiae point with CN = 3.

)3 )2 )0

)6 ) )"

)1 )7 )8

(a) Pixel positionin a 3× 3 window

(b) CN=1 (ridgeending)

(c) CN=3 (bifur-cation)

Figure 4.3: Pixel position and crossing number for two different types of minutiae.

61


ANSI/NIST-ITL format [152] is used to store the minutiae points. The origin of thefingerprint image is located at the bottom-left corner and y-coordinate is measured fromthe bottom to upward. The position and the angle of a minutiae feature are stored in(x, y, θ, T ) format where (x, y) is the location coordinate of a minutiae in the fingerprintimage, θ is the minutiae angle and T is the type of minutiae. The value of T is R for ridgeend type minutiae and B for bifurcation type minutiae. Minutiae angle (θ) is measuredanti-clockwise with the horizontal x-axis. The angle of a ridge ending is calculated bymeasuring the angle between the horizontal x-axis and the line starting at the ridge endpoint and running through the middle of the ridge as shown in Fig. 4.4(a). The angleof a bifurcation is calculated by measuring the angle between the horizontal x-axis andthe line starting at the minutia point and running through the middle of the interveningvalley between the bifurcating ridges as shown in Fig. 4.4(b).

Extracted minutiae using the above mentioned method may have many false minutiaedue to breaks in the ridge or valley structures. These artifacts are typically introduced bythe noise at the time of taking fingerprint impression. A break in a ridge causes two falseridge endings to be detected, while a break in a valley causes two false bifurcations. Inorder to remove all these false minutiae, the shorter distance minutiae removal methodof Chatterjee et al. [59] is followed. In this method, the distance between two ridgeending points or two bifurcation points, and orientations of that points are checked.If the distance is less than some minimum distance say δ, and the orientations of twopoints are opposite, then both minutiae points are removed. Here, the distance betweentwo points is considered as Euclidean distance. Euclidean distance (ED) between twominutiae points A and B with positions (xA, yA) and (xB, yB) is defined as:

ED(A,B) =√

(xA − xB)2 + (yA − yB)2 (4.7)

R

4�<��5

�

�

(a) Ridge ending

R

4�<��5

�

�

(b) Bifurcation

Figure 4.4: Minutiae position and minutiae orientation.

62


Let two minutiae points A and B are represented as (xA, yA, θA, TA) and (xB, yB, θB, TB),respectively. The Euclidean distance between two minutiae points is represented asED(A,B). The following heuristic is applied to remove the false minutiae points.

IF (TA = TB) THEN

IF (ED(A,B) < δ) AND (210 ≤ |θA − θB| ≤ 150) THEN

REMOVE A and B from the list of minutiae

ENDIF

ENDIF

In the above heuristic, the value of δ is chosen as the average distance between tworidges in the fingerprint image whose value is typically taken as 10. The difference ofthe orientations (|θA − θB|) is checked between 150 to 210 degree for the oppositenessof the minutiae points. Note that the exact value of |θA − θB| is 180 degree when theminutiae points are opposite. The tolerance value of ±30 degree is considered due to thecurvature nature of the ridges.


Our feature extraction step consists of two tasks: two closest point triangulation andtriplet generation. Detailed descriptions of these tasks are given in the following.

4.2.1 Two Closest Points Triangulation

To capture the structural information from the distribution of minutiae points, we pro-pose the concept of two closest point triangulation. We define a two closest point trianglewith three minutiae points such that for any point, the distance from other two pointsare minimum. In other words, let mi be a minutiae point. Other two points say mj andmk form a two closest point triangle if the distances between mi and mj , and mi andmk are minimum compared to all other minutiae points in the fingerprint. Figure 4.5(a)illustrates the definition of the two closest point triangle with four minutiae points. Itmay be noted that the triangulation covers less geometric area of a fingerprint and im-plies a meaningful feature. We can obtain a set of two closest point triangles from agiven fingerprint image. Figure 4.5(b) shows the two closest point triangles obtainedfrom minutiae points of a sample fingerprint.

63


4.2.2 Triplet Generation

A two closest point triangle with three minutiae points represents a triplet. We identifyall such triplets. Let M = {m1,m2, . . . ,m|M |} be a set of |M | minutiae points in afingerprint image. A set of minutiae triplets T = {t1, t2, . . . , t|T |} from M need to beidentified. The number of triplets in the set T is |T |, where |T | = |M |. First, wecalculate the distance matrix D as shown in Eq. (4.8). Each entry in D denotes thedistance between two minutiae points. For example, di,j represents the distance betweenthe ith and jth minutiae points. Here, di,j is calculated as the Euclidean distance usingEq. (4.7).

D =

⎡⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎣

d1,1 d1,2 . . . d1,j . . . d1,|M |d2,1 d2,2 . . . d2,j . . . d2,|M |...

......

......

...di,1 di,2 . . . di,j . . . di,|M |...

......

......

...d|M |,1 d|M |,2 . . . d|M |,j . . . d|M |,|M |

⎤⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎦

(4.8)

The triplet corresponding to the ith minutiae mi in the triplet set T is representedwith ti, where ti =< mi,mj ,mk > and 1 ≤ i, j, k ≤ |M |. The minutiae points mj andmk of triplet ti are determined by the criteria defined in Eq. (4.9).

mj = minIndex(D) ∃ di,j ∈ D : 1 ≤ j ≤ |M | and i = j

mk = minIndex(D) ∃ di,k ∈ D : 1 ≤ k ≤ |M | and i = j and i = k(4.9)

In Eq. (4.9), minIndex function returns the minutiae point having the minimumdistance from the ith minutiae. Each ti generated using the above mentioned criteriais added to the triplet set T. It may be noted that there may be duplicate tripletsin T because the minutiae points with minimum distance may be the same for twoneighborhood minutiae points. Figure 4.5(c) and (d) show the closest point triangles andcorresponding triplets generated from the five minutiae points of Fig. 4.5(c). Initially,the number of triplets in T is 5. We can see that the triplets, t1 and t3, generated fromminutiae points m1 and m3, are same. Hence, we remove one of these duplicate triplets(t1 or t3) from T. After removing the triplet t3, the set contains t1, t2, t4 and t5.

In this way, we generate the triplet set T, where |T | is the number of triplets generatedfrom |M | minutiae points. It may be noted that |T | is at most equal to |M | for afingerprint image.

64

4.3. Index Key Generation

��

�%

�(

��

�<�

�<%

�<(

(a) Two closest point triangulation

:

6:

"6:

06:

6:

"::

0::

":: 0::"6: 06:

(b) Two closest point triangle with sam-ple minutiae points of a fingerprint image

�" 48:<�"0:5

�0 49:<�"":5

�2 47"<�"::5

�6 496<�9:5

�3 4"::<�"265

(c) Duplicate two closest point triangles

� � ��

�6 ?�4�6<��0<��25

�3 ?�4�3<��"<��05

� � �� 0 ?�4�0<��"<��65

(d) Duplicate triplets

Figure 4.5: Two closest point triangulation with sample fingerprint image.


A fingerprint image is represented with a triplet set T. We generate a number of indexkeys from each triplet in T for a fingerprint image. An index key is characterized with a 8-dimensional feature vector which consists of geometric information of the triplet, relativeorientation information of each minutia point associated with the triplet and textureinformation of the local ridge structure around each minutiae point of the triplet.

Let ti be the ith triplet in T where 1 ≤ i ≤ |T | and indxi be the index key generatedfrom ti, which contains of eight feature values (f i(1), f i(2), f i(3), f i(4), f i(5), f i(6), f i(7)

and f i(8)). The three minutiae points m1,m2 and m3 of ti are the vertices of a triangle ina 2-D space (see Fig. 4.6). Let l1, l2 and l3 be the lengths of the sides of the triangle such

65


that l1 ≤ l2 ≤ l3, and θ1, θ2 and θ3 be the orientations of the minutiae points m1,m2 andm3, respectively for the triplet ti. We denote the angle of the longest side of the trianglewith respect to the horizontal axis by θL. Figure 4.6 shows the different components ofthe triangle corresponding to ti. The feature values (f i(1), f i(2), · · · , f i(8)) of the ith

index key are computed as follows.

f i(1): The feature f i(1) is defined as the ratio of the two smallest sides of triangle andcalculated using Eq. (4.10).

f i(1) =l2l1

(4.10)

f i(2): The feature f i(2) of index key is defined as interior angle between the two smallestsides of triangle. Here, f i(2) is the angle between l1 and l2 as in Eq. (4.11).

f i(2) = ∠m2m3m1 (4.11)

f i(3), f i(4) and f i(5): These features are the orientation of minutiae m1,m2,m3 withrespect to the angle of the longest side of triangle θL. We calculate f i(3), f i(4) and f i(5)

using Eq. (4.12).

f i(3) = |θL − θ1|f i(4) = |θL − θ2|f i(5) = |θL − θ3|

(4.12)

f i(6), f i(7) and f i(8): These three features represent the Gabor energy features at thelocation of three minutiae (m1, m2 and m3) of the triplet, respectively. We use a 2-D

�"

�0

�2

�"

�0

�2

R"

R0

R2

R/

Figure 4.6: Components of a minutiae triplet.

66


Gabor filter on 20×20 block size centered at each minutiae point of enhanced fingerprintimage. The block size 20 is considered because the inter ridge distance of fingerprintimage captured at 500 dpi is 10. The Gabor energy feature gives numerical summary oflocal ridge configuration around the minutiae.

Let I(x, y) be an enhanced fingerprint image and G(ω, θ) be the Gabor filter [96]

with frequency ω and orientation θ. The Gabor response GRω,θ is obtained by 2-Dconvolution of the image I(x, y) and Gabor filter G(ω, θ). Here, ω is the average localridge frequency of the fingerprint and can be computed using the method given in [96],w is the size of the filter and θ is the angle of the minutia (θ = θ1, θ2, θ3). The responseof Gabor filter is computed using Eq. (4.13).

GRω,θ(x, y) =

w2∑

u=−w2

w2∑

v=−w2

G(ω, θ)I(x− u, y − v) (4.13)

These responses are also called Gabor coefficient values which are complex. Wecalculate the magnitude value of a Gabor coefficient at each pixel in the block of aminutiae point using Eq. (4.14). Finally, the Gabor energy (GEω,θ) at each minutiae iscalculated as the sum of squares of the magnitude values of GRω,θ at every pixel usingEq. (4.15).

|GRω,θ(x, y)| =√

Re(GRω,θ(x, y))2 + Im(GRω,θ(x, y))2 (4.14)

GEω,θ =∑x,y

[|GRω,θ(x, y)|]2 (4.15)

We calculate f i(6), f i(7) and f i(8) using Eq. (4.16).

f i(6) = GEω,θ1

f i(7) = GEω,θ2

f i(8) = GEω,θ3

(4.16)

We generate all index keys corresponding to all triplets of the fingerprint imagesof each subject. The values of each feature are in different range for all index keys offingerprint images. To bring the values of each feature into a common range we applymin-max normalization [107] on each feature of fingerprint images. Suppose, there areP number of subjects and each subject has Q number of samples to be enrolled into thedatabase. We represent the index keys of the qth sample of the pth subject in Eq. (4.17),where p = 1, 2, . . . , P and q = 1, 2, . . . , Q. In Eq. (4.17), the ith index key of the qth

sample of the pth subject is denoted as indxip,q. Let |Tp,q| be the number of index keys

67


generated from the fingerprint image of the qth sample of the pth subject. Hence, the totalnumber of index keys obtained from all persons is N =

∑P,Qp=1,q=1 |Tp,q|. Each index key

can be viewed as a single point in an 8-dimensional hyperspace. Thus, each fingerprintimage is a collection of points residing in the 8-dimensional space. We need to storethese index keys and the corresponding subject’s identities into the database so that wecan easily retrieve the identities of subjects at the time of searching. We represent theidentity of the qth sample of the pth subject with Idpq .

indx1p,q = [ f1p,q(1) f1

p,q(2) · · · f1p,q(fl) · · · f1

p,q(8) ]

indx2p,q = [ f2p,q(1) f2

p,q(2) · · · f2p,q(fl) · · · f2

p,q(8) ]

· · · · · · · · · · · ·indxip,q = [ f i

p,q(1) f ip,q(2) · · · f i

p,q(fl) · · · f ip,q(8) ]

· · · · · · · · · · · ·indx

Tp,qp,q = [ f

Tp,qp,q (1) f

Tp,qp,q (2) · · · f

Tp,qp,q (fl) · · · f

Tp,qp,q (8) ]

(4.17)

4.4 Storing

In this work, we explore three different storing techniques to index the fingerprint data.First, we investigate the linear storing technique where all index keys and correspondingsubjects’ identities are stored linearly. Then we examine clustering technique. In thistechnique, all index keys are divided into a number of groups based on the feature valuesof index keys. The index keys and the corresponding subject’s identities in each groupare stored linearly for each cluster. Finally, we analyze the clustered kd-tree technique tostore all index keys. In this technique, we divide all index keys into a number of groups.The index keys and the corresponding subject’s identities in each group are stored in akd-tree [40].

4.4.1 Linear Index Space

In this approach, we store all index keys and subject identities linearly. In other words,we create a 2-D index space to store index keys and the corresponding identities. Thesize of the index space would be N × (FL+1) where N is the total number of index keysgenerated from all persons and FL (FL = 8) is the number of feature values in an indexkey. The first eight columns of each row in the 2-D space contain the eight feature valuesof an index key and the last column contains the identity of the subject corresponding tothe index key. The method of creating linear index space is summarized in Algorithm 4.1.

68

4.4. Storing

In Step 2 to 12 of Algorithm 4.1, all index keys and identity of a person are stored in thelinear index space. Among these steps, Step 5 to 7 store the eight key values into thefirst eight columns of the linear index space and Step 8 stores the identity of the personinto the 9th column of the linear index space.

Figure 4.7 shows the structure of the linear index space in the database. In Fig. 4.7,LNINDX is the two dimensional index space of size N× (FL+1). In this structure, wecan store N =

∑P,Qp=1,q=1 |Tp,q| number of index keys from the P subjects with Q samples

each. Here, we first store all index keys of the 1st subject, then all index keys of the 2nd

subject and so on. In Fig. 4.7, the entry < f ip,q(1) f

ip,q(2) · · · f i

p,q(d) · · · f ip,q(8) > and

Idpq in LNINDX represent the ith index key and the identity of the qth sample of thepth subject, respectively.

4.4.2 Clustered Index Space

In this approach, we divide all index keys into a number of groups. The grouping isdone by performing k-means clustering on all index keys using the 8-dimensional featurevalues. We choose

√N/2 number of clusters which is the rule of thumb of clusters for any

unsupervised clustering algorithm [151] to balance between the number of clusters and

Algorithm 4.1 Creating linear index space

Input: All index keys of all samples of all subjects (indx11,1, indx21,1, . . ., indxT1,1

1,1 ;

indx11,2, indx21,2, . . ., indxT1,2

1,2 ; . . . ; indx1P,Q, indx2P,Q, . . ., indxTP,Q

P,Q )Output: Linear Index Space (LNINDX[ ][ ])1: inc = 1

// P is the number of subjects2: for p = 1 to P do

// Q is the number of samples3: for q = 1 to Q do

// Tp,q is the number of index keys for the qth sample of the pth subject4: for i = 1 to Tp,q do5: for fl = 1 to 8 do6: LNINDX[inc][fl] = indxip,q[fl] // Copy eight feature values into the first eight

columns7: end for8: LNINDX[inc][9] = Idpq // Copy identity of the subject in the ninth column9: inc = inc+ 1


69

4. Fingerprint Biometric Data Indexing"

��

�%��

"<"*

"

+U"

"<@

@?"

* W"�+

"<@

@?"

*�

"

"<"� 4"5 "

"<"� 405

"<"*

"<"� 4"5

�

"<@� 4"5

"

"<+� 4"5

"<+*

"<+� 4"5

�

�<@� 4"5

"

)<"� 4"5

)<"*

)<"� 4"5

�

)<@� 4"5

"

)<+� 4"5

)<+*

)<+� 4"5

)��

�

%��

)U"<+

�<@

�?"<@?"

* W"�

)<+

�<@

�?"<@?"

* W"�

)U"<+

�<@ )<"

�?"<@?"

* W*�

)<+U"

�<@

�?"<@?"

* W"�

"<@�

�<@�

)<@�

"

"<"� 425 "

"�"

"<"� 435 "

"<"� 465"

"<"� 415 "

"<"� 475 "

"<"� 485

"<"*

"<"� 405 "<"*

"<"� 425 "<"*

"<"� 435 "<"*

"<"� 465 "<"*

"<"� 415 "<"*

"<"� 475 "<"*

"<"� 485

�

"<@� 405 �

"<@� 425 �

"<@� 435 �

"<@� 465 �

"<@� 415 �

"<@� 475 �

"<@� 485

"

"<+� 405 "

"<+� 425 "

"<+� 435 "

"<+� 465 "

"<+� 415"

"<+� 475 "

"<+� 485

"<+*

"<+� 405 "<+*

"<+� 425 "<+*

"<+� 435 "<+*

"<+� 465 "<+*

"<+� 415 "<+*

"<+� 475 "<+*

"<+� 485

�

�<@� 405 �

�<@� 435�

�<@� 465 �

�<@� 415�

�<@� 475�

�<@� 425�

�<@� 485

"

)<"� 405

)<"*

)<"� 405

�

)<@� 405

"

)<+� 405

)<+*

)<+� 405

"

)<"� 425

)<"*

)<"� 425

�

)<@� 425

"

)<+� 425

)<+*

)<+� 425

"

)<"� 435

)<"*

)<"� 435

�

)<@� 435

"

)<+� 435

)<+*

)<+� 435

"

)<"� 465

)<"*

)<"� 465

�

)<@� 465

"

)<+� 465

)<+*

)<+� 465

"

)<"� 415

)<"*

)<"� 415

�

)<@� 415

"

)<+� 415

)<+*

)<+� 415

"

)<"� 475

)<"*

)<"� 475

�

)<@� 475

"

)<+� 475

)<+*

)<+� 475

"

)<"� 485

)<"*

)<"� 485

�

)<@� 485

"

)<+� 485

)<+*

)<+� 485

"

"�

"

@�

"

+�

"

+�

�

@�

)

"�

)

"�

)

@�

)

+�

)

+�

��(��

��"��

��(��

��@��

��(��

��+��

��(��@��

� ��%��

��(��

��"��

��(��

��@��

��(��

��+��

/'�'�X

Figure 4.7: Linear index space.

the number of items within a single cluster. An eight dimensional index key is treatedas a eight dimensional point in the cluster space. Before doing k-means clustering, wepreprocess the fingerprint data so that clustering algorithm converges to a local optima.In preprocessing, we compute the distances of all points from the origin and find thedensity of the points from the distance histogram. From this histogram, we select thetop

√N/2 picks and randomly initialize the cluster centers with these points. Then, we

apply the k-means clustering on all eight dimensional points in the cluster space (i.e. allindex keys) and find out the centroid for each cluster. The cluster centroid representsthe point which is equal to the average values of all the eight dimensional points inthe cluster. Then, we assign key points to the clusters such that the distance of a keypoint from the centroid of assigned cluster is the minimum than the distances from thecentroids of other clusters. If we find any key points with almost same distances from thetwo or more cluster centers and or some isolated points with large distance from clustercenter, we remove those points from the resulting partition to reduce the effect of outliers.

70

4.4. Storing

The distance is measured as Euclidean distance between key point and cluster centroid.Finally, when all points are assigned, we recalculate the centroids of the clusters. Westore the centroid of each cluster in a two dimensional space (see Fig. 4.8). The size ofthe space is K × (FL + 1), where K is the number of clusters (here, K =

√N/2) and

FL (FL = 8) is the number of features in an index key.Let Ni be the number of index keys present in the ith cluster and f j(l) be the lth

(l = 1, 2, . . . , 8) feature value of the jth index key in the ith cluster. Then, the lth centroidvalue dcni(l) of the ith cluster is calculated using Eq. (4.18).

dcni(l) =1

Ni

Ni∑j=1

f j(l) for l = 1 to 8 and i = 1 to K (4.18)

Thus, we can calculate eight centroid index values (dcni(1) dcni(2) · · · dcni(8)) ofthe ith cluster using Eq. (4.18) and store the centroid values in a 2-D space (CLTCNTRD

in Fig. 4.8). In Fig. 4.8, CLTCNTRD is the array for storing the centroid values of allclusters. The first eight columns of each row store the 8-dimensional centroid values

�4"5 �405 �425 �435 �465 �415 �475 �485 �

"

0

'"

��Y"ZU[/'�'�X

��4"5

�/*�'*$�

��405 ��425 ��435 ��465 ��415 ��475 ��485 ��

��"4"5 ��"405 ��"��"425 ��"435 ��"465 ��"415 ��"475 ��"485

��\4"5 ��\405 ��\��\425 ��\435 ��\465 ��\415 ��\475 ��\485

"

\

�

"

0

'�

�

"

0

'\

��4"5 ��405 ��425 ��435 ��465 ��415 ��475 ��485�

��Y�ZU[/'�'�X

��Y\ZU[/'�'�X

�4"5 �405 �425 �435 �465 �415 �475 �485

�4"5 �405 �425 �435 �465 �415 �475 �485

Figure 4.8: Clustered index space.

71


(dcn(l), l = 1, 2, . . . , 8) for a cluster and the last column stores the cluster id (it isrepresented as clid in Fig. 4.8) which uniquely identifies each cluster. For each cluster,we create a linear index space (Cluster → LNINDX) as shown in Fig. 4.8. In Fig. 4.8,Cluster[i] → LNINDX is the linear index space for the ith cluster. The feature valuesof an index key in Cluster[i] → LNINDX are denoted as < f(1), f(2), · · · , f(8) > andthe identity of the subject corresponding to the index key is represented as id. Note thatthe index keys within a cluster are stored in a linear fashion. The method for creatingclustered index space is summarized in Algorithm 4.2. In Step 2 to 12 of Algorithm 4.2,all index keys of all samples of all subjects are stored into a linear index space. In Step13, k-meansClustering function takes the linear index space as input and divides allindex key into K groups. Each group is stored in Cluster[ ] → LNINDX[ ][ ] 2-D spaceof the clustered index space. In Step 15, Centroid function calculates the centroid ofa cluster. The centroid values of a cluster are stored in CLTCNTRD of cluster indexspace. The cluster id of each cluster (here cluster number) is stored in the 9th column ofCLTCNTRD in Step 16.

4.4.3 Clustered kd-tree Index Space

First, we create clusters of all index keys as discussed above. Then each index key ofa cluster is stored into a kd-tree. A kd-tree is a data structure for storing a finite setof points from a k-dimensional space [40]. The kd-tree is a binary tree in which everynode stores a k-dimensional point. In other words, the node of a kd-tree stores an eightdimensional point. The node structure of kd-tree is shown in Fig. 4.9(a). In a node ofa kd-tree, keyV ector field stores the k-dimensional index key and Split field stores thesplitting dimension or a discriminator value. The leftTree and rightTree store a kd-treerepresenting the pointers to the left and right of the splitting plane, respectively. Foran example, let there are twelve three dimensional points (P1, P2, . . . , P12) as shownin Fig. 4.9(b) and a kd-tree with these points is shown in Fig. 4.9(c). We insert thefirst point P1 at the root of the kd-tree. At the time of insertion, we choose one of thedimension as a basis (Split) of dividing the rest of the points. In this example, the valueof Split in the root node is 1. In other words, if the value of the first dimension of thecurrent point to be inserted is less than the same of the root, then the point is storedin leftTree otherwise in rightTree. This means all items to the left of root will havethe first dimension value less than that of the root and all items to the right of the rootwill have greater than (or equal to) that of the root. The point P2 is inserted in theright kd-tree and P3 is inserted in the left kd-tree of the root. When we insert the pointP4, we first compare the first dimension value of P4 with the root and then compare

72

4.4. Storing

Algorithm 4.2 Creating clustered index space


1,1 ;

indx11,2, indx21,2, . . ., indxT1,2


P,Q )Output: Cluster centroids (CLTCNTRD[ ][ ]) and linear index spaces (Cluster[ ] →

LNINDX[ ][ ]) corresponding to each cluster

// Creating linear index space1: inc = 1

// P is the number of subjects2: for p = 1 to P do

// Q is the number of samples3: for q = 1 to Q do

// Tp,q is the number of index keys for the qth sample of the pth subject4: for i = 1 to Tp,q do5: for fl = 1 to 8 do6: LNINDX[inc][fl] = indxip,q[fl] // Copy feature values to the first eight columns7: end for8: LNINDX[inc][9] = Idpq // Copy identity of the subject in the ninth column9: inc = inc+ 1


// Grouping of all index keys into K different clusters13: Cluster[ ] → LNINDX[ ][ ] = K-meanClustering(LNINDX[ ][ ],K)

// K is the number of clusters14: for k = 1 to K do

// Calculate the centroid of each cluster15: CLTCNTRD[k][ ] = Centroid(Cluster[k] → LNINDX)16: CLTCNTRD[k][9] = k // Copy the cluster id of linear index space corresponding to

a cluster in the ninth column of clustered index space17: end for

the second dimension value of P4 with the second dimension value of P2 at next level.The point P4 is inserted in the right kd-tree of the point P2. Similarly, we insert the allother points into the kd-tree. First dimension will be chosen again as the basis (Split=1 )for discrimination at level 3. We store all eight dimensional points (index keys) within acluster in a kd-tree data structure. We use the FLANN library [15,34,162] to implementthe kd-tree in our work. The maximum height of the optimized kd-tree with N numberof k-dimensional point is �log2(N)� [40].

Figure 4.10 shows the schematic of a clustered kd-tree index space. In Fig. 4.10,CLTCNTRD stores the centroid of the K clusters. The centroid value of the ith cluster

73


��#�(U��

�� #��

��#�&�(U��

��

��#�&�(U��

��

(a) Components of a kd-tree node

)"��?�]0:��"1��"8[

)0��?�]03��"0��"7[

)2��?�]"7��01��"3[

)3��?�]07��00��27[

)6��?�]"9��"6��08[

)1��?�]"8��20��07[

)7��?�]07��""��03[

)8��?�]"8��23��22[

)9��?�]06��"7��08[

)":�?�]"0��0:��26[

)""�?�]2:��""��09[

)"0�?�]"6��20��"8[

(b) 3D points

��?"

)"?Y��<�"1<�"8Z

��?0

)2?Y"7<��<�"3Z

��?0

)0?Y03<�� "7Z

��?2

)6?Y"9<�"6<��Z

��?2

)1?Y"8<�20<��Z

��?2

)7?Y07<�""<��Z

��?2

)3?Y07<�00<��Z

��?"

)9?Y��<�"7<�08Z

��?"

)""?Y��<�""<�09Z

��?"

)8?Y��<�23<�22Z

��?"

)"0?Y��<�20<�"8Z

��?"

)":?Y��<�0:<�26Z

(c) Kd-tree example

Figure 4.9: Structure of a kd-tree and an example of kd-tree with three dimensional points.

�/*�'*$�

��

��"

��\

��

(U��"

(U��

(U��\

��4"5 ��405 ��425 ��435 ��465 ��415 ��475 ��485

��"4"5 ��"405 ��"425 ��"435 ��"465 ��"415 ��"475 ��"485

��\4"5 ��\405 ��\425 ��\435 ��\465 ��\415 ��\475 ��\485

"

\

��4"5 ��405 ��425 ��435 ��465 ��415 ��475 ��485�

Figure 4.10: Clustered kd-tree index space.

is represented as < dcni(l) dcni(2) · · · dcni(8) >. We store the index keys within acluster into a kd-tree data structure and assign a unique id to the each kd-tree. The id ofa kd-tree is stored at the last column of the CLTCNTRD. In Fig. 4.10, rooti representsthe id of the ith kd-tree. The kd-treei stores the index keys of the ith cluster as shownin Fig. 4.10.

74

4.4. Storing

The methods for creating clustered kd-tree index spaces are summarized in Algo-rithm 4.3. In Step 2 to 12 of Algorithm 4.3, all index keys of all samples of all subjectsare stored in a linear index space. Step 13 takes the linear index space as input anddivides all index keys into k groups using k-meansClustering function. In Steps 14 to20 of Algorithm 4.3, the clustered kd-tree index space is created. In Step 15, Centroid

Algorithm 4.3 Creating clustered kd-tree index space


1,1 ;

indx11,2, indx21,2, . . ., indxT1,2


P,Q )Output: Cluster centroids (CLTCNTRD[ ][ ]) and kd-trees (kd− treek) corresponding

to each cluster// Creating linear index space

1: inc = 1// P is the number of subjects

2: for p = 1 to P do// Q is the number of samples

3: for q = 1 to Q do// Tp,q is the number of index keys for the qth sample of the pth subject

4: for i = 1 to Tp,q do5: for fl = 1 to 8 do6: LNINDX[inc][fl] = indxip,q[fl] // Copy eight feature values into the first eight

columns7: end for8: LNINDX[inc][9] = Idpq // Copy identity of the subject in the ninth column9: inc = inc+ 1


// Grouping of all index keys into K different clusters13: Cluster[ ] → LNINDX[ ][ ] = K-meanClustering(LNINDX[ ][ ],K)

// K is the number of clusters14: for k = 1 to K do

// Calculate the centroid of each cluster15: CLTCNTRD[k][ ] = Centroid(Cluster[k] → LNINDX)

// Each key of a cluster inserted into a kd-tree index space// Nk is the number of index keys in kth cluster

16: for j = 1 to Nk do17: Insert Cluster[k] → LNINDX[j][ ] into Kd-treek18: end for19: CLTCNTRD[k][9] = Kd-treek // Copy the id of kd-tree corresponding to a cluster

in the ninth column of clustered index space20: end for

75


function calculates the centroid of a cluster. The centroid values of a cluster are storedin CLTCNTRD of clustered index space. In Step 16 to 18, each index key of the kth

cluster is inserted into the kth kd-tree and in Step 19, the id of the kth kd-tree is storedin the 9th column of kth row of clustered index space (CLTCNTRD).

4.5 Retrieving

Retrieving is the process of generating a candidate set (CSET ) from the enrolled fin-gerprints for a given query fingerprint. Note that the fingerprints in the CSET are themost likely to match with the query fingerprint. To obtain the CSET for a given queryfingerprint image, first we generate the index keys corresponding to each triplet of thequery fingerprint using index key generation technique discussed in Section 4.3 and nor-malize each dimension using min-max normalization [107] to bring each dimension intocommon range. Let Tt be the total number of index keys of the query fingerprint (seeEq. (4.19)). In Eq. (4.19), indxit denotes the ith index key of the query fingerprint. Thenwe compare each index key of the query fingerprint with the stored index keys in thedatabase and generate the CSET . The CSET contains two fields: id and vote. The id

and vote fields of the CSET store the identity of the subject and the number of vote

received for that subject. When an index key of the query fingerprint is matched witha stored index key then a vote is counted corresponding to that identity. Finally, wecalculate the rank based on the vote received by each identity. The first rank is givencorresponding to the maximum vote received, second rank is given corresponding to thenext maximum vote received and so on.

indx1t = [ f1t (1) f1

t (2) · · · f1t (d) · · · f1

t (8) ]

indx2t = [ f2t (1) f2

t (2) · · · f2t (d) · · · f2

t (8) ]

· · · · · · · · · · · ·indxit = [ f i

t (1) f it (2) · · · f i

t (d) · · · f it (8) ]

· · · · · · · · · · · ·indxTt

t = [ fTtt (1) fTt

p,q(2) · · · fTtt (d) · · · fTt

t (8) ]

(4.19)

In this work, we investigate three different searching techniques to retrieve the finger-print from the database for three different storage structures discussed in the previoussection. These three searching techniques are linear search (LS), clustered search (CS)and clustered kd-tree search (CKS) for linear, clustered and clustered kd-tree index spaces,respectively. These search techniques are described in the following.

76

4.5. Retrieving

4.5.1 Linear Search (LS)

In this approach, we search the linear index space for a query fingerprint. For an in-dex key of the query fingerprint, we search whole linear index space and calculate thedistances with all index key stored in linear index space. We calculate the distance asEuclidean distance. The Euclidean distance between the ith index key (indxit) of thequery fingerprint and the jth index key (indxjp,q) of the qth sample of the pth subject inthe database is calculated as in Eq. (4.20).

EDi,j =

√8∑

fl=1

(f it (fl)− f j

p,q(fl))2 where i = 1, 2, . . . , Tt

j = 1, 2, . . . , Tp,q ; ∀p and ∀q(4.20)

We store the identity and cast a vote on that identity in the CSET corresponding tothe minimum distance. If the jth index key in the database produces minimum Euclideandistance for the ith index key of query fingerprint, then we add a vote for the subjectidentity (Idj) of the jth index key. In the similar way, we cast the votes for other indexkeys of the query fingerprint. The identity which receives the highest number of votes willbe ranked first in the CSET . The method for generating CSET using linear search isdescribed in Algorithm 4.4. Step 3 to 7 of Algorithm 4.4 calculate the distances betweena query index key and all stored index keys. In Step 8, the minimum distance among allcalculated distances is determined. Match index corresponding to the minimum distanceis selected in Step 9 and the id is retrieved from linear index space at that index in Step10. Step 11 to 18 generate the CSET and CSET is sorted according to received votesin Step 20.

4.5.2 Clustered Search (CS)

In this technique, we search the clustered index space to retrieve the candidate set. Foran index key of query fingerprint, first we find a cluster which contains index keys similarwith the query index key. Then we retrieve the all identities from the identified cluster.To do this, first we calculate the Euclidean distance between an index key of the queryfingerprint and all stored centroid values of clusters using Eq. (4.21). In Eq. (4.21), indxitis the ith index key of the query fingerprint and dcnj is the centroid values of the jth

cluster.

EDi,j =

√√√√ 8∑fl=1

(f it (fl)− dcnj(fl))2 where j = 1, 2, . . . ,K and i = 1, 2, . . . , Tt (4.21)

77


Algorithm 4.4 Candidate set generation in LS from linear index space

Input: All index keys from query fingerprint image (indx1t , indx2t , . . ., indxTtt ) and linear

index space (LNINDX[ ][ ])Output: Candidate Set (CSET [ ] → (id, vote))1: inc = 1

// Tt is the total number of query index key2: for i = 1 to Tt do

// Calculate distance from a query index key to all stored index key3: for j = 1 to N do

// N is the total number of index keys in the database

4: EDi,j =

√8∑

fl=1

(f it (fl)− LNINDX[j][fl])2

5: end for// Select minimum distance

6: EDmin = min(EDi)7: Select j corresponding to EDmin and assign to idmin

// Select id from linear index space for the minimum distance8: id = LNINDX[idmin][9]9: subid = SubjectId(id) // Copy subject id from id

// Generate candidate set10: if subid /∈ CSET [ ] → (id) then11: CSET [inc] → id = subid12: CSET [inc] → vote = CSET [inc] → vote+ 113: inc = inc+ 114: else15: CSET [id] → vote = CSET [id] → vote+ 116: end if17: end for

18: Sort CSET [ ] → (id, vote) in descending order based on vote

We select the cluster which has the minimum distance between a query index key andthe centroid values of that cluster. If centroid values of the jth cluster in the databaseproduce minimum distance, then we retrieve all identities from the linear index space(Cluster[i] → LNINDX[ ][ ]) of the jth cluster in the database. Similarly, we find theclusters for all other index keys of the query fingerprint. The voting is done with theretrieved identities from the clusters to generate the CSET . The method for generatingCSET from the clustered index space is summarized in Algorithm 4.5. Step 2 to 7 ofAlgorithm 4.5 find the matched cluster for an index key of query fingerprint. Step 8 to19 generate the CSET for cluster search. Finally, CSET is sorted in Step 20.

78

4.5. Retrieving

Algorithm 4.5 Candidate set generation in CS from clustered index space

Input: All index keys from query fingerprint image (indx1t , indx2t , . . ., indxTtt ), Clus-

ter centroid index space(CLTCNTRD[ ][ ]) and linear index space for each cluster(Cluster[ ] → LNINDX[ ][ ])

Output: Candidate Set (CSET [ ] → (id, vote))

1: inc = 1// Tt is the total number of query index key

2: for i = 1 to Tt do// Find matched cluster, K is the number of clusters

3: for k = 1 to K do

4: EDi,k =

√8∑

fl=1

(f it (fl)− dcnk(fl))2

5: end for6: EDmin = min(EDi)7: Select cluster id corresponding to EDmin and assign to CL

// Retrieve the all identities from Cluster[CL] → LNINDX[ ][ ]

// Let NCL be the total number of index key in the CLth cluster8: for i = 1 to NCL do9: id = Cluster[CL] → LNINDX[i][9] // Select id from linear index space

10: subid = SubjectId(id) // Select the subject id from id11: if subid /∈ CS[ ] → (id) then

// Generate candidate set12: CSET [inc] → id = subid13: CSET [inc] → vote = CSET [inc] → vote+ 114: inc = inc+ 115: else16: CSET [id] → vote = CSET [id] → vote+ 117: end if18: end for19: end for20: Sort CSET [ ] → (id, vote) in descending order based on vote

4.5.3 Clustered kd-tree Search (CKS)

In this approach, first we find the cluster to which a query index key belongs. Thenwe search the kd-tree corresponding to that cluster. We find the cluster which have theminimum Euclidean distance between a query index key and the stored centroid indexkey. The method is discussed in the previous section. If centroid values of the ith clusterproduces minimum Euclidean distance, we search the ith kd-tree in the database. Themethod for creating CSET from the clustered kd-tree index space is summarized in

79


Algorithm 4.6 Candidate set generation in CKS from clustered kd-tree index space

Input: All index keys from query fingerprint image (indx1t , indx2t , . . ., indxTtt ), Cluster

centroid index space(CLTCNTRD[ ][ ]) and linear index space for each cluster (Kd−tree[ ])

Output: Candidate Set (CSET [ ] → (id, vote))1: inc = 1

// Tt is the total number of query index key2: for i = 1 to Tt do

// Find matched cluster, K is the number of clusters3: for k = 1 to K do

4: EDi,k =

√8∑

fl=1

(f it (fl)− dcnk(fl))2

5: end for6: EDmin = min(EDi)7: Select cluster id corresponding to EDmin and assign to CL

// Find nearest neighbor of a query index key from Kd-tree (Kd-tree[CL])8: NN = findNearestNeighbour(Kd-tree[CL], indx

Qq ) // Find the nearest neighbor

9: id = retrieveIdFromNN(NN) // Select id of nearest neighbor10: subid = SubjectId(id) // Select the subject id from id11: if subid /∈ CSET [ ] → (id) then

// Generate candidate set12: CSET [inc] → id = subid13: CSET [inc] → vote = CSET [inc] → vote+ 114: inc = inc+ 115: else16: CSET [id] → vote = CSET [id] → vote+ 117: end if18: end for19: Sort CSET [ ] → (id, vote) in descending order based on vote

Algorithm 4.6. Step 2 to 7 of Algorithm 4.6 find the matched cluster for an index keyof the query fingerprint. Step 8 finds the nearest neighbors of query index key from thekd-tree corresponding to matched cluster. We use FLANN library [15, 34, 162] to searchthe kd-tree. We find the subject identity of the nearest neighbor in Step 9 and 10. Thesorted CSET is generated in Step 11 to 19.


To study the efficacy of the proposed fingerprint data indexing approach, we have con-ducted a number of experiments on different fingerprint databases with different exper-

80


imental setups. The efficacy of the proposed method is measured with different aspectslike accuracy, efficiency, searching time and memory requirement. In this section, first,we present the descriptions of different fingerprint databases and the experimental se-tups used in our approach. Then we present the experimental results of the proposedapproach.

4.6.1 Databases

In our experiments, we consider six widely used fingerprint databases. The first twodatabases are obtained from National Institute of Science and Technology (NIST) Spe-cial DB4 database [32, 189]. The remaining databases are obtained from FingerprintVerification Competition (FVC) 2004 databases [16, 147]. Detailed description of eachdatabase is given in the following.

NIST DB4: The NIST Special Database 4 [32, 189] contains 4000 8-bit gray scale fin-gerprint images from 2000 fingers. Each finger has two different image instances (F andS). The fingerprint images are captured at 500 dpi resolution from rolled fingerprint im-pressions scanned from cards. The size of each fingerprint image is 512 × 512 pixelswith 32 rows of white space at the bottom of the fingerprint image. The fingerprints areuniformly distributed in the five fingerprint classes: arch (A), left loop (L), right loop(R), tented arch (T) and whorl (W).

NIST DB4 (Natural): The natural distribution of fingerprint images [32, 114] is A =3.7%, L = 33.8%, R = 31.7%, T = 2.9% and W = 27.9%. A subset of NIST DB4 isobtained by reducing the cardinality of the less frequent classes to resemble a naturaldistribution. The data set contains 1,204 fingerprint pairs (the first fingerprints fromeach class have been chosen according to the correct proportion).

FVC2004 DB1: The first FVC 2004 database [16,147] contains 880 fingerprint imagesof 110 different fingers. Each finger has 8 impressions in the database. The images arecaptured at 500 dpi resolutions using optical sensor (“V300" by CrossMatch) [147]. Thesize of each image is 640 × 480 pixels.

FVC2004 DB2: The second FVC 2004 database [16, 147] contains 880 fingerprint im-ages captured from 110 different fingers and 8 impressions per finger. The images arecaptured at 500 dpi resolutions using optical sensor (“U.are.U 4000" by Digital Per-sona) [147]. The size of each image is 328×364 pixels.

FVC2004 DB3: The third FVC 2004 database [16,147] contains 880 fingerprint imagesof 110 different fingers captured at 512 dpi resolutions with thermal sweeping sensor

81


(“Finger Chip FCD4B14CB" by Atmel) [147]. Each finger has 8 impressions in thedatabase. The size of each image is 300×480 pixels.

FVC2004 DB4: The fourth FVC 2004 database [16,147] is generated synthetically usingSFinGe V3 tool [25, 56]. This database consists of 880 fingerprint images of 110 fingersand 8 impression per finger. The images are generated with about 500 dpi resolutionsand the size of images is 288×384 pixels.


To evaluate our proposed indexing methods, we have partitioned each fingerprint databaseinto two sets: Gallery and Probe. The Gallery set contains the fingerprint images en-rolled into the index database and the Probe set contains the fingerprint images used forquery to search the index database. In our experiment, we create one type of Gallery set(G1) and one type of Probe set (P) for each NIST database and three types of Gallery

set (G1, G3 and G5) and one type of Probe set (P) for each FVC 2004 database toconduct the different experiments.

• G1: The gallery set is created with the first fingerprint impression of each finger inthe NIST databases and one fingerprint impression from the first six impressionsof each finger in FVC 2004 databases.

• G3: In this gallery set, three fingerprint impressions from the first six impressionsof each finger in FVC 2004 databases are used.

• G5: This gallery set contains five fingerprint impressions from the first six impres-sions of each finger in FVC 2004 databases.

• P: The probe set is created with the second fingerprint impression of each fingerin NIST databases and the seventh and eighth impressions of each finger in FVC2004 databases.

All methods described in our approach are implemented using C language and com-piled with GCC 4.3 compiler on the Linux operating system. Methods are evaluatedwith Intel Core2Duo processor of speed 2.00GHz and 2GB RAM.

4.6.3 Evaluation

We evaluate our approach with different experiments to measure the performance of theproposed indexing method. The descriptions of these experiments and the results withthese experiments are given bellow.

82


4.6.3.1 Accuracy

We measure the accuracy of the proposed fingerprint data indexing approach with re-spect to the parameters: HR, PR and CMS. The definition of these parameters aregiven in Section 3.7.1. Here, we study the accuracy of the proposed method with singlesample enrollment as well as multiple sample enrollments into the database.

Performance of three search techniques: To investigate the performance of the LS,CS and CKS with single sample enrollment into the database, we use G1 and P asGallery and Probe sets of all NIST (NIST DB4 and NIST DB4 Natural) and FVC 2004(FVC 2004 DB1, FVC 2004 DB2, FVC 2004 DB3 and FVC 2004 DB4) databases. Weenroll G1 into the database and probe with the samples in P. Results on rank one HR,PR and searching time (ST ) for LS, CS and CKS are given in Table 4.1. From Table 4.1,we observe that LS gives the maximum rank one HR whereas CKS gives the minimumrank one HR. However, the PR is 100% for LS whereas the PR for CKS is 13.55%. Alsothe ST in CKS is less than the other two searching techniques.

We also substantiate the performances of LS, CS and CKS with CMS at differentranks. The CMS is represented by CMC curve. Figure 4.11 shows the CMC curves ofLS, CS and CKS for two NIST databases and four FVC 2004 databases. In Fig. 4.11,we can see that the CMS is increased at higher rank. We attain 95.90%, 95.05% and94.20% CMS for NIST DB4 database (see Fig. 4.11(a)) and 96.20%, 95.55% and 95.10%CMS for NIST DB4 Natural database (see Fig. 4.11(b)) at the rank five in LS, CS andCKS, respectively. From Fig. 4.11(a) and (b), we observe that if the rank increases from5 to 15, then the CMS increases to 99.65%, 98.50% and 97.55% in LS, CS and CKSfor NIST DB4 database, respectively. With reference to NIST DB4 (Natural) database,

Table 4.1: HR, PR and searching time (ST ) for different fingerprint databases with singleenrollment (gallery G1) into the database

DBLS CS CKS

HR PR ST (ms) HR PR ST (ms) HR PR ST (ms)

FVC2004DB1 90.91 100 33.102 89.09 28.38 10.342 86.36 11.93 0.426

FVC2004DB2 90.45 100 32.693 88.18 32.03 7.852 85.45 13.17 0.485

FVC2004DB3 83.64 100 117.725 80 33.44 25.887 78.18 14.75 1.57

FVC2004DB4 89.09 100 26.925 85.91 25.56 9.395 84.09 10.89 0.349

NISTDB4 85.3 100 325.928 80.4 32.56 25.455 78.7 15.34 1.193

NIST DB4Natural 87.7 100 213 83.45 28.67 18.411 81.1 15.22 0.842

83


for the same increment in rank, the CMS are 99.70%, 99.10% and 98.80% in LS, CSand CKS, respectively. We achieve 94.09%, 92.73% and 90.45% CMS for FVC2004DB1 (Fig. 4.11(c)), 94.09%, 91.82% and 90.0% CMS for FVC2004 DB2 (Fig. 4.11(d)),

�

�

�

�

�

�

�

�

�

�

�

� �

8:

86

9:

96

"::

: 6 ": "6 0: 06 2:

��

�

$ �(

��

/��"

��"

�\��"

(a) CMC curves with NIST DB4 database�

�

�

� �

8:

86

9:

96

"::

: 6 ": "6 0: 06 2:

��

�

$ �(

��

/��"

��"

�\��"

(b) CMC curves with NIST DB4 Natural database�

�

� �

8:

86

9:

96

"::

: 6 ": "6 0: 06 2:

��

�

$ �(

!"#��

/��"

��"

�\��"

(c) CMC curves with FVC 2004 DB1 database

Figure 4.11: CMC curves with one enrolled sample per fingerprint in LS, CS and CKS fordifferent databases.

84


�

�

� �

8:

86

9:

96

"::

: 6 ": "6 0: 06 2:

��

�

$ �(

!"#��

/��"

��"

�\��"

(d) CMC curves with FVC 2004 DB2 database�

�

� �

8:

86

9:

96

"::

: 6 ": "6 0: 06 2:

��

�

$ �(

!"#��

/��"

��"

�\��"

(e) CMC curves with FVC 2004 DB3 database�

�

�

�

�

� �

8:

86

9:

96

"::

: 6 ": "6 0: 06 2:

��

�

$ �(

!"#��

/��"

��"

�\��"

(f) CMC curves with FVC 2004 DB4 database

Figure 4.11: CMC curves with one enrolled sample per fingerprint in LS, CS and CKS fordifferent databases (contd.).

88.18%, 87.27% and 85.91% CMS for FVC2004 DB3 (Fig. 4.11(e)) and 95.91%, 90.45%and 88.18% CMS for FVC2004 DB4 (Fig. 4.11(f)) at rank five in LS, CS and CKS, re-spectively. Figure 4.11(c), (d), (e) and (f) show that the CMS increases to 100%, 98.64%

85


and 94.55% for FVC2004 DB1, 100%, 97.27% and 95.91% for FVC2004 DB2, 97.73%,95.0% and 94.09% for FVC2004 DB3 and 100%, 98.64% and 95.0% for FVC2004 DB4when rank increases to 15% in LS, CS and CKS, respectively.

Effect of multiple samples enrollment: We study the effect of enrollments of multi-ple samples per fingerprint into the index database with LS, CS and CKS performances.We evaluate the performances on G1, G3 and G5 Gallery sets and P Probe set of eachFVC 2004 database (FVC 2004 DB1, FVC 2004 DB2, FVC 2004 DB3 and FVC 2004DB4). The evaluation results of three searching techniques are presented in the following.

a) LS: We have given the results of LS with different number of samples enrollment intothe database in Table 4.2. We observe that rank one HR increases for more number ofenrollments.

We also analyze the performance with CMS of LS when one (G1), three (G3) andfive (G5) samples per fingerprint are enrolled into the database. In Fig. 4.12 (a), (b), (c)and (d) show the CMC curves of LS for FVC2004 DB1, DB2, DB3 and DB4 databases,respectively. We observe that LS gives better result when five samples per fingerprintare enrolled in the database.

b) CS: The performance of CS with one (G1), three (G3) and five (G5) samples en-rollment per fingerprint into the database is reported in Table 4.3. From Table 4.3, weobserve that CS is also perform better with respect to rank one HR when multiple num-ber of samples are enrolled into the database. For multiple enrollment, PR is increaseda little but required searching time is more.

The analyze the performance of CS with CMS for multiple samples enrollment. TheCMC curves are obtained in CS with FVC2004 DB1, DB2, DB3 and DB4 databases withmultiple samples enrollment are shown in Fig. 4.13 (a), (b), (c) and (d), respectively. Weobserve that CS is produced better performance when more number of samples per fin-gerprint are enrolled in the database.

Table 4.2: HR, PR and searching time (ST) in LS with different number of enrollments(Gallery G1, G3 and G5) into the database

DBG1 G3 G5


FVC2004DB1 90.91 100 33.102 92.27 100 109.13 92.73 100 159.045

FVC2004DB2 90.45 100 32.693 91.36 100 77.155 91.82 100 155.873

FVC2004DB3 83.64 100 117.725 84.55 100 356.29 85.91 100 671.011

FVC2004DB4 89.09 100 26.925 90 100 103.115 90.45 100 170.812

86

4.6. Performance Evaluation�

�

�

� �

8:

86

9:

96

"::

: 6 ": "6 0: 06 2:

��

�

$ �(

!"#��

/��"

/��2

/��6

(a) CMC curves of LS with gallery G1, G3 and G5for FVC 2004 DB1 database

�

�

�

� �

8:

86

9:

96

"::

: 6 ": "6 0: 06 2:

��

�

$ �(

!"#��

/��"

/��2

/��6

(b) CMC curves of LS with gallery G1, G3 and G5for FVC 2004 DB2 database

�

� �

8:

86

9:

96

"::

: 6 ": "6 0: 06 2:

��

�

$ �(

!"#��

/��"

/��2

/��6

(c) CMC curves of LS with gallery G1, G3 and G5 forFVC 2004 DB3 database

Figure 4.12: CMC curves of LS with different number of enrolled samples for differentdatabases.

87


�

�

8:

86

9:

96

"::

: 6 ": "6 0: 06 2:

��

�

$ �(

!"#��

/��"

/��2

/��6

(d) CMC curves of LS with gallery G1, G3 and G5for FVC 2004 DB4 database

Figure 4.12: CMC curves of LS with different number of enrolled samples for differentdatabases (contd.).

Table 4.3: HR, PR and searching time (ST ) in CS with different number of enrollments(gallery G1, G3 and G5) into the database

DBG1 G3 G5


FVC2004DB1 89.09 28.38 10.342 89.55 26.89 20.035 90 25.91 25.594

FVC2004DB2 88.18 32.03 7.852 89.55 28.88 14.396 90.45 27.96 18.69

FVC2004DB3 80 33.44 25.887 80.91 30.74 43.221 82.73 29.52 65.78

FVC2004DB4 85.91 25.56 9.395 87.27 25.52 13.745 88.18 24.14 21.356

�

�

� �

8:

86

9:

96

"::

: 6 ": "6 0: 06 2:

��

�

$ �(

!"#��

��"

��2

��6

(a) CMC curves of CS with gallery G1, G3 and G5for FVC 2004 DB1 database

Figure 4.13: CMC curves of CS with different number of enrolled samples for differentdatabases.

88


�

�

� �

8:

86

9:

96

"::

: 6 ": "6 0: 06 2:

��

�

$ �(

!"#��

��"

��2

��6

(b) CMC curves of CS with gallery G1, G3 and G5for FVC 2004 DB2 database

�

�

� �

8:

86

9:

96

"::

: 6 ": "6 0: 06 2:

��

�

$ �(

!"#��

��"

��2

��6

(c) CMC curves of CS with gallery G1, G3 and G5for FVC 2004 DB3 database

�

�

� �

8:

86

9:

96

"::

: 6 ": "6 0: 06 2:

��

�

$ �(

!"#��

��"

��2

��6

(d) CMC curves of CS with gallery G1, G3 and G5for FVC 2004 DB4 database

Figure 4.13: CMC curves of CS with different number of enrolled samples for differentdatabases (contd.).

89


c) CKS: We have presented the results of CKS with different number of samples enrolledinto the database in Table 4.4. We can see that rank one HR increases for more numberof enrollments without affecting the PR. We can also observe that CKS requires veryless time for large number of samples enrollment.

The performance of CKS is also substantiate with CMS. The CMS at differentranks of cluster kd-tree search with multiple samples enrollment per fingerprint into thedatabase is shown in Fig. 4.14. The results are obtained with the G1, G3 and G5Gallery sets and P Probe set of all FVC 2004 databases. The CMC curves with thisapproach for FVC2004 DB1, DB2, DB3 and DB4 databases are shown in Fig. 4.14 (a),(b), (c) and (d), respectively. We can see that CKS is also performed better when morenumber of samples per fingerprint are enrolled in the database.

Table 4.4: HR, PR and searching time (ST ) in CKS with different number of enrollments(gallery G1, G3 and G5) into the database

DBG1 G3 G5


FVC2004DB1 86.36 11.93 0.426 87.27 12.02 0.706 88.64 12.32 1.041

FVC2004DB2 85.45 13.17 0.485 86.82 13.87 0.55 87.27 14.54 0.768

FVC2004DB3 78.18 14.75 1.57 79.55 15.18 2.121 81.82 15.61 2.885

FVC2004DB4 84.09 10.89 0.349 85 10.88 0.766 85.45 11.29 0.684

�

� �

8:

86

9:

96

"::

: 6 ": "6 0: 06 2:

��

�

$ �(

!"#��

�\��"

�\��2

�\��6

(a) CMC curves of CKS with gallery G1, G3 and G5for FVC 2004 DB1 database

Figure 4.14: CMC curve of CKS with different number of enrolled samples for differentdatabases.

90


�

�

� �

8:

86

9:

96

"::

: 6 ": "6 0: 06 2:

��

�

$ �(

!"#��

�\��"

�\��2

�\��6

(b) CMC curves of CKS with gallery G1, G3 and G5for FVC 2004 DB2 database

�

�

� �

8:

86

9:

96

"::

: 6 ": "6 0: 06 2:

��

�

$ �(

!"#��

�\��"

�\��2

�\��6

(c) CMC curves of CKS with gallery G1, G3 and G5for FVC 2004 DB3 database

�

�

8:

86

9:

96

"::

: 6 ": "6 0: 06 2:

��

�

$ �(

!"#��

�\��"

�\��2

�\��6

(d) CMC curves of CKS with gallery G1, G3 and G5for FVC 2004 DB4 database

Figure 4.14: CMC curves of CKS with different number of enrolled samples for differentdatabases (contd.).

91


4.6.4 Searching Time

In this section, we calculate the searching time complexity of the proposed three searchingtechniques. We also report the execution time and the average number of comparisonsrequired in LS, CS and CKS with different number of enrolled samples for differentdatabases.

a) LS: Let Tn be the number of index keys from N fingerprints enrolled in the databaseand each fingerprint has Tp number of index keys on the average. Let Tq be the averagenumber of index keys in the query fingerprint. In LS, Tn × Tq number of comparisonsare required to retrieve a set of similar index keys with their identities and Tq log Tq

comparisons are required to sort the retrieved identities based on their rank. Thus, wecan calculate the average time complexity of LS (denoted as TLS) as follows.

TLS = Tn × Tq + Tq log Tq

= N × Tp × Tq + Tq log Tq

= O(N)

(4.22)

b) CS: Let Tn index keys are stored in K (K=√

Tn/2) clusters (rule of thumb of clustersfor any unsupervised clustering algorithm [151] for balance between number of clustersand the number of points in a single cluster) and each cluster contains Tk (Tk=

√2Tn)

number of index keys on the average. Thus, Tq ×K number of comparisons are requiredto find the matched clusters. A set of Tq ×Tk similar index keys with their identities areretrieved for a query fingerprint in Algorithm 4.5. The average number of comparisonsrequired to sort the retrieved identities according to their rank is (Tq × Tk) log(Tq × Tk).Hence, the complexity of the CS can be calculated as in Eq. (4.23).

TCS = Tq ×K + (Tq × Tk) log(Tq × Tk)

= Tq ×√

Tn2 + (Tq ×

√2Tn) log(Tq ×

√2Tn)

=Tq×

√Tn√

2[1 + 2 log Tq + log 2 + log Tn]

=Tq×

√N×Tp√2

[1 + 2 log Tq + log 2 + log(N × Tp)]

= O(√N × logN)

(4.23)

c) CKS: In CKS, Tq ×K number of comparisons are needed to find the matched clustersand Tq log Tk number of comparisons are required to retrieve a set of similar index keyswith their identities from the kd-tree for a query fingerprint. The CKS retrieves Tq

number of index keys with their identities (see Algorithm 4.6). On the average Tq log Tq

number of comparisons are needed to sort the retrieved data according to their rank.

92


Therefore, the complexity of the CKS is calculated in Eq. (4.24).

TCKS = Tq ×K + Tq log Tk + Tq log Tq

= Tq ×√

Tn2 + Tq log

√2Tn + Tq log Tq

=Tq√2

[√N × Tp +

1√2log 2 + 1√

2log(N × Tp) +

√2 log Tq

]= O(

√N)

(4.24)

The execution time (in Intel Core-2 Duo 2.00 GHz processor and 2GB RAM im-plementation environment) of LS, CS and CKS with different fingerprint databases areshown in Table 4.5. We observe that the execution time for CKS is less than the othertwo searching methods. It is also observed that there is no significant changes in theexecution time of the CKS when the number of enrolled sample increases. On the otherhand, the execution time rapidly increases for LS and CS for enrollment of multiple sam-ples. The average number of comparisons required for LS, CS and CKS are shown inTable 4.5. Hence, we may conclude that CKS requires the least number of comparisonsto retrieve and rank the similar identities for a given query.

Table 4.5: Execution time and average number of comparisons required in LS, CS andCKS with different number of enrolled samples (ES)

DB ESExecution time (in ms) Average number of comparisons

LS CS CKS LS CS CKS

FV

C20

04D

B1

110 33.102 10.342 0.426 122600 36937 1938

330 109.130 20.035 0.706 404187 71555 3210

550 159.045 25.594 1.041 722931 98437 4165

FV

C20

04D

B2

110 32.693 7.852 0.485 112734 32717 1732

330 77.155 14.396 0.550 335457 59984 2749

550 155.873 18.690 0.768 577309 81263 3489

FV

C20

04D

B3

110 117.725 25.887 1.570 511847 117669 5232

330 356.290 43.221 2.121 1549088 216107 8483

550 671.011 65.780 2.885 2580813 286002 10687

FV

C20

04D

B4

110 26.925 9.395 0.349 117064 32397 1746

330 103.115 13.745 0.766 343716 59760 2734

550 170.812 21.356 0.684 569373 79097 3420

NIST DB4 2000 325.928 25.455 1.193 1303712 110676 4418

NIST DB4Natural

1204 213.304 18.411 0.842 790016 83685 3509

93


4.6.5 Memory Requirements

In our approach, each value of an index key requires 4 bytes memory. There are 8 keyvalues in the each index key and one identity field is required for each index key. Wecan store 216 identities with 2 bytes identity field. Hence, 34 bytes is required to storean index along with identity in linear index space. In clustered index space, we need thememory to store the cluster centroid additional to the linear index space. Each clustercentroid requires 32 bytes memory and 2 bytes memory for cluster id. On the other hand,in clustered kd-tree index space, we need to store the cluster centroid and the kd-treesfor all clusters. Similar to the clustered index space, each cluster centroid requires 32bytes memory and 2 bytes memory for cluster id. A single node of kd-tree requires 46bytes memory. Table 4.6 shows the memory requirements to store different number offingerprints for different databases for linear, cluster and clustered kd-tree index spaces.


There are several work [38,43,51,54,114,136,175,196] for fingerprint identification meth-ods. The most of the existing approaches are tested on the NIST DB4 and NIST DB4

Table 4.6: Memory requirements (in KB) of linear, cluster and clustered kd-tree indexspaces

Indexspace

Fingerprint indatabase

FVC2004DB1

FVC2004DB2

FVC2004DB3

FVC2004DB4

NISTDB4

NIST DB4Natural

Lin

ear

110 108.18 113.12 233.95 120.96 92.7 93.27

330 357.03 336.95 708.46 355.47 278.08 279.84

550 638.73 579.99 1180.44 588.96 463.48 466.4

1204 1014.62 1021.16

2000 1685.26

Clu

ster

110 109.50 114.48 235.91 122.39 93.93 94.50

330 359.46 339.30 711.88 357.9 280.23 281.99

550 641.98 583.08 1184.85 592.08 466.27 469.19

1204 1018.74 1025.28

2000 1690.54

Clu

ster

edkd

-tre

e

110 134.96 141.10 290.96 150.85 115.74 116.44

330 443.46 418.58 878.57 441.54 345.66 347.84

550 792.27 719.55 1462.60 730.66 575.33 578.94

1204 1257.47 1265.55

2000 2087.07

94

4.7. Comparison with Existing Work

Natural database and reported the results with error rate (ERR) at different number oftop matches. To compare our work with existing work, we compute ERR as 100−HR

at different ranks. Here, HR denotes hit rate and defines in Eq. (3.19). Figure 4.15shows the comparison of ERR at different ranks of our three proposed techniques withexisting work. Figure 4.15(a) and (b) report the comparison on NIST DB4 databaseand Fig. 4.16(a) and (b) report the comparison on NIST DB4 Natural database. FromFig. 4.15(a), it is evident that all three proposed techniques outperform other existingapproaches reported (on NIST DB4 database at all penetration rates). Figure 4.15(b)shows that Cappelli et al. (2011) [54] indexing performs better than the proposed in-dexing techniques at low rank (with respect to the result with NIST DB4 database).However, our proposed indexing methods perform better at a little higher rank (top 11rank). From Fig. 4.16(a), it is clear that our proposed approaches outperform otherreported work; although Fig. 4.16(b) shows that Cappelli (2011) [51] and Cappelli et al.

�

�

�

�

�

�

�

� �

:

6

":

"6

0:

06

: ": 0: 2:

.$

$

$ �(

�� * ��40::25 �� M�4"9975

�� , � ��$��40::85 /�� M�40::75

)��/� )��

)��\�

(a) NIST DB4

�

�

�

�

�

:

6

":

"6

0:

06

: ": 0: 2:

.$

$

$ �(

� ��40:""5 � �� M�40:""5

^� �� M�40::15 )��/�

)�� )��\�

(b) NIST DB4

Figure 4.15: Comparison with existing work for NIST DB4 database.

95


�

�

�

�

�

� �

:

6

":

"6

0:

06

: ": 0: 2:

.$

$

$ �(

� �� M�4"9995 ^� �� M�40::15

/�� M�40::65 /�� M�40::15

/�� M�4"9975 )��/�

)�� )��\�

(a) NIST DB4 Natural

�

�

�

�

�

�

:

6

":

"6

0:

06

: ": 0: 2:

.$

$

$ �(

� ��40:""5 � �� M�40:""5

/�� M�40::15 /�� M�40::75

)��/� )��

)��\�

(b) NIST DB4 Natural

Figure 4.16: Comparison with existing work for NIST DB4 Natural database.

(2011) [54] methods perform better than the proposed methods at lower rank for NISTDB4 Natural database. However, proposed approaches are comparable with Cappelli(2011) [51] and Cappelli et al. (2011) [54] methods at a little higher rank (top 13 rank).

Ross and Mukherjee (2007) [175], Liang et al. (2007) [132] and Zhang et al. (2008) [196]

publish the results on FVC 2004 DB1 database. Liang et al. (2007) [132] also publish theresult using Bhanu and Tan (2003) [43] and Bebis et al. (1999) [38] method for FVC 2004DB1 database. Zhang et al. (2008) [196] also report the result of Liu et al. (2007) [137]

method on FVC 2004 DB1 database. We have compared our proposed work with theseexisting works. Figure 4.17 shows the comparison of ERR at different ranks of our threeproposed techniques with existing work. From Fig. 4.17 it is clearly evident that ourapproaches outperform the approaches in [38, 43, 136, 175, 196] method. However, Lianget al. (2007) [132] method is performed better at rank but our methods perform betterfrom at a little higher rank (top 13 rank).

96

4.8. Summary��

�

�

�

:

6

":

"6

0:

06

: ": 0: 2: 3: 6:

.$

$

$ �(

$�� (��%��40::75 /�� M�40::75

�� * ��40::25_ �� M�4"9995_

`� �� M�40::85 /�� M�40::75__

)��/� )��

)��\� _��/� �� M�40::75�

__��`� �� M�40::85�

Figure 4.17: Comparison with existing work for FVC2004 DB1 database.

Table 4.7: Comparison of search complexity and execution time of the existing work

Methods Search complexity Searchtime (ms)

Implementationenvironment

FP inDB

Bhanu and Tan [43] O(N) 1000 Sun Ultra 2 2000

Cappelli [51] O(N +NT logNT ) 1.6 Intel Core 2 Quad 2.66 GHz 2000

Cappelli et al. [55] O(N) 900 Pentium 200MHz 1204

Cappelli et al. [54] O(|V | × L× bs) 16 Intel Quad Core, 2.66 GHz 2700

Jiang et al. [114] O(N) 67 Intel Pentium 4, 2.26 GHz 2000

Proposed LS O(N) 326 Intel Core 2 Duo, 2.00 GHz 2000

Proposed CS O(√N × logN) 25 Intel Core 2 Duo, 2.00 GHz 2000

Proposed CKS O(√N) 1 Intel Core 2 Duo, 2.00 GHz 2000

We also compare our approach with the existing approaches with respect to searchingcomplexity and the execution time. The result is reported in Table 4.7. From the reportedresult we can observe that proposed clustered kd-tree search executes faster than the allother approaches.

4.8 Summary

A new set of eight-dimensional feature vectors for fingerprint indexing has been proposedin this work. The eight-dimensional rotation and scale invariant feature vectors arecalculated from the local topology of the minutiae triplets which are generated by twoclosest point triangulation of minutiae points. The invariance properties of the featurevector make the fingerprint indexing technique accurate for the different fingerprints

97


captured from the different devices and sensors. We explore three different searchingtechniques, LS, CS and CKS with three different index spaces. It has been observed thatLS technique produces the highest HR but this technique is computationally expensivethan the other two approaches. On the other hand, CKS perform faster retrieval thanthe other two searching techniques though the HR is little lower than others. The CSgives moderate hit rate and also takes moderate time for retrieval. The results show thatCS approach have good balance between response time and accuracy. Further, it hasbeen observed that enrollment of multiple samples of a subject improve the performanceof indexing system. Finally, the comparison with the existing indexing techniques showsthat the performance of the proposed approach is better with respect to accuracy andthe searching time.

98

Chapter 5

Face Biometric Data Indexing

Face-based biometric identification system with a large pool of data requires huge com-putation time to search an individual’s identity from database due to the high dimension-ality of the face biometric features. In this work, we propose a new indexing techniquefor face-based biometric identification system to narrow down the search space. In thischapter, we describe the proposed indexing technique for a face biometric data. Our tech-nique is based on the Speed Up Robust Feature (SURF) key points and descriptors [36].We generate index keys for a face image from the key points and descriptors of the faceimage. A two level index space is created based on the extracted key points. A storingand retrieving techniques have been proposed to store and retrieve the face data to orfrom the proposed index space. Figure 5.1 gives an overview of our proposed approach.

The rest of the chapter is organized as follows. Different steps of preprocessingtechnique for face image are outlined in Section 5.1. We discuss the feature extractionmethodology for face in Section 5.2. Index key generation method for face biometric is

)��(��

��

��

��

� ��

� � � ��

*��

� ��

+��

��

��

$��,��

��(��

��

��

�� )��

.��

+��,��

Figure 5.1: An overview of our proposed face biometric data indexing approach.

99

5. Face Biometric Data Indexing

presented in Section 5.3. The storing and retrieving techniques for face data are givenin Section 5.4 and Section 5.5, respectively. Performance of our proposed method withdifferent face databases is analyzed in Section 5.6. Section 5.7 presents the comparisonwith existing work. The chapter is concluded in Section 5.8.

5.1 Preprocessing

Input face image of a biometric system contains background and is usually of not goodquality. To extract the features from a face image, we need to remove the background andenhance the image quality. This makes feature extraction task easy and also ensures thequality of the extracted features. We propose to follow some essential preprocessing tasksfor the above mentioned reason. The steps followed in the preprocessing are describedin the following.

5.1.1 Geometric Normalization

All input face images may not be of same size and aligned in the same direction (e.g. dueto the movement of head at the time of capturing). We follow geometric normalizationprocess proposed by Blome et al. [46] and Beveridge et al. [42] to align and scale theimages so that the face images are in the same position, same size and at the sameorientation. To get geometric normalized image, first, we find out the eye coordinatesfrom the face image using Blome et al. [45, 47] method. Let (XL, YL) and (XR, YR)be the detected left and right eye coordinates, respectively. We calculate the angle (θ)between horizontal axis and the line passing through the two eye coordinates as shownin Fig. 5.2(a). The angle θ is calculated using Eq. (5.1).

θ = tan−1(YR − YLXR −XL

) (5.1)

To lines up the eye coordinates, we rotate the eye image by −θ angle. Let I(x′, y′)be transformed image of I(x, y) after rotation by −θ angle. The rotated image is shownin Fig. 5.2(b). Transformation relation between original image pixel and resultant imagepixel is given in Eq. (5.2).

[x′

y′

]=

[cos(−θ) − sin(−θ)

sin(−θ) cos(−θ)

]×[

x

y

](5.2)

After rotating the face image, we detect the face part from the rotated image. Todo this, we use Viola and Jones’s [187] face detection algorithm. A snapshot of the

100

5.1. Preprocessing

detected face is shown in Fig. 5.2(c). To make the face image scale invariant, we map thedetected face part (DW ×DH) into a fixed size image (FW × FH) by applying scalingtransformation. The scaling factor s is calculated using Eq. (5.3).

s =FW

DW(5.3)

In our approach, we consider FH = 150 and FW = 130, because, in general, aspectratio of a face image would be 1.154 [46]. We map the detected image into fixed sizeimage by applying translation and followed by scaling. Let I(x, y) and I(x′′, y′′) be thedetected and fixed size face image. The transformation is done using Eq. (5.4). InEq. (5.4), (tx, ty) is the top left coordinate of the detected face image (see Fig. 5.2(c)).Figure 5.2(d) shows the fixed image of width FW and height FH.

[x′′

y′′

]=

[s 0

0 s

]×([

x

y

]−[

tx

ty

])(5.4)

R

4X/<�b/5

4X$<�b$5

(a) Face image beforealign

(b) Face image afteralign

4��<��5

�D

�L

(c) Detected faceboundary

�D

�L

(d) Fixed size faceimage

(e) Masked face im-age

(f) Enhanced faceimage

Figure 5.2: Face images after different preprocessing tasks.

101


5.1.2 Face Masking

Masking is the process of separating the foreground region from the background regionin the image. The foreground region correspond to the clear face area containing thesignificant feature value, which is called the area of interest. We have done the maskingto ensure that the face recognition system does not respond to features correspondingto background, hair, clothing etc. We use an elliptical mask [42, 46] such that only theface from forehead to chin and cheek to cheek is visible. Let (xc, yc) is the center of theelliptical mask which is just below the eye level. We apply the following mathematicalmodel [46] to each face image to mask the background region.

Let I(x, y) be the detected face image obtained after geometric normalization. Themasked image I ′(x, y) is calculated as per the following rule.

I ′(x, y) =

⎧⎪⎨⎪⎩

I(x, y) if (x−xc)2

a2+ (y−yc)2

b2− 1 ≤ 0

0 otherwise

where, a and b are minor axis and major axis of elliptical shape, respectively. Figure 5.2(e)shows the face image after applying mask on geometric normalized image where only facepart is present in the image and the background has been masked out.

5.1.3 Intensity Enhancement

Intensity enhancement is required to reduce the variations due to lighting and sensordifferences. We have done the intensity enhancement in two steps [46]. First, we equalizethe histogram of the unmasked face part and then normalize the intensity of the imageto a mean of zero and standard deviation of one.

To equalize the histogram of an image, we calculate the cumulative distribution (cdf)of the histogram. The cdf at each gray level v represents the sum of its occurrence andthe occurrence of the previous gray level in image. Then the histogram equalization isdone by Eq. (5.5).

h(v) = round

(cdf(v)− cdfmin

(FW × FH)− cdfmin

)× (L− 1) (5.5)

where cdfmin is the minimum value of the cumulative distribution function, FW and FH

are the width and height of image, and L is the number of gray levels used.Next, we normalize the intensity values of the equalized face image. Intensity nor-

malization is used to standardize the intensity values in an image by adjusting the rangeof gray level values so that it lies within a desired range of values. Let I(i, j) represents

102


the gray level value at pixel (i, j) of histogram equalized image and N(i, j) representsthe gray level value at pixel (i, j) in normalized image. The normalization is done usingEq. (5.6).

N(i, j) =

⎧⎪⎪⎨⎪⎪⎩

M0 +

√V0×(I(i,j)−M)2

V if I(i, j) > M

M0 −√

V0×(I(i,j)−M)2

V otherwise

(5.6)

where M and V are the estimated mean and variance of I(i, j), respectively, and M0 andV0 are the desired mean and variance values, respectively. We scale the pixel values tohave a mean of zero and a standard deviation of one in our experiment. Figure 5.2(f)shows the intensity enhancement image of the geometric normalization image.


Speed-Up-Robust-Feature (SURF) [36, 37] method is known as a good image interestpoints (also called key points) and feature descriptors detector. We apply SURF methodin our approach. We use this method because it has several advantages over other featureextraction methods. The most important property of an interest point detector usingSURF method is its repeatability. Repeatability means that the same interest points willbe detected in different scales, orientations and illuminations of a given image. Anotheradvantage is that the SURF method is computationally very fast. In addition to these,the SURF feature provides scale, rotation and illumination invariant feature descriptors.

The feature extraction method consists of three different tasks: key point detection,orientation assignment and key point descriptor extraction. In this section, first wepresent the different steps of key point detection method followed by orientation assign-ment of each key point. Then we describe the method of feature descriptor extraction ateach key point.

5.2.1 Key Point Detection

We follow Bay et al. [36,37] method to detect key points from a face image. The methodis based on Hessian-matrix approximation [36] which uses integral images to reduces thecomputation time of key points detection. The method consists of three steps. First, wecreate the scale space of the preprocessed image, which helps us to detect key points atdifferent scales of the image. Next, we calculate Hessian-matrix [36, 134] at each pixelposition in the different scale space images and compute the value of the determinant of

103


the Hessian Matrix. Note that all detected key points are not necessarily discriminantbecause the determinant of the Hessian Matrix does not produce local maximum orminimum response at all detected points. Hence, we localize the most discriminant keypoints from all detected key points. In the following, we present the detailed descriptionsof the above mentioned steps.

5.2.1.1 Scale Space Creation

Scale space of an image is represented by an image pyramid as shown in Fig. 5.3(a). Tocreate an image pyramid, a fixed size Gaussian filter is applied to an image and then subsampled filtered image [108]. This process should be done repeatedly to get the higherlayered images of a scale space. Since, the repeated sub sampling of image demands morecomputation time, we perform this in a different way. We apply Gaussian filter into theimage keeping the image size fixed but increasing the filter size (see Fig. 5.3(b)). Thisgives us the same result of decreasing the size of image and find Gaussian convolutionusing same size of filter and then scale up the image due to the cascading properties ofGaussian filter [36, 108].

Scale space is made of number of octaves; where an octave represents a series offilter responses obtained by convolving the same input image with doubling the filtersize. Again, an octave is divided into a constant number of scale levels. To create scalespace we follow Bay et al. [36] method. We construct eight distinct Gaussian filterswith different sizes and different standard deviations. The Gaussian filter of size m×m

and standard deviation σ is represented as Gm×m(σm) The eight Gaussian filters areG9×9(σ9), G15×15(σ15), G21×21(σ21), G27×27(σ27), G39×39(σ39), G51×51(σ51), G75×75(σ75)and G99×99(σ99) (for details please see the reference [36]). We create base level of scalespace with filter size 9× 9 and standard deviation (σ) 1.2 (also referred as scale s=1.2).

��

��

N�

D��

/�A

��N��

��

(a) Scale space creation by re-ducing image size

��

��

��N�

D��

/�A

��N��

��

(b) Scale space creation by in-creasing filter size

Figure 5.3: Approaches to scale space creation.

104


Next, scale spaces are obtained by changing the size of filter. As we change the filtersize to obtain subsequent scale space we need to change the associate Gaussian scale aswell so that ratios of layout remain constant. We can calculate this standard deviation(scale) of current filter using Eq. (5.7).

σc = σb × SizecSizeb

(5.7)

In Eq. (5.7), σc and σb be the standard deviation of current filter and base filter,respectively, and Sizec and Sizeb denote the current filter size and base filter size, re-spectively.

5.2.1.2 Hessian Matrix Creation

SURF detector [36] is based on determinant of Hessian matrix [36, 134]. We detectblob-like structures which differ in properties like brightness or color compared to thesurrounding at locations where the determinant of Hessian matrix is maximum. Letf(x, y) be a function of two variables, then Hessian matrix H is defined as in Eq. (5.8).

H(f(x, y)) =

[∂2f∂x

∂2f∂x∂y

∂2f∂x∂y

∂2f∂y

](5.8)

Hence, determinant of above matrix will be as follow:

det(H(f(x, y))) =∂2f

∂x

∂2f

∂y−(

∂2f

∂x∂y

)2

(5.9)

We can compute the maxima and minima from the above determinant formula. Thiswill help us to classify the points into maxima and minima based on the sign of result.Let x (x, y) be a point in an image I(x, y). Then Hessian matrix is calculated as follows.

H(x, σm) =

[Lxx(x, σm) Lxy(x, σm)

Lxy(x, σm) Lyy(x, σm)

](5.10)

Here, Lxx(x, σm) = ∂2Gm×m(σm)∂x2 , Lyy(x, σm) = ∂2Gm×m(σm)

∂y2and Lxy(x, σm) = ∂2Gm×m(σm)

∂x∂y .Further, Lxx(x, σm) represents the convolution of the Gaussian second order derivativeof Gaussian filter at scale σm with the image I(x, y) at point x; similarly the Lxy(x, σm)

and Lyy(x, σm). These entire derivatives are known as Laplacian of Gaussian [108]. Wecalculate determinant of each pixel to find the interest points.

To reduce the computation time of calculating Laplacian of Gaussian, we follow [36]

approximation method. The approximation of the Laplacian of Gaussian is done us-

105


ing box filter representations of the respective Gaussian filters. Figure 5.4 shows theweighted box filter approximations of discretised and cropped second order Gaussianderivatives in the x, y and xy-directions. Suppose, Dxx, Dyy and Dxy represent theweighted box filter response [36] of the second order Gaussian derivatives Lxx, Lyy andLxy, respectively. Then Hessian determinant with box filter approximation is calculatedusing Eq. (5.11) [36].

det(Happrox) = DxxDyy − (0.9Dxy)2 (5.11)

The above determinant value is referred to as the blob response at location x = (x, y)with scale σm. We apply this method of Hessian matrix creation for each point of eachscale space for a face given image.

5.2.1.3 Key Point Localization

We detect key points based on the value determinant of the Hessian Matrix. As thehigh determinant value at a point represents more discriminant [37] point, we considerthose points where the determinant values are high. For this reason, first, we choose a

(a) Derivative in x-direction (Lxx)

(b) Derivative in y-direction (Lyy)

(c) Derivative inxy- direction (Lxy)

(d) Approximatedderivative in x- di-rection (Dxx)

(e) Approximatedderivative in y- di-rection (Dyy)

(f) Approximatedderivative in xy- di-rection (Dxy)

Figure 5.4: Gaussian partial 2nd order derivative and their approximation.

106


threshold value and remove the points whose values are less than the threshold. Bayet al. [37] show that the threshold value 600 is good for detecting the discriminant keypoint from an image with average contrast and sharpness. In our experiment, we choosethe threshold value as 600. Next, we perform non-maximal suppression [108] to findthe candidate key points. To do this, we compare each point of scale space with its26 neighbors (see Fig. 5.5) and find the local maxima. We consider 8 points from thescale space at which non-maximal suppression is done and 9 points from both above andbellow the scale space of that. Finally, to localize key points we interpolate the maximaof the determinant of the Hessian matrix in scale and image space. To do this, we usethe method proposed by Brown and Lowe [48].

5.2.2 Orientation Assignment

We assign an orientation to each key point to extract rotation invariant features from theinput face image. The orientation is important because we extract the feature descriptorsrelative to this orientation in later stage. To find the orientation of a key point, first, wecreate a circular area centered with the key point of radius 6σ where σ is the standarddeviation of the Gaussian filter [36] at the current scale space. Then we calculate the Haarwavelet response [113] at each point within the circular area in x and y directions. Weuse Haar filter of size 4σ. The Haar filter responses in x and y directions are calculatedusing two filters as shown in Fig. 5.6(a) and (b). Then, we calculate weighted response ofHaar wavelet response with Gaussian filter. The Gaussian filter is centered with interestpoint and the value σ of Gaussian is taken as 2. The weighted responses are representedby points in vector space as shown in Fig. 5.6(c). To find dominating orientation, wecalculate the resultant vector in window of size 60 degree as shown in Fig. 5.6(c). Thelongest vector is taken as the orientation of that key point.

A set of key points is detected from an image and we estimate orientation of each keypoint. A key point can be represented with position, orientation, scale space in which

��

��

Figure 5.5: Non-maximal suppression by checking the nearest neighbor in subsequent scalespaces.

107


(a) Haar filter in x-direction

(b) Haar filter in y-direction

(c) Orientation assign-ment to a key point

Figure 5.6: Haar filters in x- and y- directions and orientation of a key point.

the key point is detected, Laplacian value and the determinant of Hessian matrix. Letk1, k2, . . ., kL be the L detected key points of an input image. We represent the keypoints of an image as shown in Eq. (5.12).

k1 = x1 y1 θ1 σ ls1 hs1

k2 = x2 y2 θ2 σ ls2 hs2

. . . . . .

kL = xL yL θL σ lsL hsL

(5.12)

In Eq. (5.12), (x, y) and θ represent the position and orientation of a key point,respectively; σ (σ ∈ σ9, σ15, σ21, σ27, σ39, σ51, σ75, σ99) denotes scale space at which keypoint is detected (see Section 5.2.1.1); ls and hs represent the Laplacian value anddeterminant of Hessian matrix, respectively.

5.2.3 Key Point Descriptor Extraction

In this step, we extract the feature descriptors at each key point from the scale spaceimages as follows. Scale space images are created by applying Gaussian filter on theimages (as discussed in Section 5.2.1.1). We follow SURF method [36] to extract thefeature descriptors from the face image. First, we create a square window of size 20σ

where σ is the scale or standard deviation of the Gaussian filter at which key point isdetected. The window is centered at key point position and the direction of window isthe same with the orientation of the key point (see Fig. 5.7). Now, the window is dividedinto 4× 4 square sub regions and within each sub-region 25 (5× 5) regularly distributedsample points are placed. We calculate Haar wavelet responses [113] at each samplepoint of a sub-region in x and y directions. We weight the Haar wavelet responses with

108


f�

fj�j

f�

fj�j

6Q6��

��

Figure 5.7: SURF descriptor extraction at a key point.

a Gaussian filter with standard deviation 3.3σ centered at key point to reduce the effectof geometric deformations and localization errors. Let dx and dy be the Haar waveletresponses at each sample point within each sub-region. We consider

∑dx,

∑ |dx|, ∑ dy

and∑ |dy| as four features at each sub-region. Hence, we create 64 (4×4×4) descriptors

corresponding to each key point. The extracted feature descriptors from a face image isshown in Eq. (5.13). In Eq. (5.13), d1, d2, . . . , di, . . . , dT represent descriptors of T keypoints and fdij represents the jth descriptor of the ith key point.

d1 = fd11 fd12 fd13 . . . fd1j . . . fd164d2 = fd21 fd22 fd23 . . . fd2j . . . fd264. . . . . .

di = fdi1 fdi2 fdi3 . . . fdij . . . fdi64. . . . . .

dT = fdT1 fdT2 fdT3 . . . fdTj . . . fdT64

(5.13)


We extract all key points and feature descriptors from all face images. We can representa face image with a set of index keys. The set of index keys are generated from theextracted key points of a face image such that for each key point there is an index key.More precisely, we use the key point information, feature descriptors and identity of aperson as the constituents of an index key. We represent an index key as a row vector ofsixty nine elements. The first four values of an index key contain the sign of Laplacian,position and orientation of a key point. Next sixty four values of the index key hold thefeature descriptors corresponding to the key point and the last value keeps the identity of

109


a person. The first four values are used to index the database and the feature descriptorsare used to search the identity of a person. Suppose, there are P number of subjects andeach has Q number of samples. We generate index keys for all samples of all subjects. LetTp,q be the number of key points (k1p,q, . . . , k

Tp,qp,q ) and feature descriptors (d1p,q, . . . , d

Tp,qp,q )

extracted from the face image of the qth sample of the pth subject. Note that Tp,q mayvary from one face image to another. Thus, Tp,q number of index keys are generated forthe face image the qth sample of the pth subject. We represent the index keys of the qth

sample of the pth subject in Eq. (5.14). The ith index key (indxip,q) of the qth sample of thepth subject is generated by the ith key point (kip,q) and corresponding feature descriptors(dip,q), and the identity (Idpq) of the qth sample of the pth subject. In Eq. (5.14), lsip,q,xip,q, yip,q, θip,q and dip,q represent the sign of Laplacian, x and y positions, orientation andfeature descriptors of the ith key point (kip,q), and Idpq represents the identity of the faceimage of the qth sample of the pth subject.

indx1p,q = ls1p,q x1p,q y1p,q θ1p,q d1p,q Idpq

indx2p,q = ls2p,q x2p,q y2p,q θ2p,q d2p,q Idpq

. . . . . .

indxip,q = lsip,q xip,q yip,q θip,q dip,q Idpq

. . . . . .

indxTp,qp,q = ls

Tp,qp,q x

Tp,qp,q y

Tp,qp,q θ

Tp,qp,q d

Tp,qp,q Idpq

(5.14)

5.4 Storing

Once all index keys are generated, we have to store the face data into the database.To do this, first, we create a two-level index space in the database. Then we store theface data into the index space in two ways: linear structure and kd-tree structure. Thetechniques of index space creation and storing are described in the following.


We have to create an index space to store all index keys into the database. The createdindex space helps us to find a match corresponding to a query in fast and accuratemanner. To create index space, we use first four components of an index key. These arethe sign of Laplacian (ls), positions (x and y) and orientation (θ) of a key point. Allindex keys can be classified into two groups based on the sign of the Laplacian value (ls)because this value distinguishes the brightness (dark and light) at a key point position.Note that all face images are aligned in the same direction and scaled to the same size

110

5.4. Storing

in the preprocessing step. Hence, a key point will occur at the same or near to thesame position in the image and the orientation of the key point will remain almost samealthough, the face images are captured at different time. So, we can divide the indexkeys in each group into sub-groups based on the positions (x and y) and orientation (θ)of a key point.

Due to the above characteristics of key points we propose a two-level index space tostore the index keys. In the first level, we divide the index space based on the value ofls of index keys. The value of ls can be either ‘−1’ for low intensity value or ‘+1’ forhigh intensity value for a key point. Hence, the first level index space (LS) is dividedinto two sub-index spaces (LS1 and LS2) as shown in Fig. 5.8. In the second level, eachsub-index space is divided into a number of cells based on the positions (x and y) andorientation (θ) of key points. We represent the second level index space (INDX) as athree dimensional index space. The three dimensions of index space are x, y and θ. Eachdimension is in different scales. To bring each dimension in the same scale, we normalizeeach dimension. To do this, we quantize each dimension of the second level index spaceinto the same number of units. Each dimension is quantized into δ number of units. Thevalue of δ is decided experimentally (discussed in Section 5.6.3). We refer each threedimensional index space in the second level as an index cube. Each index cube containsδ3 number of cells. Figure 5.8 shows two three-dimensional index cubes for storing theindex keys.

Now, we store all index keys into the index space based on the first four values ofthe index keys. Note that a number of index keys may map into a single cell of an indexcube because index values of a set of index keys may fall within the same range. We findthe cell positions for all index keys. To do this we define a set of hash functions based onthe sign value of Laplacian, positions and orientation of a key point. Let ls, (x, y) andθ be the sign value of Laplacian, positions and orientation of a key point, respectively.Then the hash functions are defined in Eq. (5.15). In Eq. (5.15), ls′, x′, y′, and θ′ arethe normalized cell index of the two-level index space and FH and FW represent theheight and width of the normalized face image, respectively.

ls′ = Hls(ls), where Hls(ls) = ls+32

x′ = Hx(x), where Hx(x) = (x+1)×δFW �

y′ = Hy(y), where Hy(y) = (y+1)×δFH �

θ′ = Hθ(θ), where Hθ(θ) = (θ+1)×δ360 �

(5.15)

We illustrate the storing of an index key into the proposed two-level index space withan example. Let indx =< ls = −1, x = 55, y = 89, θ = 45, fd1, fd2, . . . , fd64, Id > be

111


/�

" 0 2 J

"02

J

"0

2

J

�'�X

" 0 2 J

"02

J

"02

J

�'�X

/�"

"

/�0

0

��,��

��,��

��

R

��

R

Figure 5.8: Proposed index space to store all index keys of all face images.

an index key of a face image of size 130 × 150 (FW × FH) , fd1 to fd64 are the 64feature descriptors of that key point and Id be the identity of the subject. To storethe feature descriptors (fd1, fd2, . . . , fd64) and the identity (Id) into the index space,we apply the hash functions (defined in Eq. (5.15)) on ls, x, y and θ of the key point.The feature descriptors will be stored into the first index cube (LS1) of the first level ofindex space because the value of ls′ is 1 after applying the hash function on ls. The cellposition of the index cube in second level index space (INDX) is decided by applyingthe hash function on x, y and θ. Let us assume that each dimension of second level indexspace is divided into 15 units (i.e. δ = 15). After applying the hash function on x, y andθ the value of x′, y′ and θ′ are 7, 9 and 2, respectively. Hence, the feature descriptors(fd1, fd2, . . . , fd64) and identity (Id) of the index key (indx) is stored at [7,9,2] locationin the first index cube which is represented as LS1 → INDX[7][9][2].

We may note that a cell of an index cube can contain a set of feature descriptorsand identities of index keys. To store the feature descriptors and identities of index keys,we propose two storing structures: linear storing structure and kd-tree storing structure.These storing structures are discussed in the following.

112

5.4. Storing

5.4.2 Linear Storing Structure

In this technique, we create a 2-D linear index space (LNINDX) for each cell of theindex cube. Each linear index space is assigned a unique identity (lid) and this identityis stored into the corresponding cell of the index cube. Note that there are δ3 number ofcells in each index cube. Hence, 2δ3 number of linear index spaces are created to storeall index keys using linear storage structure. The linear index space (LNINDX) storesthe 64-dimensional feature descriptors (fd1, fd2, . . . , fd64) and identities (Id) of indexkeys. Figure 5.9 shows the linear index space (LNINDXi) for the ith cell of the firstindex cube (LS[1] → INDX). The cell stores the identity (lidi) of the linear index space(LNINDXi). The ith cell of the index cube LS[1] → INDX is represented as CELL[i].We store all index keys of the ith cell into the linear index space (LNINDXi). Hence,the size of the linear index space (LNINDXi) is Ni × (64 + 1) where Ni is the numberof index keys in the ith cell and an index key contains 64-dimensional feature descriptor.The method for creating linear index space is summarised in Algorithm 5.1. In Step 7 ofAlgorithm 5.1, we find the index cube from the first level index space. The cell positionof the index cube is found in Step 8 to Step 10. Step 11 calculates the identity of linearindex space and Step 12 assigns that identity to a cell of an index cube. We copy thedescriptor values and the identities of index keys in Step 13 and 14.

5.4.3 Kd-tree Storing Structure

We create a kd-tree for each cell of an index cube and assign a unique identity (kid)to each kd-tree. The identity of the kd-tree is stored into the corresponding cell of theindex cube. There are 2δ3 number of cells in the index space. Hence, the total numberof kd-trees required is 2δ3. All index keys of a cell are stored into a kd-tree. A kd-tree isa data structure for storing a finite set of points from a k-dimensional space [34,35,160]

and the structure is defined in Section 4.4.3.

/'�'�X�

2

�" �0 �2 MMM �13 �

"

0

%

'�U"

'�

��

/�Y"Zq�'�XY�ZY�ZYRZ�

4�.//Y�Z5

Figure 5.9: Linear index space to store all index keys of the ith cell.

113


Algorithm 5.1 Creating index space with linear storing structure

Input: All index keys of all person’s face image (indx11,1, indx21,1, . . ., indxT1,1

1,1 ; indx11,2,

indx21,2, . . ., indxT1,2


P,Q ). Two level index space(LS[ ] → INDX[ ][ ][ ]).

Output: Index space (LS[ ] → INDX[ ][ ][ ]) with linearly stored index keys for eachcell (CELL[ ] → LNINDX[ ][ ])

1: for c = 1 to 2× δ3 do2: inc[c] = 1 // Initialize linear index counter3: end for4: for p = 1 to P do5: for q = 1 to Q do6: for i = 1 to Tp,q do7: ls = Hls(ls

ip,q) // Decide first level index space

// Decide cell location of second level index space8: x = Hx(x

ip,q)

9: y = Hy(yip,q)

10: θ = Hθ(θip,q)

11: lid = (ls− 1)× δ3+(x− 1)× δ2+(y− 1)× δ+ θ // Calculate linear index space id12: LS[ls] → INDX[x][y][θ] = lid // Copy id of linear index space into a cell13: CELL[lid] → LNINDX[inc[lid]] = dip,q // Copy descriptor values of index key

into linear index space14: CELL[lid] → LNINDX[inc[lid]][65] = Idpq // Copy identity of person into lin-

ear index space15: inc[lid] = inc[lid] + 1 // Increment linear index counter16: end for17: end for18: end for

We store all sixty four dimensional points (descriptors of an index key) within a cellinto a kd-tree data structure. To store the index keys we apply the method proposedby Arya and Mount [34, 35] which follows Bentley [40] kd-tree insertion method. Themaximum height of the optimized kd-tree with N number of k-dimensional point is�log2(N)� [40]. The kd-tree structure for the ith cell is shown in Fig. 5.10. In Fig. 5.10,KDINDXi is the kd-tree for the ith cell of the first index cube (LS[1] → INDX). Thecell stores the identity (kidi) of the ith kd-tree (KDINDXi). The ith cell of the firstindex cube LS[1] → INDX is represented as CELL[i]. We summarize the method forcreating kd-tree index space in Algorithm 5.2. Step 7 of Algorithm 5.2 finds the positionof first level index space and Step 8 to Step 10 calculate the cell position in the indexcube. In Step 11, we calculate the unique identity of the kd-tree. We assign the identity

114

5.4. Storing

of the kd-tree into a cell of an index cube in Step 12. Finally, we insert the descriptorvalues and the identity of a subject into the kd-tree in Step 15.

\��'�X�

/�Y"Zq�'�XY�ZY�ZYRZ�

4�.//Y�Z5

\��'�X

(��

Figure 5.10: Kd-tree to store the index keys of the ith cell of the index space.

Algorithm 5.2 Creating index space with kd-tree storing structure

Input: All index keys of all person’s face image (indx11,1, indx21,1, . . ., indx

T1,1

1,1 ; indx11,2,

indx21,2, . . ., indxT1,2


P,Q ). Two level index space(LS[ ] → INDX[ ][ ][ ]).

Output: Index space (LS[ ] → INDX[ ][ ][ ]) with kd tree stored index keys for eachcell (CELL[ ] → KDINDX).

1: for c = 1 to 2× δ3 do2: inc[c] = 1 // Initialize linear index counter3: end for4: for p = 1 to P do5: for q = 1 to Q do6: for i = 1 to Tp,q do7: ls = Hls(ls

ip,q) // Decide first level index space

// Decide cell location of second level index space8: x = Hx(x

ip,q)

9: y = Hy(yip,q)

10: θ = Hθ(θip,q)

11: kid = (ls− 1)× δ3+(x− 1)× δ2+(y− 1)× δ+ θ // Calculate kd-tree id12: LS[ls] → INDX[x][y][θ] = kid // Copy id of the kd-tree into a cell13: Temp[ ] = dip,q // Copy descriptors of index key into temporary vector14: Temp[65] = Idpq // Copy identity of person into the temporary vector15: Insert Temp into Kd-treekid16: inc[kid] = inc[kid] + 1 // Increment kd-tree index counter17: end for18: end for19: end for

115


5.5 Retrieving

Querying is the process of retrieving a set of candidates from the enrolled face templatescorresponds to a query template. The retrieved templates are most likely to match withthe query face template. We investigate two different searching techniques to retrieve theface templates from the database of two different storage structures discussed previously.In each searching technique, first we generate the index keys corresponding to the queryface using index key generation technique (discussed in Section 5.3). Let the index keysgenerated from a query face are represented in Eq. (5.16).

indx1t = ls1t x1t y1t θ1t d1t

indx2t = ls2t x2t y2t θ2t d2t

. . . . . .

indxit = lsit xit yit θit dit

. . . . . .

indxLtt = lsLt

t xLtt yLt

t θLtt dLt

t

(5.16)

The ith query index key is represented as indxit = lsit, xit, y

it, θ

it, d

it, where lsit, xit, yit

and θit, dit represent the sign of Laplacian, x and y position, orientation, feature descriptorof the ith key point (kit) of the query face image, respectively. Then, we apply indexingon the first level index space using the value of ls of the query index key. The indexingis done using hash functions defined in Eq. (5.15). The first level of indexing selects theindex cube for a query index key. Let us assume that the value of ls of the ith index keyof query is −1. Then index cube (LS[1] → INDX) in the first level index space (LS) isselected for the ith query index key. Next, we apply hash functions (defined in Eq. (5.15))on the value of x, y and θ of the query index key to find the cell position of the indexcube in the second level index space. Then, the candidate set (CSET ) is generated bycounting the vote received for each identity of the retrieved index keys from the database.The CSET contains two fields: id and vote. The id and vote fields of the CSET storethe identities of subjects and the number of votes received for an identity. To generatethe CSET , we search the linear or kd-tree structures corresponding the identity stored inthe selected cell of an index cube and find the closest match in the linear or kd-tree indexspace. If x, y and θ of the ith index key of a query select the LS[1] → INDX[x][y][θ]

cell of the index cube LS[1] → INDX and retrieve the jth linear or kd-tree identitythen we find the closest match in the LNINDXj linear index space for linear search andKDTREEj for kd-tree search. Finally, ranks are calculated based on the votes receivedby each identity. The search techniques are described in the following.

116

5.5. Retrieving

5.5.1 Linear Search

In linear search, first we find the cell position in an index cube for a query index key. Then,we search the linear index space of that cell. We compute Euclidean distance betweenfeature descriptor of a query index key and all the feature descriptors stored in the linearindex space to find a match. Let the jth cell in the index cube is selected for the ith

index key of a query. Then, we select the linear index space (CELL[j] → LNINDX)corresponding to the jth cell to find a match. We compute the Euclidean distancesbetween the feature descriptors of the ith index key of the query and all the descriptorsstored in the linear index space (CELL[j] → LNINDX) using Eq. (5.17). We retrievethe identity corresponding to the minimum distance. The retrieved identity is thenplaced in the CSET and cast a vote for this identity. We follow the same procedurefor all other index keys of the query face. We summarize the linear searching method inAlgorithm 5.3. In Step 2 of Algorithm 5.3, we initialize the length of each linear indexspace. Step 6 and Step 7 to 9 find the index of the first and second level index spaces,respectively. We calculate the cell id in Step 10. In Step 11 to 19, we find the minimumdistance for an index key of query face and retrieve the identity corresponding to theminimum distance. Step 23 to 31 generate the CSET for a query index key. Finally, wesort the CSET in Step 35.

EDi,j = EuclidDist(dj , dit),

where, EuclidDist(dj , dit) =∑64

f=1(fdjf −t fd

if )

2(5.17)

5.5.2 Kd-tree Search

In kd-tree search, first we find the cell position in an index cube for an index key ofa query. Then we retrieve the identity of a kd-tree (kid) from the cell and search thekd-tree corresponding to the retrieved kd-tree identity. We apply hash functions to findthe cell position in the index cube. Let the jth cell in the index cube is selected forthe ith query index key. Then we search kd-tree index space (CELL[j] → KDINDX)corresponding to the jth cell to find a match. We apply approximate nearest neighborsearch [15,34,35,162] to reduce the searching time. Arya and Mount’s [15,34] approximatek nearest neighbor search method is used to search the kd-tree. In this technique, weexamine only the k closest bins of the kd-tree and use a priority queue to identify theclosest bins based their distances from query. The expected searching complexity of thenearest neighbor search can be reduced to O(kdlogn) and space complexity is O(dn). Forthis purpose, a public domain library (FLANN) [15, 162] for faster approximate nearest

117


Algorithm 5.3 Candidate set generation in linear search from index space

Input: All index keys from query face image (indx1t , indx2t , . . ., indxLtt ), index space

(LS[ ] → INDX[ ][ ][ ]) with linearly stored index keys for each cell (CELL[ ] →LNINDX[ ][ ]).

Output: Candidate Set (CSET [ ] → (id, vote))

1: for cellid = 1 to 2× δ3 do // Initialize length in each linear index space2: KEY S[cellid] = Number of keys in CELL[cellid] → LNINDX[ ][ ]3: end for4: idc = 15: for i = 1 to Tt do // Tt is the total number of query index key6: ls′ = Hls(ls

it) // Find first level index space

// Find cell location of second level index space7: x′ = Hx(x

it)

8: y′ = Hy(yit)

9: θ′ = Hθ(θit)

10: cellid = LS[ls] → INDX[x′][y′][θ′] // Calculate cell id of an index cube// Retrieve the matched identities from CELL[cellid] → LNINDX[ ][ ]

11: MinDist = ∞12: for j = 1 to KEY S[cellid] do // KEY S[cellid] is the total number of index key

in the cellidth cell13: EDi,j = EucleadDist(CELL[cellid] → LNINDX[j], dit)14: if EDi,j ≤ MinDist then // Find match identities corresponding to the minimum

distance15: m = 116: MatchId[m] = CELL[cellid] → LNINDX[j][65]17: else if EDi,j = MinDist then18: m = m+ 119: MatchId[m] = CELL[cellid] → LNINDX[j][65]20: end if21: end for22: for j = 1 to m do23: id = MatchId[m]24: subid = SubjectId(id) // Copy subject id from id25: if subid /∈ CSET [ ] → (id) then // Generate candidate set26: CSET [idc] → id = subid27: CSET [idc] → vote = 128: idc = idc+ 129: else30: idx = IdIndex(CSET [], subid) // Return index of subject id in CSET

31: CSET [idx] → vote = CSET [idx] → vote+ 132: end if33: end for34: end for35: Sort CSET [ ] → (id, vote) in descending order based on vote

118

5.5. Retrieving

neighbors search is available. In our approach, we utilize this library for implementingkd-tree algorithms. We retrieve the identities corresponding to the closest match fromthe kd-tree. The retrieved identities are then placed in the CSET and cast a vote forthis identity. We follow the same procedure for all other index keys of the query face.The searching method for kd-tree index space is summarized in Algorithm 5.4. Step 3and Step 4 to 6 of Algorithm 5.4 find the index of the first and second level index spaces,respectively. In Step 7, we calculate the cell identity of an index cube for a query indexkey. Step 8 finds the approximate nearest neighbors for a query index key and Step 9retrieves the identities corresponding to the nearest neighbors. In Step 12 to 17, wegenerate the CSET for a query index key and sort the CSET in Step 20.

Algorithm 5.4 Candidate set generation in kd-tree search from index space

Input: All index keys from query face image (indx1t , indx2t , . . ., indxLtt ) with linearly

stored index keys for each cell (CELL[ ] → KDINDX[ ]).Output: Candidate Set (CSET [ ] → (id, vote))

1: idc = 12: for i = 1 to Lq do // Lq is the total number of query index key3: ls′ = Hls(ls

it) // Find first level index space

// Find cell location of second level index space4: x′ = Hx(x

it)

5: y′ = Hy(yit)

6: θ′ = Hθ(θit)

7: cellid = LS[ls] → INDX[x][y][θ] // Calculate cell id of an index cube

// Retrieve the matched identities from Kd-tree (Kd-tree[cellid])8: NN = findANN(Kd-tree[cellid], dit) // Find approximate nearest neighbors9: id = retrieveIdFromNN(NN) // Select id of nearest neighbor

10: subid = SubjectId(id) // Copy subject id from id// Generate candidate set

11: if subid /∈ CSET [ ] → (id) then12: CSET [idc] → id = subid13: CSET [idc] → vote = 114: idc = idc+ 115: else16: idx = IdIndex(CSET [], subid) // Return index of subject id in CSET

17: CSET [idx] → vote = CSET [idx] → vote+ 118: end if19: end for20: Sort CSET [ ] → (id, vote) in descending order based on vote

119



We have evaluated our proposed method with different face databases in different exper-imental setups to measure the performance of our proposed face indexing approach. Theperformance metrics are defined in Section 3.7.1. In this section, first we describe theface databases used in our experiment and then different experimental setups which areused. Then, we present experimental results observed in our experiments with respectto the different performance metrics defined in Section 3.7.1. We also give a comparisonof our approach with existing face indexing approaches in this section.

5.6.1 Database

We perform our experiments on two widely used large face databases, namely ColorFERET [12, 20, 168] and FRGC V2.0 [21, 166, 167]. We also carry out our experimentson CalTech 256 [4, 85] face database. Detailed description of each database is given inthe following.

Color FERET Face Database: The FERET database is developed for the FacialRecognition Technology (FERET) program [12, 20]. The database is designed by theDefense Advanced Research Products Agency (DARPA) [20] during 1993 to 1997 to givecommon standard for face recognition experiments. The database contains 11338 imagesfrom 994 different subjects. These images are collected in different sessions. The res-olution of the captured images is 256 × 384 pixel. The database contains 2722 frontalimages with different facial expressions (Neutral and Alternate). There are 1364 imageswith neutral expression and 1358 images with alternate expression. Figure 5.11(a) and(b) shows the four images with different facial expressions of two different subjects.

FRGC 2.0 Face Database: FRGC Still version 2.0 data set [21, 166, 167] is collectedat University of Notre Dame as a part of Face Recognition Grand Challenge (FRGC)program. The primary goal of the FRGC program is to promote and advance the facerecognition technology. This database contains color face images, which are taken indifferent lightning conditions and environments. The database consists of 24038 frontalface images of 466 subjects. These images are captured in Fall 2003 and Spring 2004semesters of 2003-2004 academic year. A total of 16024 images from all subjects arecaptured in indoor environment with two different protocols (FERET and Mugshot)and two different facial expressions (Neutral and Smiley) [166]. The resolution of eachimage is either 1704 × 2272 pixel or 1200 × 1600 pixel. The images are collected in4007 subject sessions. Four images (FERET-Neutral, FERET-Smiley, Mugshot-Neutral

120


and Mugshot-Smiley) are captured in each subject session. The database contains 4007FERET-Neutral, 4007 FERET-Smiley, 4007 Mugshot-Neutral and 4007 Mugshot-Smileyface images. Figure 5.11(c) and (d) shows four images with two facial expressions of twodifferent subjects. FRGC Still version 2.0 data set [21,166,167] contains 8014 face imageswhich are captured in outdoor environment with different backgrounds and illuminations.Figure 5.11(e) shows two face images of two different subjects in different backgrounds.

CalTech 256 Face Database: Caltech-256 object category data set [4, 85] containsa total of 30607 images from 256 different categories. In our experiment, we use facecategory images of the Caltech-256 data set. The face category set consists of 432 faceimages from 28 subjects. Each face image is captured in complex background withdifferent facial expressions. Figure 5.11(f) shows two face images of two different subjectsfrom CalTech 256 face database.


To evaluate our proposed indexing method, we have partitioned each face database intotwo sets: Gallery and Probe. The Gallery set contains the face images which are enrolledinto the index database and Probe set contains the face images which are used as queries

(a) Frontal face im-age samples without ex-pression from FERETdatabase

(b) Frontal face imagesamples with expressionfrom FERET database

(c) Frontal face imagesamples without ex-pression from FRGCdatabase

(d) Frontal face imagesamples with expressionfrom FRGC database

(e) Sample face imagesin outdoor environmentfrom FRGC database

(f) Sample face imagesfrom CalTech database

Figure 5.11: Sample images from FERET, FRGC and CalTech256 databases.

121


to search the index database. In our experiment, we create different Gallery and Probe

sets for each database. The descriptions of different Gallery and Probe sets for FERET,FRGC and CalTech256 databases are given in Table 5.1.

All methods described in our approach are implemented using C programming lan-guage and OpenCV [22] image processing library in the Linux operating system. Allmethods are evaluated with Intel Core-2 Duo processor of speed 2.00GHz and 2GBRAM.

Table 5.1: Description of Gallery and Probe sets of FERET, FRGC and CalTech256 facedatabases.

Data-base

Name # images#

subjectsDescription

FE

RE

T

Gallery11 994 994First face image with neutral facial expression of first session forall subjects.

Gallery12 994 994First face image with neutral facial expression of first session forall subjects.

Probe11 992 992First face image with alternate facial expression of first sessionfor all subjects.

Probe12 370 250Face images with neutral facial expressions of other sessions forall subjects.

Probe13 366 247Face images with alternate facial expressions of other sessionsfor all subjects.

Probe14 736 250Face images with neutral and alternate facial expressions ofother sessions for all subjects.

Probe15 228 75Face images with neutral and alternate facial expressions ofother sessions for all subjects. But images are captured withminimum twelve months difference.

FR

GC

Gallery21 466 466First face image with neutral facial expression of first session forall subjects. Images are captured with FERET protocol.

Gallery22 932 466First face image with neutral and smiley facial expressions offirst session for all subjects. Images are captured with FERETprotocol.

Probe21 466 466First face image with smiley facial expression of first session forall subjects. Images are captured with FERET protocol.

Probe22 3541 411Face images with neutral facial expressions of other sessions forall subjects. Images are captured with FERET protocol.

Probe23 3541 411Face images with smiley facial expressions of other sessions forall subjects. Images are captured with FERET protocol.

Probe24 7082 411Face images with neutral and smiley facial expressions of othersessions for all subjects. Images are captured with FERET pro-tocol.

Probe25 1134 193

Face images with neutral and smiley facial expressions of othersessions for all subjects. Images are captured with FERET pro-tocol. The time difference from first captured image is minimumsix months.

Probe26 466 466First face image with neutral facial expression of first session forall subjects. Images are captured with Mugshot protocol.

Continued to next page

122


Table 5.1 – continued from previous pageData-base

Name # images#

subjectsDescription

Probe27 466 466First face image with smiley facial expression of first session forall subjects. Images are captured with Mugshot protocol.

Probe28 3541 411Face images with neutral facial expressions of other sessions forall subjects. Images are captured with Mugshot protocol.

Probe29 3541 411Face images with smiley facial expressions of other sessions forall subjects. Images are captured with Mugshot protocol.

Probe30 7082 411Face images with neutral and smiley facial expressions of othersessions for all subjects. Images are captured with Mugshotprotocol.

Probe31 1134 193

Face images with neutral and smiley facial expressions of othersessions for all subjects. Images are captured with Mugshot pro-tocol. The time difference from first captured image is minimumsix months.

Probe32 8014 466 Face images with at outdoor environment.

Cal

Tec

h

Gallery41 28 28One face image from each subject. Face image is selected ran-domly.

Gallery42 350 26Eighty percent face images of each subject. Face images areselected randomly.

Probe41 26 26One face image from each subject. Face image is selected ran-domly from the rest of the Gallery41.

Probe42 404 26 All face images of each subject except the Gallery41 face images.Probe43 82 26 All face images of each subject except the Gallery42 face images.

5.6.3 Validation of the Parameter Value

The experimental results we will be reported in this chapter is subjected to the validityof the value of the parameter used. The value of dimension quantization of the secondlevel index space (δ) is considered as an important parameter in our experiment. Thevalidity is assessed on the basis of experimental evaluation of the variable. We validatethe value of δ in the following.

To determine the value of the number of quantization (δ) of each dimension of thesecond level index space, we have done the following experiment. We perform kd-treebased search for a set of query images with different values of δ. The value of δ is variedfrom 2 to 50 with increment of 2. This experiment is conducted with FERET, FRGCand CalTech databases. We use Gallery11 and Probe11 for FERET database, Gallery21

and ProbeP21 for FRGC database, and Gallery41 and Probe41 for CalTech database.The rank one HR and PR for different values of δ as observed are shown in Fig. 5.12.From Fig. 5.12(a), we observe that for FERET database rank one HR decreases nearly2% when the value of δ is changed 2 to 14 whereas rank one HR decreases more than4% when the value of δ is changed 16 to 20. On the other hand, the PR decreases morethan 85% when the value of δ is changed 2 to 14 but PR decreases only 5% for the

123

5. Face Biometric Data Indexing�

�

�

�

�

� �

:

0:

3:

1:

8:

"::

: ": 0: 2: 3: 6:

w

J

�.$.*�4� ��""U)��""5

)$

D$

(a) HR and PR with Gallery11 andProbe11 for different values of δ

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

:

0:

3:

1:

8:

"::

: ": 0: 2: 3: 6:

w

J

�$��4� ��0"U)��0"5

)$

D$

(b) HR and PR with Gallery21 andProbe21 for different values of δ

�

�

�

� �

:

0:

3:

1:

8:

"::

: ": 0: 2: 3: 6:

w

J

� �*��4� ��3"U)��3"5

)$

D$

(c) HR and PR with Gallery41 andProbe41 for different values of δ

Figure 5.12: HR and PR with FERET, FRGC and CalTech256 databases for differentvalues of δ.

change of δ value 14 to 20 for FERET database. The similar trend is observed for FRGCdatabase also (see Fig. 5.12(b)). On the other hand, the value of δ equal to 12 givesbetter performance than the other values of δ for Caltech database (see Fig. 5.12(c)).Hence, in our other experiments, we choose the value of δ equal to 15 for FERET andFRGC databases, and 12 for Caltech database, respectively. However, user may choosethe other values of δ according to their requirements.

5.6.4 Evaluation

We have conducted a number of experiments to evaluate the accuracy, searching timeand memory requirement of our proposed face indexing method. The descriptions ofthese experiments and the result of each experiment are given in the following.

124


5.6.4.1 Accuracy

The accuracy is measured with respect to HR, PR and CMS metrics. In the first experi-ment, we show the performance of proposed method with and without applying indexing.Next, we present the performances of the proposed indexing method with different Probe

sets. The effect of multiple number of samples enrollment into the database is shownin the next experiment. Finally, we measure the FPIR and FNIR for the face-basedidentification system with the proposed indexing technique.

Performances with and without indexing: In this experiment, we compare the per-formance of the system with and without applying the proposed indexing technique. Weuse Gallery11 and Probe11 for FERET, Gallery21 and Probe21 for FRGC databases,and Gallery41 and Probe41 for CalTech databases. The CMC curves with and withoutindexing for different databases are shown in Fig. 5.13. Figure 5.13(a) shows that theapproach without indexing gives better cumulative match score for FERET database.Whereas, from Fig. 5.13(b) and (c), we can see that the approach with and withoutindexing gives almost the same CMS after the 15th rank for FRGC database and afterthe 7th rank for CalTech database. We have also reported the rank one HR, PR andaverage searching time (ST ) for linear and kd-tree search using indexing and withoutindexing in Table 5.2. From Table 5.2, we can see that kd-tree search with indexingachieves 95.57%, 97% and 92.31% HR with 9.70%, 12.55% and 7.14% PR for FERET,FRGC and CalTech databases, respectively. We also observe that kd-tree search requiresless average searching time for all three databases.

Table 5.2: Performance of the proposed approach with and without indexing using linearand kd-tree search

Database PerformanceLinear Kd-tree

Withoutindexing

Withindexing

Withoutindexing

Withindexing

FERETHR 97.28 95.57 97.28 95.57

PR 100 10.54 49.43 7.90

ST (ms) 4.46× 105 114.00 2.21× 105 85.40

FRGCHR 98.93 97.00 98.93 97.00

PR 100.00 16.36 51.47 12.55

ST (ms) 2.78× 105 92.55 1.43× 105 71.02

CalTechHR 96.15 92.31 96.15 92.31

PR 100 26.91 61.78 23.72

ST (ms) 1.24× 104 8.10 7.64× 103 7.14

125

5. Face Biometric Data Indexing�

�

�

�

� �

96

91

97

98

99

"::

: ": 0: 2: 3: 6:

��

�

$ �(

�.$.*�4� ��""U)��""5

L��

L��

(a) CMC curve with and without indexingfor FERET Gallery11 and Probe11 sets

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

91

98

"::

: ": 0: 2: 3: 6:

��

�$ �(

�$��4� ��0"U)��0"5

L��

L��

(b) CMC curve with and without indexingfor FRGC Gallery21 and Probe21 sets

�

�

�

�

�

�

� �

9:

96

"::

: 0 3 1 8 ":

��

�

$ �(

� �*��061�4� ��3"U)��3"5

L��

L��

(c) CMC curve with and without indexingfor CalTech256 Gallery41 and Probe41 sets

Figure 5.13: CMC curve with and without indexing for FERET, FRGC and CalTech256databases.

Performance with different probe sets: In this experiment, we check the perfor-mance of linear and kd-tree search with the proposed indexing method for different Probe

sets. The Probe sets are created with different conditions as discussed in Table 5.1. Weenroll the all images of Gallery11 (with neutral expression), Gallery21 (with neutralexpression) and Gallery41 into the database for FERET, FRGC and CalTech databases,respectively and use all probe sets to test the indexing performances of linear and kd-treesearch. Figure 5.14(a), (b) and (c) show the CMC curves of all probe sets of FERET,FRGC, CalTech256 databases, respectively. From Fig. 5.14(a) and (b), we can note thatthe CMSs are reduced for the Probe sets which contain the face images captured inmore than six month difference. On the other hand, face images captured with differentexpressions but in the same session give the better results than the others. We observe

126


�

�

� �

8:

86

9:

96

"::

: ": 0: 2: 3: 6:

��

�

$ �(

�.$.*�4� ��""5

)��"" )��"0

)��"2 )��"3

)��"6

(a) CMC curves for different Probe sets withFERET Gallery11

�

�

� �

76

8:

86

9:

96

"::

: ": 0: 2: 3: 6:

��

�

$ �(

�$��4� ��0"5

)��0" )��00 )��02)��03 )��06 )��01)��07 )��08 )��09)��2: )��2" )��20

(b) CMC curves for different Probe sets withFRGC Gallery21

�

�

�

�

�

�

�

� �

9:

96

"::

: 6 ": "6 0: 06

��

�

$ �(

� �*��061�4� ��3"5

)��3"

)��30

)��32

(c) CMC curves for different Probe sets withCalTech256 Gallery41

Figure 5.14: CMC curves for different Probe sets with single enrolment of a subject withFERET, FRGC and CalTech256 databases.

that face images with complex background (Probe32 of FRGC database) give less CMS

than the others. However, face images with complex background for CalTech databasegive above 90% CMS. We have also given the rank one HR, PR and ST for linear andkd-tree search for different Probe sets in Table 5.3. We observe that the PR and ST forkd-tree based search are less for all Probe sets.

Performance of multiple enrolments of a subject into the index space: We havedone the experiment to check the effect of multiple enrolments on the performance. Inthis experiment, we enroll all samples of Gallery12 for FERET, Gallery22 for FRGC

127


Table 5.3: Performance of different Probe sets with single enrolment of a subject in linearand kd-tree search

DB Probe setLinear Kd-tree

HR PR ST (ms) HR PR ST (ms)

FE

RE

T

Probe11 95.57 10.54 114.00 95.57 7.90 85.40

Probe12 87.44 11.13 121.98 87.44 8.34 91.38

Probe13 86.11 11.28 122.32 86.11 8.45 91.64

Probe14 86.75 11.20 121.29 86.75 8.39 90.87

Probe15 82.44 10.62 114.29 82.44 7.96 85.62

FR

GC

Probe21 97.00 16.36 92.55 97.00 12.55 71.02

Probe22 90.40 14.66 82.48 90.40 11.25 63.29

Probe23 80.41 16.13 92.01 80.41 12.38 70.60

Probe24 85.40 15.40 87.66 85.40 11.81 67.27

Probe25 87.31 15.69 88.48 87.31 12.04 67.89

Probe26 98.07 14.62 81.73 98.07 11.22 62.71

Probe27 96.36 16.20 91.73 96.36 12.43 70.39

Probe28 89.70 14.41 81.72 89.70 11.06 62.71

Probe29 79.39 16.07 91.24 79.39 12.33 70.01

Probe30 84.54 15.24 84.86 84.54 11.69 65.12

Probe31 85.46 15.42 86.88 85.46 11.83 66.67

Probe32 81.40 15.47 86.38 81.81 11.87 66.28

Cal

Tec

h Probe41 92.31 26.91 8.19 92.31 23.72 7.22

Probe42 94.55 27.52 8.29 94.55 24.27 7.31

Probe43 93.90 27.72 8.48 93.90 24.44 7.47

and Gallery42 for CalTech databases, and test with all Probe sets of FERET, FRGCand CalTech databases. The CMC curves of all Probe sets are shown in Fig. 5.15. FromFig. 5.15, we can see that 100% CMS is achieved for Probe11 and Probe21 becausethe images in the Probe11 and Probe21 sets are also in the Gallery12 and Gallery22,respectively. We observe that in multiple enrolments of a subject, CMSs for other probesets are increased than that of the single enrolments. We have computed the PR, rankone HR and searching time for linear and kd-tree based search with multiple enrolments.The results are summarized in Table 5.4. From this experiment, we observe that betterrank one HR is achieved without affecting the PR. Nevertheless, a higher searching timeis required to search the database with multiple enrolments.

128


�

�

�

�

�

�

� �

8:

86

9:

96

"::

: ": 0: 2: 3: 6:

��

�

$ �(

�.$.*�4� ��"05

)��"" )��"0

)��"2 )��"3

)��"6

(a) CMC curves for different Probe sets withFERET Gallery12

�

�

�

�

�

�

�

� �

8:

86

9:

96

"::

: ": 0: 2: 3: 6:

��

�

$ �(

�$��4� ��005

)��0" )��00)��02 )��03)��06 )��01)��07 )��08)��09 )��2:)��2" )��20

(b) CMC curves for different Probe sets withFRGC Gallery22

�

�

�

�

�

� �

93

91

98

"::

: 6 ": "6 0: 06

��

�

$ �(

� �*��061�4� ��305

)��3"

)��30

)��32

(c) CMC curves for different Probe sets withCalTech256 Gallery42

Figure 5.15: CMC curves for different Probe sets with multiple enrolment of a subjectwith FERET, FRGC and CalTech256 databases.

FPIR and FNIR: We have analysed the performance of our proposed system withrespect to FPIR and FNIR. We use Gallery11, Gallery21 and Gallery42 sets asGallery sets, and Probe11, Probe21 and Probe43 as Probe sets for FERET, FRGC andCalTech databases, respectively. We have computed 992, 466 and 1297 genuine scores and985056, 216690 and 27403 imposter scores for FERET, FRGC and CalTech databases,respectively. The trade-off between FPIR and FNIR for the identification system withoutindexing and with indexing is shown in Fig. 5.16. The equal error rates (EER) of thesystem without indexing are 6.38%, 5.12% and 15.78% for FERET, FRGC and CalTechdatabases, respectively. Whereas, the EERs with indexing are 6.06%, 4.87% and 14.36%for FERET, FRGC and CalTech databases, respectively.

129


Table 5.4: Performance of different Probe sets for multiple enrolments of a subject in linearand kd-tree search

DB Probe setLinear Kd-tree

HR PR ST (ms) HR PR ST (ms)

FE

RE

T

Probe11 100 9.33 207.66 100 6.74 149.98

Probe12 88.94 9.85 215.93 88.94 7.11 155.96

Probe13 86.56 9.99 216.59 86.56 7.21 156.43

Probe14 87.73 9.92 214.91 87.73 7.16 155.22

Probe15 82.31 9.40 205.35 82.31 6.79 148.31

FR

GC

Probe21 100 13.69 164.07 100 10.14 121.54

Probe22 91.11 12.27 146.87 91.11 9.09 108.79

Probe23 93.93 13.50 163.63 93.93 10.00 121.21

Probe24 92.52 12.88 153.78 92.52 9.54 113.91

Probe25 95.59 13.13 157.14 95.59 9.73 116.40

Probe26 98.93 12.23 148.34 98.93 9.06 109.88

Probe27 99.57 13.56 160.79 99.57 10.04 119.11

Probe28 90.23 12.06 144.72 90.23 8.93 107.20

Probe29 93.31 13.45 160.29 93.31 9.96 118.73

Probe30 91.77 12.75 151.78 91.77 9.45 112.43

Probe31 93.30 12.90 151.07 93.30 9.56 111.90

Probe32 87.00 12.94 153.26 87.44 9.59 113.53

Cal

Tec

h Probe41 96.15 14.21 55.70 96.15 11.12 43.58

Probe42 97.52 14.54 57.35 97.52 11.38 44.87

Probe43 98.78 14.64 57.85 98.78 11.46 45.27


We analyze the time complexity of linear and kd-tree based search techniques in theproposed index space. Let N be the total number of face images enrolled in the databaseand Tp be the average number of index keys in each enrolled face image. Thus, the totalnumber of index keys in the index space is Tn = Tp ×N . If there are K number of cellsin all index cubes, then the average number of index keys in each cell is Tk = Tn/K.Let Tq be the average number of index keys in each query face image. To perform linearsearch or kd-tree based search in the index space, first, we find the index cell positionfor an index key of a query using hash functions defined in Eq. (5.15). This operationrequires O(1) computation time for both type of searching. The time complexity anal-ysis of linear and kd-tree based searching within the located cell are given in the following.

130

5.6. Performance Evaluation�

�

�

�

�

�

�

� �

:

6

":

"6

0:

: ": 0: 2: 3: 6: 1:

�'

�$

�)�$

L��

L��

(a) FPIR vs. FNIR curve for FERETdatabase

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

:

6

":

"6

0:

: ": 0: 2: 3: 6: 1:

�'

�$

�)�$

L��

L��

(b) FPIR vs. FNIR curve for FRGCdatabase

�

�

��

:

6

":

"6

0:

06

2:

: ": 0: 2: 3: 6: 1:

�'

�$

�)�$

L��

L��

(c) FPIR vs. FNIR curve for FRGCdatabase

Figure 5.16: FPIR vs. FNIR curve with and without indexing for FERET, FRGC andCalTech256 databases.

Linear search: In linear search, Tk×Tq number of comparisons are required to retrievea set of similar index keys and their identities, and Tq log Tq comparisons are requiredto sort the retrieved identities based on their ranks. Thus, we can calculate the averagetime complexity of linear search (denoted as TLS) as follows.

TLS = O(1)× Tq + Tk × Tq + Tq log Tq

= O(1)× Tq +N×Tp

K × Tq + Tq log Tq

= O(N)

(5.18)

Kd-tree search: The number of comparisons required in kd-tree based search to finda set of nearest index keys and their identities are log Tk × Tq, and to sort the retrievedidentities based on their ranks are Tq log Tq. Thus, we can calculate the average timecomplexity of kd-tree based search (denoted as TKS) as follows.

131


TKS = O(1)× Tq + log Tk × Tq + Tq log Tq

= O(1)× Tq + logN×Tp

K × Tq + Tq log Tq

= O(logN)

(5.19)

We compute the searching time and the average number of comparisons required forlinear and kd-tree based search techniques with the proposed index space. To computethese, we have enrolled different number of samples into the index space for FERET,FRGC and CalTech databases. The execution time (in Intel Core-2 Duo 2.00 GHz pro-cessor and 2GB RAM implementation environment) of linear and kd-tree search withFERET, FRGC and CalTech databases are shown in Fig. 5.17(a), (b) and (c), respec-tively. We observe that the execution time for kd-tree search is less than the linear search

�

�

� �

:

"

0

: 6:: "::: "6:: 0:::

��

��

��

��

4��5

Q":0

O��

�.$.*��

/��

\U��

(a) Average searching time for FERETdatabase

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

:

:M6

"

"M6

: 0:: 3:: 1:: 8:: ":::

��

��

��

��

4��5

Q":0

O��

�$��

/�� \U��

(b) Average searching time for FRGCdatabase

�

�

�

�

�

�

�

�

�

�

�

�

�

:

0:

3:

1:

: ":: 0:: 2::

��

��

��

��

4��5

O��

� �*��061��

/��

\U��

(c) Average searching time for CalTech256database

Figure 5.17: Average searching time with different sizes of databases for FERET, FRGCand CalTech256 databases.

132


method. It is also observed that the rate of increment in execution time for kd-tree basedsearch is less when the number of enrolled sample increases. We have given the averagenumber of comparisons for linear and kd-tree based search in Fig. 5.18. From Fig. 5.18,we can see that the rate of increment in number comparisons is also less for kd-tree basedsearch. Hence, we may conclude that to retrieve the similar identities for a given query,kd-tree based search within index cell is better than the linear search.


Here, we analyze the memory requirement for two proposed storing structures. Let b1

and b2 bytes memory are required to store the reference of the index cubes into the first

�

�

� �

:

6

":

"6

: 6:: "::: "6:: 0:::

O��

��

� �

��

��

Q":2

O��

�.$.*��

/�� \U��

(a) Average number of comparisons forFERET database

�

�

�

� �

:

6

":

: 0:: 3:: 1:: 8:: ":::

O��

��

� �

��

��

Q":2

O��

�$��

/�� \U��

(b) Average number of comparisons forFRGC database

�

� �

:

0

3

: ":: 0:: 2::

O��

��

� �

��

��

Q":2

O��

� �*��061��

/�� \U��

(c) Average number of comparisons for Cal-Tech256 database

Figure 5.18: Average number of comparisons with different sizes of databases for FERET,FRGC and CalTech256 databases.

133


level index space and the reference of linear or kd-tree index spaces into the index cube,respectively. Let m bytes are required to store a feature value of an index key and 2 bytesare required to store an identity of a subject. If there are P subjects and each subject hasQ samples then we can compute the memory requirement for linear and kd-tree indexspaces using Eq. (5.20) and (5.21), respectively. In Eq. (5.20) and (5.21), Tp,q representsthe number of index keys of the qth sample of the pth subject and δ denotes the numberof quantization levels of the second level index space.

MLS = 2× (b1 + δ3 × b2) +

P∑p=1

Q∑q=1

(64×m+ 2)× Tp,q (5.20)

MKD = 2× (b1 + δ3 × b2) +

P∑p=1

Q∑q=1

(64×m+ 14)× Tp,q (5.21)

In our approach, 2 bytes are required to store the reference of index cube into a cell offirst level index space and 4 bytes are required to store the reference of linear or kd-treeindex space into a cell of index cube. There are 64 feature values in an index key and 4bytes are required to store a feature value. We also store the identity of a subject witheach index key. The identity field requires 2 bytes extra memory for each index key.We can store 216 identities with 2 bytes identity field. Hence, a total of 258 bytes arerequired to store an index key along with the identity in linear index space. In kd-treebased index space, a single node of kd-tree requires 270 bytes memory. Figure 5.19(a),(b) and (c) show the memory requirements for linear and kd-tree index spaces to storedifferent number of samples for FERET, FRGC and CalTech databases, respectively.From Fig. 5.19, we observe that the memory requirements are almost same for the linearand kd-tree based index spaces.


We compare our approach with three existing face indexing approaches [123, 133, 159].To compare our proposed work, we use Gallery11, Gallery21 and Gallery41 as Gallery

sets, and Probe11, Probe21 and Probe41 as Probe sets for FERET, FRGC and CalTechdatabases, respectively. The comparison result is reported in Table 5.5. The comparisonis done with respect to rank one HR, PR and average searching time (ST ). FromTable 5.5, we can see that our approach gives better performance than the existingapproaches.

134

5.8. Summary�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

:

":

0:

2:

3:

: 6:: "::: "6:: 0:::

��

��

�4�

�5

O��

�.$.*��

/��

\U��

(a) Memory requirements for FERETdatabase

�

�

�

�

�

�

�

�

�

�

�

�

�

�

�

:

6

":

"6

0:

06

: 0:: 3:: 1:: 8:: ":::

��

��

�4�

�5

O��

�$��

/�� \U��

(b) Memory requirements for FRGCdatabase

� �

:

0

3

1

8

: ":: 0:: 2::

��

��

�4�

�5

O��

� �*��061��

/�� \U��

(c) Memory requirements for CalTech256database

Figure 5.19: Memory requirements with different sizes of databases for FERET, FRGCand CalTech256 databases.

5.8 Summary

In this chapter, we discuss an approach of creating a two-level indexing mechanism fora face biometric-based identification system. The proposed indexing mechanism reducessearch space for face-based identification system. In this technique, we calculate a setof sixty nine dimensional index keys using SURF feature extraction method from a faceimage. Among sixty nine dimensions, we consider only four dimensions to create thetwo-level index space. In the first level of indexing, we group the index keys based onthe sign of Laplacian value and in the second level, we group the index keys based onthe position and the orientation. We retrieve a set of similar identities for a query fromthe two-level index space using a hashing technique. The hashing technique requires

135


Table 5.5: Comparison of the proposed approach with existing approaches

Approach Performance FERET FRGC CalTech

Lin et al. [133]HR 83.87 85.19 73.08

PR 41.07 45.49 83.99

ST (ms) 2697.40 1738.35 895.27

Mohanty et al. [159]HR 94.15 95.06 80.77

PR 25.58 24.95 60.5

ST (ms) 745.61 623.56 46.77

Kaushik et al. [123]HR 95.87 97.64 88.46

PR 16.96 18.61 27.49

ST (ms) 456.24 398.78 14.59

ProposedHR 95.57 97.00 92.31

PR 7.90 12.55 23.72

ST (ms) 85.40 71.02 7.22

O(1) time complexity to retrieve the identities. We propose linear and kd-tree basedsearching mechanism to search the identities within the two-level index space. We havetested our approach with FERET, FRGC and CalTech face databases. The experimentalresult shows that kd-tree based search performs better than the linear search. We canachieve 95.57%, 97% and 92.31% rank one HR with 7.90%, 12.55% and 23.72% PR forFERET, FRGC and CalTech databases, respectively. Further, our approach gives betterHR when multiple samples of a subject are enrolled into the database. We achieve onthe average 8.21%, 11.87% and 24.17% search space reduction for different probe sets ofFERET, FRGC and CalTech, respectively. With our proposed indexing approach, weachieve the computation time advantage without compromising the accuracy comparedto traditional face-based person identification systems. The limitations of our approachis that it does not give good results under different poses (e.g. left or right profile) offace images.

136

Chapter 6

Multimodal Biometric DataIndexing

Multi-biometric based person identification is gaining its importance and present trendis to process a large amount of biometric data which is in the order of millions. Theproblem in such situations is to deal with high dimensional features from two or morebiometric traits. This demands high computation time to identify a query template.In this work, we propose an approach to index a large pool of multi-biometric dataso that the matching process can be accomplished in a real time without compromisein accuracy of person identification. Our proposed indexing technique is based on therelative scores. First, we select a small set of reference subjects. Then we enroll (retrieve)the subjects into (from) the database using proposed indexing approach. At the timeof enrollment (retrieving), the relative scores are calculated against the set of referencesubjects corresponding to each trait. We combine the scores using SVM based score levelfusion technique. These scores are used to generate index key for a subject. Based on theindex code values we store the subject identity into the database. We create index spacesin the database and store subjects’ identities into index space based on the relative indexkey values. At the time of querying, we retrieve a candidate set for a query index keycorresponding to each biometric trait. We introduce a new rank level fusion techniqueon the retrieved candidate sets using SVM rank. The different steps in our proposedapproach is shown in Fig. 6.1. Figure 6.1(a) shows an overview of the reference subjectselection and Fig. 6.1(b) shows the different steps of enrolling a subject into the databaseand retrieving a set of subjects from the database.

The rest of the chapter is organized as follows. Feature extraction and score cal-culation methodologies from multi-biometric traits is discussed in Section 6.1 and 6.2,

137

6. Multimodal Biometric Data Indexing

��

��

��

��

� ��

��

� ��

$��%��

��

� ��

��

��%��

��

$��M��%��'

$��M��%��0

$��M��%��"

��

��

� ��

$$$

(a) Reference subject selection

� � � ��

$��

��

� ��

��\��

��

��,��

��

�� N ��

��

��

$��M��%��

��

��

� ��

$��

��

� ��

��\��

�� $��,��

��,��

��

�� N ��

��

��

$ �(

��,��

��

��

��

��

��

.��%��

��

��

��

� ��

+��%��

��

��

��

� ��

$��,��

��%��

� ��

��

��

+��,��

.��

(b) Identification system with indexing

Figure 6.1: Overview of the proposed approach.

respectively. In Section 6.3, we describe the techniques of reference subject selection. Sec-tion 6.4 briefs about the score calculation against the selected reference subjects. Scorefusion technique is described in Section 6.5. Section 6.6 explains the proposed index keygeneration method for multimodal biometric system. The storing and retrieving tech-niques are discussed in Section 6.7 and Section 6.8, respectively. Section 6.9 introducesthe proposed rank level fusion technique. Performances of the proposed indexing methodare presented in Section 6.10. Section 6.12 summarizes the chapter.


In this work, we consider a subject having a multiple number of samples. A sampleis characterized with the images of multiple biometric traits. A sample of a subject isidentified with a unique identity and all samples of a subject are grouped together. Let usassume that there are P number of subjects and each having Q number of samples, andwe consider three biometric traits for each sample. Figure 6.2 shows the representationof samples for a subject with multiple biometric traits. In Fig. 6.2, B1, B2, and B3

138


� ��+

*� ��"

*� ��0

*� ��2

*� ��"

*� ��0

*� ��2

� ��@� ��"

*� ��"

*� ��0

*� ��2

�%��

&��

�'��

�

��

�%��

&��

�%��

&��

��

� ��"

*� ��"

*� ��0

*� ��2

� ��@

*� ��"

*� ��0

*� ��2

� ��+

*� ��"

*� ��0

*� ��2

(%��(

&��(��

(��

(%��(

&��(��

(%��(

&��'

�� ( ��

� ��"

*� ��"

*� ��0

*� ��2

� ��@

*� ��"

*� ��0

*� ��2

� ��+

*� ��"

*� ��0

*� ��2

)%��)

&��)��

)��

)%��)

&��)��

)%��)

&��'

�� ) ��

Figure 6.2: Representation of multimodal biometric samples.

represent three biometric traits and B1pq denotes the qth sample of the pth subject forthe biometric trait B1.

The extracted feature set of a subject for all samples and all biometric traits isdenoted by Eq. (6.1). F (B1pq) represents the feature set of the qth sample of the pth

subject for the biometric trait B1. The number of feature vectors for a biometric traitdepends on the feature extraction method. F (B1pq) may contains a single feature vectoror a set of feature vectors. Eq. (6.2) shows the feature vectors of the F (Bp

q ) where B

denotes a biometric trait. In Eq. (6.2), KB and LB represent the number of featurevectors extracted from a sample and the number of features in a single feature vector forthe biometric B, respectively.

F (B1p1) F (B2p1) F (B3p1)

F (B1p2) F (B2p2) F (B3p2)

· · ·F(B1pq

)F(B2pq

)F(B3pq

)· · ·

F(B1pQ

)F(B2pQ

)F(B3pQ

)(6.1)

F(Bp

q

)=

⎡⎢⎢⎢⎢⎢⎢⎢⎣

fd11(Bpq ) fd12(B

pq ) · · · fd1d(B

pq ) · · · fd1LB

(Bpq )

fd21(Bpq ) fd22(B

pq ) · · · fd2d(B

pq ) · · · fd2LB

(Bpq )

· · · · · · · · · · · · · · · · · ·fdk1(B

pq ) fdk2(B

pq ) · · · fdkd(B

pq ) · · · fdkLB

(Bpq )

· · · · · · · · · · · · · · · · · ·fdKB

1 (Bpq ) fdKB

2 (Bpq ) · · · fdKB

d (Bpq ) · · · fdKB

LB(Bp

q )

⎤⎥⎥⎥⎥⎥⎥⎥⎦

(6.2)

139


In our approach, we consider three biometric traits namely iris, fingerprint and face.These biometric traits (in our case, B1, B2 and B3 are iris, fingerprint and face biometrictraits, respectively) are commonly used in different applications [103, 107] due to theiruniqueness, stability, reliability and accessibility. We use the state of art techniquesto extract features from these biometric traits. We apply Daugman’s IrisCode-basedmethod [66,67], Jain’s filterbank-based method [106] and Du’s SURF-based method [75]

to extract features from iris, fingerprint and face biometric traits, respectively. Summaryof the extracted features from iris, fingerprint and face biometric traits are given inTable 6.1.

6.2 Score Calculation

We measure the similarity between two samples of a biometric trait and represent thesimilarity as a score value. Let F (Bp

q ) and F (Bmn ) be the feature vectors of the qth sample

of the pth subject and the nth sample of the mth subject, respectively for a biometrictrait B. We represent the score for a biometric trait B as Sm,n

p,q (B) and calculate usingEq. (6.3).

Sm,np,q (B) = SCOREB(F

(Bp

q

), F (Bm

n )) (6.3)

Thus, using Eq. (6.3), we can measure the similarity score value between two sam-ples belong to a particular biometric trait. It may be noted that SCOREB() func-tion is different for different biometric traits. We use existing methods to calculate thescores for different biometric traits and describe in Table 6.1. Daugman’s Hamming dis-tance method [67], Jain’s Euclidean distance method [106] and Du’s Euclidean distancemethod [75] are used to calculate the scores for iris, fingerprint and face, respectively.

Table 6.1: Feature summary of different biometric traits

Biometrictrait (B)

Featureextractionmethod

# of featurevectors

/sample (KB)

Length offeature vector

(LB)

# of totalfeatures/sample

Score calculationmethod

(SCOREB())

Iris (B1) Daugman’sIrisCode [67] 1 2048 2048

Hamming

distance [67]

Fingerprint(B2)

Jain’s Filter-Bank [106] 8 64 512

Jain’s Euclideandistance [106]

Face (B3) Du’s SURFfeature [75] 70 to 130 64 4200 to

8320Du’s Euclideandistance [75]

140

6.3. Reference Subject Selection

6.3 Reference Subject Selection

Our proposed multimodal biometric indexing approach is based on the relative scores.To calculate relative scores we have to decide the reference subjects. All individuals’data will be inserted into the database based on the relative scores with respect to thereference subjects. We choose the reference subjects with a single sample in such waythat the scores generated by individuals with respect to the reference subjects give ahigh degree of distinctiveness. Let < Bp

1 , Bp2 , . . . , B

pQ > be all Q samples of the pth

subject (p = 1, 2, . . . , P ) where P is the total number of subjects with the biometric traitB. We have to select M subjects (B1

r1 , B2r2 , . . . , B

MrM

) with single sample as referencesubjects from all samples of all P subjects for the biometric trait B and let BM

rMdenotes

the selected sample of the M th subject. The selection of reference subjects with singlesample for a biometric trait is done in two steps. The details of these steps are describedin the following.

6.3.1 Sample Selection

In this step, we choose a distinct sample for each subject for a biometric trait which giveshigh variance in scores compared to other samples for that biometric trait of that subject.Let Bp

r , the rth sample of the pth subject for a biometric trait B, gives the maximumvariance among all samples of the pth subject for the biometric B. To select Bp

r , first, wecalculate scores of the ith sample (Bp

i ) of the pth subject with all other samples (Bpj ; i = j

and j = 1, 2, . . . , Q) of the pth subject for the biometric trait B and find the variancevp,i(B) of the scores for the ith sample. We represent scores for the ith sample with allother samples for the pth subject in Eq. (6.4) and calculate the score variance for the ith

sample of the pth subject in Eq. (6.5).

Sp,i(B) =< Sp,jp,i (B) > ∀j; and j = i; j = 1, 2, . . . Q; (6.4)

vp,i(B) =1

Q− 1

∑1≤j≤Qj �=i

(Sp,jp,i (B)− μp,i(B))2 (6.5)

In Eq. (6.4), Sp,i(B) denotes the score vector of length Q − 1 and Sp,jp,i (B) denotes

the score between the ith and jth samples of the pth subject for B. In Eq. (6.5), μp,i(B)

represents the mean of all scores for the ith sample of the pth subject for biometric traitB and calculated using Eq. (6.6).

141


μp,i(B) =1

Q− 1

∑1≤j≤Qj �=i

Sp,jp,i (B) (6.6)

We calculate variance for the all samples (Bpi ; i = 1, 2, . . . , Q) of the pth subject using

Eq. (6.4), (6.5) and (6.6) and create a variance vector vp(B) for the pth subject (seeEq. (6.7)).

vp(B) =< vp,1(B), vp,2(B), . . . , vp,i(B), . . . , vp,Q(B) > (6.7)

We find the maximum variance from vp(B) and select the corresponding sample as adistinct sample of the pth subject for the biometric trait B. Let vp,r(B) be the maximumvariance in vp(B). We select the rth sample (Bp

r ) as a distinct sample for the pth subjectfor B and represent as Bp

rp . In this way, we select the distinct samples (B1r1 , B

2r2 , . . . , B

PrP

)for all P subjects of biometric trait B.

6.3.2 Subject Selection

Now, we have all subjects with one distinct sample (B1r1 , B

2r2 , . . . , B

PrP

) for a biometrictrait. In this step, we select M subjects as reference subjects from these subjects whichyield the high diversity in scores with respect to other subjects. In other words, we haveto select B1

r1 , B2r2 , . . . , B

MrM

as reference subjects which give top M variances among allP subjects. It may be noted that a reference subject contains single biometric templatefrom each trait. To choose the reference subjects, we follow the same maximum variationfinding strategy as in sample selection. Here, the main difference is that we compute thevariation among all subjects rather than all samples of a subject. First, we calculatea score vector Sp,r(B) for the pth subject (see Eq. (6.8)) with all other subjects for abiometric trait B and compute the score variance for the pth subject in Eq. (6.9).

Sp(B) =< Sq,rq

p,rp(B) > ∀q; and q = p; q = 1, 2, . . . P ; (6.8)

vp(B) =1

P − 1

∑1≤q≤Pq �=p

(Sq,rq

p,rp(B)− μp(B))2 (6.9)

In Eq. (6.8), rp and rq are the selected samples for the pth and qth subject, respectivelyand Sq,rq

p,rp(B) is the score between the selected samples of the pth and qth subjects for thebiometric trait B. The length of the score vector Sp,r(B) is P − 1. In Eq. (6.9), μp(B)

142

6.3. Reference Subject Selection

is the mean of all scores between the pth subject and all other subjects for the biometrictrait B. We calculate μp(B) using Eq. (6.10).

μp(B) =1

P − 1

∑1≤q≤Pq �=p

Sq,rq

p,rp(B) (6.10)

We calculate a variance vector of all subjects for the biometric trait B using Eq. (6.9)and the vector is represented with v(B) (see Eq. (6.11)).

v(B) =< v1(B), v2(B), . . . , vp(B), . . . , vP (B) > (6.11)

We find the top M variances from v(B) and select M subjects corresponding to thetop variances as a reference subjects for each biometric trait. Let Bm

rm (m = 1, 2, . . . ,M)be the subject with top mth variances. We select B1

r1 , B2r2 , . . . , B

MrM

as reference subjectsfor the biometric trait B and represent as R1(B), R2(B), . . . , RM (B). The referencesubjects contain single template for each biometric trait. In this way, we select referencesubjects for all other biometric traits. The extracted features of three biometric traits(B1, B2 and B3) of reference subjects are defined in Eq. (6.12). We represent the featurevector of the mth (m = 1, 2, . . . ,M) reference subject for biometric trait B in Eq. (6.13).In Eq. (6.13), KB and LB denote the number of feature vectors extracted from thebiometric trait B and the length of a feature vector, respectively.

F(B1R

1)

F(B2R

1)

F(B3R

1)

F(B1R

2)

F(B2R

2)

F(B3R

2)

· · ·F(B1R

m)F(B2R

m)F(B3R

m)· · ·

F(B1R

M)

F(B2R

M)

F(B3R

M)

(6.12)

F(BRm)=

⎡⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢⎣

fd11(BRm

) fd12(BRm

) · · · fd1d(BRm

) · · · fd1LB(BRm

)

fd21(BRm

) fd22(BRm

) · · · fd2d(BRm

) · · · fd2LB(BRm

)

· · · · · · · · · · · · · · · · · ·fdk1(B

Rm) fdk2(B

Rm) · · · fdkd(B

Rm) · · · fdkLB

(BRm)

· · · · · · · · · · · · · · · · · ·fdKB

1 (BRm) fdKB

2 (BRm) · · · fdKB

d (BRm) · · · fdKB

LB(BRm

)

⎤⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥⎦

(6.13)

143


6.4 Reference Score Calculation

We calculate scores of all samples against all reference subjects using Eq. (6.3). Wedenote match score of the qth sample (q = 1, 2, . . . , Q) of the pth subject (p = 1, 2, . . . , P )against the mth reference subject (m = 1, 2, . . . ,M) for B as Sm

p,q (see Eq. (6.14)).

Smp,q(B) = SCORE(F

(Bp

q

), F(BRm)

) (6.14)

The scores for the qth samples of the pth subject with respect to all reference subjectsare given in Eq. (6.15). In Eq. (6.15), Sm

p,q denotes a tuple which contains scores of theqth samples of the pth against the mth reference subject for all biometric traits. Wecalculate scores for all p (p = 1, 2, . . . , P ) and all q (q = 1, 2, . . . , Q), where P and Q arethe number of subjects and number of samples for each subject, respectively.

S1p,q = < S1

p,q(B1) S1p,q(B2) S1

p,q(B3) >

S2p,q = < S2

p,q(B1) S2p,q(B2) S2

p,q(B3) >

· · · · · ·Smp,q = < Sm

p,q(B1) Smp,q(B2) Sm

p,q(B3) >

· · · · · ·SMp,q = < SM

p,q(B1) SMp,q(B2) SM

p,q(B3) >

(6.15)

6.5 Score Level Fusion

After the calculation of scores for all samples of each biometric trait our next task isto combine the scores of all biometric traits for a sample. Note that the scores of abiometric trait is in different scales. Hence, we normalize the scores of each biometrictrait before combining it. We combine the scores of three biometric traits using SVMclassifier [116,119]. In the following, we discuss the score normalization technique followedby score fusion (combining) technique.

6.5.1 Score Normalization

Several score normalization techniques (min-max, z-score, tanh, sigmoid, reduction ofhigh-scores effect (RHE), etc.) exist in literature [93, 102, 176, 179]. Table 6.2 showsthe effectiveness of the different score normalization methods with respect to robustness,scalability, efficiency and ease of use. From Table 6.2, we can see that RHE method [93]

is more robust and efficient among all other normalization techniques. Hence, we applyRHE normalization technique in our work. We normalize all scores of each biometric

144

6.5. Score Level Fusion

Table 6.2: Characteristics of the different score normalization methods

Score normalization method Robustness Scalability Efficiency Ease of use

Min-max [102,176] No No High Easy

Decimal scaling [102,176] No No Low Easy

tanh [102,176] Yes No Moderate Hard

Sigmoid [102,176] Yes No Moderate Hard

z-score [93,102,176] No Yes High Hard

Reduction of High Score(RHE) [93] Yes Yes High Moderate

trait using Eq. (6.16).

smp,q(B) =Smp,q(B)−min(S(B))

{mean(S∗(B)) + std(S∗(B))}+min(S(B))(6.16)

In Eq. (6.16), smp,q(B) is the normalized score of Smp,,q(B) for the biometric B. S(B)

denotes the distribution of all scores and S∗(B) denotes the genuine score distribution [93]

for the biometric B.

6.5.2 Score Fusion

The normalized scores are used to calculate the multimodal score for a sample. Thereexist several score fusion methods (maximum rule, minimum rule, sum rule, productrule, weighted sum rule, likelihood ratio, SVM and etc.) [93, 102,121,173,174,176]. Thecharacteristics of the different fusion methods are shown in Table 6.3. From Table 6.3,we can see that the SVM based score fusion is the most efficient with respect to thedifferent parameters like accuracy, robustness and scalability. Hence, SVM classifier ispreferable to classify the genuine and imposter subjects using the normalized scores [77,93, 97, 125, 128, 178]. In our work, we use SVM classifier [116, 119] to calculate themultimodal score because SVM-based classification performs better for biometric dataclassification [77,93,178].

Let smp,q =< smp,q(B1) smp,q(B2) smp,q(B3) > be the normalized score vector of the qth

sample of the pth subject with respect to the mth reference subject. Then we representthe SVM classification function for smp,q using kernel trick [116,192] (see Eq (6.17)).

F (smp,q) =Ns∑i=1

αiyiK(Si, smp,q) (6.17)

145


Table 6.3: Characteristics of the different score fusion methods

Fusion method Robustness Scalability Accuracy Remarks

Maximum rule [121,176] No No Moderate No training required

Minimum rule [121,176] No No Low No training required

Sum rule [121,176] Yes No Moderate No training required

Product rule [121,176] No No Low No training required

Weighted sumrule [121,176] Yes Yes High

Accuracy depends on theweight of classifiers. Hence,deciding the weights of the

classifiers is an important task.

Likelihoodratio [121,176] Yes Yes High Score statistics is required in

advance and training required

SVM [125,176] Yes Yes High Training required

In Eq. (6.17), Ns is the number of support vectors, Si denotes the ith support vector,yi is the class associated with Si, K(·, ·) represents a non-linear kernel function and αi

represents Lagrange multiplier associated with Si. The sign of F (smp,q) represents theclass of the smp,q and the absolute value of F (smp,q) indicates the confidence of smp,q beingin that class. In our approach, we consider the confidence value F (smp,q) as a combinedscore. We refer the combination of all biometric traits as multimodal biometric trait anddenote it as B4. The multimodal score of the qth sample of the pth subject with respectto the mth reference subject is represented by Sm

p,q, that is, Smp,q = F (smp,q).

To decide the support vectors (Si), classes (yi) of each support vector and Lagrangemultipliers (αi) associated with each support vector, we train the SVM classifier with aset of training samples. We use 280134 training data to train the SVM classifier. Thesedata are created from 100 users of the Gallery set. Detail description of the trainingprocedure is given in Section 6.10.3.

Note that the multimodal scores (Smp,q(B4)) for all samples (q = 1, 2, . . . , Q) of all

subjects (p = 1, 2, . . . , P ) against all reference subjects (m = 1, 2, . . . ,M) calculated asmentioned above (also see Eq. (6.17)) are in different scales than the unimodal biometrictraits. Hence, we propose to normalize the multimodal scores using RHE normalizationtechnique (see Eq. (6.16)) and represent the normalized multimodal score of the qth

sample of the pth subject against the mth reference subject with smp,q(B4).Now, we have four normalized score values for a sample with respect to a reference

subject. Four normalized score values of the qth sample of the pth subject against the mth

reference subject is denoted with a vector < smp,q(B1) smp,q(B2) smp,q(B3) smp,q(B4) >. We

146


generate the normalized score vector for all samples of all subjects against all referencesubjects and represent it in Eq. (6.18) where p = 1, 2, . . . , P and q = 1, 2, . . . , Q. Thesescore vectors are used to generate index key for a sample. Index key generation techniqueis discussed in the next section.

< s1p,q(B1) s1p,q(B2) s1p,q(B3) s1p,q(B4) >

< s2p,q(B1) s2p,q(B2) s2p,q(B3) s2p,q(B4) >

· · · · · ·< smp,q(B1) smp,q(B2) smp,q(B3) smp,q(B4) >

· · · · · ·< sMp,q(B1) sMp,q(B2) sMp,q(B3) sMp,q(B4) >

(6.18)


To generate an index key for a sample of a subject we use the normalized score valuesobtained as discussed above. Note that there are four normalized score values withrespect to a reference subject and there are M reference subjects. Hence, we create M

index keys and each index key is of four dimensions corresponding to four normalizedscore values. Each dimension is called a key feature of an index key. To compute anindex key feature we consider the distribution of the scores of a biometric trait withrespect to a reference subject for all samples of all subjects. Let sm(B) denotes a vectorwhich contains normalized scores of all samples of all subjects against the mth referencesubject for the biometric trait B. We represent sm(B) in Eq. (6.19).

sm(B) =< sm1,1(B) sm1,2(B) . . . smp,q(B) . . . smP,Q(B) > (6.19)

The distribution of the score vector sm(B) may not be uniform. But our target is togenerate uniformly distributed index keys which help us to store the identities of subjectsinto the database in a well distributed manner. So, we construct uniformly distributedscore vector prior to index key generation. To do this we apply histogram equalization [84]

technique on the score values of the vector sm(B). The equalized score fmp,q(B) for a

score smi (B) of the qth sample of the pth subject against the mth reference subject forthe biometric B is calculated in Eq. (6.20). In Eq. (6.20), T (·) is a transfer functionon the input normalized score to get the equalized score, p

Sm(B)denotes probability

density function (PDF) of scores against the mth reference subject for the biometric traitB, minsm(B) represents the minimum value in the vector sm(B) and r is the dummy

147


variable for integration.

fmp,q(B) = T (smp,q(B)) =

∫ smp,q(B)

s=minsm(B)

psm(B)

(r)dr (6.20)

The equalized score in Eq. (6.20) is used as a key feature of an index key. Werepresent the index key of the qth sample of the pth subject for the mth reference subjectwith indxmp,q. We generate M index keys for a sample of a subject. All M index keys ofthe qth sample of the pth subject are shown in Eq. (6.21).

indx1p,q = < f1p,q(B1) f1

p,q(B2) f1p,q(B3) f1

p,q(B4) >

indx2p,q = < f2p,q(B1) f2

p,q(B2) f2p,q(B3) f2

p,q(B4) >

. . . . . .

indxmp,q = < fmp,q(B1) fm

p,q(B2) fmp,q(B3) fm

p,q(B4) >

. . . . . .

indxMp,q = < fMp,q(B1) fM

p,q(B2) fMp,q(B3) fM

p,q(B4) >

(6.21)

6.7 Storing

We have generated index keys for all samples of all subjects according to the proposedindex key generation method. To store the identities of subjects based on the indexkeys, first we create index spaces into the database and then store the identities into theindex spaces. The methods for index space creation and storing data are discussed inthe following.


We create an index space into the database corresponding to each reference subject. Ifthere are M reference subjects then we create M index spaces. Figure 6.3 shows anoverview of the mth index space which is corresponding to the mth reference subject.Each index space contains one table for each biometric trait. There are four biometrictraits B1, B2, B3 and B4 where B4 represents multimodal trait in our approach. Wecreate four tables in each index space. In Fig. 6.3, table for a biometric trait B is denotedwith TablemB (in the mth index space) and the length of the table is denoted with LB.The length of each table depends on the enrollments of subjects into the database and isdecided experimentally (see Section 6.10.5.2). The index values of the cells for the table

148

6.7. Storing

" ��/

0 ��/

2 ��/

/�" ��/

�

�* ��"

" ��/

0 ��/

2 ��/

/�0 ��/

�

�* ��0

" ��/

0 ��/

2 ��/

/�2 ��/

�

�* ��2

" ��/

0 ��/

2 ��/

/�3 ��/

�

�* ��3

��

Figure 6.3: Index space for the mth reference subject.

TablemB are represented with 1, 2, . . . , LB. Each cell of the table contains a list calledIDL which stores a set of identities of subjects (see Fig. 6.3).

6.7.2 Storing Multimodal Biometric Data

We store the identity of each sample into the database based on the generated indexkeys which is stated as follows. For the key value fm

p,q(B) of the index key indxmp,q, weselect the mth index space and table TablemB . We represent the unique identity of the qth

sample of the pth subject as Idpq where p and q represent the subject number and samplenumber of the subject, respectively. We store the identity Idpq into the identity list IDL

corresponding to a cell of the table TablemB . The cell position in the table is calculatedbased on the index key value. We find the cell position (tB) in the table TablemB forthe key value fm

p,q(B) using Eq. (6.22) and Eq. (6.23). In Eq. (6.22) and Eq. (6.23), LB

denotes the length of the table, and minfm(B) and maxfm(B) represent the minimum andmaximum values of all key values of all samples of all subjects against the mth referencesubject for the biometric B, respectively and calculated using Eq. (6.24).

tB =⌈fmp,q(B)−minfm(B)

Δ + Δ2

⌉(6.22)

where, Δ =maxfm(B)−minfm(B)

LB−1 (6.23)

maxfm(B) = max(fmp,q(B)) ∀p ∈ P and ∀q ∈ Q

minfm(B) = min(fmp,q(B)) ∀p ∈ P and ∀q ∈ Q

(6.24)

Note that if there are P number of subjects with Q number of samples each, M

number of reference subjects and each subject has B biometric traits then there are M

index spaces and each index space has B number of tables. We store N = P ×Q numberof identities in each table.

149


Illustration: We illustrate the storing technique with an example. Let there be fivesubjects (P = 5) and each subject has four samples (Q = 4) to be enrolled into thedatabase. We also assume that there are three reference subjects (R1, R2 and R3). Wegenerate the four dimensional index keys for the third sample (q = 3) of the fifth subject(p = 5) with respect to all three reference subjects which is shown in Fig. 6.4(a). Wecreate three index spaces corresponding to each reference subject in the database andeach index space contains four tables. Figure 6.4(b) shows all tables of the first indexspace. Let us assume that the length of each table is 5, and the range of key values forB1, B2, B3 and B4 with respect to the first reference subject are 0 to 0.8, 0 to 0.7, 0to 0.5, and 0 to 1, respectively. Now, we show how we to store the identity of a subjectfor the first index key (highlighted row in Fig. 6.4(a)). We select the first index space tostore identity Id53 corresponding to the first index key indx15,3. The identity will be storedinto all tables (Table1B1, Table1B2, Table1B3 and Table1B4) of the first index space. Wecalculate the cell positions tB1, tB2, tB3 and tB4 of the Table1B1, Table

1B2, Table

1B3 and

Table1B4 for the indx15,3 using Eq. (6.22), respectively. The calculated cell positions forfour tables are 4, 3, 2 and 4, respectively. We store sample identity (Id53) into the identitylist (IDL) at the 4th, 3rd, 2nd and 4th cell positions of Table1B1, Table

1B2, Table

1B3 and

Table1B4 respectively. The highlighted cells in Fig. 6.4(b) show the positions for storingthe identity (Id53).

" 6

6<2 2

0 6

6<2 2

2 6

6<2 2

�� # :M7"2 :M398 :M027 :M733 �

�� # :M786 :M6:6 :M063 :M761 �

�� # :M712 :M611 :M220 :M780 �

*+�, �� *

(a) Index key of the 3rd sample of the 5th

subject

"

0

2

3

6

3

0;� <MMM=2

";� <MMM="

0;� <MMM=

; <MMM=��

2

2;� <MMM=

"

�"* ��

"

0

2

3

6

"

2;� <MMM=2

3;� <MMM=

6

";� <MMM=

; <MMM=��

6

3;� <MMM=

"

�0* ��

"

0

2

3

6

3

2;� <MMM=

; <MMM=��3

0;� <MMM=2

2;� <MMM="

3;� <MMM=

"

�2* �� "

�3* ��

"

0

2

3

6

0

0;� <MMM=3

";� <MMM=

2; <MMM=��

"

0;� <MMM=

6

3;� <MMM=

(b) Database of the 1st index space

Figure 6.4: Example of storing the 3rd sample of the 5th subject into the 1st index spaceof the database.

150

6.8. Retrieving

6.8 Retrieving

Once all samples of individuals are enrolled into the database, we can use it to retrievedata for a match. In retrieving, we find the samples from the database which are themost similar to a query sample. We generate the index keys for the query sample asdiscussed in Section 6.6 and represent the query index keys in Eq. (6.25) where indxm

denotes the query index key correspond to the mth reference subject, fm(B) denotes keyfeature value corresponding to the biometric trait B and M represents the number ofreference subjects.

indx1 = < f1(B1) f1(B2) f1(B3) f1(B4) >

indx2 = < f2(B1) f2(B2) f2(B3) f2(B4) >

. . . . . .

indxm = < fm(B1) fm(B2) fm(B3) fm(B4) >

. . . . . .

indxM = < fM (B1) fM (B2) fM (B3) fM (B4) >

(6.25)

Now, we retrieve a set of identities of subjects for each query index key. For thispurpose, we maintain a candidate set for each biometric trait. Note that there are fourbiometric traits and each trait is related to a dimension of an index key. Let the candidatesets are CSET (B1), CSET (B2), CSET (B3) and CSET (B4) for the biometric traitsB1, B2, B3 and B4, respectively. Each candidate set consists of two fields: id and vote.We store the retrieved subject identities into the id field and number of occurrences ofthe subject identities into the vote field. To retrieve the subject identities for the mth

query index key indxm, we search the mth index space and retrieve the list of subjectidentities IDL from a particular cell of the table in the mth index space for a key featurevalue of the indxm. For the key feature value fm(B) of the indxm, we retrieve theIDL from the t th

B cell of the TablemB . The cell position tB in the table is calculatedby Eq. (6.22) and Eq. (6.23). The retrieved identities for the fm(B) are stored into id

field of the candidate set CSET (B). We count the number of occurrence of each subjectidentities and store the count value in the vote field of the candidate set CSET (B)

corresponding to the subject identity. Note that a subject have more than one sample inthe database, and the superscript and subscript values refer to the subject and sampleidentifiers, respectively. The count is done based on the subject identifier. For example,if we retrieve Id52, Id

51, Id

32 identities for biometric B then we store Id5 and Id3 into the

id field, and 2 and 1 in the vote field of candidate set CSET (B), respectively.

The key values calculated corresponding to a reference subject for a query may differfrom the key values of the matched subjects. This may leads to decrease the accuracy

151


of the proposed system. To address this limitation, we consider the ±δ neighbor cellpositions of the cell tB in a table. We retrieve the identity lists (IDL from tB−δ to tB+δ

cell positions of the table. The value of δ is decided experimentally (see Section 6.10.5.3).In this way, we generate all candidate sets CSET (B1), CSET (B2), CSET (B3) and

CSET (B4) for all biometric traits B1, B2, B3 and B4. These candidate sets are usedfor rank level fusion to rank the retrieved identities.

Illustration: We illustrate the retrieving of candidate set from the database for a givenquery subject with an example. In this example, we consider 3 reference subjects togenerate the index key. Let us assume that the database stores the identities for 5subjects and each subject has four samples. The database consists of four index spaces.Figure 6.5(a) shows the stored identities in the tables Table1B1, Table

2B1 and Table3B1 of all

index spaces ( for 1st, 2nd and 3rd index spaces, respectively) for the biometric trait B1.In each table, the identities of all subjects are stored. The index keys of a query subjectare shown in Fig. 6.5(b). We generate the candidate set CSET (B1) for the B1. Togenerate CSET (B1), we consider the f(B1) key values of all index keys (see Fig. 6.5(b))and find the respective positions in the three tables using Eq. (6.22). The positions inthe Table1B1, Table

2B1 and Table3B1 for the key values f(B1) = 0.417, 0.465, 0.378 are 3,

3 and 2. We retrieve the identities from the 3rd, 3rd and 2nd positions of the Table1B1,Table2B1 and Table3B1, respectively, and count the number of occurrences of each subject.From Fig. 6.5(a), we can see that Id1, Id4 and Id5 occur 10, 4 and 7 times, respectively.Therefore, Id1, Id4 and Id5 in CSET (B1) receive 10, 4 and 7 votes, respectively (seeFig. 6.5(c)). Similarly, we can generate CSET (B2), CSET (B3) and CSET (B4) for thebiometric traits B2, B3 and B4, respectively.

6.9 Rank Level Fusion

For a query subject we retrieve a candidate set corresponding to each trait. A candidateset for a biometric trait contains a set of retrieved subjects and a vote for each retrievedsubject. Therefore, each retrieved subject has different order of similarities based ondifferent biometric traits. To decide the order of similarity for the retrieved subjectswith the query, we have done rank level fusion among all candidate sets. Rank levelfusion is done in two steps. These steps are described in the following.

6.9.1 Creating Feature Vector for Ranking

In Section 6.8, we have retrieved four candidate sets (CSET (B1), CSET (B2), CSET (B3)

and CSET (B4)). From these candidate sets, we create a set of feature vectors for rank-

152

6.9. Rank Level Fusion

ing. Let us consider a total of N subject identities are retrieved in all four candidate setsfor a query. Each subject identity has own vote in each candidate set. We represent thevotes of the retrieved subjects in each candidate set as shown in Table 6.4. In Table 6.4,Idi denotes the identity of the ith retrieved subject and vi(B) denotes the vote for thatsubject identity in the candidate set CSET (B). For each retrieved subject, we createa feature vector from the number of votes of that subject in each candidate set. Thefeature vector for the subject identity Idi is represented in Eq (6.26).

vi = < vi(B1) vi(B2) vi(B3) vi(B4) > (6.26)

6.9.2 SVM Ranking

In this step, we rank each feature vector using SVM ranking method [117, 192]. Un-like SVM classification function, which outputs a distinct class for a feature vector, theranking function gives an ordering of feature vectors. The ranking function outputs ascore for each feature vector, from which a global ordering of feature vectors is con-structed [117,192]. Let vi feature vector is preferred to vj feature vector and specified asvi � vj . The objective is to find a global function F (·) which outputs a score such thatF (vi) > F (vj) for any vi � vj . The global ranking function F (·) on a feature vector

"

�"* ��

"

0

2

3

6

" 0 3

" 2 0;� < � <� =

0 0 0 2

" 0 3 ";� <� < � < � =

� � ��

-� �� .

2 6 6

3 2 3;� <� <� =

2 2 3 3

0 2 " 3;� <� <� <� =

0

�"* ��

0 0 3

0 2 0;� <� < � =

2 2 2

0 2 3;� <� <� =

;

=

� � � ��

� � ��

� ��

0 3 3 6

3 " 3 2;� <� <� <� =

0 2 6

" " ";� < � <� =

"

0

2

3

6

2

�"* ��

2 2 2

" 0 3;� <� <� =

;

=

� � � ��

� � � ��

� ��

" 0 0

2 0 3;� <� <� =

2 3 6

2 0 2;� <� <� =

0 0 3

" 2 2;� < � <� =

"

0

2

3

6

(a) Tables of the biometric trait (B1) from all index spaces of the database

"

0

2

�� # :M3"7 :M"61 :M397 :M19"

�� # :M316 :M03" :M36" :M136

�� # :M278 :M"10 :M328 :M760

*+�, ��

(b) Index keys for query subject

��.*4�"5

�" ":

�3 3

�6 7

��

��.*4�05

�2 8

�" 1

�0 2

��

��.*4�25

�0 6

�2 9

�" 2

��

��.*4�35

�2 1

�3 9

�" 6

��

(c) Candidate sets for all biometric traits

Figure 6.5: Example of retrieving candidate sets for a given query.

153


Table 6.4: Retrieved subjects and their votes in each candidate set

Subject Id Vote

CSET (B1) CSET (B2) CSET (B3) CSET (B4)

Id1 v1(B1) v1(B2) v1(B3) v1(B4)

Id2 v2(B1) v2(B2) v2(B3) v2(B4)

. . . . . . . . . . . . . . .

Idi vi(B1) vi(B2) vi(B3) vi(B4)

. . . . . . . . . . . . . . .

IdN vN (B1) vN (B2) vN (B3) vN (B4)

vi can be computed using SVM [117, 192] and represented in Eq. (6.27). In Eq. (6.27),K(·, ·) is kernel function, Ns is the number of support vectors, (Vk − Vl) is the pairwise difference support vector and αk,l is the Lagrange multiplier associated with the(Vk − Vl).

F (vi) =

Ns∑k,l

αk,lK(Vk − Vl,vi) (6.27)

The pair wise difference support vector (Vk−Vl) and the Lagrange multiplier αk,l forthe global ranking function F (·) are computed from a set of labeled training data. Thedetail method to train the SVM for ranking can be found in [117,119]. To generate the setof training data, we randomly select T number of queries from the gallery set and createa set of feature vectors for each query using the method discussed in Section 6.9.1. LetRt be the training data generated from the tth query (t = 1, 2, . . . , T ) and is representedin Eq. (6.28).

Rt = {(v1t , r

1t ), (v

2t , r

2t ), · · · , (vi

t, rit) · · · , (vNT

t , rNTt )} (6.28)

In Eq. (6.28), vit denotes the ith feature vector for the tth query and rit is the rank

of vit. Note that the training data Rt follows strict ordering [117], that is, rit < rjt if

vit � vj

t . The ranks of the feature vectors in the training data are given by manuallyor any automatic ranking method [117, 119]. In our approach, we assign a rank to eachfeature vector of a training data by an automatic method which is discussed in thefollowing.

Our ranking method for training data is based on a weighted feature vector. Let wvitbe the weighted value of the ith feature vector vi

t for the tth training data. The weightedvalue wvit is computed by Eq. (6.29).

wvit = [vit(B1) + vit(B2) + vit(B3) + vit(B4)]× wit (6.29)

154

6.9. Rank Level Fusion

In Eq. (6.29), wit represents the weight of the ith feature vector vi

t for the tth trainingdata. The weight is assigned to each feature vector of a training data in such a waythat if the identity related to the feature vector is present in all candidate sets then itgets higher preference than the other feature vector. The weight wi

t is computed usingEq. (6.30).

wit =

B4∑B=B1

eB where eB =

{0 if vit(B) = 0

1 otherwise.(6.30)

We calculate the weighted values for all feature vectors of a training data usingEq. (6.29) and rank the feature vector based on the weighted values. We give rank oneto the feature vector with the highest weighted value, rank two to the feature vector withthe second highest weighted value and so on. In this way, we assign rank to each featurevector of all training sets.

Illustration: We illustrate the SVM-based ranking of each retrieved subject with anexample. Figure 6.6(a) shows the retrieved candidate sets for a query. There are 5unique subjects among all candidate sets. We generate a feature vector for each subjectidentity. For the subject identity Id1, there are 10, 6, 3 and 5 votes in CSET (B1),CSET (B2), CSET (B3) and CSET (B4). Hence, the feature vector v1 correspondingto Id1 is < 10 6 3 5 >. The feature vectors for all retrieved subjects are shown inFigure 6.6(b). We compute the value of SVM ranking function for each feature vectorwhich is shown in Figure 6.6(c). In Figure 6.6(c), we can see that the feature vector v1

related to Id1 gives the maximum value for SVM-ranking function. We assign rank 1 tothe subject identity Id1.

��.*4�"5 ��.*4�05

��.*4�25 ��.*4�35

�" ":

�3 3

�6 7

�� 2 8

�" 1

�0 2

��

�0 6

�2 9

�" 2

�� 2 1

�3 9

�" 6

��

(a) Candidate setswith retrieved data

�"# /" ?�]":��1��2��6[

�0# /0 ?�]�:��2��6��:[

�2# /2 ?�]�:��8��9��1[

�3# /3 ?�]�3��:��:��9[

�6# /6 ?�]�7��:��:��:[

(b) Feature vectors for allsubjects in candidate sets

/" "M1

/� �4/�5

/0 :M1

/2 :M9

/3 :M7

/6 :M6

"

$ �(

3

0

2

6

�"

�

�0

�2

�3

�6

(c) Subject with rank af-ter SVM

Figure 6.6: An example of SVM-based rank to each retrieved subject for a query.

155



To study the efficacy of the proposed multimodal biometric data indexing approach, wehave conducted a number of experiments on virtual multimodal database and measurethe performances with the metric defined in Section 3.7.1. This section starts withthe description of database. Then, we present the experiments carried out and theexperimental results observed.

6.10.1 Database

There are few publicly available multimodal databases with iris, fingerprints and facebiometric traits. Out of which, WVU [31, 63] and BioSecure [3] are the two multimodaldatabase which contain three biometric traits, namely iris, fingerprint and face. But theauthority of the WVU database does not disclose the privacy of the users. Hence, WVUface database is also not available to the research community. On the other hand, BioSe-cure [3] database contains less number of users and this database is not freely available tothe research community. As a way out, we create a set of virtual users form three publiclyavailable large unimodal databases and perform the experiments on these virtual userdataset. We use CASIAV3I [7], WVU [31, 63] and FRGC Still version 2.0 [21, 166, 167]

unimodal biometric databases for iris, fingerprint and face biometric traits, respectively.The description of these databases are given in the following.

Iris Database: The CASIAV3I database [7] contains 2639 eye images from 395 eyes of249 persons. One to twenty six images are captured from each eye. It may be notedthat the iris data of left and right eyes of a person are different. Hence, we treat eacheye as a unique subject. The summary of CASIAV3I iris database is given in Table 6.5.Figure 6.7(a) shows two sample eye images of CASIA database. We consider at leasttwo samples from each subject so that we can use at least one sample for enrollmentinto the database and one sample for probing. There are 372 subjects which have atleast two samples for each subject. These subjects are used to create virtual users in ourexperiments.

Fingerprint Database: The WVU fingerprint database [31, 63] contains 7136 imagesof 270 persons. The images are captured from 4 different fingers (left index, left thumb,right index, right thumb). Three to twenty images are captured from each finger. Aseach finger of a person is different, we consider the each finger as unique subject. Hence,there are 1080 unique subjects in the WVU fingerprint database. We could not extractthe features from the fingerprint images of 320 subject. So, we use 750 subjects to create

156


virtual users in our experiment. The summary of WVU fingerprint database is given inTable 6.5. Two Sample fingerprint images selected randomly from WVU database areshown in Fig. 6.7(b).

Face Database: FRGC Still version 2.0 database [21, 166, 167] contains 16028 frontalface images of 466 subjects. The images from each subject are captured in indoor en-vironment with two different protocols (FERET and Mugshot) and two different facialexpressions (Neutral and Smiley) [166]. In our experiment we have taken a subset of4007 face images of 466 persons with neutral expression captured with FERET protocol(FERET-Neutral). These images are captured in Fall 2003 and Spring 2004 semester of2003-2004 academic year. The images from each subject are captured in indoor environ-ment. A brief summary of the FRGC database is given in Table 6.5. All these imagesare used for virtual user creation. Figure 6.7(c) shows two face images of two differentsubjects.

Virtual Users: A sample (instance) of a virtual user consists of three images from threebiometric traits, namely iris, fingerprint and face. The virtual users are created from theabove mentioned databases. We consider those subjects from the databases which haveat least two samples because we need at least one sample for enrollment and one samplefor probe. Note that there are 372, 1080 and 466 subjects have at least two samples foriris, fingerprint and face biometric databases, respectively. From these three databaseswe can create 372 virtual users (subjects) with three biometric traits. To create virtualuser (subject), first, we select all samples of a subject from the database of each biometrictrait. The subject is selected randomly. Then, a sample image from each biometric traitof the subject is taken and used as a sample image of virtual user. It may be noted thateach subject has some uniqueness with respect to each biometric trait. Hence, we arenot using the subjects or samples to create a virtual user which are already selected for

Table 6.5: Summary of biometric databases and virtual users

Database Modality # uniquesubjects

# sample/subject Size Remark

CASIAV3I Iris 395 1 to 26 2639 372 subjects have at least 2 samplesand use to create virtual users

WVU Fingerprint 1080 3 to 20 7136Minutiae points are extractedsuccessfully from 750 subjects whichare used to create virtual users.

FRGC Face 466 2 to 44 4007 All 466 subjects are used to createvirtual users

Virtualusers

iris +fingerprint +

face372 2 to 20 2625 Subjects are selected randomly from

each trait

157


(a) Sample iris images (b) Sample finger-print images

(c) Sample faceimages

Figure 6.7: Sample images of CASIAV3I iris, WVU fingerprint and FRGC face databases.

a virtual user. This helps us to create virtual users with unique multi biometric traits.In this procedure, we create 372 virtual users and each virtual user has 2 to 20 samples,resulting in a total of 2625 samples. The summary of virtual users is given in Table 6.5.


To evaluate the performance of the proposed approach, we divide all samples of all virtualusers into two sets: Gallery and Probe. The samples in the Gallery set are enrolled intothe index database and samples in the Probe set are used as query to search the indexdatabase. We use 80% samples of each subject to create the Gallery set and other 20% tocreate the Probe set. We select samples of the Gallery set randomly from each subject.

We have done our experiments with an Intel Core2Duo processor (2.00 GHz) and2.0-GB memory. We use GCC 4.3 compiler to develop our program.

6.10.3 Training of SVM-based Score Fusion Module

We use SVM-based score fusion techniques to combine the scores of different traits andthe SVM needs to train before using it in score fusion. We train the model [98,118,119]

with known training samples. The training samples contain the scores of all traits andtheir classes. The class of the training sample is genuine if the score is calculated from

158


the samples of the same subject and imposter if the samples are from different subjects.To create training samples, first, we select 100 subjects with 2 to 10 samples per subjectform the Gallery set. From these selected samples, we create 2,498 genuine and 2,77,636imposter pairs. Then, we calculate the scores between each pair of sample for eachbiometric trait using Eq. (6.3) given in Section 6.2 and normalized the scores using theRHE score normalization method described in Section 6.5.1. Thus, a total of 2,80,134training data have been created. Each training data has been assigned either genuineor imposter class based on pairs (genuine or imposter). We train the SVM with thesetraining data and perform 5-fold Cross validation [98, 119] to measure the performancewith training data. We observed 96.75% cross validation accuracy, that is, the SVMtraining is of good classification accuracy. We use SVMligtht tool [118, 119] for trainingand testing the SVM model.

6.10.4 Training of SVM-based Ranking Module

To train the SVM in rank level fusion, we need an enrolled database, a set of querysamples and the retrieved candidate sets for each query with their ranking. For thispurpose, we randomly select 1 sample from 100 subjects as query from the gallery setand enroll the rest of the samples of Gallery set into the database. For each querysample, we retrieve four candidate sets corresponding to four biometric traits using theproposed retrieving technique (see Section 6.8). To create training data for SVM ranking,we have to assign a rank to each candidates retrieved in the candidate set. To do this,we create feature vector corresponding to each candidate retrieved candidate set usingthe proposed method described in Section 6.9.2. We give an initial rank to each trainingdata using Eq. (6.29) as mentioned in Section 6.8. In this way, we create 100 trainingsets from the 100 query samples. We perform 5-fold cross validation [98,120] with thesetraining data and observe 92.46% cross validation accuracy. The SVMRank tool [117,120]

is used to train and test the SVM in ranking module.

6.10.5 Validation of the Parameter Values

We use three parameters in our proposed indexing approach. The first parameter is thenumber of reference subjects (M) which is used in index key generation. The secondparameter is the size of the table (LB) in index space which is used in storing thebiometric data into the index space. The third parameter is the number of neighbor cells(δ) of a table. We validate the values of these three parameters experimentally. Theexperimental validations are given in the following.

159


6.10.5.1 Number of Reference Subjects (M)

In our approach, we generate the index keys with respect to a number of referencesubjects. The number of index keys depends on the number of reference subjects. If weconsider more number of index keys from more number of reference subjects then we canachieve better HR, however, it also increases the PR. So, we should choose the numberof reference subjects in such a way that it gives good HR with low PR. To do this,we perform experiment with different number of reference subjects (M = 5, 10, 15, 20and 25). To evaluate the result without biasing toward reference subject we remove thesamples of all reference subjects from the gallery and probe sets and measure the HR

and PR. The result is shown in Fig. 6.8. We observe that the HR remains almost samethough the number of reference subjects is more than 10 and the PR increases for morethan 10 reference subjects. Hence, we consider 10 reference subjects in our approach.

6.10.5.2 Size of Table (LB)

The HR and PR of the proposed indexing technique depend on the number of entriesinto a cell of the index table. If number of entries into the cell are more then the chancesof finding a query subject within the cell increase; as a result HR increases and vice-versa. At the same time, the PR also increases as more number of samples in the cellwhich need to be retrieved at the time of querying. The number of entries depends onthe number of cells of a table which is referred as table size. The number of entriesinto a cell decreases when table size increases and vice versa. Hence, we measure theHR and PR of different number of enrolled samples with different table sizes. In our

�

�

�

�

�

�

�

�

� �

:

":

0:

2:

3:

6:

1:

7:

8:

9:

"::

6 ": "6 0: 06

w

O�$��%��

D$

)$

Figure 6.8: HR and PR with different number of reference subjects.

160


experiment, we calculate sizes of the table by taking different percentages of the totalnumber of enrolled samples. We consider the table size 10% to 100% of enrolled sampleswith step of 10% increment. We have done our experiments with 500, 1000, 1500 and2000 enrolled samples and results are presented in Fig. 6.9. From Fig. 6.9(a), we observethat HR decreases rapidly beyond the table size as 30% of the total enrolled samples fordifferent number of enrollments. Whereas in Fig. 6.9(b), we can see that PR decreaseswhen the table size is less than 20% of the total enrolled samples for different numberof enrollments. Hence, we choose the size of the table as 20% of the total number ofenrolled samples.

6::

":::

"6::

0:::

":w0:w2:w3:w6:w1:w7:w8:w9:w"::w

7:

8:

9:

"::

O��

��

* ��N�

D$

(a) HR with different number of enrolled samples and differenttable sizes

6::

":::

"6::

0:::

":w0:w2:w

3:w6:w1:w8

":

"0

"3

"1

"8

0:

O��

��

* ��N�

)$

(b) PR with different number of enrolled samples and differenttable sizes

Figure 6.9: HR and PR with different number of enrolled samples and different tablesizes.

161


6.10.5.3 Number of Neighbor Cells (δ)

For an index key, we retrieve a set of stored identities from a cell and its neighborhoodcells of a table. If we consider more number of cells then the HR will increase butthe PR will also increase. Hence, there should be a choice for selecting the number ofneighborhood cells (δ). For this purpose, we have conducted an experiment with differentvalues of δ. We choose the values of δ from 1 to 10 and measure the HR and PR ateach δ value. The result is presented in Fig. 6.10. From Fig. 6.10, we can see that byincrementing the value of δ from 1 to 5 the HR is increased from 95.81% to 99.55% withthe increase of PR 2.57%. However, incrementing the value of δ from 5 to 10, the HR

increases only 0.30% with the increase of PR 4.93%. The value of δ = 5 gives 99.55%HR at 13.86% PR. We select the value of δ = 5 in our experiment.

6.10.6 Evaluation

We judge the efficiency of our proposed indexing technique with respect to accuracy,searching time and memory requirement. After selecting the 10 reference subjects, ourGallery set contains 1957 samples and Probe set contains 668 samples from 372 subjectswith iris, fingerprint and face biometric traits. We evaluate the experiments with theseGallery and Probe sets. In our experiment, we perform the indexing using iris, fingerprintand face trait separately, and different combinations of these three traits (multimodal)also. �

�

�

�

�

�

�

�

�

�

:

0:

3:

1:

8:

"::

: 0 3 1 8 ":

w

J

)$

D$

Figure 6.10: HR and PR for different values of δ with the proposed indexing technique.

162


6.10.6.1 Accuracy

To analyze the accuracy of the proposed approach, we measure HR and PR. We reportthe HR and PR achieved by indexing with iris, fingerprint, face and different combina-tions of these three traits in Table 6.6. From Table 6.6, we can see that the indexingwith three traits gives 99.55% HR which is higher than the indexing with uninodal orcombination of any two traits. However, the PR is little higher than the unimodal orcombination of any two traits.

We also substantiate our result in terms of CMS which gives the probability of atleast one correct identity present within a top rank. How CMS varies with rank isshown in Fig. 6.11 as CMC curve for indexing with each unimodal tarits as well as forthe multimodal indexing. From Fig. 6.11, we see that 91.62%, 92.96% and 86.98% CMSs

Table 6.6: HR and PR of the proposed indexing technique with unimodal and multimodaltraits

Biometric trait HR PR

Iris 93.56 14.63

Fingerprint 95.96 12.98

Face 90.27 15.86

Iris+Fingerprint 97.75 17.62

Fingerprint+Face 97.21 16.44

Iris+Face 93.86 17.18

Iris+Fingerprint+Face 99.55 17.77

�

�

�

�

�

�

�

�

� �

7:

76

8:

86

9:

96

"::

: 6 ": "6 0: 06 2:

��

�

$ �(

��W��W� ��

��

��

� ��

��W��

��W� ��

��W� ��

Figure 6.11: CMC curves of the proposed indexing technique with different combinationof biometric traits.

163


are possible at the top 30 ranks for indexing with iris, fingerprint and face, respectively.On the other hand, indexing with the combination of three traits gives 99.25% CMS atthe 30th rank.

Further, we analyze the performance of the proposed method with respect to FPIR

and FNIR. To do this, first we calculate FMR and FNMR as follows. We match eachquery template of the Probe set with each template in the Gallery set using SVM clas-sification. We choose 3738 genuine pairs and 13,03,538 imposter pairs from the Gallery

and Probe, and calculate the genuine score and imposter scores for each genuine andimposter pairs, respectively. Finally, we calculate FNIR and FPIR for the identifica-tion system without indexing and with indexing using Eq. (3.20) and (3.21), respectively,The trade-off between FPIR and FNIR for the identification system without indexingis shown in Fig. 6.12. Figure 6.12 also shows the trade-off between FPIR and FNIR

for indexing with iris, fingerprint, face and combining of three traits. From our experi-mental results, it may be interpreted that 5.22% FNIR can be achieved at 1% FPIR

without indexing where as 2.72%, 2.43%, 3.04% and 2.42% FNIR can be achieved at1% FPIR for iris, fingerprint, face and multimodal (combination of iris, fingerprint andface) indexing. From Fig. 6.12, also we can observe that using our proposed indexingapproach we can achieve a lower FPIR for an FNMR.

�

�

�

� �

:

"

0

2

3

6

1

7

8

9

":

: ": 0: 2: 3: 6: 1:

�'

�$

�)�$

L��

��A��

��A��

��A��

��

Figure 6.12: FPIR versus FNIR curves of the proposed identification system withoutany indexing and with iris, fingerprint, face and multimodal based indexing.

164



First, we analyze the run-time complexity with big-O notation for gallery match cor-responding to a query sample. Let N be the total number of samples enrolled in thedatabase from P number of individuals and an individual has L number of samples. Inour approach, for given a query template, constant time is required to find a positionin the table and retrieve the list of identities (IDL) stored at the position for a keyvalue of a query index key. Adding these list into a set SB can be done in constanttime. There are 10 index keys and each has 4 key values. Hence, the time complexity ofretrieving candidates from the table for a given query is O(1). We process each retrievedcandidate to create feature vector for rank level fusion. Let IL be the average number ofcandidates retrieved from the database. Note that IL << N , N be the total number ofsamples enrolled in the database. Then, the feature generation process for ranking canbe accomplished in time O(IL). We can compute rank of all IL candidates using SVMranking in O(IL) time. In the worst case, when all samples are stored in one position,then IL is equal to N , which is very unlikely to occur.

We also analyze the search efficiency by measuring the average time taken to retrievethe templates from the database for a given query sample. Let tp be the average time toperform operation like addition, subtraction, and assignment operations. Our indexingapproach requires six comparisons to retrieve candidates corresponding to one key valuesof a query index key and a candidate set of size IL for a single trait is retrieved using10 key values of all index keys, one corresponding to each reference subject. There aretotal four candidate sets. Therefore, the time taken to retrieve a candidate set of sizeIL is (tp × 6) × 40. The number of features generated for ranking is IL and requiresIL× tp × 4. Let tsv be the time taken to calculate SVM rank score for a feature vector.Therefore, IL×tsv time is required to calculate rank of IL feature set. Let ts be the timeto compare query template with one stored template for matching. Hence, the searchtime using our proposed indexing approach is (tp×6)×40+IL×tp×4+IL×tsv+IL×ts

where tp < tsv < ts. On the other hand, linear search requires N × ts time. Thus, ourindexing approach takes less time than the linear search approach because IL << N .

We have given the average retrieving time taken in indexing with iris, fingerprint,face and multimodal for different database sizes in Table 6.7. The average time taken tosearch a query without indexing is also reported in Table 6.7. We can observe that theaverage retrieving time remains almost same with the increasing size of the databases.From Table 6.7, we can see that our approach perform faster than the searching withoutindexing.

165


Table 6.7: Average retrieving time for a query with different database sizes.

With indexing Without indexing

Retrieving time (in ms) Searching time (in ms) Searching time (in ms)

DBSize Ir Fp Fa Multi Ir Fp Fa Multi Ir Fp Fa Multi

500 0.012 0.013 0.013 0.038 0.041 0.151 0.214 0.751 0.157 1.785 5.214 8.451

1000 0.015 0.013 0.014 0.041 0.059 0.195 0.317 1.611 0.249 2.912 12.847 12.791

1500 0.014 0.015 0.013 0.039 0.082 0.227 0.392 1.957 0.412 5.765 14.972 23.857

1957 0.016 0.015 0.015 0.042 0.117 0.231 0.479 2.685 0.578 6.181 18.298 35.685

Ir → Iris, Fp → Fingerprint, Fa → Face, Multi → Iris + Fingerprint + Face


We calculate the memory requirement to store the identities into the database. In ourapproach, we consider four tables corresponding to each reference subject in an indexspace and ten reference subjects in the database. Hence, there are 40 tables in thedatabase. Each table stores the identities of all samples. Let 4 bytes memory is requiredto store the subject identity of a sample and 1 byte is required to store the sampleidentity of the subject. We can store 232 subjects and 256 samples per subject using 5bytes memory. Suppose, each cell requires 4 bytes memory to store the reference of theidentity list (IDL). The table size (LB) depends on the number of enrollments. In ourapproach, we consider the table size is 20% of the number of enrollments. If there are N

samples need to be enrolled, then we can calculate the memory requirement (MemoryN )using Eq. (6.31).

MemoryN = (4× LB +N × 5)× 40 (6.31)

We calculate the memory requirement for different number of enrollments and resultis shown in Fig. 6.13. We can see that the memory requirement increases linearly withthe increment of database size.


The objective of our work matches with the work done by Gyaourova et al. [90]. Gyaourovaet al. [90] proposes multimodal indexing by applying two fusion techniques. In first tech-nique, they concatenate the index code of face and fingerprint and then retrieve the datafrom database using concatenated index code. In another technique, they first retrievethe data from database using individual index code of face and fingerprint and then take

166

6.12. Summary

�

�

� �

:

6

":

"6

0:

: 6:: "::: "6:: 0::: 06::

��

��

�4\

�5

Q"

::

:

O��

Figure 6.13: Memory requirements for different number of enrolled samples in the proposedindexing technique.

the union of the retrieved identities. To compare our approach, we evaluate our approachand their approaches on the same database of fingerprint and face trait. The score used inexisting an proposed approach is calculates by our score calculation method. We comparethe results with respect to HR, PR, dimensionality of index code, search complexity andretrieving time. Table 6.8 shows the comparison results. From Table 6.8, we see that ourapproach gives the almost same HR at low PR with low dimensional index key than theapproaches proposed in [90]. Here, we use only two unimodal traits (fingerprint and face)and combination of fingerprint and face as multimodal trait. Hence, our approach usesa total of 30 dimensions index keys (10 index keys each with 3 dimensions) for face andfingerprint based indexing whereas Gyaourova et al. approach [90] uses 500 dimensionsindex keys (2 index keys with 250 dimensions). Further, the searching complexity of ourapproach is O(1) however, O(N) for the approach [90]. Again our approach retrieves thecandidate set from the database in less time than the approach in [90].

6.12 Summary

In this chapter, we propose a new indexing mechanism to reduce the search space for amultibiometric-based identification system. In our approach, only ten reference subjectsare used to generate index code. The reference subject-based index key generation givesthe high discriminant index codes from different subjects. Our proposed storing andindexing mechanism for multimodal biometric identification system is novel. The storingand indexing mechanisms support for any number of biometric traits. Our indexing

167


Table 6.8: Comparison with existing multimodal indexing techniques [90]

Traits HR PRSearch

complexityDimensionalityof index code

Retrievingtime (ms)

Concatenating in-dex code [90]

fingerprint+ face 95.46 19.78 O(N) 500 0.183

Union of candi-date [90]

fingerprint+ Face 96.95 23.85 O(N) 500 0.275

Proposed approach fingerprint+ face 97.21 16.44 O(1) 30 0.042

mechanism allows us to retrieve a small set of identities from the database in O(1)

time. We also propose SVM-based rank level fusion to combine the retrieved identitiesusing different traits. Proposed indexing technique is tested with iris, fingerprint andface biometric traits. We have generated only four dimensional index key. We havetested our approach with a set of virtual users which is created from the most popularCASIAV3I iris, WVU fingerprint and FRGC face databases. The experimental resultsshows that 99.55% HR can be achieved at 17.77% PR with the combination of iris,fingerprint and face biometric traits. From the experimental results using unimodaland different combination of the traits, we can also observe that combining differenttraits gives better performance than any unimodal traits. Comparison with existingwork shows that our approach gives better performance than existing approaches. Ourapproach takes on the average 0.042 millisecond to retrieve a small set of identities withdatabase size 1957 for a query sample. Please note that, this time remains constant eventhe database size increases. Also, we may conclude that our approach is applicable tobiometric identification systems with large multimodal biometric data and accomplishesan identification in real time without compromising the accuracy.

168

Chapter 7

Conclusions and Future Research

The main objective of our research is to explore efficient indexing mechanisms for someunimodal traits as well as multimodal biometrics. The outcomes of our research arediscussed in Chapter 3 to 6 in this thesis. In this chapter, we summarize some salientfeatures so far the research contributions are concerned. This research considers someassumptions, which may raise about the validity of our claim. We point out the threatsto validity of those considerations. Finally, we give some future research directions inour area of research.

7.1 Dimensionality of Index Key Vector

In biometric data indexing, an important judgement is about the dimensionality of theindex key vector. The dimensionality of index vector should be in such a manner that itreduces the search space significantly and at the same time retrieves a minimum numberof candidates without compromise in accuracy. In the following, we summarize how wehave achieved this objective with respect to different biometric data.

Majority of the iris-based indexing techniques consider the creation of index key fromiris texture pattern [153,154,163,164]. Among these texture based methods, SIFT featurebased method [154] gives the better result to the best of our knowledge. This method uses128-dimensional index key vector. In contrast to the existing practices, this is the firsttime we propose iris biometric based indexing mechanism using Gabor energy features ofiris texture. The Gabor energy features allow us to derive only 12-dimensional index keyfor iris biometric. Indeed this is quite low dimension of index key vector in comparisonto the several existing work. The summary of iris feature representation is shown inTable 7.1.

169

7. Conclusions and Future Research

From our study of existing literature, we have observed that minutiae based fin-gerprint indexing uses low dimensional index key vector for fingerprint data indexing.Moreover, the MCC based fingerprint indexing [54] is treated as the most efficient minu-tiae based indexing technique. This technique utilizes at least 15 dimensional index keyvector. On the other hand, our feature extraction technique using two closest point tri-angulation method generates 8-dimensional index key vectors. To add more, the scaleand rotation invariant properties of index keys make our proposed indexing scheme morerobust compared to other higher dimensional index key vector reported elsewhere. Ta-ble 7.1 shows the summary of fingerprint features.

To achieve the improved accuracy of the indexing system, the existing face biometricdata indexing techniques advocate high dimensional index key vector [123,133,159]. Ac-cording to the reported work, we have observed that the method proposed by Kaushik etal. [123] gives better accuracy. This technique uses 128-dimensional index key vectors. Incontrast, our proposed face biometric data indexing mechanism considers 69-dimensionalindex key vectors. We may note that in our 69-dimensional face indexing mechanism,the first four dimensions of the index key vector are constituted from the key point infor-mation, which are used to index the face data. The next 64 dimensions, which containthe SURF feature descriptors information, are used to match the face template. The lastdimension of the index key vector is the identity of a subject. We represent the summaryof face feature in Table 7.1.

To the best of our exhaustive literature survey on multimodal biometric data index-ing, only a few number of research work have been reported. Out of these reported work,the approach proposed in [90] performs better than the other approaches. However, thisapproach uses 256-dimensional index key vector to index the multimodal biometric dataof face and fingerprint traits. On the other hand, our approach generates 4-dimensional

Table 7.1: Feature representations of the different indexing approaches

Indexing methodFeature representation

Features Dimensions

Iris Gabor energy features 12

Fingerprint Geometric and Gabor energy featuresextracted from two-closest point triangles 8

Face SURF key points and descriptors 69

Multimodal Relative scores 4

170

7.2. Storing and Retrieving

index key vector from the relative match scores (see Table 7.1). Further, the relativescores are calculated against a set of reference subjects corresponding to different uni-modal traits as well as multimodal traits. It may please be noted that we consider threebiometric traits: iris, fingerprint and face biometric traits and combine the scores usingSVM-based score level fusion technique which enable us to solve the problem with sucha low-dimensional index key vector.

7.2 Storing and Retrieving

Storage structure is another important issue in any indexing mechanism. It may be notedthat storage structure should vary depending on the index key and the data structurealso highly influences the retrieving efficiency. Further, indexing technique needs extramemory overhead. We summarize our investigation to achieve the best storage structurefor the indexing of iris, fingerprint, face and multimodal biometric data in the following.

Existing iris data indexing techniques use different storing structures which are morerelevant to the traditional retrieval techniques [112, 164, 185]. The tree-based storagestructure is commonly used to store the iris data. Although, the retrieving efficiency intree-based storing may not be acceptable for a large number of entries into the database.On the contrary, we use table-based storage structure in our iris biometric data indexingtechnique. We have maintained a table for a dimension of the iris index key vector andstored all iris data into the table based on the value of that dimension in the proposedindex space. This proposed storage structure helps us to retrieve a small set of candidatesfrom the database in constant time using our proposed retrieving mechanism. Further,the low dimensionality of the index key vector reduces the memory overhead for the irisindexing system.

From the current literature, we can see that majority of the fingerprint indexing ap-proaches [43,51,88] follow continuous classification mechanism and for this purpose theyuse linear storage structure. On the other hand, to store the fingerprint data, we haveused three different storing structures: linear, clustered and clustered kd-tree. In our fin-gerprint based indexing approach, k-means clustering technique is applied on fingerprintindex keys to cluster the fingerprint data. We have experimented three search techniquesfor these three storing structures. We have observed that clustered and clustered kd-tree searches efficiently reduce the search space of fingerprint data. Our approach dealswith low dimensional index key vector, hence, we need less indexing memory overhead.Nevertheless, our clustered kd-tree based storage requires an extra memory overhead tostore the cluster and kd-tree information.

171


Existing face-based indexing techniques hardly explore the storing structures of theface-based indexed data [123, 159]. In our proposed face based indexing, biometric dataare stored into a two level index space in the database. In the first level, we have dividedthe face data into two groups based on the first dimension of the index keys and insecond level, a 3-dimensional index cube is created based on the next three dimensionsof the index keys. We have used a linear or kd-tree structure within a cell of an indexcube to store the face data. A hash function is applied on the index keys to store andretrieve the face data. The proposed storing and retrieving methods allow us to efficientlygenerate an accurate candidate set with similar template for a given query. However, theproposed storing technique needs a small memory to store the two level index space intothe database as we use only four dimensions of the index key vector to create the indexspace.

Existing work on multimodal biometric data indexing use linear or kd-tree storagestructures and follow linear and kd-tree based searching [90,111]. On the other hand, weuse table-based storage structure to store the identities of multimodal biometric data andfollow hashing technique to retrieve the identities from the database. We have stored thesubject identities into the database based on the feature values corresponding to differenttraits in index key vector and retrieved a candidate set for each biometric trait. Further,we apply rank level fusion technique by applying SVM rank to combine the retrievedcandidates. As we store the multiple entries of a subject identity, we would admit thatour approach requires a bit extra memory for this purpose. As a leverage of the extramemory overhead, our proposed technique sufficiently narrow downs the search spaceand precisely retrieves the most similar templates.

7.3 Performance of Indexing Techniques

The performance of a biometric based identification systems relies on the efficiency andaccuracy of the indexing technique. The experimental results of the proposed indexingmechanisms for unimodal and multimodal biometric traits substantiate the efficiencyand accuracy of our proposed methods. In Table 7.2, we compare the performancesof different indexing techniques exercised in our research. We summarize the outcomesof our proposed indexing techniques for iris, fingerprint, face and multimodal biometrictraits in the following.

The index space organization of iris data allows us to retrieve iris data from thedatabase in constant time without compromising the accuracy. An exhaustive evaluationhas been done with different iris databases. We have achieved on the average 82.79% rank

172

7.3. Performance of Indexing Techniques

one HR and 13.78% PR for all iris databases (see Table 7.2). Our approach is also capableto achieve 96.75% CMS on an average at the 30th rank as shown in Fig. 7.1. The methodhas also been shown to perform better than the best of the existing approaches [154].The experimental results indicate that our iris biometric data indexing approach can beapplied to any real time iris biometric-based identification system which deals with alarge number of iris biometric data.

Our fingerprint-based indexing approach ables to achieve a higher accuracy with theproposed low dimensional index key. We have performed an extensive study with differentfingerprint databases and the average results of all fingerprint databases in clustered kd-tree based indexing are reported in Table 7.2 which indicates that fingerprint indexingcan achieve on the average 83.83% rank one HR and 14.05% PR and retrieve a set ofsimilar fingerprint templates in the order of milliseconds. From Fig. 7.1, we can see

Table 7.2: Performances of the different indexing approaches

Indexing method HR PRRetrieving time

(ms)Search

complexity

Iris 82.79 13.78 0.015 O(1)

Fingerprint 83.83 14.05 1.236 O(√N)

Face 93.52 9.3 104.1 O(logN)

Multimodal 96.11 13.86 0.042 O(1)

�

�

�

�

8:

86

9:

96

"::

: 6 ": "6 0: 06 2:

��

�

$ �(

��

��

� ��

��

Figure 7.1: CMC curves of different indexing approaches.

173


that our approach is able to achieve 99.83% CMS at the 30th rank. Further, betteraccuracy can be achieved in linear or cluster based fingerprint data indexing though theretrieving time is higher than clustered kd-tree based indexing technique. Depending theapplications’ requirement, we can utilize any one of these indexing technique. We mayconclude that the proposed clustered kd-tree based indexing technique outperforms theexisting techniques.

Experimental results on different face databases prove that our proposed face indexingapproach provides better results than the existing techniques [123,133,159]. The averageperformance on different face databases is reported in Table 7.2. We achieve on theaverage 93.52% rank one HR and 9.30% PR. Our approach, in fact, performs the facedata retrieval in the order of millisecond. Also, our approach can achieve on the average96.69% CMS (see Fig. 7.1). Moreover, the experimental results establish the potentialof the proposed approach for handling a large face database of a face biometric basedidentification system.

Comprehensive evaluation on a virtual user database shows the effectiveness of ourproposed multimodal biometric data indexing approach. The proposed multimodal bio-metric indexing approach can achieve 96.11% rank one HR and 13.86% PR and retrievesa candidate set in constant time (see Table 7.2). We can also achieve 99.25% CMS atthe 30th rank (see Fig. 7.1). We have observed that our approach performs better thanthe existing approaches [90, 111]. It is evident from the experimental results that ourproposed multimodal biometric indexing approach can be applied to any large scale ap-plication. Further, the proposed approach can handle any number of biometric traits fora multimodal biometric based identification system.

7.4 Threats to Validity

Our experiments of proposed indexing mechanisms are involved with different biomet-ric databases and different parameters. Eventually, the experimental results we havereported in this thesis are subjected to the validity of the available resources and as-sumptions on values of parameters. In the following, we discuss the validity of ourexperiments and experimental results.

7.4.1 Internal Validity

In our proposed indexing mechanisms, some parameters may affects the experimentalfindings. Hence, we asses the internal validity [126] of the proposed systems based onthe different parameters in the following.

174

7.4. Threats to Validity

Our iris biometric data indexing approach uses table based storing structure wherethe length of the tables are decided a priory from the range of feature vectors. If wewant to enroll a new sample whose feature values are the out of the range, then we needto reorganize the table. Again, multiple samples of a subject are used at the time ofenrollment for iris biometric data indexing. If some subjects have only single sample, theaccuracy of the indexing system may be affected.

We use unsupervised k-means clustering to group the fingerprint data. In our pro-posed fingerprint-based indexing technique, cluster centers need to recalculate if a newsubject comes for enrollment. Further, the number of cluster we have chosen

√N as a

rule of thumbs for k-means clustering where N denotes the total number of fingerprintdata to be enrolled. However, this can be decided by examining the cluster distributionfor different number of clusters when the number of data to be enrolled is huge.

The number of cells of an index cube in the second level index space for face dataindexing also affects the experimental results of the proposed face indexing approach.The number of cells may need to change when the number of face data in databasechanges.

In our multimodal biometric indexing system, we use SVM to combine the scores ofunimodal biometric traits and rank the retrieve data. We have adequately experimentedto select the training data from the virtual multimodal database for SVM based scorefusion module and rank module. However, the performance of SVM modules is dependanton the training data set. Hence, the use of different multimodal databases in the proposedmultimodal biometric needs to retrain the SVM modules.

7.4.2 External Validity

We validate the factors which may limit the generalization of experimental results. Allof our proposed unimodal indexing methods are tested with the different unimodaldatabases (BATH, CASIAV3I, CASIAV4T, MMU3 and WVU iris database for iris bio-metric indexing, NIST DB4, NIST DB4 Natural and FVC 2004 fingerprint databases forfingerprint indexing, and FERET and FRGC database for face biometric indexing) withmoderate size which are available for the research communities. Most of these databasesare created in controlled experimental setup. So, we should not claim the results appli-cable to any type of biometric databases. Moreover, to establish the results it needs tobe validated with other databases, which we could not access during our experiments.Further, the size of the databases are in the order of thousands. Hence, the use of verylarge size databases which are in the order of millions may slightly affect the performanceof the proposed approaches. We only consider the frontal face images in face biometric

175


based indexing system. Hence, results of face biometric indexing may be affected forthe other face profiles (left face profile, right face profile, etc.) and occluded face imagesas the feature extraction is difficult from these types of images. Further, the proposedmultimodal biometric indexing technique is adequately tested with the virtual users’database. Hence, we should not claim that the performance of the proposed multimodalindexing will remain same for the users’ datanase with real multibiometric data.

7.4.3 Construct Validity

We would also like to assess how well the theories are implemented into actual programs.Our proposed method used Gabor feature extraction technique [149] for iris, Hong etal. [96] method to extract minutiae points for fingerprint and SURF method [36] forface features. The existing literature shows that these methods are well established andgive better performance than the other methods. However, the other feature extractiontechniques could be applied to cross validate the outcomes. Further, in our multimodalbiometric based indexing system, we use Daugman’s [66, 67], Jain et al. [106] and Duet al. [75] method to extract features and calculate scores for iris, fingerprint and facebiometric traits, respectively. These feature extraction and score calculation methodsare treated as the best in their respective domains. Nevertheless, the other techniquesfor feature extraction and score calculation methods could also be checked to confirm theresults.

The performance of our system is measured by two metrics: HR, PR, CMS andtrade of between FPIR and FNIR. These metrics are commonly used to measurethe performance of biometric indexing systems [43, 164, 169]. This way our approachconfirms the construct validity. However, metrics such as identification probability [153,154] and bin miss rate [154] could be of other interest. Since, these are the metrics usuallyinsignificant when we use HR, PR, CMS and trade-off between FPIR and FNIR toestablish the claim of efficiency, we ignore these in our work like the research practice inthis area of research.

7.5 Future Scope of Work

While we have made significant improvement in the development of biometric data index-ing methods that facilitate the design of fast and reliable biometric identification systems,this thesis opens up several future avenues for research. Some of them are mentionedbelow.

176

7.5. Future Scope of Work

• Most of the algorithms for different biometric data indexing, including ours, arestudied with the biometric databases whose sizes are in the order of thousands.There is no such large database in the order of millions for the research community.So, creating a large biometric database with different biometric traits is a promisingtopic of research.

• The performance of the biometric data indexing algorithms degrade when the qual-ity of the captured biometric traits is low. Hence, there is a scope for research tohandle the low quality biometric traits for indexing purpose.

• In this thesis, we have used the biometric images which are captured from coop-erative users. That means users cooperate with the system when the biometricsamples are taken from the users. Biometric data indexing with non-cooperativeusers’ data is a challenging research area.

• We investigate the Gabor energy features in our iris data indexing technique. Infuture, other texture feature extraction techniques for iris biometric can be exploredto improve the accuracy of iris data indexing.

• A well-known limitation of the k-mean clustering is that an inappropriate choice ofk may yield poor results. In our fingerprint data indexing approach, we use k-meanclustering technique to group the fingerprint data. Other clustering techniques canbe investigated in future research.

• Apart from the geometry of the minutiae triplets and texture pattern around theminutiae point, many minutiae-based fingerprint matchers use additional attributeslike minutia type, ridge counts, ridge curvature, ridge density and local texturefeatures to achieve high recognition rates. These attributes can also be incorporatedinto the fingerprint-based indexing in future study.

• We have proposed face biometric data indexing for frontal face image using SURFfeatures. However, SURF method may not good for side profile or occluded faceimages to extract features. So, these types of images demand great attention forfurther research.

• In our multimodal biometric data indexing, we have explored SVM-based scorefusion technique. Though, to the best of our knowledge this technique performsbetter than other reported techniques till date. On the other hand, accuracy ofSVM-based technique rely on the training data. Hence, further research can possi-ble to improve the accuracy of the score fusion technique.

177


• In this thesis, we have used three biometric traits. However, there are several otherbiometric traits used in different applications. The indexing with the biometrictraits other than iris, fingerprint and face are yet to be addressed.

• Security of the biometric templates is another concern in recent days. To preventthe theft of biometric data, cancelable templates are generated from the differentbiometric traits. Generating cancelable biometric index key for the security ofbiometric data would be thought as a new area of research.

• Finally, a formal analysis for cost-benefit of a biometric indexing system based onparameters such as performance gain (HR, PR, CMS), speed up, physical cost ofthe system and security needs to be developed in order to enable biometric systemdevelopers to rapidly design an indexing system that is most appropriate for theapplication on hand.

178

References

[1] “AADHAAR,” Unique Identification Authority of India. [Online]. Available:http://uidai.gov.in/ (Accessed on July, 2012).

[2] “Bath Iris,” University of Bath Iris Image Database. [Online]. Available:http://www.bath.ac.uk/elec-eng/research/sipg/irisweb/index.html (Accessed onAugust, 2008).

[3] “BIOSECURE,” The BioSecure Multimodal Database. [On-line]. Available: http://biosecure.it-sudparis.eu/AB/index.php?option=com_content&view=article&id=11&Itemid=14 (Accessed on November, 2012).

[4] “CalTech 256,” CalTech 256 Face Database. [Online]. Available: http://www.vision.caltech.edu/Image_Datasets/Caltech256/ (Accessed on October,2012).

[5] “Canadian Passenger Accelerated Service System (CANPASS),” Canada BorderServices Agency. [Online]. Available: http://www.cbsa-asfc.gc.ca/prog/canpass/menu-eng.html (Accessed on September, 2012).

[6] “CANPASS - Air.” [Online]. Available: http://www.cbsa-asfc.gc.ca/prog/canpass/canpassair-eng.html (Accessed on November, 2010).

[7] “CASIA Iris Database,” CASIA-IrisV3-Interval. [Online]. Available: http://www.cbsr.ia.ac.cn/IrisDatabase.htm (Accessed on September, 2010).

[8] “CASIA Iris Database,” CASIA-IrisV4-Thousand. [Online]. Available: http://www.cbsr.ia.ac.cn/IrisDatabase.htm (Accessed on March, 2012).

[9] “CLEAR Program,” CLEAR Airport Security and Automate Security. [Online].Available: http://clearme.com/index (Accessed on September, 2012).

179

References

[10] “Crossing U.S. Borders.” [Online]. Available: http://www.dhs.gov/crossing-us-borders/ (Accessed on September, 2011).

[11] “Entering the UK,” Home Office UK Border Agency. [Online]. Available:http://www.ukba.homeoffice.gov.uk/customs-travel/Enteringtheuk/ (Accessed onSeptember, 2012).

[12] “FERET,” The Facial Recognition Technology (FERET) Database. [Online]. Avail-able: http://www.itl.nist.gov/iad/humanid/feret/feret_master.html (Accessed onDecember, 2011).

[13] “Find Biometrics,” Mobile Biometrics, PDAs & Laptop Fingerprint Readers.[Online]. Available: http://www.findbiometrics.com/mobile-biometrics/ (Accessedon May, 2012).

[14] “Find Biometrics,” Physical Access Control Biometrics. [Online]. Available:http://www.findbiometrics.com/physical-access/ (Accessed on June, 2012).

[15] “FLANN - Fast Library for Approximate Nearest Neighbors.” [Online]. Available:http://www.cs.ubc.ca/~mariusm/index.php/FLANN/FLANN (Accessed on May,2012).

[16] “FVC2004,” FVC2004: the Third International Fingerprint VerificationCompetition. [Online]. Available: http://bias.csr.unibo.it/fvc2004/databases.asp(Accessed on December, 2011).

[17] “IRIS,” Using the Iris Recognition Immigration System (IRIS). [Online].Available: http://www.ukba.homeoffice.gov.uk/travellingtotheuk/Enteringtheuk/usingiris/ (Accessed on November, 2010).

[18] “Iris Scans at Amsterdam Airport Schiphol.” [Online]. Available: http://www.schiphol.nl/Travellers/AtSchiphol/Privium/Privium/IrisScans.htm (Accessed onOctober, 2012).

[19] “MMU2,” Multimedia University: MMU2 Iris Image Database. [Online]. Available:http://pesona.mmu.edu.my/~ccteo/ (Accessed on November, 2008).

[20] “NIST FERET,” The Color FERET Database: Version 2. [Online]. Available:http://www.nist.gov/itl/iad/ig/colorferet.cfm (Accessed on February, 2012).

[21] “NIST FRGC,” Face Recognition Grand Challenge (FRGC). [Online]. Available:http://www.nist.gov/itl/iad/ig/frgc.cfm (Accessed on May, 2012).

180

References

[22] “OpenCV,” The Open Source Computer Vision (OpenCV) Library. [Online]. Avail-able: http://opencv.willowgarage.com/wiki/Welcome (Accessed on December,2011).

[23] “Passport Canada: ePassport.” [Online]. Available: http://www.ppt.gc.ca/eppt/index.aspx?lang=eng (Accessed on August, 2012).

[24] “Personal Identity Verification (PIV) of Federal Employees and Contractors.”[Online]. Available: http://www.nist.gov/itl/csd/set/piv.cfm (Accessed on August,2012).

[25] “SFinGe,” Synthetic Fingerprint Generator. [Online]. Available: http://bias.csr.unibo.it/research/biolab/sfinge.html (Accessed on January, 2012).

[26] “The Eyes Have It.” [Online]. Available: http://www.accessexcellence.org/WN/SU/irisscan.php (Accessed on September, 2010).

[27] “TSA,” Transportation Security Administration. [Online]. Available: http://www.tsa.gov/ (Accessed on September, 2012).

[28] “UIDAI,” Unique Identification Authority of India from Wikipedia. [Online].Available: http://en.wikipedia.org/wiki/Multipurpose_National_Identity_Card(Accessed on January, 2012).

[29] “UIDAI Strategy Overview,” Creating a Unique Identity Number for EveryResident in India. [Online]. Available: http://uidai.gov.in/UID_PDF/Front_Page_Articles/Documents/Strategy_Overveiw-001.pdf (Accessed on May, 2012).

[30] “US-VISIT Program.” [Online]. Available: http://ipv6.dhs.gov/files/programs/usv.shtm (Accessed on September, 2012).

[31] “WVU Iris Database,” Multimodal Biometric Dataset Collection, BIOMDATA,Release 1,. [Online]. Available: http://citer.wvu.edu/multimodal_biometric_dataset_collection_biomdata_release1 (Accessed on September, 2010).

[32] “NIST Special Database 4,” NIST 8-Bit Gray Scale Images of Fingerprint ImageGroups(FIGS), 1992. [Online]. Available: http://www.nist.gov/srd/nistsd4.cfm(Accessed on January, 2012).

[33] M. Abellanas, P. Bose, J. Garcia, F. Hurtado, M. Nicolas, and P. A. Ramos, “OnProperties of Higher Order Delaunay Graphs with Applications,” in Proceedings of

181

References

the 21st European Workshop on Computational Geometry, Eindhoven, The Nether-lands, March 2005, pp. 119–122.

[34] S. Arya and D. M. Mount, “Algorithms for Fast Vector Quantization,” in Proceed-ings of DCC ’93: Data Compression Conference, Snowbird, UT, 1993, pp. 381–390.

[35] S. Arya, D. M. Mount, N. S. Netanyahu, R. Silverman, and A. Y. Wu, “An Opti-mal Algorithm for Approximate Nearest Neighbor Searching in Fixed Dimensions,”Journal of the ACM, vol. 45, no. 6, pp. 891–923, 1998.

[36] H. Bay, A. Ess, T. Tuytelaars, and L. V. Gool, “SURF: Speeded Up Robust Fea-tures,” Computer Vision and Image Understanding, vol. 110, no. 3, pp. 346–359,2008.

[37] H. Bay, T. Tuytelaars, and L. V. Gool, “SURF: Speeded Up Robust Features,” inProceedings of the 9th European Conference on Computer Vision, vol. LNCS-3951,Graz, Austria, May 2006, pp. 404–417.

[38] G. Bebis, T. Deaconu, and M. Georgiopoulos, “Fingerprint Identification usingDelaunay Triangulation,” in Proceedings of the IEEE International Conference onIntelligence, Information, and Systems, Bethesda, USA, October-November 1999,pp. 452–459.

[39] P. Belhumeur, J. Hespanha, and D. Kriegman, “Eigenfaces vs. Fisherfaces: Recog-nition using Class Specific Linear Projection,” IEEE Transactions on Pattern Anal-ysis and Machine Intelligence, vol. 19, no. 7, pp. 711–720, 1997.

[40] J. L. Bentley, “Multidimensional Binary Search Trees Used for Associative Search-ing,” Communications of the ACM, vol. 18, no. 9, pp. 509–517, 1975.

[41] M. D. Berg, O. Cheong, M. V. Kreveld, and M. Overmars, Computational Ge-ometry: Algorithms and Applications, 3rd ed. Springer-Verlag New York Inc.,2008.

[42] J. R. Beveridge, D. Bolme, B. A. Draper, and M. Teixeira, “The CSU Face Identi-fication Evaluation System,” Machine Vision and Applications, vol. 16, no. 2, pp.128–138, 2005.

[43] B. Bhanu and X. Tan, “Fingerprint Indexing Bsed on Novel Features of Minu-tiae Triplets,” IEEE Transactions on Pattern Analysis and Machine Intelligence,vol. 25, no. 5, pp. 616–622, 2003.

182

References

[44] A. Bishnu, B. B. Bhattacharya, M. K. Kundu, C. A. Murthy, and T. Acharya,“Euler Vector for Search and Retrieval of Gray-tone Images,” IEEE Transactionson Systems, Man, and Cybernetics, Part B: Cybernetics, vol. 35, no. 4, pp. 801–812,2005.

[45] D. S. Bolme, J. R. Beveridge, and B. A. Draper, “FaceL: Facile Face Labeling,” inProceedings of the 7th International Conference on Computer Vision Systems, vol.LNCS-5815, Liege, Belgium, October 2009, pp. 21–32.

[46] D. S. Bolme, J. R. Beveridge, M. L. Teixeira, and B. Draper, “The CSU Face Iden-tification Evaluation System: Its Purpose, Features and Structure,” in Proceedingsof the 3rd International Conference on Computer Vision Systems, Graz, Austria,April 2003, pp. 304–313.

[47] D. S. Bolme, B. A. Draper, and J. R. Beveridge, “Average of Synthetic ExactFilters,” in Proceedings of the Computer Vision and Pattern Recoginition, Miami,Florida, June 2009, pp. 2105–2112.

[48] M. Brown and D. G. Lowe, “Invariant Features from Interest Point Groups,” inProceedings of the British Machine Vision Conference, Cardiff, Wales, September2002, pp. 656–665.

[49] M. Burrows and D. J. Wheeler, “SRC Research Report: A Block-sorting LosslessData Compression Algorithm,” Systems Research Center, Digital EquipmentCorporation, Palo Alto, CA, USA, Technical Report, 1994. [Online]. Available:http://www.eecs.harvard.edu/~michaelm/CS222/burrows-wheeler.pdf

[50] A. Califano and I. Rigoutsos, “FLASH: A Fast Look-up Algorithm for String Ho-mology,” in Proceedings of the IEEE Computer Society Conference on ComputerVision and Pattern Recognition, NY , USA, June 1993, pp. 353–359.

[51] R. Cappelli, “Fast and Accurate Fingerprint Indexing Based on Ridge Orientationand Frequency,” IEEE Transactions on Systems, Man, and Cybernetics, Part B:Cybernetics, vol. 41, no. 6, pp. 1511–1521, 2011.

[52] R. Cappelli, M. Ferrara, and D. Maio, “Candidate List Reduction based onthe Analysis of Fingerprint Indexing Scores,” IEEE Transactions on InformationForensics and Security, vol. 6, no. 3, pp. 1160–1164, 2011.

183

References

[53] R. Cappelli, M. Ferrara, and D. Maltoni, “Minutia Cylinder-Code: A New Rep-resentation and Matching Technique for Fingerprint Recognition,” IEEE Transac-tions Pattern Analysis and Machine Intelligence, vol. 32, no. 12, pp. 2128–2141,2010.

[54] R. Cappelli, M. Ferrara, and D. Maltoni, “Fingerprint Indexing Based on MinutiaCylinder-Code,” IEEE Transactions on Pattern Analysis and Machine Intelligence,vol. 33, no. 5, pp. 1051–1057, 2011.

[55] R. Cappelli, A. Lumini, D. Maio, and D. Maltoni, “Fingerprint Classification byDirectional Image Partitioning,” IEEE Transactions on Pattern Analysis and Ma-chine Intelligence, vol. 21, no. 5, pp. 402–421, 1999.

[56] R. Cappelli, D. Maio, and D. Maltoni, “Synthetic Fingerprint-Database Genera-tion,” in Proceedings of the 16th International Conference on Pattern Recognition,vol. 3, QC, Canada, August 2002, pp. 744–747.

[57] H. Caulfield, J. Fu, and S.-M. Yoo, “Artificial Color Image Logic,” InformationSciences, vol. 167, no. 1–4, pp. 1–7, 2004.

[58] R. Chakravarti and X. Meng, “A Study of Color Histogram based Image Retrieval,”in Proceedings of the Sixth International Conference on Information Technology:New Generations, Las Vegas, NV, 2009, pp. 1323–1328.

[59] A. Chatterjee, S. Mandal, G. M. A. Rahaman, and A. S. M. Arif, “Finger-print Identification and Verification System by Minutiae Extraction using ArtificialNeural Network,” International Journal of Computer and Information Technology(IJCIT), vol. 1, no. 1, pp. 12–16, 2010.

[60] P.-H. Chen, C.-J. Lin, and B. Schlkopf, “A Tutorial on ν-Support Vector Machines,”Applied Stochastic Models in Business and Industry, vol. 21, no. 2, pp. 111–136,2005.

[61] M. M. S. Chong, H. N. Tan, L. Jun, and R. K. L. Gay, “Geometric Framework forFingerprint Image Classification,” Pattern Recognition, vol. 30, no. 9, pp. 1475–1488, 1997.

[62] Y. D. Chun, N. C. Kim, and I. H. Jang, “Content-based Image Retrieval usingMulti-resolution Color and Texture Features,” IEEE Transactions on Multimedia,vol. 10, no. 6, pp. 1073–1084, 2008.

184

References

[63] S. Crihalmeanu, A. Ross, S. Schuckers, and L. Hornak, “A Protocol for Multibio-metric Data Acquisition, Storage and Dissemination,” West Virginia University,Lane Department of Computer Science and Electrical Engineering, Morgantown,WV, Tech. Rep., 2007.

[64] J. Daugman, “Uncertainty Relation for Resolution in Space, Spatial Frequency,and Orientation Optimized by Two-dimensional Visual Cortical Filters,” Journalof The Optical Society of America, vol. 2, no. 7, pp. 1160–1169, 1985.

[65] J. Daugman, “Biometric Personal Identification System Based on Iris Analysis,”US Patent (5 291 560), 1994.

[66] J. Daugman, “The Importance of being Random: Statistical Principles of IrisRecognition,” Pattern Recognition, vol. 36, no. 2, pp. 279–291, 2003.

[67] J. Daugman, “How Iris Recognition Works,” IEEE Transactions on Circuits andSystems for Video Technology, vol. 14, no. 1, pp. 21–30, 2004.

[68] J. Daugman, Encyclopedia of Biometrics. New York: Springer, 2009, ch. IrisRecognition at Airports and Border-Crossings, pp. 819–825.

[69] J. G. Daugman, “Complete Discrete 2d Gabor Transforms by Neural Networks forIimage Analysis and Compression,” IEEE Transactions on Acoustics, Speech, andSignal Processing, vol. 36, no. 7, pp. 1169–1179, 1988.

[70] J. G. Daugman, “High Confidence Visual Recognition of Persons by a Test ofStatistical Independence,” IEEE Transactions on Pattern Analysis and MachineIntelligence, vol. 15, no. 11, pp. 1148–1161, November 1993.

[71] S. Dey and D. Samanta, “An Efficient Approach to Iris Detection for Iris BiometricProcessing,” International Journal of Computer Applications in Technology, vol. 35,no. 1, pp. 2–9, 2009.

[72] S. Dey and D. Samanta, “An Efficient and Accurate Pupil Detection Method forIris Biometric Processing,” International Journal of Computers and Applications,vol. 32, no. 2, pp. 141–148, 2010.

[73] S. Dey and D. Samanta, “Fast and Accurate Personal identification based on IrisBiometric,” International Journal of Biometrics, vol. 2, no. 3, pp. 250–281, June2010.

185

References

[74] S. Dey and D. Samanta, “Iris Data Indexing Method using Gabor Energy Features,”IEEE Transactions on Information Forensics and Security, vol. 7, no. 4, pp. 1192–1203, 2012.

[75] G. Du, F. Su, and A. Cai, “Face Recognition using SURF Features,” in Proceedingsof the SPIE Pattern Recognition and Computer Vision (MIPPR 2009), vol. SPIE-7496, Yichang, China, October-November 2009, pp. 749 628–1–749 628–7.

[76] J. Feng and A. Cai, “Fingerprint Indexing using Ridge Invariants,” in Proceedingsof the 18th International Conference on Pattern Recognition, vol. 4, Hong Kong,August 2006, pp. 433–436.

[77] J. Fierrez-Aguilar, J. Ortega-Garcia, D. Garcia-Romero, and J. Gonzalez-Rodriguez, “A Comparative Evaluation of Fusion Strategies for Multimodal Bio-metric Verification,” in Proceedings of the IAPR International Conference on Audioand Video-based Person Authentication, Guildford, UK, June 2003, pp. 830–837.

[78] J. Fu, H. J. Caulfield, S.-M. Yoo, and V. Atluri, “Use of Artificial Color Filteringto Improve Iris Recognition and Searching,” Pattern Recognition Letter, vol. 26,no. 14, pp. 2244–2251, 2005.

[79] K. Fukunaga, Introduction to Statistical Pattern Recognition, 2nd ed. AcademicPress, 1990.

[80] D. Gabor, “Theory of Communication,” Journal of the Institution of ElectricalEngineers - Part III: Radio and Communication Engineering, vol. 93, no. 26, pp.429–457, 1946.

[81] R. B. Gadde, D. Adjeroh, and A. Ross, “Indexing Iris Images using the Burrows-Wheeler Transform,” in Proceedings of the IEEE International Workshop on Infor-mation Forensics and Security, Seattle, USA, December 2010, pp. 1–6.

[82] R. S. Germain, A. Califano, and S. Colville, “Fingerprint Matching using Trans-formation Parameter Clustering,” IEEE Computational Science and Engineering,vol. 4, no. 4, pp. 42–49, 1997.

[83] A. Gionis, P. Indyk, and R. Motwani, “Similarity Search in High Dimensions viaHashing,” in Proceedings of the 25th International Conference on Very Large DataBases, Edinburgh, Scotland, UK, September 1999, pp. 518–529.

[84] R. C. Gonzalez and R. E. Woods, Digital Image Processing, 2nd ed. Prentice Hall,2002.

186

References

[85] G. Griffin, A. Holub, and P. Perona, “Caltech-256 Object Category Dataset,” Cal-ifornia Institute of Technology, Pasadena, CA, USA, Tech. Rep., 2007.

[86] J. Gudmundsson, M. Hammar, and M. V. Kreveld, “Higher Order Delaunay Tri-angulations,” Computational Geometry, vol. 23, no. 1, pp. 85–98, 2002.

[87] Z. Guo and R. W. Hall, “Parallel Thinning with Two-subiteration Algorithms,”Communications of the ACM, vol. 32, no. 3, pp. 359–373, 1989.

[88] A. Gyaourova and A. Ross, “A Novel Coding Scheme for Indexing Fingerprint Pat-terns,” in Proceedings of the 7th International Workshop Statistical Pattern Recog-nition, Orlando, USA, December 2008, pp. 755–764.

[89] A. Gyaourova and A. Ross, “A Coding Scheme for Indexing Multimodal BiometricDatabases,” in Proceedings of the IEEE Computer Society Conference on ComputerVision and Pattern Recognition Workshops, Miami, FL, June 2009, pp. 93–98.

[90] A. Gyaourova and A. Ross, “Index Codes for Multibiometric Pattern Retrieval,”IEEE Transactions on Information Forensics and Security, vol. 7, no. 2, pp. 518–529, 2012.

[91] J. Hammond, “Biometric Traveler Screening Introduced in Orlando, DenverNext,” November 2010. [Online]. Available: http://www.examiner.com/article/biometric-traveler-screening-introduced-orlando-denver-next/ (Accessed on July,2012).

[92] J. Han and K.-K. Ma, “Rotation-invariant and Scale-invariant Gabor Features forTexture Image Retrieval,” Image and Vision Computing, vol. 25, no. 9, pp. 1474–1481, 2007.

[93] M. He, S.-J. Horng, P. Fan, R.-S. Run, R.-J. Chen, J.-L. Lai, M. K. Khan, and K. O.Sentosa, “Performance Evaluation of Score Level Fusion in Multimodal BiometricSystems,” Pattern Recognition, vol. 43, no. 5, pp. 1789–1800, 2010.

[94] K. Hollingsworth, T. Peters, K. W. Bowyer, and P. Flynn, “Iris Recognition usingSignal-Level Fusion of Frames From Video,” IEEE Transactions on InformationForensics and Security, vol. 4, no. 4, pp. 837–848, 2009.

[95] L. Hong and A. Jain, “Integrating Faces and Fingerprints for Personal Identifica-tion,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 20,no. 12, pp. 1295–1307, 1998.

187

References

[96] L. Hong, Y. Wan, and A. Jain, “Fingerprint Image Enhancement: Algorithm andPerformance Evaluation,” IEEE Transactions on Pattern Analysis and MachineIntelligence, vol. 20, no. 8, pp. 777–789, 1998.

[97] S.-J. Horng, Y.-H. Chen, R.-S. Run, R.-J. Chen, J.-L. Lai, and K. O. Sentosal, “AnImproved Score Level Fusion in Multimodal Biometric Systems,” in Proceedings ofthe International Conference on Parallel and Distributed Computing, Applicationsand Technologies, Higashi Hiroshima, Japan, December 2009, pp. 239–246.

[98] C.-W. Hsu, C.-C. Chang, and C.-J. Lin, “A Practical Guide to SupportVector Classification,” Department of Computer Science and InformationEngineering, National Taiwan University, Taipei, Taiwan, Tech. Rep., 2003.[Online]. Available: http://www.ee.columbia.edu/~sfchang/course/spr/papers/svm-practical-guide.pdf

[99] J. Huang, S. Kumar, M. Metra, W. J. Zhu, and R. Zabith, “Spatial Color Indexingand Applications,” International Journal of Computer Vision, vol. 35, no. 3, pp.245–268, 1999.

[100] T. Huang, Z. Xiong, and Z. Zhang, Handbook of Face Recognition, 2nd ed.Springer, 2011, ch. Face Recognition Applications, pp. 617–638.

[101] P. Indyk and R. Motwani, “Approximate Nearest Neighbors: Towards Removingthe Curse of Dimensionality,” in Proceedings of the 30th Annual ACM Symposiumon Theory of Computing, Dallas, Texas, United States, May 1998, pp. 604–613.

[102] A. Jain, K. Nandakumar, and A. Ross, “Score Normalization in Multimodal Bio-metric Systems,” Pattern Recognition, vol. 38, no. 12, pp. 2270–2285, 2005.

[103] A. K. Jain, P. Flynn, and A. Ross, Eds., Handbook of Biometrics, 1st ed. NewYork: Springer-Verlag, 2007.

[104] A. K. Jain, L. Hong, S. Pankanti, and R. Bolle, “An Identity-authentication Systemusing Fingerprints,” Proceedings of the IEEE, vol. 85, no. 9, pp. 1365–1388, 1997.

[105] A. K. Jain, S. Prabhakar, and L. Hong, “A Multichannel Approach to FingerprintClassification,” IEEE Transactions on Pattern Analysis and Machine Intelligence,vol. 21, no. 4, pp. 348–359, 1999.

[106] A. K. Jain, S. Prabhakar, L. Hong, and S. Pankanti, “Filterbank-based FingerprintMatching,” IEEE Transactions on Image Processing, vol. 9, no. 5, pp. 846–859,2000.

188

References

[107] A. K. Jain, A. Ross, and S. Prabhakar, “An Introduction to Biometric Recognition,”IEEE Transactions on Circuits and Systems for Video Technology, vol. 14, no. 1,pp. 4–20, 2004.

[108] R. Jain, R. Kasturi, and B. Schunck, Machine Vision, 1st ed. New York: McGraw-Hill Inc, 1995.

[109] U. Jayaraman, S. Prakash, Devdatt, and P. Gupta, “An Indexing Technique forBiometric Database,” in Proceedings of the International Conference on WaveletAnalysis and Pattern Recognition, vol. 2, Hong Kong, China, August 2008, pp.758–763.

[110] U. Jayaraman, S. Prakash, and P. Gupta, “Indexing Multimodal BiometricDatabases using kd-tree with Feature Level Fusion,” in Proceedings of the 4th Inter-national Conference on Information Systems Security, Hyderabad, India, December2008, pp. 221–234.

[111] U. Jayaraman, S. Prakash, and P. Gupta, “An Efficient Technique for IndexingMultimodal Biometric Databases,” International Journal of Biometrics (IJBM),vol. 1, no. 4, pp. 418–441, 2009.

[112] U. Jayaraman, S. Prakash, and P. Gupta, “An Efficient Color and Texture basedIris Image Retrieval Technique,” Expert Systems with Applications, vol. 39, no. 5,pp. 4915–4926, 2012.

[113] A. Jensen and A. L. Cour-Harbo, Ripples in Mathematics: The Discrete WaveletTransform, ser. Berlin, Heidelberg. 1st: Springer-Verlag, 2001.

[114] X. Jiang, M. Liu, and A. C. Kot, “Fingerprint Retrieval for Identification,” IEEETransactions on Information Forensics and Security, vol. 1, no. 4, pp. 532–542,2006.

[115] X. D. Jiang, “Extracting Image Orientation Feature by using Integration Operator,”Pattern Recognition, vol. 40, no. 2, pp. 705–717, 2007.

[116] T. Joachims, Advances in Kernel Methods - Support Vector Learning. MIT Press,1999, ch. Making Large-Scale SVM Learning Practical, pp. 169–184.

[117] T. Joachims, “Optimizing Search Engines using Clickthrough Data,” in Proceedingsof the ACM Conference on Knowledge Discovery and Data Mining, New York, USA,August 2002, pp. 133–142.

189

References

[118] T. Joachims, “Training Linear SVMs in Linear Time,” in Proceedings of the ACMConference on Knowledge Discovery and Data Mining, New York, USA, August2006, pp. 217–226.

[119] T. Joachims, “SVMlight: Support Vector Machine,” Cornell University, 2008.[Online]. Available: http://svmlight.joachims.org/ (Accessed on May, 2012).

[120] T. Joachims, “SVMrank: Support Vector Machine for Ranking,” CornellUniversity, 2009. [Online]. Available: http://www.cs.cornell.edu/People/tj/svm_light/svm_rank.html (Accessed on May, 2012).

[121] T. Joshi, S. Dey, and D. Samanta, “Multimodal Biometrics: State of the Art inFusion Techniques,” International Journal of Biometrics (IJBM), vol. 1, no. 4, pp.393–417, 2009.

[122] K. Karu and A. K. Jain, “Fingerprint Classification,” Pattern Recognition, vol. 29,no. 3, pp. 389–404, 1996.

[123] V. D. Kaushik, A. K. Gupta, U. Jayaraman, and P. Gupta, “Modified GeometricHashing for Face Database Indexing,” in Proceedings of the 7th international con-ference on Advanced Intelligent Computing Theories and Applications: with Aspectsof Artificial Intelligence, Zhengzhou, China, August 2011, pp. 608–613.

[124] J. Kim, S. Cho, J. Choi, and R. J. Marks-II, “Iris Recognition using WaveletFeatures,” Journal of VLSI Signal Processing Systems, vol. 38, no. 2, pp. 147–156,2004.

[125] D. R. Kisku, P. Gupta, and J. K. Sing, “Fusion of Multiple Matchers using SVMfor Offline Signature Identification,” in Proceedings of the International ConferenceFuture Generation Information Technology-Security Technology, vol. CCIS-58, JejuIsland, Korea, December 2009, pp. 201–208.

[126] B. A. Kitchenham, S. L. Pfleeger, D. C. Hoaglin, and J. Rosenberg, “PreliminaryGuidelines for Empirical Research in Software Engineering,” IEEE Transactions onSoftware Engineering, vol. 28, no. 8, pp. 721–734, 2002.

[127] E. Krichen, S. Garcia-Salicetti, and B. Dorizzi, “A New Phase-Correlation-BasedIris Matching for Degraded Images,” IEEE Transactions on Systems, Man, andCybernetics, Part B: Cybernetics, vol. 39, no. 4, pp. 924–934, 2009.

[128] A. Kumar and D. Zhang, “Personal Recognition using Hand Shape and Texture,”IEEE Transactions on Image Processing, vol. 15, no. 8, pp. 2454–2461, 2006.

190

References

[129] S. Lee, Y. Kim, and G. Park, “A Feature Map Consisting of Orientation andInter-Ridge Spacing for Fingerprint Retrieval,” in Proceedings of the Audio andVideo-Based Biometric Person Authentication 9, New York, USA, July 2005, pp.184–190.

[130] M. S. Lew, N. Sebe, C. Djeraba, and R. Jain, “Content-based Multimedia Informa-tion Retrieval: State of the Art and Challenges,” ACM Transactions on MultimediaComputing Communication and Application, vol. 2, no. 1, pp. 1–19, 2006.

[131] J. Li, W. Y. Yau, and H. Wang, “Fingerprint Indexing Based on Symmetrical Mea-surement,” in Proceedings of the 18th International Conference on Pattern Recog-nition, vol. 1, Hong Kong, China, August 2006, pp. 1038–1041.

[132] X. Liang, A. Bishnu, and T. Asano, “A Robust Fingerprint Indexing Scheme us-ing Minutia Neighborhood Structure and Low-Order Delaunay Triangles,” IEEETransactions on Information Forensics and Security, vol. 2, no. 4, pp. 721–733,2007.

[133] K. H. Lin, K. M. Lam, X. Xie, and W. C. Siu, “An Efficient Human Face IndexingScheme using Eigenfaces,” in Proceedings of the IEEE International Conferenceon Neural Networks and Signal Processing, Nanjing, China, December 2003, pp.920–923.

[134] T. Lindeberg, “Feature Detection with Automatic Scale Selection,” InternationalJournal of Computer Vision, vol. 30, no. 2, pp. 79–116, 1998.

[135] M. Liu, , and P.-T. Yap, “Invariant Representation of Orientation Fields for Fin-gerprint Indexing,” Pattern Recognition, vol. 45, no. 7, pp. 2532–2542, 2012.

[136] M. Liu, X. Jiang, and A. C. Kot, “Fingerprint Retrieval by Complex Filter Re-sponses,” in Proceedings of the 18th International Conference on Pattern Recogni-tion, vol. 1, Hong Kong, China, August 2006, pp. 1042–1045.

[137] M. Liu, X. Jiang, and A. Kot, “Efficient Fingerprint Search Based on DatabaseClustering,” Pattern Recognition, vol. 40, no. 6, pp. 1793–1803, June 2007.

[138] T. Liu, G. Zhu, C. Zhang, and P. Hao, “Fingerprint Indexing based on Singu-lar Point Correlation,” in Proceedings of the International Conference on ImageProcessing, vol. 2, Genoa, Italy, September 2005, pp. 293–296.

[139] D. G. Lowe, “Distinctive Image Features from Scale-invariant Keypoints,” Interna-tional Journal on Computer Vision, vol. 60, no. 2, pp. 91–110, 2004.

191

References

[140] G. Lu, “Indexing and Retrieval of Audio: A Survey,” Multimedia Tools and Appli-cations, vol. 15, no. 3, pp. 269–290, 2001.

[141] H. Lu, X. Xue, and Y.-P. Tan, “Content-Based Image and Video Indexing andRetrieval,” in Proceedings of Joint Chinese-German Workshop, Cognitive Systems,Shanghai, China, 2007, pp. 118–129.

[142] J. Lu, K. Plataniotis, and A. Venetsanopoulos, “Face Recognition using LDA-BasedAlgorithms,” IEEE Transactions on Neural Networks, vol. 14, no. 1, pp. 195–200,2003.

[143] A. Lumini, D. Maio, and D. Maltoni, “Continuous versus Exclusive Classificationfor Fingerprint Retrieval,” Pattern Recognition Letters, vol. 18, pp. 1027–1034,1997.

[144] L. Ma, T. Tan, Y. Wang, and D. Zhang, “Personal Identification based on Iris Tex-ture Analysis,” IEEE Transaction on Pattern Analysis and Machine Intelligence,vol. 25, no. 12, pp. 1519–1533, 2003.

[145] L. Ma, T. Tan, Y. Wang, and D. Zhang, “Efficient Iris Recognition by Characteriz-ing Key Local Variations,” IEEE Transactions on Image Processing, vol. 13, no. 6,pp. 739–750, 2004.

[146] W. Y. Ma and B. S. Manjunath, “Image Indexing using a Texture Dictionary,”in Proceedings of the SPIE International Conference on Digital Image Storage andArchiving Systems, vol. SPIE 2606, Philadelphia, USA, October 1995, pp. 288–296.

[147] D. Maio, D. Maltoni, R. Cappelli, J. L. Wayman, and A. K. Jain, “FVC2004:Third Fingerprint Verification Competition,” in Proceedings of the InternationalConference on Biometric Authentication, Hong Kong, China, July 2004, pp. 1–7.

[148] D. Maltoni, D. Maio, A. K. Jain, and S. Prabhakar, Handbook of FingerprintRecognition, 2nd ed. London: Springer, 2009.

[149] B. S. Manjunath and W. Y. Ma, “Texture Features for Browsing and Retrieval ofImage Data,” IEEE Transactions on Pattern Analysis and Machine Intelligence,vol. 18, no. 8, pp. 837–842, 1996.

[150] P. Mansukhani, S. Tulyakov, and V. Govindaraju, “A Framework for EfficientFingerprint Identification using a Minutiae Tree,” IEEE Systems Journal, vol. 4,no. 2, pp. 126–137, 2010.

192

References

[151] K. V. Mardia, J. T. Kent, and J. M. Bibby, Multivariate Analysis. AcademicPress, 1980.

[152] R. M. McCabe and E. M. Newton, Information Technology: American NationalStandard for Information Systems - Data Format for the Interchange of Finger-print, Facial, & Other Biometric Information - Part 1, ANSI/NIST-ITL 1-2007Revision of ANSI/NIST-ITL 1-2000, National Institute of Standards and Technol-ogy Std., 2007.

[153] H. Mehrotra, G. S. Badrinath, B. Majhi, and P. Gupta, “Indexing Iris BiometricDatabase using Energy Histogram of DCT Subbands,” in Proceedings of the In-ternational Conference on Contemporary Computing, vol. CCIS-40, Noida, India,August 2009, pp. 194–204.

[154] H. Mehrotraa, B. Majhia, and P. Gupta, “Robust Iris Indexing Scheme using Ge-ometric Hashing of SIFT Keypoints,” Journal of Network and Computer Applica-tions, vol. 33, no. 3, pp. 300–313, 2010.

[155] B. M. Mehtre, “Fingerprint Image Analysis for Automatic Identification,” MachineVision and Applications, vol. 6, no. 2-3, pp. 124–139, 1993.

[156] A. Mhatre, S. Chikkerur, and V. Govindaraju, “Indexing Biometric Databasesusing Pyramid Technique,” in Proceedings of the 5th International Conference onAudio- and Video-Based Biometric Person Authentication, vol. LNCS-3546, NewYork, USA, July 2005, pp. 841–849.

[157] A. Mhatre, S. Palla, S. Chikkerur, and V. Govindaraju, “Efficient Search and Re-trieval in Biometric Databases,” in Proceedings of the SPIE Defense and SecuritySymposium, vol. SPIE-5779, Florida, USA, March 2005, pp. 265–273.

[158] N. Mittal and E. Walia, “Face Recognition using Improved Fast PCA Algorithm,”in Proceedings of the Congress on Image and Signal Processing, Sanya, Hainan,May 2008, pp. 554–558.

[159] P. Mohanty, S. Sarkar, R. Kasturi, and P. J. Phillips, “Subspace Approximation ofFace Recognition Algorithms: An Empirical Study,” IEEE Transactions on Infor-mation Forensics and Security, vol. 3, no. 4, pp. 734–748, 2008.

[160] A. Moore, “A Tutorial on Kd-trees,” Extract from PhD Thesis, University ofCambridge, Tech. Rep., 1991. [Online]. Available: http://www.cs.cmu.edu/~awm/papers.html

193

References

[161] G. M. Morton, “A Computer Oriented Geodetic Data Base and a New Techniquein File Sequencing,” International Business Machines Company, Ottawa, Canada,Tech. Rep., 1966. [Online]. Available: http://books.google.co.in/books?id=9FFdHAAACAAJ

[162] M. Muja and D. G. Lowe, “Fast Approximate Nearest Neighbors with AutomaticAlgorithm Configuration,” in Proceedings of the International Conference on Com-puter Vision Theory and Applications (VISAPP’09), 2009, pp. 331–340.

[163] R. Mukherjee, “Indexing Techniques for Fingerprint and Iris Databases,” Master’sthesis, College of Engineering and Mineral Resources at West Virginia University,2007.

[164] R. Mukherjee and A. Ross, “Indexing Iris Images,” in Proceedings of the 19th In-ternational Conference on Pattern Recognition, Tampa, Florida, USA, December2008, pp. 1–4.

[165] T. Ojala, M. Pietikainen, and T. Maenpaa, “Multiresolution Gray-Scale and Ro-tation Invariant Texture Classification with Local Binary Patterns,” IEEE Trans-action on Pattern Analysis and Machine Intelligence, vol. 24, no. 7, pp. 971–987,2002.

[166] J. Phillips and P. J. Flynn, “Overview of the Face Recognition Grand Challenge,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition,San Diego, USA, June 2005, pp. 947–954.

[167] P. J. Phillips, P. J. Flynn, T. Scruggs, K. W. Bowyer, and W. Worek, “PreliminaryFace Recognition Grand Challenge Results,” in Proceedings of the 7th InternationalConference on Automatic Face and Gesture Recognition, Southampton, UK, April2006, pp. 15–24.

[168] P. J. Phillips, H. Moon, S. A. Rizvi, and P. J. Rauss, “The FERET EvaluationMethodology for Face-Recognition Algorithms,” IEEE Transactions on PatternAnalysis and Machine Intelligence, vol. 22, no. 10, pp. 1090–1104, 2000.

[169] N. B. Puhan and N. Sudha, “A Novel Iris Database Indexing Method using the IrisColor,” in Proceedings of the 3rd IEEE Conference on Industrial Electronics andApplications (ICIEA 2008), Singapore, June 2008, pp. 1886–1891.

194

References

[170] N. K. Ratha, S. Y. Chen, and A. K. Jain, “Adaptive Flow Orientation-based FeatureExtraction in Fingerprint Images,” Pattern Recognition, vol. 28, no. 11, pp. 1657–1672, 1995.

[171] C. Rathgeb and A. Uhl, “Iris-Biometric Hash Generation for Biometric DatabaseIndexing,” in Proceedings of the 20th International Conference on Pattern Recog-nition, Istanbul, August 2010, pp. 2848–2851.

[172] K. Rerkrai and V. Areekul, “A New Reference Point for Fingerprint Recognition,”in Proceedings of the International Conference on Image Processing, Vancouver,Canada, September 2000, pp. 499–502.

[173] A. Ross and A. Jain, “Information Fusion in Biometrics,” Pattern Recognition Let-ters, vol. 24, no. 13, pp. 2115–2125, 2003.

[174] A. Ross and A. K. Jain, “Multimodal Biometrics: An Overview,” in Proceedings ofthe 12th European Signal Processing Conference, Vienna, Austria, September 2004,pp. 1221–1224.

[175] A. Ross and R. Mukherjee, “Augmenting Ridge Curves with Minutiae Tripletsfor Fingerprint Indexing,” in Proceedings of the SPIE Conference on BiometricTechnology for Human Identification IV, vol. 6539, Orlando, USA, April 2007, pp.65 390C–65 390C–12.

[176] A. Ross, K. Nandakumar, and A. K. Jain, Handbook of Multibiometrics. Springer,2006.

[177] C. Sanchez-Avila, R. Sanchez-Reillo, and D. de Martin-Roche, “Iris-based BiometricRecognition using Dyadic Wavelet Transform,” IEEE Aerospace and ElectronicSystems Magazine, vol. 17, no. 10, pp. 3–6, 2002.

[178] R. Singh, M. Vatsa, and A. Noore, “Intelligent Biometric Information Fusion usingSupport Vector Machine,” in Proceedings of the Soft Computing in Image Process-ing: Recent Advances, New Jersey, USA, December 2006, pp. 327–350.

[179] R. Snelick, U. Uludag, A. Mink, M. Indovina, and A. K. Jain, “Large-Scale Eval-uation of Multimodal Biometric Authentication using State-of-the-Art Systems,”IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 27, no. 3,pp. 450–455, 2005.

195

References

[180] R. Thai, “Fingerprint Image Enhancement and Minutiae Extraction,” HonoursProgramme Report, School of Computer Science and Software Engineering, TheUniversity of Western Australia, 2003. [Online]. Available: http://www.csse.uwa.edu.au/~pk/studentprojects/raymondthai/RaymondThai.pdf (Accessed on June,2011).

[181] M. Turk and A. Pentland, “Eigenfaces for Recognition,” Journal of Cognitive Neu-rosicence, vol. 3, no. 1, pp. 71–86, 1991.

[182] M. Turk and A. Pentland, “Face Recognition using Eigenfaces,” in Proceedings ofthe IEEE Conference on Computer Vision and Pattern Recognition, Maui, Hawaii,USA, 3-6 June 1991, pp. 586–591.

[183] U. Uludag, S. Pankanti, S. Prabhakar, and A. K. Jain, “Biometric Cryptosystems:Issues and Challenges,” Proceedings of the IEEE, vol. 92, no. 6, pp. 948–960, 2004.

[184] M. Vatsa, R. Singh, and A. Noore, “Reducing the False Rejection Rate of IrisRecognition using Textural and Topological Features,” International Journal ofSignal Processing, vol. 2, no. 1, pp. 66–72, 2005.

[185] M. Vatsa, R. Singh, and A. Noore, “Improving Iris Recognition Performance usingSegmentation, Quality Enhancement, Match Score Fusion, and Indexing,” IEEETransactions on Systems, Man, and Cybernetics, Part B, vol. 38, no. 4, pp. 1021–1035, 2008.

[186] S. P. Vingron, Switching Theory: Insight Through Predicate Logic. Springer-Verlag, 2004, ch. Karnaugh Maps, pp. 57–76.

[187] P. Viola and M. J. Jones, “Robust Real-time Face Detection,” International Journalof Computer Vision, vol. 57, no. 2, pp. 137–154, 2004.

[188] Y. Wang, J. Hu, and D. Phillips, “A Fingerprint Orientation Model based onFourier Expansion (FOMFE) and its Application to Singular-point Detection andFingerprint Indexing,” IEEE Transactions on Pattern Analysis and Machine Intel-ligence, vol. 29, no. 4, pp. 573–585, 2007.

[189] C. J. Watson and C. L. Wilson, “NIST Special Database 4, Fingerprint Database,”National Institute of Standards and Technology, Tech. Rep., 1992.

[190] H. Wolfson and I. Rigoutsos, “Geometric Hashing: An Overview,” IEEE Compu-tational Science and Engineering,, vol. 4, no. 4, pp. 10–21, 1997.

196

References

[191] D. Wu and L. Wu, “Image Retrieval based on Subband Energy Histograms ofReordered DCT Coefficients,” in Proceedings of the 6th International Conferenceon Signal Processing, vol. 1, Australia, December 2002, pp. 26–30.

[192] H. Yu and S. Kim, Handbook of Natural Computing . Springer, 2010, vol. I, ch.SVM Tutorial: Classification, Regression, and Ranking.

[193] L. Yu, D. Zhang, K. Wang, and W. Yang, “Coarse Iris Classification using Box-counting to Estimate Fractal Dimensions,” Pattern Recognition, vol. 38, no. 11, pp.1791–1798, 2005.

[194] W. Yu, X. Teng, and C. Liu, “Face Recognition using Discriminant Locality Pre-serving Projections,” Image and Vision Computing, vol. 24, no. 3, pp. 239–248,2006.

[195] P. Yuen and J. Lai, “Face Representation using Independent Component Analysis,”Pattern Recognition, vol. 35, no. 6, pp. 1247–1257, 2002.

[196] Y. Zhang, J. Tian, K. Cao, P. Li, and X. Yang, “Improving Efficiency of Finger-print Matching by Minutiae Indexing,” in Proceedings of the 19th InternationalConference on Pattern Recognition, Tampa, FL, December 2008, pp. 1–4.

[197] W. Zhao, R. Chellappa, A. Rosenfeld, and P. J. Phillips, “Face Recognition: ALiterature Survey,” ACM Computing Surveys, vol. 35, no. 4, pp. 399–458, 2003.

[198] Y. Zhu, T. Tan, and Y. Wang, “Biometric Personal Identification based on Iris Pat-terns,” in Proceedings of the 15th International Conference on Pattern Recognition,vol. 2, Barcelona , Spain, September 2000, pp. 801–804.

[199] Z.Wang and X. Li, “Face Recognition Based on Improved PCA Reconstruction,”in Proceedings of the 8th World Congress on Intelligent Control and Automation,Jinan, China, July 2010, pp. 6272–6276.

197

Publications out of this work

Published• Somnath Dey and Debasis Samanta, “Iris Data Indexing Method Using Gabor

Energy Features”, IEEE Transactions on Information Forensics and Security, Vol-7, No-4, pp. 1192-1203, 2012.

• Somnath Dey and Debasis Samanta, “An Efficient and Accurate Pupil DetectionMethod for Iris Biometric Processing”, International Journal of Computers andApplications, Vol-32, No-12, pp. 2-9, 2009.

• Somnath Dey and Debasis Samanta, “An Efficient Approach to Iris Detectionfor Iris Biometric Processing”, International Journal of Computer Applications inTechnology, Vol-35, No-2, pp. 141-148, 2010.

• Tejas Joshi, Somnath Dey and Debasis Samanta, “Multimodal Biometrics: Stateof the Art in Fusion Techniques”, International Journal of Biometrics (IJBM),Special Edition on: Multimodal Biometric Systems and Biometric Fusion, Vol-1,No-4, pp. 393-417, 2009.

• Tejas Joshi, Somnath Dey and Debasis Samanta, “A Two-stage Algorithm forCore Point Detection in Fingerprint Images”, Proceedings of the International Con-ference of IEEE R10- TENCON, IEEE Xplorer, pp.1-6, 2009, Singapore.

Communicated• Somnath Dey and Debasis Samanta, “An Efficient Approach to Multi-biometric

Data Indexing”, IEEE Transaction on Systems, Man, and Cybernetics, Part B.(Paper id: SMCB-E-2012-12-1219, Submitted on 06 November, 2012)

• Somnath Dey, Jyotirmay Dewangan and Debasis Samanta, “Indexing Face Im-ages for Person Identification Problem”, International Journal of Biometrics andBioinformatics (IJBB). (Paper id: 1278, Submitted on April, 2013)

• Somnath Dey, Om Prakash Singh and Debasis Samanta, “Fingerprint IndexingUsing Minutiae-Based Invariable Set of Multidimensional Features”, InternationalJournal of Biometrics (IJBM). (Paper id: IJBM_58991, Submitted on 8 June,2013)

199