*This research is supported by the CICYT project DPI 2010-17112.

1

Small Partial to Full Palmprint Matching based on Candidate Positions and Multiple

Correspondences*

by Carlos Moreno

MASTER THESIS

Mentored and Reviewed by

Francesc Serratosa

2

INDEX

1 INTRODUCTION………………………………………………………….………………………………….……4

2 STATE OF THE ART……………….…………………………………………………………………….……….6

2.1 Typical structure of a palmprint-based recognition system…………………..6

2.1.1 Image Acquisition……………………………………………………………………7

2.1.2 Preprocessing…………………………………………………………………..…….9

2.1.3 Feature Extraction…………………………………………………………………12

A. Holistic-Based Approaches……………………………………………12

B. Feature-Based Approaches……………………………………..……13

C. Hybrid Approaches…………………………………………………..….17

2.1.4 Classification Methods……………………………………………….…………17

A. Low Resolution Method Based on Image Correlation…19

B. Low Resolution Method Based on Principal Lines….……20

C. High Resolution Methods based on Minutiae………….….20

2.2 Minutiae-Based Palmprint Matching…………………………………………………….21

2.2.1 Minutiae Extraction……………………………………………………………….21

A. Edge Detection ……………………………………………………………..22

B. Filtered Directional Matrix …………………………………………...24

C. Segmentation…………………………………………………………………26

D. Binarization……………………………………………………………………28

E. Skeletonization……………………………………………………………...29

F. Minutiae Extraction……………………………………………………….32

2.2.2 Classical Minutiae-Based Matching System………………………….35

2.2.3 Current Minutia-Based Matching Systems………………………….38

2.3 Justification for a new algorithm…………………………………………………………..40

3 METHODS ………………………………………………………………………………………………………42

3.1 Problem with the classical minutiae matching approach……………………42

3.2 Partial to Full Palmprint Matching ……………………………………………………...43

3.2.1 Partial Palmprint Representation ……………………………………………44

3

3.2.2 Selecting position candidates ………………………………………………….45

3.2.3 Best candidate selection through multiple correspondences …47

4 PRACTICAL EVALUATION………………………………………………………………………………..51

4.1 Palmprints used …………………………………………………………………………………..51

4.1.1 Selected radius ……………………………………………………………………….52

4.1.2 Noise aggregation and rotation ………………………………………………53

4.2 Experiment conditions …………………………………………………………………………54

4.3 Tests for the location of the correct center ……………………………………………58

4.4 Tests for identification of a sample amongst a data set ………………………..60

5 CONCLUSIONS AND FURTHER WORK ……………………………………………………………..65

6 REFERENCES …………………………………………………………………………………………………..66

4

1 INTRODUCTION

Over the course of history, humanity has looked for a method in which a certain

individual can be identified and discerned from the rest of the society. Several methods

have been studied since the XVII century, when Dr. Nehemiah Grew first discovered

the uniqueness of certain elements in the human palm of an individual compared to

another one [1]. Meanwhile, other individuals focused their studies in aspects like the

human phrenology (the shape of the head) or the anthropometry (measurements of the

body). It was not until the year of 1823 when Jan Evangelista Purkinje realized that the

grooves and ridges of the human finger and palm seem to be unique to such individual.

This observation led to the foundation of dactyloscopy, a discipline led by Juan

Vucetich in 1888 who, as a police officer of Buenos Aires, Argentina, conducted the

process of taking the fingerprints of the current prison inmates and registering them by

inking them and printing them over paper. After this technique proved to be useful for

the solution of a criminal case [2], police agencies all over the world began adapting the

system to identify and connect individuals to a certain crime scene.

Figure 1.1 Fingerprint of Francisca Rojas, first person to be identified as guilty of assassination given

the existence of her fingerprint on a crime scene

Nowadays, researchers have identified numerous characteristics of the human being

that can be collected from a certain individual to try to differentiate him from the rest. A

chart presented on Figure 1.2 [3] shows the main features that biometric studies have

been recently focused in

5

Figure 1.2 Biometrical characteristics most commonly studied in human beings

Out of all of these systems, palmprint-based technology has proven to be a reliable

source of biometrical identification, since the human hand contains a variety of features

that can be extracted to perform a match between two images.

We intend to implement a novel algorithm for palmprint matching based on the

common limitations of other approaches, leaving aside the palmprint’s feature

extraction, which we consider has been researched broadly and fully developed. Our

method must not only be fast and reliable in the number of correct matches, but also

should match partial images as tiny as possible with their full counterparts, therefore

requiring as less sample as possible for the matching. Our system must be minutiae-

based (this terminology will be later explained on section 2) to be capable of performing

a latent and real matching. Finally, we also intend to design a system which is rotation

invariant, meaning that no matter the position of the partial palmprint found (i.e. a crime

scene palmprint), the system can still relate such sample with its full counterpart.

Over the course of this article, we will start on section 2 by describing the existent

state of the art on palmprint analysis. Then on section 3 we will explain our

methodology to find a correspondence from small to big sample. On section 4 we will

expose our practical evaluation. Section 5 is dedicated for conclusions and future work

for this project, ending with section 6 which is reserved for references.

6

2 STATE OF THE ART

2.1 Typical structure of a palmprint-based recognition system

Biometrics is a mean for automatic recognition of individuals based on their unique

anatomical characteristics, which results to be very useful to authenticate a person or to

identify people who left trails in a crime scene [4]. As computer interfaces become more

natural, biometrics have much boarder relevance since user authentication tends to be

more natural too. Although fingerprint has been the most popular biometrics solution,

lately, palmprint based systems are gaining the attention, since palmprint has more

features to be taken into consideration depending on the image resolution.

Every biometric authentication requires the comparison of an enrolled biometrical

sample with a new sample taken from another site. The enrollment is a three-step

process (capture, process, enroll) followed by a matching process. The matching can be

a verification (i.e., there is a given user id and its authenticity is verified) or

identification (i.e., find a user by a sample palmprint). Figure 2.1 shows the process

starting with the image acquisition, which will be processed for feature extraction, and

then enrolled in the system. Only relevant and novel methods will be described for each

step mentioned.

Figure 2.1 Biometric authentication processes

7

2.1.1 Image Acquisition

A palmprint system’s first step is always the image acquisition. Depending on the

image resolution, measured in dots per inch (dpi), different features can be extracted:

Low resolution images: With less than 400 dpi, we can appreciate the hand’s

geometry or identify wrinkles, principal lines and texture.

High resolution images: With at least 400 dpi or more, for features such as

singular points, minutiae and ridges can be obtained [5].

Palmprint identification systems rely on the knowledge of the parts of the human

hand and the characteristics that it possesses on the palm. To clear some concepts, the

following definitions are presented:

a) Hand Geometry: A hand can be seen as a whole geometrical figure and its outer

shape can be analyzed for human identification. Systems relying on the human

hand geometry are fast and simple to implement, however, they don’t work well

with a big dataset since as the number of people analyzed grows, the chances of

encountering two or more individuals with the same hand geometry increases.

Systems like this will also be deficient identifying and differencing the left and

the right hand, as well as discerning between a pair of twins. For the classification

phase, algorithms relying on graph matching are typically implemented. An

example of a hand with its geometrical structure extracted can be seen in the

following Figure 2.2

Figure 2.2 Hand geometry structure extracted from an image

8

b) Principal Lines: These are the three biggest creases that one can see in its own

hand which are formed due to the folding of our palm through the course of our

lifetime [6]. Figure 2.3 is presented to better identify these lines. A system based

on principal line extraction and classification would be more complex than a hand

geometry system, however the implementation and imaging will be still easy

since the features are still visible and have a notorious geometry,

Figure 2.3 A captured palmprint with line definition

c) Wrinkles: The wrinkles are the small lines that overlap the principal lines. These

lines have the same thickness of a principal line; however they have no individual

name since every human being has a set of different wrinkles. Wrinkles tend to be

ignored for biometrical purposes [6].

d) Ridges: Ridges are the thin creases that cover the entire palm and finger in a

quasi-continuous way. These lines are harder to see and discern for the human

eye, however, they can be obtained with a meticulous ink to paper method or with

commercial scanners. Ridges guarantee the uniqueness of a sample from

individual to individual.

e) Minutiae: Essential for fingerprint systems. Minutiae are understood as the part or

parts of a ridge where an alteration of the trajectory happens (i.e. a termination or

a bifurcation of the ridge). Minutiae have the special characteristic of

proportioning not only a location but a direction angle as well. Minutiae-based

systems will be discussed broadly on section 2.2 since the system presented on

this article relies on them. On Figure 2.4 we present a palmprint with some ridges

and a set of the most common minutiae (bifurcation, terminal and dot) selected.

9

Figure 2.4 A ridge pattern on a human palmprint with some minutiae selected

Initially, palmprint research focused on high resolution images. Recently research on

low resolution images is being done since it is more suitable for civil and commercial

applications because of smaller file sizes and shorter computation times. Therefore, they

are very useful for real-time applications.

According to the speed of a palmprint system, identification falls into two main

categories: offline and online systems. Offline systems use previously captured samples,

such as inked or scanned images, to perform verification. On the other hand, online

systems include a device which captures the samples directly transmitted to the

computer. Usually these devices are CCD (charge-coupled device), cameras or

scanners. Figure 2.3 shows an image captured with a CCD. In the past research, offline

systems were the main focus, but nowadays online systems are becoming more popular

because of real-time applications.

There are other alternative possibilities for image acquisition which will possibly be

very popular in the future, like 3D palmprint recognition. This alternative exploits the

3D structural information of the palm surface. In this approach the structured-light

imaging is used to acquire the 3D palmprint data [6].

2.1.2 Preprocessing

After acquiring the palmprint image according to the system’s needs, a step known as

“preprocessing” should be implemented. Preprocessing involves correcting distortions

and putting all palmprints under the same parameters so that the expected area of

10

analysis can be obtained for feature extraction [7]. According to the imaging method

used and the speed of the acquisition (offline or online), different preprocessing

methods are proposed.

Zhang [7] describes a method for offline imaging using datum point registration for

the palmprint alignment. First, the two endpoints “a” and “b” of the hand are located at

the beginning and the end of the head line. This mapping is better understood by

looking at the coordinates “a” and “b” marked in Figure 2.2. Afterwards, datum point

determination is implemented. The basic idea of this phase is to locate the endpoint of

each principal line and then, according to their regularity, the principal lines and their

endpoints are detected by using a directional projection algorithm. As a result, every

image is located with the midpoint “o” overlapped on the coordinate plane’s origin.

Another proposal included in [7] is discussed for offline imaging in inked images. To

develop such preprocessing technique, the coordinate system should be defined and then

the y-axis should be determined. First, a coordinate plane is defined outside of the hand

parallel to the outer bound. Afterwards, origin determination is made to ensure that the

coordinate plane origin is aligned with the beginning of the heart line (i.e., point b in

Figure 2.2). Finally, image rotation and translation is made to set every picture at the

same angle. The final result resembles the one on the Figure 2.5 This method allows

more computational efficiency than the previous one; however, it only works with inked

images since the origin cannot be established without the clear definition of the heart

line’s rear end.

Figure 2.5 Preprocessing result using the proposed system for inked palmprints

11

Somvanshi et al. [8] describe preprocessing for online images in five steps: Image

binarization, boundary extraction, detection of key points, definition of a coordinate

system and extraction of the central part. The binarization is made via a thresholding

method, while the boundary extraction and detection of key points may be made with

tangent or bisector based approaches. While some authors [9] opt for Otsu’s

thresholding algorithm, other proposals like [10] use an adaptive threshold algorithm

with calculation of Euclidean distances for the three first steps. Such algorithm leads to

simplicity, accurate positioning and high noise endurance. The definition of the

coordinate system is then made so that the axis are constant in every image, and finally

the image’s central part is removed in either a circular or rectangular shape, to focus

only on the main features of the hand, which are located on the sides. A circular central

part ensures a better rotation [11], while a rectangular extracted center simplifies the

image translation [8]. The image on Figure 2.6 clearly explains how a circular section

method proposed by Qichuan et al. [11] selects the circular central part of the image to

extract the main features of the palmprint.

Figure 2.6 Preprocessing of an online high definition palmprint

12

2.1.3 Feature Extraction

Feature extraction is the part of the process where the central part of the palmprint

image is taken, and a set of different characteristics is recognized for the matching. The

three most widely recognized approaches are holistic-based, feature-based and hybrid.

A. Holistic-based approaches

Holistic-based approaches are the ones in which the hand is considered as a whole,

hence, characteristics such as the geometry or inside lines are not essential. The

palmprint image is first represented as a spatial or transform domain, therefore treated

as any particular image. Subsequently, three different holistic feature extractions may be

used [12]. In the subspaced method, the image is converted into a vector, a two-

dimensional matrix or a second order tensor, with the aid of principal component

analysis (PCA). For the transform-based methods, Fourier transform or Discrete-Cosine

Transformation (DCT) is used to extract features form the image’s transform domain.

Finally, in the invariant moment-based technique proposed by Wu et al. [13], algebraic

features let the system consider each pixel of the palmprint image as a coordinate in a

high-dimensional image space. Although robust and effective for simple matching,

holistic-based approaches are mostly considered for low resolution images in which the

palm inner features are not easily seen. Moreover, a non-digital palmprint located on a

crime scene would be impossible to match with these methods, since the inked image

will share no characteristics with the palmprint on the database. With the aid of novel

techniques for image acquisition and higher resolution image databases, these methods

tend to be less used. A summary presented in Table 1 [12] presents the three categories

of holistic based approaches, as well as the most representative approaches of each

subcategory.

13

Table 1 Holistic-Based approach and its most representative works.

B. Feature-based approaches

A palmprint image, depending on the image resolution and acquisition method, may

have one or several of the following features, which were previously described on

section 2.1.2:

Hand Geometry (2D or 3D)

Principal lines

Wrinkles

Ridges

Minutiae

From here, given the best located feature in the image, techniques such as line-based

methods and coding-based methods are used for the classification and matching [12].

Line-based methods deal with the representation of certain lines chosen from the

palmprint, such as the principal lines, wrinkles or ridges (according to the resolution).

For example, the proposal made by Wu et al. [14] uses the first and second order

derivatives of a Gaussian 1-D function to convolve it with the original image matrix and

14

locate a certain line with a certain direction. Once the new matrix has been obtained, the

image is rotated to obtain several lines according to their direction. Finally, every

obtained image is overlapped to generate the total feature extraction. On Figure 2.7, the

structure of the line-based algorithm presented in [14] is shown, starting from the

original image and finishing with the extracted features.

Figure 2.7 (a) Original Image (b) 0o directional lines (c) 45o directional images (d) 90o directional

images (e) 135o directional images (f) resultant palm lines and (g) lines overlapped with the original palm

On the other hand, coded-based methods rely on encoding the response of a bank of

filters into bitwise code [12]. Since the palmprint features are represented as bits, this

type of approaches offers more speed and lower memory requirement, thus being very

popular for online palmprint matching. For example, the PalmCode method [15]

convolves the palmprint image with a series of 2D Gabor Filters, and then encodes the

phase of the filter responses as bitwise features. On figure 2.8 we can appreciate the

result of a feature extraction using such process, where the chosen feature was the

texture of the palmprint (a combination of the wrinkles and the visible ridges). A

preprocessed image (a) is convolved with a series of Gabor Filters resulting in (b) and

(c), with their respective masks (d). It can be clearly noticed that a feature extraction

done by an approach of this nature has less appeal for the human eye than a line-based

15

extraction like the one presented previously, however, its speed makes it more attractive

for palmprint matching.

Figure 2.8 (a) Preprocessed Image (b) real parts of texture image (c) imaginary parts of texture

features (d) corresponding masks

An improvement called the FusionCode method presented in [16] does the same

convolution; however, the Gabor Filters have different orientations, thus avoiding the

correlation of several bitwise features of the same palmprint, which happened often in

the PalmCode method. Figure 2.9 shows a FusionCode process and the use of the

images generated by PalmCode to form the new feature extraction image.

16

Figure 2.9 (a) Preprocessed Image (b) and (c) Filtered image’s real parts (Column 1) and imaginary

parts (Column 2) and PalmCode image’s real parts (Column 3) and imaginary parts (Column 4). (d)

FusionCode result.

This approach is even less appealing for the human eye in terms of recognition of the

features extracted; however, it still offers more speed than a line-based method. On

Table 2, which was originally presented in [12], a summary of the most significant

feature-based feature extraction methods is shown.

17

Table 2 Feature-Based approaches and their most representative works.

C. Hybrid Approaches

The characteristics of the previous systems can be combined to create a feature

extraction made with both holistic and feature-based characteristics. The combination

and accumulation of several features is reflected in higher accuracy for the matching, as

proven by Kong et al. and Kumar et al. mentioned by [12]. The disadvantage of these

approaches resides in the increase of computational resources and the complexity of

developing parallel algorithms working for a same image’s vast different characteristics,

reflected in a much more diverse research group and thus, a bigger economical cost.

These approaches should only be implemented for high security applications where the

economic cost is not a factor, such as military devices.

2.1.4 Classification Methods

Classification methods which rely on high resolution images take into account

minutiae to relate the sample of a previously obtained palmprint with a new one. An

algorithm to classify may resolve in two different ways [17]:

Both palmprints are similar according a certain scale (typically from 0 to 1)

A binary decision (Match/Non-Match).

18

In order for a certain palmprint matching algorithm to resolve that two samples are

the same, a certain threshold once again must be applied. Since the output of the first

solution is a continuous value, whenever such output surpasses the threshold, then a

Match is declared (assuming that 0 means no similarity and 1 means complete

similarity). Otherwise, the system would consider that both samples are from a different

person.

When comparing two palmprints and , where represents a previously registered

template on the system and represents a new input image in the system, various factors

can make both samples have a great variability. This continuous problem is the basis of

constant development of new matching algorithms. Some of the factors that increase

variability between two samples are:

1. Displacement: The same palmprint can be put in different positions in a certain

lecture area in two separate acquisitions. This generates a positional shift in the

image. For example, a displacement of just 2 mm can generate a shift of the

register of approximately 40 pixels with a common scanner of 500 dpi.

2. Rotation: The same palm can be put in different angles with respect to the

sensor’s vertical axis. A rotation that surpasses 20o

of separation usually cannot

relate two palms.

3. Partial Overlapping: Such displacement and rotation may cause that certain

areas of the print are lost.

4. Non-Linear Distortion: Provoked by the hand’s plasticity and the fact that a

tridimensional surface (as the skin) is captured by a bidimensional sensor, the

captured sample will have a distortion.

5. Skin Conditions: Ridges and valleys (the spaces between a pair of ridges) are

captured by the sensor if they are touching the sensor in a uniform way. However,

factors like non-constant pressure to the sensor, sweat drops, skin diseases, dirt or

scars can make the image change in two different samplings.

6. Noise: The same sensor can generate noise on the sample and distort the image.

For example, grease or dust which is left in the sensor’s screen may perturb the

image to be taken. Depending on the crystal’s quality, the image may vary from

sensor to sensor. One last example can be a system communication error between

19

the sensor and the computer storing the data, where a distortion of the original

data may occur according to the cable’s length or the communication protocol

used.

For forensic applications, this variability is even higher than the one in more ideal

applications like door sensors or live identification, since the sample is obtained from

a complete different environment that and most of the times we don’t even get a

complete sample, but just a piece of it. Since the matching algorithm depends on the

image resolution, different approaches are used; hence, low-resolution methods and

high-resolution methods can be applied.

A. Low Resolution Method Based on Image Correlation

A method described in [17] proposes two vectorized images and , corresponding

to two palmprints and , where represents the vector of a previously registered

template on the system and represents the vector of a new input image in the system.

A vectorized image is one which matrix has been converted into a vector by simply

concatenating the columns of the image.

One method to calculate the diversity of both images is by calculating the Sum of the

Squared differences (SSD) between the pixel intensities

where the super index denotes the transposed matrix.

If the terms 2 and 2 are constant and independent from the image correlation, the

diversity between the two vectorized images is minimized whenever the crossed relation

( ) t is maximized.

Since the term 2 t appears in a negative form in the diversity function of SSD,

then we can say that ( ) is a function of the similarity between both images. The

application of an algorithm like this requires the assumption that both images are

20

perfectly taken, and no noise or rotation is applied. However, on the practice we will see

that images are never taken equally from the same subject, and samples tend to have a

series of differences that we must consider before. If an algorithm like this wants to be

used, we must first realize that it should be made for each and every possible angle

rotation of the image, which will make the algorithm very slow.

B. Low Resolution Method Based on Principal Lines

A low resolution approach can use the palm’s principal lines, with algorithms such as

the stack filter [18], but these principal lines do not contribute adequately to high

accuracy because they are very similar between different people. Wrinkles are a good

option in low-resolution palmprint identification, but they are difficult to extract,

therefore texture analysis to palmprint recognition is applied. For doing so, a Gabor

filter is used. The Gabor filters are very popular in texture analysis and other types of

biometrics [19]. Each of the points of the resultant image from the Gabor filter is coded

to two bits. The feature vector matching is based on a normalized hamming distance.

Basically, a hamming distance is an XOR operation, which means that it is relatively

fast to compute. The matching score supports the image translation, but it does not

support totally the rotation of images. This can be solved by registering sub images

rotated by some degrees (e.g., -6º, -3º, 3º, 6º) and then extracting and storing their

features [5]. The disadvantage of this method is that template size becomes very big.

C. High Resolution Methods based on Minutiae

Low-resolution palmprint recognition is based on encoding and matching creases,

which are less reliable than ridges. Traditionally, ridges have been studied for

fingerprint matching techniques, however three main advantages can be found on

analyzing ridge patterns on palmprints rather than on fingerprints. First, a palmprint has

much more minutiae than a fingerprint. Second, minutiae tend to deform as a person

presses his finger when capturing the ridges with an ink on paper techniques. Third, on

a fingerprint there are many small regions to take into consideration; hence, they are

unable to appropriately handle distortion and noise due to a high computational cost. On

21

section 2.2 the minutiae based approaches will be discussed with a much broader

spectrum since the system presented on this article belongs to such category.

In an effort to summarize all the palmprint methodology discussed on section 2.1,

Figure 2.10 is presented.

Figure 2.10 Diagram of a typical palmprint matching system and the most common used options on

each step.

2.2 Minutiae-Based Palmprint Matching

2.2.1 Minutiae Extraction

Whenever we want to perform a system based on minutiae matching, both fingerprint

and palmprint systems offer the same complexity for minutiae extraction. Therefore, it

shouldn’t be strange if fingerprint examples are used to demonstrate the steps performed

in a minutiae palmprint extraction process. On this section, we will present the basic

notions to perform an offline feature extraction, line-based algorithm, for minutiae

extraction either on fingerprint or palmprint images.

22

A. Edge Detection

The first step to be performed in either a finger or a palmprint is the edge detection.

This procedure is not only used for biometric systems, but for other systems such as

image recognition and computer vision, where computer cameras need to identify a

certain shapes to perform diverse actions. Among the wide range of options that we can

encounter to perform edge detection, the majority of the different methods may be

grouped into two categories: Sobel Methods [20] and Laplace Methods. Figure 2.11

shows the expected result of an edge detection algorithm and the difference between the

two classes. Because of their simplicity and fast computational time, Sobel-based edge

detection methods are widely used in biometric systems.

Figure 2.11 Difference on edge detection between Sobel and Laplace Methods

Sobel Based Methods rely on the derivate of the original image, where it is shown

that a maximum at the center of the edge of the original signal can be located. To

perform a Sobel edge detection, first a derivate (or gradient) has to be extracted from the

image [17].

Let I be the original image in which we want to perform the edge detection. The

gradient ( ) of the coordinate ( ) of I is a bidimentional matrix such that

23

Where the components and are the horizontal and vertical derivatives of I in

the coordinate( ). To perform this step, the well-known Sobel algorithm relies on the

convolution of a predefined mask with the whole image I. Usually, the following 3x3

masks are used to generate ( ) and ( ) respectively. On Figure 2.12 we also

present an example of the gradients and formed in a fingerprint image.

Figure 2.12 Masks used to obtain the horizontal and vertical gradients

Images and show the change of intensity of the image on the horizontal or

vertical direction respectively, however, there is still more information that can be

deduced. Once that and are formed, two more images known as the consistency

and directional matrixes can be constructed as well. The consistency matrix ( ) is

defined as the norm of the gradient vector on the form

where ( ) )

Simultaneously, the orientation image ( ) represents the orthogonal angle of the

vector ( ), which is the direction of a hypothetical ridge (which so far is a change in

intensity from dark to light grey) in a certain coordinate ( ) . Therefore, to compute

( ) the following operation must be done

24

where ( ) ( ]

Since the orientation of a ridge is always orthogonal to the gradient’s angle, to form

the directional image ( ) we assume that

where ( ) ( ]

Given an original image I, on Figure 2.13 it can be appreciated the formation of the

four images (horizontal gradient ( ), vertical gradient ( ), consistency image

( ) and directional image ( )).

Figure 2.13 Generation of the Sobel Edge Detection Images on a palmprint

B. Filtered Directional Matrix

Once the previous images have been obtained, we need the formation of a Filtered

Directional Matrix (FDM), which is an image that contains the directional angles of the

25

original image, but with the important characteristic of the discarding of the angles not

belonging to the portions of the image that are not a hand. For this to happen, Ratha et

al. [21] propose the calculation of the covariance and variance between the vertical and

horizontal gradients, by using an E environment of n x n pixels, where n is typically set

to 9 [17]. After calculating the three gradients

We can finally calculate the final value of the mean of each angle by performing

where Ď represents the matrix containing the Filtered Directional image. On figure 2.14

we can see the evolution of a palmprint image evolving from its original representation I

(a), its directional matrix (b) its consistency matrix (c) and the resulting FDM (d). We

can clearly notice that (d) is an image which only resembles (a) in shape but not in the

interior design, since it is a representation of all the areas in which the directionality of

the ridges is similar.

Figure 2.14 Evolution of the processing of a palmprint image

26

C. Segmented Image

It may occur that the original image captured by a sensor doesn’t come specifically cut

for only the area of interest. In this type of circumstances, a segmentation algorithm is

needed to cut the parts of the image which don’t represent the finger or palm. Figure

2.15 explains that, by using the original image and the FDM, we can perform this

segmentation.

Figure 2.15 Generation of a Segmented Image

A method proposed also in [21] proposes the formation of a signature S, which is

defined as a one-dimensional rectangle centered on the pixel (x,y), which has n x n

elements and an inclination corresponding to an angle α [3]. This rectangle will go

through the whole image, meaning that every single pixel of the image will at some

point be the center of the signature. The inclination α of this signature will correspond

to the angle located on the coordinate (x,y) of the Ď. The constant n is preferably chosen

as an odd number, therefore creating a square that has an exact center pixel (x,y). On

Figure 2.16 (a) we can observe an example of how S (expressed as a white rectangle)

positions its center (x,y) at a certain pixel (xi,yi) of the image.

27

Afterwards, the signature is inclined α degrees, where α represents the angle located on

Ď(xi,yi). Finally, every value of a pixel inside S is used to create a histogram (b)

representing the column on the x axis and the summation of the color values of each

column on the y axis. In this example we could assume that n=33, therefore creating 33

columns of pixel color data in the histogram.

Figure 2.16 Signature execution and histogram formation

Therefore in this example, each column of the histogram will represent the 33 pixel

values of the 33 pixels of each of the columns of S. Once that this procedure has been

executed for a certain (x,y), we must calculate the variance of the data f by performing

where μ(f) represents the mean, which can be calculated as

If v(f) surpasses a certain threshold ts (previously set according to experimentation),

then the routine concludes that there is a lot of variation on that section of the image and

(xi,yi) will be considered as image. Otherwise, when few variation is found, the routine

concludes that the pixel (xi,yi) is part of the background, and therefore such pixel is

removed from the FDM by painting it with another color (such as red like in the

28

example of Figure 2.15). We will now refer to the FDM Ď as the Segmented Filtered

Directional Matrix (SFDM) Ď’.

D. Binarization

Usually palmprint and fingerprint identification methods rely on grayscale pictures

and samples, since color identification is not essential when identifying a subject. Once

the segmentation process is done, the original image I has to be transformed to a binary

image I’ in which ridges area fully colored in one tone, and the background in the

opposite tone. Image binarization is done in other computer vision processes, and in

palmprint identification is no exception, since this step is fundamental to find minutiae

or other singularities. Figure 2.17 shows a standard binarization process for any given

image.

Figure 2.17 (a) Original Image (b) Image after a binarization process

A typical and easy method for binarization consists in a threshold-based algorithm.

The algorithm goes over each and every pixel of the original image. If the value f of the

color of such pixel surpasses a threshold tc, then the pixel’s value is changed for 255 in

a gray scale. Otherwise, the value is changed for 0. On grayscale to black/white image

transformation, tc is commonly set to 128. An improvement of this method proposed by

[22] denotes that first, a proper threshold (called the optimum threshold) for the image

must be obtained before performing the binarization. This optimum threshold may be

obtained by various methods, amongst the most common one being first obtaining the

mean color of the image and then setting the optimum threshold around such value. Is

important to point out that a binarization algorithm applied in palmprint and fingerprint

29

imagery is enhanced if first we automatically convert all the pixels where Ď’ = {red}

(where the segmentation process indicated no image is found). Figure 2.18 resumes the

procedure previously described.

Figure 2.18 Generation of a binarized image

E. Skeletonization

In order for an algorithm to be implemented to find minutiae either in fingerprint or

palmprint samples, the ridges that have been found must be thinned as much as possible.

A pixel in an skeletonized image E can only take two values E(x,y) ϵ {0 , 255}, where

the color black E(x,y) = 0 represents the pixel of a ridge which thickness is equal to 1

30

pixel, and the color white E(x,y) = 255 represents the pixel of a valley (the section

between two ridges). Therefore, given an image I an algorithm known as medial axis

transform is implemented, where the skeleton of such image is constructed. A certain

pixel (xe,ye) is part of the skeleton (E(x,y) = 0) if the distance between it and at least two

other edge pixels (the ones adjacent to the valley) is the same. Figure 2.19 shows a pixel

“P” which would accomplish this condition with two pixels “M” in a blob image.

Figure 2.19 A pixel “P” is considered as part of a skeleton.

Therefore, if a proper skeletonization algorithm is applied to, for example, an image

containing the letter “B”, the expected result would be a thin line like the one shown

inside the letter on Figure 2.20.

Figure 2.20 Skeletonization (white line) of the letter B

Gonzalez and Woods [23] present an algorithm to perform this transformation. First,

we define the neighborhood of a certain pixel p=(x,y) as

31

Then, we define a set of rules to label pixels according to their neighbors [17]:

p is a “left edge” of a ridge if p4 is a valley (white) and p0 is the ridge

(black).

p is a “right edge” of a ridge if p0 is a valley (white) and p4 is the ridge

(black).

p is an “upper edge” of a ridge if p2 is a valley (white) and p6 is the ridge

(black).

p is a “lower edge” of a ridge if p6 is a valley (white) and p2 is the ridge

(black).

Afterwards, we define the logic value ni=TRUE for a given ridge pi. Four

combinatory expressions are defined to express potential pixels that will be converted

from a ridge to a valley (the edges).

An algorithm goes over the entire image I’ eliminating every edge pixel (converting

it to white) until no more ridges are found (when the conditions previously defined

aren’t met by any pixel). It is recommended that, when doing the binarization of an

image, all the ridges are set to a fixed thickness in such matter that the skeletonization

32

algorithm is run the same amount of times for every ridge. In the end, we expect to

obtain a result like the one shown on Figure 2.21.

Figure 2.21 From left to right. Original Image I, Binarized Image I’ and Skeletonized Image E.

F. Minutiae Extraction

The last and most important phase for the image processing is the minutiae

extraction. From the skeletonized image we will be able to extract the minutiae. Each

minutia is defined as a vector ( ), where and represent the position

of the minutia on the image. Moreover, the value is the directional angle of the ridge

at the minutia point and represents the type of minutia (termination or bifurcation)

[3]. Arcelli and Baja [24] presented a method called crossing numbers, in which given a

pixel of an skeletonized p=(x,y) and its value E[x,y]=E[p], the number crossing cn(p) is

defined as the half of the sum of the differences between a pair of adjacent pixels in the

eight neighbors of such pixel [17], which is defined with the equation

where p0, …, p7 are the neighbor pixels of p that belong to a certain sequence. We can

reuse the sequence previously defined for the skeletonization algorithm

Since we consider that a pixel can only take two values {0,255} for black and white

respectively, a certain pixel p=(x,y) that belongs to a ridge E[x,y]=0 is:

33

a) An isolated coordinate if cn(p) = ½ * 0 * 255 = 0

b) A terminal minutiae if cn(p) = ½ * 2 * 255 = 255

c) A regular ridge point if cn(p) = ½ * 4 * 255 = 510

d) A bifurcation minutiae if cn(p) = ½ * 6 * 255 = 765

e) A crossing of two ridges if cn(p) = ½ * 8 * 255 = 1020

On Figure 2.22 we expose a visual representation of three of these scenarios. The one

on the left represents c), the center image belongs to b) and the image on the right is a

case when d) occurs.

Figure 2.22 From left to right. Normal ridge point, terminal minutia and bifurcation minutia

Now we store the coordinate of p(x,y) as and in the minutia vector and the

corresponding type in . The only missing variable to define would be , which must

be described as ϵ [ 0 , 2π ) degrees. We can obtain from the SFDM matrix and its

orientation angle information. Since the SFDM image only has information of two

quadrants Ď’ ϵ (-π/2 , π/2), the conversion to a four quadrant angle must be made. In

case that a terminal minutia has been located (cn(p)=255) we can find two conversion

scenarios given an angle α from the SFDM matrix

a) If α ϵ [ 0 , π/2 ] and a situation like this is presented

Figure 2.23 Terminal Minutia scenarios

then .

34

a) b) If α ϵ [ 0, -π/2 ] and a situation like this is presented

Figure 2.24 Terminal Minutia scenarios

then

b) for the rest of the cases.

In case that a bifurcation minutia has been found (cn(p)=765) we could also find the

proper angle by using a case algorithm. However, since the number of cases is

superior this time, several proposals [17] suggest that, given the binarized image I’

invert the values. This way, ridges and valleys will be inverted and when performing the

skeletonization, the features that now resemble terminations will be the original

bifurcations, and the previous algorithm will satisfy the new conditions.

It’s also worthy to point out that impostor minutiae can appear by using the previous

algorithm, since noise is always present in a captured image. Xiao and Rafaat [25]

proposed a method to eliminate impostor minutiae by deducing several common

structures that are not minutiae. The ridges are then connected as shown in Figure 2.25

if the distance between two ridges is smaller than a certain threshold defined by

previous experimentation and according to the type of image used. This procedure shall

be applied before the minutiae extraction and right after the skeletonization algorithm.

35

Figure 2.25 Impostor minutiae transformed to regular ridges

2.2.2 Classical Minutiae-Based Matching System

Given two images of a template and an input image , a minutiae extraction process

is executed to extract a set of minutiae of the form

where m and n represent the number of minutiae on the sets of and respectively [17].

The position of the minutiae in the image is represented by and y, the orientation by ,

and the type of the minutiae is where ϵ {termination , bifurcation}.

We will consider two minutiae i and j paired if the following three conditions are

met:

36

where 0 represents a certain distance threshold and 0 represents an angular threshold.

Both of this thresholds are set since we must remember that the information on and

may have some variability. We must point out that the function takes only the

minimum between j i| and j i| in order to consider the circularity of an

angle. Else, a case where, for example, j=355o and i=10

o will deduct a o

instead of o.

To maximize the number of coincidences, two minutiae i and j must be aligned

whenever they are paired. A correct alignment requires the knowledge of a certain

displacement on and , and a rotation of to overlap one minutia with the other. A

transformation of the set of minutiae of called mapping (represented as the function

map( )) is executed, where a new set is formed so that

where

The new position and direction of the minutiae are calculated as

Finally, we will define a function mm( ) as a method that returns

so that, for every possible combination of i and j, a result is emitted. This procedure

is commonly known as “casting a vote” for a certain alignment displacement and

37

rotation. Therefore, in order to obtain the correct alignment for j, we just need to

seek for the maximum number of votes obtained with the mm( ) function for a certain

combination of , , and .

where ( ) can be any function which determines the labelling of two minutiae. A

certain minutia has either one or no pair, but cannot have more than one.

One of the most common reviewed systems to perform this process is the one

proposed by Ratha et al. in the year of 1996 [26]. This method is based on the

comparison of the two sets of minutiae according to four different factors using a Hough

transform-based alignment [3]. This algorithm is often considered suboptimal, which

means that maybe there is a better solution but no one has been able to find it [17]. The

Hough Transform approaches are derived from the Hough transform developed by

Ballard [27] in 1980.

The method proposed in [26] is based on the following. First, the possible space of

transformations is defined as triplets ( , , ). Then, each parameter is discretized

(denoted with the + symbol) into a finite set of values

A three dimensional array A with one entry of each discretization parameter is

initially reset and the following algorithm is applied [3].

38

At the end of the accumulative process, the best alignment ( *, *

, *) is obtained

as

To increase robustness, it is common to cast a vote not only for the exact discretized

combination but also its nearest neighbors [3].

As we can see, the algorithm presented known hereafter as “Hough Algorithm” is a

very efficient method to pair two minutiae, however, the following problems can be

found when applying the methodology in forensic partial to full applications. First, on

the practice is very difficult to agree in a proper discretization scheme ( +, +

, +)

since the more discretization made, the slower the algorithm is. Second, the algorithm is

not ideal for partial to full comparisons, since the alignment is made with one piece of

image versus the whole counterpart, and a lot of time is wasted trying to pair and label

minutiae that were never related.

2.2.3 Current Minutia-Based Matching Systems

Several methods have been presented for palmprint recognition, most notably in [3],

[4] and [12]. Along many other studies, an initial approach modeled by Zhang & Kong

[5] proposes a low resolution image matching based on the palmprint ridges. Funada et

al. [28] created a novel algorithm for minutiae matching using crease elimination.

Nevertheless, these models cannot be applied while performing partial to full palmprint

matching. Derivate from such approach, Jain & Feng [29] propose a latent palmprint

matching technique consisting on partial to full palmprints for forensic applications,

39

improving the feature extraction algorithm proposed by [28]. Although the

appropriateness of this approach is proven, we encounter the problem of large

computational demand on Discrete Fourier Transform and Gabor Filtering, but mostly

an apparent requirement of about 150 minutiae per partial palmprint. Additionally, to

achieve an acceptable classification rate, a fusion of multiple partial palmprints is

needed from the same palmprint for the system to work. This is an important demand,

since in some scenarios, only one sample is available.

More recent approaches propose different matching criteria than merely minutiae

analysis. A document presented by Dai & Zhou [30] proposes a multi-feature fusion

algorithm that, compared to the latent matching elaborated in [29], presents an improved

matching percentage of 91.7%, however, the method is not presented for partial to full

matching. On 2012, Dai et al. [31] propose a robust ridge-based matching algorithm

which outperforms the techniques presented in [29] and [30].

The proposal is composed by three modules: palmprint registration, segment-based

matching and fusion, and cascade filtering. In the registration an orientation field is

used, since palmprints share a lot of common ridge flow patterns. This transforms

different rotations and displacements into the common coordinate system. Then,

registered palmprints are split into smaller segments, this way when distortion is

detected the segments will shift and rotate in order to correct the distortion. The

similarity score is calculated at each segment for each feature. Different weights are

defined for various features at different segments. These weights are determined by

learning using a naïve Bayesian. Finally, a cascade filtering is applied to discard

segments which are not distinctive in palmprints. The matching between the three

characteristics described and the proposed modules are pictured on Figure 2.26.

40

Figure 2.26 Relation between palmprint characteristics and matching algorithms

However, it is again unclear if the method works for partial palmprints and, since it’s

based in more features than the minutiae, requires a higher definition and quality than

images found on, for example, a crime scene.

A method based on wavelets has just been presented this year [32]. Although the

results are interesting, this method does not work on partial palmprints due to the need

of the representation of the whole ridges. Nibouche et. al. presented in [33] a method to

obtain a palmprint distance, but it needs several samples of the same full palmprint

since it is based on PCAs. In a proposal made by Badrinath et. al. [34], PCAs are also

used, but each palmprint is divided into several square-overlapping blocks. This is done

to classify these blocks into either a good block or a non-palmprint block. Finally, in the

paper presented by Wang et. al. [35], palmprint images are decomposed by 2D Gabor

wavelets. The drawback of this methodology is that this decomposition is very sensitive

to the length of the obtained palmprint and again, cannot be used to satisfy partial to full

matching requirements.

2.3 Justification for a new algorithm

While most of the palmprint matching approaches are based on a full-to-full

association, our contribution is based on a tiny partial section of the palmprint being

associated with its complete counterpart, and the importance of only one sample

available, which is a more plausible scenario in forensics studies. Only the method

presented in [29] considered this fact, although they need a larger partial palmprint than

the one we expect to use. Our algorithm is designed and developed not only to perform

41

a verification of two samples, but also to declare to which section of the full sample

does the partial sample corresponds.

We also want to demonstrate that it is possible to develop a method which is more

optimal than the “Hough Method” presented in [26], obtaining similar results in less

computational time. Finally, we need to demonstrate that our method is not only

resistant to noise aggregation on the input image I, but also rotation invariant, which

means that no matter the position of the partial sample found on a scene, our method

must be capable of finding a correct correspondence amongst several templates T.

42

3 METHODS

3.1 Problem with the classical minutiae matching approach

Let a Palmprint Identification Register (PIR) be represented as a set { },

where represents a minutia. We must point out again that each minutia is defined as

a vector ( ), where and represent the position of the minutia on the

image. Moreover, the value is the directional angle of the ridge at the minutia point

and represents the type of minutia (termination or bifurcation) [3].

Suppose that we want to find the correspondences between a pair of PIRs P and F.

Also, suppose that is an injective non-subjective function that maps minutiae of P to

minutiae of F, . Some minutiae in P may not be mapped through this

function. Thus, the aim of the palmprint matching is to find the best correspondence

such that the sum of distances between the minutiae in the sub-set is minimized.

{ }

{ ∑ ( )

( )

}

The matching distance is defined as

( ) ∑ ( )

( )

Parameters and represent a global displacement and rotation of one of the

palmprints assuming an affine homography that would not have to influence on the final

distance value. Suppose minutia becomes minutia after applying the

displacement and rotation . Then, the distance function is defined as follows,

( ) ( ) ( ) ((

) ( ))

43

where is the angular distance and is the displacement distance. Parameter

] weights both distances and it is application dependent. The restriction R in

imposed to the summation of the first formula leads to both minutiae have to belong to

the same type and the distances have to be lower than a given threshold.

( ) ( ) ((

) ( ))

Due to this restriction, some minutiae in both sets remain non-mapped and for this

reason, the number of mapped minutiae may be lower than the cardinality of both sets.

Without more formality, we define as the number of mapped minutiae. If the

thresholds and are small, then the method tends to consider most of the minutiae

as outliers and so the number of matched minutiae is small. Thus, the matching leads

to a small number of minutiae dictating the final distance ( ) . Contrarily, if these

thresholds are high, then increases. This fact makes the distance ( ) increase as

well, since more minutiae distance terms are added.

While we previously stated that there are several algorithms that seek for a

suboptimal solution in polynomial time [26], these algorithms usually fall in a local

minimum that is considered far away from the best solution when the difference

between the cardinality of both sets of minutiae is high, or when the input images are

very noisy. For this reason, our proposal is based on using an algorithm that has in its

core the fact that one of the sets of minutiae P is a small subset of the other one B. That

is, instead of matching two sets of minutiae treated as equals, we look for the best subset

of B where P has the best fit.

3.2 Partial to Full Palmprint Matching

Assume P is a partial palmprint and B is a full palmprint, and we seek for ( )

and the number of paired minutiae N. In this section, we will explain the three main

steps of our algorithm. In the first one, we encounter P and its set of minutiae must be

extracted correctly. Then, some positions in B are selected as candidates to be the center

of P. Finally, B is split in several PIRs, in which the center of each is a candidate

44

position. In this third step, a matching between P and each of the PIRs is performed, and

the best correspondence is stated as the output. We assume there is a database of full

and noiseless B1, B2… Bn.

3.2.1 Partial Palmprint Representation

A partial palmprint is used as an input image I instead of a complete one for two main

reasons:

1. The sample is gotten from a crime scene, where the perpetrator or suspect only

pressed his hand against an object partially.

2. The sensor that obtains I is smaller than the one which obtained the template T,

therefore we only have part of the palmprint. This scenario is plausible in mobile

applications, where a sensor for a human’s whole hand cannot be implemented.

For our method to work, we propose the following agreement:

1. Start by locating a partial palmprint in either one of the previously described

scenarios.

2. After performing an image acquisition, preprocessing and feature extraction to the

partial palmprint, we obtain a small set of minutiae.

3. These minutiae need to somehow have a reference point, since these minutiae’s

positions , y are now totally different from the original template, and θ depends

on the rotation that the partial image was obtained.

4. Therefore, the partial palmprint is enclosed within a circle of radius r and a center

located exactly within one of the minutiae. We then adjust this circle tangent to

the coordinate plane, leading to the chosen central minutia be positioned in the

coordinate (r/2 ,r/2). A recursive method is called to calculate

a. The distance π between every minutia inside the circular area and the

central minutia.

b. The angle φ between every minutia and the central one.

5. We finally get a Floating Palmprint Identification Register (FPIR) which

represent a set { }, where represents a minutia. Each minutia is

defined as a vector ( ), where is the directional angle of the

ridge at the minutia point and represents the type of minutia (termination or

45

bifurcation). Additionally, the position and may also be added (keeping in

mind that this position is taken with respect of the tangent coordinate access and

not with respect of the full palm), and two new variables and are introduced.

The variable represents the distance from the minutiae to the center of the

circle and represents the angle formed by both of them.

On Figure 3.1 we depict this procedure graphically in an effort to describe the

algorithm better.

Figure 3.1 Process followed to declare a partial palmprint

3.2.2 Selecting position candidates

Let a FPIR be represented as a set { }, where represents a floating minutia.

Each floating minutia is represented as a vector ( ), where

represent the Euclidean distance from the respecting minutia to the center of the set. The

angle is the angle from the minutia to the mass center of the set. Values and are

the same as on the PIR. Note that we call the input a Floating PIR since through this

representation; the absolute positions of the minutiae have been converted into

relative positions that depend on the values of the set.

We now perform an algorithm to select the positions in the full palmprint that are

candidates to be the center of the partial palmprint, that is, the center of the partial in the

full. To do so, the method considers the spatial relation of the minutiae in the partial

palmprint and seeks for these relations in the full palmprint. The relation between

minutiae is considered through the radius and angle from each minutia in the

partial to the center of the minutiae set. For this reason, it is crucial to represent the

partial palmprint as a FPIR and the full palmprint as a PIR.

46

In this step, the angle of the minutiae is not taken into consideration, and the type

of minutia is used to discard combinations between minutiae of both sets of minutiae

(minutiae that are not of the same type). To achieve this aim we used the Generalized

Hough Transform (GHT) [3] [27][36]. The method is composed of two subparts. In the

first one, a matrix G is filled with the same height and weight than the full palmprint

image. Each cell ] represents the position in the full palmprint where certain

consider its center to be, when mapped with a certain minutia of the full palmprint.

Thus, G[i,j] is computed as

] ( ( ) ( ) )

One of the key aspects to point out is that the algorithm is rotation invariant so far

given the fact that the angle used for the generation of G is and not . When matrix G

is filled, then the algorithm generates an ordered list {(

) (

)} of these

positions through a clustering and voting process. Only the cases, in which both

minutiae are of the same type, are taken into consideration. The whole

candidates in C must have achieved a voting value greater than a voting threshold Tv.

With this list, the method generates a set of PIRs { }. Each component of

represents a candidate (

), and its minutiae is a subset of minutiae in the full

palmprint defined with the following criteria

{ ((

) (( )) { } }

Value makes the radius larger, and so some additional minutiae from the full

palmprint may be included into the PIR , making the obtained correspondences more

adaptable. Notice that while considering these sets of minutiae, what we have done is to

consider different displacements since the information of these displacements is

implicit in the value of the centers (

). Figure 3.2 resumes the second step of the

algorithm.

47

Figure 3.2 Generation of the set of PIRs { }.

3.2.3 Best candidate selection through multiple correspondences

This last step of the method consists on finding the distances ( );

and the suboptimal correspondences between the partial palmprint P and each in

. Several algorithms can be used to find these correspondences. For instance we can

use the Hungarian method [37] (where no outliers are considered), the bipartite graph

matching without the second order information [38] or a greedy algorithm that simply

selects the best option without considering the other candidates. Any of these matching

algorithms has to be designed to consider the restrictions.

( ) ( ) (( ) ( ))

We must keep in mind that, since the requirements are done for the position and the

direction, some adjustments on P and each have to be made.

First, to make this methodology rotation invariant and be able to use any directional

angle and , all the calculations must be done with the relative angles and not with

and themselves. To obtain the relative angles and of P and a given , we

must perform the following operations

48

∑ ( )

{ }

∑ ( )

{ }

( (

)

(

))

∑ ( )

{ }

( )

∑ ( )

{ }

( )

( (

)

(

))

( )

Afterwards, for every and of both P and a given respectively, we can obtain

and by performing

)

)

Once the rotation invariance is set, we need to transfer the position of the

minutiae of P to resemble the coordinates described by a certain . This is solved by

49

aligning P to and calculating new positions for each of P by doing the

following:

where ( ) is the center of the FPIR P calculated at the beginning, and ( ,

) is

the candidate center of the PIR to be matched with. Notice that this spatial

transformations must be done separately on P for each to be matched, Therefore, it is

convenient to create n separate FPIRs P for each position adjustment, where n is the

number of PIRs on . Therefore, we will match a set of PIRs { } with a

set of FPIRs { }

Once a matching algorithm and the angle adjustment are executed, the winner

candidate is the one that obtains the minimum distance or the one that obtains the

maximum amount of pairs N. Then the method returns either the distance ( )

{ } or simply the maximum N. It also returns the correspondences used

to compute this distance since the method assumes , where {

}. Breaking down the full palmprint in a set of candidates has two advantages. On

one hand, the computational cost of obtaining the distances is lower than obtaining

the value ( ). On the other hand, the algorithm tends to obtain a more precise local

minimum.

Figure 3.3 Computation of the final distances and correspondences

50

If this procedure is executed only once, the information obtained is the possible center

of the FPIR over the PIR image, therefore no subject identification done. Contrarily,

when executing this process using an input FPIR and a series of different template PIRs

belonging to different subjects, we will obtain both an identification of the most likely

subject to which the FPIR belongs and the most likely center where the FPIR is aligned

over the PIR.

51

4 PRACTICAL EVALUATION

Two different metrics are used while comparing palmprints based on minutiae. The

first one is the number of matches found between both sets of minutiae. This is the

simplest, metric but it depends on the number of minutiae in the sets (more minutiae,

more matches). The second option is the distance normalized by the number of

matches represented as ( ) . It is not usual to consider directly the distance

as a metric that gauges the quality of the match, since this distance depends on the

number of minutiae matched, and a lower distance does not necessarily mean that a

correct match has been found. If few minutiae are matched, the distance value tends to

be smaller. Thus if the distance has to be consider, it must be normalized by the number

of minutia mappings.

4.1 Palmprints used

We have used images contained in the Tsinghua 500 PPI Palmprint Database [29]. It

is a public high-resolution palmprint database composed of 500 palmprint images of a

resolution of 2040 x 2040 pixels, which were captured with a commercial palmprint

scanner from Hisign.

Figure 4.1 A sample of a palmprint contained on the Tsinghua database

52

We used the algorithm presented in [29], [30] and [31] to extract the minutiae from

each image. Since the enrolment algorithm clearly states that it only works with full

2040x2040 palmprint images, a feature extraction of a partial image is not possible. For

this reason, we generate the partial palmprints as explained on Figure 4.2.

Given a full palmprint, we first extracted the minutiae. Then, we select only the

minutiae that have a distance from their position to the center ( ) smaller than a

fixed radius . Finally, we can add a rotation to every minutia based on a factor β and

some distortion on the position and the minutiae angle is added based on a Gaussian

noise with standard deviations and and mean equal to zero. The center of

the circular partial is taken randomly for each test.

Figure 4.2 Generation of a partial palmprint.

4.1.1 Selection of the Radius

One of the key aspects of the system’s design is to perform multiple tests by first

defining the size of the radius of the partial palmprint found. Thus, we want our

system to be efficient for the smallest possible .

Our system’s efficiency is directly correlated to two factors: number of minutiae

found on the partial PIR and noise aggregation presence. Therefore, by increasing the

number of minutiae found on the partial FPIR, the probability of performing a correct

match increases (as long as noise aggregation is not high enough to distort the original

data). We decided to test the ability of the partial palmprint to be correctly classified

with a full template by cropping circular section S of the original palmprints within a

radius (150 to 300 pixels). We chose this values based on our

goals and expectations of the system to relate partial palmprints as small as possible (1,5

53

cm radius being acceptably small) with full templates. On Figure 4.3 we show an

approximate of how much a circular area with a radius of 0,75 cm (small circle) and 1,5

cm (big circle) represent in a human palm.

Figure 4.3 Circles with radius r = 150 px = 0,75cm (small, blue) and radius r = 300 px = 1,5cm (big, red).

4.1.2 Noise Aggregation and Rotation

As mentioned before, our system’s experimentation will also consider the presence of

noise in the partial image found. Noise on a sample may be present because of the

variability factors described on section 2. To simulate the effect that such factors would

cause to the data on the partial PIR, a Gaussian noise with a standard deviations and

54

and zero mean is added to the partial PIR data before attempting the matching,

where represents the angular noise on and represents the position noise on

. Both are always affected with the same noise simultaneously. Right before

the application of this noise, the angle of every minutia of the partial print may be

rotated by a factor β ϵ [0,360o) to consider the fact that a partial print is never found on a

“straight” position on forensic or mobile applications.

The noise aggregation follows the conditions

0≤ ≤18, {2 , ϵ N}

0≤ ≤18, {2 , ϵ N}

We selected a maximum noise of standard deviation of 18 since once that a

palmprint has been altered with this noise, it is almost impossible to relate the altered

palmprint with the original one. We can see an example of the original values and the

altered values with a configuration = 18 | = 18 on Figure 4.4.

Figure 4.4 Comparison between minutiae with no noise and minutiae with noise aggregation.

The reason to increase the noise in a step of 2 (from 0, 2, 4… , 16, 18) was just to

perform 10 tests exactly.

4.2 Experiment Conditions

. We count with a total of 100 palmprints (50 left hands and 50 right hands) and a

total of eight samples per palm, giving as a result of 800 templates. Since a left hand has

55

always different features than the right one [7], we could consider that we have samples

of 100 different individuals. On Figure 4.5 we present four images: (a) corresponds to a

sample of a certain person right hand, and (b) corresponds to another sample of the

same person’s right hand. Notice the difference between (a) and (b) and the variability

between them. On image (c) we present the same person’s left hand and (d) corresponds

to a second sample for such left hand. The same phenomenon described before occurs.

We can see that (a) and (b) area very different to (c) and (d), which confirms the

difference between the features of a human’s left and right hand.

(a)

(b)

(c)

(d)

Figure 4.5 Samples of the right palm (a) and (b) and of the left hand (c) and (d) of the same person.

56

Our first proposal is to perform a fast test in which we only compare one small print

with one template, and obtain the similarity between both samples (according to either

or ( ) ) and the most likely center of the small partial over the full template.

To do this, we first choose one subject randomly and select one of the eight samples Ti

of his or her hand randomly as well. We then select a small partial print Pi from it (with

a random center (xc , yc), a fixed radius , a certain fixed noise

and an a fixed rotation β). Afterwards we compare Pi with Ti and see if the system

has been able to correctly locate Pi within Ti.

Figure 4.6 Proposed algorithm to relate a single partial palmprint with a single full template

Likewise, we also propose a global identification test, when trying to find the

correspondence of a partial palmprint found amongst a set of n samples. We first choose

one subject randomly and select one of the eight samples of his or her hand randomly as

well. We then select a small partial print Pi from it (with a random center (xc , yc), a

fixed radius , a certain fixed noise and an a fixed rotation β).

Then we proceed to execute the algorithm 100 times, comparing the small print with the

57

best quality template of each individual {T1, T2, … T100}. The output is composed by a

verdict of the best relation of Pi with a certain Ti (according to or ( ) ) and the

most likely center where Pi is located within Ti.

Figure 4.7 Proposed algorithm to relate a single partial palmprint with several full templates

The palmprints used have an average of 800 minutiae and the partial palmprint have

an average of 40 to 80 minutiae (according to the radius r chosen). The threshold on the

pixel proximity was set to 5 pixels, and the threshold on the angle difference

varies in such way that we can discern which value is the most appropriate one for each

method. We compare our method using first the Greedy Matching and then the

58

Hungarian Method Matching with the results given by the “Hough Algorithm” proposed

in [26] implemented by ourselves. For the Hungarian Method, instead of varying we

use a value named cost represented by C, where ]. The cost is a reference

distance value which has to be higher than the distance ( ) of two minutiae in

order for those minutiae of P and F to be paired. The three algorithms are implemented

using the Matlab software.

When implementing the Hough Algorithm we set the values of discretization ( +,

+,

+) to +

ϵ {0,2,4,…,2038,2040}, + ϵ {0,2,4,…,2038,2040} and +

ϵ

{0o,10

o,20

o,…,340

o,350

o}. This means that, the Hough Algorithm will do a review of

every possible position with a leap of 2 pixels, and 36 possible rotations of the minutiae

from [0o,360

o) to perform a match for a minutia of P on T. We realize this will make the

“Hough Algorithm” very exhaustive and computational demanding, but it is the only

way in which we can compare the “Hough Algorithm” to our rotation and position

invariant methodology.

4.3 Tests for the location of the correct center (single partial print with single full

template)

This test was only performed for our method and not with the “Hough Algorithm”,

since one of the key aspects of our methodology is that we offer as well the option of

finding the position of P within the large sample T. We only performed tests with the

Greedy Matching Method and left the comparison with the Hungarian Method for later.

Even though our main interest is to develop a system that can relate a partial with a set

of full palms, it is interesting to demonstrate that our proposal is capable of finding as

well the location of a small palmprint P over the coordinate plane of a full template T

when applying noise to the small sample and when the correspondence PT is well

known. For this test we obtain a random partial P from the same template T with which

it will be compared, since the only interest is to see whether the algorithm can perform a

center location or not. We created 10 different samples with a radius r=300 {P1, P2, …,

P10} with a fixed value of noise | and a random rotation β for each Ti in {T1, T2,

…, T10}. Afterwards, we attempted to match each Pi with its correspondent Tj, keeping in

59

memory the original center (xa , ya) where the partial Pi was taken from Tj. Every time

the matching algorithm was done, as described on Figure 4.4, we obtain the tentative

center (xf , yf) where the partial Pi could come from. If (xa , ya) is located within a

maximum distance of 5 pixels of (xf , yf), we consider the test a success (1), otherwise a

failure (0). In the end, we average the 100 results (10 out of each palm) for noises |

from 0 to 18 with a step of 2, and random rotations β where 0≤β≤360, ϵ Z.

Figure 4.8 Results for the simple test using the Greedy Matching Algorithm

We can notice that results are not that constant according to the increase of the noises

| . For example, a configuration = 18 | = 8 has a success rate of 91%,

while the configuration = 0 | = 10 (which evidently represents less noise) has a

success rate of 94%. This is caused because experimentation is entirely random, and it

may occur that on some experiments the chosen center (xa , ya)which generates Pi

includes fewer minutiae than other experiments. However, we can notice that on the

chart on Figure 4.7 diagonal values (when = ), a representative decrease in the

results can be charted. We can clearly notice that even when the noise configuration is

= 18 | = 18

Figure 4.9 Decrease in success for the single test as noise increases

0

0,2

0,4

0,6

0,8

1

1,2

0 2 4 6 8 10 12 14 16 18

% o

f Su

cces

s

increment of noise rxy and ro

60

4.4 Tests for identification of a sample amongst a data set (multiple single prints

with multiple full templates)

After analyzing the functionality of the algorithm, we are ready to present the

comparison between the “Hough Algorithm” and our method, in which simultaneously

we tested both the “Greedy Matching” and the “Hungarian Matching”. Since we could

notice on the last section that the results when = are the most representative and

constant of the whole set of combinations, for this test we increased and alike

from a standard deviation of 0 to 18, once again applying every single time a random

rotation β that satisfies 0≤β≤360, ϵ Z.

We performed 100 tests per noise configuration using one partial out of each subject

and doing the comparison versus the whole database (the best sample of each hand). We

therefore obtain a 100x100 matrix of results where we can see either the number of

pairings or the normalized distance ( ) .

On Figure 4.9 we expose a smaller example using the first 10 subjects to explain the

methodology faster. We can see that, when using the criteria , we start by comparing

P1 with {T1, T2, …, T9, T10}. Then, the results of for each test are accommodated in

order on the result matrix. After comparing the rest of the Pi, and once the 100

comparisons are made (10 partials with 10 templates), since the results are ordered, we

know that the diagonal results must be the correct matches. We then perform a routine

where we analyze each row of this resulting matrix. In the case of a criteria , if the

result on the position [n,n] is the biggest for the nth

column, then we obtained a success.

Otherwise, the system wasn’t able to find the correct match. This procedure is

repeated until no more rows are left to analyze.

61

Figure 4.9 Results of a test comparing ten partials with ten templates

Using the criteria , we obtained the following result. We decided to run four main

tests. Each case represents the percentage of success over the 100 tests done with each

hand. We don’t include the results of the “Hungarian Algorithm” since this last

methodology uses a cost C instead of Ta, and there is no possible way to do equivalence

between C and Ta, meaning that it isn’t meaningful to compare one metric with the

other.

1. Radius = 300, Ta = 180º (for “Hough Algorithm” and “Greedy Matching”)

Table 3

0,82

0,84

0,86

0,88

0,9

0,92

0,94

0,96

0,98

1

1,02

0 2 4 6 8 10 12 14 16 18

% o

f Su

cce

ss

Noise Increment

Hough

Greedy

62

2. Radius = 150, Ta = 180º (for “Hough Algorithm” and “Greedy Matching”)

Table 4

3. Radius = 300, Ta = 90º (for “Hough Algorithm” and “Greedy Matching”)

Table 5

0

0,2

0,4

0,6

0,8

1

1,2

0 2 4 6 8 10 12 14 16 18

% o

f Su

cce

ss

Noise Increment

Hough

Greedy

0,8

0,85

0,9

0,95

1

1,05

0 2 4 6 8 10 12 14 16 18

% o

f Su

cce

ss

Noise Increment

Hough

Greedy

63

4. Radius = 150, Ta = 90º (for “Hough Algorithm” and “Greedy Matching”)

Table 6

As we can see, our method using the “Greedy Matching” outperforms the “Hough

Algorithm” in every case of the four shown. There is no doubt that our methodology is

superior than the one proposed seriously, not only in the quality of results for increasing

noise, but it is also superior in smaller radius (the improvement in r=150 is better than in

r=300) with the added value of the center localization, which is not shown in any of the

result charts for practical reasons. Additionally, tests done with our method ran two

times faster in a computer with identical characteristics that the one that ran the “Hough

Algorithm”.

We present two additional charts to compare each method according to the size of

the radius of the partial palmprint, this we can decide not only which the best method is,

but also which the best threshold configuration Ta for such radius is.

0

0,2

0,4

0,6

0,8

1

1,2

0 2 4 6 8 10 12 14 16 18

% o

f Su

cce

ss

Noise Increment

Hough

Greedy

64

r = 300

r = 150

Figure 4.10 Efficiency of the methods according to the radius of the small partial palmprint used

When the circular area of the partial palmprint has a radius of 300 pixels, it is

convenient to use a Ta of 90o, which proved to be more stable and when dealing with big

noise quantities. As for the r = 150 circular areas, both thresholds Ta work almost

equally well on our method.

65

5 CONCLUSIONS AND FURTHER WORK

Several palmprint matching algorithms have been presented through the course of

biometrics study. Nevertheless, most of these methods need both images to show the

full palmprint. These methods are useful in applications such as verification to give

access into restricted areas. For forensics applications, it is usual to have very tiny

partial palmprints (around 2 cm weight and height images). In these cases, most of the

algorithms fail. This happens because the core of these algorithms is to find a full-to-full

matching instead of a partial-to-full matching. We have presented an algorithm to

perform a partial-to-full palmprint matching. Two main steps compose it. In the first

one, several position candidates are selected and in the second one, only one position is

obtained through a multiple assignment method. Results show that with circular partial

palmprints starting from 0,75 cm radius, the recognition ratio keeps high compared to

one of the most used minutiae matching algorithms.

There is much work left to do in this project and plenty to demonstrate. First and most

important, it is essential to compare our system with current methodologies developed

by other scholars. It is hard to do this but not impossible though, since getting

algorithms from other authors and making them run under the same conditions that our

is an interesting pending work.

As a further work, we want to include second order information and represent the

partial palmprint as an attributed graph. Thus, the structure of the partial palmprint

would help the method to find the minimum distance and the best correspondences.

Even though the computational cost could increase, this fact will need to be considered.

Moreover, we want to include an active an interactive method [39], [40]. This means

that a human specialist could correct the current correspondences and help the method

to find the minimum distance.

Finally, we also have considered pertinent to use other palmprint databases and even

developing our own minutiae extractor algorithm so that the methodology can be

applied directly on small partial palmprints, instead of having to do a cut of the full

template.

66

6 REFERENCES

1. Anonymous. “Early Fingerprint Pioneers. Ridges and Furrows”.

http://www.ridgesandfurrows.homestead.com/early_pioneers.html.

2. History.com. “This Day in History: A bloody fingerprint elicits a mother’s evil tale in Argentina”. http://www.history.com/this-day-in-history/a-bloody-fingerprint-elicits-a-

mothers-evil-tale-in-argentina.

3. Jain, A. K., Flynn, P., Arun A. 2009 “Handbook of Biometrics”. Springer. 4. Jain, A. K., Ross, A., Prabhakar, S. 2004. “An Introduction to Biometric Recognition”.

IEEE Trans. Circuits Syst. Video Technol. 14, 1, 420.

5. Kong, W. K., Zhang, D. 2002. “Using Low-Resolution Palmprint Images and Texture Analysis for Personal Identification”. ICPR.

6. Zhang, D. 2008. “Three Dimensional Palmprint Recognition using Structured Light

Imaging. Biometrics: Theory, Applications and Systems”. BTAS. 2nd IEEE International

Conference on Biometrics. 7. Zhang, D., 2004. “Palmprint Authentication”. Kluwer Academic Publishers.

8. Somvanshi, P., Rane, M. 2012. “Survey of Palmprint Recognition”. International Journal of

Scientific & Engineering Research. 9. Kong, W.K., Zhang, D. Li, W. 2003. “Palmprint feature extraction using 2-D Gabor filters”.

Pattern Recognition, vol. 36, pp. 2339-2347.

10. Zhong, Q., Zheng-yong, W. 2010. “Research on Preprocessing of Palmprint Image Based

on Adaptive Threshold and Euclidian Distance”. Sixth International Conference on Natural Computation (ICNC 2010). IEEE.

11. Qichuan, T., Ziliang, L., Yanchun, Z. 2010. “A Novel Palm-print Segmentation and

Recognition. Algorithm”, International Conference on Intelligent Computation Technology and Automation.

12. Zhang, D., Zuo, W., Yue, F. 2012. “A Comparative Study of Palmprint Recognition

Algorithms”. ACM Computing Surveys, Vol 44, No 1, Article 2. 13. Wu, X., Zhang, D., Wang, K., 2003. “Fisherpalms based palmprint recognition”. Pattern

Recognition Letters 24, 2829-2838. Elsevier.

14. Wu, X., Zhang, D., and Wang, K. 2006. “Palm line extraction and matching for personal

authentication”. IEEE Trans. Syst. Man Cybern. Part A 36, 5, 978–987. 15. Zhang, D., Kong, W. K., You, J., and Wong, M. 2003. “Online palmprint identification”.

IEEE Trans. Pattern Anal. Mach. Intell. 25, 9, 1041–1050.

16. Kong, A., Zhang, D., and Kamel, M. 2006. “Palmprint identification using feature-level fusion”. Pattern Recognition 39, 3, 478–487.

17. Serratosa, F. Solé, A. 2008. “Biometría”. Material Docente de la UOC. Spain.

18. P.S. Wu and M. Li. Pyramid edge detection based on stack filter. Pattern Recognition Letters, vol. 18, no. 4, pp. 239-248, 1997.

19. Duc, B., Fischer S., Bigun, J. 1999. “Face authentication with Gabor information on

deformable graphs”. IEEE Transactions on Image Processing, vol. 8, no. 4, pp. 504- 516.

20. Sobel, I., Feldman, G. 1968. "A 3x3 Isotropic Gradient Operator for Image Processing". Unpublished.

21. Ratha N.K., Chen S., and Jain A.K. 1995. “Adaptive Flow Orientation Based Feature

Extraction in Fingerprint Image,” Palmprint Recognition Vol 28, No 11 pp 1657-1672. 22. Mukherjee, A., Kanrar, S. 2010. “Enhancement of Image Resolution by Binarization”.

International Journal of Computer Applications, Volume 10 – No. 10, November.

23. Gonzalez, R., Woods, R. 2001. “Digital Image Processing: Second Edition.” Pretience Hall.

United States. 24. Arcelli, C., Baja, G. 1984. “An Approach to Figure Decomposition using width

Information”. Computer Vision, Graphics, and Image Preprocessing, Volume 1, Issue 26,

pp 61-72,

67

25. Xiao, Q., Raafat, H. 1991. “Finger Image Postprocessing: A combined statistical and

structural approach”. Pattern Recognition Vol 24 Issue 10, pp 985-992.

26. Ratha N. K., Karu K., Chen S., Jain A.K. 1996. “A Real-Time Matching System for Large Fingerprint Databases,” IEEE Transactions on Pattern Analysis and Machine Intelligence,

18 (8), pp. 799−813.

27. Ballard, D.H. 1980. “Generalizing the Hough Transform to Detect Arbitrary Shapes”, Ridge Based Palmprint Matching”, IEEE Transactions on Pattern Analysis and Matching

Intelligence.

28. Funada, J., Ohta, N., Mizoguchi, M., Temma, T., Nakanishi, K., Murai, A., Sugiuchi T.,

Wakabayashi, T., Yamada, Y. 1998. “Feature Extraction Method for Palmprint Considering Elimination of Creases,” Proc. 14th Int. Conf. Pattern Recognition, pp. 1849-1854,

29. Jain, A. K., Feng, J. 2009. “Latent Palmprint Matching”. IEEE Transactions on Pattern

Analysis and Matching Intelligence. 30. Dai, J. and Zhou, J. 2011. “Multifeature-Based High-Resolution Palmprint Recognition”.

IEEE Trans. Pattern Analysis and Machine Intelligence, 33 (5), pp: 945-957.

31. Dai, J., Feng, J., Zhou, J. 2012. “Robust and Efficient Ridge Based Palmprint Matching”, IEEE Transactions on Pattern Analysis and Matching Intelligence. 34 (8).

32. Wang, X., Liang, J., Wang, M. 2013. “On-line fast palmprint identification based on

adaptive lifting wavelet scheme”, Knowledge-Based Systems, 42, pp: 68-73.

33. Nibouche, O., Jiang, J., Trundle, P. 2012. “Analysis of performance of palmprint matching with enforced sparsity”, Digital Signal Processing, 22 (2), pp: 348-355.

34. Badrinath, G.S., Gupta, P. 2012. “Palmprint based recognition system using phase-

difference information” Future Generation Computer Systems, 28 (1), pp: 287-305. 35. Wang, X., Lei, L., Wang, M. 2012. “Palmprint verification based on 2D – Gabor wavelet

and pulse-coupled neural network” Knowledge-Based Systems, 27, pp: 451-455.

36. Kassim, A.A., Tan, T., Tan, K.H. 1999. “A comparative study of efficient generalised

Hough transform techniques”. Image and Vision Computing 17, pp: 737–748. 37. Kuhn, H.W. 1955. “The Hungarian method for the assignment problem” Export. Naval

Research Logistics Quarterly 2(1-2), 83–97.

38. Riesen, K., Bunke, H. 2009. “Approximate graph edit distance computation by means of bipartite graph matching”, Image Vision Computing. 27(7), pp: 950-959.

39. Serratosa, F., Cortés, X., Solé, A.. 2012. “Interactive Graph Matching by means of

Imposing the Pairwise Costs”, International Conference on Pattern Recognition, ICPR2012, Tsukuba, pp: 1298 - 1301.

40. Cortés, X., Serratosa, F., Solé, A. 2012. "Active Graph Matching based on Pairwise

Probabilities between nodes", Syntactic and Structural Pattern Recognition, SSPR2012,

Hiroshima, Japan, LNCS 7626, pp: 98-106.