Signature Verification

History• Sumerians used intricate seals applied to clay cuneiform tablets to authenticate

their writings.• Documents were authenticated in the Roman Empire (AD 439) by affixing

handwritten signatures to the documents.• In 1677 England passed a an act to prevent frauds and perjuries by requiring

documents to be signed by the participating parties.• In 1977, the first studies of both off-line and on-line signature verification

algorithms were published– Nagel and Rosenfeld “off-line system” IEEE T. Comp.– Liu and Herbst “on-line system” IBM J. Res. Dev.

• Much research has followed, attempting various methods for both feature extraction and matching

– Yang et. al. “Application of Hidden Markov Models for Signature Verification” (HMM)– Lam et. al. “Signature Recognition through Spectral Analysis” (FFT)– Hangai et. al. “Writer Verification using Altitude and Direction of Pen Movement”

(DTW)– Lejman et. al. “On-line Handwritten Signature Verification using Wavelets and Back-

propagation Neural Networks” (Neural Network)– Crane et. al. “Automatic Signature Verification using a Three-axis Force-sensitive Pen”

(Parametric)

Current State of the Art• No common agreement on benchmark databases and protocols in

the research community.• 1st International Signature Verification Competition in 2004 (100

users; 20 genuine and 20 forgeries):– Web: http://www.cs.ust.hk/svc2004/– On-line signature: 2 Tasks.– No companies participated (or participated but remained anonymous).– Difficult task (pen tablet without visual feedback, synthetic signatures,

forgers were given the dynamics of the signatures to imitate, English and Chinese signatures, etc.).

– Best system (3% EER Skilled Forgeries, 1.5% EER Random Impostors)

• Human performance:– Expert (0.5%FAR @ 7%FRR), Layperson (6.5%FAR @ 26%FRR)

Signature Verification vs. Handwriting Recognition

Paola Garcia

HandwritingRecognition

Signature Recognition(Verification)

• On-Line:

• Off-Line

Altitude (0°-90°)

90°

270°

0°

Azimuth (0°-359°)

180°

XY

PA

z

0 100 200 300

Al

SCANNER

ApplicationsApplications• On-line:

– SOFTPRO (http://www.signplus.com/)

– CYBERSIGN (http://www.cybersign.com/)

– CIC (http://www.cic.com/)

• Off-line:– APP-DAVOS (http://www.app-

davos.ch/)– NUMEDIA

(http://www.sapura.com.my/NuMedia/check.htm)

• Advantages of signature verification:– User-friendly.– Well accepted socially and legally.– Non invasive.– Already acquired in a number of applications.– Acquisition hardware:

• Off-line: ubiquitous (pen and paper).• On-line: inexpensive and already integrated in some devices (Tablet

PC).– If compromised, can be changed.– Long experience in forensic environments.

• Disadvantages:– High intra-class variability– Forgeries– Higher error rates than other traits– Affected by the physical and emotional state of the

user– Large temporal variation

Pattern Recognition System(On-line Signature Verification)

Pattern Recognition System

• Acquisition devices:

XY

PA

zA

l

0 100 200 300

Pattern Recognition Process

Preprocessing• For online signatures no segmentation needs to be

performed– All parts of the signature are known after sensing

• Attempt to eliminate noise from the capturing device, speed of writing, and the writing itself

• Minimize the potential of eliminating writer dependencies

• Solutions1) Size Normalization2) Position Normalization3) Smoothing4) Re-sampling5) Ligature

-1500 -1000 -500 0 500 1000 1500 2000 2500-1500

-1000

-500

0

500

1000

-500 0 500 1000 1500 2000 2500 3000 3500 4000-800

-600

-400

-200

0

200

400

600

800

-2000 -1500 -1000 -500 0 500 1000 1500 2000-1000

-500

0

500

1000

1500

-2000 -1500 -1000 -500 0 500 1000 1500 2000 2500-800

-600

-400

-200

0

200

400

600

800

• Position Normalization:

Initial Sample

Center of Mass

Resample• In order to compare two signatures with respect to their

shape, they must be re-sampled to eliminate the dependencies on speed– Sample rate: 100 samples/second

• Temporal features must be extracted beforehand since all local speed information is lost during this process

Resampling

• Ensures that the signature is uniformly smoothed– Segments of high writing velocity will be

smoothed more than segments that are written slow

Smoothing

∑−=

+=σ

σ

2

2*

i

origiti

filteredt xfx• A one dimensional

Gaussian filter is used in both the x and y directions– Small changes in the signal

are smoothed out while the overall structure is kept

• Each segment between critical points is smoothed separately in order to retain their absolute positions

where

∑−=

−

−

=σ

σ

σ

σ

2

2

2

2

2

2

2

2

j

j

i

i

e

ef

Stroke Concatenation

• A stroke is the points input between a pen down and pen up sequence

• All strokes are connected into one long string– This is done in order to facilitate the use of the string

matching procedure

Critical Points• Def: points that carry more information than others

– Endpoints of strokes– Points of trajectory change

Critical Points

• Re-sampling or smoothing of these points will discard important information about the structure and speed of the signature– Accordingly, these points are never changed

throughout preprocessing– The speed information is stored at each of

these points

Preprocessing Steps

• Original

• Critical points

Preprocessing Steps

• Fine re-sampling

• Gaussian filter

Preprocessing Steps

• Coarse re-sampling

• Stoke concatenation

Pattern Recognition Process

Functional vs. Parametric• Functional Approaches (local)

– Complete signals (e.g. x(t), y(t), p(t), v(t), a(t), etc.) are considered as mathematical time functions whose values are directly correlated with the feature set.

– Difficulties are encountered in the matching step (temporal differences and non-linear distortions)

– Feature extracting is relatively simple– More computationally intensive (slower), higher accuracy

• Parametric Approaches– m parameters are computed as features from the measured signals– Feature extraction is very difficult (selection of meaningful features)– Simple matching techniques for comparing 2 sets of parameters can be

used– Very fast matching, lower accuracy

Feature Extraction• Local features

– Spatial: static features extracted from the shape of the signature

• Change of the distance between two consecutive points (δx, δy)

• Absolute y coordinate (y)• Sine and cosine of the angle with the x axis (sin α and cos α)• Curvature (β)• Grey values in a 9x9 pixel neighborhood

– Temporal: features using the ordering (timing) of the signature

• Absolute and relative speed at each re-sampled point• Absolute and relative speed between two critical points

– Pressure value

Local Features• δx and δy for point pi are the

changes with respect to the subsequent point pi+1

• y is the y-coordinate of each re-sampled point after preprocessing

• α is the angle between the x-axis and the line through points pi and pi+1– α is not used (ex. 1 and 359)– sin and cos are both used for

directional information• β is the angle between the

straight lines pi -pi-2 and pi -pi+2

Local Features• The 9x9 pixel neighborhood

is divided into nine 3x3 squares– Grey values are computed by

summing pixel values in each 3x3 square

– Most costly operation• No features are extracted

for the last point of a stroke• Total number of features

– 15

Temporal Features• Absolute and relative speeds are defined as

distance per unit time– Tablet PC captures the position of the pen 100 times

per second– Distance is measured in pixels– Only distance between points is necessary to define

the speed• Speed is normalized by dividing the local speed

at each sample point by the average writing speed of the signature– Overall speed may vary but the relative speeds

should be more stable

Global Features

Pattern Recognition System

String Matching• To compute this alignment, dynamic time warping (DTW) is

used

– D(i,j) is the optimal alignment up to point i of the first string and point j of the second string

– de(i,j) is the Euclidian distance between points i and j

• The overall dissimilarity for a signature

⎪⎩

⎪⎨

⎧

++−

+−−=

Penalty Spurious1Penalty Missing),1(),()1,1(

),()D(i,j-jiD

jidjiDMinjiD

E

),(_),(),(

2

JI NNFactorNormJIDJIDist =

DTW Example• Match strings ‘abcacac’ and ‘bcab’

Difference Score Matrix Reverse Path

Result

String Matching• Each point is represented by an n-ary feature vector

– Feature reduction is performed• Euclidean distance is used as the metric to compare two feature

vectors• Each feature is normalized using the z-score

–

σµ−

=′ff

• A set of pairings between the template and input string is found where the sum of the differences between each pair of aligned points in minimal

User Dependent Normalization• Training set:

– Pair-wise distances between all training samples is calculated (DTW)

– The sample with the smallest average distance is selected as the template

– Normalization Statistics:• Average distance from template• Average maximum distance• Average minimum distance

• Test sample:– Compute DTW against all training samples

• Record: distance from template, maximum distance and minimum distance

– Normalize the 3 distances by dividing them by the set’s average statistics

Dimension Reduction• PCA

– Transform a 3D vector of highly correlated data to 1D– Find a linear transformation W that maps the original

vector (X) to projection coefficient vector (Y)

– Compute Average:

– Covariance Matrix:

– Eigen-decomposition:

XWY T=

∑=

=N

iix

N 1

1µ

∑=

−−=N

i

TiiT xxS

1))(( µµ

eeST λ=

PCA• Project the distance vector (max, min,

template) to 1 dimension

Validation Set• PCA needs samples to calculate the projection

coefficients– Using data from the training or testing sets will bias the results

• Divide data into training, validation, and testing• SVC 2004

– 40 users: 20 genuine signatures, 20 skilled forgeries per user.– Testing procedures:

• Train on 5 genuine signatures• Test on 10 genuine signatures• Test on 20 skilled forgeries• Test on 10 random forgeries

• 2-Fold Cross Validation– Select data from the second half of 20 users (genuine, skilled)

for validation set to train PCA– Test on the first half of 20 users with the calculated coefficients– Repeat this process, switching the 2 groups

SVC Examples

Genuine

Imposter

Mahalanobis Distance• Introduced by P. C. Mahalanobis in 1936• A distance measure which utilizes the

correlations between the features•

• M is the squared Mahalanobis distance• s represents the within-group covariance matrix• y is the vector of the means of the scores for a

group• x is the vector containing the individual scores of a

sample• In our work, a diagonal covariance matrix is

assumed

)()(),( 1 yxSyxyxd M −′−= −

Pattern Recognition System

Score Fusion

Score Fusion

Global FeaturesLocal Features

Weighted Sum Rule

• Assign a test sample to wi if–

And to wa otherwise)|()|()|()|( gaglalgiglil xwPWxwPWxwPWxwPW +>+

Biometric Comparison