Group Sparse Autoencoder for Extracting Latent …faculty.iitmandi.ac.in/~aditya/cs671/cs671_2017/data/...it is quite di cult to de ne a model for extracting minutiae. Figure:Latent

Group Sparse Autoencoder for ExtractingLatent Fingerprint Minutia Patterns

Reference Paper(Group Sparse Autoencoder)Published in (Image and Vision Computing,2017)

Presented ByAvantika Singh

School of Computing and Electrical Engineering (SCEE)Indian Institute of Technology, Mandi

May 9, 2017

Outline

Introduction

Problem Statement

Autoencoders

Regularization

Group Sparse Autoencoders(GSAE)

Majorization and Minimization

Proposed System

Experimental Results

Conclusion

References

Avantika Singh (SCEE, IIT-Mandi) [email protected] May 9, 2017

Introduction

Fingerprints can be broadly classified into two categories.

Patent FingerprintsLatent Fingerprints

Figure: Patent Fingerprint


Introduction

A good fingerprint normally consists of 40 to 100 minutia points.

Ridge ending and ridge bifurcation are the two main types of minutias.

Figure: (a)Ridge Ending (b)Ridge Bifurcation


Introduction

Figure: Patent Fingerprint Image Patches(Image taken from [1])


Problem StatementMain objective is to extract features from latent fingerprint.

Latent fingerprints are the fingerprints that are left on the surface bydeposits of oils or perspiration from the finger.

It is not usually visible to the naked eye but can be detected by usingspecial techniques.

Figure: Latent Fingerprint


Problem StatementOur Goal is to extract minutia patterns from latent fingerprint.Latent fingerprint minutia patches lack a definite structure as well asthe of minutia points are less typically between 20 to 30Due to the non-uniform and uncertain variations in latent fingerprints,it is quite difficult to define a model for extracting minutiae.

Figure: Latent Fingerprint Image Patches(Image taken from [1])


Autoencoders

An autoencoder is a neural network that tries to reconstruct its input.It mainly has three layers: an input layer, a hidden layer (encodinglayer) and a decoding layer.

The network is trained to reconstruct its input, which forces thehidden layer to learn good representation of the inputs.

It learns a non-linear mapping function between data and its featurespace.

For building an autoencoder just three things are needed. They are:

An Encoding FunctionA Decoding FunctionA distance function between the amount of information loss betweenthe compressed representation of your data and the decompressedrepresentation (i.e loss function).

It is unsupervised in nature.


Autoencoders

Figure: Autoencoder block diagram representation


Autoencoders

Figure: Autoencoder block diagram representation


Major Application Areas of Autoencoder

Data Denoising

Dimensionality Reduction

Figure: Noisy Images

Figure: Reconstructed Images


Regularized Autoencoder

It uses a loss function that encourges the model to have otherproperties besides the ability to copy its input to its output.

These properties include sparsity of the representation, robustness tonoise or to missing values.


Regularization

It is the technique used in machine learning to prevent overfitting ofthe data,by preventing the coefficients to fit perfectly.

Figure: Comparision between unregularized and regularized results (Images takenfrom Prof. Alexander Ihler Presentation)


Regularization FunctionsRegularizer is defined as

Figure: Different regularization functions (Images taken from Prof. AlexanderIhler Presentation)


Regularization Functions

Total loss = Data loss + Regularizer term

Estimate balance between the data term and the regularization term.

Lasso generates sparse solution than a quadratic regularizer.

Figure: (Images taken from Prof. Alexander Ihler Presentation)


Group Sparse Autoencoder (GSAE)

GSAE is the supervised version of autoencoder,which utilizes thelabelled information of the data.

The learning is performed such that the features of a single class willhave the same sparsity signature.

This is achieved by incorporating L2,1 norm regularization.

L2,1 norm of a matrix M ∈ Rn×m is given as:

L2,1 norm of a matrix is the sum of the euclidean norms of thecolumns of the matrix.



Let X be the input data as shown in the figure below

Figure: Original Data (Image taken from )

Here, C is the number of classes.

Data is organized such that all the data columns belonging to class 1appear first, followed by data columns of class 2, and so on.


Group Sparse Autoencoder (GSAE)The loss function J is defined as

Here X is input data, W is the encoding weights, U is the decodingweights , φ is any non-linear activation function and R(W) is aregularization term,here it is L2,1 norm.

Figure: Block diagramAvantika Singh (SCEE, IIT-Mandi) [email protected] May 9, 2017


Loss function:

Here the inner L2 norm promotes dense solution within the selectedrows whereas the outer L1 norm enforces sparsity in selecting therows.

In this way regularizer enforces group sparsity within each class byadding the constraint that the features from the same group/classshould have the similar sparsity signature.

This makes the optimization supervised as the information regardingthe class labels is utilized during training, which further makes GSAEa supervised version of autoencoder.



The original loss function J is defined as

This function is non-convex in nature so we can split it into two partsas:

Step 1 is simple linear least squares regression problem.It can beeasily solved.

For solving Step 2 majorization and minimization technique is used.



Majorization Step Let, J(x) is the function to be minimized. Startwith an initial point (at k=0) xk . A smooth function Gk(x) isconstructed through xk which has a higher value than J(x) for allvalues of x apart from xk , at which the values are the same.

Figure: Original Function(Image taken from [2] )



The function Gk(x) is constructed such that it is smooth and easy tominimize.

Figure: Majorization(Image taken from [2] )



Minimization Step At each step, minimize Gk(x) to obtain the nextiterate xk+1. A new Gk+1(x) is constructed through xk + 1 which isnow minimized to obtain the next iterate xk+2. In this way finalsolution is obtained iteratively.

Figure: Original Function(Image taken from [2])


Proposed System

Figure: Proposed System Block diagram(Image taken from [1])


Experimental Results

Proposed system accuracy is verified on two publicly available latentfingerprint datasets namely: NIST SD-27 and MOLF.

Here CCA is Correct Classification Accuracy, MDA is MinutiaDetection Accuracy and NMDA is Non-Minutia Detection Accuracy.

Figure: Classification results obtained on the NIST SD-27 and MOLF latentfingerprint datasets(Image taken from [1])


Conclusion

A novel supervised regularization method for autoenocoders usingL2,1 norm is proposed (GSAE) which utilizes class labels to learnsupervised features for the specific task at hand.

Using GSAE, an automatic latent fingerprint minutia extractionalgorithm is formulated as a binary classification algorithm.


References

Anush Sankaran, Mayank Vatsa , Richa Singh and Angshul Majumdar,G roup sparse autoencoder,Image and Vision Computing,volume 60,year 2017,

Vanika Singhal, and Angshul Majumdar, Majorization andMinimization technique for optimally solving Deep Dictionarylearning,Neural Process Letters.,year 2017


THANK YOU


Documents

Group Sparse Autoencoder for Extracting Latent …faculty.iitmandi.ac.in/~aditya/cs671/cs671_2017/data/...it is quite di cult to de ne a model for extracting minutiae. Figure:Latent