Neural Assignment

8/14/2019 Neural Assignment

1/76

NEURAL ASSIGNMENT(MEE 10403)

NEURAL NETWORK FOR FACE RECOGNITION

Code of Course MEE 10403

Nae of Course COMPUTATIONAL INTELLIGENCE

NAME 1!NAME "!

DINESHWARAN GUNALAN (HE120104)TUAN MOHD MUSTAQIM (HE120092)

LECTURER NAME PROF. MADYA DR. JIWA !" A#DULLAH

FACULT# OF ELECTRICAL AN$ ELECTRONIC ENGINEERING


2/76

Fa%u&' of E&e%'r%a& a*d E&e%'ro*% E*+*eer*+

T!$%& ' NEURAL NETWOR FOR FACE RECOGNITION A&"*&"$ D+$& 13,04,2013

Table Of Contents PageTable Of Contents 2

List Of Figures 4

List Of Tables 6

Abstract 7

Aim and Objective 8

Ca!ters

"#$%T&O'(CT$O% )

"#* Project Overvie+ )

2#L$T,&AT(&, &,-$,. "*

2#* %eural %et+or/ 0asic "*

2#" 1istorical Pers!ective "*

2#2 1istorical Of Face &ecognition "2

#%,(&AL %,T.O&3 A&C1$T,CT(&, "4

#* Classification of %eural %et+or/s "4

#" 0ac/!ro!agation %et+or/s "

#2 %et+or/ 'esign 2*

# %et+or/ Arcitecture in 5imulin/ 2"

4#%,(&AL %,T.O&3 T&A$%$% 24

4#* T!e Of Training 24

4#"0ac/!ro!agation Algoritm 2

4#2 &esilient 0ac/!ro!agation Algoritm 9&P&OP: 2

4# Conjugate radient Algoritms 27

4#4 5caled Conjugate radient Algoritm 95C: 28

4# Transfer Function 2)

4#6 ive Training $mages To %et+or/ *

JEP, FKEE (Semester 1 2012/2013) 2


3/76



4#7 5etting Training Parameters 2

4#8 Training .it ;ore Faces 2

#%,(&AL %,T.O&3 T,5T$% 4

#* %et+or/ 5imulation 4

#" Post


4/76



Figure #"@ Arcitecture of 0ac/!ro!agation net+or/ "6

Figure #2@ Log sigmoid transfer function "7Figure# &etro

Figure #@ Arcitecture of te net+or/ 2"

Figure #6@ %eural net+or/ diagrams 2"

Figure #7@ %eural net+or/ laers 22

Figure #8@ First laer of te net+or/ 22

Figure #) 5econd laer of te net+or/ 22

Figure #"*@ First and 5econd laer +eigts of te net+or/ 2Figure 4#"@ %eural net+or/ diagrams# 24

Figure 4#2 Log


5/76



Figure 7#"4@ Person 0 train times gra!s

Figure 7#"@ Person 0 train "* times gra!s 4Figure 7#"6@ Person 0 train 2* times gra!s

Figure 7#"7@ Original image 6

Figure 7#"8@ &econstructed image 6

Figure 7#")@ &econstructed image 6

Figure 7#2*@ &econstructed image 6

Figure 7#2"@ &econstructed image 6

Figure 7#22@ Person A train " time gra!s )

Figure 7#2@ Person A train times gra!s 6*Figure 7#24@ Person A train "* times gra!s 6"

Figure 7#2@ Person A train 2* times gra!s 62

Figure 7#26@ Original image 6

Figure 7#27@ &econstructed image 6

Figure 7#28@ &econstructed image 6

Figure 7#2)@ &econstructed image 6

Figure 7#*@ &econstructed image 6

Figure 7#"@ Person 0 train " time gra!s 66

Figure 7#2@ Person 0 train times gra!s 67

Figure 7#@ Person 0 train "* times gra!s 68

Figure 7#4@ Person 0 train 2* times gra!s 6)

Figure 7#@ Original image 7*

Figure 7#6@ &econstructed image 7*



Figure 7#)@ &econstructed image 7*

List Of Tables

Page

Table 2#"@ 1istorical notes of neural net+or/ "*

Table 7#"@ Target value com!are +it training value 4)

JEP, FKEE (Semester 1 2012/2013)


6/76


7/76



!rocessing !erformed at com!uting elements or nodes B2# Peo!le in com!uter vision and

!attern recognition ave been +or/ing on automatic recognition of uman faces for te last 2*ears B6# Face recognition as establised itself as an im!ortant sub


8/76



2# To understand and e=amine !roblem from a %eural %et+or/s !oint of vie+ b

!resenting +or/ on face recognition## To stud and a!!l face recognition tecni?ue#

Objectives

Te objectives of tis !roject are mainl@

"# To introduce ;ATLA0

%eural %et+or/ Toolbo= in training and recogni>ing te

face#

2# To be able to describe te !rinci!al stages of face recognition +it %eural %et+or/

using good design metodolog#

# To address te benefits of %eural %et+or/#

4# To cec/ and corroborate te reliabilit of te said net+or/ +ic +as conducted#

C1APT,& "

$%T&O'(CT$O%

"#* Project Overvie+

Artificial %eural %et+or/s 9A%%: or sim!l %eural %et+or/s can be loosel defined

as large sets of interconnected sim!le units +ic e=ecute in !arallel to !erform a common

global tas/# Tese units usuall undergo a learning !rocess +ic automaticall u!dates

JEP, FKEE (Semester 1 2012/2013) 8


9/76



net+or/ !arameters in res!onse to a !ossibl evolving in!ut environment# Te units are often

igl sim!lified models of te biological neurons found in te animal brain B8#Face recognition as ra!idl gained im!ortance +itin te field of !attern recognition

+it a variet of interesting a!!lications in areas suc as securit or inde=ing of image and

videodatabases.

;ATLA0 %eural %et+or/ Toolbo= +as used for tis !roject# A standard

0ac/!ro!agation feed for+ard t+o


10/76



visual !erce!tion s!eec understanding and sensor information !rocessing and in ada!tivel

as +ell as intelligent decision ma/ing in general come from te organi>ational andcom!utational !rinci!les e=ibited in te igl com!le= neural net+or/ of te uman brain#

,=!ectations of faster and better solutions !rovide us +it te callenge to build macines

using te same com!utational and organi>ational !rinci!les sim!lified and abstracted from

neurobiological studies of te brain B"#

2#" 1istorical Pers!ective

Te istor of neural net+or/s is usuall !resented as a series of G+aves of researcG#

Tis as !robabl led cnics to discount neural net+or/ researc as igl vulnerable to Gerd

mentalitG and to brand te field as noting more tan G!eG and GfasionG B8# Table 2#" +as

te istorical notes about neural net+or/#

Table 2#"@ 1istorical notes of neural net+or/

Year Description

1943

McCulloch and Pitts (start of the modern era of neural

networks).

Logical calculus of neural net+or/s# A net+or/ consists of sufficient

number of neurons 9using a sim!le model: and !ro!erl set sna!tic

connections can com!ute an com!utable function#

1949

e!!"s !ook #The organization of behavior#.

An e=!licit statement of a !siological learning rule for synaptic

modification+as !resented for te first time# 1ebb !ro!oses tat te

connectivit of te brain is continuall canging as an organism

learns differing functional tas/s and tat neural assemblies are

created b suc canges# 1ebbHs +or/ +as immensel influential

among !scologists

JEP, FKEE (Semester 1 2012/2013) "*


11/76



19$%

&osen!latt introduced Perceptron

A novel metod of su!ervised learning#

Perce!tron convergence teorem#

Least mean


12/76



Te subject of face recognition is as old as com!uter vision bot because of te

!ractical im!ortance of te to!ic and teoretical interest from cognitive scientists#Pera!s te most famous earl e=am!le of a face recognition sstem is due to 3oonen

B +o demonstrated tat a sim!le neural net could !erform face recognition for aligned and

normali>ed face images# Te t!e of net+or/ e em!loed com!uted a face descri!tion b

a!!ro=imating te eigenvectors of te face imageHs autocorrelation matri=D tese eigenvectors

are no+ /no+n as Ieigenfaces#H

3irb and 5irovic 9")8): B4 later introduced an algebraic mani!ulation +ic made

it eas to directl calculate te eigenfaces and so+ed tat fe+er tan "** +ere re?uired to

accuratel code carefull aligned and normali>ed face images# Tur/ and Pentland 9"))": B

ten demonstrated tat te residual error +en coding using te eigenfaces could be used bot

to detect faces in cluttered natural imager and to determine te !recise location and scale of

faces in an image# Te ten demonstrated tat b cou!ling tis metod for detecting and

locali>ing faces +it te eigenface recognition metod one could acieve reliable real


13/76



C1APT,&

%,(&AL %,T.O&3 A&C1$T,CT(&,

#* Classification of %eural %et+or/s

Artificial %eural %et+or/s 9A%%s: also called !arallel distributed !rocessing sstems

9P'Ps: and connectionist sstems are intended for modeling te organi>ational !rinci!les of

te central nervous sstems +it te o!e tat te biologicall ins!ired com!uting

ca!abilities of te A%% +ill allo+ te cognitive and sensor tas/s to be !erformed more

easil and more satisfactor tan +it conventional serial !rocessors B"#

%eural %et+or/ models can be classified in a number of +as# (sing te net+or/ arcitecture

as basis tere are tree major t!es of neural net+or/s B8@

Recurrent networks< te units are usuall laid out in a t+oation !rocess +ere te net+or/ units cange teir activation values and slo+l

evolve and converge to+ard a final configuration of Glo+ energG# Te final

configuration of te net+or/ after stabili>ation constitutes te out!ut or res!onse of te

net+or/# Tis is te arcitecture of te1o!field ;odel#

JEP, FKEE (Semester 1 2012/2013) "
http://www.comp.nus.edu.sg/~pris/AssociativeMemory/HopfieldModel.htmlhttp://www.comp.nus.edu.sg/~pris/AssociativeMemory/HopfieldModel.htmlhttp://www.comp.nus.edu.sg/~pris/AssociativeMemory/HopfieldModel.html


14/76



Feed forward networksJ tese net+or/s distinguis bet+een tree t!es of units@ in!ut

units idden units and out!ut units# Te activit of tis t!e of net+or/ !ro!agatesfor+ard from one laer to te ne=t starting from te in!ut laer u! to te out!ut laer#

5ometimes called multilaered net+or/s feed for+ard net+or/s are ver !o!ular

because tis is te inerent arcitecture of te0ac/!ro!agation ;odel#

Competitive networksJ tese net+or/s are caracteri>ed b lateral inibitor

connections bet+een units +itin a laer suc tat te com!etition !rocess bet+een

units causes te initiall most active unit to be te onl unit to remain active +ile all

te oter units in te cluster +ill slo+l be deactivated# Tis is referred to as a

G+inner

"#

JEP, FKEE (Semester 1 2012/2013) "4
http://www.comp.nus.edu.sg/~pris/ArtificialNeuralNetworks/MultiLayeredPerceptrons.htmlhttp://www.comp.nus.edu.sg/~pris/ArtificialNeuralNetworks/MultiLayeredPerceptrons.htmlhttp://www.comp.nus.edu.sg/~pris/SelfOrganizingMaps/SelfOrganizingMapsIndex.htmlhttp://www.comp.nus.edu.sg/~pris/ArtificialNeuralNetworks/MultiLayeredPerceptrons.htmlhttp://www.comp.nus.edu.sg/~pris/SelfOrganizingMaps/SelfOrganizingMapsIndex.html


15/76



$n feedfor+ard activation units of idden laer " com!ute teir activation and out!ut values

and !ass tese on to te ne=t laer and so on until te out!ut units +ill ave !roduced te

net+or/Hs actual res!onse to te current in!ut# Te activation value a /of unit / is com!uted as

follo+s#

As illustrated above =iis te in!ut signal coming from unit i at te oter end of te

incoming connection# +/iis te +eigt of te connection bet+een unit / and unit i# (nli/e in

te linear tresold unit te out!ut of a unit in a bac/!ro!agation net+or/ is no longer based

on a tresold# Te out!ut /of unit / is com!uted as follo+s@

JEP, FKEE (Semester 1 2012/2013) "

Figure #"@ Arcitecture of 0ac/!ro!agation net+or/


16/76



Te function f9=: is referred to as te out!ut function# $t is a continuousl increasing

function of tesigmoidt!e asm!toticall a!!roacing * as = decreases and asm!toticalla!!roaces " as = increases# At = * f9=: is e?ual to *##

Figure #2@ Log sigmoid transfer function

Once activation is fed for+ard all te +a to te out!ut units te net+or/s res!onse is

com!ared to te desired out!ut di+ic accom!anies te training !attern# Tere are t+o t!es

of error# Te first error is te error at the output layer# Tis can be directl com!uted as

follo+s@

Te second t!e of error is teerror at the hidden layers# Tis cannot be com!uted

directl since tere is no available information on te desired out!uts of te idden laers#

Tis is +ere te retro

,ssentiall te error at te out!ut laer is used to com!ute for te error at te idden

laer immediatel !receding te out!ut laer# Once tis is com!uted tis is used in turn to

com!ute for te error of te ne=t idden laer immediatel !receding te last idden laer#

Tis is done se?uentiall until te error at te ver first idden laer is com!uted# Te retroe of +eigt

adjustments de!ending on te actual out!ut f9=:# $n te case of te sigmoid function above its

first derivative 9slo!e: f9=: is easil com!uted as follo+s@

%ote tat te cange in +eigt is directl !ro!ortional to te error term com!uted for

te unit at te out!ut end of te incoming connection# 1o+ever tis +eigt cange is

JEP, FKEE (Semester 1 2012/2013) "8


19/76



controlled b te out!ut signal coming from te in!ut end of te incoming connection# $t can

infer tat ver little +eigt cange 9learning: occurs +en tis in!ut signal is almost >ero#Te +eigt cange is furter controlled b te term f9a /:# 0ecause tis term measures

te slo!e of te function and /no+ing te sa!e of te function +e can infer tat tere +ill

li/e+ise be little +eigt cange +en te out!ut of te unit at te oter end of te connection

is close to * or "# Tus learning +ill ta/e !lace mainl at tose connections +it ig !re

Documents

Neural Assignment