If you can't read please download the document
Upload
phungdien
View
240
Download
16
Embed Size (px)
Citation preview
1Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
M s: CT384, 3 Tn ch(KT in t VT, KT iu khin v C in t)
TS. Nguyn Ch NgnB mn T ng Ha
Khoa K thut Cng nghEmail: [email protected]
---2008---
I HC CN TH
2Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
Ni DungChng 1: Tng quan v mng nron nhn to (ANN)Chng 2: Cu trc ca ANNChng 3: Cc gii thut hun luyn ANNChng 4: Mt s ng dng ca ANN (MATLAB) n mn hcChng 5: Mng nron m (Fuzzy-Neural Networks) Chng 6: Mt s nh hng nghin cu (Case Studies)n tp v tho lun
Tham kho:1. Nguyn Ch Ngn, iu khin m hnh ni v Neural network: Chng 2 Mng n-
ron nhn to, Lun n cao hc, HBK Tp. HCM, 2001.2. Nguyn nh Thc, Mng nron Phng php v ng dng, NXBGD, 2000.3. Simon Haykin, Neural Networks a comprehensive foundation, Prentice Hall, 1999. 4. Howard Demuth, Mark Beale and Martin Hagan, Neural Networks toolbox 5
Users Guide, The Matworks Inc., 2007.
3Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
T chc mn hcThi lng mn hc: 3TC
2 TC l thuyt v bi tp trn lp1 TC n mn hc (03 SV thc hin 1 ti)
Lch hc:Tun 1: Chng 1Tun 2 3: Chng 2 + Bi tpTun 4 5: Chng 3 + Bi tpTun 6 7: Chng 4 + Bi tpTun 8 11: n mn hcTun 12 13: Chng 5 + Bi tpTun 14: Chng 6Tun 15: n tp v tho lun
nh gi n mn hc: 45%Thi ht mn: 55%
4Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
Chng 3Cc gii thut hun luyn ANN
Gii thiuCc phng php hun luynMt s gii thut thng dng
Hm mc tiuMt li v cc im cc tiu cc b
Qui trnh thit k mt ANNCc k thut ph tr
Minh ha bng MATLABBi tp
5Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
Gii thiuGi thiu v cc phng php hun luynTm hiu mt s gii thut thng dng hun luyn ANN. Phn ny tp trung vo gii thut Gradient descent v cc gii thut ci tin ca nHm mc tiuMt li v cc im cc tiu cc bMt s v d v phng php hun luyn mng bng MATLABQui trnh thit k mt ANNCc k thut ph trHin tng qu khp ca ANN
6Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
Cc phng php hun luynHun luyn mng l qu trnh thay i cc trng s kt niv cc ngng ca n-ron, da trn cc mu d liu hc, sao cho tha mn mt s iu kin nht nh.
C 3 phng php hc:Hc gim c st (supervised learning)Hc khng gim st (unsupervised learning)Hc tng cng (reinforcement learning).Sinh vin tham kho ti liu [1].
Gio trnh ny ch tp trung vo phng php hc c gimst. Hai phng php cn li, sinh vin s c hc trongchng trnh Cao hc.
7Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
Gii thut hun luyn ANN (1)Trong phn ny chng ta tm hiu v gii thut truyn ngc(backpropagation) v cc gii thut ci tin ca n, p dng chophng php hc c gim st.Gii thut truyn ngc cp nht cc trng s theo nguyn tc:
wij(k+1) = wij(k) + g(k)trong :
wij(k) l trng s ca kt ni t n-ron j n n-ron i, thi im hin ti l tc hc (learning rate, 0< 1)g(k) l gradient hin ti
C nhiu phng php xc nh gradient g(k), dn ti c nhiugii thut truyn ngc ci tin.
8Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
Gii thut hun luyn ANN (2) cp nht cc trng s cho mi chu k hun luyn, gii thuttruyn ngc cn 2 thao tc:Thao tc truyn thun (forward pass phase): p vect d liu votrong tp d liu hc cho ANN v tnh ton cc ng ra ca n.
Thao tc truyn ngc (backward pass phase): Xc nh sai bit (li) gia ng ra thc t ca ANN v gi tr ng ra mong mun trong tp dliu hc. Sau , truyn ngc li ny t ng ra v ng vo ca ANN vtnh ton cc gi tr mi ca cc trng s, da trn gi tr li ny.
p1(k)
p2(k)
pj(k)
pR(k)
wij(k)
wi2(k)
wi1(k)
wRj(k)
f ti(k)
ei(k)
ai(k)+ -
Minh ha phng php iu chnh trng s n-ron th j ti thi im k
9Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
Gii thut gradient descent (1)Xt mt MLP 2 lp:
pn
p1
p2
pia2j
a21
a2m
w2ij l trng s lp ra, t j n iw1ij l trng s lp n,
t j n i
Ng ra n-ron n l a1i
Lp n
10Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
Gii thut gradient descent (2)Thao tc truyn thun
Tnh ng ra lp n (hidden layer):n1i (k) = j w1ij (k) pj (k) ti thi im k a1i(k) = f1( n1i(k) )
vi f1 l hm kch truyn ca cc n-ron trn lp n. Ng ra ca lp n l ng vo ca cc n-ron trn lp ra.
Tnh ng ra ANN (output layer):n2i (k) = j w2ij (k) a1j (k) ti thi im k a2i(k) = f2( n2i(k) )vi f2 l hm truyn ca cc n-ron trn lp ra
11Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
Gii thut gradient descent (3)Thao tc truyn ngc
Tnh tng bnh phng ca li:vi t(k) l ng ra mong mun ti kTnh sai s cc n-ron ng ra:
Tnh sai s cc n-ron n:
Cp nht trng s
Sinh vin tham kho ti liu [3], trang 161-175.
=i
ii kaktkE22 )()(
21)(
[ ] )(')()()(
)()( 222 knfkaktknkEk iii
ii =
=
=
=j
jijii
i wkknfknkEk )()('
)()()( 111
)()()()1(
)()()()1(122
1
kakkwkw
kpkkwkw
jiijij
jiijij
+=+
+=+
lp n:
lp ra:
12Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
Minh ha gii thut hun luyn (1)Xt mt ANN nh hnh v, vi cc n-ron tuyn tnh.
Minh ha gii thut truyn ngc nh sau
a21
a22
p1
p2
w111= -1
w121= 0
w112= 0w122= 1
b11= 1b12= 1
w211= 1
w221= -1
w212= 0w222= 1
b21= 1 b22= 1
Mr DuongSticky Notej ben phai, i ben trai
Mr DuongSticky Notevi noron la lop an dau tien nen p=a. cap nhat trong so phai dua vao delta va a(p) cua 2 no ron nam 2 ben trong so.
13Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
Minh ha gii thut hun luyn (2) n gin, ta cho c cc ngng bng 1, v khng v ra y
Gi s ta c ng vo p=[0 1] v ng ra mong mun t=[1 0]Ta s xem xt tng bc qu trnh cp nht trng s ca mngvi tc hc =0.1
a21
a22
p1=0
p2=1
w111= -1
w121= 0
w112= 0
w122= 1
w211= 1
w221= -1
w212= 0
w222= 1
14Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
Minh ha gii thut hun luyn (3)Thao tc truyn thun. Tnh ng ra lp n:
a21
a22
p1=0
p2=1
w111= -1
w121= 0
w112= 0
w122= 1
w211= 1
w221= -1
w212= 0
w222= 1
a12 = 2
a11 = 1
a11 = f1(n11) = n11 =(-1*0 + 0*1) +1 = 1
a12 = f1(n12)=n12 = (0*0 + 1*1) +1 = 2
15Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
Minh ha gii thut hun luyn (4)Tnh ng ra ca mng (lp ra):
a21=2
a22=2
p1=0
p2=1
w111= -1
w121= 0
w112= 0
w122= 1
w211= 1
w221= -1
w212= 0
w222= 1
a12 = 2
a11 = 1
a21 = f2(n21) = n21 =(1*1 + 0*2) +1 = 2
a12 = f2(n22)=n22 = (-1*1 + 1*2) +1 = 2
Ng ra a2 khc bit nhiu vi ng ra mong mun t=[1 0]
16Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
Minh ha gii thut hun luyn (5)Thao tc truyn ngc
p1=0
p2=1
w111= -1
w121= 0
w112= 0
w122= 1
w211= 1
w221= -1
w212= 0
w222= 1
a12 = 2
a11 = 11= -1
2= -2
Vi ng ra mong mun t =[1, 0],Ta c cc error ng ra:
1 = (t1 - a21 )= 1 2 = -12 = (t2 - a22 )= 0 2 = -2
)()()()1( 122 kakkwkw jiijij +=+
Mr DuongSticky Notemang no ron la tuyen tinh nen dao ham = 1.
17Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
Minh ha gii thut hun luyn (6)Tnh cc gradient lp ra
p1=0
p2=1
w111= -1
w121= 0
w112= 0
w122= 1
w211= 1
w221= -1
w212= 0
w222= 1
a12 = 2
a11 = 11a11= -1
2a12= -4
1a12= -2
2a11= -2
18Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
Minh ha gii thut hun luyn (7)Cp nht trng s lp ra
p1=0
p2=1
w111= -1
w121= 0
w112= 0
w122= 1
w211= 0.9
w221= -1.2
w212= -0.2w222= 0.6
a12 = 2
a11 = 1
)()()()1( 122 kakkwkw jiijij +=+
w211= 1 + 0.1*(-1) = 0.9w221= -1 + 0.1*(-2) = -1.2w212= 0 + 0.1*(-2) = -0.2w222= 1 + 0.1*(-4) = 0.6
Mr DuongSticky Notecap nhat trong so,ta su dung tin hieu ben phai
19Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
1= -1
2= -2
1 w11= -1
2 w21= 21 w12= 0
2 w22= -2
Minh ha gii thut hun luyn (8)Tip tc truyn ngc
p1=0
p2=1
w111= -1
w121= 0
w112= 0
w122= 1
S dng li cc trng s trc khi cp nht cho lp ra, tnh gradient lp n
=j
jijii wkknfk )()(')(11
20Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
Minh ha gii thut hun luyn (9)Tnh cc error trn lp n
p1=0
p2=1
w111= -1
w121= 0
w112= 0
w122= 1
1 = 1 w11 + 2 w21 = -1 + 2 = 1
2 = 1 w12 + 2 w22 = 0 - 2 = -2
1= -1
2= -2
1= 1
2 = -2
21Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
Minh ha gii thut hun luyn (10)Tnh gradient lp n
p1=0
p2=1
w111= -1
w121= 0
w112= 0
w122= 1
1= -1
2= -2
1 p1 = 0
2 p2 = -2
2 p1 = 0
1 p2 = 1
22Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
Minh ha gii thut hun luyn (11)Cp nht trng s lp n
p1=0
p2=1
w111= -1
w121= 0
w112= 0.1w122= 0.8
)()()()1(1 kpktwkw jiijij +=+
w111= -1 + 0.1*0 = -1w121= 0 + 0.1*0 = 0w112= 0 + 0.1*1 = 0.1w122= 1 + 0.1*(-2) = 0.8
1 p1 = 0
2 p2 = -2
2 p1 = 0
1 p2 = 1
23Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
Minh ha gii thut hun luyn (12)Gi tr trng s mi:
w111= -1
w121= 0
w112= 0.1w122= 0.8
w211= 0.9
w221= -1.2
w212= -0.2w222= 0.6
Qu trnh cp nht cc gi tr ngng hon ton tng t.
24Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
Minh ha gii thut hun luyn (13)Truyn thun 1 ln na xc nh ng ra ca mng vi gi trtrng s mi
w111= -1
w121= 0
w112= 0.1w122= 0.8
w211= 0.9
w221= -1.2
w212= -0.2w222= 0.6
p1=0
p2=1
a12 = 1.6
a11 = 1.2
25Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
Minh ha gii thut hun luyn (14)
w111= -1
w121= 0
w112= 0.1w122= 0.8
w211= 0.9
w221= -1.2
w212= -0.2w222= 0.6
p1=0
p2=1
a12 = 1.6
a11 = 1.2
a22 = 0.32
a21 = 1.66
Gi tr ng ra by gi l a2 = [1.66 0.32]gn vi gi tr mongmun t=[1 0] hn.Bi tp: T kt qu ny, sinh vin hy thc hin thao tctruyn ngc v cp nht trng s ANN 1 ln na.
26Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
Hun luyn n khi no? s thi k (epochs) n nh trcHm mc tiu t gi tr mong munHm mc tiu phn k
27Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
Hm mc tiu MSE (1)Mean Square Error MSE l li bnh phng trung bnh, c xc nh trong qu trnh hun luyn mng. MSE cxem nh l mt trong nhng tiu chun nh gi s thnhcng ca qu trnh hun luyn. MSE cng nh, chnhxc ca ANN cng cao.nh ngha MSE:
Gi s ta c tp mu hc: {p1,t1}, {p2,t2}, , {pN,tN}, vi p=[p1, p2, pN] l vect d liu ng vo, t= [t1, t2, , tN] l vect dliu ng ra mong mun. Gi a=[a1, a2, , aN] l vect d liu rathc t thu c khi a vect d liu vo p qua mng. MSE:
c gi l hm mc tiu=
=N
iii atN
MSE1
21
28Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
Hm mc tiu MSE (2)V d 1: Cho ANN 2 lp tuyn tnh nh hnh v.p=[1 2 3; 0 1 1] l cc vect ng vo.t=[2 1 2] l vect ng ra mong mun.Tnh MSE.Gii:
Ng ra lp n: a11=f(n11)=n11=[1.5 3.5 4.5]
a12=f(n12)=n12=[0.2 2.2 2.2]
Lp ra: a2=f(n2)=n2=[3.2 9.2 11.2]
MSE =
MSE = 51.1067
.2
.5
p1
p2
2
1
1
2
1
1
0
1a2
a11
a12 1
0
( ) ( ) ( ) 2223
1
2 )2.112(2.912.3231
31
++==i
ii at
29Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
Hm mc tiu MSE (3)M phng: net=newff([-5 5; -5 5], [2 1], {'purelin', 'purelin'}); net.IW{1,1}=[1 1; 0 2]; % gn input weights net.LW{2,1}=[2 1]; % gn layer weights net.b{1}=[.5; .2]; % gn ngng n-ron lp n net.b{2}=0; % gn ngng n-ron lp ra p=[1 2 3; 0 1 1]; % vect d liu vo t=[2 1 2]; % ng ra mong mun a=sim(net,p) % ng ra thc t ca ANN mse(t-a) % tnh mse
ans =51.1067
30Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
Hm mc tiu MSE (4)MSE c xc nh sau mi chu k hun luyn mng (epoch) vc xem nh 1 mc tiu cn t n. Qu trnh hun luyn ktthc (t kt qu tt) khi MSE nh.
31Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
V d v GT Gradient descent (1)Bi ton: Xy dng mt ANN nhn dng m hnh vo ra cah thng iu khin tc motor DC sau:
32Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
V d v GT Gradient descent (2)Nguyn tc:
(k)=fANN[V(k), V(K-1), V(K-2), (k-1), (k-2)]
Cc bc cn thit:Thu thp v x l d liu vo ra ca i tngChn la cu trc v xy dng ANNHun luyn ANN bng gii thut gradient descentKim tra chnh xc ca m hnh bng cc tn hiu khc
Motor DC
M hnhANN
ng vo V ng ra
~
e = - ~
gradient descent
33Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
V d v GT Gradient descent (3)Thc hin:
M hnh Simulink thu thp d liu: data_Dcmotor.mdl
Chun b d liu hun luyn: ANN_Dcmotor.mload data_DCmotor; % nap tap du lieu hocP=[datain'; dataout(:, 2:3)']; % [V(k), V(k-1), V(k-2), (k-1), (k-2)]T=dataout(:,1)'; % theta(k)
V(k), V(k-1), V(k-2) (k), (k-1), (k-2)
34Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
V d v GT Gradient descent (4)Thc hin:
To ANN v hun luyn: ANN_Dcmotor.m>> net=train(net, Ptrain,Ttrain, [], [], VV,TV);
Nhn xt:Tc hi t ca giithut Gradient descent qu chm.
Sau 5000 Epochs, MSE ch t 6.10-4.
Kt qu kim tra chothy li ln.
Cn gii thut ci tin.
35Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
V d v GT Gradient descent (5)Thc hin:
M hnh kim tra chnh xc ANN: Test_Dcmotor.mdl
(k)=fANN[V(k), V(K-1), V(K-2), (k-1), (k-2)]
36Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
V d v GT Gradient descent (6)Kt qu:
0 5 10 15 20 25 30 35 40 45 50-0.4
-0.3
-0.2
-0.1
0
0.1
0.2
0.3
0.4
Time(s)
(ra
d)
Testing result of DC_motor model
DC_motor outputModel output
37Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
Cc tiu cc bnh hng ca tc hc
Qu trnh hun luyn mng, giithut cn vt qua cc im cctiu cc b (v d: c th thay ih s momentum), t cim global minimum.
E
WLocal minimum global
minimum
38Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
Mt li
Cc tiu mong mun
39Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
Gradient des. with momentum (1)Nhm ci tin tc hi t ca gii thut gradient descent, ngi ta a ra 1 nguyn tc cp nht trng s ca ANN:
vi g(k): gradient; : tc hcij(k-1) l gi tr trc ca ij(k) : momentum
t hiu qu hun luyn cao, nhiu tc gi nghi gi tnggi tr ca moment v tc hc nn gn bng 1:
[0.8 1]; [0 0.2]
)1()()(
)()()1(
+=
+=+
kwkgkw
kwkwkw
ijij
ijijij
40Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
Gradient des. with momentum (2)p dng cho bi ton nhn dng m hnh ca motor DC:
Nhn xt: Sau 5000 Epochs,MSE t 4.10-4, nhanh hn giithut gradient descent.
41Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
GD vi tc hc thch nghi (1)Tc hi t ca gii thut Gradient descent ph thuc vo tc hc . Nu ln gii thut hi t nhanh nhng bt n. Nu nh thi gian hi t s ln.Vic gi tc hc l mt hng s sut qu trnh hun luyn, t ra km hiu qu. Mt gii thut ci tin nhm thay i thchnghi tc hc theo nguyn tc:
42Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
GD vi tc hc thch nghi (1)p dng cho bi ton nhn dng m hnh ca motor DC:
Nhn xt: Sau 5000 Epochs,MSE t 10-4, nhanh hn giithut gradient descent with momentum
43Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
(1)
L gii thut gradient descent ci tin, m c tc hc v hs momentum c thay i thch nghi trong qu trnh hunluyn. Vic thay i thch nghi c thc hin tng th nh victhch nghi tc hc .Th tc cp nht trng s ging nh gii thut GD with momentum:
vi g(k): gradient; : tc hc thch nghiij(k-1) l gi tr trc ca ij(k) : momentum thch nghi
)1()()()()()1(+=
+=+
kwkgkwkwkwkw
ijij
ijijij
44Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
(2)
p dng cho bi ton nhn dng m hnh ca motor DC:
Nhn xt: Sau 5000 Epochs,MSE t 3.10-5, nhanh hn giithut gradient descent with thch nghi
45Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
Gradient direction
GT. truyn ngc Resilient (1)Cc hm truyn Sigmoid nn cc ng vo v hn thnh cc ng rahu hn lm pht sinh mt im bt li l cc gradient s c gi trnh, lm cho cc trng s ch c iu chnh mt gi tr nh, mc dn cn xa gi tr ti u.Gii thut Resilient c pht trinnhm loi b im bt li ny bngcch s dng o hm ca hm li quyt nh hng tng/gim cagradient.
Nu cng du:
wij(k+1) c tng thm 1 lng inc
Nu khc du: wij(k+1) c gim i 1 lng dec
+
ij
k
ij
k
wE
wE 1&
+
ij
k
ij
k
wE
wE 1&
46Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
GT. truyn ngc Resilient (2)
47Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
GT. truyn ngc Resilient (3)p dng cho bi ton nhn dng m hnh ca motor DC:
Nhn xt: Sau 5000 Epochs,MSE t 8.10-6, nhanh hn giithut gradient descent with & thch nghi
48Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
Gii thut BFGS Quasi-NewtonSinh vin t c ti liu [1] v [3]p dng cho bi ton nhn dng m hnh motor DC:
Nhn xt: Sau 1100 Epochs,MSE t 2.10-7, nhanh hn giithut Resilient
49Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
Gii thut Levenberg-Marquardt (1)Gii thut Levenberg-Marquardt c xy dng t tc hi t bc 2 m khng cn tnh n ma trn Hessian nh giithut BFGS Quasi-Newton.Ma trn Hessian c tnh xp x: H=JTJ v gi tr gradient c xc nh: g=JTetrong , J l ma trn Jacobian, cha o hm bc nht ca hmli (e/wij), vi e l vect li ca mng.Nguyn tc cp nht trng s:
wij(k+1)=wij(k) [JTJ + mI]-1 JTe
Nu m=0, th y l gii thut BFGS Quasi-Newton.Nu m c gi tr ln n l gii thut gradient descent.Gii thut Levenberg-Marquardt lun s dng gi tr m nh, do gii thut BFGS Quasi-Newton tt hn gii thut gradient descent.
50Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
Gii thut Levenberg-Marquardt (1)p dng cho bi ton nhn dng m hnh ca motor DC:
Nhn xt: Sau 20 Epochs,MSE t 1,7.10-7, nhanh hn giithut Newton
51Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
So snh cc gii thutSo snh trn bi ton nhn dng m hnh mt i tng phi tuyn, c trnh by trong ti liu [1]:
52Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
Cc gii thut ca NN. toolboxTrainb Batch training with weight & bias learning rules.Trainbfg BFGS quasi-Newton backpropagation.Trainbr Bayesian regularization.Trainc Cyclical order incremental training w/learning functions.Traincgb Powell-Beale conjugate gradient backpropagation.Traincgf Fletcher-Powell conjugate gradient backpropagation.Traincgp Polak-Ribiere conjugate gradient backpropagation.Traingd Gradient descent backpropagation.Traingdm Gradient descent with momentum backpropagation.Traingda Gradient descent with adaptive lr backpropagation.Traingdx Gradient descent w/momentum & adaptive lr backpropagation.Trainlm Levenberg-Marquardt backpropagation.Trainoss One step secant backpropagation.Trainr Random order incremental training w/learning functions.Trainrp Resilient backpropagation (Rprop).Trains Sequential order incremental training w/learning functions.Trainscg Scaled conjugate gradient backpropagation.
53Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
Qui trnh thit k mt ANN
B A
B A
54Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
55Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
Tin x l d liu (1)Phng php chun ha d liu:Chun ha tp d liu nm trong khong [-1 1].
Gi p[pmin, pmax] l vect d liu vo
ps l vect d liu sau khi chun ha, th:
Nu ta a tp d liu c x l vo hun luyn mng, thcc trng s c iu chnh theo d liu ny. Nn gi tr ng raca mng cn c thao tc hu x l.Gi a l d liu ra ca mng, at gi tr hu x l, th:
1pp
pp2pminmax
mins
=
D liu vop[pmin, pmax]
Chun haps[-1, 1]
ANN Hu x la
at
( )( ) minminmaxt p pp1a21a ++=
56Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
Tin x l d liu (2)Phng php chun ha d liu (v d)
57Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
Tin x l d liu (3)Phng php tr trung bnh v lch chun:Tin x l tp d liu c tr trung bnh bng 0 (mean=0) v lch chun bng 1 (standard deviation=1).
Gi p l vect d liu vo, c tr trung bnh l meanp v lch chun l stdp, th vect d liu c x l l:
Gi a l vect d liu ra ca ANN, th vect d liu ng ra saukhi thc hin thao tc hu x l:
p
ps std
eanmpp
=
ppt meanstdaa += *
58Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
Tin x l d liu (4)Phng php tr trung bnh v lch chun (v d)
Mean = 0
Std = 1
Mean 0
Std 1
59Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
Nng cao kh nng tng qut ha (1)Mt vn xut hin trong qu trnh hun luyn mng, lhin tng qu khp (overfitting). Khi kim tra mng bng tp d liu hun luyn, n cho ktqu tt (li thp). Nhng khi kim tra bng d liu mi, kt qurt ti (li ln). Do mng khng c kh nng tng qut ha cctnh hung mi (hc vt).C 2 phng php khc phc: Phng php nh ngha li hmmc tiu v phng php ngng sm.
60Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
Nng cao kh nng tng qut ha (2)Phng php nh ngha li hm mc tiu:
Thng thng hm mc tiu c nh ngha l:
Hm mc tiu c nh ngha li bng cch thm vo i lngtng bnh phng trung bnh ca cc trng s v ngng, MSW, khi :
Vi l mt hng s t l vn tng s trng sv ngng ca mng
61Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
Nng cao kh nng tng qut ha (3)Phng php ngng sm:
Phng php ny i hi chia tp d liu hc thnh 3 phn, gm d liu hun luyn, d liu kim tra v dliu gim st.
P = [Ptrain, Ptest, Pvalidation]
Sau li thi k hun luyn, tp d liu dm st Pvalidationc a vo mng kim tra li. Nu li thu cgim, qu trnh hun luyn c tip tc. Nu li thuc bt u tng (hin tng qu khp bt u xy ra), qu trnh hun luyn c dng li gi l ngng sm.Sinh vin c thm ti liu [1].
62Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
Minh ha bng MATLABBi ton:
Nhn dng m hnh h thng m t, cphng trnh vi phn m t h:
Vi i(t) [0, 4A] dng in ng voy(t) l khong cch t nam chm vnh cun nam chm in.cc tham s: =12, =15, g=9.8 v M=3.
63Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
Bi tp1. Sinh vin thc hin li bi ton nhn dng m hnh
motor DC v hun luyn mng bng tt c cc giithut ca NN toolbox ca MATLAB. So snh tc hi t ca cc gii thut.
64Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
Chng 4Mt s ng dng ca ANN
Gii thiuNhn dng k t (OCR)
Nhn dng ting niThit k cc b iu khin
Kt lunBi tp
65Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
Gii thiuGii thiu mt s hng ng dng ANNPht trin thnh Lun vn tt nghip hay tiNCKH sinh vin
66Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
Nhn dng k tMa trn ha bitmap ca k tGi lp cc hnh thc nhiuTp hp d liu hun luyn,mi k t l 1 vect d liung vo
Qui c ng ra, gi s l mASCII tng ng ca k t.
Xy dng cu v hun luyn ANN
67Mng N-ron nhn to, Ts. Nguyn Ch Ngn, 2007 Chng 3 & 4
Nhn dng ting ni
Trch c trng tn hiu ting niTp hp d liu vo, qui c d liu raHun luyn v th nghimPhng php LPC & AMDF xc nh c trng ting ni
Untitled