Proceedings of the 2007 International Conference on Information AcquisitionJuly 9-11, 2007, Jeju City, Korea

Identification of Nonlinear System Based on aNew Hybrid Gradient-Based PSO Algorithm

Sanfeng Chen"2 Tao Mei 1 ,Minzhou Luol and Xiuqing Yang" 2Centerfor Biomimetic Sensing and Control Research 2Department ofAutomation

Institute ofIntelligent machines, Chinese Academy ofScience University ofScience and Technology ofChinaHefei, Anhui Province, China Hefei, Anhui Province, China

csf8262740@ 163 .com {tmei&lmz} @iim.ac.cn

Abstract - Because the number of iterations necessary to lo- izes a "competitive" strategy. Thus, suboptimal solutions incate the global best solution is not known a priori, it's problem- PSO algorithm can survive and contribute to the search proc-atic to make a proper choice of inertial weights CO and constric- ess at the latter period of iteration. The PSO algorithm has

tion coefficients / of Particle Swarm Optimization(PSO) algo- been shown to perform better than GA in optimizing parame-rithm in advance. The existing PSO algorithms are sensitive to ters of neural network [2]. It has become an important algo-the above two parameters. In addition, standard PSO algorithms rithm in identification of complicated nonlinear system.convergence slowly and coarsely in the latter period. A new hy- Whereas, the number of iterations necessary to locate thebrid PSO algorithm is proposed to overcome the above short- global best solution is not known a priori of PSO algorithm,comings. The new algorithm utilizes original PSO algorithm for it's problematic to make a proper choice of inertial weightslocating approximately a good local minimum, and then a conju-gate gradient based local search is done with the best solution ..and constin cO algorithm. i ad-found by the PSO algorithm as its starting point for finding local vance The existingPI 0 algorithms are sensithve to the aboveminimum accurately. A new optimization circle begins with the two parameters. In addition, P50 algorithms convergenceaccurate local minimum as global best particle. The simulation slowly and coarsely in the latter period. A new hybrid PSOresults show that the new algorithm convergences more fast and algorithm is proposed to overcome the above shortcomings.accurately than GA. It also shows better performance than GA in We utilized the new hybrid PSO algorithm for optimizingidentifying the parameters ofRBF neural networks. parameters of RBFNN. The experimental results have shown

that the new hybrid PSO algorithm is effective and out-Index Terms - PSO, nonlinear system identification, conjugate standing.

gradient algorithm, RBF neural network.II. MODELLING BASED ON RBFNN

I. INTRODUCTIONRBFNN is a feedforward neural networks with three lay-

Radial basic function neural networks (RBFNN) has in- ers, i.e., input layer, hidden layer and output layer. The topo-creasingly been used in many areas for solving some compli- logical structure of RBFNN is as shown in Fig. 1:cated problems. This is due to its approximation capability,i.e., to the property that any continuous function can be ap- X1 Aproximated within an arbitrary accuracy. /t

Accordingly, BFNN is widely used in identification ofnonlinear systems. The key problems of RBFNN are as fol-lowing: determining centers and widths of radial basic func-tion, the number of hidden nodes, weights between hidden I \layer and output layer, and the parameters of hidden layer are I

optimized locally not globally. Output LayerIn strongly nonlinear system, parameters should be 4I

searched globally to achieve accurate identification. GeneticAlgorithm (GA) is a typical global optimization algorithm [1],which is widely used in identification of nonlinear systemeffectively. However, the optimization process via GA iscomparatively complex.

The Particle Swarm Optimization (PSO) algorithm is a Input Lernew optimization algorithm inspired by social behavior Hi dden Layer(schooling) in birds and fish. The PSO algorithm has fewer Fig.l Topological structureofRFNNthan evolutionary computation schemes for a variety ofbenchmark optimization problems. Like GA, PSO algorithm The input vector is X= (Xi, X2¢ ............Xn) ,and Gaussiansearches multiple solutions in parallel. However, PSO algo- Fucto wrsathacitonfcin. Ths th uptorithm employs "cooperative" strategy unlike GA, which util-

1-4244-1220-X/07/$25.OO ©C2007 IEEE 265

each each hidden node is as following: hidden node is as fol- x, (k) -the jh component of the state for particle i at itera-lowing:

tion k-(X-ci)2 r, r2j - uniformly distributed random numbers

Hi(x) = e . (1) p,/ (k) - the jth component of the best solution found by par-

ticle i at iteration k so far

Where Ci is the i th RBFNN center, X is the input vector, p, (k) -the jth component of the best solution found glob-and 07 is the kernel width. The output of networks is: ally at iteration k so far.

m C1' C2-the learning rateY(x)= w1H1(x) (2) w -the inertial weight

The individuals in the swarm are labelled from I to m .

Thus, in order to approximate a given nonlinear mapping, Shi and Eberhart found that adjusting inertial weights wthe networks parameters Ci I 07, w, have to be determined

can control the particle's velocity effectively, and it can im-the networks parametersChavetobedeterprove the algorithm's performance greatly[5]. Whereas, the

Where Ci =( Cil, Ci2,.Cin ) , 0i =( u7il, Ti2. )( research results by Ozcan and Mohan [6] have shown that

i=1,2...m. Let adjusting w can't improve the performance effectively, forthe contribution of inertial weight w may be neglected for

Q = C... Cm ' 5T n .. (M , WI}. (3) searching randomly. To maintain the balance between explo-Y(X) =Y (X) ration and exploitation in PSO algorithm, Clerc proposed a

Where 0 Q. Acordingly,such a nural netwrkshave new PSO algorithm with constriction coefficient / as follow-Where 0 E Q . Accordingly, such a neural networks have ing:to estimate 2mn + m parameters. vJ (k + 1) = l(vj (k) + clrlj (p1 (k) - xJ (k))

The error function is as following: +c2r2(p (k) - xJ (k))) (8)

1= ['l(Xk)-YO(Xk)] (4)2Nk=l WhereThus, system identification is to search the Q globally and 2find O'e Q (9)

E(O') = min(E(O)) (OE Q). (5) 2-C-C C2-4CIt is actually to optimize the parameters of RBFNN [3]. From and C = c1 + c2,C > 4(3),(4), we know it is complicated to optimize parameters of However, since the location of best solution is not knownRBFNN. Accordingly, there are many parameters to be esti- in advance, proper choice of inertial weights or constrictionmated in RBFNN. PSO algorithm has shown outstanding per- coefficients is problematic because the number of iterationsformance in searching large space and globally optimizing. In necessary to locate the best solution is not known a priori. Aaddition, PSO algorithm converges fast. new hybrid conjugate gradient -based PSO algorithm is pro-

III STANDARD PSO ALGORITHM AND NEW HYBRID posed to solve the above problems.In the new hybrid algorithm, the original PSO algorithm

is firstly used to approximately locate a good local minimum.PSO algorithm is a newly proposed algorithm of evolu- Then a conjugate gradient based local search is done with the

tionary computation. It was firstly introduced by Doctor Eber- best solution found by the PSO algorithm as its starting point.hart and Doctor Kennedy[4]. If the best solution found by the PSO algorithm (G) has a lar-

In the standard PSO algorithm, the particle in the kth it- ger cost than the final solution found by local search duringeration is updated as following equations: the previous iteration (L), then L is used as the starting point

for local search. This ensures that the local search is done invi, (k + 1) = w * v. (k) + clrtj (p, (k) -xJ (k)) the neighborhood of the best solution found by the GPSO in

(6) all previous iterations. Thus, the original PSO algorithm is+c2r2 (pgj(k)-Xi (k)) used to go near the vicinity of good local minima, and the

xi (k + 1) = xi (k) + vi (k + 1) (7) conjugate gradient schemes is used to find the local minimumaccurately. Next, this accurately computed local minimum is

where used as the global best solution in the PSO algorithm to iden-

Vjj (k) - the ]tl component of the velocity for particle i at tify still better local minima, and the cycle is repeated. In thisiteration k ~~~~~~~way the hybrid conjugate gradient -based PSO algorithm lo-

266

cates the global minimum by locating progressively better 3) Determine the fitness of each particlelocal minima. 4) Update the velocity and position via the proposed

In the new PSO algorithm, the balance between explora- hybrid PSO algorithmtion and exploitation is achieved by using the original PSO 5) Set the number of iterations and error. If the algo-algorithm without constriction coefficients or inertia weights rithm meets any one requirement, then end.for global exploration and a deterministic local search algo-rithm for accurate location of good local minima. This ap-proach is advantageous because it allows for exploration of The nonlinear system to be identified is as following:new regions of the search space while retaining the ability to y(t -l)y(t-2)(y(t-1) + 2.5)improve good solutions already found. y(t) - 1+ y(t 1)2 + y(t - 2)2 u(t-1) (10)

Specifically, the new proposed algorithm is as following:{ The input data u are uniformly distributed random num-Do for each particle { bers. GA and the new proposed hybrid PSO algorithm are

Randomly initialize Xi, Vi and Pi utilized to train the neural networks respectively. The maxi-mum number of iteration is 200, the evolution of training erroris as shown in Fig.2.

Let G = minf(Xi)L=GDo for each particle{

Modify velocity and position based on Pav I~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

and C according to the original PSO up-date rule:

v,j (k + 1) =v.j (k) + rlj/ (p/ (k) - xzj (k))+ '2 j (py,(k) 9x.k

xu (k +1)= x (k) + v. (k +1)if f(Xi) <f(Pi) then replace Pi with Xi v

if f x1) <f(G) then replace C with X

Do for NG iterations {Deterministic local search with C as

starting point} Iteration numberif f(G) <f(L) ,Then{ L = C } Fig.2 The evolution of training error via the proposed PSO algorithm and GA

Else I G L ~~~~~~~~~~~~~~~respectively. The proposed algorithm --Blue; GA--- red

} While maximum iteration or minimum error criteria is notreached The training results show that compared with GA , the

IV NONLNEARSSTEM IENTIFITION BSED ONproposed PSO algorithm converges more fast, stably and ac-curately in identifying nonlinear system.

RBFNN VIA HYBRID PSO ALGORITHM Let u(t) = sin(2;Tt / 25),According to the above analysis, the procedure of identi- x(t) =[y(t -1), y(t - 2), u(t -1)]

fying nonlinear system via RBFNN is as following:TaesmldtaetD Y Xkk=12..101o1) Determine the number of hidden layer nodes via Tk apedt e (,xkk=12. 2}tRPCCL automatically, which actually is the process test the trained model. The clustering number is 20, which isof determining the optimal clustering number of in- determined by RPCCL. Accordingly , the neural networksput data. RPCCL a new clustering method, which have three input nodes , 20 hidden nodes and one output node.performs clustering fast, accurately and automati- The number of iteration is 200. The identification results bycally. RPCCL has shown outstanding performance GA and the proposed PSO algorithm are as shown in Fig.3in determining the number of radial basic function and Fig.4.ofRBFNN in [9]. The simulation result via PSO algorithm fits the model

2)Wil Intaizem ithepraticle swrm.nmu muchr betteritha thtvaG,whrociaiotinliPeahe 0he 0trai =12.. }weeN stenm- ut in(2ng/5Itsu has aloshow nthat proposedwihybridth

ber ~~~ ~ ~ ~ ~ ~ ~~ ~pooe ofO particlesrgsmoefat salyad c

IV ONINARSYTE IENIFATIN ASD N urtey n denifin nnlner267em

PSO is more effective in identification of nonlinear system above problems. The proposed algorithm is used to identifybased on RBFNN. the parameters ofRBFNN. The results show that the new al-

The identification reulitvihthe proposed hiflid PSO alrithm gorithm is effective.output of RBFNNoutput of expectation REFERENCES

_ I J \ / \ I X 0 * [1] Meng Zu-qiang and Cai Zi-xing, "Identification method of nonlinearsystems based on parallel genetic algorithm" , Control and Decision

: [ \ 2003,18(3):367-374[2] R. C. Eberhart and Y. Shi, "Comparison between genetic algorithms and

particle swarm optimization ", Evolutionary Programming VII: Proc.2 7th Ann. Conf. Evolutionary Programming Conf., San Diego, CA. Ber-

3 / l \ \ lin: Springer-Verlag 1998I i \ t [3] Ding Hong-kai, "Study on Simulation of Nonlinear System Identifica-

tion Based on PSO-RBF NN ",Journal of system simulation, 2005,17- / /// l / l * (8) ,1826-829.

0,{ f'l / / / [4] J Kennedy, R Eberhart. Particle Swarm Optimization. Proc. IEEE Int.Conf. on Neural Networks [C]. 1995, 1942-1948.

_K14 | SJ % i ix [5] Shi Y. Eberhart RC. Parameter Selection in Particle Swarm Optimiza--1 ~~~~~~~~~~~~~~~~~tion.In Evolutionary Programming VII: Proc. EP98. New York

Springer-verlag,1998,591-600[6] Ozcan E,Mohan C, Particle Swarm Optimization : Surfing The Waves.

.2 l l ! l ! IIn: Proc, 1999 Congress on Evolutionary Piscataway, NJ: IEEE Serviceo20 40 60 so 100 120Discrettime Center, 1999,1939-1944

Fig. 3 The identification result via the proposed PSO algorithm [7] Clerc M, Kennedy J. The Particle Swarm: Explosion, Stability, andConvergence in Multi-Dimension Complex Space[J]. IEEE Transactionson Evolutionary Computation, 2002, 16(1): 58-73

The identification result Mi GA [8] Mathew M.Noel, "Explorations in swarm algorithms Hybrid particlej expectation output swarm optimization and adaptive culture model algorithms",[D],2005

output of RBFNN [9] Ohn, I, Ansari, N., "A novel algorithm to configure RBF networks"[C],4 K' IEEE Trans. Neural Networks, 1997 June 9-12, vol(3), 1809-1814

- I1-%lX1) AV1 l/>N IK >\ IJA1

0 /

a 20 40 Go SO 100 120Discret lime

Fig. 4 The identification result via GA algorithm

The simulation result via PSO algorithm fits the modelmuch better than that via GA, where activation signal isu(t) = sin(2;Tt / 25) It has also shown that proposed hybridPSO is more effective in identification of nonlinear systembased on RBFNN.

VI CONCLUNSIONS

Since the location of best solution is not known in ad-vance, proper choice of inertial factors or constriction coeffi-

cintis prbeai beas the nube of itrtin necessar to loat th bes souto is no knw apir, andP5alorth covre slwl in the lae pe iod A e hbicojuat grdin -P5 alorih is prpoe to sov h

1Vq1S 1$#fS IllJ X 1kSl 268