Soft Computing (Immune Networks)

In Artificial Intelligence Yasuhiko Dote

Muroran Institute of Technology Mizumoto 27-1, Muroran 050 8585,Japan

dote@,csse.muroran-it.ac.jp

ABSTRACT

iliis paper proposes a novel reactive distnbuted artificial ~ntcIiigeiice (dvnaniic)using immune networks and other soft Loiiiptitiiig inethods Fusth. extended sot? computing is defined

In .idding iiiuiiuiie networks and chaos theory including fractal and ivavelet to conventional sott computing which is the fusion or

coinbinatioii of tiizzv systenis.neural networks and genetic .~lgoritlinis and is suitable to cognitive distnbuted artificial

~ i i~c l l i~ence (static) Next, a novel fuuv neural net(genera1

parameter radial based function neural network) is developed in

order to use it for communication among agents in immune

iictnorhs The geiieral paraineter method is &ended to an adaptive structured genetic algorithm to obtain much faster convergence rate An unbiasedness criterion using distorter( a

radial based ftiiiction network in order to optimize parameters

resultiiy i n die reactive distributed artificial intelligence hnd of

(;MD!i) is applied to better generalization propertes. Then, t h s developed I'tvrv neural net is extended to a h g h performance

1. INTRODUCTION I<eactivit\ is a hehavior-bewd model of activitv,as opposed to

the svmbol inanipulation model u.wd in planning.This leads to the

iiotioii of cognitive c0st.i e.. the complexity of the over ,Irchitecture needed to achieve a task Cognitive agents support a coinplz\ architecture which inems that their cognitive cost is IiigIi.Copnitive agents have intenml representation of the world \ \ l i i ~ l i iiiiist he i n adequation with the morld itself The process of rslating the tiitenial representation and the world is considered as

'I ~oinple\- task On the other hand. reactive agents are simple. cash to uiiderstwd and do not support intemal representation of the world. Ilius. their cognitive cost is low, and tend to what is

~ a l l e d cognitive economy. the property of being able to perfom cvcn compls\r actions with simple architectures Because of their

complexit\. cognitive agents are otteii considered as self- ~iitlicicrit the\ can nork alone or nith a ten other agents.0n die c o i i t r m . reackive agcnts need companionshp 'I'hey can iiot work isolated and they usually achieve their tasks in groups. Reactive agents me companionship. They can not work isolated and thev

0-7803-4778-1 198 $10.00 0 1998 IEEE ,

usualh achieve theu tasks i n groups. llcactive a y i t s are sittiatcd

they do not take past e~eiits into account. a i d can iiot Ibrcsee rlic

ftiture. Their action is based on \\hat happens no\\. ho\\ the\ ~ C I I ~ ~

distmzuish situations ui Ilie aorld. on the ~ \ a v thev resognve world indexes and react accordingly llius. reacuve agents can not

plan ahead what they will do But, what can be considered as a

weakness is one of theu strengths because the! do not ha\^ to

revise their world model when perturbations chaiige the \\orld i n

an tme\pected u a ) Robustness and tatilt toleraiict arc tno 01 the

main properties of reactive agent swttiiis. j 2 group of r e ; ~ c t ~ w agents can coinplete tasks even when one o l them b r d s doun. The loss of one agent does iiot prohibit the coinpietion o l the whole task, because allocation of roles is achieved locall\ bv

perception of the enviroiunental needs. 'Thus, reactive ageiit

svstems are considered as ve: flexible and adaptive because[ I 1 In this paper ;I no\zcaI re;ictive distributrt l ;irtif'i(,i:il

int r llige nrr is proposvd us1 t i g Ish igrii ro's in i i i i i i ne

nr twork ;11~[~r(~~1t~Iil'l;in'li:(i : i nc l ot1ic.r +oft rwmputin?

approaches In section 11. soft coinpiit ing propoawl I)? L1r.L . i .Zadrh[-i] is r x t r n t l t d by :Ititling rhaos coinpiti ing

and iinmunr net,work theory. i\ novel fuzzy neural nrhvork with grneral p;lr;imrt.c-r statistics calculus taking advantages of both fiii.z\ ins arid neural iictnorhs i n section

111 In section IV this IS eaciided to a high perfonnoiice radial hasis tiiiiction iieural iitt\\urh using oii adaptive structure genetic

algorithinl5 I close to the seneral parciiiieter iiicthod arid o kind 01'

(iMnH[61. In section V these developed nct\\orks are applied to

optiiiiize Ishiguro's uimiune network reactive distributed artiticial intelligence.

x

11. EXTENDED SOFT COMPUTING Soft coiiiputing is proposed bv I)r I* A./.adch/-ll to coiistrtict

ne\\ generation Ai [ macliiiie .intelligeiicc quatient) and to solve

noiiliiiear and inatliematicallv iiimioJelld systems prohlenis

(tractability) especiallv for cognitive artilicial intelligence In this

section by adding chaos coiiiptituig arid iiiuiitiiie network thron.

1382

ilii extended soft computing is detined for explaining, what thev

call. complex svstems(7). hunune networks are promising

approachos to construct reactive artiticial intelligence[21 and [ 3 ]as Illustnltcd 111 Fig I

Hiinian I~eir ig l i k e .AI Cognir iw

Fuzzy 1)istribur~d

~~-

Systern ;\I (Stnticl

Fig.1 Soft computing in AI

111. NOVEL FUZZY NEURAL NET

I irst. id consider the (iP approach to KHFN weights adjust~ng. ;\s soon iis IIRI,'N IS linear on its \\eights. the (iP method may bo impIementc.d in a straightfonxard manner The equation

dcscribiiig (if'-RBFN for a single output network is

U IirrI, 11.: fisrcl init ial va lues of' network we1ght.s: p :

si:;il;ir grnrr;il p a r a m e t e r t,o be adjusted with t h e

fol I ow 1 ng algori t hin

s t a r t i n g from the same initi;il conrhtions. In t h e first

case. :I coiivcsiit ioniil I2HF" wiis uswl w i t h in~livi~Iu;i l

adjusting of i t s \vciights. I n t h e s;c.c:ontl c:ise. (~ l ' - l i l$ l~N w a s simuliitrtl with leiirning iilgorit hili (2) 'l'h~, vfficirnc? of both mrthorls w e r r ~ o n i i ~ ~ r r d by using

t h e following m r a s u r e of convergrncr sprrd

I ' J

Figure2. The simplest GP-RRFN

0' 400 800 1

Figure 3 Im;trning algorithm c:onvt!rgc!nc:o:

t i ) conventional IZUCN: 11) [;I'-ltt3FN

1383

~linic.nsioniilit> IeiirninC sprrtl of (;P-RBFS hits iiicreasril reliitivelj- convent.iona1 RBFN. IitlFN to be used in adaptive fuzzy system (AFS) . in

comnion case. is assumed to be t.ririned by means of t h e iiiiniiiiuiii necessary nunibrr of rules (hidden unit

number) ilc.trrniin:ition a n d adjusting of t h e mean and

v;iri:incc~ vwtorh of' iiithvidu:il hidd[,n nodes a s well as

thrJir \\eight5 In th i s p:iper. t he simplest CP RBFN li;isc~tl :idapt ive fuzzy system for aut.oinatic fuzz!- rule niiinbrr tletemiination is proposed (Fig4). Only t h e nrtworli weights have been assumed t.o be adjust.ed by the' (:P algorithm while t he c rn t r e s a n d widt.hs of unit +nsit I V P zon6.s \yere ooiiipletel>- tleteriiiined wi th the n( , tworl i input Gign;il r:iiigr :inil u n i t

1

, CP KBFN BASED AFS - - ___

Figure4. G P RBFN adapt ive furLy system

nuiiihrr during riich t ra ining rpoch

.\ "s:iniplr" fuzz!- systrm has been present.ed by RBFN U i t h I he "unknown" number of hidden uni ts (i.e.. fuzzy

rules) Star t ing t'rom the single-unit-(;P-RBFN. the nr.twork learning has been performed by the scii1:ir grneriil piiriiiiieter iirljusting in the Learning l ' r ~ ~ c c ~ ~ l u r ~ ~ blorli The stratly st iitr general parameter

( ~ ~ ~ I T I : i t ion f < [ f l \ ; i n d viiriiince D { P ) have been

c,:ilcul:ittd hy GP Statist,ics Estimnt.or. The ;~pproxiniation quality cnter ion (1B) was evalutit.etl for ( h p current (:P KRFN st . ructurr . rind decision on rh;inging of' nrtwork s t ructure p:ir:iiiirter iicljusting

i i i t I I P 1,riiriiing Prow[iurr, L~lock. The stezicly s ta le

gr~iii~riil p:ir:iniet<Jr ~~spec t ; i t i on E[P}antl

viiriiince U(P: have been c;ilculatrd by GP Stat.istics

l%timwt.or. The approx" t . ion quality crit.enon (1:3)

\viis n ~ : i I i i ; i t c d fort he current (:P RBFN st.ructure. and

D(P: Q=-

Thereforr. the C:P KHFNAFS Jetc,rniines I h c , " t ru r "

fuzz! rulrj number b; incrt~iiir~nt;lli! rwrui i iny: I 1 1 ~

r ak i i l basis fuiiction uni ts ant1 cant inuous est in i i i t ion

of t.he approxlmtition quality through critrriii (4)

evwluat.ion for each fixed GP R B F S structure . The network t o be determined is the network with 1r:ist

v:ilue oi' i, anr! its unit n u m l ~ r r I C :issiinic,~l t o h.

c l i t * " s : l n i l i l t ~ " r,qu;il t o t he f'uzzy r d c ~ nuiii lwr C I ~

i'uzz> .ystc"

Let consider t he proposed procrdure i n c1et.d for r h r siiiiplest case of the (:P RBFN AFS Lvith sciil;rr input

signal

i n p u t slgniil I I (E ' :u : =0) iintl lino\vn nuni1ii.r of

(:aussian uni ts r] (for t he first stage. y = I ) t h r

sensitive zone center coor&n;ites :ire calculiitcd by relationship (5).

whrrr. I is i i current unit number For y = I ;incl I = I

for rsiiiiiplr. onr ['tin recrivr (': = 0

3 ) The initial (basic) sensitive zone w i d t h rqu;il i'or all netu.orli uni ts I:, c;ilcul;itt.tl as ((5)

1384

IC; p r t o r n i e d biised on input-out,put sample d;itii In this section. the 1Jnbiasediiess Criterion tisiiig Distorter I I K I ) ) ;icwrtlingly to the ;iIgorithm ( 2 ) . Simiiltmeousl>- t h r approach( 8 I is used. which has been sho\\n provldlng iiiiproved

features i n coiiipare to conveiitioiial methods. such as ~ k a i k e

Infomiation Cntenon ( A I C ) [ 9 ] and its modification for neural networks Network Infonnation Criterion (MC) [ I O ] , f i n l n i u m

Descnption Length (MDL)[ I I].

gc'nrr;il p;ir:iiii~ter iJspwt ;it ion E { P ) and viiriiince

D[,& :ire es t imated with some conventional method.

for rs-ample. by t h e movlng average calculation. Let consider the IJCD method application to the GP RBFN AFS The overall svstein block diagram IS shown ui Fig. 6.5

Both of them are (iP RBFN with a lemiing procedurs llie same signals are ted uito the network inputs The diiYerelici: I \ 111

the u a y of the teaclung signal usage While the reaching signill is

fed mto uppa loop without any changes, the lower iietuork is

trained by "distorted", i.e. nonliiirarly traiisfonned, sample d ~ t a

The output of the lower network is also changed hv the

transfoniier of the same transfer function as fir teachins sgiitl

The critenon ol' the iietuork structure optimality is derivedI61.

nhich IS otthe tonnc 7) :1 0 6

%ax (*,; (:2 - 0-. C: umax I .( 'D = 5 (U ) - I.-? (7 ) ]

/ = I

( 7 ) IJiguref,. Definition of GP RBFN basic parameters

where ' ' j-th set (vector) ofthe network input data. 17 overall

c.v;iIii,ii N I : i i i ( I iiiemorizrtl. variables of the both networks. Tlie structure of the netnork n i th

the least value of the cntenon 7 1 is assumed to be a soliition ot the problem

- , - - - - ':I ,- ' 5 ... \ : , ,_ :,! \ : ... '

8 ) The strucciirr of GP RHFN is modified by one inore '. - . . . [:iiussiaii un i t recruiting: y = q + l . The st.eps 1) - 6)

: I r i a rvl)i.at 6.i I .Y , .VI

The, r rs i i l t of the algorif hni 1 ) - 8) imp1ement;ition is :I

111 I'uiivtion uni ts i n c:P IiBFS $1 h i ( . h provic1c.s I he best :ipproxiniating accur:icy In the

car ti is of' fuzxy system theory i t iiieiiiis t h e fuzzy rule ii ii m1)c.r clrtc~rminiition problriii solution. [8]

1V 1 IJ(il.1 PERFOIWANCE RI3k.N l'he prohiein of the reliahiliiy n1' the denved model is one of thc

iiiost iiiipottaiit ones. ansing duruig the identitication task solving

Hic model over-titting prevention IS a crucial point tor inam

y l c t i c a l iinplrmeiitations 11s i t WJS discussed ui the preceding sections, there are several approaches to cope with this ditficultv

Fig.6 Determinution of number of units by dibtorter The proposed general paaiiieter method in scctioii Ill I,

again illustrated i n Fig.7.. This idear is extended to aii adaptive structure genetic algonthm[j]. Geiiotvpe has an adaptive

structure . The string representation is constructed by two l a y s

One is nanied locus l a y . the other .operon l+er as slio!!ii i i i

IFig 8 For this reprcseiitatioii .live ne\\ genetic o~)er~i~i~ii is iirs

detined in order to scll~orgaiii/t: the siring itriicture and dsvclo1)

adaptive genctic change 111 the evolutioiial pro approach bnngs attractive optiiiiiimoii results fbr probizins

including (iA-dilticultv.Suice genetic algorithm and chaos

1385

Loinputnips are heuristic approaches, they have capabilities of a creative thinking ivav or evolution

H i these techniques the Iuzzv neural net in section III turns Into <I high pcrlbnnancr radial basis fuiictlon neural network

Fig.7 General parameter method

S t r i n g

fashion.Namelv.onlv one antibodv is allowved lo activate and act

its corresponding its action to the ivorld 11' its coiiceiitratioii

surpasses the prespecitied the threshhold As sho\vii i n Fig10 . ilic

concentration of the aiitibodv is influenced b! the stimulatioii iuid

suppression from other antibodies . the stiiiiulation froin antigeii. and the dissipation Factor t i c. natural death ). The concentration 01

I-th antibody .which is denoted by a, . is calculated b! ( X ) ( I and

0 are the rate of interaction ainong antigens and antibodird.

+. ..... .... +

~ a l u e list t i x e d lenzth .~

. . . . . . . . _ - ~ _ . . _ ~ ~ - ~ ............ General Parameter Locur libel V V V

.~ ......

................. ....................

N!:eight layer (fixed nominal value) - ....... -. __ . .- -- / I ; , ... Ili,, ......... li:, II, ... It,;,"

_- r

-.: * .. ~

.-- -.; ._. . ~

............ -~ . ? -- ~ ~-~ *. i ~ _ _ _ :-

Inputs : blutually

1

Inputs : Mutually ' Correlated - - _ _ - I Correlated

... - .... I - .....

Fig.8 Adaptive string structure o f genetic algorithm

V. SOFT COMPUTLNG IN REACTIVE DISTRIBUTED ARTIFICIAL

INTELLIGENCE \ Is1 l l G [ J R O 3 REACTIVE IIISTRIBIJTED ARTIFICIAL

IN'TEI.Ll(;ENCE WITH MMJNE NETWORKS[Z] and [i] i 'he detected current situation and competence modules as .\iitigciis and Antibod~es,respzctiveI~ lo inake a iinonoido(antihody) select a suitable antibodv against ilw wrreiit antigen, it IS highlv important I i o ~ the antibodies

arc described .Moreover.it is noticed that the unmunogical

dntration inecliamsm select an antibody in bottom up manner by ~ommuiiicating aiiioiig the antibodies. To rwlize the above

rcquireineiits. the descnptioii the description of antibodies are

defined as follons The identitv of a specific antibody is generally ilcleniiinzd h? the stncture of its paratope and idiotope F i g 5 dcplcts thc represetitation of antibodies As shown iii this tigure.a

pair of precondition action t o paratope .the nuinher of l l~wllo\\rd antibodies and thc degrce ot' disallowance to idiotope

,irc respectively assigned In addition, the structure of paratope is J I \ ided into four portions: objects, direction,distance, and action.

For adequate selection of antibodies . one state variable called

concentration is assigned to each antibody. The selection of

;Ilitibodics I S simply carried out in a wiimer-take a lb

N N N tlA,(tvdt=( (L ( XI11 i l ( 1 ) XI11 ) n i Llll .<I. ( 1 )

J - I 1 1 k 1

I X IN:, - 0 111: ~ k. ii: ( t)

il. ( t - I ) -1.. (l.rxp(O. 5 - A . ( t ) ) )

(8)

k = I

\\liere N IS die number of antibodies. a i d nil denotc~ inatclinis

ratio hrtneen antibod! I and antigen .m), that denotes dcgrce 01

disallo\\ance of antibod\ I for antibod! I 'The first and sccond

tenns of nght hand side denote the stiiiiulatioii and supprzssioti

from other antibodies, respectively The thrd tenii represents lhr

stimulation from antigen, and the forth tenn thtl natural death . . ~ _ _ _ ~ - .

Idiotour - 7 z E E D -~

. . . ~ ~~

Food Bark Middle H w k w u d Obsmclr I v t l FW KlEhi EnrrgY and c , r . i.cn ni>d et,' . - _ ~

Fig.9 Represent;rtion of antibodies

1386

hi order to optimize this reactive distributed artificial intelligence.

h e deve1opr:d ftiziv neural net is applied to communication

aiiioiig agents( antigens and antibodies ) The developed radial

hasis function neural net is used to optimize parameters in (8) and lbr a inetadyaniics whch produces and removes antigens and ailtibodies to make reactive tables.[f]

VL. CONCLUSION 1111s paper proposes extaidtxl sott computing to construct 10%

cos^ reactive distrihuted artificial intelligence resutmg in excellent decision iiiahng. Table I shows the comparison of the proposed system vvith fuzzy svstems on decision making.

Tirblel Comparison of immune network- based with fuzzy reiisoning approach

Iiiiiiiuiic iietnork-bawd T'wn reasoning

t3ottoiii-up decentralized Top-dow~ centralized IIsplicit uiteraction Implicit interaction 1)viiamir: static

REFERENCES

_1 lcrhcr."Reactive I)istnhwed Arti ticial

Intt.lli~eiice.Principles and Applications".Chapter

I I .I:oundations of Distnbuted Artificial

hitelligence,cdited bv GM.P.O'harc and N. R.Jemngs,John Wilev&Sons Innc.,New York, 1 9 9 6 . ~ ~ 2 8 7 - 3 14.

A.Ishiguro.T.Kondo.Y.Watanabe and Y.lJchka\va."ki Iniiiiunogical Approach to Behavior Control of .~lutoiioimious Mobile Ilobots-Coiistructioii Immune Netuorks Through Leanimg--'.Proceedmgs of the

hitexnational Workshop on Solt Computing in

Industry( IWSCI'96),Muroran,Jap~i.April17-28.l996,pp 253-267.

A 1shiguro.Y Watanahe.'l:Kondo and Y I Ichil;a\\a."Constrctioii of a Decentralized

Consensus-Maklng Netaork Based on the Inunune

S! stem-Application lo Action Arbitration for an Autonornous Mobile Robot-",The SlCE

Trans. .Vol.33,No.h. I097,pp 524-5.32(inJapanese). I. A %adeh?The Role of Soti Computing mid Fuzzv

Logic iii the Conception.Design. Development of 1111211 igeiit Svstems".€'roc Of the

IWSC I'Oh.Mtiroraii.Japaii.ApnI27-ZX. I990 PP I .3b-

I .37. (I'leiian: Speaker)

IC Ohkura and K.11eda..'Srlf-Orgaiii/;ing of Stnng Structure arid Adaptive < imetic Swrch".Proceedings of the IWSC 1'06 pp 172- I77

t r TaLeuclu and T Mpos1ii.H Ishihashi and H.Tanaka,"A

Heunstic Model Selection Cnterion I king Distorter and

Its Application to Detenmiumatioii of the Nuinher oI

Hidden IJIUIS in RBFN', .louiial o t rhr: .lap Soc 01'

Syt.Contr. and Inf.,Vol Il,N0.2,l99X.pp6 1-70 Y.Dote,"Sott Coniputmg( Immune Networks) 111

Artificial Intelligence". Web.site:http-//bik.csse Muroraim.Japan. I997 D FhE;hntetov.Y.Dote and M S ShaiMi."S\striii

Identilicetion bv the (iciieral l'urumeier N e t d Netuorks nith Fuzzy self-or~anizaiion"f'rep. o t the I I " '

IFAC SVmP on Svsrelll IdentiIication,Kitak~shu,Japaii,Vol.2, I 997.~~829-8.34 H.Al;aike."A New Look at the Statistical Model

Ideiiti!ication".IEEE Tran. On AC.Vol 19.I974.pp71b

72 3

M.Murata.S Yosluka\va uid S.Aiiian."Nt.r\~orL

Infonnation Cntenoii-l)eieniuiuimg die Nuinher ol'

Hidden IJiUts for Anilicial Neural Nelnork

Model".IEEE Tran. on Neural Net,Vol.j,No.j, I994,pp865-872. J kssanen,"A IJniversal Prior tor Integers and

Estimation bv M " u m Descriptloii I .engtIiC. Annals 01'

Statistics.Vol I I.No 12.l9X3.pp4l(~-i.~l

1387