Dyna mi cs of Comple x Sys t e ms# 29412 Cust: AddisonWesley Au: Bar-Yam Pg. No. iTitle: Dynamics Complex SystemsShor t / Normal / LongFMadBARYAM_29412 3/10/02 11:05 AM Page i# 29412 Cust: AddisonWesley Au: Bar-Yam Pg. No. iiTitle: Dynamics Complex SystemsShor t / Normal / LongSt udi e s i n Nonli ne a ri t ySer ies Editor : Rober t L. DevaneyRa lph Abra h a m, Dyn a mics: Th e Ge o me t ry of Be h a vio rRa lph H. Abra h a m a nd Ch rist oph e r D. Sh a w, Dyn a mics: Th e Ge o me t ry ofBe h a vio rRobe rt L. De va ne y, Ch a o s, Fra ct a ls, a n d Dyn a mics: Co mp u t e r Exp e rime n t sin Ma t h e ma t icsRobe rt L. De va ne y, A First Co u rse in Ch a o t ic Dyn a mica l Syst e ms: Th e o rya n d Exp e rime n tRobe rt L. De va ne y, An I n t ro d u ct io n t o Ch a o t ic Dyn a mica l Syst e ms, Se co n dEd it io nRobe rt L. De va ne y, J a me s F. Ge orge s, De lbe rt L. J oh nson , Ch a o t icDyn a mica l Syst e ms Soft wa reGe ra ld A. Edga r ( e d. ) , Cla ssics o n Fra ct a lsJ a me s Ge orge s, De l J oh nson , a nd Robe rt L. De va ne y, Dyn a mica l Syst e msSo ft wa reMich a e l McGuire, An Eye fo r Fra ct a lsSt e ve n H. St roga t z, No n lin e a r Dyn a mics a n d Ch a o s: Wit h Ap p lica t io n s t oPh ysics, Bio lo gy, Ch e mist ry, a n d En gin e e rin gNich ola s B. Tufilla ro, Tyle r Abbot t , a n d J e re mia h Re illy, An Exp e rime n t a lAp p ro a ch t o No n lin e a r Dyn a mics a n d Ch a o sFMadBARYAM_29412 3/10/02 11:05 AM Page iiYaneer Bar-YamDyna mi cs ofComple x Sys t e msThe Advanced Book ProgramAddison-WesleyReading, Massachusetts# 29412 Cust: AddisonWesley Au: Bar-Yam Pg. No. iiiTitle: Dynamics Complex SystemsShor t / Normal / LongvwwFMadBARYAM_29412 3/10/02 11:05 AM Page iiiFigure 2.4.1 1992 Benjamin Cummings, from E. N. Marieb/Human Anatomy andPhysiology. Used with permission.Figure 7.1.1 (bottom) by Br ad Smith, Elwood Linney, and the Center for In Vivo Microscopyat Duke Universit y (A National Center for Research Resources, NIH). Used with per mission.Many of the designations used by manufacturers and sellers to dist inguish their products areclaimed as tr ademarks. Where those designations appear in this book and Addison-Wesleywas aware of a tr ademar k claim, the designations have been printed in initial capital letters.Library of Congress Cataloging-in-Publication DataBar-Yam, Yaneer.Dynamics of complex systems / Yaneer Bar-Yam.p. cm.Includes index.ISBN 0-201-55748-71. Biomathematics. 2. System theor y. I. Title.QH323.5.B358 1997570' .15' 1DC21 96-52033CIPCopyr ight 1997 by Yaneer Bar-YamAll rights reser ved. No par t of this publication may be reproduced, stored in a retr ieval sys-tem, or transmitted, in any form or by any means, electronic, mechanical, photocopying,recording, or otherwise, without the pr ior written permission of the publisher. Pr inted in theUnited States of America.Addison-Wesley is an imprint of Addison Wesley Longman, Inc.Cover design by Suzanne Heiser and Yaneer Bar-YamText design by Jean HammondSet in 10/12.5 Minion by Carlisle Communications, LTD1 2 3 4 5 6 7 8 9MA0100999897First printing, August 1997Find us on the World Wide Web athtt p://www.aw.com/gb/# 29412 Cust: AddisonWesley Au: Bar-Yam Pg. No. ivTitle: Dynamics Complex SystemsShor t / Normal / LongFMadBARYAM_29412 3/10/02 11:05 AM Page ivThis book is dedicated with love to my familyZvi, Miriam, Aureet and SageetNaomi and our children Shlomiya, Yavni, Maayan and TaeerAureets memory is a blessing.# 29412 Cust: AddisonWesley Au: Bar-Yam Pg. No. vTitle: Dynamics Complex SystemsShor t / Normal / LongFMadBARYAM_29412 3/10/02 11:05 AM Page v# 29412 Cust: AddisonWesley Au: Bar-Yam Pg. No. viTitle: Dynamics Complex SystemsShor t / Normal / LongFMadBARYAM_29412 3/10/02 11:05 AM Page vivii# 29412 Cust: AddisonWesley Au: Bar-Yam Pg. No. viiTitle: Dynamics Complex SystemsShor t / Normal / LongCont e nt sPre fa ce xiAcknowle dgme nt s xv0 Ove rvi e w: The Dyna mi cs of Complex Sys t e ms Exa mple s,Que s t i ons, Me t hods a nd Conce pt s 10. 1 Th e Fie ld of Complex Syst e ms 10. 2 Exa mple s 20. 3 Que st ions 60. 4 Me t hods 80. 5 Conce pt s: Eme rge n ce a nd Complexit y 90. 6 For t he I n st ruct or 141 I nt roduct i on a nd Pre li mi na ri e s 161. 1 It e ra t ive Ma ps ( a n d Ch a os) 191. 2 St och a st ic I t e ra t ive Ma ps 381. 3 Th e rmodyna mics a nd St a t ist ica l Me ch a n ics 581. 4 Act iva t e d Proce sse s ( a nd Gla sse s) 951. 5 Ce llula r Aut oma t a 1121. 6 St a t ist ica l Fie lds 1451. 7 Comput e r Simula t ions ( Mon t e Ca rlo, Simula t e d An ne a ling) 1861. 8 I n forma t ion 2141. 9 Comput a t ion 2351. 10 Fra ct a ls, Sca lin g a nd Re norma liza t ion 2582 Ne ura l Ne t works I : Subdi vi s i on a nd Hi e ra rchy 2952. 1 Ne ura l Ne t works: Bra in a n d Mind 2962. 2 At t ra ct or Ne t works 300FMadBARYAM_29412 3/10/02 11:05 AM Page vii2. 3 Fe e dforwa rd Ne t works 3222. 4 Subdivide d Ne ura l Ne t works 3282. 5 An a lysis a nd Simula t ions of Subdivide d Ne t works 3452. 6 From Subdivision t o Hie ra rch y 3642. 7 Subdivision a s a Ge ne ra l Phe nome non 3663 Ne ura l Ne t works I I : Mode ls of Mi nd 3713. 1 Sle e p a nd Subdivision Tra in ing 3723. 2 Bra in Funct ion a n d Mode ls of Mind 3934 Prot e i n Foldi ng I : Si ze Sca li ng of Ti me 4204. 1 Th e Prot e in - Folding Proble m 4214. 2 I nt roduct ion t o t h e Mode ls 4274. 3 Pa ra lle l Proce ssing in a Two- Spin Mode l 4324. 4 Homoge n e ous Syst e ms 4354. 5 I n homoge n e ous Syst e ms 4584. 6 Conclusions 4715 Prot e i n Foldi ng I I : Ki ne t i c Pa t hwa ys 4725. 1 Ph a se Spa ce Ch a n ne ls a s Kine t ic Pa t h wa ys 4735. 2 Polyme r Dyn a mics: Sca lin g The ory 4775. 3 Polyme r Dyn a mics: Simula t ions 4885. 4 Polyme r Colla pse 5036 Li f e I : Evolut i onOri gi n of Complex Orga ni s ms 5286. 1 Livin g Orga n isms a nd En viron me nt s 5296. 2 Evolut ion Th e ory a nd Ph e nome nology 5316. 3 Ge nome, Phe nome a n d Fit ne ss 5426. 4 Explora t ion , Opt imiza t ion a nd Popula t ion I nt e ra ct ion s 5506. 5 Re product ion a n d Se le ct ion by Re source s a nd Pre da t ors 5766. 6 Colle ct ive Evolut ion : Ge n e s, Orga n isms a nd Popula t ions 6046. 7 Conclusions 619viii C o n t e n t s# 29412 Cust: AddisonWesley Au: Bar-Yam Pg. No. viiiTitle: Dynamics Complex SystemsShor t / Normal / LongFMadBARYAM_29412 3/10/02 11:05 AM Page viii7 Li f e I I : Deve lopme nt a l Bi ologyComplex by De s i gn 6217. 1 De ve lopme n t a l Biology: Progra mmin g a Brick 6217. 2 Diffe re nt ia t ion : Pa t t e rns in An ima l Colors 6267. 3 De ve lopme nt a l Tool Kit 6787. 4 Th e ory, Ma t he ma t ica l Mode ling a nd Biology 6887. 5 Principle s of Se lf- Orga n iza t ion a s Orga n iza t ion by De sign 6917. 6 Pa t t e rn Forma t ion a nd Evolut ion 6958 Huma n Ci vi li za t i on I : De f i ni ng Complexi t y 6998. 1 Mot iva t ion 6998. 2 Comple xit y of Ma t he ma t ica l Mode ls 7058. 3 Comple xit y of Physica l Syst e ms 7168. 4 Comple xit y Est ima t ion 7599 Huma n Ci vi li za t i on I I : A Complex( i t y) Tra ns i t i on 7829. 1 I nt roduct ion : Complex Syst e ms a n d Socia l Policy 7839. 2 I nside a Comple x Syst e m 7889. 3 Is Huma n Civiliza t ion a Complex Syst e m? 7919. 4 Towa rd a Ne t worke d Globa l Econ omy 7969. 5 Conse que nce s of a Tra n sit ion in Complexit y 8159. 6 Civiliza t ion I t se lf 822Addi t i ona l Re a di ngs 827I ndex 839C o n t e n t s ix# 29412 Cust: AddisonWesley Au: Bar-Yam Pg. No. ixTitle: Dynamics Complex SystemsShor t / Normal / LongFMadBARYAM_29412 3/10/02 11:05 AM Page ix# 29412 Cust: AddisonWesley Au: Bar-Yam Pg. No. xTitle: Dynamics Complex SystemsShor t / Normal / LongFMadBARYAM_29412 3/10/02 11:05 AM Page x# 29412 Cust: AddisonWesley Au: Bar-Yam Pg. No. xiTitle: Dynamics Complex SystemsShor t / Normal / LongPre fa ceCom p l ex is a word of t he t i m e s , as in the of ten - qu o ted growing com p l ex i t y ofl i fe . S c i en ce has begun t o tr y to under st and com p l ex i t y in natu re , a co u n terpoint tothe trad i ti onal scien tific obj ect ive of u n derstanding t he fundamental simplicity ofl aws of n a tu re . It is bel i eved ,h owever, that even in t he stu dy of com p l ex i t y ther e ex-ist simple and therefor e com preh en s i ble laws . The field of s tu dy of com p l ex sys tem sholds that t he dynamics of com p l ex sys t ems are fo u n ded on universal pr inciples thatm ay be used to de s c ri be dispara te probl ems ra n ging from part i cle physics t o the eco-n omics of s oc i et i e s . A coro ll a ry is that tr a n s fer r ing ideas and re sult s from inve s ti ga-tors in hitherto dispara te areas wi ll cro s s - fer ti l i ze and lead to impor tant new re su l t s .In this text we introduce sever al of the problems of science that embody the con-cept o f complex dynamical systems. Each is an active area of research that is at theforefront of science.Our presentation does not tr y to provide a compr ehensive reviewof the research literature available in each area. Instead we use each problem as an op-por tunity for discussing fundamental issues that are shared among all areas and there-fore can be said to unify the study of complex syst ems.We do not expect it to be possible to provide a succinct definition of a complexsystem. Instead, we give examples of such systems and provide the elements of a def-inition. It is helpful to begin by describing some of the att ributes that char acter izecomplex syst ems. Complex syst ems contain a large number of mutually int er actingparts. Even a few inter acting objects can behave in complex ways. However, the com-plex systems that we are interested in have more than just a few parts. And yet there isgener ally a limit to the number of parts that we are interested in. If there are too manyparts, even if these parts are strongly interacting, the propert ies of the system becomethe domain of conventional ther modynamicsa unifor m mater ial.Thus far we have d efined complex syst ems as being within the mesoscopic d o-maincontaining more than a few, and less than too many parts.However, the meso-scopic regime describes any physical system on a par ticular length scale,and this is toobr oad a definition for our purposes. Another character istic of most complex dynam-ical systems is that they are in some sense purposive. This means that the dynamics ofthe syst em has a d efinable objective or function. There oft en is some sense in whichthe syst ems are engineered. We address this t opic dir ectly when we discuss and con-tr ast self-organization and organization by design.A centr al goal of this text is to develop models and modeling techniques that areuseful when applied to all complex systems. For this we will adopt both analytic toolsand computer simulation. Among the analyt ic t echniques are statistical mechanicsand stochastic dynamics. Among the computer simulation techniques are cellular au-tomata and Monte Carlo. Since analytic treatments do not yield complete theories ofcomplex systems, computer simulations play a key role in our understanding of howthese systems work.The human br ain is an important example of a complex system for med out of itscomponent neurons. Computers might similarly be understood as complex inter act-ing systems of tr ansistor s.Our brains are well suited for under standing complex sys-xiFMadBARYAM_29412 3/10/02 11:05 AM Page xixii P re f a c e# 29412 Cust: AddisonWesley Au: Bar-Yam Pg. No. xiiTitle: Dynamics Complex SystemsShor t / Normal / Longtems, but not for simulating them. Why are computers bett er suited to simulations ofcomplex systems? One could point to the need for precision that is the t raditional do-main of the computer. However, a better reason would be the difficulty the brain hasin keeping tr ack of many and ar bitr ary interact ing objects or eventswe can typicallyremember seven independent pieces of information at once. The reasons for this arean impor tant part of the design of the brain that make it power ful for other purposes.The architect ure of the brain will be discussed beginning in Chapter 2.The study o f the dynamics of complex syst ems creates a host o f new int erdisci-plinary fields. It not only breaks down barr iers between physics, chemistr y and biol-ogy, but also between these disciplines and the so-called soft sciences of psychology,sociology, economics,and anthropology. As this breakdown occurs it becomes neces-sary to introduce or adopt a new vocabular y. Included in this new vocabulary arewords that have been considered taboo in one area while being extensively used in an-other. These must be adopted and adapted to make them part of the interdisciplinar ydiscourse. One example is the word mind. While the field of biology studies thebrain,the field o f psychology considers the mind. However, as the study of neural net-works progresses,it is anticipat ed that the funct ion of the neural network will becomeident ified with the concept of mind.An o t h er area in wh i ch scien ce has trad i ti on a lly been mute is in the con cept of m e a n-ing or purpo s e . The field of s c i en ce trad i ti on a lly has no con cept of va lues or va lu a ti on .Its obj ective is to de s c ri be natu ral ph en om ena wi t h o ut assigning po s i tive or nega tivecon n o t a ti on to the de s c ri pti on . However, the de s c ri pti on of com p l ex sys tems requ i res an o ti on of p u r po s e ,s i n ce the sys tems are gen era lly purpo s ive . Within the con text of p u r-pose there may be a con cept of va lue and va lu a ti on . If , as we wi ll attem pt to do, we ad-d ress soc i ety or civi l i z a ti on as a com p l ex sys tem and iden tify its purpo s e ,t h en va lue andva lu a ti on may also become a con cept that attains scien tific sign i f i c a n ce . Th ere are evenf u rt h er po s s i bi l i ties of i den ti f ying va lu e ,s i n ce the ver y con cept of com p l ex i ty all ows usto iden tify va lue with com p l ex i t y thro u gh its difficult y of rep l acem en t . As is usual wi t ha ny scien t ific adva n ce ,t h ere are both dangers and opportu n i ties with su ch devel opm en t s .Finally, it is curious that the origin and fate of the uni verse has become an ac-cepted subject of scientific discoursecosmology and the big bang theor ywhile thefate of humankind is gener ally the subject of religion and science fiction. There ar eexceptions to this rule, particular ly surrounding the field o f ecologylimits to pop-ulation growth, global warminghowever, this is only a limited selection o f topicsthat could be addressed. Over coming this limitation may be only a matter of havingthe appropr iate tools. Developing the t ools to address questions about the dynamicsof human civilization is ap propr iate within the study of complex syst ems. It shouldalso be recognized that as science expands to address these issues, science itself willchange as it redefines and changes other fields.Different fields are often distinguished more by the t ype o f questions they askthan the syst ems they study. A significant effor t has been made in this t ext to ar ticu-late questions, though not always to p rovide complete answers, since questions thatdefine the field of complex syst ems will inspire more p rogress than answers at thisearly stage in the development of the field.FMadBARYAM_29412 3/10/02 11:05 AM Page xiiLike other fields, the field of complex systems has many aspects, and any textmust make choices about which mater ial to include. We have suggested that complexsystems have more than a few parts and less than too many of them. There are two ap-proaches to this inter mediate regime. The first is to consider systems with more thana few parts, but still a denumerable numberdenumerable,that is, by a single personin a reasonable amount of t ime. The second is to consider many parts, but just fewerthan too many. In the first approach the main task is to describe the behavior of a par-ticular system and its mechanism of oper ationthe funct ion of a neural network ofa few to a few hundred neurons, a few-celled organism, a small protein,a few people,etc. This is done by describing completely the role of each of the parts. In the secondapproach, the precise number of parts is not essential,and the main task is a statisti-cal study of a collect ion of systems that differ fr om ea ch other but share the samestruct urean ensemble o f systems. This approach t reats general p roperties of pro-teins, neural networks, societ ies, etc. In this text, we adopt the second approach.However, an interesting twist to our discussion is that we will show that any complexsystem requires a description as a par ticular few-part syst em.A complementary vol-ume to the present one would consider examples of systems with only a few parts andanalyze their function with a view toward extract ing general pr inciples. These pr inci-ples would complement the seemingly more general analysis of the statisticalapproach.The order of pr esentation of the topics in this text is a matter of taste. Many ofthe chapter s are self-contained discussions of a particular system or question. The firstchapter contains mat er ial that p rovides a foundation for the rest. Part of the role ofthis chapter is the introduction of simple models upon which the remainder of thetext is based. Another role is the r eview of concepts and t echniques that will be usedin lat er chap ters so that the text is mo re self-contained. Because of the interdiscipli-nary nature of the subject matter, the first chapter is considered to have particular im-portance. Some of the mat er ial should be familiar to most graduate students, whileother mater ial is found only in the p rofessional literature. For example, basic proba-bility theor y is reviewed, as well as the concepts and pr opert ies of cellular automata.The pur pose is to enable this t ext to be read by students and researchers with a var i-ety of backgrounds. However, it should be apparent that digesting the variety of con-cepts aft er only a br ief presentation is a difficult task. Additional sources of mater ialare listed at the end of this text.Throughout the book, we have sought to limit advanced for mal discussions to aminimum. When possible, we select models that can be described with a simpler for-malism than must be used to t reat the most gener al case possible. Where additionallayers of formalism are particular ly appropriate, reference is made to other liter ature.Simulations are described at a level of detail that,in most cases,should enable the st u-dent to perform and expand upon the simulations described. The graphical display ofsuch simulations should be used as an integr al part of exposure to the dynamics ofthese syst ems. Such displays are generally effective in d eveloping an intuition aboutwhat are the impor tant or relevant proper ties of these systems.P re f a c e xiii# 29412 Cust: AddisonWesley Au: Bar-Yam Pg. No. xiiiTitle: Dynamics Complex SystemsShor t / Normal / LongFMadBARYAM_29412 3/10/02 11:05 AM Page xiii# 29412 Cust: AddisonWesley Au: Bar-Yam Pg. No. xivTitle: Dynamics Complex SystemsShor t / Normal / LongFMadBARYAM_29412 3/10/02 11:05 AM Page xiv# 29412 Cust: AddisonWesley Au: Bar-Yam Pg. No. xvTitle: Dynamics Complex SystemsShor t / Normal / LongAcknowle dgme nt sThis book is a composite of many ideas and r eflects the efforts of many individualsthat would be impossible to acknowledge. My personal efforts to compose this bodyof knowledge into a coherent framework for fut ure study are also ind ebted to manywho contr ibuted to my own development. It is the earliest teachers, who we can nolonger identify by memor y, who should be acknowledged at the completion of a ma-jor effort. They and the teachers I remember from elementary school through gradu-ate school, especially my thesis advisor John Joannopoulos, have my deepest grati-tude. Consistent with their dedication, may this be a reward for their effor ts.The study of complex systems is a new endeavor, and I am grateful to a few col-leagues and teachers who have inspired me to pursue this path. Charles Bennettthrough a few joint car trips opened my mind to the possibilities of this field and thepaths less trodden that lead to it. Tom Malone, through his course on networked cor-porations, not only cont ributed significant concepts to the last chap ter of this book,but also motivated the creation of my course on the dynamics of complex systems.Ther e are colleagues and students who have inspir ed or cont ributed to my un-derstanding of various aspects of mat erial cover ed in this text. Some of this cont r ibu-tion arises fr om reading and commenting on various asp ects of this text, or throughdiscussions of the mat erial that eventually made its way here. In some cases the dis-cussions were or iginally on unrelated matters, but because they were eventually con-nected to these subjects,they are here acknowledged. Roughly divided by area in cor-respondence with the order they appear in the text these include: GlassesDavidAdler; Cellular AutomataGerard Vichniac, Tom Toffoli, Norman Margolus, MikeBiafore, Eytan Domany, Danny Kandel; ComputationJeff Siskind; Multigr idAchiBrandt, Shlomi Taasan, Sorin Costiner; Neural NetworksJohn Hopfield, SageetBar-Yam, Tom Kincaid, Paul Appelbaum, Charles Yang, Reza Sadr-Lahijany, JasonRedi, Lee-Peng Lee, Hua Yang, Jerome Kagan, Ernest Har tmann; Protein FoldingElisha Haas, Charles DeLisi, Temple Smith, Robert Davenport, David Mukamel,Mehran Kardar ; Polymer DynamicsYitzhak Rabin, Mark Smith, Bor is Ost rovsky,Gavin Crooks, Eliana DeBer nardez-Clark; EvolutionAlan Perelson, Derren Pier re,Daniel Goldman, Stuart Kauffman, Les Kaufman; Developmental BiologyIr vingEpstein, Lee Segel, Ainat Rogel, Evelyn Fox Keller ; ComplexityCharles Bennett,Michael Wer man, Michel Baranger; Human Economies and SocietiesTom Malone,Har r y Bloom, Benjamin Samuels, Kosta Tsipis, Jonathan King.A special acknowledgment is necessary to the students of my course from BostonUniver sity and MIT. Among them are students whose p rojects became incorporatedin parts of this t ext and are mentioned above. The int erest that my colleagues haveshown by attending and participating in the course has bright ened it for me and theircontributions are meaningful: Lewis Lipsitz, Michel Baranger, Paul Barbone, GeorgeWyner, Alice Davidson,Ed Siegel, Michael Werman,Lar r y Rudolfand Mehran Kardar.Among the rea der s of this t ext I am par ticularly ind ebted to the detailed com-ments of Bruce Boghosian, and the suppor tive comments of the series editor BobDevaney. I am also indebted to the suppor t of Charles Cantor and Jerome Kagan.xvFMadBARYAM_29412 3/10/02 11:05 AM Page xv# 29412 Cust: AddisonWesley Au: Bar-Yam Pg. No. xviTitle: Dynamics Complex SystemsShor t / Normal / LongI would like to acknowledge the constr uctive effor ts o f the edit ors at Addison-Wesley star ting from the initial contact with Jack Repcheck and continuing withJeff Robbins. I thank Lynne Reed for coordinating production, and at CarlisleCommunicat ions: Susan Steines, Bev Kraus, Faye Schilling, and Kathy Davis.The software used for the t ext, graphs, figures and simulations of this book, in-cludes: Microsoft Excel and Word, Deneba Canvas, Wolframs Mathematica, andSymantec C. The hardware includes: Macintosh Quadra,and IBM RISC wor kstations.The contr ibutions of my family, to whom this book is dedicated, cannot be d e-scribed in a few words.Yaneer Bar-YamNewton, Massachusetts, June 1997xvi Ac k n ow l e d g m e n t sFMadBARYAM_29412 3/10/02 11:05 AM Page xvi1# 29412 Cust: AddisonWesley Au: Bar-Yam Pg. No. 1Title: Dynamics Complex SystemsShor t / Normal / Long0Ove rvi ew:The Dyna mi cs of Comple x Sys t e ms Exa mple s, Que s t i ons, Me t hods a nd Conce pt sThe Fi e ld of Comple x Sys t e msThe study o f complex systems in a unified framework has become r ecognized in r e-cent years as a new scientific discipline, the ultimate of int erdisciplinary fields. It isstrongly r ooted in the advances that have been made in diverse fields ranging fromphysics to anthropology, from which it dr aws inspir ation and to which it is relevant.Many of the syst ems that surround us are complex. The goal of understandingtheir properties motivates much if not all of scientific inquiry. Despite the great com-plexity and variety of systems, universal laws and phenomena are essential to our in-quiry and to our understanding. The idea that all matter is f ormed out o f the samebuilding blocks is one of the original concepts of science. The moder n manifestationof this conceptatoms and their constituent par ticlesis essential to our recogni-tion of the commonality among syst ems in science. The universality of constituentscomplements the universality of mechanical laws (classical or quantum) that governtheir motion. In biology, the common molecular and cellular mechanisms of a largevariet y of organisms for m the basis of our studies. However, even more univer sal thanthe constituents are the dynamic processes of variation and selection that in somemanner cause organisms to evolve. Thus, all scientific endeavor is based, to a great eror lesser degree, on the existence of universality, which manifests itself in diverse ways.In this context,the study o f complex systems as a new endeavor str ives to increase ourability to under stand the universalit y that ar ises when systems are highly complex.A dict ionary d efinition of the word complex is: consisting of interconnectedor int er woven parts. Why is the nature of a complex system inherently related to itsparts? Simple systems are also formed out of parts. To explain the difference betweensimple and complex syst ems, the t erms interconnected or interwoven are some-how essential.Qualitat ively, to understand the behavior of a complex system we mustunderstand not only the behavior of the parts but how they act t ogether to form thebehavior of the whole. It is because we cannot describe the whole without describingeach part, and b ecause ea ch part must be described in relation to other parts, thatcomplex systems are difficult to under stand. This is relevant to another definition ofcomplex: not easy to understand or analyze. These qualitative ideas about what acomplex system is can be made more quantitat ive. Ar ticulating them in a clear way is0 . 100adBARYAM_29412 9/5/00 7:26 PM Page 1both essential and fruitful in pointing the way toward progress in understanding theuniversal pr opert ies of these systems.For many years, professional sp ecialization has led science to progressive isola-tion of individual disciplines. How is it possible that well-separated fields such as mol-ecular biology and economics can suddenly become unified in a single discipline?How does the study of complex systems in general per tain to the detailed efforts de-voted to the study of part icular complex syst ems? In this r egard one must be car efulto acknowledge that there is always a dichotomy between universality and specificit y.A study of universal pr inciples does not replace detailed description of part icularcomplex systems. However, univer sal pr inciples and t ools guide and simplify our in-quiries into the study of specifics. For the study of complex systems,universal simpli-fications are particularly impor tant. Somet imes universal pr inciples are intuitivelyappr eciated without being explicitly stated. However, a car eful ar t iculation o f suchprinciples can enable us to approach par ticular syst ems with a systematic guidancethat is often absent in the study of complex syst ems.A pictorial way of illustrating the relationship of the field of complex systems tothe many other fields o f science is indicat ed in Fig. 0.1.1. This figure shows the con-ventional view of science as progressively separating into disparate disciplines in or-der to gain knowledge about the ever larger complexity of syst ems. It also illust ratesthe view of the field of complex systems, which suggests that all complex systems haveuniversal proper ties. Because each field develops tools for addressing the complexityof the systems in their domain, many of these tools can be adap ted for more generaluse by recognizing their univer sal applicabilit y. Hence the mot ivation for cross-disciplinar y fer tilizat ion in the study of complex systems.In Sections 0.20.4 we initiate our study of complex syst ems by discussing ex-amples, questions and methods that are relevant to the study of complex systems.Ourpur pose is to int roduce the field without a strong bias as to conclusions, so that thestudent can develop independent perspectives that may be useful in this new fieldopening the way to his or her own cont ributions to the study of complex systems. InSection 0.5 we introduce two key conceptsemergence and complexitythat willar ise through our study of complex systems in this text.Exa mple s0 . 2 . 1 A few exa mplesWhat are com p l ex sys tems and what proper ties ch a racteri ze them? It is hel pful to star tby making a list of s ome examples of com p l ex sys tem s . Ta ke a few minutes to make yo u rown list. Con s i der actual sys tems ra t h er than mathem a tical models (we wi ll con s i derm a t h em a tical models later ) . Ma ke a list of s ome simple things to con trast them wi t h .Examples of Complex SystemsGover nmentsFamiliesThe human bodyphysiological perspective0 . 22 O ve r v i e w# 29412 Cust: AddisonWesley Au: Bar-Yam Pg. No. 2Title: Dynamics Complex SystemsShor t / Normal / Long00adBARYAM_29412 9/5/00 7:26 PM Page 2# 29412 Cust: AddisonWesley Au: Bar-Yam Pg. No. 3Title: Dynamics Complex SystemsShor t / Normal / LongSimple systemsPhysicsChemistry BiologyMathematicsComputer ScienceSociologyPsychologyEconomicsAnthropologyPhilosophySimple systemsComplex systems(a)(b)ChemistryBiologyPsychologyPhysicsMathematicsComputer ScienceSociologyEconomicsAnthropologyPhilosophyFi gure 0 . 1 . 1 Con ce pt ua l illust ra t ion of t h e spa ce of scie n t ific in quiry. ( a ) is t h e con ve n t ion a lvie w wh e re disciplin e s dive rge a s kn owle dge in cre a se s be ca use of t h e in cre a sin g comple xit yof t h e va rious syst e ms be in g st udie d. I n t h is vie w a ll kn owle dge is spe cific a n d kn owle dge isga in e d by providin g more a n d more de t a ils. ( b) illust ra t e s t h e vie w of t h e fie ld of complexsyst e ms wh e re comple x syst e ms h a ve un ive rsa l prope rt ie s. By con side rin g t h e common prop-e rt ie s of comple x syst e ms, on e ca n a pproa ch t h e spe cifics of pa rt icula r comple x syst e ms fromt h e t op of t he sph e re a s we ll a s from t he bot t om.00adBARYAM_29412 9/5/00 7:26 PM Page 3A personpsychosocial per spectiveThe brainThe ecosystem of the worldSubworld ecosystems: desert, r ain forest, oceanWeatherA corpor ationA computerExamples of Simple SystemsAn oscillatorA pendulumA spinning wheelAn orbiting planetThe purpose of thinking about examples is to develop a first under standing of thequestion, What makes systems complex? To begin to address this question we can startdescribing systems we know intuitively as complex and see what properties they share.We tr y this with the first two examples listed above as complex systems.Government It has many different funct ions:milit ar y, immigrat ion,t axat ion,income distr ib-ut ion, transpor tation, regulation. Each function is itself complex. There are different levels and t ypes of government: local, state and federal; townmeeting, council,mayoral. There are also various governmental forms in differ-ent count ries.Family It is a set of individuals. Each individual has a relationship with the other individuals. Th ere is an interp l ay bet ween the rel a ti onship and the qu a l i t ies of the indivi du a l . The family has to interact with the outside world. There are different kinds of families: nuclear family, extended family, etc.These descriptions focus on function and st ructure and diverse manifestation.We can also consider the role that time plays in complex systems. Among the proper-ties of complex systems are change, growth and death, possibly some for m of life cy-cle. Combining time and the environment, we would point to the ability of complexsystems to adapt.One of the issues that we will need to address is whether there are differ ent cate-gor ies of complex syst ems. For example, we might contr ast the systems we just de-scribed with complex physical systems: hydrodynamics (fluid flow, weather), glasses,composite mater ials, earthquakes. In what way are these syst ems similar to or differ-ent from the biological or social complex systems? Can we assign funct ion and discussst ructure in the same way?4 O ve r v i e w# 29412 Cust: AddisonWesley Au: Bar-Yam Pg. No. 4Title: Dynamics Complex SystemsShor t / Normal / Long00adBARYAM_29412 9/5/00 7:26 PM Page 40 . 2 . 2 Cent ra l propert ies of complex syst emsAfter beginning to describe complex systems,a second st ep is to identify commonal-ities. We might make a list o f some of the char acteristics of complex systems and as-sign each of them some measure or att r ibute that can provide a first method of clas-sification or descript ion. Elements (and their number) Interactions (and their strength) For mation/Oper ation (and their time scales) Diversit y/Variability Environment (and its demands) Activit y(ies) (and its[their] objective[s])This is a first step toward quantifying the pr opert ies of complex systems.Quantifyingthe last three in the list requires some method of counting possibilities. The problemof counting possibilities is cent ral to the discussion of quantitative complexit y.0 . 2 . 3 Emergence: From element s a nd pa rt s t o complex syst emsThere are two approaches to organizing the proper ties of complex syst ems that wil lserve as the foundation of our discussions. The first of these is the relationship b e-tween elements,parts and the whole. Since there is only one proper ty of the complexsystem that we know for sure that it is complexthe primary question we can askabout this relationship is how the complexity of the whole is related to the complex-ity of the parts. As we will see, this question is a compelling question for our under-standing of complex systems.From the examples we have indicat ed above, it is appar ent that parts of a com-plex syst em are oft en complex syst ems themselves. This is reasonable, because whenthe parts o f a syst em are complex, it seems intuit ive that a collection of them wouldalso be complex. However, this is not the only possibilit y.Can we describe a syst em composed of simple parts where the collective behav-ior is complex? This is an important possibility, called emergent complexit y. Any com-plex system formed out of atoms is an example. The idea of emergent complexity isthat the behaviors of many simple parts inter act in such a way that the behavior of thewhole is complex.Elements are those parts of a complex system that may be consid-ered simple when descr ibing the behavior of the whole.Can we describe a system composed o f complex parts where the collective b e-havior is simple? This is also possible, and it is called emergent simplicit y. A usefulexample is a planet or biting around a star. The behavior of the planet is quite simple,even if the planet is the Earth, with many complex systems upon it. This example il-lustrates the possibility that the collective syst em has a behavior at a different scalethan its parts. On the smaller scale the system may behave in a complex way, but onthe larger scale all the complex details may not be relevant.E xa m p l e s 5# 29412 Cust: AddisonWesley Au: Bar-Yam Pg. No. 5Title: Dynamics Complex SystemsShor t / Normal / Long00adBARYAM_29412 9/5/00 7:26 PM Page 50 . 2 . 4 Wha t is complexit y?The second approach to the study of complex systems begins from an understandingof the relationship of systems to their descriptions. The cent r al issue is defining quan-titatively what we mean by complexity. What, after all, do we mean when we say thata system is complex? Better yet, what do we mean when we say that one system is morecomplex than another? Is there a way to identify the complexity of one system and tocompare it with the complexity of another system? To develop a quantitat ive under-standing of complexity we will use tools of both statistical physics and computer sci-enceinfor mation theor y and computation theor y. Accor ding to this understanding,complexity is the amount of information necessary to describe a system. However, inorder to arr ive at a consistent definition,care must be taken to specify the level of de-tail provided in the descr iption.One of our targets is to understand how this concept of complexity is related toemergenceemergent complexity and emergent simplicit y. Can we understand whyinformation-based complexity is related to the description of elements,and how theirbehavior gives rise to the collective complexit y of the whole syst em?Section 0.5 of this overview discusses further the concepts of emergence andcomplexity, providing a simplified preview of the more complete discussions later inthis text.Que s t i onsThis text is st ruct ured around four questions r elated to the char acterization of com-plex systems:1. Space: What are the character istics of the st ructure o f complex syst ems? Manycomplex systems have substructure that extends all the way to the size of the sys-tem itself. Why is there substructure?2. Time: How long do dynamical processes take in complex systems? Many complexsystems have specific responses to changes in their environment that requirechanging their internal structure. How can a complex structure respond in a rea-sonable amount of t ime?3. Self-organization and/versus organization by design: How do complex syst emscome into existence? What are the dynamical processes that can give rise to com-plex syst ems? Many complex syst ems und ergo guid ed d evelopmental p rocessesas par t of their formation. How are developmental processes guided?4. Com p l ex i t y: What is com p l ex i ty? Com p l ex sys tems have va rying degrees of com-p l ex i ty. How do we ch a racteri ze / d i s tinguish t he va r ying degrees of com p l ex i ty ?Chapter 1 of this text plays a special role. Its ten sect ions introduce mathematicaltools. These tools and their related concepts are integral to our under standing of com-plex system behavior. The main part of this book consists of eight chapters,29. These0 . 36 O ve r v i e w# 29412 Cust: AddisonWesley Au: Bar-Yam Pg. No. 6Title: Dynamics Complex SystemsShor t / Normal / Long00adBARYAM_29412 9/5/00 7:26 PM Page 6chapter s are paired.Each pair discusses one of the above four questions in the contextof a par ticular complex syst em. Chapters 2 and 3 discuss the role of substr ucture inthe context of neural networks. Chapters 4 and 5 discuss the time scale of dynamicsin the context of protein folding. Chapters 6 and 7 discuss the mechanisms of orga-nization of complex systems in the context of living organisms. Chapters 8 and 9 dis-cuss complexity in the context of human civilization. In each case the first of the pairof chapters discusses mor e gener al issues and models. The second t ends to be morespecialized to the system that is under discussion. There is also a patter n to the degreeof analytic, simulation or qualitative treatments. In gener al,the first of the two chap-ters is more analytic, while the second relies more on simulations or qualitative treat-ments. Each chap ter has at least some discussion of qualitative concepts in additio nto the formal quantitative discussion.Another way to regard the text is to distinguish between the two approaches sum-marized above. The first deals with elements and interactions. The second deals withdescriptions and information. Ultimately, our object ive is to relate them, but we do sousing questions that progress gradually from the elements and interactions to the de-scriptions and infor mation. The former dominates in earlier chap ters, while the lat -ter is important for Chapter 6 and becomes dominant in Chapters 8 and 9.While the discussion in each ch a pter is pre s en ted in the con text of a spec i f i ccom p l ex sys tem , our focus is on com p l ex sys tems in gen era l . Thu s , we do not at-tem pt (nor would it be po s s i ble) to revi ew the en ti re fields of n eu r al net work s , pr o-tein fo l d i n g, evo luti on , devel opm ent al bi o l ogy and social and econ omic scien ce s .Si n ce we are intere s t ed in universal aspect s of these sys tem s , t he topics we covern eed not be t he issues of con tem por a r y impor t a n ce in the stu dy of these sys tem s .Our approach is to motiva te a qu e s ti on of i n t erest in the con text of com p l ex sys-tems using a par ticular com p l ex sys t em , t h en to step back and adopt a met h od ofs tu dy that has rel eva n ce to all com p l ex sys t em s . Re s e a rch ers intere s ted in a par ti c u-lar com p l ex sys tem are as likely t o find a discussion of i n terest to t hem in any on eof the ch a pters , and should not focus on the ch a pter with the par ticular com p l exs ys t em in it s ti t l e .We note that the text is interrupted by questions that are, with few exceptions,solved in the text. They are given as questions to promote independent thought aboutthe study of complex systems. Some of them develop further the analysis of a systemthrough analyt ic work or through simulations. Others are designed for concept ual de-velopment. With few except ions they should be considered int egr al to the text, andeven if they are not solved by the reader, the solut ions should be read.Que s t i on 0 . 3 . 1 Consider a few complex systems. Make a list of their el-ements, interact ions between these elements, the mechanism by whichthe system is formed and the activities in which the system is engaged.Solut i on 0 . 3 . 1 The following table indicates proper ties of the systems thatwe will be discussing most int ensively in this t ext. Q u e s t i o n s 7# 29412 Cust: AddisonWesley Au: Bar-Yam Pg. No. 7Title: Dynamics Complex SystemsShor t / Normal / Long00adBARYAM_29412 9/5/00 7:26 PM Page 7Ta ble 0 . 3 . 1 : Complex Syst e ms a nd Some At t ribut e s8 O ve r v i e w# 29412 Cust: AddisonWesley Au: Bar-Yam Pg. No. 8Title: Dynamics Complex SystemsShor t / Normal / LongSystem El ement Interacti on Formati on Acti vi tyProteins Amino Acids Bonds Protein folding Enzymatic activityNervous system Neurons Synapses Learning BehaviorNeural networks ThoughtPhysiology Cells Chemical Developmental Movementmessengers biology PhysiologicalPhysical support functionsLife Organisms Reproduction Evolution SurvivalCompetition ReproductionPredation ConsumptionCommunication ExcretionHuman Human Beings Communication Social evolution Same as Life?economies Technology Confrontation Exploration?and societiesCooperationMe t hodsWhen we think about methodology, we must keep purpose in mind.Our purpose instudying complex systems is to extract general principles.General pr inciples can takemany forms. Most pr inciples are ar ticulated as relationships b etween propertieswhen a system has the propert y x, then it has the proper t y y. When possible, r elation-ships should be quantitat ive and expressed as equations. In order to explore such re-lationships, we must constr uct and study mathematical models. Asking why theproper ty x is related to the proper ty y requires an understanding of alter natives. Whatelse is possible? As a bonus, when we are able to generate systems with various p rop-ert ies, we may also be able to use them for pract ical applicat ions.All appr oaches that are used for the study of simple systems can be applied to thestudy of complex systems. However, it is impor tant to recognize features of conven-tional approaches that may hamper progress in the study of complex syst ems. Bothexper imental and theoretical methods have been developed to overcome these diffi-culties. In this text we introduce and use methods of analysis and simulation that arepar ticular ly suit ed to the study of complex syst ems. These methods avoid standar dsimplifying assump tions, but use other simplifications that are better suit ed to ourobject ives. We discuss some of these in the following paragraphs. Dont take it apart. Since interactions between parts of a complex system are es-sential to understanding its behavior, looking at parts by themselves is not suffi-cient. It is necessary to look at parts in the context of the whole. Similarly, a com-plex syst em int er acts with its environment, and this e nvironmental influence is0 . 400adBARYAM_29412 9/5/00 7:26 PM Page 8impor tant in describing the behavior of the system. Exper imental tools have beendeveloped for studying systems in situ or in vivoin context. Theoretical analyt icmethods such as the mean field ap proach enable parts of a system to be studiedin context. Computer simulations that tr eat a system in its entiret y also avoidsuch problems. Dont assume smoo t h n e s s . Mu ch of the qu a n ti t a tive stu dy of simple sys tems make suse of d i f feren tial equ a ti on s . Di f feren tial equ a ti on s ,l i ke the wave equ a ti on ,a s su m ethat a sys tem is essen ti a lly uniform and that local details dont matter for the be-h avi or of a sys tem on larger scales. These assu m pti ons are not gen er a lly valid forcom p l ex sys tem s . Al tern a te static models su ch as fract a l s , and dynamical models in-cluding itera t ive maps and cellular automata may be used inste ad . Dont assume that only a few parameters are important. The behavior of complexsystems depends on many independent pieces of information. Developing an un-derstanding of them requires us to build mental models. However, we can onl yhave in mind 72 independent things at once. Analytic approaches, such asscaling and renormalization,have been developed to identify the few relevant pa-rameters when this is possible. Information-based approaches consider the col-lection of all parameters as the object of study. Computer simulations keep trackof many parameters and may be used in the st udy of dynamical processes.There are also tools needed for communication of the results of studies.Conventional manuscripts and oral pr esentations are now being augmented by videoand int er active media. Such novel approaches can increase the effectiveness of com-municat ion,par ticular ly of the results of computer simulations. However, we shouldavoid the cute picture syndrome, where pictures are presented without accompany-ing discussion or analysis.In this t ext, we int roduce and use a variety of analyt ic and computer simulationmethods to address the questions list ed in the previous section. As mentioned in thepreface, there are two gener al methods f or stud ying complex syst ems. In the first, aspecific syst em is selected and each of the parts as well as their int eractions are iden-tified and described. Subsequently, the objective is to show how the behavior of thewhole emerges from them. The second approach considers a class of systems (ensem-ble), wher e the essential characteristics of the class are described,and statistical anal y-sis is used to obtain properties and behaviors of the systems. In this t ext we focus onthe latter approach.Conce pt s : Eme rge nce a nd Comple xi t yThe object ives of the field o f complex syst ems are built on fundamental conceptsemergence, complexityabout which there are common misconceptions that are ad-dressed in this sect ion and throughout the book.Once under stood,these concepts re-veal the context in which universal propert ies of complex syst ems arise and specificuniversal phenomena, such as the evolution of biological systems, can be betterunder stood.0 . 5Co n c e p t s : Em e r g e n c e a n d c o m p l e x i t y 9# 29412 Cust: AddisonWesley Au: Bar-Yam Pg. No. 9Title: Dynamics Complex SystemsShor t / Normal / Long00adBARYAM_29412 9/5/00 7:26 PM Page 9A complex system is a syst em formed out of many components whose behavioris emergent,that is,the behavior of the system cannot be simply infer red from the be-havior of its components. The amount of infor mation necessary to describe the be-havior of such a system is a measure of its complexity. In the following sect ions wediscuss these concepts in greater detail.0 . 5 . 1 EmergenceIt is impossible to understand complex systems without recognizing that simple atomsmust somehow, in large numbers, give rise to complex collect ive behaviors. How andwhen this occurs is the simplest and yet the most profound problem that the study ofcomplex syst ems fa ces. The p roblem can be appr oached first by developing an un-derstanding o f the t erm emergence. For man y, the concept o f emergent behaviormeans that the behavior is not captured by the behavior of the parts. This is a seriousmisunderstanding. It arises because the collective behavior is not readily understoodfrom the behavior of the parts. The collective behavior is, however, contained in thebehavior of the parts if they are studied in the context in which they are found. To ex-plain this, we discuss examples of emergent proper ties that illustrate the differ ence be-tween local emergencewhere collect ive behavior appears in a small part of the sys-temand global emergencewhere collective behavior p er tains to the system as awhole. It is the latter which is par ticularly relevant to the study of complex systems.We can speak abo ut em er gen ce wh en we con s i der a co ll ecti on of el em ents and theproper ties of the co ll ective beh avi or of these el em en t s . In conven ti onal phys i c s , t h emain arena for the st u dy of su ch proper ties is therm odynamics and stati s tical me-ch a n i c s . The easiest therm odynamic sys t em to think abo ut is a gas of p a r ti cl e s . Twoem er gent properties of a gas are its pre s su re and tem pera tu re . The re a s on they areem er gent is that they do not natu ra lly arise out of the de s c ri pti on of an indivi dual par-ti cl e . We gen era lly de s c ri be a par ti cle by spec i f ying its po s i ti on and vel oc i t y. Pre s su reand tem pera tu re become rel evant on ly wh en we have many part i cles toget h er. Wh i l ethese are em er gent propert i e s , the way they are em er gent is very limited . We call theml ocal em er gent propert i e s . The pre s su re and tem pera tu re is a local proper ty of the ga s .We can take a ver y small sample of t he gas aw ay from the rest and st i ll define and mea-su re the (same) pre s su re and tem pera tu re . Su ch propert i e s ,c a ll ed inten s ive in phys i c s ,a re local em er gent proper ti e s . Ot h er examples from physics of l oc a lly em er gent be-h avi or ar e co ll ect ive modes of exc i t a ti on su ch as sound wave s , or light prop a ga ti on ina med iu m . Phase tr a n s i ti ons (e.g. , solid to liquid) also repre s ent a co ll ective dy n a m i c sthat is vi s i ble on a mac ro s copic scale, but can be seen in a micro s copic sample as well .Another example of a local emergent p roper t y is the formation o f water fr omatoms of hydrogen and oxygen. The proper ties of water are not apparent in the prop-er ties of gasses of oxygen or hydrogen. Neither does an isolated water molecule revealmost propert ies of water. However, a microscopic amount of water is sufficient.In t he stu dy of com p l ex sys tems we are par ti c u l a rly intere s ted in gl obal em er gen tproperti e s . Su ch proper ties depend on the en ti re sys tem . The mat hem a tical t re a tm en tof gl obal em er gent propert ies requ i res some ef fort . This is one re a s on that em er gen ceis not well apprec i a ted or unders tood . We wi ll discuss gl obal em er gen ce by su m m a ri z-10 O ve r v i e w# 29412 Cust: AddisonWesley Au: Bar-Yam Pg. No. 10Title: Dynamics Complex SystemsShor t / Normal / Long00adBARYAM_29412 9/5/00 7:26 PM Page 10ing the re sults of a classic mathem a tical t re a tm en t , and then discuss it in a more gen-eral manner that can be re ad i ly apprec i a ted and is useful for sem i qu a n ti t a tive analys e s .The classic analysis of global emergent behavior is that of an associative memor yin a simple model of neural networks known as the Hopfield or attractor network.The analogy to a neural network is useful in order to be concrete and relate this modelto known concepts. However, this is more gener ally a model o f any syst em for medfrom simple elements whose states are cor related. Without such correlations, emer-gent behavior is impossible. Yet if all elements are correlated in a simple way, then lo-cal emergent behavior is the outcome. Thus a mo del must be sufficiently rich in or-der to cap ture the phenomenon o f global emergent behavior. One of the importantqualities of the attractor network is that it displays global emergence in a part icularlyelegant manner. The following few paragraphs summarize the oper ation of the at -t ractor network as an associat ive memory.The Hopfield networ k has simple binary elements that are either ON or OFF. Thebinary elements are an abstraction of the firing or quiescent state of neurons. The el-ements interact with each other to create cor relations in the firing patterns. The in-teractions represent the role of synapses in a neural networ k. The network can workas a memor y. Given a set o f preselected patterns, it is possible to set the interact ionsso that these patt erns are self-consistent states of the networkthe networ k is stablewhen it is in these firing patterns. Even if we change some of the neur ons, the or igi-nal pattern will be recovered. This is an associative memor y.Assume for the moment that the pattern of firing repr esents a sentence, such asTo be or not to be,that is the question. We can recover the complete sentence by pre-senting only part of it to the networ k To be or not to be, that might be enough. Wecould use any part to retrieve the whole,such as,to be,that is the question. This kindof memor y is to be contrasted with a computer memor y, which works by assigning anaddress to each st or age location. To access the information stored in a par ticular lo-cation we need to know the address. In the neural network memor y, we specify par tof what is located there, rather than the analogous address: Hamlet, by WilliamShakespeare, act 3, scene 1, line 64.More centr al to our discussion,however, is that in a computer memor y a partic-ular bit of information is st ored in a part icular switch. By cont rast,the network doesnot have its memor y in a neuron. Instead the memor y is in the synapses. In the model,there are synapses between each neuron and every other neuron. If we remove a smallpart of the networ k and look at its propert ies,then the number of synapses that a neu-ron is left with in this small part is only a small fr action of the number of synapses itstarted with. If there are more than a few patterns st ored, then when we cut out thesmall part of the network it loses the ability to remember any of the patterns, even thepar t which would be represented by the neurons contained in this par t.This kind of behavior char acterizes emergent p roper ties. We see that emergentproperties cannot be studied by physically taking a system apart and looking at theparts (reductionism). They can,however, be studied by looking at each of the parts inthe context of the syst em as a whole. This is the nature o f emergence and an indica-tion of how it can be studied and under stood.Co n c e p t s : Em e r g e n c e a n d c o m p l e x i t y 11# 29412 Cust: AddisonWesley Au: Bar-Yam Pg. No. 11Title: Dynamics Complex SystemsShor t / Normal / Long00adBARYAM_29412 9/5/00 7:26 PM Page 11The above discussion reflects the analysis of a relatively simple mathematicalmodel of emergent behavior. We can,however, provide a more qualitat ive discussionthat ser ves as a guide for thinking about diverse complex systems. This discussion fo-cuses on the proper ties of a system when part of it is removed. Our discussion of lo-cal emergent proper ties suggested that taking a small part out of a large system wouldcause little change in the proper ties of the small part, or the propert ies of the largepart.On the other hand, when a system has a global emergent proper t y, the behaviorof the small part is different in isolation than when it is par t of the larger system.If we think about the system as a whole, rather than the small part of the system,we can identify the system that has a global emergent propert y as being for med out ofinterdependent parts. The t erm interdependent is used here instead of the t ermsinter connected or inter woven used in the dict ionary definition of complexquoted in Section 0.1, because neither of the latter terms per tain directly to the influ-ence one part has on another, which is essential to the proper ties of a dynamic system.Interdependent is also distinct from inter acting, because even strong interact ionsdo not necessarily imply int erdependence of behavior. This is clear fr om the macr o-scopic propert ies of simple solids.Thus, we can characterize complex systems through the effect of removal of partof the system. There are two natural possibilities. The first is that properties of the partare affected, but the rest is not affected. The second is that propert ies of the rest are af-fected by the r emoval of a part. It is the latt er that is most appealing as a model of at ruly complex system. Such a system has a collective behavior that is dependent on thebehavior of all of its parts. This concept becomes more precise when we connect it toa quantitat ive measure of complexit y.0 . 5 . 2 Complexit yThe second concept that is central to complex syst ems is a quantitative measure ofhow complex a syst em is. Loosely speaking, the complexity of a system is theamount of information needed in order to describe it. The complexity d epends onthe level of detail r equired in the description. A more for mal definition can be un-derstood in a simple way. If we have a system that could have many possible states,but we would like to specify which state it is actually in, then the numb er of binar ydigits (bits) we need to specify this par ticular state is r elated to the numb er of statesthat are possible. If we call the number of states then the number of bits of infor-mation needed isI log2() (0.5.1)To understand this we must realize that to specify which state the system is in, we mustenumerate the states. Representing each state uniquely requires as many numbers asthere are states. Thus the number of states of the representation must be the same asthe number of states of the syst em. For a st ring of N bits there are 2Npossible statesand thus we must have 2N(0.5.2)12 O ve r v i e w# 29412 Cust: AddisonWesley Au: Bar-Yam Pg. No. 12Title: Dynamics Complex SystemsShor t / Normal / Long00adBARYAM_29412 9/5/00 7:26 PM Page 12which implies that N is the same as I above. Even if we use a descript ive English textinstead of numbers,t her e must be the same number of possible descriptions as thereare states, and the information content must be the same. When the number of pos-sible valid English sent ences is properly accounted for, it turns out that the best est i-mate of the amount of infor mation in English is about 1 bit per character. This meansthat the infor mation content of this sentence is about 120 bits, and that of this bookis about 3 106bits.For a microstate of a p hysical syst em, where we specify the positions and mo-menta of each of the par ticles, this can be r ecognized as proport ional to the ent ropyof the syst em, which is defined asS k ln() k ln(2)I (0.5.3)wh ere k 1.38 1 02 3Jo u l e / Kelvin is the Boltzmann constant wh i ch is rel evant toour conven ti onal ch oi ce of u n i t s . Using measu red en tropies we find t hat en tropies oforder 10 bits per atom are typ i c a l . The re a s on k is so small is that the qu a n ti ties of m a t terwe t yp i c a lly con s i der are in units of Avoga n d ros nu m ber (moles) and the nu m ber ofbits per mole is 6.02 1 02 3times as large . Thu s , t he inform a ti on in a piece of m a ter-ial is of order 1024 bi t s .There is one point about Eq.(0.5.3) that may r equire some clarification. The po-sitions and momenta of particles are real numbers whose specification might requireinfinitely many bits. Why isnt the infor mation necessary to sp ecify the microstate ofa system infinite? The answer to this question comes from quantum physics, which isresponsible for giving a unique value to the ent ropy and thus the information neededto specify a state of the syst em. It does this in two ways. First, it t ells us that micro-scopic states are indistinguishable unless they differ by a discrete amount in positionand momentuma quantum diff erence given by Plancks constant h. Second, it in-dicates that part icles like nuclei or atoms in their ground state are uniquely specifiedby this state,and are indistinguishable from each other. There is no additional infor-mation necessary to specify their int er nal st r ucture. Under standard conditions, es-sentially all nuclei are in their lowest energy state.The rel a ti onship of en tropy and inform a ti on is not acc i den t a l , of co u rs e , but it is thes o u rce of mu ch con f u s i on . The con f u s i on arises because the en t ropy of a physical sys-tem is largest wh en it is in equ i l i br iu m . This su ggests that the most com p l ex sys tem is as ys tem in equ i l i briu m . This is co u n ter to our usual understanding of com p l ex sys tem s .Equ i l i brium sys tems have no spatial stru ctu re and do not ch a n ge over ti m e . Com p l exs ys tems have su b s t a n tial inter nal stru ctu re and this str u ctu re ch a n ges over ti m e .The problem is that we have used the definition of the information necessary tospecify the microscopic state (microstate) of the system rather than the macroscopicstate (macrostate) of the syst em. We need to consider the information necessary todescribe the macrostate o f the system in o rder to define what we mean by complex-it y. One of the important p oints to realize is that in order for the macrostate of thesystem to require a lot of information to describe it,there must be cor relations in themicrostate of the syst em. It is only when many microscopic atoms move in a coher-ent fashion that we can see this motion on a macroscopic scale. However, if manyCo n c e p t s : Em e r g e n c e a n d c o m p l e x i t y 13# 29412 Cust: AddisonWesley Au: Bar-Yam Pg. No. 13Title: Dynamics Complex SystemsShor t / Normal / Long00adBARYAM_29412 9/5/00 7:26 PM Page 13microscopic atoms move t ogether, the syst em must be far from equilibr ium and themicroscopic information (entropy) must be lower than that of an equilibr ium system.It is helpful, even essential, to define a complexity profile which is a func tion ofthe scale of obser vation. To obtain the complexity profile, we obser ve the system at apar ticular length (or time) scale,ignoring all finer-scale details. Then we consider howmuch information is necessary to describe the obser vations on this scale. This solvesthe problem of distinguishing between a microscopic and a macroscopic description.Moreover, for different choices of scale, it explicitly cap tures the dependence of thecomplexit y on the level of detail that is required in the descr iption.The complexity profile must be a monotonically falling function of the scale. Thisis because the infor mation needed to describe a system on a larger scale must be a sub-set of the information needed to describe the system on a smaller scaleany finer-scale description contains the coarser-scale description. The complexity profile char-acterizes the properties of a complex system. If we wish to point to a part icularnumber for the complexity of a system,it is natural to consider the complexity as thevalue of the complexity profile at a scale that is slightly smaller than the size of the sys-tem itself. The b ehavior at this scale includes the movement o f the syst em throughspace, and dynamical changes of the system that are essentially the size of the systemas a whole. The Earth orbiting the sun is a useful example.We can make a dir ect connection between this definition of complexity and thediscussion of the for mation of a complex system out of parts. The complexity of theparts of the system are described by the complexity profile of the system evaluated onthe scale of the parts. When the behavior of the system depends on the behavior of theparts, the complexity of the whole must involve a descript ion of the parts, thus it islarge. The smaller the parts that must be described to describe the behavior o f thewhole, the larger the complexit y of the entire system.For t he I ns t ruct orThis text is designed for use in an introductory graduate-level course, to present var-ious concepts and methodologies of the study of complex systems and to begin to de-velop a common language for researchers in this new field. It has been used for a one-semester course, but the amount of mater ial is large, and it is better to spread thematerial over two semesters.A two-semest er course also provides more oppor tunitiesfor including various other approaches to the study of complex systems, which are asvaluable as the ones that are covered her e and may be more familiar to the instr uctor.Consistent with the objective and purpose of the field,students attending such acourse tend to have a wide variety of backgrounds and interests. While this is a posi-tive development, it causes difficult ies for the syllabus and framework of the cour se.One approach to a course syllabus is to include the int roductor y mat erial givenin Chapter 1 as an int egral part of the course. It is better to int erleave the later chap-ters with the relevant materials from Chapter 1. Such a course might proceed:1.11.6;2; 3; 4; 1.7; 5; 6; 7; 1.81.10; 8; 9. Including the materials of Chapter 1 allows the dis-0 . 614 O ve r v i e w# 29412 Cust: AddisonWesley Au: Bar-Yam Pg. No. 14Title: Dynamics Complex SystemsShor t / Normal / Long00adBARYAM_29412 9/5/00 7:26 PM Page 14cussion of impor tant mathematical methods,and addresses the diverse backgr oundsof the stud ents. Even if the int roductor y chap ter is covered quickly (e.g., in a one-semester course),this establishes a common base of knowledge for the remainder ofthe course. If a high-speed approach is taken,it must be emphasized to the studentsthat this mater ial ser ves only to expose them to concepts that they are unfamiliar with,and to review concepts for those with pr ior knowledge of the t opics covered.Unfortunately, many students are not willing to sit through such an extensive (and in-tense) int roduction.A second approach begins from Chapter 2 and int roduces the material fromChapter 1 only as needed. The chapters that are the most t echnically difficult,and r elythe most on Chapter 1,are Chapters 4 and 5. Thus, for a one-semester course,the sub-ject of protein folding (Chapter s 4 and 5) could be skipped. Then much of the intro-ductor y mater ial can be omitted, with the except ion of a discussion of the last part ofSection 1.3,and some int roduct ion to the subject of entropy and information eitherthrough thermodynamics (Section 1.3) or information theor y (Sect ion 1.8), pr efer-ably both. Then Chapters 2 and 3 can be covered first, followed by Chapters 69, withselected mater ial introduced from Chapter 1 as is ap propr iate for the background ofthe students.Ther e are two additional recommendations.First,it is better to run this course asa project-based course rather than using graded homewor k. The varied backgroundsof students make it difficult to select and fairly gr ade the problems. Projects for indi-viduals or small groups of students can be tailo red to their knowledge and int erests.There are many new areas of inquir y, so that projects may approach resear ch-levelcontributions and be exciting for the students. Unfor tunately, this means that stu-dents may not devote sufficient effor t to the study of course material,and rely largelyupon exposure in lectures. There is no optimal solution to this problem. Secon d,if itis possible,a seminar ser ies with lecturers who wor k in the field should be an integralpart of the course. This pr ovides additional exposure to the varied approaches to thestudy of complex systems that it is not possible for a single lecturer or text to provide.F o r t h e i n s t r u c t o r 15# 29412 Cust: AddisonWesley Au: Bar-Yam Pg. No. 15Title: Dynamics Complex SystemsShor t / Normal / Long00adBARYAM_29412 9/5/00 7:26 PM Page 1516# 29412 Cust: AddisonWesley Au: Bar-Yam Pg. No. 16Title: Dynamics Complex SystemsShor t / Normal / Long1I nt roduct i on a nd Pre li mi na ri e sConce pt ua l Out li neA deceptively simple model of the dynamics of a system is a deterministiciterative map applied to a single real variable. We characterize the dynamics by look-ing at its limiting behavior and the approach to this limiting behavior. Fixed points thatattract or repel the dynamics, and cycles, are conventional limiting behaviors of asimple dynamic system. However, changing a parameter in a quadratic iterative mapcauses it to undergo a sequence of cycle doublings (bifurcations) until it reaches aregime of chaotic behavior which cannot be characterized in this way. This deter-ministic chaos reveals the potential importance of the influence of fine-scale detailson large-scale behavior in the dynamics of systems. A system that is subject to complex (external) influences has a dynamicsthat may be modeled statistically. The statistical treatment simplifies the complex un-predictable stochastic dynamics of a single system, to the simple predictable dy-namics of an ensemble of systems subject to all possible influences. A random walkon a line is the prototype stochastic process. Over time, the random influence causesthe ensemble of walkers to spread in space and form a Gaussian distribution. Whenthere is a bias in the random walk, the walkers have a constant velocity superim-posed on the spreading of the distribution.While the microscopic dynamics of physical systems is rapid and complex,the macr oscopic behavior of many materials is simple, even static. Before we can un-derstand how complex systems have complex behaviors, we must understand whymater ials can be simple. The origin of si mplicity is an averaging over the fast micr o-scopic dynamics on the time scale of macroscopic observations (the ergodic t heorem)and an averaging over microscopic spatial variat ions. The aver aging can be perf ormedtheoretical ly using an ensemble represent ation of t he physical system that assumesall microscopic states are realized. Using this as an assumption, a statistical treat mentof microscopic states descri bes the macr oscopic equil ibrium behavior of syst ems. Thefi nal part of Secti on 1.3 int roduces concepts that play a cent ral role in the rest of thebook. It di scusses the dif ferences between equilibrium and complex syst ems.Equilibrium systems are divi sibl e and satisfy t he ergodic theor em. Complex systems 1 . 3 1 . 2 1 . 1 01adBARYAM_29412 3/10/02 10:15 AM Page 16are composed out of i nterdependent par ts and violate the ergodic theor em. They havemany degr ees of f reedom whose time dependence is very slow on a microscopic scale.To understand the separation of time scales between fast and slow de-grees of freedom, a two-well system is a useful model. The description of a particletraveling in two wells can be simplified to the dynamics of a two-state (binary vari-able) system. The fast dynamics of the motion within a well is averaged by assumingthat the system visits all states, represented as an ensemble. After taking the aver-age, the dynamics of hopping between the wells is represented explicitly by the dy-namics of a binary variable. The hopping rate depends exponentially on the ratio ofthe energy barrier and the temperature. When the temperature is low enough, thehopping is frozen. Even though the two wells are not in equilibrium with each other,equilibrium continues to hold within a well. The cooling of a two-state system servesas a simple model of a glass transition, where many microscopic degrees of freedombecome frozen at the glass transition temperature.Cellular automata are a general approach to modeling the dynamics ofspatially distributed systems. Expanding the notion of an iterative map of a single vari-able, the variables that are updated are distributed on a lattice in space. The influ-ence between variables is assumed to rely upon local interactions, and is homoge-neous. Space and time are both discretized, and the variables are often simplified toinclude only a few possible states at each site. Various cellular automata can be de-signed to model key properties of physical and biological systems.The equilibr ium state of spati al ly distri buted syst ems can be modeled byfi elds that are treated using statistical ensembles. The simplest is the I sing model, whichcapt ures the simple cooperative behavior found in magnet s and many other systems.Cooper ative behavior is a mechani sm by whi ch microscopi c fast degrees of freedomcan become slow collective degrees of f reedom that violate the ergodic theorem andare visible macroscopicall y. Macroscopic phase tr ansitions are the dynamics of thecooperat ive degrees of f reedom. Cooper ative behavior of many i nteracting elementsis an important aspect of the behavior of complex systems. This should be contrastedto the two-state model (Section 1.4), where the slow dynami cs occurs microscopically. Computer simulations of models such as molecular dynamics or cellularautomata provide important tools for the study of complex systems. Monte Carlo sim-ulations enable the study of ensemble averages without necessarily describing thedynamics of a system. However, they can also be used to study random-walk dy-namics. Minimization methods that use iterative progress to find a local minimum areoften an important aspect of computer simulations. Simulated annealing is a methodthat can help find low energy states on complex energy surfaces.We have treated systems using models without acknowledging explicitlythat our objective is to describe them. All our efforts are designed to map a systemonto a description of the system. For complex systems the description must be quitelong, and the study of descriptions becomes essential. With this recognition, we turn 1 . 8 1 . 7 1 . 6 1 . 5 1 . 4 Co n c e p t u a l o u t l i n e 17# 29412 Cust: AddisonWesley Au: Bar-Yam Pg. No. 17Title: Dynamics Complex SystemsShor t / Normal / Long01adBARYAM_29412 3/10/02 10:15 AM Page 17to information theory. The information contained in a communication, typically astring of characters, may be defined quantitatively as the logarithm of the number ofpossible messages. When different messages have distinct probabilities P in an en-semble, then the information can be identified as ln(P) and the average informationis defined accordingly. Long messages can be modeled using the same concepts asa random walk, and we can use such models to estimate the information containedin human languages such as English.In or der to underst and the relationshi p of information to systems, we mustalso underst and what we can infer from informati on that is pr ovided. The theory of logicis concerned with inference. It is directl y linked t o computation theory, which is con-cerned with the possible (det ermini stic) operations that can be per formed on a str ingof charact ers. All operations on char acter stri ngs can be constructed out of el emen-tary logical (Boolean) operat ions on binary var iables. Using Tu r i n g s model of compu-tation, it is fur ther shown that all computations can be perf ormed by a univer sal Tu r i n gmachine, as l ong as its input character string is suitabl y constructed. Computati on t he-ory i s al so related to our concer n with the dynami cs of physical systems because it ex-plores the set of possible outcomes of discrete deterministic dynamic systems.We return to issues of structure on microscopic and macroscopic scalesby studying fractals that are self-similar geometric objects that embody the conceptof progressively increasing structure on finer and finer length scales. A general ap-proach to the scale dependence of system properties is described by scaling theory.The renormalization group methodology enables the study of scaling properties byrelating a model of a system on one scale with a model of the system on anotherscale. Its use is illustrated by application to the Ising model (Section 1.6), and to thebifurcation route to chaos (Section 1.1). Renormalization helps us understand the ba-sic concept of modeling systems, and formalizes the distinction between relevantand irrelevant microscopic parameters. Relevant parameters are the microscopicparameters that can affect the macroscopic behavior. The concept of universality isthe notion that a whole class of microscopic models will give rise to the same macro-scopic behavior, because many parameters are irrelevant. A conceptually relatedcomputational technique, the multigrid method, is based upon representing a prob-lem on multiple scales.The study of complex systems begins from a set of models that capture aspects of thedynamics of simple or complex systems. These models should be sufficiently gener alto encompass a wide range of possibilities but have sufficient structure to capt ure in-teresting features. An exciting bonus is that even the appar ently simple mo dels dis-cussed in this chapter introduce features that are not typically t reated in the conven-tional science of simple systems, but are appropriate introductions to the dynamics ofcomplex systems.Our t reatment of dynamics will often consider discrete rather thancontinuous time. Analytic t reatments are oft en convenient to for mulate in continu- 1 . 1 0 1 . 9 # 29412 Cust: AddisonWesley Au: Bar-Yam Pg. No. 18Title: Dynamics Complex SystemsShor t / Normal / Long18 I n t r o d u c t i o n a n d P r e l i m i n a r i e s01adBARYAM_29412 3/10/02 10:15 AM Page 18ous variables and differential equations;however, computer simulations are often bestformulated in discrete space-time variables with well-defined intervals. Mor eover, theassumpt ion of a smooth continuum at small scales is not usual ly a convenient star t-ing point for the study of complex systems. We are also generally interested not onlyin one example of a system but rather in a class of systems that differ from each otherbut share a char acter istic st ructure. The elements of such a class of systems are col-lectively known as an ensemble. As we introduce and study mathematical models, weshould recognize that our primary objective is to represent properties of real systems.We must therefore develop an understanding of the nature of models and modeling,and how they can per tain to either simple or complex systems.I t e ra t i ve Ma ps ( a nd Cha os )An iterative map f is a function that evolves the state of a system s in discrete times(t) f(s(t t)) (1.1.1)where s(t) describes the state of the system at time t. For convenience we will gener-ally measure time in units of t which then has the value 1,and time takes integral val-ues star ting from the initial condition at t 0.Ma ny of the com p l ex sys tems we wi ll con s i der in t his text are of the form ofEq .( 1 . 1 . 1 ) ,i f we all ow s to be a gen eral va ri a ble of a rbi t ra ry dimen s i on . The gen era l i tyof i tera t ive maps is discussed at the end of this secti on . We start by con s i dering severa lexamples of i tera t ive maps wh ere s is a single va ri a bl e . We discuss br i ef ly the bi n a r yva ri a ble case, s t1 . Th en we discuss in gre a ter detail two types of maps with s a re a lva ri a bl e , s , linear maps and qu ad ra tic maps. The qu ad ra tic iter a tive map is a sim-ple model that can display com p l ex dy n a m i c s . We assume that an itera tive map may bes t a r ted at any init ial con d i ti on all owed by a spec i f i ed domain of its sys tem va ri a bl e .1 . 1 . 1 Bina ry it era t ive ma psThere are only a few binary iterat ive maps.Question 1.1.1 is a complete enumerationof them.*Que s t i on 1 . 1 . 1 Enumer ate all possible it erative maps where the syst emis described by a single binary variable, s t1.Solut i on 1 . 1 . 1 Ther e are only four possibilities:s(t) 1s(t) 1s(t) s(t 1)(1.1.2)s(t) s(t 1)1 . 1I t e ra t i v e m a p s ( a n d c h a o s ) 19# 29412 Cust: AddisonWesley Au: Bar-Yam Pg. No. 19Title: Dynamics Complex SystemsShor t / Normal / Long*Questions are an integral part of the text. They are designed to promote independent thought. The readeris encouraged to read the question, contemplate or work out an answer and then read the solut ion providedin the text. The continuation of the text assumes that solutions to questions have been read.01adBARYAM_29412 3/10/02 10:15 AM Page 19It is inst r uctive to consider these possibilities in some detail. The main rea-son ther e are so few possibilities is that the form of the iterative map we areusing depends,at most, on the value of the system in the previous time. Thefirst two examples are constants and dont even depend on the value o f thesystem at the previous time. The third map can only be distinguished fromthe first two by obser vation of its behavior when presented with two differ-ent initial conditions.The last of the four maps is the only map that has any sustained dy-namics. It cycles between two values in per petuity. We can think about thisas representing an oscillator. Que s t i on 1 . 1 . 2a. In what way can the map s(t) s(t 1) represent a physical oscillator? b. How can we think of the stat ic map, s(t ) s(t 1), as an oscillator? c. Can we do the same for the constant maps s(t) 1 and s(t) 1?Solut i on 1 . 1 . 2 (a) By looking at the oscillator displacement with a strobeat half-cycle intervals,our measured values can be r epresented by this map.(b) By looking at an oscillator with a st robe at cycle intervals. (c) You mightthink we could, by picking a definite starting phase of the strobe with respectto the oscillat or. However, the constant map ignores the first value, the os-cillator does not. 1 . 1 . 2 Linea r it era t ive ma ps: free mot ion, oscilla t ion, deca ya nd growt hThe simplest example of an iterative map with s real, s , is a constant map:s(t ) s0(1.1.3)No matter what the initial value,this system always takes the par t icular value s0. Theconstant map may seem trivial,however it will be useful to compare the constant mapwith the next class of maps.A linear it er ative map with unit coefficient is a model of free motion or propa-gat ion in space:s(t ) s(t 1) + v (1.1.4)at su cce s s ive times the va lues of s a re sep a ra ted by v, wh i ch plays the role of the vel oc i ty.Que s t i on 1 . 1 . 3 Consider the case of zero velocit ys(t ) s(t 1) (1.1.5)How is this different from the constant map?Solut i on 1 . 1 . 3 The two maps differ in their depen den ce on the initial va lu e . 20 I n t r o d u c t i o n a n d P r e l i m i n a r i e s# 29412 Cust: AddisonWesley Au: Bar-Yam Pg. No. 20Title: Dynamics Complex SystemsShor t / Normal / Long01adBARYAM_29412 3/10/02 10:15 AM Page 20# 29412 Cust: AddisonWesley Au: Bar-Yam Pg. No. 21Title: Dynamics Complex SystemsShor t / Normal / LongRunaway growth or decay is a multiplicative iterative map:s( t ) gs(t 1) (1.1.6)We can gener ate the values of this it er ative map at all times by using the equivalentexpression(1.1.7)which is exponential growth or decay. The it er ative map can be thought of as a se-quence of snapshots of Eq.(1.1.7) at integral time. g 1 reduces this map to the pre-vious case.Que s t i on 1 . 1 . 4 We have seen the case o f free motion, and now jumpedto the case of growth. What happened to accelerated motion? Usually wewould consider acceler ated motion as the next step after motion with a con-stant velocit y. How can we wr ite acceler ated motion as an iter ative map?Solut i on 1 . 1 . 4 The description of accelerated motion requires two var i-ables: position and velocity. The iter ative map would look like:x( t ) x( t 1) + v( t 1)(1.1.8)v(t ) v(t 1) + aThis is a two-variable iterative map. To wr ite this in the notation of Eq.(1.1.1)we would define s as a vector s(t ) (x(t ), v(t )). Que s t i on 1 . 1 . 5 What hap pens in the rightmost exponential expressionin Eq. (1.1.7) when g is negative?Solut i on 1 . 1 . 5 The logarithm of a negat ive number results in a phase i .The t er m i t in the exp onent alt er nates sign ever y time st ep as one wouldexpect from Eq. (1.1.6). At this point,it is convenient to introduce two graphical methods for describingan it erative map. The first is the usual way of plotting the value o f s as a function oftime. This is shown in the left panels of Fig. 1.1.1. The second type of plot ,shown inthe right panels, has a different purpose. This is a plot of the iterative relation s(t ) asa funct ion of s(t 1). On the same axis we also draw the line for the identity maps(t ) s(t 1). These two plots enable us to gr aphically obtain the successive values ofs as follows. Pick a star ting value of s, which we can call s(0). Mark this value on theabscissa. Mark the point on the graph of s(t ) that corresponds to the point whose ab-scissa is s(0),i.e.,the point (s(0), s( 1) ) .Dr aw a hor izontal line to intersect the identit ymap. The int ersection p oint is (s(1), s(1)). Draw a ver tical line back to the iterativemap. This is the p oint ( s(1), s(2)). Successive values of s( t ) are obtained by it eratingthis graphical procedure. A few examples are plotted in the r ight panels of Fig. 1.1.1.In order to discuss the iterative maps it is helpful to recognize sever al features ofthese maps.First,intersect ion points of the identity map and the iterative map are thefixed points of the iter ative map:(1.1.9) s0 f (s0) s(t ) gts0eln(g )ts0I t e ra t i v e m a p s ( a n d c h a o s ) 2101adBARYAM_29412 3/10/02 10:15 AM Page 21Fixed points,not surpr isingly, play an impor tant role in iterative maps. They help usdescribe the state and behavior of the system after many iterations. There are twokinds o f fixed p ointsstable and unstable. Stable fixed p oints are char acter ized byattracting the result of iteration of points that are nearby. Mor e precisely, there exists22 I n t r o d u c t i o n a n d P r e l i m i n a r i e s# 29412 Cust: AddisonWesley Au: Bar-Yam Pg. No. 22Title: Dynamics Complex SystemsShor t / Normal / Long0 1 2 3 4 5 6 7s(t)t0 1 2 3 4 5 6 7s(t)=s(t1) +v(a)(b) s(t)s(t1)