Introduction to CHAOS Larry Liebovitch, Ph.D. Florida Atlantic University 2004

Introduction to CHAOSIntroduction to CHAOS

Larry Liebovitch, Ph.D.Larry Liebovitch, Ph.D.

Florida Atlantic UniversityFlorida Atlantic University

20042004

These two sets of data have the same

mean variance power spectrum

Data 1 RANDOMrandom

x(n) = RND

CHAOSDeterministic

x(n+1) = 3.95 x(n) [1-x(n)]

Data 2

etc.

Data 1 RANDOMrandom

x(n) = RND

Data 2 CHAOSdeterministic

x(n+1) = 3.95 x(n) [1-x(n)]

x(n+1)

x(n)

DefinitionCHAOS

Deterministicpredict that value

these values

CHAOS

Small Number of Variables

x(n+1) = f(x(n), x(n-1), x(n-2))

Definition

DefinitionCHAOS

Complex Output

PropertiesCHAOS

Phase Space is Low Dimensional

phase spaced , random d = 1, chaos

PropertiesCHAOS

Sensitivity to Initial Conditions

nearly identicalinitial values

very differentfinal values

PropertiesCHAOS

Bifurcationssmall change in a parameter

one pattern another pattern

Time Series

X(t)

Y(t)

Z(t)

embedding

Phase Space

X(t)

Z(t)

phase space set

Y(t)

Attractors in Phase SpaceLogistic Equation

X(n+1)

X(n)

X(n+1) = 3.95 X(n) [1-X(n)]

Attractors in Phase Space

Lorenz Equations

X(t)

Z(t)

Y(t)

X(n+1)

X(n)

Logistic Equationphase spacetime series d<1

The number of independent variables = smallest integer >

the fractal dimension of the attractor

d < 1, therefore, the equation of the time series that produced this attractor depends on 1 independent variable.

Lorenz Equationsphase spacetime series d =2.03

The number of independent variables = smallest integer >

the fractal dimension of the attractor

d = 2.03, therefore, the equation of the time series that produced this attractor depends on 3 independent variables.

X(t)

Z(t)

Y(t)

X(n+1)

n

Data 1 time series

phase spaced

Since ,the time series was producedby a randommechanism.

d

Data 2 time series

phase spaced = 1

Since d = 1,the time series

was produced by a deterministic

mechanism.

Constructed by direct measurement:Phase Space

Each point in the phase space set has coordinatesX(t), Y(t), Z(t)

Measure X(t), Y(t), Z(t) Z(t)

X(t) Y(t)

Constructed from one variablePhase Space

Takens’ TheoremTakens 1981 In Dynamical Systems and Turbulence Ed. Rand & Young, Springer-Verlag, pp. 366 - 381

X(t+ t)

X(t+2 t)

X(t)

Each point in thephase space sethas coordinatesX(t), X(t + t), X(t+2 t)

velo

city

(cm

/sec

)

Position and Velocity of the Surface of a Hair Cell in the Inner Ear

Teich et al. 1989 Acta Otolaryngol (Stockh), Suppl. 467 ;265 - 279

10-1

-10-1

-10-4 3 x 10-5displacement (cm)

stimulus = 171 Hz

velo

city

(cm

/sec

)

Position and Velocity of the Surface of a Hair Cell in the Inner Ear

Teich et al. 1989 Acta Otolaryngol (Stockh), Suppl. 467 ;265 - 279

5 x 10-6displacement (cm)

stimulus = 610 Hz

-3 x 10-2

3 x 10-2

-2 x 10-5

Data 1RANDOM

x(n) = RND

fractal demension of the phase space set

fra

cta

l dim

en

sio

n

of

ph

as

e s

pac

e s

etembedding dimension = number of values of the data taken at a time to

produce the phase space set

Data 2 CHAOSdeterministic

x(n+1) = 3.95 x(n) [1 - x(n)]

fra

cta

l dim

en

sio

n

of

ph

as

e s

pac

e s

et

fractal demension of the phase space set = 1

embedding dimension = number of values of the data taken at a time to

produce the phase space set

microelectrode

chick heart cell

current source

voltmeter

Chick Heart Cells

v

Glass, Guevara, Bélair & Shrier.1984 Phys. Rev. A29:1348 - 1357

Spontaneous Beating, No External Stlimulation

Chick Heart Cells

voltage

time

Periodically Stimulated2 stimulations - 1 beat

Chick Heart Cells

2:1

Chick Heart Cells

1:1

Periodically Stimulated1 stimulation - 1 beat

Chick Heart Cells

2:3

Periodically Stimulated2 stimulations - 3 beats

periodic stimulation - chaotic response

The Pattern of Beatingof Chick Heart Cells

Glass, Guevara, Bélair & Shrier.1984 Phys. Rev. A29:1348 - 1357

= phase of the beat with respect to the stimulus

The Pattern of Beating of Chick Heart Cells continued

phase vs. previous phase

0.5

0 0.5 1.0

1.0

0 0.5 1.0

i + 1

experiment

i

theory (circle map)

The Pattern of Beatingof Chick Heart Cells

Glass, Guevara, Belair & Shrier.1984 Phys. Rev. A29:1348 - 1357

Since the phase space set is 1-dimensional, the timing between the beats of thesecells can be described by a deterministic relationship.

ProcedureProcedure Time seriesTime series

e.g. voltage as a function of timee.g. voltage as a function of time

Turn the Time Series into a Turn the Time Series into a Geometric ObjectGeometric ObjectThis is called This is called embeddingembedding..

ProcedureProcedure Determine the Topological Determine the Topological

Properties of this ObjectProperties of this ObjectEspecially, the Especially, the fractal dimensionfractal dimension..

High Fractal DimensionHigh Fractal Dimension = Random = chance= Random = chance Low Fractal DimensionLow Fractal Dimension = Chaos = deterministic= Chaos = deterministic

The Fractal Dimension The Fractal Dimension

isis NOTNOT equal to equal to

The Fractal DimensionThe Fractal Dimension

Fractal Dimension:Fractal Dimension:How many new pieces of the How many new pieces of the Time Series are found when Time Series are found when viewed at finer time resolution.viewed at finer time resolution.

X

time

d

Fractal Dimension:Fractal Dimension:The Dimension of the Attractor in The Dimension of the Attractor in

Phase Space is related to thePhase Space is related to theNumber of Independent Number of Independent Variables. Variables.

X

time

d

x(t) x(t+ t)

x(t+2 t)

Mechanism that Generated the DataMechanism that Generated the DataChanced(phase space set)

Determinismd(phase space set) = low

Data

x(t)

t

?

C O L D

LorenzLorenz1963 J. Atmos. Sci. 20:13-1411963 J. Atmos. Sci. 20:13-141

Model

HOT

(Rayleigh, Saltzman)


Equations

X = speed of the convective X = speed of the convective circulation circulation X > 0 clockwise, X > 0 clockwise, X < 0 counterclockwiseX < 0 counterclockwise

Y = temperature difference Y = temperature difference between rising and falling between rising and falling fluidfluid

Equations


Z = bottom to top Z = bottom to top temperature minus the temperature minus the linear gradientlinear gradient

Equations


Phase Space


Z

X Y

Lorenz AttractorLorenz Attractor

X < 0 X > 0

cylinder of air rotating counter-clockwise

cylinder of air rotating clockwise

IXtop(t) - Xbottom(t)I e t = Liapunov Exponent

Sensitivity to Initial ConditionsSensitivity to Initial ConditionsLorenz EquationsLorenz Equations

X(t)

X= 1.00001

Initial Condition:

differentsame

X(t)

X= 1.

0

0

Deterministic, Non-ChaoticDeterministic, Non-Chaotic

X(n+1) = f {X(n)}

Accuracy of values computed for X(n):

1.736 2.345 3.2545.455 4.876 4.2343.212

Deterministic, ChaoticDeterministic, Chaotic

X(n+1) = f {X(n)}

Accuracy of values computed for X(n):

3.455 3.45? 3.4?? 3.??? ? ? ?

Initial ConditionsInitial Conditions X(t X(t00), Y(t), Y(t00), Z(t), Z(t00)...)...

Clockwork Universedetermimistic non-chaotic

Cancomputeall future

X(t), Y(t), Z(t)...Equations

Initial ConditionsInitial Conditions X(t X(t00), Y(t), Y(t00), Z(t), Z(t00)...)...

Chaotic Universedetermimistic chaotic

sensitivityto initial

conditionsCan notcomputeall future

X(t), Y(t), Z(t)...Equations

Lorenz Strange AttractorLorenz Strange Attractor

Trajectories from outside:

pulled TOWARDS it

why its called an attractor

starting away:

Lorenz Strange AttractorLorenz Strange Attractor

Trajectories on the attractor:

pushed APART from each othersensitivity to initial

conditions

starting on:

““Strange”Strange”attractor is fractalattractor is fractal

phase space set

not strange strange

““Chaotic”Chaotic”sensitivity to initial conditionssensitivity to initial conditions

time series

not chaotic chaotic

X(t)

t

X(t)

t

Shadowing TheoremShadowing Theorem

If the errors at each integration step are small, there is an EXACT trajectory which lies within a small distance of the errorfull trajectory that we calculated

There is an INFINITE number of trajectories on the attractor. When we go off the attractor, we are sucked back down exponentially fast. We’re on an exact trajectory, just not on the one we thought we were on.

Shadowing TheoremShadowing Theorem

4. We are on a “real”

trajectory.

3. Pulled backtowards the

attractor.

2. Error pushesus off

the attractor.

1. We start here.

Trajectorythat we actually

compute.

Trajectory that we

are trying to

compute.

Sensitivity to initial Sensitivity to initial conditions means that the conditions means that the

conditions of an experiment conditions of an experiment can be quite can be quite similarsimilar, but , but

that the results can be quite that the results can be quite differentdifferent..

TUESDAY

++

10 µlArT

10 µl

WEDNESDAY

ArT

++

A = 3.22

X(n)

n

X(n + 1) = A X(n) [1 -X (n)]

A = 3.42

X(n)

n

X(n + 1) = A X(n) [1 -X (n)]

A = 3.62

X(n)

n

Bifurcation

Start with one value of A. Start with x(1) = 0.5. Use the equation to compute x(2) from x(1). Use the equation to compute x(3) from x(2) and so on... up to x(300).

x(n + 1) = A x(n) [1 -x(n)]

Ignore x(1) to x(50), these are the transient values off of the attractor. Plot x(51) to x(300) on the Y-axis over the value of A on the X-axis. Change the value of A, and repeat the procedure again.

x(n + 1) = A x(n) [1 -x(n)]

Sudden changes of the pattern indicate bifurcations ( )

x(n)x(n)

The energy in glucose is transfered to ATP. ATP is used as an energy source

to drive biochemical reactions.

Glycolysis

+- -

periodic

TheoryMarkus and Hess 1985 Arch. Biol. Med. Exp. 18:261-271

Glycolysis

time

sugar input ATP output

chaotic

time

time time

ExperimentsHess and Markus 1987 Trends. Biomed. Sci. 12:45-48

cell-free extracts from baker’s yeast

Glycolysis

ATP measured by fluorescence glucose input time

ExperimentsHess and Markus 1987 Trends. Biomed. Sci. 12:45-48

Periodicfl

uo

resc

ence

Glycolysis

Vin

GlycolysisExperimentsHess and Markus 1987 Trends. Biomed. Sci. 12:45-48

Chaotic

20 min

GlycolysisMarkus et al. 1985. Biophys. Chem 22:95-105

Bifurcation Diagram

chaos

theory

experiment

GlycolysisMarkus et al. 1985. Biophys. Chem 22:95-105

ADP measured at the same phase each time of the input sugar flow cycle(ATP is related to ADP)

period of the input sugar flow cycle

# =

period of the ATP concentration

frequency of the input sugar flow cycle

Phase TransitionsHaken 1983 Synergetics: An Introduction Springer-Verlag

Kelso 1995 Dynamic Patterns MIT Press

Tap the left index fingerin-phase with the tickof the metronome.

Try to tap the right index

finger out-of-phase with the

tick of the metronome.



As the frequency of the metronome increases, the right finger shifts from out-of-phase to in-phase motion.

Position of Right Index FingerPosition of Left Index Finger

A. TIME SERIES



ADD

ABD

Position of Right Index Finger

360o

0o

B. POINT ESTIMATE OF RELATIVE PHASE

180o

Self-Organized Phase TransitionsHaken 1983 Synergetics: An Introduction Springer-Verlag


2 sec

This bifurcation can be explained as a change in a potential energy

function similar to the change which

occurs in a physical phase

transition.

syst

em p

ote

nti

al

scal

ing

par

amet

er

Phase TransitionHaken 1983 Synergetics: An Introduction

Springer-Verlag Kelso 1995 Dynamic Patterns MIT Press

Small changes in parameters can produce large changes in behavior.

+

10cc ArT

++

9cc ArT

Bifurcations can be used to test if a system is deterministic.

Deterministic Mathematical Model Experiment

observed bifurcationspredicted bifurcations

Match ?

The fractal dimension of the phase space set tells us if the data was

generated by a random or deterministic mechanism.

ExperimentalDatax(t)

t

X(t+ t)Phase Space

Set

X(t)


generated by a random or a deterministic mechanism.

Mechanism that generated the experimental data.

Deterministic Random

d = low d


generated by a random or a deterministic mechanism.

EpidemicsSchaffer and Kot 1986 Chaos ed. Holden,

Princeton Univ. Press

400015000

0 0

measlesNew York

time series:

phase space:

chickenpox

EpidemicsOlsen and Schaffer 1990 Science 249:499-504

dimension of attractor in phase space

measles chickenpox

Kobenhavn 3.1 3.4 Milwaukee 2.6 3.2St. Louis 2.2 2.7New York 2.7 3.3


SEIR models - 4 independent variables

S susceptible E exposed, but not yet infectious I infectious R recovered


Conclusion: measles: chaotic chickenpox: noisy yearly cycle

time series: voltageKaplan and Cohen 1990 Circ. Res. 67:886-892

normal fibrillation death

D = 1chaos

D = random

Phase spaceV(t), V(t+ t)

ElectrocardiogramECG: Electrical recording of the

muscle activity of the heart.

8

time series: voltageBabloyantz and Destexhe 1988 Biol. Cybern. 58:203-211

normal

D = 6chaos





time series: time between heartbeatsBabloyantz and Destexhe 1988 Biol. Cybern. 58:203-211

normal

D = 6chaos

fibrillation deathD = 4chaos

induced arrhythmiasD = 3chaos

Evans, Khan, Garfinkel, Kass, Albano, and Diamond 1989 Circ. Suppl. 80:II-134

Zbilut, Mayer-Kress, Sobotka, O’Toole and Thomas 1989 Biol. Cybern, 61:371-381

ElectroencephalogramEEG: Electrical recording of the nerve

activity of the brain.Mayer-Kress and Layne 1987 Ann. N.Y. Acad. Sci. 504:62-78

time series: V(t) phase space:

D=8 chaos

V(t)

V(t+ t)

Rapp, Bashore, Martinerie, Albano, Zimmerman, and Mees 1989 Brain Topography 2:99-118

Babloyantz and Destexhe 1988 In: From Chemical to Biological Organization ed. Markus, Muller, and Nicolis, Springer-Verlag

Xu and Xu 1988 Bull. Math. Biol. 5:559-565


activity of the brain.

Different groups find different dimensions

under the same experimental conditions.


activity of the brain.

mental task

quiet awake, eyes closed

quiet sleep

brain virus: Creutzfeld- Jakob

Epilepsy: petit mal

meditation: Qi-kong


activity of the brain.perhaps:High Dimension

Low Dimension

Random Markov

How to compute the next x(n):Each t pick a random number 0 < R < 1

If open, and R < pc, then close.

If closed, and R < po, then open.

Random Markov

t

closed

If closed:probability to open in thenext t=po

If open:probability to close in the next t = pcopen

Deterministic Iterated MapLiebovitch & Tóth 1991 J. Theor. Biol. 148:243-267

x(n) = the current at time nx(n+1) = f (x(n))

open

closed

x(n+1)

x(n)

0 x(1) 0 x(2)0

x(3)

0

x(2)

Deterministic Iterated MapLiebovitch & Tóth 1991 J. Theor. Biol. 148:243-267

How to compute the next x(n):

Tacoma Narrows Bridge

Thursday November 7, 1940Good modern review (explaining why the explanation given in physics textbooks is wrong): Billah and Scanlan 1991 Am. J. Phys. 59:118-124


Equation of simple, forced resonance:x + Ax + Bx = f ( t )

Equation of flutter that destroyed the Tacoma Narrows Bridge:

x + Ax + Bx = f ( x, x )


Scanlan and Vellozzi 1980 in Long Span Bridges ed. Cohen and Birdsall pp. 247-263 NYAS

Wind Tunnel Tests

AIRFOILORIGIONAL

TACOMA NARROWS(


The drag on an airplane wing (A) increases with wind speed.

Wind Tunnel Tests

0.3

0.2

0.1

0

0.1

0.2

A2OTN

U NB

A

*The drag on the OTN (original Tacoma Narrows) bridge changes sign as the wind speed increases, it enters into positive feedback.

Like a small molecule, relentlessly kicked by the surrounding heat fromone state to another.

The change of states is driven bychance kT thermal fluctuations.

CLOSEDrandom

OPEN

ener

gy

Random

DeterministicLike a lilttle mechanical machine with sticks and springs.The change of states is driven by coherent motions that result from the structure and the atomic, electrostatic, and hydrophobic forces in the channel protein.

CLOSED OPEN

ener

gy

deterministic

Analyzing Experimental Data

In principle, you can tell if thedata was generated by a random or a deterministic mechanism.

The Good News:

Analyzing Experimental Data

In practice, it isn’t easy.

The Bad News:

Need Lots of Data

• Very large data sets: 10d?• Sampling rate must cover the attractor evenly.

Sample too often: only see 1-d trajectories.

Sample too rarely; don’t see the attractor at all.

Why it’s Hard to Tell Random from Deterministic Mechanisms


Analyzing the Data is Tricky• Choice of lag time t for the

embedding.– lag too small: the variable

doesn’t change enough, derivatives not accurate.

– lag too long: the variable changes too much,

derivatives not accurate.• Method of evaluating the dimension.

Mathematics is Not Known• Embedding theorems are only proved

for smooth time series.


How Many Times Series Values?

N = Number of valuesin the time series

needed to correctlyevaluate the dimension

of an attractorof dimension D

NwhenD = 6


Smith 1988Phys. Lett. A133:283 42D 5,000,000,000

Wolff et al. 1985Physica D16:285 30D 700,000,000

Wolf et al. 1985Physica D16:285 10D 1,000,000


Nerenberg & Essex 1990Phys. Rev. A42:7065

D+22

_______1________

kd1/2[A In (k)](D+2)/2

D/22(k-1) ((D+4)/2) (1/2) ((D+3)/2)

x[ ]

200,000


Ding et al. 1993Phys. Rev. Lett. 70:3872 10D/2

(D/2)! D/2

10

1,000

Gershenfeld 1990 preprint 2D

Lorenz

t 0X(t+ t)

X(t)

Lorenz

X(t+ t)

X(t)

t Just Rightt correlation time

Lorenz

X(t+ t)

X(t)

8t

Takens’ TheoremIf

X(t+ t)

X(t)

Then,the lag plot constructed from the data

dX(t)dt

X(t)

Is a linear transformation of the real phase space

dX(t)dt

because

X(t+ t) - X(t) t

Since the fractal dimension is invariant under a linear

transformation, the fractal dimension of the lag plot is

equal to the fractal dimension of the real phase space set.

Takens’ Theorem

If the data does not satisfy these assumptions then we are not guaranteed that the fractal dimension of the lag plot is equal to the fracfal

dimension of the real phase space set.

The ion channel current is not smooth, it is fractal (bursts within bursts) and therefore

not differentiable.

Thus the assumptions of the theroem are not met and we are not guaranteed that the fractal dimension of the lag plot is equal to the fractal

dimension of the real phase space set.

For example:

Osborne & Provenzale 1989 Physica D35:381

They used a Fourier series to generate a fractal time series whose

power spectra was 1/f . They randomized the phases of the terms

in the Fourier series so that the fractal dimension of the real phase space set was infinite. But, they found that the

fractal dimension of the lag plots was as low as 1.

For example:

RANDOMLY pick numbers

Pathological example where an infinite dimensional random process

has a LOW dimension attractor

6 6 6 6 6 6 6

6 6 6 6 6 6 6

6 6 6 6 6 6 6

6 6 6 6 6 6 6

Time Series: 6, 6, 6, 6, 6, 6, 6, 6 ... Phase Space:

Pathological example where an infinite dimensional random process

has a LOW dimension attractor

D = 0

6

6 6

Organization of the Vectors in the Phase Space Set

Kaplan and Glass 1992 Phys. Rev. Lett. 68:427-430

small

average direction

Random no uniform flow

Organization of the Vectors in the Phase Space Set

Kaplan and Glass 1992 Phys. Rev. Lett. 68:427-430

large

Deterministic

average direction

uniform flow

Surrogate Data SetTheiler et al. 1992 Physica D58:77-94

original phase space set

surrogate phase space set

same

original time series

surrogate time series

RANDOM

same first ordercorrelations

higher orders scrambled

Surrogate Data SetTheiler et al. 1992 Physica D58:77-94

surrogate phase space set

different

surrogate time series

same first ordercorrelations

higher orders scrambled

DETERMINISTIC

original phase space set

original time series

Time Series phase space

Experiments

Dimension

Low = deterministicHigh = random

examples: ECG, EEG

WEAK

vary a parameter

Experiments

predicted by a nonlinear

model

STRONGsee behavior

electrical stimulation of cells, biochemical reactions

examples:

Control

system outputNon-Chaotic System

control parameter

Control

system outputChaotic System

control parameter

Control of Chaoslight intensity of a laser

Roy et al. 1992 Phys. Rev. Lett. 68:1259-1262

NO CONTROL

0 0.5 msec

Intensity



CONTROL

0 0.2 msec

Intensity

Control



CONTROL

0 0.2 msec

Intensity

Control

Control of Chaosmotion of a magnetoelastic ribbon

Ditto, Rauseo, and Spano 1990 Phys. Rev. Lett. 65:3211-3214

electromagnets

magnetoelasticribbon

B = 0

B > B1





sensor

X

Xn = X (t = nT)

2 TB = Bo sin ( t)



iterationnumber

0 - 2359

2360 - 4799

4800 - 7099

7100 - 10000

none

period 1

period 2

period 1

control



4.5

4.0

3.5

3.0

2.50 2000 4000 6000 8000 10000

Iteration Number

Xn

Control of Biological Systems

The Old WayBrute Force Control.

BIG machine

BIG powerHeart

Amps

Control of Biological Systems

The New WayCleverly timed, delicate pulses.

little machine

little power

mA

Heart

The Old WayForces drive the system between stable states.

How do we think of biological systems?


Force D Force E

Stable State B

Stable State A Stable State C


The New Way

Hanging around for a

while in one condition

forces the system into

another condition.

Dynamics of A

Dynamics of B


Unstable State B

Unstable State A Unstable State C

Summary of Chaos

FEW INDEPENDENT VARIABLES

Behavior is so complex that it mimics random behavior.

Summary of Chaos

The value of the variables at the next instant in time can be calculated from their values at

the previous instant in time.

xi (t+ t) = f (xi (t))

DYNAMICAL SYSTEMDETERMINISTIC

Summary of Chaos

x1(t+ t) - x2(t+ t) = Ae t

SENSITIVITY TO INITIAL CONDITIONS

NOT PREDICTABLE IN THE LONG RUN

Summary of Chaos

STRANGE ATTRACTOR

Phase space is low dimensional (often fractal).

Books About Chaos

J. GleickChaos: Making a New Science 1987 Viking

introductory

Books About Chaos

F. C. MoonChaotic and Fractal Dynamics 1992 John Wiley & Sons

intermediate mathematics

Books About Chaos

J. Guckenheimer & P. Holmes Nonlinear Oscillations, Dynamical Systems, and Bifurcations of Vector Fields 1983 Springer-VerlagE. Ott Chaos in Dynamical Systems 1993 Cambridge Univ. Press

advanced mathematics

A. V. Holden Chaos 1986 Princeton Univ. Press

E. & L. Moskilde Complexity, Chaos and Biological Evolution 1991 Plenum

reviews of chaos in biologyBooks About Chaos

Books About Chaos

J. Bassingthwaighte, L. Liebovitch, & B. West Fractal Physiology 1994 Oxford Univ. Press

reviews of chaos in biology

Documents

Introduction to CHAOS Larry Liebovitch, Ph.D. Florida Atlantic University 2004