Distributed Control

The Importance of Signals and Boundaries[ 边界 ]

Nothing is less real than realism. Details are confusing. It is only by selection, by elimination, by emphases that we get at the real meaning of things.

--Georgia O’Keefe

Outline

1) Introduction to Distributed Control. (pp. 3-14)

2) Reaction Networks. (pp. 15-25)

3) Urn[缸] models of uncontrolled reactions. (pp. 26-31)

4) Hierarchical[层级] control. (pp. 32-42)

5) Markov models of hierarchical control. (pp. 43-48)

In traditional control theory:An external controller directs a system (“plant”) using signals

generated by the plant and external feedback.

Distributed Control occurs when the control originates internally through interaction of components of the system

Networks of interaction (Reaction Networks) have a central role in the study of Distributed Control.

Introduction to Distributed Control (1)

Introduction to Distributed Control (2)

Adaptive Control contrasted to Distributed Control

Adaptive Control

Distributed Control

Different agents have different controls.

Connections between agents are frequently made and broken.

Fitness = 1/operating cost

Adaptive control in many complex systems, such as flight controllers, is often implemented by a program -- a linked set of IF/THEN rules.

As we will see, reaction nets can also be represented by a linked IF/THEN rules, and they are distributed.

Reaction nets are a natural way to study distributed control.

Introduction to Distributed Control (3)

Introduction to Distributed Control (4)

Reactions in biological cells are primarily controlled through enzymes[ 酶 ] (catalysts[ 催化剂 ]) and a hierarchy of enclosures by semi-permeable membranes (selective filters).

This hierarchy yields adaptive control orders of magnitude better thanwe can obtain with artificial systems.

The details of the hierarchy are immensely complex.

Even comparative physiology[ 比较生理学 ], comparing very simple cells to much more complex organisms, gives only vague ideas about the hierarchy.

[ 半 - 渗透膜 ]

Introduction to Distributed Control (5) Spontaneous[ 自发的 ] Emergence of Levels

no water

waterSpore孢子

Slime mold (Discoideum) [ 粘液菌 ]

When the environment is unfavorable, individual cells aggregate, form boundaries[ 边界 ], and specialize.

aggregationof cells

differentiationof cells[ 细胞分化 ]

single cell

Introduction to Distributed Control (6)

Genetically-specified signal/boundary interactions in plants:

mitochondria[ 线粒体 ](energy conversion), chloroplasts (photosynthesis[ 光合作用 ]) mycorrhizal fungi[ 真菌 ] (nutrient and water uptake) debris-decomposing bacteria (recycling) pollinators[ 授粉者 ] (e.g., bees), seed-transporting organisms (e.g., fruit eaters) predators [ 捕食者 ] (e.g., plant eaters)

Signals and boundaries control the network of interactions at each level of the hierarchy.

It is difficult to analyze these interactions using traditional mathematics.

Introduction to Distributed Control (6)

Genetically-specified signal/boundary interactions in plants:

mitochondria[ 线粒体 ](energy conversion), chloroplasts (photosynthesis[ 光合作用 ]) mycorrhizal fungi[ 真菌 ] (nutrient and water uptake) debris-decomposing bacteria (recycling) pollinators[ 授粉者 ] (e.g., bees), seed-transporting organisms (e.g., fruit eaters) predators [ 捕食者 ] (e.g., plant eaters)

Signals and boundaries control the network of interactions at each level of the hierarchy.

It is difficult to analyze these interactions using traditional mathematics.

Introduction to Distributed Control (6)

Genetically-specified signal/boundary interactions in plants:

mitochondria[ 线粒体 ](energy conversion) chloroplasts (photosynthesis[ 光合作用 ]) mycorrhizal fungi[ 真菌 ] (nutrient and water uptake) debris-decomposing bacteria (recycling) pollinators[ 授粉者 ] (e.g., bees), seed-transporting organisms (e.g., fruit eaters) predators [ 捕食者 ] (e.g., plant eaters)

Signals and boundaries control the network of interactions at each level of the hierarchy.

It is difficult to analyze these interactions using traditional mathematics.

Introduction to Distributed Control (6)

Genetically-specified signal/boundary interactions in plants:

mitochondria[ 线粒体 ](energy conversion) chloroplasts (photosynthesis[ 光合作用 ]) mycorrhizal fungi[ 真菌 ] (nutrient and water uptake) debris-decomposing bacteria (recycling) pollinators[ 授粉者 ] (e.g., bees), seed-transporting organisms (e.g., fruit eaters) predators [ 捕食者 ] (e.g., plant eaters)

Signals and boundaries control the network of interactions at each level of the hierarchy.

It is difficult to analyze these interactions using traditional mathematics.

Introduction to Distributed Control (6)

Genetically-specified signal/boundary interactions in plants:

mitochondria[ 线粒体 ](energy conversion) chloroplasts (photosynthesis[ 光合作用 ]) mycorrhizal fungi[ 真菌 ] (nutrient and water uptake) debris-decomposing bacteria (recycling) pollinators[ 授粉者 ] (e.g., bees) seed-transporting organisms (e.g., fruit eaters) predators [ 捕食者 ] (e.g., plant eaters)

Signals and boundaries control the network of interactions at each level of the hierarchy.

It is difficult to analyze these interactions using traditional mathematics.

Introduction to Distributed Control (6)

Genetically-specified signal/boundary interactions in plants:

mitochondria[ 线粒体 ](energy conversion) chloroplasts (photosynthesis[ 光合作用 ]) mycorrhizal fungi[ 真菌 ] (nutrient and water uptake) debris-decomposing bacteria (recycling) pollinators[ 授粉者 ] (e.g., bees), seed-transporting organisms (e.g., fruit eaters) predators [ 捕食者 ] (e.g., plant eaters)

Signals and boundaries control the network of interactions at each level of the hierarchy.

It is difficult to analyze these interactions using traditional mathematics.

Introduction to Distributed Control (6)

Genetically-specified signal/boundary interactions in plants:

mitochondria[ 线粒体 ](energy conversion) chloroplasts (photosynthesis[ 光合作用 ]) mycorrhizal fungi[ 真菌 ] (nutrient and water uptake) debris-decomposing bacteria (recycling) pollinators[ 授粉者 ] (e.g., bees), seed-transporting organisms (e.g., fruit eaters) predators [ 捕食者 ] (e.g., plant eaters)

Signals and boundaries control the network of interactions at each level of the hierarchy.

It is difficult to analyze these interactions using traditional mathematics.

Introduction to Distributed Control (7)

Systems where signal/boundary interactions are critical:

Even beyond biology, it is important to understand hierarchical distributed control because it occurs in many different complex adaptive systems (cas).

Biological cell (chromosome/protein communication; organelles) Ecosystem Geopolitics [地缘政治 ] Reaction network (with phases and/or membranes[ 膜 ]) Central Nervous System regions Language Markets (tags) Psychology (induction and discovery)

Introduction to Distributed Control (8)Spontaneous Boundary Formation

A boundary is the interface between two distinct sets of reactants.

(E.g., the boundary between Mandarin[ 普通话 ] and Cantonese[ 广东话 ].)

Before

After

Later, boundaries will be represented by tagged urns.

厌水蛋白质不能溶解在水里，但是在油里是可溶解的并且粘在一起

Introduction to Distributed Control (9)

As is usual in theoretical physics, we try to use simplified exploratory models to obtain ideas about where to look in real systems.

Then we formulate critical experiments to test our hypotheses.

As we will see, tagged urn models [ 带标志的缸模型 ] are helpful in studying distributed control.

Introduction to Distributed Control (10)

Tagged urn models of hierarchical control offer the followingadvantages:

1) Complicated hierarchies of enclosures and distributed control are easily presented.

2) There is a relevant mathematics, Markov processes, and a relevant

search technique, the Monte Carlo algorithm.

3) Using Monte Carlo algorithms, it is easy to find communities of reactants[ 反应物 ] (niches[ 小生境 ]) -- communities that persist

because of recirculation[ 再流通 ] and autocatalysis[ 自催化 ].

4) Because all reactants and membranes are presented as strings, the adaptive co-evolution of tagged urn models is easily simulated using a genetic algorithm.

Introduction to Distributed Control (11)

In a complex adaptive system (cas) , where there are multiple interacting agents that learn , control is necessarily distributed.

Boundaries separate agents into communities (modularity), allowing specialization and higher efficiency.

Signals allow one community to partially control others.

Signals and Boundaries are both modified by evolution.

Introduction to Distributed Control (12) Distributed Control of Reaction Nets

All cas can be presented as networks of reactions (rule based agents) and

resource flows.

Reaction networks give us a single formalism for studying both cas and distributed control.

After reviewing reaction networks we will see how boundaries and signals make possible distributed control of networks.

Overview of Reaction Networks (1)

Reaction networks result from spatially distributed sets of interactions

between molecules, signals, and/or resources.

E.g., a collection of reactions of the kind A+B C+D distributed

over a 2-dimensional geometry.

Overview of Reaction Networks (2)

The reactants [ 反应物 ] (e.g. molecules) have tags [ 标志 ] (active sites)

that

determine the reactions that take place.

Reactants come into contact through random elastic collisions.

“Billiard-ball” [ 撞球 ] mechanics.

Reactants in contact react with a probability determined by the reaction rate.

An urn model [ 缸模型 ] of the state of the reaction network allows a Markov matrix analysis.

Overview of Reaction Networks (3)

Objectives

1) Observe the robustness and organization of the evolving reaction networks.

2) Observe the kinds of agents (coordinated sets of reactions), if any, that form through the evolution of tags.

3) Develop a concept of niche [ 小生境 ] (local resource enrichment) suitable for dynamic, perpetually [ 永久地 ] changing networks.

These objectives have a major role in understanding any cas.

Overview of Reaction Networks (4)Emergent Phenomena Expected

1) Boundaries arise through constraints provided by tags on reactants. Boundaries allow the formation of agents with individual histories.

2) Agents become building blocks for still more complex agents.A ‘layered’ use of tags evolves (similar to the membrane, organelle,

cell, organ, … hierarchy of biological cells).

3) Cycles in the reaction net provide locally increased concentrations [ 浓度 ] of reactants.

The resulting niches [ 小生境 ] offer possibilities for agent exploitation.

4) Increasing diversity of the rules, signals, and agents arises as the reaction network evolves (under a genetic algorithm).

A Quick Review of Reaction Networks (1) Reactants

Reactants are classified according to active sites (tags).

Reactants diffuse and collide at random (a ‘billiard ball’ mechanics), undergoing mass

action chemistry depending upon their active sites.

A Quick Review of Reaction Networks

(2) A Binary Reaction

R1 + R2 R3 + R4

A Quick Review of Reaction Networks (3) A Binary Reaction Based on Tags

[ 连接酶 ]

[ 受体 ]

prefix [ 前缀 ]

A Quick Review of Reaction Networks

(4) A Sequence of Binary Reactions

A Quick Review of Reaction Networks

(5) The Effect of Recycling (Material Feedback)

A Quick Review of Reaction Networks

(9) Summarizing: Reactions, Conditions, & Tags

A reaction requires a given combination of tags (active sites) for each

input reactant; any reactant with that combination can serve as that

input.

The combination of tags required for a reaction can be specified by using

using ‘don’t care’ (#) symbols for the non-tag locations (as in a

classifier rule).

A Quick Review of Reaction Networks

(7) Embedding a Simple Ecosystem in a Reaction Network

Substrate [ 培养基 ]: {a, b, c, #}

multiple copies of each element

Plant: a + a + a + b + b + c => aababc;

aa# => aa#

Herbivore [ 食草动物 ]: aa# + b+ b => bb#c + a + a;

bb# => bb#

Carnivore [ 食肉动物 ]: bb# +c + c => cc# + b + b;

cc# => cc#

Bacterium [ 细菌 ]: #c => {elements in string #};

#c => #c

[The two reactions in the bacterium compete]

Uncontrolled Reactions

In uncontrolled reactions, the reactants are uniformly mixed and reactions occur when the reactants undergo elastic collisions.

Ui = reactant type i, i = 1, …, M.ni(t) = number of copies of reactant type i at time t.

N = i n i = total number of copies of reactants, a constant.

pi(t) = ni(t)/N = the proportion of reactant type i at time t.

rij = the proportion of collisions resulting in a reaction between reactantsi and j = the forward reaction rate.

An Urn Model for Uncontrolled Reactions (1)

In this urn model, the number of balls in urn Uj is proportional to pj .

Assume that the reaction between reactants indexed by Uh and Ui produces the products Uj and Uk.

The probability that the products will be produced is given by the reaction equation

pj = pk = rhiphpi

An Urn Model for Uncontrolled Reactions (2)

The state of the system at time t is S(t) = ( n1(t), n2(t), …, nM(t)),= the number of balls in each urn

In the urn model, to go from S(t) to S(t+1), pick two balls at random from the urns and produce the products with probability rhi.

If the products are producedS(t+1) = ( …, nh(t)-1, …, ni(t)-1, …, nj(t)+1, … nk(t)+1, … ),

elseS(t+1) = S(t).

An Urn Model for Uncontrolled Reactions (3)

This simple urn model can be presented as a Markov process with one dimension for each possible distribution of balls in the urns.

Notation: Let [X,Y] be the number of different ways of choosing Y objects from a total of X objects.

There are b = [N+M-1, M-1] distinct distributions of N balls in M urns.

b is easily derived by considering the number of binary numbers of length N+M-1 having exactly M-1 ones,

(For example, with N=3 and M=2, 0100

=> 1 ball in the first urn 2 balls in the second urn.)

An Urn Model for Uncontrolled Reactions (4)

R is a bxb matrix having Ruv as its component at row h and column i.Row u corresponds to one possible distribution of balls in urns,

and column v represents a possible product distribution

Let S(t), the current state, correspond to row u of the matrix.Let S(t+1) = ( …, nh(t)-1, …, ni(t)-1, …, nj(t)+1, … nk(t)+1, … ), a possible result of a random draw from the distribution S(t), correspond to column v.

Then Ruv = rhiphpi.

Note that the distribution corresponding to row u fully specifies phand pi .

Note that several different draws can yield the result S(t+1).

An Urn Model for Uncontrolled Reactions (5)

As is usual with a Markov representation

S(t+T) = S(t)RT

and the equilibrium distribution of reactants is given by the eigenvector S* corresponding to the positive eigenvalue e of the matrix R

S*(t+1) = eS*(t).

A Spatial Urn Model

Each site (square in the array) contains a set of urns representing the reactanttypes present in that area.

The number of balls in each urn gives the local concentration of that reactant type.

Each reactant species (same active sites) is assigned a distinct color.

Diffusion takes place by moving balls at random between the urns.

Urn Models of Tag-based Reactions

Control viaSemi-permeable Membranes

A semi-permeable membrane is a filter. It allows only reactants with specific tagsto diffuse from one side of the membrane to the other.

Outside: High diversity of reactants with low individual concentrations=> Low reaction rates

Semi-permeable boundary filters flow=> Increased reaction rates for selected reactants=> Locally increased concentration of selected reactants

Inside: Increased concentrations=> Catalysts become effective as "switches" determining reaction sequences (much like a computer program)

Outside Inside

Urn Model of a Semi-permeable Membrane

Each urn is assigned entry and exit conditions.

For a ball to enter (exit) an urn under diffusion, its tags must match the

corresponding condition.

Using Reactants to Define Urn Entry/Exit Conditions

A new urn is formed each time the system forms a different kind of reactant with an urn tag.

Reaction Inside a Semi-permeable Membrane

Distributed control results from the control of diffusion provided by boundaries, combined with the control of reactions provided by tags (signals).

Urn Model ofCoupled Membrane-enclosed Reactions

x = 111111y = 111000

Hierarchical Membrane-enclosed Reactions

By controlling concentrations, this hierarchy selects and amplifies a particular sequence of reactions.

Corresponding Hierarchical Urn Model

The suffix on the each urn entry tag specifies the urn(s) from which incoming balls may be drawn.

Review of the Urn Model of Controlled Diffusion

Steps in executing the model:

(1) A ball is chosen at random from one of the urns.

(2) The match between the ball’s tags and the exit tags of an urn determines its probability of

leaving the urn.

(3) The match between the ball’s tags and the entry tags of the other urns determines its

probability of entering another urn.

These steps are repeated to obtain the effect of simultaneous diffusion.

Markov Process Corresponding tothe Urn Model of Controlled Diffusion (1)

h = index of the ball type chosen.x = index of the urn containing the ball y = index of the target urn.

When a ball h moves from urn x to urn y then

S(t+1) = ( …, nhx(t)-1, …,nky(t)+1, … ),

where nhx(t) is the number of balls of type h in urn x at time t.

Markov Process Corresponding tothe Urn Model of Controlled Diffusion (2)

qhxy = the probability that ball h will move from urn x to urn y, as determined by the match scores between the tags.

As with the uncontrolled reactions, the state S(t) at time t is given by the distribution of the balls in the urns.

A Markov matrix D can be used to define this process, where

Duv = qhxyph.

With b diffusion steps followed by c reaction steps, the result is

S(t+b+c) = S(t)DbRc.

Markov Process Corresponding tothe Urn Model of Controlled Diffusion (3)

Because the Markov matrix D is sparse -- most Duv = qhxyph = 0 --

S(t+b+c) = S(t)DbRc can be quickly calculated.

By using states with high occupation probabilities as starting points, Monte Carlo simulations can quickly determine communities of states – niches – with large exit times.

Simple examples of interactions in reaction networks with boundaries and signals:

• mass action and unbalanced flows

• counter-current flows

Some themes:

• persistent patterns in flows

• niche formation and specialization

• diversity and increasing complexity

• modules, motifs, and building blocks

Evolution Generates Feedback and Controlsin Complex Adaptive Systems

Three examples from natural systems:

Niche formation: The “devil’s” garden in Peru.

Octopus “joints” and convergent evolution.

Octopus mimicry of snakes and fish.

Evolution can increase interactions, boundaries, and signals in reaction networks.

Recapitulation [ 摘要重述 ]

Hypotheses: Local concentrations of resources, induced by feedback and recycling, provide opportunities for the formation and adaptive radiation of agents.

This process of agent formation leads to increasing diversity of agents and progressively larger amounts of resources “tied up” in agents.

Under ‘tranquil[ 安静的 ]’ conditions, increasing agent specialization should be observed.

If any of these hypotheses can be established we will have substantiallyincreased our understanding of cas.

Outline

Details

A Quick Review of Reaction Networks

(6) A Rule-based Version of a Reaction Network

t1##...# & t2##...# => t3##...#

A Quick Review of Reaction Networks (8) Formalisms for Reaction Nets

Difference Equations

x(t+1) = x(t) – rx(t)y(t) + r*u(t)v(t),

where x, y. … are concentrations and r, r*, … are reaction rates.

Urn Model

Billiard Ball [ 撞球 ] Mechanics (Markov Process)

Rule-based Signal Processing a1 a 2 a 3 … a k & b1b2… bk a 1 …a k b 1 …b k

Urn Models of Tag-based Reactions (1)