Complex Network Flow Architecture: A cartoon guide€¦ · Complex Network Architecture: A cartoon...

Error/flow control

Relay/MUX

Global

E/F control

Relay/MUX

Physical

Application

E/F control

Relay/MUX

Complex

Network

Architecture:

A cartoon guide

Reactions

Protein level

Reactions

RNA level

Reactions

DNA level

John Doyle

John G Braun Professor

Control and Dynamical Systems

BioEngineering, Electrical Engineering

Caltech

Outline

• Not merely “complexity, networks,

abstraction, recursion, modularity,…”

• But very specific forms of these.

• Formal methods have great potential

• Illustrate with case studies and cartoons:

Internet versus bacterial biosphere

• Implicitly: importance of formal methods, not

merely modeling and simulation

Error/flow control

Relay/MUX

Global

E/F control

Relay/MUX

Physical

Application

E/F control

Relay/MUX

Reactions

Protein level

Reactions

RNA level

Reactions

DNA level

Huge range of dynamics

• Spatial

• Temporal

Bewildering

w/out clear

grasp of

layered

architecture

Network Math and Engineering

(NetME) Challenges

• Predictive modeling, simulation, and analysis of

complex systems in technology and nature

• Theoretical foundation for design of network

architectures

• Balance rigor/relevance, integrative/coherent

• Model/simulate is critical but limited

– Predicting rare but catastrophic events

– Design, not merely analysis

– Managing complexity and uncertainty

“Architecture”

• Most persistent, ubiquitous, and global features

of organization

• Constrains what is possible for good or bad

• Platform that enables (or prevents) innovation,

sustainability, etc,

• Internet, biology, energy, manufacturing,

transportation, water, food, waste, law, etc

• Existing architectures are unsustainable

• Theoretical foundation is fragmented,

incoherent, incomplete

Stochastics in Biology

• Arkin, Gillespie, Petzold, Khammash, El-Samad, Munsky,

Paulsson, Vinnicombe, many others…

• Noise in the cellular environment

‣ Elowitz, van Oudenaarden, Collins, Swain, Xie, Elston, ...

• Stochastic Monte Carlo Simulation

‣ Kurtz, Gibson, Bruck, Anderson, Rathinam, Cao, Salis, Kaznessis, ...

• Statistical moment computations

‣ Hespanha, Singh, Verghese, Gomez-Uribe, Kimura

• Density function computations

‣McNamara, Sidje, ...

• Stochastic differential equation approximations

‣ van Kampen, Kurtz, Elf, Ehrenberg,...

• Spatial stochastic models and tools

‣ Elf, Iglesias,…

Very incomplete, idiosyncratic list

Other Influences

• Internet (Kelly/Low, Willinger, Clark, Wroclawski, Day, Chang, etc etc)

• Biology/Medicine (Savageau, G&K, Mattick, Csete, Arkin, Alon, Caporale, de Duve, Exerc Physio, Acute Care, etc etc…)

• Architecture (Alexander, Salingeros,…)

• Aerospace (many, Maier is a good book)

• Philosophy/History (Fox Keller, Jablonka&Lamb)

• Physics/ecology (Carlson)

• Management (Baldwin,…)

• Resilience/Safety/Security Engineering/Economics(Wood, Anderson, Leveson, …)

Biology versus the Internet

Similarities

• Evolvable architecture

• Robust yet fragile

• Constraints/deconstrain

• Layering, modularity

• Hourglass with bowties

• Feedback

• Dynamic, stochastic

• Distributed/decentralized• Not scale-free, edge-of-chaos, self-

organized criticality, etc

Differences

• Metabolism

• Materials and energy

• Autocatalytic feedback

• Feedback complexity

• Development and regeneration

• >4B years of evolution

• How the parts work?

Similarities

• Feedback

• Dynamics

Differences

• Metabolism

Focus on

bacterial biosphere

Robust Yet Fragile

Question: Human complexity

Efficient, flexible metabolism

Regeneration & renewal

Rich microbial symbionts and

Immune systems

Complex societies

Advanced technologies

Obesity and diabetes

Cancer

Parasites, infection

Inflammation, Auto-Im.

Epidemics, war, …

Catastrophic failures

Robust Yet Fragile

Mechanism?

Efficient, flexible metabolism

Regeneration & renewal

Fat accumulation

Insulin resistance

Inflammation

Obesity and diabetes

Cancer

Fat accumulation

Insulin resistance

Inflammation

Fluctuating

energy

Static

energy

Robust Yet Fragile

Implications/

Generalizations

Efficient, flexible metabolism Rich microbial symbionts and Immune systems Regeneration & renewal Complex societies Advanced technologies

Obesity and diabetes Parasites, infection Inflammation, Auto-Im. Cancer Epidemics, war, … Catastrophic failures

• Fragility = Hijacking, side effects, unintended… of mechanisms evolved for robustness

• Complexity is driven by control, robust/fragile tradeoffs

• Math: New robust/fragile conservation laws

computer Wireless

router

Optical

router

Physical

server

Switch

MAC MAC

Pt to Pt Pt to Pt

control

Functions

Resources

LocalLocal

Non-networked Systems

control

Geographically local

Semantic (functionally) local

Network requirements

Resources

Diverse, Distributed Functions

Local Local Resources Local

Global, universal control

Layered solution

Universal controlConstraints

Diverse Resources

Diverse Functions

That deconstrain

computer Wireless

router

Physical

Switch

computer Wireless

router

Physical

Switch

Error/flow control

Relaying/Multiplexing

Error/flow control

Relaying/MultiplexingLocal

Global

Differ in

• Details

• Scope

Wireless

router

Optical

router

Physical

server

Pt to Pt

Wireless

router

Optical

router

Physical

server

Pt to Pt

Error/flow control

Relay/MUX

Error/flow control

Relay/MUX

Global

computer Wireless

router

Optical

router

Physical

server

Switch

MAC MAC

Pt to Pt Pt to Pt

Error/flow control

Relay/MUX

Global

control

Relay/MUX

Physical

Application

control

Relay/MUX

Recursive control structure

Global

Physical

LocalLocal

Layered control

Layered solution

Layered control

How many layers are there?

As many as you need.

Local Resources

Layered solution

And layers have sublayers

Physical

CircuitCircuitCircuit

LogicalInstructions

And layers have sublayers

Physical

LogicalInstructions

Physical

LogicalInstructions

Layered control

Layered solution

Physical

LogicalInstructions

• Spatial

• Temporal

Bewildering w/out

clear grasp of

layered architecture

Physical

Switch

MAC MAC

Pt to Pt Pt to Pt

telephony

Diverse applications

telephony

Pathways (Bell)

Communications

Layers (Net)

ComputerAncient network

architecture:

“Bell-heads versus

Net-heads”

Operating

systems

Phone systems

Cyber-Physical Theories

• Thermodynamics

• Communications

• Control

• Computation

Physical

• Thermodynamics

• Communications

• Control

• Computation

• Thermodynamics

• Communications

• Control

• Computation

Internet Bacteria

Case studies

Physical

• Thermodynamics

• Communications

• Control

• Computation

• Thermodynamics

• Communications

• Control

• Computation

Promising unifications

Resources

Deconstrained

Applications

Deconstrained

Theoretical framework: Constraints that deconstrain

• Optimization

• Optimal control

• Robust control

• Game theory

• Network coding

2 2min

arg max , ,

arg max ,s sv

R R dt

x c x c

x v p p x c

Physical

ApplicationArchitecture

is not graph

topology.

Architecture

facilitates

arbitrary

graphs.

Similarities

• Feedback

• Dynamics

RNADNA Protein

From Pathways

Metabolic

pathways

“Central dogma” Network

architecture?

To Layers?

Computational Resources

Diverse Physiological Functions

Layered Brain (Hawkins)?

Computational Resources

Diverse Physiological Functions

Prediction

Actions

Catabolism

Carriers

Nucleotides

Inside every cell

Core metabolic bowtie

Catabolism

Carriers

Nucleotides

metabolism

Inside every

cell ( 1030)

Catabolism

Carriers

Catabolism

TCAPyr

F1-6BP

TCAPyr

F1-6BP

TCAPyr

F1-6BP

metabolites

Enzymatically

catalyzed reactions

F1-6BP

PEP Pyr

TCAPyr

F1-6BP

Autocatalytic

Carriers

TCAPyr

F1-6BP

Autocatalytic

produced

consumed

Rest of cell

Carriers

TCAPyr

F1-6BP

Reactions

Proteins

Control?

F1-6BP

PEP Pyr

Control

TCAPyr

F1-6BP

PEP Pyr

If we drew the feedback loops the

diagram would be unreadable.

F1-6BP

PEP Pyr

Mass &Reaction

Energyflux

Balance

dxSv x

Stoichiometry

matrix

Regulation of enzyme levels by

transcription/translation/degradation

F1-6BP

PEP Pyr

Mass &Reaction

Energyflux

Balance

dxSv x

F1-6BP

PEP Pyr

Mass &Reaction

Energyflux

Balance

dxSv x

Allosteric regulation

of enzymes

Error/flow

F1-6BP

PEP Pyr

Mass &Reaction

( ) Energyflux

Balance

dxSv x

Error/flow

Reaction

F1-6BP

PEP Pyr

Flow/error

Reactions

Protein level

F1-6BP

PEP Pyr

Flow/error

Reactions

Protein level

Layered

architecture

Diverse Resources

Diverse Functions

That deconstrain

Flow/error

Reactions

Macromolecules

Mature red blood

cells live 120 days

Running only the top layers

productsS reactions P

Enz1 reaction3 Enzyme form/activity

Reaction rate

“metabolism first”

origins of life?

Flow/error

Reactions

Protein level

Flow/error

Reactions

RNA level

Flow/error

Reactions

DNA level

Flow/error

Reactions

Protein level

Flow/error

Reactions

RNA level

Flow/error

Reactions

DNA level

Protein

Flow/error

Reactions

Protein level

Flow/error

Translation

RNA level

Flow/error

Transcription

DNA level

Diverse Reactions

DNADNADNA

Diverse Genomes

Flow/error

Protein level

Flow/error

Reactions

RNA level

Flow/error

Reactions

Conserved

control

Flow/err

Reactions

Protein level

Flow/err

Reactions

RNA level

Flow/err

Reactions

DNA level

Top to bottom

• Metabolically costly but

fast to cheap but slow

• Special enzymes to

general polymerases

• Allostery to regulated

recruitment

• Analog to digital

• High molecule count to

low (noise)

Rich Tradeoffs

Reactions

Protein level

Reactions

RNA level

Reactions

DNA level

Fragility example: Viruses

Viruses exploit the universal

bowtie/hourglass structure to

hijack the cell machinery.

proteins

Similarities

• Feedback

• Dynamics

Differences

• Metabolism

Control of the Internet

source receiver

Packets

control

packets

source receiver

signaling

gene expression

metabolism

lineage

Biological

pathways

source receiver

control

energy

materials

signaling

gene expression

metabolism

lineage

complex

feedback

source receiver

control

energy

materials

Autocatalytic feedback

source receiver

control

energy

materials

signaling

gene expression

metabolism

lineage

complex

feedback

What theory is relevant to

these more complex

feedback systems?

source receiver

control

energymaterials

complex

feedback

these more complex

feedback systems?

signalinggene expression

metabolismlineage

1ln ln

z z pS j d

Flow/error

Protein level

Catabolism

Carriers

Nucleotides

Biosynthesis

RNA level/

Transcription rate

DNA level

Biosynthesis

Transc. xRNA

Co-factors

Transc. xRNA

tRNA Ribosome

transl. Enzymes

Transc. xRNA

mRNAncRNA

Ribosome

transl. Protein

Transc. mRNA

“Central dogma”

Transc.

Protein

Transl.

tRNA Ribosome

Autocatalysis

everywhere

transl. Proteins

xRNAtransc.

All the enzymes

are made from

(mostly) proteins

and (some) RNA.

F1-6BP

charging

consumption

= dischargingRest of cell

This is just charging and discharging

Flow/error

Protein level

AMP level

F1-6BP

G6PF6P

Rest of cell

ATP supplies

energy to all

layers

Flow/error

Protein level

AMP level

cellA*P

Lots of

ways to

draw this.

RNALayered

transl. Enzymes

xRNAtransc.

trans. EnzymesAA

mRNAncRNA

Ribosome

RNA level/

Transcription rate

RNA form/activity

Transc. xRNA

S reactions P

Enz1 reaction3

Enzyme level/

Translation rate

Enzyme form/activity

Reaction rate

trans.

ncRNATransc.

productsreactions

reaction3

All products

feedback everywhere

ProteinsControl?

Error/flow control

Relay/MUX

Global

E/F control

Relay/MUX

Physical

Application

E/F control

Relay/MUX

Recursive

control

structureFlow

Reactions

Protein level

Reactions

RNA level

Reactions

DNA level

Error/flow control

Relay/MUX

Global

E/F control

Relay/MUX

Physical

Application

E/F control

Relay/MUX

Recursive

control

structureFlow

Reactions

Protein level

Reactions

RNA level

Reactions

DNA level

• Spatial

• Temporal

Bewildering w/out

clear grasp of

layered architecture

Archaea

Eukaryotes

Animals Fungi Plants

Bacteria Algae

What is locus

of early

evolution?HGT and

Shared

Protocols

Horizontal gene transfer

Architecture!?!

Archaea

Eukaryotes

Bacteria Algae

HGT and

Shared

Protocols

trans. EnzymesAA

ncRNAtRNARibosome

RNA level/

Transcription rate

form/activity

Transc. xRNA

productsS reactions P

Enz1 reaction3

Enzyme level/

Translation rate

Enzyme

form/activity

Reaction

Archaea

Eukaryotes

Bacteria Algae

DNA levelGene

Controlled,

dynamic

HGT and

Shared

Protocols

Horizontal gene transfer

• Not a static database

• Not only point mutations

Diverse

function

Diverse

components

Universal

Control

fan-in

of diverse

inputs

fan-out

of diverse

outputs

universal

carriers

Bowties: flows

within layers

Robust

fragile

Constraints

deconstrain

Essential ideas

Diverse

function

Diverse

components

fan-in

of diverse

inputs

fan-out

of diverse

outputs

Robust

yet fragile

Constraints that

deconstrain

Highly robust

• Diverse

• Evolvable

• Deconstrained

Highly fragile

• Universal

• Frozen

• Constrained

• Hijacking

Universal

Control

universal

carriers

Diverse

function

Diverse

components

Universal

Control

fan-in

of diverse

inputs

fan-out

of diverse

outputs

universal

carriers

Bowties: flows

within layers

Robust

fragile

Constraints

deconstrain

Essential ideas

source receiver

control

energymaterials

complex

feedback

these more complex

feedback systems?

signalinggene expression

metabolismlineage

1ln ln

z z pS j d

F1-6BP

PEP Pyr

F1-6BP

PEP Pyr

F1-6BP

Control

Autocatalytic

F1-6BP

Gly3p13BPG

Control

Autocatalytic

1 1(1 )

1 1 01

x q qVxky

Control

Autocatalytic

Control

output=x

Control theory cartoon

Caution: mixed cartoon

Controller

output=x

X jS j

X jS j d d

X j d U j d

Entropy rates

1ln 0S j d

Hard limits

0 5 10 15 200.8

Time (minutes)

Time response)

Fourier

Transform

of error

h hS x = F(

0 2 4 6 8 10-0.8

Frequency

Spectrum

1log loghS S

0 5 10 15 200.8

Time (minutes)

h >> 1

0 2 4 6 8 10-0.8

Frequency

Spectrum

Time response

Robust

fragile

0ln lnhS j S j

0 2 4 6 8 10-0.8

Frequency

h = 0 Robust

fragile

ln 0S j d

[a system] can have

[a property] robust for

[a set of perturbations]

Yet be fragile for

Or [a different perturbation]

[a different property] Robust

Fragile

Robust yet fragile = fragile robustness

output=x

X jS j

X jS j d d

X j d U j d

Entropy rates

1ln 0S j d

Hard limits

0 5 10 15 200.8

Time (minutes)

h >> 1

0 2 4 6 8 10-0.8

Frequency

Spectrum

Time response

Robust

fragile

0ln lnhS j S j

output=x

X jS j

1ln ln

z z pS j d

1ln 0S j d

The plant can make

this tradeoff worse.

output=x

X jS j

1ln ln

z z pS j d

1ln 0S j d

2kz p RHPzero s q k s k

All controllers:

Biological cells: =

output=x

X jS j

1ln ln

z z pS j d

1ln 0S j d

2kz p RHPzero s q k s k

Small z is bad.

Small z is bad

(oscillations and crashes)

Small z =

• small k and/or

• large qEfficiency =

• small k and/or

• large q

Correctly predicts conditions

with “glycolytic oscillations”

1ln ln

z z pS j d

output=x

X jS j

X jS j d d

X j d U j d

Entropy rates

1ln 0S j d

Hard limits

output=x

uController

1ln 0S j d

output=x

uController

1ln 0S j d

Channel

Sensor+

Channel

output=x

Controller

1ln FBS j d C

Channel

Sensor+

Channel

1ln sensorS j d C Helps

Flow/error

Reactions

Protein level

Flow/error

Trans*

*NA level

qk y zyw

xAutocatalytic

Control

Glycogen/

glucose

AA Biosyna

p1Prot1

Anaerobically on w

Control

feedback

everywhere

Nuc Biosyn n

operons

See El-Samad, Kurata, et al…

PNAS, PLOS CompBio

operons

Flow/error

Reactions

Protein level

Flow/error

Reactions

RNA level

Flow/error

Reactions

DNA level

Protein

Where are these layers?

degradation

folded unfolded

operons

DnaKRNAP

FtsH Lon

HeatmRNA

operons

DnaKRNAP

FtsH Lon

HeatmRNA

operons

Flow/error

Reactions

Protein level

Flow/error

Reactions

RNA level

Flow/error

Reactions

DNA level

Protein

Where are these layers?

Protein

FtsH Lon

operons DnaK

RNAPRNAP

rpoH Heat

Translation

was not

shown.

mRNA activity is

actively controlled.

FtsH Lon

HeatmRNA

operons

F1-6BP

PEP Pyr

Allosteric

Trans*

FtsH Lon

levels

Layered control architectures

F1-6BP

PEP Pyr

Allosteric

Trans*

FtsH Lon

levels

ratesThe greatest

complexity

here is

primarily in

the control

of rates

That is not always the case.

FtsH Lon

HeatmRNA

operons

All at the DNA layer

qk y zyw

xAutocatalytic

ControlAero-

Reaction?

Glycogen/

glucose

AA Biosyna

Reaction?

*NA1*NA2

*NA?*NA?

qk y zyw

xAutocatalytic

Control

Glycogen/

glucose

Anaerobically on w

Protein

qk y zyw

xAutocatalytic

Control

Glycogen/

glucose

AA Biosyn

degrade

Anaerobically on w

Control?

tRNA Ribosome

transl. Enzymes

Transc. xRNA

mRNAncRNA

tRNA Ribosome

Autocatalysis

everywhere

transl. Proteins

xRNAtransc.

All the enzymes

are made from

(mostly) proteins

and (some) RNA.

xAutocatalytic

Reaction?

Glycogen/

glucose

AA Biosyn

Reaction?

Aerobically on z

supply

Control?

qk y zyw

xAutocatalytic

ControlAero-

Reaction?

Glycogen/

glucose

AA Biosyn

degrade

Reaction?

trans.

ncRNATransc.

productsreactions

reaction3

All products

feedback everywhere

ProteinsControl?

qk y zyw

xAutocatalytic

Control

Glycogen/

glucose

AA Biosyna

degradex

Anaerobically on w

*NA1*NA2

Control

feedback

everywhere

qk y zyw

xAutocatalytic

ControlAero-

Reaction?

Glycogen/

glucose

AA Biosyna

Reaction?

*NA1*NA2

*NA?*NA?

qk y zyw

xAutocatalytic

Control

Glycogen/

glucose

AA Biosyna

p1Prot1

Anaerobically on w

Control

feedback

everywhere

Nuc Biosyn n

Complex Network Flow Architecture: A cartoon guide€¦ · Complex Network Architecture: A cartoon...

Documents

Architecture of Mammalian Respiratory Complex I

An Ensemble Architecture for Learning Complex …web.engr.oregonstate.edu/~tgd/publications/zhang-etal-an...An Ensemble Architecture for Learning Complex Problem-Solving Techniques

00 An Ensemble Architecture for Learning Complex Problem ...rakaposhi.eas.asu.edu/gila-tist.pdf · 00 An Ensemble Architecture for Learning Complex Problem-Solving Techniques From

Probing nuclear pore complex architecture with … · Probing nuclear pore complex architecture with proximity-dependent biotinylation Dae In Kima, Birendra KCa, Wenhong Zhub,

Pareto-optimization of complex system architecture …...ORIGINAL PAPER Pareto-optimization of complex system architecture for structural complexity and modularity Kaushik Sinha1 •

Enterprise Architecture of Emergent Complex … Architecture of Emergent Complex Adaptive Systems ... The Enterprise • Approach as Enterprise ... • An enterprise architecture and

Understanding the genetic architecture of complex traits

Pathogenic alleles, clan genomics and the complex architecture of

OSX Complex Application Challenge Architecture

Reiterative growth in the complex adaptive architecture …users.humboldt.edu/tomescuam/pdf/Tomescu 2008 Plant Systematic… · Reiterative growth in the complex adaptive architecture

Probing nuclear pore complex architecture with proximity … · Probing nuclear pore complex architecture with proximity-dependent biotinylation Dae In Kima, Birendra KCa, Wenhong

Complex Programmable Logic Device (CPLD) Architecture and Its Applications

The Polygenic Nature and Complex Genetic Architecture of

Masterpieces of Indian Art & Architecture New Delhi Capital Complex

Architecture of the herpesvirus genome-packaging complex ... · R ESEARCH ARTICLE Architecture of the herpesvirus genome-packaging complex and implications for DNA translocation Yunxiang

Puthia temple complex: Developing tourism through architecture

Molecular architecture of human polycomb repressive complex 2

Polymers with Complex Architecture by Living Anionic ... · Polymers with Complex Architecture by Living Anionic Polymerization Nikos Hadjichristidis,* Marinos Pitsikalis, Stergios

Architecture Analysis of Evolving Complex Systems ... - Architecture Analysis of Evolving Complex Systems of Systems ... 171484;197406-198403;203388-204385;220337-498500. Actual CFPD

Masterpieces of Indian Art & Architecture New Delhi Capital Complex