Ohayon - UCSD - Cogs 1 - May 19, 2009 1

1

Autonomous Neurodynamics: How Does all the Stuff Going on in our Brains

add up to Freedom?

Elan Liss Ohayon

UCSDCogs 1: Introduction to Cognitive Science

May 19, 2009

2

How are we free?

How are we conscious?

What is the relation?

3Invasion of the Body Snatchers (1956) 4

I) Determinism

II) Modeling Epilepsy and Cognition in Embodied Autonomous Agents:

- Dynamics: - Categorization of Network Dynamics - Intermittency

- Structure: - Spatial Networks- Neurodegenerative Disorders- Persistent Activity

- Behavior: - Embodied modeling

III) Implications:

- Clinical (Epilepsy, Transitions) - Cognition (e.g., Attention) - Ethics (Humans, Robots)

Concepts Covered / Overview Of Talk

5

(I) DETERMINISM

6

Laplace's Demon

Pierre-Simon, marquis de Laplace (1749 – 1827)

"We may regard the present state of the universe as the effect of its past and the cause of its future. An intellect which at a certain moment would know all forces that set nature in motion, and all positions of all items of which nature is composed, if this intellect were also vast enough to submit these data to analysis, it would embrace in a single formula the movements of the greatest bodies of the universe and those of the tiniest atom; for such an intellect nothing would be uncertain and the future just like the past would be present before its eyes."

Ohayon - UCSD - Cogs 1 - May 19, 2009 2

7

Determinism and Freedom: The Great Debate

Cicero, Marcus Tullius(106 BCE – 43 BCE)

On fate (De fato)

image: thanks Wikipedia!

8

Determinism and Freedom: The Great Debate

Democritus (460 BCE – c. 370 BCE)

Vs.

Epicurus(341 BCE – Athens, 270 BCE)

Swerving Atoms

image: Wikipedia image: Wikipedia

9

Descartes: Mind-Body Problem

Descartes (1596 –1650) - Body (Machine) vs. Soul - Interaction at pineal gland - Snag: Dualism; Mind-body problem / dichotomy

image: Wikipedia

image: Wikipedia

image: Wikipedia

10

Determinism - Epistemic Issues (what we can know…)

- E.g., Chaos - deterministic but unpredictable

- Ontological Issues (of the nature of things…)- Newtonian world, Einstein - Quantum Mechanics?

- Compatibilism?

- Implications:What you do won't make a difference!?Will choosing to study for the exam change the outcome?

A matter of Life and Death? Time and place of death inevitable?

Possible hint: Internal contradiction!

11

(II) Modeling Epilepsy and Cognition in Embodied

Autonomous Agents

12

I) Analysis of Brain Dynamics

II) Computer Modeling• Random Networks• Spatial Networks• Autonomous Agents (Robots)

III) Model Verification, Bridging I and II

Overview of Research Epilepsy & Cognition

Ohayon - UCSD - Cogs 1 - May 19, 2009 3

13

PROBLEM OF EPILEPSY / DISCHARGE: • Hyper-Excitation• Self-Sustained• Oscillations / Periodicity / Synchrony• Propagation

How does this state come about? Can we model epilepsy?

(EASY!)

(HARD QUESTION!)

What about the “typical" brain? How does it maintain activity and interactivity??

14

Close Return Analysis: Generalized Seizure

Subject 3932418456; Fp1-Pz (Channels 0); t = 4 min: 23-33 sec; 200 samples / sec

15

Categorization of EEG in Children with Seizuresusing Close Return Analysis

16

EPILEPSY AND COGNITION• EPILEPSY:

– What are the mechanisms by which networks become susceptible to transitions from interictal to ictal states?

– What initiates a given transition?

– How is an unprovoked seizure initiated?

• COGNITION:– Attention, Recognition, Decisions, Action / Behavior

• COMMONALITY → Transitions– How do systems change from one state (or phase) to another?

17

How do Neural Systems ChangeDynamical States?

- Change in unit properties - Change input (perception, noise)- Change in connectivity properties

- Autonomous Neurodynamics

18

Neural Network Modeling

Ohayon - UCSD - Cogs 1 - May 19, 2009 4

19

NEURAL NETWORKS

Output Layer

Input Layer

Hidden Layer

Levels of Analysis: somewhere between ion channels and universal equations.

20

NEURAL NETWORK FUNDAMENTALS

• Processing units• States of activation• Pattern of connectivity• Efficacy of synaptic transmission – weights• Activation rule

….• Memory and processing are not separate

21

NEURAL NETWORKS

Simple Neural Unit Transfer Function

Afferent inputs to a unit (S1..Sj) are multiplied by the respective weights (w1..wj) and summed. This weighted sum Ei is then fed through a sigmoidal function S(Ei) which in turn yields the activation value for that unit.

22

NEURAL NETWORKS

Output Layer

Input Layer

Hidden Layer

RECURRENT

23

CATEGORIES OF DYNAMICS

(in Fully ConnectedRandom Networks)

24

Architecture and Dynamics in Fully interconnected Recurrent Neural Networks

Paper Fig. 1. Illustration of recurrent neural network architecture

Ohayon - UCSD - Cogs 1 - May 19, 2009 5

25

Categories of Dynamics in Random Networks

Randomly connected networks exhibited (a) fixed point (b) periodic oscillations (c) close returns (d) turbulent / complex dynamics. Periodic behavior by far dominated these first generations (Ohayon et al, J. Physiology – Paris, 2004).

26

The distribution of network dynamics as a percentage of population in randomly connected networks (Five populations of 100 networks; each network was run for 1000 iterations).

27

Activity Dynamics of a Plastic Network(Hebbian Nightmares)

• A network's progression from turbulent activity through synchrony and ultimate convergence to a fixed-point all within 200 iterations.

• Note the transitional synchrony between the first and fifth unit for the same data.

• Networks were run with the addition of a naive Hebbian plasticity rule.

• Networks with this plasticity tended to quickly synchronize and ultimately all settled to fixed point dynamics.

28

Oscillations in Networks

• Simple networks: connectionist • Synchrony as a result of collective network properties• Units don't need to oscillate individually. No need for a

central pacing mechanism• High probability of synchrony

• Questions: What does this have to do with behavior or epilepsy? How do networks AVOID synchrony?

- Unbinding problem- Hebbian nightmares

How do networks switch states? (transitions)

29

Intermittency

Acknowledgments:

• Stichting Epilepsie Instellingen Nederland (SEIN)& University of Amsterdam

• Dr. Stilliyan Kalitzin • Dr. Piotr Suffczynski• Dr. Fernando Lopes da Silva

30

(Superimposed activity traces for 5 units in a recurrent neural network showing intermittent activity)

Intermittency in Network Activity

Ohayon - UCSD - Cogs 1 - May 19, 2009 6

31

Modification of Intermittency with Weight Changes

Changes in connectivity could modulate intermittency in a generally continuous manner. (a), (b) and (c) show increases in the duration of laminar periods corresponding to increasing the strength of a single connection.

1.5

-11.5

-11.5

-1

Time (Iterations)

(a)

(b)

(c)

32

Implications - Epilepsy

• Intermittency in the model may offer a novel way to understand seizure transitions (new category)

• Once intermittency is in place ictal events can occur intermittently without the need for further changes or extrinsic triggering.

• However, these autonomous transitions are not exclusionary of other mechanisms…

• Intermittency + weight changes: The genesis and abolition of seizure susceptibility

• Clinical implications: basic understanding, classification, prediction, amelioration and reversal of susceptibility

33

Implications - Cognition

• Seizures as a window/door to cognition

• Sensitive to input but avoids lockup

• Transitions in attention and behavior

• Intermittency does not require plasticity → rapidity of response

• Modifiable (context and learning)

34

Network Structure, Boundaries and Activity

(in spatial network models)

Acknowledgments:

• Computational Neurobiology LaboratoryThe Salk Institute

• Maxim Bazhenov• Terry J. Sejnowski

35

SPATIAL NETWORK

36

Small single-layer networks with the connection radius defined as (A) radius=1 (B) radius=2 and (C) radius=1 with wraparound connections.

A B C

Ohayon - UCSD - Cogs 1 - May 19, 2009 7

37

• 3D recurrent network architectures • recurrent local connection within a given geometry and radius (in x, y, z planes)• on-line stimulation, cell deletion and recording • single, two-layer and multi-layered networks• up to 12,800 elements sigmoid or spiking activation functions (80x80x2)

General Purpose Simulation Environment Unit Connectivity ZX ; ZY Projections XY Projection

38

Xn

Train of action potential generated by the model

SPIKING MODELS (MAP-BASED)

α/(1-v) + y,

α + y,

-1,

v ≤ 0

0 < v < α+y

v ≥ α+y

f (v,y)=

Rulkov, Timofeev, Bazhenov, Journal of Computational Neuroscience 17, 203–223, 2004

39

• Each layer represented distinct inhibitory and excitatory populations• Inter-layer and columnar intra-layer connectivity• Toroidal connectivity (to eliminate boundaries)

TWO-LAYER NETWORKS:

40

41

Post Lesion Video:

42

Variable Deletion Probability

Pd = 0 Pd = 0.2 Pd = 0.5

Ohayon - UCSD - Cogs 1 - May 19, 2009 8

43

Epilepsy and Aging

From Hauser, WA. Seizure disorders: changes with age, Epilepsia 1992; 33(Suppl 4:S9); reprinted in GubermanA., and Bruni J. (1999) Essentials of Clinical Epilepsy, 2nd ed. Butterworth-Heinemann, Boston, MA.

Age specific incidence of generalized-onset and partial-onset seizuresin Rochester, Minnesota: 1935-1984.

44

Cell Loss and Dynamics in Aging and Neurodegenerative Diseases

• Previously shown structure can be critical in dynamics of post-traumatic epilepsy (focal cell deletion)

• Could diffuse connectivity changes cause changes in dynamics seen in EEG accompanying cell loss in aging and neurodegeneration?

• What about persistent activity in healthy brains?

45

Homogenous Network (pd = 0)

46

Heterogeneous Network (pd = 0.4)

47

Heterogeneous Network (pd = 0.9)

48

Ohayon - UCSD - Cogs 1 - May 19, 2009 9

49

• Variation in activity as a function of changes in network structure

• Changes in dynamics are complex

• Structural mechanisms may be sufficient to bring about changes in oscillations seen in aging and neurodegenerative disorders

• The healthy brain: heterogeneity in structure as basis for complex and persistent activity required for cognitive processing

• May need to look beyond: Intrinsic cell properties and inhibitory / excitatory balance

Heterogeneous Nets - Summary

50

(a) The raster plot of activity for all remaining cells in a 40 by 40 heterogeneous network with diffuse cell deletion p=0.4.

(b) The population activity for the same simulation. Note that distinct low-frequency, high-amplitude, waves appear at beginning of the run and then return at approximately iterations # 1600-2200 and iterations # 3800-4300.

(c) The autonomous transitions in and out of low frequency are reflected in the wavelet transform of the population activity. No structural changes, stimuli or noise were applied during the simulation -- the transitions in population activity are thus autonomous of any external factors. These autonomous frequency transitions correspond to ongoing changes in the spatial nature of the propagating wavefronts.

Autonomous Transitions in Spatial Networks

3200

0

200

400

600

800

1000

1200

1400

1600

1800

2000

2200

2400

2600

2800

3000

50001 200 400 600 800 1000 1200 1400 1600 1800 2000 2200 2400 2600 2800 3000 3200 3400 3600 3800 4000 4200 4400 4600 4800

Cel

l #

(a) Raster Plot

Time (samples)

0

5

10

15

20

25

30

35

Wavelet Transform

40

0

Freq

uenc

y (H

z)

10

0.0

2.0

4.2

6.4

8.4

(c) Wavelet Transform

Time (samples)

60.0

0.0 # of

Act

ive

Cel

ls

60

0

(b) Population Activity

Time (samples)

51

Oscillations in Evolved Embodied Networks

52

• Dr. McIntyre Burnham • Dr. Hon Kwan

• Dr. Paul Hwang• Dr. Peter Carlen

AcknowledgmentsUniversity of Toronto Group

• Canadian Institutes of Health Research (CIHR)• U of T International Exchange Program• NIH

• Kathryn Hum• Ann Lam• Deborah Lonsdale• Kirk Nylen• Brian Scott

• Frank Jin• Ruth Liu• Heather Chun• Maggie MacDonald• Thomas Cheng

53

Recurrent Networksand Autonomous Agents

Autonomous Agent Network and Body Diagrams: eight input units, five hidden units and two output units connected to the wheels. Activity in the recurrent network responded in real-time to the sensory input and recurrent activity. 54

Recurrent Network Driving Autonomous Agent

Ohayon - UCSD - Cogs 1 - May 19, 2009 10

55

FITNESS FUNCTION

• A fitness function that encouraged forward movement, turning and obstacle avoidance successfully eliminated motor seizures.

56

EVOLUTIONARY ALGORITHM

Initial network weights were randomly generated.

The probability for a given network architecture (its set of weights) to be passed on to the next generation was directly proportional to the behavioral success of the network (fitness)

The most successful networks were passed on without change (elitism).

Other networks were randomly mutated. The flow chart illustrates the iterative process.

57

Video - Embodied Behavior - Generation 0

58

Video - Embodied Behavior - Generation 12

59

POPULATION EVOLUTION

• The population almost completely evolved out of epilepsy within 10 generations from the initial set of 20 random individuals

60

EVOLVING OUT OF EPILEPSY

• Contrast of a second generation’s highly periodic neural activity (top panel) with the more complex activity seen in a twelfth generation network.

• As population fitness evolved, a change in spectral characteristics from narrowband to broadband activity was observed. Narrowband characteristics are often associated with seizure or tremor-like behavior.

Ohayon - UCSD - Cogs 1 - May 19, 2009 11

61

EVOLVING OUT OF EPILEPSY

• Initial generations included a high percentage (up to 70%) of individuals with simple periodic neural activity and concomitant behavioral oscillations (seizures as default).

• Through interaction with the world the genetic algorithm was able to evolve networks away from this motor behavior.

• The use of embodied computational models thus offers a novel way of modeling epilepsy and pathology in general.

• This use of an evolutionary approach can help provide understanding of the escape from pathology in humans as well as other autonomous agents (robotic systems).

62

ADVANTAGES OF EMBODIED MODELINGCompared to biological preparations:

Full access to the activity levels across the entire network Full access to network connectivity weight Repeated experiments with same network over differing circumstances Can study a complete population through its evolutionCan interact with the network directly in a non-perturbing manner

Compared to traditional computational model:

Physical behavioral correlateDirect interaction with environment Explicit consideration of real world parameters

(gravity, light, physical intervention, etc.)

Robots as research partners!

63

Not damage… work toward health!!!

Constructive Not Destructive Modeling

64

(III) Implications:

On Embodiment, Body Snatchers and Emotional

Agents

65

Why “Robot”?

Robots and Autonomous Agents

66

The word comes from the Czech word robota, compulsory labour or work (also used in a sense of a serf), first used by Karel Čapek in his science fiction play R.U.R. (Rossum'sUniversal Robots) in 1921, and according to Čapek, was coined by his brother, painter Josef Čapek (see also etymology of robot). The word was brought into popular Western use by famous science fiction writer Isaac Asimov.

(Wikipedia entry on Robots)

Robots and Autonomous Agents

Ohayon - UCSD - Cogs 1 - May 19, 2009 12

67

A key scene in "R.U.R." (Rossum's Universal Robots, 1923) has a central human character, Helena, witnessing a robot having a seizure.

The Psychologist-in-Chief dismisses this activity: "Occasionally they seem somehow to go off their heads. Something like epilepsy, you know. We call it Robot's cramp... It's evidently some breakdown in the mechanism."

The Chief Engineer and Head of the Physiological Department concur. The General Manager asserts: "A flaw in the works. It'll have to be removed."

But Helena -- clearly conveying Čapek's sympathies and outlook -- disagrees and cautions that this supposed flaw may be an important first indication of life: "Perhaps it's just a sign that there's a struggle. Oh, if you could infuse them with it." (Capek, 1923)

Robots, Epilepsy and Autonomy

68

- Not so quick to judge epilepsy as pathology- Daniel Tammet Case…- Extraordinary abilities connected to cognitionand emotions

Epilepsy, Emotions and Cognition

69Video - From Brainman (2005), Documentary on Daniel Tammet 70

- Daniel Tammet case –> increased ability - Car driving case –> inverse- Actor case –> no relation - Current thinking:

- Consciousness related to most complex tasks - Ridiculing of Freud…. - But... Freud alerted us to the fact that

some of the most complex tasks may be taking place unconsciously!

Relation between (1) consciousness/ emotions

and (2) capabilities / behavior

=> not trivial!

71

X Free

Zombies?Not-Free

Conscious Not-Conscious

72

- Qualia, Consciousness (see Chalmers) - Philosophical Zombies / Pods - False Negative (think no emotions but have them) - False Positive (i.e., no emotions, displace cognizant beings)

Grave Concerns:

- Creation of something with emotions and treating it as if it doesn't have emotions (Frankenstein, RUR, Blade Runner, Hal, A.I….)

- Creating something that doesn't have emotions and we think it does (Zombies, Pods)

Implications:- Ethics, philosophy of mind, computation, physicalism (see Seager)

Easy vs. Hard vs. Profound Questions