System Level Neuroengineering Modeling of the Nervous System Cancun Dec 2008 I. Steve Massaquoi i.The Problem ii.Overview of sensorimotor control system

System Level Neuroengineering Modeling of the Nervous System

Cancun Dec 2008

I. Steve Massaquoi

i. The Problemii. Overview of sensorimotor control systemiii. Attention to cerebellar system architectureiv. RIPID control model as example of medium high level modelv. Preliminary implications of model

II. Kazutaka Takahashi

i. Cerebrocerebellar system architecture in greater detailii. RICSS Quantitative model of internal signal generationiii. Model adequacy and modeling issuesiv. Relation of dynamic models to point process signal analysis

I. The Problem to develop increasingly comprehensive, integrated, multi-resolution engineering models of nervous system structure and function

MotivationsScientific: i. understanding the principles of human behavior and intelligence in quantitative, mechanistic terms ii. understanding the principles of neurological and psychiatric disease in quantitative, mechanistic terms

Engineering: i. design of devices to mimic and potentially supersede human behavior artificially ii. development of devices that better interact with the nervous system to study, restore or extend human function

System Level Neuroengineering Modeling of Nervous System I. Massaquoi CDC Cancun 2008

I The Problem (cont’d)

Key Challenges (constraints) for Neuroengineering models:

A. Models must respect known or highly suspected neuroantomical connections

B. Models must respect known time delays and phase lags (these are non-trivial)C. Models must utilize functions that are known to be achievable by collections

of neurons. (e.g. Quantitative multiplication ?)D. Models must be consistent with both normal human function, and

pathological dysfunction.E. Models of human function should be able to be related to those of animals in

a manner consistent with natural evolution (i.e. continuously)F. Models must ultimately account for the firing behavior of system neurons (i.e. must be multi-resolution)

System Level Neuroengineering Modeling of Nervous System I.System Level Neuroengineering Modeling of Nervous System I. Massaquoi CDC Cancun 2008


Style of approach: Ideal features of Neuroengineering models:

A. Models should seek to explain the nervous system in terms of established engineering principles and abstractions

(e.g. filters, estimators, feedback/feedforward controllers, registers, switches

not simply to formulate a computational model)

B. Models should point to potentially fruitful areas of engineering research where current engineering methods are lacking.

• Models are more profitably driven by science, and guided by engineering principles. . . Rather than vice versa

(editorial remark!)



Neuroengineering strategy Modeling and analysis Behavior & Intelligence: Bottom up Primitive complex mimicking evolution editorial comment #2 Intelligence will emerge



Neuroengineering strategy Modeling and analysis Behavior & Intelligence: Bottom up Primitive complex mimicking evolution editorial comment #2 Intelligence will emerge Modeling and analysis of Physiology: Simultaneous multi-resolution neuron circuits architecture

Massaquoi




Takahashi




Emphasis for rest of session




Investigate both health and disease


• The problem

• Overview of the sensorimotor control system -- Multi-loop architecture, feedback and delays -- Internal dynamic models?

• RIPID Model -- Physiology -- Stability analysis -- Cerebellar linear gainscheduling?

• Application to bipedal balance and locomotion


Neural signals

executive

sensory

Mtr CtxBrainstem orSpinal Cord

Segment

Im AntCblL Ant

Cbl

Putamen& GP

Caudate& GP

Frontal & ParietalAssoc Ctx

BodyForce/Motion

Muscle & tendon,Joints, skin

“highest level”PLANS (strategy)

“middle level” PROGRAMS (tactics)

“lower level”ACTION

(force, velocity)

“Motor Servo”

Vestib

Visual

M. Cbl Flocc Cbl

• Human motor control principal information flow (adapted from V. Brooks, 1986)


Brainstem orSpinal Cord

Segment

Im AntCblL Ant

Cbl

Putamen& GP

Caudate& GP


BodyForce/Motion





(force, velocity)“Proprioceptive Motor Servo ?”

Vestib

Visual

M. Cbl Flocc Cbl

• Motor Servo Concept?

Actuator ?

Mtr Ctx


Mtr CtxBrainstem orSpinal Cord

Segment

Im AntCblL Ant

Cbl

Putamen& GP

Caudate& GP


BodyForce/Motion





(force, velocity)

(programmed) Stimulus- Response Loops?

Vestib

Visual

M. Cbl Flocc Cbl

• Programmed Triggered Response Loops?


Mtr Ctx

Im AntCbl

L AntCbl

Putamen &GP, SN

M. Cbl Flocc Cbl

“high level”PROGRAMS

(discrete control) (tactics: trajectories

cues)


“highest level”PLANS,

ALGORITHMS (free assoc, strategy)

“intermediate level”CONTROL

(continuous control)(stability, tracking, stiffness,

scaling, movement time)

• Possible refinement of upper portion

L Post Crerebellum

Caudate & GP

Primary & Peri-Sensorimotor Ctxs

“Proprioceptive Motor Servo ?”

(programmed) Stimulus- Response Loops?

Cognitive Programming?


Brainstem orSpinal Cord

Segment

Im AntCblL Ant

Cbl


BodyForce/Motion


• Important sensorimotor control issues: 1) (often) low plant stiffness

2) significant time delays/phase lags~35-40 ms round trip to elbow+ low pass filtering of neural signal at muscle

if not compensated, cause significant instability

Mtr Ctx

T

~10 ms

~8 ms

T

2/(s+)2

Primary and Peri-Sensorimotor Ctxs


• As a result, considerable thinking views cerebellum as an adaptive compensator that incorporates internal models of inverse and/or forward dynamics. Eg:

–

+ G PT

T2–+

T

ref

Miall et al, 1993

Smith Predictor?

ˆ P

P– +

++G T

T

ref

Kawato & Gomi, 1992

Feedback error learning ofinverse dynamics?

ˆ P 1


• The problem


• RIPID Model -- Structure and performance -- Stability analysis -- Cerebellar gainscheduling?



Simpler control approaches may also fit better with what is known about cerebellar architecture

Primary Processing Elements:Purkinje Cells (PC), 1.5x107

Branch of Major Input Channels:Parallel Fibers (PF) ~1011, 1012

3-6 mm in humans (very long)& thin, very slowly conducting 0.5 m/sec

medio-lateral

Cortex of folium

• Specifically, control must be implemented by simple lattice-like modular architecture of cerebellum

(phylogenetically strongly conserved)

Deep white matter

PCPF

Operation?Activated “beam” of parallel fibers(known) ….

u(t)y(t)

control signal input (?) (Mossy Fiber to PF and DCN)

–MF

PFs

DCN

direct excitation

Inhibition by sidepath

+

y(t)= 1u(t)2u(tt)

y(t) =(12)u(t) + 2(u(t) u(tt))

Y(s)( gbs + gk)U(s)

• In light of its connectivity, we can consider that the lateral cerebellum may compute proportional and derivative signals

2 represents adaptable weight

s represents Laplace complex frequency variable

PD

u(t)1

2t

y(t)

CbCtx

Dn

y(t) (3 4 )u()d0

tY(s) ((3 4 )

1

s)U(s)

• …. and that the intermediate cerebellum may compute integral signals

I

u(t)

Ipy(t)

z(t)

CbCtx

RNmc LRN

MF

Ip

• …. Together, cerebellar regions may implement “PID” (control) circuitry

IPD

u(t)1

2t

y(t)

CbCtx

Dn

u(t)

Ipy(t)

z(t)

CbCtx

RNmc LRN

MF

Ip

spinomusculoskeletal plant with low-pass muscle activation dynamics

• … so a simple linear control system structure may be considered.

i1

–

+

s gb

–

+gk

1/sf2

+

++

ref spr

i2

–P(s)

spr

ia/s mc

neural “long-loop”signal transmissiondelays


i1

–

+

s gb

–

+gk

1/s

ia/s

f2

+

++

ref spr

i2

–P(s)

spr

mc

peri- and primary sensorimotor cortex including integrator anddirect paths


i1

–

+

s gb

–

+gk

1/s

ia/s

f2

+

++

ref spr

i2

–P(s)

spr

mc

Linear cerebellar processing


i1

–

+

s gb

–

+gk

1/s

ia/s

f2

+

++

ref spr

i2

–P(s)

spr

mc


Proposal: Stabilized feedback PID control model of cerebellum

“PID”ProportionalDerivativeIntegral

i1

–

+

s gb

–

+gk

1/s

ia/s

f2

+

++

ref spr

i2

–P(s)

spr

mc


Proposal: Stabilized feedback PID control model of cerebellum key added feature: integrator in feedback path to add phase-lead that stabilizes against transmission delays

i1

–

+

s gb

–

+gk

1/s

ia/s

f2

+

++

ref spr

i2

–P(s)

spr

mc


Key Feature: Recurrent integrator that adds phase lead to PID= “RIPID” model

i1

–

+

s gb

–

+gk

1/s

ia/s

f2

+

++

ref spr

i2

–P(s)

spr

mc


Key Feature: Recurrent integrator that adds phase lead to PID re-represented showing zero at origin

i1

–

+

sgb

P(s,T)T

–

+

T

ref gk

1/s

ia/s

f2

+

+

s

s i2 +

f3

0 0.5 1 1.5 2 2.5 3

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

1.2

Figure R1. Impulsive disturbance response of elbow as function of f1

seconds

segmental reflex gain f1 = 0segmental reflex gain f1 = 0.4segmental reflex gain f1 = 1.5

• Postural regulation: Impulse response of linear elbow joint model to 1 Nt force for different values of segmental reflex gain

radi

ans

0 0.5 1 1.5 2 2.5 3-0.5

0

0.5

1

Figure R2a. Impulsive disturbance response of elbow with long-loop control

seconds

radi

ans

(+)segmental reflex, (-)long-loop(+)long-loop, (-)segmental reflex(+)long-loop, (+)segmental reflex

0 0.5 1 1.5 2 2.5 3-0.5

0

0.5

1

Figure R2b. Impulsive disturbance response of elbow with cocontraction, long-loop and segmental reflex control

seconds

radi

ans

(+)cocontraction, (+)segmental reflex, (-)long-loop(+)cocontraction, (+)long-loop, (-)segmental reflex(+)cocontraction, (+)long-loop, (+)segmental reflex

• Postural regulation: Impulse response of plant to 1 Nt force with contribution of RIPID-stabilized long-loop responses with and without muscular coactivation.

-60 -40 -20 0 20 40 60 80 100 120

0

50

100

150

200

250Figure 4a. Force response to sinusoidal position disturbance input

joint (angular) stiffness Nm/rad

join

t (an

gula

r) v

isco

sity

Nm

-s/r

ad

Data*Simulation

• Postural regulation: Viscous and Elastic force responses to small amplitude sinusoidal position disturbance (* Rack, 1981)

i1

–

+P(s,T)T

–

+

T

ref gk

i2

ia/s

f2

+

+– +

f3

• Effect of cerebellar lesions: lateral gk, and/or intermediate recurrent integrator i2

∫

gbd/dt

X

X

0 0.5 1 1.5 2 2.5 3-0.5

0

0.5

1

Figure R5a. Elbow disturbance response with decreased cerebellar gains i2 and gk, no cocontraction

seconds

radi

ans

i2 normal, gk normal (Figure 2a)i2 normal, gk reducedi2 reduced, gk reduced

0 0.5 1 1.5 2 2.5 3-0.5

0

0.5

1

Figure R5b. Elbow disturbance with decreased cerebellar gains i2 and gk and cocontraction

seconds

radi

ans

i2 normal, gk normal, (-)cocontractioni2 normal, gk reduced, (+)cocontractioni2 reduced, gk reduced, (+)cocontraction

• Postural regulation (pathological): Effect of reducing cerebellum-related gains gk and/or i2 , without and with muscular co-activation

• The problem


• RIPID Model -- Structure & performance -- Stability analysis -- Cerebellar gainscheduling?



• Without RIPID controller, cortical integrator and bypass path Start with a simple nominal linear model of spinomuscular apparatus with long loop gain K, and delays.

CNS

udesc

+

–

0

+Tcns(s)

Tcns(s)

K

Tpr(s)

(1+0.1s)

spinomusculoskeletal plant including peripheral nervedelays

P(s,T)

Natural frequencies for long-loop control, as K≥0.

-140 -120 -100 -80 -60 -40 -20 0 20-60

-40

-20

0

20

40

60

Real Axis

Ima

g A

xis

TextEnd

Root locus of total plant, delay and sf2

CNS

–

+P(s,T)

ref

ia/s

f3

Tcns(s)

Consider behavior for various values of gain K

gbs2 gks i1s i2

K

Tcns(s)1+sf2/iaf3Tpr(s)

(1+0.1s)

• With RIPID Controller, cortical integrator and bypass path

-140 -120 -100 -80 -60 -40 -20 0 20-60

-40

-20

0

20

40

60

Real Axis

Ima

g A

xis

TextEnd

PZ map of compensator, cortical integrator, total plant, loop delay, spindle and sf2

Natural frequencies (x) of RIPID model for K=0, and zeros (o) in complex plane:

Poles relatedto E-A coupling

musculoskeletaldynamics

thalamocorticalintegrator

x x x xx

x

Recurrent Integrator

-140 -120 -100 -80 -60 -40 -20 0 20-60

-40

-20

0

20

40

60

Real Axis

Ima

g A

xis

TextEnd

PZ map of compensator, cortical integrator, total plant, loop delay, spindle and sf2

Natural frequencies (x) of RIPID model for K=0, and zeros (o) in complex plane:

primary spindleafferent

recurrent integrator

controller

cortical integrator bypass path

Natural frequencies (x) of RIPID model for K > 0, and zeros (o) in complex plane:

-140 -120 -100 -80 -60 -40 -20 0 20-60

-40

-20

0

20

40

60

Real Axis

Ima

g A

xis

TextEnd

Root locus of total plant, delay and sf2

• The problem





Operation?Activated “beam” of parallel fibers(known) …. Two classes (?, speculation)

u(t)y(t)

s(t)context-specificselector input (?)(Mossy Fiber to PF)

control signal input (?) (Mossy Fiber to PF and DCN)

–

MF

MF

PFs

DCN

direct excitation

Inhibition by sidepath

+

laterally inhibited Purkinje Cells

Operation?Activated “beam” of parallel fibersfocused by lateral inhibition (reasonably established).

u(t)y(t)

s(t)context-specificselector fiber(Mossy Fiber to PF)

input signal(Mossy Fiber to PF)

–

MF

MF

PFs

+

Operation?Activated “beam” of parallel fibersand lateral inhibition.Together,Possible selection mechanism?(net behavior not definitivelyestablished)

u(t)y(t)

s(t)context-specificselector fiber(Mossy Fiber to PF)

input signal(Mossy Fiber to PF)

–

MF

MF

PFs

common teaching input Climbing fibers (CF)

+

Adaptation of selected synaptic weight?(PF - PC synapse known to be adaptive (decreases) under coincident PF - CF activity, Ito, 1984)

u(t)y(t)

s(t)context-specificselector fiber (MF-PF)

input signal (MF-PF)+–

teaching signalresponding to behavioral error(CF)

cerebellum

cerebralcortex

• A slightly enhanced version suited for gainscheduling studies:

–+ spr

–

+

ref

i2

1/s

ia/s

f2

+

+

–

+

i3

–s gb(i) S()

gk(i) S()

I1(i) S()

+mc

spr

e

P(s)+

+

+

scheduling variables“intent”

“motor command”

cerebellum

cerebralcortex

• A slightly enhanced version suited for gainscheduling studies:

–+ spr

–

+

ref

i2

1/s

ia/s

f2

+

+

–

+

i3

–s gb(i) S()

gk(i) S()

I1(i) S()

+mc

spr

e

P(s)+

+

+

scheduling variables“intent”

“motor command”

statespr

• The problem





Linear combinations of joint state signals

Gainscheduled RIPID model (with torque feedback)

applied to upright balance control

From Jo & MassaquoiBiol Cybern ***

Torque feedback


Gainscheduling by region in state space: 1(t)×3 (t)×d1/dt n = [ n1, n2, n3 ] arbitrary state space direction vectorn = [ -1, n1, n2, n3] augmented direction vectorq(t) = [ 0, 1(t), 3(t), d/dt] bias and state signal vector

Switching plane 0 = n11(t) + n23(t) + n3d1/dt 0 = n q(t)

1(t)

3(t)d3/dt



1(t)

3(t)d3/dt

Balancing region




1(t)

3(t)

Gainset Switching criterion(criteria for activity on given PF): -n q(t) ≥ 0 {Gk

(1), I1(1)}

n q(t) ≥ 0 {Gk(2), I1

(2)}

d3/dt

Balancing region



1(t)

3(t)

Balance coordination trajectories in response to sudden backward platform translations (projected onto 1(t) × 3(t))

RIPID Gainscheduled model Experimental data from ***


RIPID-based models applied to bipedal locomotion

Simulation 1: Model of steady state walking (Jo & Massaquoi, Biol Cybern **)

Model contains

i. Two anti-synchronized 5-state on-off command signals that specify gait cadence and phase for each leg

ii. RIPID based gainscheduled feedback control of trunk pitch.iii. Fixed, linear combinations of muscle activations (synergies)iv. Piecewise linear control onlyv. No internal dynamic models


RIPID-based models applied to bipedal locomotion

Simulation 2: Integrated model of balance and walking (Massaquoi) Model contains

i. Two loosely coupled 5-state state machines with state transition logic that specify gait control phase for each leg

ii. No explicit specification of cadenceiii. RIPID based gainscheduled feedback control of upright posture

and COM position.iv. Fixed, linear combinations of muscle activations (synergies) based

on frog studiesv. Cerebellar control of leg movement dynamicsvi. Piecewise linear control onlyvii. No internal dynamic models


How does the RIPID-based model of balance and walking fare?



B. Models must respect known time delays and phase lags (these are non-trivial)C. Models must utilize functions that are known to be achievable by collections








B. Models must respect known time delays and phase lags (these are non-trivial) model includes full delays and lagsC. Models must utilize functions that are known to be achievable by collections









of neurons. model uses -- piecewise linear functions and low order linear filters, -- no quantitative multiplication of signals with each other, -- neural parameter specification to two significant digits or less.D. Models must be consistent with both normal human function, and








of neurons. model uses -- piecewise linear functions, -- no quantitative multiplication of signals with each other, -- neural parameter specification to two significant digits or less.D. Models must be consistent with both normal human function, and

pathological dysfunction. Comparing with ataxic gait in progressE. Models of human function should be able to be related to those of animals in

a manner consistent with natural evolution (i.e. continuously)








a manner consistent with natural evolution (i.e. continuously) () -- model uses leg control synergies very similar to those of frogs








a manner consistent with natural evolution (i.e. continuously) () -- model uses leg control synergies very similar to those of frogs




F. Models must ultimately account for the firing behavior of system neurons (i.e. must be multi-resolution) () Dr. Takahashi working in

this direction.


Preliminary implications of the model for locomotor physiology and robot control

A. Position/impedance control with series elastic actuation appears to be sufficient

B. Internal dynamic models not required for at least basic locomotor behavior

C. Internal locomotor computations may be simple, slow and require only modest accuracy

D. As a result of C, low power control hardware may be sufficient


Major specific remaining goals for this type of system model

A. More demanding locomotion: Running, negotiating uneven or slippery surfaces, push disturbances etc

B. Self tuning/ adaptation after certain structural parameter change (e.g. trunk or foot mass change)

C. More complex behaviors: skipping, stair climbing, one leg hopping, reaching manual manipulation

likely to need Basal Ganglia modeling


Documents

System Level Neuroengineering Modeling of the Nervous System Cancun Dec 2008 I. Steve Massaquoi i.The Problem ii.Overview of sensorimotor control system