Introduction to modern methods and tools for biologically plausible modeling of neurons and neural networks (2)

Southern Federal University

A.B.Kogan Research Institute for Neurocybernetics

Laboratory for Detailed Analysis and Modeling of Neurons and Neural Networks

Ruben A. Tikidji – [email protected]

2010

Introduction to modern methods and tools for biologically plausible

modeling of neurons and neural networks

Lecture II

mailto:[email protected]

Previous lecture in a nutshell1. There is brain in head of human and animal. We use it for thinking.2. Brain is researched at different levels. However physiological methods

is constrained. To avoid this limitations mathematical modeling is widely used.

3. The brain is a huge network of connected cells. Cells are called neurons, connections - synapses.

4. It is assumed that information processes in neurons take place at membrane level. These processes are electrical activity of neuron.

5. Neuron electrical activity is based upon potentials generated by selective channels and difference of ion concentration in- and outside of cell.

6. Dynamics of membrane potential is defined by change of conductances of different ion channels.

7. The biological modeling finishes and physico-chemical one begins at the level of singel ion channel modeling.

8. Instead of detailed description of each ion channel by energy function we may use its phenomenological representation in terms of dynamic system. This first representation for Na and K channels of giant squid axon was supposed by Hodjkin&Huxley in 1952.

9. However, the H&H model has not key properties of neuronal activity. To avoid this disadvantage, this model may be widened by additional ion channels. Moreover, the cell body may be divided into compartments.

10.Using the cable model for description of dendrite arbor had blocked the researches of distal synapse influence for ten years up to 80s and allows to model cell activity in dependence of its geometry.

11.There are many types of neuronal activity and different classifications.12.The most of accuracy classification methods use pure mathematical

formalizations.13.Identification of network environment is complicated experimental

problem that was resolved just recently. The simple example shows that one connection can dramatically change the pattern of neuron output.

Previous lecture in a nutshell

Phenomenological models of neuronIs it possible to model only phenomena of neuronal activity

without detailed consideration of electrical genesis?

Hodjkin-Huxley style models

Integrate-and-Fire style models

Acc

urac

y ne

uron

des

c rip

tion

Sim

plifi

catio

n

Sop

hist

icat

ion

Reduction of base equations or/and number of compartments

or/and simplification of equations for currents

Spe

ed u

p an

d di

men

s ion

of

net

wor

k

Description of neuron dynamics by formal function

FitzHugh-Nagumo's modelR. FitzHugh«Impulses and physiological states in models of nerve membrane» Biophys. J., vol. 1, pp. 445-466, 1961.

v '=ab vc v2d v3

−u u'= e v−u

Izhikevich's model

v '=0.04 v25v140−u

u '=a bv−uif v30 then v=c ,u=ud

where a,b,c,d – model parameters

Eugene M. Izhikevich«Which Model to Use for Cortical Spiking Neurons?»IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 15, NO. 5, SEPTEMBER 2004

Izhikevich's model

Integrate-and-Fire model

⌠│dt⌡

dudt

=∑ I syn−u t

Simple integrator:

Threshold function – short circuit of membrane:

if u thenu=0

Integrate-and-Fire model

⌠│dt⌡

dudt

=∑ I syn−u t

Simple integrator:

Threshold function – short circuit of membrane:

if u thenu=0

Master and slave integrators

dut dt

=rI t rrap

uapt −u t −ut ap

duap

dt=

1ap

u t −uapt

Adaptive threshold

dui t

dt={

ar

ut −uit if u t ui t

a f

u t −ui t if u t ui t =uit cth

−−

+=

<−<−−−

+=

<−+−−

+=

случаяхостальныхвсехвоtu

CR

tututI

Cdt

tdu

ttеслиUtu

CR

tututI

Cdt

tdu

ttеслиUtu

CR

tututI

Cdt

tdu

ap

ap

firefire

fire

s

sap

ap

fire

fire

s

sap

ap

τ

)()()()(

1)(

τ'2τ

τ

2

τ

)()()()(

1)(

2τ

'τ

2

τ

)()()()(

1)(

Pulse generator:

Modified Integrate-and-Fire model



Com

p ara

tive

char

act e

ristic

s o f

ne

u ron

mod

els

by

Izhi

k evi

ch

Synapses: chemical and electrical

Synapses: chemical and electrical

Chemical synapse models (ion model)

I s=g s u−E s g st =g st s−t

g st =g su ps , t

Phenomenological models

u ps , t =1−1

1expups−

u ps , t =1−1

1exp upst− t −

g st =g su ps , t ,[Ma2+]o ,

u ps , t =P u ps ,t

u ps , t , [Ma2+]o=ups , t g∞

g∞=1[Ma2+]o e

−u

−1

Chemical synapse models (Phenomenological models)

I s={ 0 if tt s

e t s−t if otherI s={

0 if tt s

t s−t

exp1− t s−t if other

I s={0 if tt s

e

t s−t1 −e

t s−t2

1−2

if other

dmi t

dt={

ms

r

−mi

f

if t−t sr

−mi

f if t−t sr

I s=mi t

Learning, memory and neural networksGerald M. Edelman

The brain is hierarchy of non-degenerate neural group

The Group-Selective Theory of Higher Brain

Function

Learning, memory and neural networks

Sporns O., Tononi G., Edelman G.M.

Theoretical Neuroanatomy: Relationg Anatomical and Functional Connectivity in Graphs and Cortical Connection Matrices

Cerebral Cortex, Feb 2000; 10: 127 - 141


Gerald M. Edelman – Brain Based Device (BBD)

Krichmar J.L., Edelman G.M. Machine Psychology: Autonomous Behavior, Perceptual Categorization and Conditioning in a Brain-based Device Cerebral Cortex Aug. 2002; v12: n8 818-830


Gerald M. Edelman – Brain Based Device (BBD)

McKinstry J.L., Edelman G.M., Krichmar J.K.

An Embodied Cerebellar Model for Predictive Motor Control Using Delayed Eligibility Traces

Computational Neurosci. Conf. 2006

Learning, memory and single neuron

Donald O. Hebb


Guo-qiang Bi and Mu-ming Poo

Synaptic Modifications in Cultured Hippocampal Neurons:Dependence on Spike Timing, Synaptic Strength, andPostsynaptic Cell Type

The Journal of Neuroscience, 1998, 18(24):10464–1047

Long Term Depression(LTD)

Long-Term Potentiation(LTP)

Spike Time-Dependent Plasticity(STDP)


Gerald M. Edelman – Experimental research

Vanderklish P.W., Krushel L.A., Holst B.H., Gally J. A., Crossin K.L., Edelman G.M.

Marking synaptic activity in dendritic spines with a calpain substrate exhibiting fluorescence resonance energy transfer

PNAS, February 29, 2000, vol. 97, no. 5, p.2253 2258

Learning and local calcium dynamicsFeldman D.E.

Timing-Based LTP and LTD at Vertical Inputsto Layer II/III Pyramidal Cells in Rat Barrel Cortex

Neuron, Vol. 27, 45–56, (2000)

Learning and local calcium dynamicsShouval H.Z., Bear M.F.,Cooper L.N.

A unified model of NMDA receptor-dependentbidirectional synaptic plasticity

PNAS August 6, 2002 vol. 99 no. 16 10831–10836

Learning and local calcium dynamicsMizuno T., KanazawaI., Sakurai M.

Differential induction of LTP and LTD is not determinedsolely by instantaneous calcium concentration: anessential involvement of a temporal factor

European Journal of Neuroscience, Vol. 14, pp. 701-708, 2001

Kitajima T., Hara K.

A generalized Hebbian rule for activity-dependent synaptic modification

Neural Network, 13(2000) 445 - 454

Learning and local calcium dynamics


Urakubo H., Honda M., Froemke R.C., Kuroda S.

Requirement of an Allosteric Kinetics of NMDA Receptors for Spike Timing-Dependent Plasticity

The Journal of Neuroscience, March 26, 2008 v. 28(13):3310 –3323


Letzkus J.J., Kampa B.M., Stuart G.J.

Learning Rules for Spike Timing-Dependent PlasticityDepend on Dendritic Synapse Location

The Journal of Neuroscience, 2006 26(41):10420 –1042


Letzkus J.J., Kampa B.M., Stuart G.J.

Learning Rules for Spike Timing-Dependent PlasticityDepend on Dendritic Synapse Location

The Journal of Neuroscience, 2006 26(41):10420 –1042

Frey & Morris, 1997

Learning and MemoryOpen issues

from: Frankland & Bontempi (2005)

Learning and MemoryOpen issues

Tools for biologically plausible modelingSimulator Publicat

ions Version

Firstrelease

Latestrelease

Primaryauthor

License MSWindows

Mac OS X Linux Other ActiveCommunity

Language

Emergent (formerlyPDP++ and PDP)

AisaMingusOReilly07

4.0 1986 2008 Dr. RandyO'Reilly

GNU GPL XP, 2003,Vista

Intel, PPC Any,Fedora,Ubuntu

Any Unix emergent-users list,Wiki

C++

GENESIS (the GEneralNEural SImulationSystem)

BeemanEtAl07

2.3 1988 2007 Dr. JamesBower &Dr. DaveBeeman

GNU GPL Cygwin Intel, PPC Yes Any Unix SourceForgelist

C

NEURON (originallyCABLE)

Hines93HinesCarnevale97HinesEtAl06

6.2 1986 2008 Dr. MichaelHines

GNU GPL 95+ Intel, PPC Debian Any Unix NEURONForum

C, C++

SNNAP (Simulator forNeural Networks andAction Potentials)

Unknown

8.1 2001 2007 Dr. JohnByrne & Dr.DouglasBaxter

Proprietary Java Java Java Java Availablebut defunct

Java

Catacomb2 (ComponentsAnd Tools for AccessibleCOmputer Modeling inBiology

Unknown

2.111 2001 2003 RobertCannon

GNU GPL Java Java Java Java No Java

Topographica NeuralMap Simulator

BednarEtAl04

0.9.4 1998 2008 Dr. James A.Bednar

GNU GPL Vista, XP,NT

Build fromsource

Build fromsource

Build fromsource

Mailing list,boards

Python/C++

NEST (NEuralSimulation Tool)

DiesmannEtAl95DiesmannGewaltig02GewaltigEtAl02Djurfeldt08

2.0 2004 2006 Unknown Proprietary Unknown Unknown Unknown Any Unix,build fromsource

NEST-userslist

Unknown

http://grey.colorado.edu/emergent/index.php/Comparison_of_Neural_Network_Simulators

Tools for biologically plausible modeling

Simulator Publications

Version

Firstrelease

Latestrelease

Primaryauthor

License MSWindows

Mac OS X Linux Other ActiveCommunity

Language

KInNeSS - KDEIntegratedNeuroSimulationSoftware

GorchotechnikovEtAl04GrossbergEtAl05

0.3.4 2004 2008 Dr. AnatoliGorchetchnikov

GNU GPL No No KDE 3.1required

No No C++

XNBC VibertAzmy92VibertEtAl97VibertEtAl01

9.10-h

1988 2006 Dr. Jean-FrançoisVIBERT

GNU GPL 9x, 2000,XP

Build fromsource

RPM(Fedora),Build fromsource

Tru 64,Ultrix, AIX,SunOS,HPux

No C++

PCSIM: A Parallel neuralCircuit SIMulator

Unknown

0.5.0 2008 2008 Dr. DejanPecevskiDr. ThomasNatschlager

GNU GPL No No Build fromsource

No No Python/C++

NeuroCAD Unknown

0.00.21a

2003 2007 Dr. RubenTikidji -Hamburyan

GNU GPL No No Yes Any Unix No C

http://grey.colorado.edu/emergent/index.php/Comparison_of_Neural_Network_Simulators

NeuroCAD – Problem definition

To create a computer environment, combining flexibility and universality of script machines, with efficacy of monolithically compiled, high

optimized application.

It would be very nice, if found solution allows to perform computations in homogeneous, heterogeneous and SMP system. Thereby parallelism is included in background of

NeuroCAD project.

NeuroCAD – how to make model?

Step II:Link its by NeuroCAD Engine

shared memory

Step III:Export variable blocks in shared memory of NeuroCAD Engine Step IV:

Connect variables.

Step IV:Connect variables.

Step I:Select and export required modules from modules data bases as c-code and compile it Modules

(shared objects files *.so)Step V:Make modules runtime scheduler and run.

NeuroCAD Benchmarks

NeuroCAD vs. GENESIS ~ 5 – 15 times

NeuroCAD -normal NeuroCAD – tab Neuron – tab0.2740 0.1955 1.1740

1 0.71 4.28NeuroCAD -normal1 6.01NeuroCAD – tab

1Neuron – tab

http://nisms.krinc.ru/[email protected]

http://nisms.krinc.ru/neurocad.org

The big model of Purkinje CellE. DeSchutter J.M. Bower«An Active Membrane Model of the Cerebellar Purkinje Cell»J. Neurophysiology Vol. 71, No. 1, January 1994.

●1600 compartments●12 types of ion channels●Ca2+ concentration dynamics ●Ca2+ dependent K+ channels●Two synaptic types●Three types of dendritic zones ●More than 60 tests and real data comparisons (runtime for some tests in 1994 was approximately two weeks)




●Neuron model – hybrid of H-H and IaF with 4 types of ion channels.●5 types of synapses. Synaptic model includes mediator waste effect.●Predominant anisotropy of network with local formed ensembles.

S. Hill, G. Tononi «Modeling Sleep and Wakefulness in the Thalamocortical System»J. Neurophysiology Vol. 93, 1671-1698, 2005.

●approximately 65000 neurons●approximately 1.5 million synapses●ration number of neurons in model and average cat 1:9

●Three cortex layers and two thalamus layers with modeling of primary and secondary zones of visual perception

Detailed model of thalamo-cortical part of cat vision system

Detailed model of thalamo-cortical part of cat vision system

Bert Sakmann, 2001, Jerusalem

”У меня есть все эти данные – типы клеток,условия их срабатывания, связи, возбудимость дендритов, динамика синапсов, .....Но я не могу понять этого. Я вынужден этомоделировать”

”I have all this data – cell types, firing properties, connectivity, dendritic excitability, synaptic dynamics, ..... But I don’t understand it. I need to model it”

Education

Introduction to modern methods and tools for biologically plausible modeling of neurons and neural networks (2)