Autonomous learning with complex dynamics

Autonomous Learning with Complex Dynamics Hans Liljenstrom* SANS-Studies of Artificial Neural Systems, Dept. of Numerical Analysis and Computing Science, Royal Institute of Technology, S-700 44 Stockholm, Sweden

Traditionally, associative memory models are based on point attractor dynamics, where a memory state corresponds to a stationary point in state space. However, biological neural systems seem to display a rich and complex dynamics whose function is still largely unknown. We use a neural network model of the olfactory cortex to investigate the functional significance of such dynamics, in particular with regard to learning and associative memory. The model uses simple network units, corresponding to populations of neurons connected according to the structure of the olfactory cortex. All essential dynamical properties of this system are reproduced by the model, especially oscillations at two separate frequency bands and aperiodic behavior similar to chaos. By introducing neuromodulatory control of gain and connection weight strengths, the dynamics can change dramatically, in accordance with the effects of acetylcholine, a neuromodulator known to be involved in attention and learning in animals. With computer simulations we show that these effects can be used for improving associative memory performance by reducing recall time and increasing fidelity. The system is able to learn and recall continuously as the input changes, mimicking a real world situation of an artificial or biological system in a changing environment. 0 1995 John Wiley & Sons, Inc.

I. INTRODUCTION

Most natural environments are complex and dynamic, and organisms have evolved nervous systems that can interact efficiently with such environments. In particular, organisms with a nervous system can adapt to environmental changes at three different time scales: (1) a very long one based on genetic changes (evolution), (2) an intermediate one based on permanent synaptic changes (learning), and (3) a short one based on neuronal activity and transient synaptic changes (perception-action). The latter should be in the order of a few hundred milliseconds, or less. The genetic adaptation have resulted in an ini-

*Phone: +46-8-790 6909; Fax: +46-8-790 0930; Email: [email protected]

INTERNATIONAL JOURNAL OF INTELLIGENT SYSTEMS, VOL. 10, 119-153 (1995) 0 1995 John Wiley & Sons, Inc. CCC 0884-8173/95/010119-35

120 LILJENSTROM

tial, more or less “hard-wired,” connectivity of the individual’s neural network. When the individual is confronted with an environment some of its neural connections are modified as learning occurs. These changes to the network connectivity are relatively permanent throughout the lifetime. Fast, highly temporal changes in the network activity appear as part of the neurodynamical processes going on constantly in the nervous system.

For any autonomous system that interacts with an environment it is important that this interaction is efficient in terms of energy, time, and accuracy. Whether the system is biological or artificial, this implies that the response to an external input should give a sufficiently “accurate” and robust input at an acceptable (low) cost in terms of time and energy. It should be able to solve the problem of being stable to noise and external perturbance and, at the same time, be flexible and adapt to significant environmental changes. In general, to achieve a high accuracy requires a long processing time, whereas if the system can suffice with a lower accuracy, processing time can be reduced. In many cases it is most important to minimize processing time. For example, the response or reaction time of an animal in a natural environment or a robot in an industrial real-time operation.

The rich dynamics of biological systems has often been neglected in artificial neural networks (ANNs). Most ANNs are modeled as point attractor systems, where the system dynamics is supposed to bring the network into a stable stationary state, which may represent a memory state of a stored pattern. However, dynamical systems can in general be characterized by three types of attractor behaviors: point attractor, limit cycle attractor, and strange attractor states. Further, many ANNs are also given a static input, which simply may be the initial state. Their dynamics does not evolve in response to a continuous, changing input. It is likely that advances in ANN will depend on including such additional dynamical behaviors. This might imply a closer resemblance with biological neural systems, which have evolved through millions of years to solve problems like pattern recognition, discrimination, categorization, and association. Thus, instead of point attractors, memories could be stored in the network connections corresponding to limit cycle, or even strange attractor states.

Much recent work along these lines, where neural network models are used for the study of real neural systems and their dynamics, deal with or are inspired by the olfactory system. Most of these models consider the olfactory bulb (OB),14 the first relay station of this system, but also the olfactory cor- t e ~ ’ - ~ and the interaction between bulb and cortexlOJ1 has been modeled. For several reasons the olfactory system is considered an ideal model system for the study of associative memory and neurodynamics. Its relatively simple structure (in comparison with, e.g., the visual system) is well characterizedI4J5 and its dynamics has been extensively studied and analyzed. 13~1w3 It displays well-defined oscillations (primarily at theta and gamma frequencies, i.e., around 5 Hz and 40 Hz, respectively) and has a close functional and structural resemblance of associative memory models.

The olfactory system processes odor information, determining the quality

AUTONOMOUS LEARNING 121

and quantity of odor objects in a very fluctuating environment. Odor information arriving from the blfactory receptor cells is “preprocessed” in the olfactory bulb and then transferred as oscillating bursts to the olfactory cortex via the lateral olfactory tract (LOT). The very low number of steps from sensory input to cortical areas is another feature that makes the olfactory system a better candidate for modelers than, e.g., the complex and hierarchical visual system, where there are about ten steps between the sensory input at the retina and the higher cortical areas.I9 Typical for odor information processing is that it results in highly distributed spatio-temporal activity patterns. In contrast to the visual cortex, there seems to be no simple (one to one) mapping of the input and the activity patterns found in the olfactory cortex. The olfactory system seems to lack anything similar to feature-detectors, and the activity patterns associated with a particular input seems more related to the response of the animal rather than directly to the input itself.”

Using EEG (electroencephalogram) and multiunit recordings of bulb and cortex Freeman et al. have found, in addition to regular oscillations, evidence for chaotic-like behavior, i.e., aperiodic behavior different from noise and with some degree of order.13 What is the significance of such behavior? Presumably, the dynamics of the olfactory system, and perhaps of the brain as a whole, reflects an evolutionary pressure to minimize the reaction time to sensory input. Accordingly, the oscillations can serve to enhance weak signals and sustain an input pattern for more accurate information processing, and the chaotic behavior could increase the sensitivity in initial, exploratory states. The neurodynamics of this and related systems can be regulated by neuromodulators, such as acetylcholine. Cholinergic agonists have been shown to affect the oscillatory behavior and, for instance, induce theta rhythm oscillatory patterns in the olfactory cortexZo and in the hippocampus.21*22 Neuromodulators are able to control activity level (excitability), threshold, synaptic transmission, etc. They also seem to be involved in various learning and memory functions.23 Other findings indicate that theta rhythm oscillatory activity is important for memory behavior and for the induction of long-term p ~ t e n t i a t i o n . ~ ~

We have constructed a neural network model of the olfactory cortex which is able to use its complex nonlinear dynamics for a fast information processing in associative memory tasks. Recognition of noisy input patterns can be sped up by oscillatory or near-chaotic dynamics. We have also included neuromodulatory effects in our model in order to investigate the role of controlled neurodynamics for learning and associative memory. By changing a gain parameter the dynamical behavior can shift dramatically and have a significant effect on memory performance. Changing synaptic transmission during learning can enhance performance further. Neuromodulatory gain control can be used in regulating the accuracy and rate of the recognition process, depending on the current demands. In particular, it is demonstrated that neuromodulatory effects can reduce recallh-eaction time considerably. Indeed, intrinsic “neuronal” noise can be used in the same way. We show that such regulatory mechanisms can automatically make the system more or less sensitive to external input and determine the speed and accuracy in the recognition of unknown noisy input

122 LILJENSTROM

patterns, which may change with time. This may have implications both for biological and industrial purposes.

We have tried to make our system as “autonomous” as possible, i.e., with as little interference as possible from the programmer. We give the system an initial network connectivity, grossly resembling that of an evolved real system (the olfactory cortex). Regardless if the system is aimed at simulating aspects of an organism or an artificial automathobot the system self-organizes and gives an output pattern that is dependent on the present connection strengths and the current input pattern. An input pattern is provided by the programmer and the network treats this input in an “autonomous way,” comparing it to previously stored patterns and, if a match is found it gives an appropriate response, otherwise it stores the current input pattern as a novel memory. The new memory is associated with a label given by the programmer (in the general case this could be done by another cortical area or subsystem). An essential idea here is that the system should perform an efficient interaction with the environment mainly by giving a fast (and sufficiently accurate) response to any input. Even though this study is based on the structure and dynamical properties of the olfactory cortex, we believe .it can be generalized to many types of pattern recognition and associative memory.

The general structural and dynamical properties of the current model are summarized in Sec. 11. (A more detailed description is given in Ref. 8). Simula- tion results with neuromodulatory effects on the dynamics and on memory performance are presented in Sec. 111. Finally, in Sec. I V we discuss some implications of oscillatory and chaotic dynamics for biological and artificial information processing.

11. NETWORK MODEL

In order to study the possible relationship between cortical dynamics and associative memory we use a previously developed model of the olfactory cortex.8 This model is of an intermediate complexity with simple network units and realistic connections. The network units correspond to populations of neurons with a continuous input-output relation, describing pulse density characteristics. The gross connectivity is in accordance to known facts about the architecture of the olfactory cortex, mainly based on the circuitry as determined by Habe1-1y.l~ This implies a three-layered structure with two layers of inhibitory units and one layer of excitatory units (see Fig. 1) . The top layer consists of inhibitory “feedforward interneurons, ” which receive inputs from an external source (“olfactory bulb”) and from the excitatory “pyramidal cells” in the middle layer. They project only locally to the excitatory units. The bottom layer consists of inhibitory ‘‘feedback interneurons,” receiving inputs only from the excitatory units and projecting back to those. The two sets of inhibitory units are characterized by two different time constants and somewhat different connections to the excitatory units. In addition to the feedback from inhibitory units, the excitatory units receive extensive inputs from each


+ 4-

Figure 1. The general principle for connections between excitatory units (open circles) in the model, corresponding to pyramidal cells of the olfactory cortex, and inhibitory units (filled circles), corresponding to feedforward (top layer) and feedback (bottom layer) interneurons. Dashed lines indicate that connection strengths decrease with distance.

other and from the “olfactory bulb.” All connections are modeled with time delays for signal propagation, corresponding to the geometry and fiber characteristics of the real cortex.

The time evolution for a network of N neural units is given by a set of coupled nonlinear first-order differential equations for all the N internal states (u). With external input, Z ( t ) , characteristic time constant, T ~ , and connection weight wii between units i and j, separated with a time delay So, we have for each unit activity, ui( t ) , at time t ,

N du. dt Ti j+i 2 = - 3 + c wijgj[uj(t - S,)] + Zi ( f ) + # $ ( I )

The input-output function, gi (ui ) , is a continuous sigmoid function, experimentally determined by Freeman25 and with a single gain parameter, Q, determining slope, threshold, and amplitude of the curve. Spontaneous neuronal activity is added to the system as a Gaussian noise function, #$(t) .

Figure 2 shows the sigmoid curves of Eq. (2) for three different Q values. In this report we let all excitatory units be determined by the same constant Q value, Qi = Qex, and all inhibitory units by a constant Qin . In a more detailed description of specific cholinergic effects the gain parameter Qi of each unit i is made dependent upon the previous unit activity. Then, it is not the gain per se that is changed, but instead an increased excitability is implemented as a suppression of neuronal adaption. This is described elsewhere26 and is not further discussed here.

External input, Zi(t), is given with realistic time delay to each one of the excitatory units as well as to the feedforward inhibitory units, simulating the

124

_ _ - -

Figure 2. The sigmoid input-output relation, g(u), used in the model. A single parameter, Q, determines the slope, “threshold,” and amplitude of the curve, which here is drawn for three different Q values.

afferent input to the cortex from the olfactory bulb via LOT, the lateral olfactory tract. The input is in most cases a random pattern that may be constant or varying in time. When simulating shock pulses to the LOT all input “fibers” will be activated with high amplitude for two simulated milliseconds.

To allow for learning and associative memory the connection weights wij are incrementally changed, according to a learning rule of Hebbian type, suit- able for the dynamics of this particular system. It takes into account that there is a conduction delay, 6ij, between the output (presynaptic) activity of one network unit and its (postsynaptic) effect on the receiving unit. The change in connection strength is also dependent on the absolute value of that connection weight, so that this value cannot exceed some maximum weight strength, w,,,. With learning rate 7 the change at time t in connection weight between unit j and i is given by

( 3 ) Awij = ~gilui(t)lgj[uj(t - &i)(wrnax - wij>

The input, patterns, as they appear in the network activity, is made up of the product of the input vector, and the random connection matrix for input connections to excitatory network units. Each “input fiber” has a (small) probabil- ity of connecting to each one of the 100 excitatory (and feedforward inhibitory) units. This connection matrix is set initially, and will remain the same for all input patterns during a learning/recall session. This would correspond to the “genetically” determined connectivity of an individual. Experimental evidence suggests that there is no synaptic modification of these afferent connection^.^^ Thus, an “ordered” vector like I ( t ) = P f ( t ) , where P is an input pattern vector andf(t) is a (periodic) function of time, will give rise to a random spatial pattern in the network (see Fig. 13). The input patterns are on the form


P = c . ( 11111000001111100000 ........), whereeachoneoftheN= 100elements are multiplied by the same constant c. In the simple case, where we have two input patterns, A and B, their overlap is given by the dot product A . B. The pattern A' is a degraded version of the originally stored pattern A, and may have an amplitude cA, different from cA. It is important to note that the input pattern, as it initially appears, will result in an activity pattern that is not necessarily exactly the same as the input. It is this activity pattern that will be stored in the network connections.

Neuromodulatory effects are here modeled by changing the values for the excitatory units Qex and in some cases also for the inhibitory units, Qin . When neuromodulatory effects on synaptic transmission are included, we change a weight constant that is multiplied with all connection weights, we, for excitatory-excitatory and win for feedback inhibitory-excitatory connections. In cases where the gain and connection strengths change equally for excitatory and inhibitory units we denote the common gain parameter by Q = Qex = Qin , and the common weight factor by w, = we, = win. In the simulations we use 100 excitatory units and 100 of each of the inhibitory types (i.e., totally 300 units). The conduction delays are set so that the network of 10 X 10 units correspond to a 10-mm square of the cortex. In the next section we will present computer simulations where we study the control of the dynamical behavior of this network and the consequences for learning and associative memory. The more direct relationship with biological experiments has been reported elsewhere26 and will only briefly be mentioned here.

In addition to the graphical representation of time series and phase plane plots, we use two of the classical quantitative measures, correlation dimension and Lyapunov exponent, for characterizing the various dynamical states. The correlation dimension is a direct measure of the degrees of freedom, or level of complexity of the system and the Lyapunov exponent is a measure of the exponential divergence (positive values) or convergence (negative values) of two initially nearby trajectories in the phase space of a dynamical system (see the Appendix).

111. RESULTS

A. Complex Neurodynamics

The current model has been shown to reproduce the essential characteristics of olfactory cortex dynamics.* The model describes intrinsic oscillatory properties of olfactory cortex and reproduces response patterns associated with a continuous random input signal and with a shock pulse given to the cortex. In the latter case, waves of activity move across the model cortex consistent with corresponding global dynamic behavior of the functioning cortex. A strong pulse gives a biphasic response with a single fast wave moving across the surface, whereas a weak pulse results in an oscillatory response, showing up as a series of waves with diminishing amplitude. For a constant random input, the network is able to oscillate with two separate frequencies

126 LILJENSTROM

simultaneously, around 5 Hz (theta rhythm) and around 40 Hz (gamma rhythm), purely as a result of its intrinsic network properties. A balance between inhibition and excitation, in terms of connection strength and timing of events, is necessary for coherent frequency and phase of the oscillating neural units. Under certain conditions the system can also display chaotic-like behavior similar to that seen in EEG traces.lJ3 In Figure 3 the principal simulation results are compared with Freeman’s experimental results. Figure 4 shows the oscillatory network response to a constant random input. The time evolution of spatio-temporal activity patterns in the excitatory layer are displayed for one simulated second, with a higher unit activity represented as a larger black square. Each graph represents the network layer of 10 x 10 excitatory units at time intervals of 50 msecs, going from top left to bottom right. Note how the activity moves from one part of the network to another, as a result of the delays in the excitatory and inhibitory connections.

All of these phenomena were shown to depend critically upon the network structure, in particular feedforward and feedback inhibitory loops and long- range excitatory connections modeled with distance dependent time delays. Details concerning neuron structure or spiking activity are not necessary for this type of dynamic behavior.

With a proper choice of parameter values the network can be viewed as two sets of coupled oscillators, each characterized by an intrinsic eigen frequency, One set consists of the excitatory units connected to the “fast feedback” inhibitory units, producing high frequency oscillations. The other set is made up of the excitatory units connected to the “slow feedforward” inhibitory units, producing low frequency oscillations. If the time constants (and

A B C

S t r o n g p u l s e

Weak p u l s e

EEG

Figure 3. Comparing model results with experimental data adopted from Freeman. The response of a neuron and (excitatory) network unit to (A) a strong shock pulse and (B) a weak shock pulse. Real and simulated EEG traces are shown under (C).


I.::.. . * . I ........ :““I . . . ... . . . . . . . . . . . . .

.. .. .. . . . . . . . . ...

.. m.. ..... . m.. .... . . . . . . . ....

’ .. . ..... .. . . . 8 . .... . . . . .

......... . . . . . . . . . . . . . . . . . . _ . . . . . . . . . ‘ a . . . . . . . . . .... . . . . . . . . . . . . . . . . . . . ........ .........

Figure 4. Graphs showing the oscillatory network response to a constant random input. Each graph displays the activity of the 10 X 10 units in the excitatory layer at time intervals of 50 ms, going from top left to bottom right.

delays) in the “two sets of coupled oscillators” differ by a factor of five or more, the system can oscillate with two different frequencies simultaneously, corresponding to the experimentally determined rhythms of the real cortex. If the significant time constants instead are very close, the system locks into a single frequency oscillation. However, for values in between, the two “natural frequencies” of the system interfere with each other and give rise to aperiodic, or chaotic, behavior. (There are no inhibitory-inhibitory connections in this model, since there is no clear evidence for their existence in the real system. Yet, in a test simulation such disinhibitory effects resulted in decreased frequencies in the network oscillations. The oscillations ceased completely for large connection strengths between feedback inhibitory units).

These results are in no way surprising, they are well in accordance with findings from electrical oscillatory circuits .28 Oscillations and chaos are known to appear in systems with feedback loops and nonlinearities under certain conditions, especially when time delays are included. What we have shown is that the circuitry of the brain also can have these characteristics and that the complex dynamic behavior described in the literature does not require any detailed knowledge at the single cell level. Time delays given by the geometri-

128 LILJENSTROM

cal structure of the present model are necessary for the spatiotemporal patterns-single or consecutive waves of activity across the cortical surface-but oscillations and chaotic activity behavior can arise locally even without them.

B. Neuromodulatory Control

Is there any way to regulate the complex dynamics described in the previous section? How can the overall activity of the network be controlled? Several ways of changing the system behavior are possible, but the biological solutions which we consider here seem most elegant and efficient. Yet, these “solutions” may not be primarily aimed at regulating the dynamics of the system, but the dynamical changes could be secondary effects. A very general “self-regulatory” mechanism of neuronal activity is the intrinsic neuronal adaptation, mentioned in the previous section. This property causes the firing frequency of a neuron to slow down after some time of activity, probably due to the accumulation of intracellular calcium that affects calcium-dependent potas- sium channels.2y Hence, the system can avoid run-away (avalanche) effects and exhaustion of (energy and material) resources. Neuronal adaptation, which in itself is an intrinsic effect, can however be regulated by external means. (By external we here mean that it can be determined from systems outside the neurons or network itself, for example from nearby brain areas with centrifugal prqjections to the neurons in the current network.)

In the brain there is, in addition to the network effects due to pure synaptic connections between neurons, a diffuse action of neuromodulators, such as acetylcholine and serotonin. These substances can change the excitability of a large number of neurons simultaneously, as well as the synaptic transmission between them. The concentration of neuromodulators is directly related to the level of arousal or motivation of the individual,13 and can have large effects on the neural dynamics and memory functions. Such effects include suppression of neuronal adaptation, thereby making cells more excitable, and suppression of synaptic transmission of certain synapses. Put in other terms, neuromodulators can affect the weights and gains in the neural network where they act, and this may change the oscillatory properties of cortical structures.2b22

To study the regulation of the network dynamics we here simulate neuromodulatory effects by (1) increasing the gain parameter Q and by (2) suppressing excitatory and inhibitory synaptic transmission (reducing w, and win). The first effect is primarily associated with the level of arousal or alertness of the animal, which in turn could be connected to any of the known neuromodulators.I3 The second effect is characteristic of the particular neuromodulator acetyl~hol ine.~~ This neuromodulator is also known to increase the excitability by suppressing neuronal adaptation, an effect similar to that of increasing the gain in general.2y

The frequencies of the network oscillations depend primarily upon intrinsic time constants and delays, whereas the amplitudes depend predominantly upon connection weights and gains, which are under neuromodulatory control. Implementation of these neuromodulatory effects in the model caused changes


analogous to those seen in physiological experiments. In particular, the increase in excitability and the suppression of synaptic transmission could induce the appearance of theta (and/or gamma) rhythm oscillations within the model, even when starting from an initially quiscent state with no oscillatory activity (in “spontaneous potentials”). The increase in excitability could also result in an enhancement of gamma frequency oscillations in the response to an external input (“evoked potentials”). In Figure 5 we show such effects when the excitability is increased due to suppression of neuronal adaptation, but the same effect is seen if the gain parameter (Q) is increased.

The neuromodulatory effects on the neurodynamics can sometimes be dramatic. For example, the (predominant) frequency of the oscillations in an oscillatory mode can be shifted from high to low (in our case from 40 Hz to 5 Hz), by raising the level of neuromodulation. (This depends on the relative connection strengths involved in the “fast feedback” and “slow feedforward” loops, which allows either one of the natural frequencies predominating). It can also move to a nonoscillatory, constant activity mode or to an aperiodic, chaotic-like behavior. Within the chaotic region, the correlation dimension and the Lyapunov exponent, indicating “the degree of chaos,” can obtain quite different values. For “normal” values of Q, i.e., for 10 < Q < 20 in our case, the correlation dimensions and Lyapunov exponents calculated for the time series produced by the computer simulations agreed with those obtained for real (human) EEG trace^,^'"^ with values between 4 and 5 for the correlation dimension and close to zero for the Lyapunov exponent.

In Figure 6 we show how different oscillatory modes can be induced by neuromodulatory control: increasing gain and decreasing connection weights. The activity evolution of one particular (arbitrarily chosen) excitatory network

Figure 5. Simulated evoked potentials (A) without and (B) with neuromodulatory increase of neuronal excitability.

130 LILJENSTROM

100 ms

Figure 6. Neuromodulators, such as acetylcholine, can change the oscillatory properties of a neural network. To simulate the the cholinergic effects on olfactory cortex dynamics the gain (Q) was increased and the synaptic transmission reduced (by a factor w, for excitatory and win for inhibitory units) when the system was in a nonoscillatory mode. The three traces show the activity of an excitatory unit in three different cases: (A) No reduction of connection strength (w, = win = 1.0); (B) Moderate reduction of excitatory (w, = 0.6) and inhibitory (win = 0.4) connection strengths; (C) Moderate reduction of excitatory (w, = 0.4) and large reduction of inhibitory (win = 0.2) connection strengths. In all cases the gain was increased from Q = 2 to Q = 10 at the onset of neuromodulatory action.

unit is shown for three different levels of suppression of connection strengths. Starting from a noisy, quiscent state with a low Q-value (Q = 2) and momentar- ily increasing it to Q = 10, the system begins to oscillate. The oscillatory mode is determined by the decrease of connection strengths, which is made simultaneously with the increase of gain. For example, if Q = 10.0 and w, = win = 1.0 (i.e., no suppression of synaptic transmission) we can get an oscillatory mode with two different frequencies (approx. 5 Hz and 40 Hz) present simultaneously. This is shown in trace A of Figure 6. If Q is constant (= 10) and w, and win reduced successively the high frequency component can get weaker and eventually be totally eliminated. In trace B of Figure 6 the connection strengths were decreased by 40% for all excitatory units (w, = 0.6) and by 60% for all inhibitory units (win = 0.4). Trace C shows the result for w, = 0.4 and win = 0.2. In the latter case only the low frequency component appears. If the excitatory connection strengths are decreased further, i.e., if w, = 0.3 or less, no oscillations were obtained.

By plotting the activity of any one (excitatory) network unit against any other, we can obtain phase plane plots of various shapes and sizes. These trajectories in phase space can reveal different types of attractor states, characterizing the dynamical behavior in this region (e.g., stable, oscillatory, chaotic).


-10‘

U l l WltI

Figure 7. Phase plane diagram showing the activity of two arbitrarily chosen excitatory units plotted against each other for one simulated second with Q = 10 and we, = win = 1 (case A of Fig. 6).

Figure 7 shows a phase plane plot with the activities of two excitatory units, each with an activity evolution as in trace A of Figure 6.

To explore the dynamical state space as a single parameter is systemati- cally changed, we increase the gain parameter Q incrementally, corresponding to higher levels of arousal (or to the concentration of some neuromodulator in the system). For each Q value the system evolved, after a transient period, to a particular dynamical mode (attractor state), in general more complex for higher Q values. For a particular input pattern, point attractors were obtained for Q 5 4.2. Increasing Q gave limit cycle states up to Q = 6.5. For larger Q values the system behavior evolved into various strange attractor states. If Q was chosen very large, e.g., Q = 50, we again obtained limit cycle states, i.e., oscillations with very high amplitudes. The effect of increasing Q alone is exemplified by Figures 8 to 10. In Figure 8(A) the time evolution for the activity of one excitatory unit is shown when Q = 3.2 . The initial 100 msec of one simulated second is removed in the figure. After a transient period the system goes to a stationary, point attractor state. Figure 8(B) shows the phase plane plot of two excitatory units for Q = 3.2. Figure 9 shows three examples of limit cycle attractors for three different Q values. Chaotic-like behavior is shown in Figure 10 for Q = 15.5, with correlation dimension equal to 4.6 and largest Lyapunov exponent positive, but close to zero. Transient “chaotic” states that eventually converge to a limit cycle (in less than one simulated second), are obtained for certain Q values. We cannot rule out the possibility that all our “chaotic” states eventually would go to limit cycles, if the simulations were allowed to run long enough.

Dynamics of high complexity (with correlation dimension around 4), between that of limit cycles and strange attractors, is obtained for intermediate Q

132

1.68

LILJENSTROM

-

1.675.-

- 3 1.67.

1.665-

4-3.2. 1800 points

I ' l 3 1

I

1

1.125-

1.12

- B 1.115-

1.11

1.105-

1.11 (B) 1.655 1.66 1.665 1.61 1.675 1.68 1 ~ ( 2 4 )

-

-

85

Figure 8. Point attractor dynamics for Q = 3.2. (A) The time course of one excitatory unit during one simulated second. (B) The phase plane plot of two excitatory unit activities plotted against each other. For clarity, the transients during the initial 100 ms of the simulation has been removed.

values (Q - 10). This behavior resembles real EEG data under certain circumstances, and could in fact be more interesting from a physiological point of view than the other dynamical behaviors. In Figure 11 we show a simulated EEG trace along with its (1og)power spectrum and several different phase plane plots for the activities of arbitrarily chosen pairs of excitatory network units. Charac-

AUTONOMOUS LEARNING

0.9

0.8 G N

0.7

0.6

0.5

133

1 -

-

-

-

-

-

1.2 1.4 1.6 1.8 2 U(M)

0.8 (A)

Q=5.2,500 mscc. 1.1,

2

0 0.5 1 1.5 2 2.5 0.4'

(B) U ( W

Figure 9. Limit cycle dynamics for three different Q-values. Phase plane plots of two excitatory unit activities, when the initial 500 ms of the one-second simulation has been removed, for (A) Q = 4.2, for (B) Q = 5.2, and for (C) Q = 6.4. (D) Log power spectrum for Q = 4.2, showing a clear frequency peak at around 40 Hz.

teristic of this dynamic behavior is that it is more ordered than the strange attractor states obtained for higher Q values.

In addition to neuromodulatory gain control, the system can increase its excitability by raising the mean (background) firing frequency of cortical neurons. Neuronal noise, due to, e.g., spontaneous neuronal activity, may have

134

Q4.2 10

8 -

6 -

4 -

LILJENSTROM

Q=6.4.500 msec.

I 0 0.5 I 1.5 2 2.5 3

(0 ~(24)

Figure 9. Continued.

such effects, and although intrinsic in nature this too may be under external regulation. Spontaneous activity of neurons is omnipresent, and can result from single ion channel openings.34 Yet, it is not known if this has any biological significance or if it can be regulated in any way, e.g., by some neuromodulator. Noise added to our system can have several interesting effects on the system dynamics. For example, (Gaussian) noise at a certain amplitude can induce theta frequency oscillations in the system. Preliminary results also indi-


Q=l5.5

I

4

2

0

-2 - ",

-4

-6

-8

-10

Q=lS.5, power specuurn of u(3) 10

8

6

4

2 - ti * Y x o s"

-2

-4

-6

-8

-10 (B) 0 100 200 300 4C4 500 600 700 800 900 loo0 Frequency

Figure 10. Strange attractor dynamics for Q = 15.5. (A) The time course of one excitatory unit activity for one simulated second. (B) Log power spectrum obtained for one of the unit activities during one simulated second. (C) Phase plane plot of two excitatory unit activities.

cate that transitions between oscillatory or chaotic states can be induced by such noise.

Noise and the neuromodulatory effects mentioned here were also simulated in the context of learning and associative memory. Below we will discuss the results when the gain constant Qex and the connection weight constant w,

136 LILJENSTROM

Q=15.5, D2=4.6 6r

4 -

2 -

0 - h

0 I =

-2 -

-4 -

-6 -

2 4 U P )


are changed. Results with suppressed neuronal adaptation instead of increasing Qex show qualitatively similar effects. We also change the amplitude for the added Gaussian noise to investigate the effects of neuronal noise on memory performance.

C. Pattern Recognition and Associative Memory

A general notion is that sensory information can be stored in synaptic connections of cortical neural networks. Signals from the sensory receptor cells reach the cortex, and result in some kind of more or less distributed activity patterns in the cortical network. According to Hebb’s learning rule35 the simultaneous activity of two neurons can cause changes in the synaptic connection strengths between them. These changes alter the system and could create new attractor states, corresponding to a memory of a particular percept. In this way a percept, like an odor object, is stored as a pattern in the network.

In most artificial neural networks, like in the traditional Hopfield net, input patterns are stored in the network connections giving “memories” that are associated with point attractor states. Recall of a pattern is a result of a relaxa- tion, or convergence, to the point attractor which lies closest to the network activity caused by the current input pattern. Most often, this “input pattern” is only the initial state of the network, and not a continuous external input, as would be the case in a real-world situation. Depending on what time scale we use, the functioning brain would rarely be in a stationary state, but is continuously changing its activity. This would suggest that there are no point attractor states lasting a longer period of time, unless one considers deep anesthesia or death, I 3 Memories could possibly be related transiently to point attractors, but


4

i ; i 2

$ 0 u

-2

-4

-8 -IL--i 200 400 600 800 1000

-loo

(A) Time [ms]

6

4

2

- 0 f g .2

3 A -4

-

-6

-8

-10

100 200 300 400 -12'

0 (B) Frequency

0

Figure 11. Complex attractor dynamics for Q = 10. (A) Simulated EEG obtained as the weighted sum of activities of excitatory units around an imaginary measuring point; (B) Log power spectrum obtained for one of the unit activities during one simulated second; (C) Six different phase plane plots obtained when the activities of different pairs of excitatory units are plotted against each other (same simulation).

it seems more realistic to associate them with limit cycles or strange attractor states, if memories are at all related to attractor states.

Using our model of the olfactory cortex we investigate whether complex dynamics can be useful for learning and recall, and demonstrate that neuromodulatory control of this dynamics can improve performance considerably. We show that such effects can result in faster learning and recall, or alternatively, in a more accurate association of a new input pattern with a previously stored pattern. There is a trade off between reaction time and accuracy and this trade off could presumably be externally regulated. The level of neuromodulators during recall, corresponding to the state of arousal, could also determine the

138

4 -

2-

0 - - & -2- 2

-4

-6

-8

LILJENSTROM

6;

-

-

-

8 , I

6-

4 -

2- - - '1

0-

- 6 1 - -6 -4 -2 0 2 4 6

(C3) N l )


I


5,

-61 .5 -4 -3 -2 -1 0 1 2 3

~ ( 6 ) (W


i

140 LILJENSTROM

level of accuracy necessary for the problem. In our case, memories are associated with limit cycle attractor states, which can be reached quickly after a transient period of aperiodic behavior.

In all simulations here we use the recall, or convergence time as a performance measure. The program halts when the network activity has converged to any of the stored memory states within a certain accuracy, or “convergence limit,” a. During the recall phase the inner products of the current input vector (pattern) and all the previously stored vectors (patterns) are taken. If this product equals one, or exceeds the convergence limit a, for any of the stored patterns, the current input pattern is identified with that pattern. In general, the higher convergence limit the longer convergence time, i.e., it takes longer time before the program halts.

One problem when storing a new memory in a neural network, cortical or artificial, is that previously stored patterns can interfere with the new input pattern, resulting in an inefficient learning and inaccurate r e ~ a l l . ~ ~ . ~ ~ To store a new pattern efficiently it should be important to strongly activate the specified set of neurons (or network units) and at the same time suppress the synaptic transmission between these neurons and those not given afferent input. This dual effect seems to be carried out by a neuromodulator like acetylcholine in the olfactory system, and we implement that in our simulations.

Starting with random initial weights (but constrained by the overall connectivity scheme evolutionary determined for the olfactory cortex), we store activity patterns by modifying the connection weights. With the modified Heb- bian learning rule previously described we train the network with input patterns that may or may not oscillate. Oscillating input would correspond to the type of odor information that is used in the real olfactory system. The patterns are stored in the network connections, associated with limit cycle attractor states, i.e., the recalled pattern will oscillate with gamma frequency at memory re- trieval. Learning with neuromodulatory effects is faster than without, but the main effects are on recall time and accuracy. Due to a more efficient learning with suppression of memory i n t e r f e r e n ~ e , ~ ~ , ~ ~ a higher accuracy in matching (“associating”) a degraded input pattern with any stored pattern is possible.

Simulation results seem to show that oscillations do reduce recall time. For example, if two patterns have been stored with Q = 8 and constant input, the (nonoscillatory) output during recall converged to the point attractor memory state after 45.5 ms. If instead, the same two patterns were stored with oscillatory input, the system converged to a limit cycle (oscillatory) memory state after 27.5 ms, i.e., almost 50% shorter recall time than for the stationary case. However, if Q = 12 for both oscillatory and nonoscillatory input during learning and recall, the output is always oscillatory and recall time is almost equal for the two cases. It should, however, be noted that a fair comparison between an oscillatory case and a nonoscillatory one is difficult to make, since momentary amplitudes in the oscillations may have a different net effect than a constant amplitude in the stationary case (even if the mean values are approximately the same).

In order to demonstrate the effects of “neuromodulation” (changing gain


and connection strength) on the network dynamics, we simulate a semirealistic situation. An unknown input pattern is presented to the network for 200 (simulated) msecs, corresponding to the approximate length of a sniff cycle (5 Hz oscillations) for a small mammal (rodent). In the simplest case this input is given with a constant amplitude, but more elaborate functions have been tested as well (e.g., inputs with 5 Hz oscillations superimposed by 40 Hz oscillations). The random input pattern, which should correspond to the activity induced by odor molecules stuck at the olfactory receptor cells and transmitted to the cortex via the olfactory bulb, is given either with a constant amplitude, or as an oscillating input. We here present only simulations with input patterns with constant amplitude.

Learning of a new sensory (odor) input is thought to be possible by a single burst, i.e., it can take place within approximately 200 ms. (In reality, this is probably too short, but is used here for simplicity). The first 100 ms of this period in the model is devoted to “recall,” i.e., during this time the system tries to match or associate the current input pattern with any of the stored patterns. If the inner (dot) product of the vector for the current pattern and any of the stored patterns equals one, or exceeds the accuracy measure (“convergence limit”) a, or any of the stored patterns the current input pattern is identified with that pattern.

Initially, the gain constant, Qex, is kept at a medium level (simulating a “normal” arousal level of an individual). When a (degraded) input pattern is presented to the network, that pattern is “compared” with the previously stored patterns (if any) at a constant gain. If a match is made the system has converged to one of its memory states. However, if the current pattern has not been matched with any of the previous patterns within 50 ms, the gain is automatically raised and the “search” will continue for another 50 ms at this higher gain value, corresponding to a higher motivational or attentional state. If still no match is made when 100 ms have passed, the system will start to learn the current pattern for yet another 100 ms at this high gain constant, and store it as a novel pattern with an appropriate label. This latter half of the 200 ms period can in principle be without direct external input, since the current pattern can be kept by sustained oscillations. Suppression of synaptic transmission is also introduced for the learning session in some of the simulations. This is done by reducing the connection weight constants, we, win, which here are equal, w,, = win = w,, and always unaffected during recall. The procedure is repeated for each new “sniff cycle” of 200 ms: the current input pattern is compared with previously stored patterns during the first 100 ms and then, if no recall is made (no convergence to any memory state), learning of that pattern will take place. In Figure 12 this situation is displayed with 500 ms of recall and 500 ms of learning, in order to make the attractor states more clear.

Figure 13 shows a case where two patterns, “apple” and “banana,” have been stored and a grossly degraded version of “apple” is given as an “unknown” (constant) input. These three patterns are shown in the three upper frames of the figure. The second and third rows of frames show the oscillating network activity induced by the current input, each frame corresponding to a

142 LILJENSTROM

6-

5 -

P 4 . B m = 3-

I

I

2 -

Figure 12. Phase plane plot for one simulated second with recall and learning. The two separate attractors correspond to two separate memory states. A degraded version of a previously stored pattern is given as the input to the network. During the first 500 ms the network tries to recall (converge to) the stored memory state, represented by the large central attractor. When a convergence at the required accuracy fails, the system starts to learn the current input pattern as a novel attractor memory state. (See the text for further description of the simulated recall-learning situation).

snapshot every 10 ms. When no match has been made after 50 ms Qex is increased from 10 to 15 and the system can converge to the stored “apple” memory state. The simulation halts at t = 87.5 ms, when the dot product between the activity caused by the current “unknown” input pattern and the previously stored “apple” pattern exceeds the convergence limit, a = 0.9. The mean activity during one cycle is shown in the lower right corner of the figure. This pattern resembles the “apple” pattern in the upper left corner, but is not identical to it, since the network connectivity has been changed by learning a second pattern (“banana”), after the first one (“apple”) was stored.

In Table I we show how the convergence time depends on the gain and synaptic transmission during learning and recall, when the convergence limit is a = 0.9. The convergence time is given for one of two stored different (but partially overlapping) patterns. (Similar results are obtained when more patterns are stored. We have not yet investigated the storage capacity for this system, but a test with ten partially overlapping patterns still gave good results, although with slightly worse performance. Here we display the results when changes in gain constant and synaptic transmission take place only for the excitatory units, but the results are qualitatively the same when the constants for the inhibitory units are modulated as well.

The table shows that learning with increased gain and reduced connection weights results in shorter convergence time. If learning occurs with Qex = 10, there is no recall during the first 50 ms presentation of a 20% degraded input. After this time, Qex is increased from 10 to 15, and convergence takes place at t = 87.5 ms after presentation. If instead, learning is performed with Qex = 15,


k= 4.283 Hln = -6.679

uc f ina l = 1.0 Tim = 87.5 na

P s f . = 0.906

.. . . . @X f l d 15.0

Figure 13. This figure shows an example of recall in the toy situation described in the text. Two patterns, “apple” and “banana” have been stored in the network connectivity and an “unknown” pattern is given as a constant input. The network activity produced by these three patterns are shown in top row of graphs. The “unknown” pattern is a grossly degraded version of the “apple” pattern. The next two rows of graphs show the network response to the “unknown” input pattern at 10 ms time intervals. Recall, i.e., convergence to one of the memory states (“apple”) occurrs after 87.5 ms. The program halts when a preset convergence limit (a = 0.9) is passed and the mean network activity, averaged over one cycle, is shown in the graph at the lower right corner of the figure. In this particular case there were no neuromodulatory effects during learning, but gain is increased (from Q = 10 to Q = 15) during recall.

convergence occurs at t = 64 ms, after Qex had been increased from 10 to 15. If all (excitatory) connection weights are reduced by 50%, i.e., w, = 0.5, during learning with high gain, then recall is made before Qex has been increased, i.e., with Qex = 10. Beneath the line in the table the result in convergence time is shown for various values of Qex during recall, when learning had occurred with Qex = 12 and w, = 0.5. This demonstrates that the gain, or “level of arousal,” during recall also is of great importance for the convergence time, especially for low Qex values.

In Table I1 we show how the convergence time during recall, T, , depends on gain and connection strength when two partially overlapping patterns, A and

144 LILJENSTROM

Table I. Convergence limit a = 0.9. Learning Recall Convergence Time

Oar W e , om T, (ms) 10 I 10+ 15 87.5 10 1 15 48.0 10 1 20 44.0 15 1 10+ 15 64.0 15 1 20 39.5 15 0.5 10 29.0

The dependence of recall/convergence time, T, , on gain and synaptic transmission during learning and recall of two different input patterns. The gain for all excitatory network units is determined by the gain constant QeI, as given by the text. All excitatory connection weights are multiplied by a weight constant w,, which is reduced at cholinergic modulation. Arrows indicate transitions of Qcx occumng 50 ms after presentation of each input pattern. The convergence limit requires for recall is here a = 0.9. The data clearly shows that neuromodulatory effects can significantly reduce recall time.

B, have been stored. The input patterns, A' and B' are degraded versions of A and B, respectively. The input amplitudes for the undistorted patterns during learning are CA = cE = 1.5, whereas for the distorted patterns we have cAt = 1 .O and cg' = 1.2. The convergence limit is here a = 0.7, which is lower than in the simulations presented in Table I. In general, this gives shorter convergence (recall) times. If learning occurs with constant gain and synaptic transmission, and the gain (Qex) is increased during recall only, the convergence time is decreased. (Notice that the two patterns are unequally "hard to recall"). How- ever, if learning occurs with increased gain and constant synaptic transmission, learning is faster, but the convergence time during recall is longer due to a stronger interference of the input patterns. This interference can be reduced if the synaptic transmission is suppressed simultaneously with an increased gain

Table 11. Convergence limit (Y = 0.7. Convergence time

Learning Recall Pattern A ' Pattern B' Qex Wex QeX T, (ms) T, (ms)

Case I 10 1 10 27.0 33.0 10 1 15 23.5 30.0 10 1 20 22.5 28.5

Case I1 15 1 10 36.0 35.5 15 1 15 26.5 22.0

Case 111 15 0.5 10 13.5 10.0 15 0.5 15 13.0 9.5

~~ ~~ ~ ~ ~~~

Recalllconvergence time, T, , for different degrees of neuromodulatory control of gain and connection strength during learning and recall. Two partially overlapping patterns, A and B, have been stored and A ' and B' are degraded versions of A and B given as input. See text for explanation.


during learning. Then, as is seen for case I11 of Table 11, T, is reduced considerably, to one half or one third of the convergence time in the unregulated case I. It is worth noting that T, is larger for pattern B’ than for A’ in case I, but shorter in cases I1 and 111, where the gain is increased during learning. This demonstrates the effect of pattern interference when no neuromodulation of connection strengths is present. Pattern B, which intrinsically is somewhat closer to B’ than A to A’, is learned after pattern A has been stored and its recall will thus be affected by the activity caused by A’s modified connections.

Although neuromodulatory effects of acetylcholine are thought to be presented only during learning,30p36 we also demonstrate that an increased gain also during recall can have a profound effect on memory performance. Table 111 explicitly shows the effects of increasing the gain during recall when the gain has been moderately increased during learning. It is clear that recall time can be reduced significantly if neuromodulatory gain control is applied also during recall.

To summarize the results on associative memory tasks, learning with increased gain (or suppressed adaptation) is faster than with low gain. Recall time is in general also reduced, but due to interference by previously stored patterns during learning this reduction is diminished. In some cases, increased gain alone can even increase recall time due to stronger memory interference. How- ever, if learning is performed with increased gain and at the same time with suppression of synaptic transmission (reduced connection weights), both learning and recall time is reduced. This situation provides the best memory performance, with respect to time and accuracy, since memory interference is reduced during learning. For example, a 50% increase in gain together with a 50% reduction in synaptic transmission could result in up to 70% reduction in convergence time during recall. Further, a grossly degraded (50%) oscillatory input pattern can cause the system to rapidly converge, within approx. 30 msec, to an oscillating activity pattern corresponding to the stored uncorrupted pattern. These results are by no means final. For example, we have not investigated the optimal suppression level of synaptic transmission. Experimentally, cholinergic suppression of synaptic transmission is shown to have a mean value around

Table 111. Learning with Qex = 12 and w, = 0.5 Convergence Limit a = 0.9.

~~

Recall Convergence time QPX T, (ms)

5 - 10 87.5 10 34.5 15 32.0 20 26.0

Neuromodulatory increase of gain, Qex, during recall can reduce recall time, T,, considerably.

146 LILJENSTROM

70% during learning. In our model it was difficult to maintain an oscillatory activity with much larger suppression than 50%, which was used in the simulations. However, the aim here has not been to optimize the system, neither to simulate in detail a physiological realistic situation, but rather to demonstrate that recall time can be reduced by increasing gain and decreasing connection strengths in a semirealistic artificial neural network.

In addition to these neuromodulatory effects on memory performance, neuronal noise can also reduce the recall time under certain circumstances. Elsewhere37 we show that a minimum recall time is achieved for an optimal value of the amplitude of Gaussian noise added to the system. For example, a four-fold increase in noise amplitude could, for a particular case, reduce recall time by about 30%. A further increase in noise amplitude resulted in longer recall time.

IV. DISCUSSION

The complex dynamics of the brain, including oscillations at various frequencies as well as aperiodic, chaotic-like behavior, could have many explana- tions. The simplest would be to assume it is only an epiphenomenon, a bipro- duct of the network circuitry or neuronal properties. Although this possibility cannot be ruled out, there are yet indications that it may have a functional role for information processing. Several ideas linking the oscillations (in particular the 40-Hz oscillations) to attention and consciousness have been made. One such theory38 was based on the zero phase stimulus-invoked synchrony between visual cortical neurons far apart.3+” Yet, the functional role of cortical oscillations, as well as their causal origin, is still largely an open question. With our model of the olfactory cortex we approach both of these problems. We have in this article tried to demonstrate a possible link between cortical neurodynamics and associative memory, and even though we have studied a particular system we believe the qualitative results are general and can be carried over to other biological or artificial systems with the same objective.

The model has been shown to reproduce most of the dynamics of the real system as determined by physiological experiments, in particular, oscillations at two different frequency domains, as well as chaotic-like behavior under certain circumstances. Also spatiotemporal patterns associated with memory storage in the olfactory cortex have been demonstrated. Neuromodulatory control of the complex dynamics has been shown to be useful for memory performance. The primary objective has not been to maximize the memory capacity in terms of number of stored patterns, but rather to show that the complex dynamics of the network can be used for reducing recall time in associative memory tasks. Yet, at this point we have not been able to present any conclusive evidence for a functional link between cortical neurodynamics and memory. It is difficult to make a fair comparison with stationary associative memory models, since many of the properties of the current model are constrained by the anatomy and physiology of the olfactory cortex. If, as we have done in test simulations, the model is made stationary by deleting or


reducing the inhibitory connections a comparison is complicated by the altered balance between excitation and inhibition. This balance is important for the associative memory properties, as well as for the oscillations. However, some of the described effects may be due to a higher general activity as a result of an increased gain, especially in cases where the system converges to a memory state before it reaches an oscillatory mode. We investigate these problems further, using a modified version of the model and different types of learning rules.

In addition to the circuitry effects of the current model, other mechanisms may be responsible for, or contributing to, the oscillatory behavior of the real system. Thus, gamma frequency oscillations can result from spike-synchronization of a large number of individual n e u r ~ n s , ~ * ~ ~ ~ or as a resonance phenome- non where a few pacemaker neurons oscillate with this frequency.44 The theta frequency oscillations may result from neuronal adaptation, or originating from systems outside the cortical structure described here. Here we do not explicitly treat effects of individual spiking neurons or externally governed oscillations, but by introducing neuronal adaptation in the model it is possible to get oscillatory activity even without involving inhibitory circuits. In this case, the oscillation frequency depends on how fast the neuronal activity drops as an effect of adaptation. In particular, the low frequency oscillations, originally dependent exclusively on the time constant of the feedforward inhibitory units in the model, can now also be produced by this adaptation effect. It has recently been suggested that intrinsically bursting cells are necessary to explain theta rhythm oscillations found in hippocampal slices, where inhibition was blocked4’ but effects of neuronal adaptation may provide an alternative explanation. In the absence of external input and feedback inhibition, such oscillations can be induced by intrinsic noise in the system.37

Our approach here has not been to focus on the causal origin of the oscillatory dynamics, but on its functional role. We regard the complex dynamics of the brain, exemplified by the olfactory cortex, as an evolutionary optimized strategy to deal with rapid changes in the environment. It is likely that biological systems have evolved to become efficient with respect to time, energy or accuracy. A highly advanced system would be able to shift the strategy for different situations, within the limits set by evolution, and oscillations could be an appropriate way to deal with this. First, oscillatory or complex dynamics provides a means for fast response to an external input. If sensitivity to small changes in the input is desired, a chaotic-like dynamics should be optimal, but a too high sensitivity should be avoided. Oscillations can also be used for enhanc- ing weak signals and by “resonance” large populations of neurons can be activated for any input. Secondly, such “recruitment” of neurons in oscillatory activity can eliminate the negative effects of noise in the input, by canceling out the fluctuations of individual neurons. Noise can, however, also have a positive effect, which we will return to shortly. Finally, from an energy point of view, oscillations in the neuronal activity should be much more efficient than if a static neuronal output (from large populations of neurons) was required. In engineering, great efforts are made to eliminate oscillations in the system, but if

148 LILJENSTROM

the system can perform as well (or better) with oscillations, energy can be saved.

Simulation results show that oscillations during the recall phase can be used for a fast convergence to a memory state. This would be especially useful for patterns that are difficult to distinguish, and where a longer “search process’’ is necessary. Oscillatory “resonance” could thus quickly recruit all network nodes belonging to a particular memory pattern (cell assembly). For simple cases, when the required recall time is less than a cycle, it is doubtful if the dynamics of the system can yield a faster recall time than a fixed point attractor model. Further, with the learning rule we used here we have not been able to show that oscillations during the learning phase also would be benefi- cial. At present we are investigating learning rules that better will make use of the complex dynamics and thus presumably further improve memory performance. From experimental studies on hippocampal tissue it is known that low frequency (around 5 Hz) oscillations are optimal for inducing LTP (long term p~ ten t i a t ion ,~~ thought to be related to learning in this system.

If the dynamical behavior is important for the performance of the system, “neuromodulatory’ ’ regulation of gain and/or connection strengths should be an efficient way to control this dynamics. It is known that the level of arousal and neuronal excitability can be regulated by neuromodulators, such as acetylcholine and serotonin, but also by the spontaneous activity of the neurons themselves. A complex system (like the brain) can be kept near a threshold to a highly active state by noise or “chaos,” due to the more or less uncorrelated activity of its constituents (the neurons). Away from any threshold, noise can also have a stabilizing effect on the system. An “appropriate” activity, e.g., finding a particular memory state, could emerge as a collective behavior of the parts if the system is pushed above this threshold by any correlated input, e.g., presented as a spatio-temporal pattern of activity. As demonstrated, an oscillatory mode can be induced by increasing the neuronal excitability, but also by noise in a system with only excitatory network units modeled with neuronal adaptation. Preliminary results also show that recall time can be minimized for optimal noise amplitudes, and that transitions between different attractor states can be induced by noise.

Maintaining an adequate accuracy in matching an unknown input to any stored memory, and subsequently to an appropriate action program, is dependent on, e.g., the degree of overlap or interference between different memory states. This kind of interference can be reduced by suppressing synaptic transmission during learning, an effect possibly carried out by acetylcholine. In fact, the cholinergic dysfunction accompanying memory deficiency in Alzheimer’s disease has been suggested to be linked to a runaway synaptic modification due to interference during learning.30 Here we have shown that suppression of synaptic transmission in the learning phase can dramatically reduce the recall time with maintained accuracy. Alternatively, the accuracy in pattern matching would increase if recall time is kept constant.

If neuromodulatory effects are to act differently during learning and recall, as is the idea in the recall-learning situation described above, there should be


some mechanism that triggers the onset and offset of the neuromodulatory action. Conceivably, the onset and offset of neuromodulatory release could be sensitive to the coherent neuronal activity in the system. Such coherent activity can be produced by the oscillations corresponding to a limit cycle memory state. If no coherent network activity occurs within a certain time, i.e., if there is no convergence to a memory state, neuromodulators could be released and their primary action could be to enhance the neuronal excitability. The suppression of synaptic transmission could possibly be delayed and be effective at a later stage, allowing for memory recall at a higher gain, before learning of the input pattern takes place. Neuromodulatory action would be turned off when a (limit cycle) attractor state has been reached and maintained for some time.

High speed in the neural system of an organism may in many cases be much more important for survival than high accuracy. There is a trade-off between speed and accuracy, so even if high accuracy is preferable, it may cost too much time to be practically realizable. (Accuracy also costs energy, but that should usually be abundant in the brain and not limiting). It is likely that evolution has optimized the neural system in such a way that high speed has been favored at the expense of a lower accuracy. The oscillatory or chaotoid states of the system would give just this. Presumably, such a strategy could be utilized in any system where a rapid recognition or response to any (sensory) input is needed, perhaps also be carried over to technical applications of artificial neural networks. Indeed, a neural network with complex dynamics has already been successively tested in an industrial app l i~a t ion .~~

There is reason to be careful when characterizing nonlinear dynamical states, since the methods used for analyzing the time series obtained may not be accurate (stable) and in principle would require “infinite” time to be correct. The rather short time available for analysis, whether of EEG traces or simulated network activity, may in general be inadequate for determining if it is “true chaos” or not. If long time series are considered, EEG traces are highly nonstationary, and deterministic chaos requires stationarity . Some chaos-Iike attractors may in fact be nonchaotic, and many of the earlier demonstrations of chaos in biological data have been shown to be spurious.47

The observed highly aperiodic and irregular behavior in the olfactory cortex, as well as in the current model of it could, in some sense, be classified as “chaotic.” Adding the electrical activity of many neurons, oscillating at different frequencies and phases, could easily yield the collective chaotic-like behavior apparent in the EEG. However, it may not be important if the dynamics of the olfactory cortex, or any other brain structure, is “true chaos” in a mathe- matical sense, or not. The importance could rather be to have a disordered aperiodic state, with a great deal of “uncertainty,” which readily switches over to an ordered oscillatory or other state. Such a “chaotoid” state would give the system flexibility and rapid response to input patterns. In this sense, “chaotic” states would probably be most useful in biological systems if they were transient, i.e., if they did not persist any longer periods of time. One of the characteristics of chaotic dynamics is that the solutions tend to diverge asymptotically when starting from initial values that are arbitrarily close. Computation in

150 LILJENSTROM

neural systems traditionally demands a convergence. For instance for associative memory, if the input is somewhat corrupted or noisy, the system should still converge to the same solution, a memory state stored in the network connections. Most computations in neural systems should be done on a time scale on the order of 100 msec. If one has to wait for a second or longer to determine any characteristics of “true chaos” it is probably of no biological significance.

We have not here been primarily concerned with characterizing the complex dynamics of our system as chaotic or nonchaotic. Our primary objective has been to show that the nonlinear dynamics displayed by this system can be useful for information processing. This nonlinear dynamics certainly yields more complex behavior than limit cycle behavior, and we have used the available techniques (correlation dimension and Lyapunov exponents) for characterizing, qualitatively, the relative complexity of the behavior. We believe that chaotic (or chaotoid) states would be most useful if they were merely transient. It could reflect an attentive, exploratory state of an animal, or the sequential tracing of memories or a thought process, as the system is moving between different limit cycle (or strange attractor) memory states. This relates to Free- man’s theory for perception and creative thinking.I3

The author is deeply grateful to Wu Xiangbao for aiding in the analysis of the nonlinear dynamics, and to Anders Lansner for valuable comments on the manuscript. Financial support from the Swedish Natural Science Research Council is also gratefully acknowledged.

APPENDIX

For analysis of the complex dynamics of our system, we use the classical quantitative measures, the correlation dimension and the Lyapunov exponent. When a system produces irregularity in one of its variables it is possible that this behavior results from randomness (implying that the number of degrees of freedom is infinite) or, that a finite, and possibly small number of degrees of freedom is correlated with the attractor (implying that the system is deterministic). The correlation dimension measures the degree of freedom of the system. Analyses of time series have been based almost exclusively on the Takens reconstruction of the multidimensional signal and on estimating its correlation dimension by the Grassberger-Procaccia algorithm.48 For a time series x( I), x ( 2 ) , . . . , x(n), one reconstructs rn-dimensional vectors (rn is called the embedding dimension), Y(i) = (x( i ) , x ( i + a), . . . , x ( i + ma - a)), for i = 1 , 2, , . . , N where N = n + 6 - ma. The correlation function for a small number, E, can hence be calculated as follows,

where H ( x ) = 0 if x < 0, otherwise H ( x ) = 1. The correlation dimension is the limit


To estimate the correlation dimension, we take the time delay 6 as the first cossing of the zero line by the signal autocorrelation function. For random time series, one could not get a saturation when the embedding dimension m is increasing. If it reaches a saturation as m increases, that value can be taken as the dimension.

The Lyapunov exponents measure quantities which constitute the exponential divergence or convergence of nearby initial points in the phase space of a dynamical system. A positive Lyapunov exponent measures the average exponential divergence of two nearby trajectories whereas a negative Lyapunov exponent measures exponential convergence of two nearby trajectories. If a discrete nonlinear system is dissipative, a positive Lyapunov exponent quantifies a measure of chaos.49 If the largest Lyapunov exponent is very close to zero, it means the system is not sensitively depending on initial conditions. The system is then not chaotic, but is regarded as strange n o n c h a ~ t i c . ~ ~ , ~ ~

Let us consider the following system of differential equations, dxldt = F ( x ) , where x belongs to Rm. The evolution of a tangent vector e in the tangent space at x ( t ) is expressed by linearizing the above, deldt = T ( x ( t ) ) e , where T is the Jacobian matrix of F. The solution for the above can be obtained as e ( t ) = U(t , e(O)), where U is the linear operator mapping the tangent vector e(0) to e ( t ) . The assymptotic behavior of this map can be characterized by following so called (largest) Lyapunov characteristic exponent,

This exponent is independent of x(0) if the system is ergodic. The excellent algorithm used here for computing the Lyapunov exponents from time series was developed by Gencay and D e ~ h e r t . ~ ~

References

1. W. J. Freeman and C. A. Skarda, “Spatial EEG patterns, non-linear dynamics and perception: The neo-Sherringtonian view,” Brain Res. Rev., 10, 147-175 (1985).

2. B. Baird, “Non-linear dynamics of pattern formation and pattern recognition in the rabbit olfactory bulb,” Physica, 22D, 150-179 (1986).

3. Z. Li and J. J. Hopfield, “Modeling the olfactory bulb and its neural oscillatory processings,” Biol. Cybern., 61, 379-392 (1989).

4. Z. Li, “A model of olfactory perception and sensitivity enhancement in the olfactory bulb., Biol. Cybern., 62, 349-361 (1990).

5. J. J. Hopfield, “Olfactory computation and object preception,” Proc. Natl. Acad. Sci., USA, 88, 6462-6466 (1991).

6. P. Erdi, T. Grobler, G. Barna, and K. Kaski, “Dynamics of the olfactory bulb: Bifurcations, learning, and memory,” Biol. Cybern., 69, 57-66 (1993).

7. R. Granger, J. Ambros-Ingerson, and G. Lynch, “Derivation of encoding characteristics of layer I1 cerebral cortex,” J . Cog. Neurosci, 1, 61-87 (1989).

8. H. Liljenstrom, “Modeling the dynamics of olfactory cortex using simplified network units and realistic architecture,” Znt. 1. NeuraZ Systems, 2 , 1-15 (1991).

152 LILJENSTROM

9. M. A. Wilson and J. M. Bower, “Cortical oscillations and temporal interactions in a computer simulation of pinform cortex,” J . Neurophys., 67, 981-995 (1990).

10. S. L. Bressler, “Relation of olfactory bulb and cortex. 11. Model for driving of cortex by bulb,” Brain Res. , 409, 294-301 (1987).

11. J. Ambros-Ingerson, R. Granger, and G. Lynch, “Simulation of paleocortex per- forms hierarchical clustering,” Science, 247, 1344-1348 (1990).

12. L. B. Haberly and J. M. Bower, “Olfactory cortex: Model circuit for study of associative memory,” Trends in Neurosci., 12, 258-264 (1989).

13. C. A. Skarda and W. J. Freeman, “How brains make chaos in order to make sense of the world,” Brain and Behav. Sci., 10, 161-195 (1987).

14. G. M. Shepherd, The Synaptic Organization ofthe Brain, Oxford Univ. Press, New York, 1979.

15. L. B. Haberly, “Neuronal circuitry in olfactory cortex: Anatomy and functional implications,” Chem. Sens., 10, 219-238 (1985).

16. W. J. Freeman, Mass Action in the Nervous System, Academic Press, New York, 1975.

17. W. J. Freeman, “Spatial properties of an EEG event in the olfactory bulb and cortex,” Elect. Clin. Neurophys., 44, 586-605 (1978).

18. S. L. Bressler and W. J. Frecman, “Frequency analysis of olfactory system EEG in cat, rabbit, and rat,” Electroenceph. Clin. Neurophysiol., 50, 19-24 (1980).

19. D. van Essen, “Functional organization of primate visual cortex,” in Cerebral Cortex, Vof . 3, Visual Cortex, A. Peters and E. G. Jones (Eds.), Plenum Press, New York, 1985.

20. M. A. Biedenbach, “Effects of anesthetics and cholinergic drugs on prepyriform electrical activity in cats,” Exp. Neurol., 16, 464-479 (1966).

21. B. H. Bland, “The physiology and pharmacology of hippocampal formation theta rhythms,” Prog. Neurobiol., 26, 1-54 (1986).

22. C. Rowntree and B. Bland, “An analysis of cholinoceptive neurons in the hippocampal formation by direct microinfusion,” Brain Res., 362, 98-1 13 (1986).

23. J. Hagan and R. Morris, “The cholinergic hypothesis of memory: A review of the animal experiments,” in Psychopharmacology of the Ageing Nervous System, L. L. Iversen, S. Iversen and S. Snyder, (Eds.), Plenum Press, New York, 1989,

24. J. Larson and G. Lynch, “Induction of synaptic potentiation in hippocampus by pattern stimulation involves two events,” Science, 232, 985-988 (1986).

25. W. J. Freeman, “Nonlinear gain mediating cortical stimulus-response relations,” Biol. Cybern., 33, 237-247 (1979).

26. H. Liljenstrom and M. Hasselmo, “Acetylcholine and cortical oscillatory dynamics,” In Computation and Neural Systems, F. Eeckman and J. M. Bower (Eds.), Kluwer, 1993, pp. 523-530.

27. M. E. Hasselmo and J. M. Bower, “Cholinergic suppression specific to intrinsic not afferent fiber synapses in rat piriform (olfactory) cortex,” J . Neurophysiol., 67,

28. M. P. Kennedy, K. R. Kriegh, and L. 0. Chua, “The Devil’s staircase: The electrical engineer’s fractal,” ZEEE Trans. Circuits and Sys. 36, 1133-1139 (1989).

29. D. V. Madison, B. Lancaster, and R. A. Nicoll, “Voltage clamp analysis of cholinergic action in the hippocampus,” J . Neurosci., 7 , 733-741 (1987).

30. M. E. Hasselmo, B. P. Anderson, and J. M. Bower, “Cholinergic modulation of cortical associative memory function,” J . Neurophysiol., 67, 1230-1246 (1992).

31. A. J. Mandell and K. A. Selz, “Some comments on the weaving of contemporane- ous minds,” Proceedings of the Conference on Measuring Chaos in the Human Brain, (D. W. Duke and W. S. Pritchard (Eds.), World Scientific, 1991, pp. 138- 155.

32. A. J. Mandell and K. A. Selz, “Period adding hierarchical protein modes and electroencephalographically defined states of consciousness,” Proceedings of the

pp. 237-324.

1222-1229 (1992).


1st Experimental Chaos Conference, S. Vohra et al. (Eds.), World Scientific, 1992,

33. A. J. Mandell and K. A. Selz, “Brain stem neural noise and neocortical reso-

34. S. Johansson, Electrophysiology of small cultured hippocampal neurons, Doctoral

35. D. 0. Hebb, The Organization ofBehauior, Wiley, New York, 1949. 36. M. E. Hasselmo, “Acetylcholine and learning in a cortical associative memory,”

Neural Computation, 5 , 32-44 (1993). 37. H. Liljenstrom and X. Wu, “The role of noise in cortical associative memory,”

manuscript in preparation. 38. F. Crick and Koch, “Towards a neurobiological theory of consciousness,” Semi-

nars Neurosci., 2 , 263-275. (1990a). 39. R. Eckhorn, R. Bauer, W. Jordan, M. Brosch, W. Kruse, M. Munk, and H. J .

Reitboeck, “Coherent oscillations: A mechanism of feature linking in the visual cortex,” Biol. Cybern., 60, 121-130 (1988).

40. C. M. Gray and W. Singer, “Stimulus-specific neuronal oscillations in orientation columns of cat visual cortex,” Proc. Natl. Acad. Sci. USA, 86, 1698-1702 (1989).

41. C. M. Gray, P. Konig, A. K. Engel, and W. Singer, “Oscillatory responses in cat visual cortex exhibit inter-columnar synchronization which reflects global stimulus properties., Nature, 338, 334-337 (1989).

42. E. FransCn, A. Lansner, and H. Liljenstrom, “A model of cortical associative memory based on Hebbian cell assemblies,” in Computation and Neural Systems, F. Eckman and J. M. Bower (Eds.), Kluwer, 1993, in press.

43. 0. Ekeberg, “Response properties of a population of neurons,” Int. J . Neural Systems, 1993, in press.

44. R. Llinas, A. A. Grace, and Y. Yarom, “Zn uitro neurons in mammalian cortical layer 4 exhibit intrinsic oscillatory activity in the 10 to 50 Hz frequency range,” Proc. Natl. Acad. Sci. USA, 88, 897-901 (1991).

45, R. Traub, R. Miles, and G . Buzsaki, “Computer simulation of carbachol-driven rhythmic population oscillations in the CA3 region of the in uitro rat hippocampus,” J . Physiol., 451, 653-672 (1992).

46. Y. Yao and W. J. Freeman, “Model of biological pattern recognition with spatially chaotic dynamics,” Neural Networks, 3 , 153-170 (1990).

47. P. E. Rapp, “Chaos in the neurosciences: Cautionary tales from the frontier,” Biologist, 40, 89-94 (1993).

48. P. Grassberg and I . Procaccia, “Characterization of strange attractors,” Phys. Rev. Lett., 50, 346-349 (1983).

49. R. Gencay and W. D. Dechert, “An algorithm for the n Liapunov exponents of an n-dimensional unknown dynamical system,” Physica, 59D, 142-157 (1992).

50. T. Kapitaniak, “On strange nonchaotic attractors and their dimensions,” Chaos Solitons and Fractals, 1(1), 67 (1991).

51. J. Brindley, T. Kapitaniak, and M. S. El Naschie, “Analytical conditions for strange chaotic and nonchaotic attractors of the quasiperiodically forced Van der pol equation,” Physica, 51D, 28-38 (1991).

pp 175-194.

nance,” J . Stat. Phys., 70, 355-373 (1993).

Thesis, Stockholm, 1991.

Documents

Autonomous learning with complex dynamics