EE141 1 Memory and Learning Janusz A. Starzyk

EE1411

Memory and LearningMemory and Learning

Janusz A. Starzyk

http://grey.colorado.edu/CompCogNeuro/index.php/CECN_CU_Boulder_OReillyhttp://grey.colorado.edu/CompCogNeuro/index.php/Main_Page

Based on book Cognition, Brain and Consciousness ed. Bernard J. Baars

courses taught by Prof. Randall O'Reilly, University of Colorado, andProf. Włodzisław Duch, Uniwersytet Mikołaja Kopernikaand http://wikipedia.org/

Cognitive ArchitecturesCognitive Architectures

EE1412

Human memory has surprising limitations and impressive capacities.

Brain evolved around tasks of survival, thus it is well prepared to deal with ill-defined problems and challenges in real world.

Its ability to remember academic information is quite recent and not as well developed in terms of storage capacities.

Humans are exceptionally flexible in learning new skills. It is amazing that practically the same brain was serving humans to live in

the stone age and was able to learn the skills needed in the age of computers and Internet.

IntroductionIntroduction Learning is the

process by which we acquire knowledge about the world.

Learning involves memory to store representations that reflect experience, behavior and values.

EE1413

Memory is the process by which that knowledge of the world is encoded, stored, and later retrieved. (Kandel 2000)

Memory storage involve synaptic changes in cortex. Correlated activities

between neurons leads to strengthening connections between them.

Temporary cell activities maintain immediate memories.

Medial temporal lobes (MTL) are important for building memories.

MemoryMemory

EE1414

General remarksGeneral remarksMemory is any persistent effect of experience.

Memory is seemingly uniform, but in reality it is very differentiated:

spatial, visual, aural, recognition, declarative, semantic, procedural,

explicit, implicit …

Here we test mechanisms, so the primary division is: Synaptic memory (physical changes in synapses), long-term and

requiring activation to have some influence on functioning. Dynamic memory, active, temporary activations, affects current

functioning. Long-term priming, based on synaptic memory, yielding to fast

modification – semantic and procedural memory are the result of

slow processes. Short-term priming, based on active memory.

EE1415

General remarksGeneral remarksMemory Types

Working memoryShort term memory

Long term memory

Declarative Nondeclarative

Facts Events Manual skills

Conditioning Priming

Emotional Motor

NeocortexCerebellumNuclei

Parietal cortexPrefrontal cortexLimbic system

STM LTM

EE1416

MTL (perirhinal cortex) include two hippocampi and olfactory area.

MTL interacts with the higher level visual area: inferior temporal lobe (IT)

Close to MTL is auditory cortex and amygdala responsible for emotions

MemoryMemory

EE1417

Thus MTL (perirhinal cortex) integrates multiple brain inputs.

It is a “hub of hubs”.

Hippocampus combines cognitive information from neocortex with emotional information from limbic areas and bids this information into memory that codes consciously experienced events.

MemoryMemory

EE1418

MTL helps to store and retrieve episodic memories. When visual cortex is activated by an image of the coffee cup it

activates memory traces through MTL. These include semantic associations of the coffee cup such as coffee

beans or the coffee aroma. Visual features like cup handle are also activated. This may activate episodic memory of yesterday’s coffee with a friend

in cafeteria and traces of conversation.

MemoryMemory

EE1419

Sensory input goes to working memory (WM). Working memory temporarily retains small amounts of

information; only 4-7 items can be held in immediate WM. WM interacts with cognitive processes to perform explicit learning

and retrieval as well as implicit learning. Explicit learning involves semantic memory (facts), episodic

memory (episodes) and perceptual memory (learning music, art). Implicit memory (fear, habits, biases, goals)

MemoryMemory

EE14110

Explicit memory is first acquired through association areas of the cerebral cortex, namely prefrontal, limbic and parieto-occipital-temporal.

Then, the information is transferred to parahippocampal cortex, entorhinal cortex dentate gyrus, hippocampus, subiculum and back to entorhinal cortex.

MemoryMemory

Damage to parahippocampal and entorhinal cortices produces greater deficits in memory storage for object recognition than does hippocampal damage.

Right hippocampal damage produces greater deficits in memory for spatial representation, whereas left hippocampal damage produces greater deficits in memory for words, objects or people.

In either case, the deficits are in formation of new, long-term memory; old memories are spared.

www.unmc.edu/physiology/Mann/mann19.html

EE14111

MemoryMemory

Current thought is that the hippocampal system does the initial steps in long-term memory storage–different parts being more important for different kinds of memory.

The results of hippocampal machinations–presumably memories–are transferred to the association cortex for storage.

The relative positions of parts of the limbic system involved in learning and memory. (Kandel, 2000

Principles of Neural Science. )

www.unmc.edu/physiology/Mann/mann19.html

EE14112

Perceptual memory refers to sensorimotor habits (skills) largely unconscious involving basal ganglia. Imagine riding a bike and you start falling to the right – WHAT TO DO? Conscious answer is to lean to the left (many cyclists say this) However when they ride a bike they instead turn their handlebar in the

direction of the fall, expressing unconscious procedural knowledge.

MemoryMemory Implicit memory

contains procedural, emotional and motor skills.

Implicit memory is often tested using priming where subjects receive subconscious perceptual or conceptual information.

EE14113

AmnesiaAmnesia Clive Wearing suffered a viral infection

that destroyed hippocampi and some frontal lobe areas.

He retained his skills as musician, but he did not remember the most recent past.

Some of his short term memory was preserved so he could converse, and be aware of the present.

However he could not remember events from the recent past. For instance he would talk to his wife and

few minutes later he forgot she was there.

He couldn't register episodic or semantic memory.

Ne couldn't recall episodic memory.

EE14114

AmnesiaAmnesia

The most important patient in cognitive neuroscience is known as HM. His medial temporal lobes were surgically removed by a surgeon who

was unaware about their importance for memories. HM cannot not remember any events in his life after the surgery. He even cannot recognize his face due to changes over the years. He also suffers from retrograde amnesia and does not remember events

from years immediately before surgery. His other cognitive functions are intact: he can reason, solve problems

and carry normal conversation.

EE14115

Impaired long term but not working memory– Amnesic people can perform normally on standard tests of intelligence– They can play chess, solve crossword puzzles, comprehend instructions, and

reason logically Impaired recent but not remote memories (anterograde amnesia). Impaired explicit but not implicit memory

– Learning, retention, and retrieval of memory without awareness is normal.

AmnesiaAmnesia HM represents amnesia in its pure form. In general amnesia is any loss of episodic

memory with otherwise normal cognitive functions.

The causes include infections, stroke, tumor, drugs, oxygen depravation, epilepsy, degenerative disease (like Alzheimer) or be of psychogenic nature.

Amnesia results from damage to MTL including hippocampi and causes: Impaired memory but preserved perception,

cognition, intelligence, and action.

EE14116

Procedural memory depends on perceptual-motor regions like basal ganglia. HM patient was able to learn and retain some motor tasks even he did not remember

learning them. Patients with impaired basal ganglia due to Parkinson’s or Huntington’s disease show not

improvement after practicing sensorimotor tasks. In serial time reaction (STR) tasks subjects are requested to retrace a series of dots

on a computer screen Amnesic patients do well on implicit STR task but poorly on explicit tasks. Patients with basal ganglia disorders like Parkinson’s disease do poorly on both tasks

AmnesiaAmnesia Implicit and procedural memories are not

damaged in amnesia. Perceptual priming involve sensory cortex Conceptual priming include word

association. Patients with amnesia perform well on

perceptual and conceptual priming tasks Patients with Alzheimer disease perform well

on perceptual but not on conceptual priming tasks

EE14117

How memories are made?How memories are made? Traditional thinking of memory as a permanent

record of past events that can be played back, examined and retrieved is false.

Memories of past events are in fact rarely accurate. Two people experiencing the same event may have

different memories of it. The process view, considers memory as a result of a

dynamic process, a reconstruction of the past influenced by present, anticipation of future events and other cognitive processes.

We forgot most of what happened within minutes or hours and what remains is distorted by our knowledge and biases. Try to reconstruct what you did two weeks ago with as

much detail and exact order as you can. Most of us will try to search for cues to figure out the

sequence of events.– Did I go shopping and which stores I visited? – What merchandise did I look at?

Memories Are Made Of This

EE14118

Stimulation of temporal lobe sometimes results in flood of conscious memories. One patient during brain stimulation experienced memory of: At four electrodes location 1-2 and 9-10 re-experiencing Flinstone cartoons

from childhood At locations 8-9 and 13-14 hearing the rock band Pink Floyd. At locations 9-10 a baseball announcer. At locations 7-8 and 12-13 a female voice singing.

How memories are made?How memories are made? You may confuse what happened two weeks ago

with what happened some other time. Patients with disorder called confabulation make up

false memories without intention of lying or awareness that they are not true.

Memories influence how other memories are formed and retrieved. They influence our thoughts and actions, and are

influenced by them.

EE14119

With repeated activation, the stimulus leads to a decrease in the number of dopamine-containing vesicles that release their contents

onto the motor neuron.

How memories are made?How memories are made? What happens in the nervous system to

produce habituation? If the siphon of the animal (Aplysia

californica ) is stimulated mechanically the animal withdraws the gill, presumably for protection.

That action is known to occur because the stimulus activates receptors in the siphon, which activates, directly or indirectly through an interneuron, the motoneuron that withdraws the gill.

This is a simple reflex circuit.

From www.unmc.edu/physiology/Mann/mann19.html

EE14120

How memories are made?How memories are made?

Autobiographical memories evoked by temporal lobe stimulation

EE14121


Possible explanation for this electrically stimulated recall of memories involves temporal lobe in neocortex.

If some neurons are activated in neocortex, this evokes an overlapping pattern of neural activation in hippocampal system (MTL).

The flow of information form neocortex to MTL causes hippocampal system to resonate with the original memory traces, to produce the original episodic experience in neocortex.

EE14122

How memories are made?How memories are made? Most synapses in cortex

are excitatory using neurotransmitter glutamate.

A large minority are inhibitory using neurotransmitters like GABA (gamma amino butyric acid).

These two processes are called long term potentiation (LTP) and long term depression (LTD).

LTP has been observed in hippocampus using single cell recording.

A schematic of a single cell recording in hippocampus

EE14123

HippocampusHippocampusAnatomy and connections of the

structures of the hippocampal

formation: signals reach from uni-

and multimodal association areas

through the Entorminal Cortex

(EC).

EE14124

More anatomyMore anatomy

Hippocampus = king of the

cortex

Bidirectional connections with the

entorhinal cortex:

olfactory bulb,

cingulate cortex,

superior temporal gyrus (STG),

insula,

orbitofrontal cortex.

EE14125

More anatomyMore anatomy

Sporadic activation

Representations in CA3 and CA1

are focused on specific

stimuli, while in the

subiculum and the entorhinal

cortex they are strongly

distributed.

EE14126

Hippocampal formationHippocampal formationModel contains structures:

dentate gyrus (DG),

areas CA1 and CA3,

entorhinal cortex (EC).

Pct Act = % of activation.

EE14127

How memories are made?How memories are made? Many millions of neurons and billions of synapses are

involved in LTP or LTD. Based on evidence from EEG, ERP, and fMRI we can

suppose that formation of long term memories involves: Episodic input is presented via neocortex. It is integrated for memory purpose in the MTL (medial

temporal lobes) involving hippocampi and related structures and perhaps thalamus and surrounding regions.

Consolidation: MTL and related regions bind and integrate a number of neocortical regions in the process that transforms temporary synaptic connections into longer lasting memory traces in both MTL and neocortex.

The main mechanism used is LTP and LTD. Normal sleep is important to form long-lasting memory

traces. More permanent memories require protein synthesis –

such as growth of dendritic spikes on the top of axons and dendrites.

EE14128


The steps of learning, binding, consolidation and remembering. When a new event is learned cortex activates MTL Cortex and MTL resonate to establish the memory traces in a binding step In consolidation the resonance continues without external support Upon presentation of the original event's cue, MTL and cortex resonate to

recall the stored memories.

EE14129

How memories are made?How memories are made? Reconsolidation turns active

neuronal connections into lasting ones.

We have two kinds of reconsolidations: cellular and system reconsolidation.

EE14130


Rapid consolidation occurs within minutes to hours from learning event. It correlates with morphological changes in synapses.

If the stimulus is intense or repeated then gene transcription and protein formation lead to long lasting changes including creation of new synapses to form long lasting memory.

EE14131

How memories are made?How memories are made? Nadel and Moskovitch concluded

that contrary to the standard consolidation model, MTL is needed to represent even old episodic memories for as long as these memories exist.

MTL neurons act as pointers to neocortical neurons that represent the experience.

Neocortex, on the other hand, is sufficient to represent repeated experiences with words, objects, people and environment.

MTL may help in initial formation of these neocortical traces, but once formed they can exist on their own.

EE14132

Varieties of MemoriesVarieties of Memories Declarative memory

can be divided into episodic and semantic memory.

Episodic memory have specific source in time, space and events. It allows us to go

back in time and relieve the experience.

Semantic memory involve facts about the world, ourselves and other knowledge. We know which city

is a capital of France or where are the great pyramids.

EE14133

Varieties of MemoriesVarieties of Memories

Episodic memories:

1. Have reference to oneself

2. Are organized around specific time period

3. Are remembered consciously such that we can relive them

4. Are susceptible to forgetting

5. Are context dependent w.r.t. time, place, relationships etc.

EE14134


Semantic memories:

1. Have reference to shared knowledge

2. Are not organized around specific time period

3. Give a feeling of knowing rather than recollection of a specific event.

4. Are less susceptible to forgetting than events.

5. Are relatively context independent.

EE14135

Varieties of MemoriesVarieties of Memories In a study subjects were

asked to tell if they remember the item or “know” the item.

The act of remembering (episodes) resulted in higher brain activation than the “feeling of knowing” (semantic)

EE14136

Varieties of MemoriesVarieties of Memories Episodic memories may

turn into semantic memories over time

Initially memories are episodic and context dependent

Over time, episodic memories are transformed into semantic memories

MTL is important for recovering episodic memories, which are linked by specific autobiographical context

Episodic memories in Fig. show a man cooking on a barbecue, giving flowers to a lady, painting a picture and playing golf.

A semantic network above combines all these specific episodes into a simplified knowledge of a man who BBQs, loves, paints, and plays golf.

EE14137

Varieties of MemoriesVarieties of Memories Learning is often thought

to require consciousness and paying attention.

It certainly helps to learn by being aware of it

It is a basic learning strategy for humans.

However there are some evidence for learning without consciousness especially with emotional stimuli.

The terms explicit and implicit memories are used in context of remembering.

Explicit (declarative) memory requires conscious awareness

Fig. from: http://universe-review.ca/R10-16-ANS.htm

EE14138

Varieties of MemoriesVarieties of Memories Prefrontal cortex (PFC) is critical for

working memory Lesions of PFC impaired performance

in delayed response tasks. Fuster (1971) experimented with

monkeys – they were trained to remember a color for a short period of time and then point to a correct color when presented with two choices.

Through implanted electrodes he observed sustained neurons activities over the delay period in the area of dorsolateral (DL) PFC

EE14139

Varieties of MemoriesVarieties of Memories Prefrontal cortex (PFC) serves to support the mental work

performed on stored information rather than as a site of storage itself.

Its primary function is to modulate activity in other cortical areas where the items in memory are stored.

PFC enhances the relevant information in the memory and inhibits irrelevant information.

When the information is relevant to a specific item in the memory, then ventral part of PFC is involved

When the information regards the relations between many items, then dorsal part of PFC is involved.

Anterior (frontal) regions of PFC are involved with coordination and monitoring activities among different PFC regions to implement higher order functions such as planning.

EE14140

Varieties of MemoriesVarieties of Memories Combined brain regions work

together for visual working memory.

Hippocampus may encode new memories, while MTL may combine them with pother modalities and IT is involved in high level visual object representation.

DL-PFC and anterior PFC is involved in short term maintenance of relations

EE14141


For effective retrieval the retrieved information must overlap with learned and encoded one – the person must have a goal to retrieve it.

MTL is mostly involved in retrieving episodic memory. Poor frontal function impairs tests on the source of memory and

temporal order. Semantic memory both learning and retrieval depend more on the left

hemisphere functions.

Clive Wearing knows that something is wrong as he always lives in present time.He has a metacognitive concept of his own cognitive functions.A person may recall an episode using semantic cue and vice versa.

EE14142

Varieties of MemoriesVarieties of Memories Other kinds of memory may involve other brain structures. The amygdala mediates fear conditioning. The cerebellum and basal ganglia are needed for habits and skills,

and subconscious conditioning. The thalamus is information hub constantly trading signals with

cortex. Perceptual and motor learning involve the dynamic organization of

cortical maps. Brain surgery can alter body maps – this is related to brain

plasticity. Life is a development process of learning, adaptation and memory

formation. New neurons are being born throughout the lifetime starting from

stem cells. The ongoing placement of the neurons involves dynamic learning

and adaptation process.

EE14143


Overview of multiple learning systems in the brain

EE14144

MemoryMemoryMemory is not uniform

1. Weights (long-term, require activation) vs activations (short-term, already activated, can influence processing)

2. Based on weights The cortex has initial states but suffers from catastrophic influences. The hippocampus can learn fast without influences, using sparse

distributed representations of images

3. Based on activation The cortex shows initial states but isn't good for short-term memory

4. Cooperation of activation and memory based on weights

5. Video 1. short-term memory in chimpanzees -30 sec2. Comparison with students– 30 sec

EE14145

Active short-term memoryActive short-term memoryShort-term priming: attention and influence on reaction speed.Besides the duration, memory content and effects resulting from similarity are like long-term priming.

Project act_priming.proj. (http://grey.colorado.edu/CompCogNeuro/index.php/CECN1_Act_Priming)

Completing roots or homophony, but without learning, only the influence of the remains after the last activation.The network has learned series IA-IB.The test has a series of images and results A and B, we show it A upon output, the network responds A; now we show the image for B but only phase is turned on – (lack of learning), the network's result is sometimes A, sometimes B.

LoadNet, View TestLogs,TestThe correlations of previous results A and B depend on the speed of fading of activation; check efekt act_decay 1 => 0, tendency to leaving a.Analyze the influence on results in test_log.

EE14146

Role of dopamineRole of dopamineBlocking of dopamine has a negative influence on working memory, and aiding it has a positive influence.

Dopamine (DA) arrives from the VTA (ventral tegmental area).DA strengthens internal activations, regulating access to working memory. VTA displays such increased activity.

Basal ganglia can also regulate PFC activity.

TD – temporal Difference in RL

EE14147

Basal Ganglia Basal Ganglia Pathways: thalamus- basal ganglia - cortex.

Red lines – inhibition, mostly GABA.

Blue lines: excitation, mostly glutamine.

Black lines: dopamine, mostly inhibition.

Malfunctions in these pathways lead to Parkinson, Huntington and other diseases.

GP – Globus Pallidus

Putamen; Substantia Nigra

Subthalamic nucleus

EE14148

A- not BA- not BInteractions between active and synaptic memory - weights have already changed but active memory is in a different state: what wins? These interactions are visible in the developing brains of children ~ 8 months (Piaget 1954), experiments done also on animals.

A toy (food) is hidden in box A and after a short delay the child (animal) can remove it from there. After several repetitions in A, the toy is hidden in box B; the children keep looking in A.

Active memory doesn't work in children as efficiently as synaptic memory, lesions in the area of the prefrontal cortex cause similar effects in adult and infant rhesus monkeys. Children make fewer errors looking in the direction of the place where the toy was hidden, than reaching for it. There are many interesting variants of this type of experiment and explanations on different levels.

EE14149

Other types of memoryOther types of memoryThe traditional approach to memory assumes functional, cognitive, monolithic, canonical representations in memory.

From modeling, it turns out that there are many systems interacting with each other which are responsible for memory, with different characteristics, variable representations and types of information.

Recognition memory: was an element of the list seen earlier?

A "recognition" signal is enough, remembering is not necessary.

A hippocampus model is also useful here, it allows for remembering, but this is too much – in recognition memory the central role seems to be played by the area of the perirhinal cortex.

Cued recall - completion of missing information.

Free recall – effects of placement on the list (best at the beginning and the end), as well as grouping (chunking) of information.

EE14150

SummarySummarySummarySummary Knowledge formed in memory is

built, dynamic, continuous, appearing Behavior and inhibition of knowledge are the result of

dynamic information processing rather than interaction structures set at the top.

Recognition is based on the ability to differentiate earlier-learned activations from new, unknown activations.

The hippocampus ensures high-quality recognition with a high threshold guaranteeing association of earlier-learned activations.

Priming contributes to slow building of inviariant representations

Two learning mechanisms Based on connection weights Based on neuron activation

EE14151

SummarySummarySummarySummary The cortex helps recognition by priming The cortex leads to unstimulated associations The cortex is responsible for working memory

cooperating with the hippocampus Sequences of grouped representations are stored in

long-term memory Memory based on activation requires combining

quick-actualizing with stable representations The hippocampus uses sparse distributed

representations for fast learning without mixing ideas Priming memory can be long-term (based on weights)

or short-term (based on activation)

Documents

EE141 1 Memory and Learning Janusz A. Starzyk