22
Models of Cognitive Processes: Historical Introduction with a Focus on Parallel Distributed Processing Models Psychology 209 Stanford University Jan 7, 2013

Psychology 209 Stanford University Jan 7, 2013

  • Upload
    mora

  • View
    20

  • Download
    0

Embed Size (px)

DESCRIPTION

Models of Cognitive Processes: Historical Introduction with a Focus on Parallel Distributed Processing Models. Psychology 209 Stanford University Jan 7, 2013. Early History of the Study of Human Mental Processes. Introspectionism (Wundt, Titchener) - PowerPoint PPT Presentation

Citation preview

Page 1: Psychology 209 Stanford University Jan 7, 2013

Models of Cognitive Processes:Historical Introduction with a Focus on Parallel Distributed Processing Models

Psychology 209Stanford University

Jan 7, 2013

Page 2: Psychology 209 Stanford University Jan 7, 2013

Early History of the Study of Human Mental Processes

• Introspectionism (Wundt, Titchener)– Thought as conscious content, but two problems:

• Suggestibility• Gaps

• Freud suggests that mental processes are not all conscious

• Behaviorism (Watson, Skinner) eschews talk of mental processes altogether

Page 3: Psychology 209 Stanford University Jan 7, 2013

Early Computational Models of Human Cognition (1950-1980)

• The computer contributes to the overthrow of behaviorism.

• Computer simulation models emphasize strictly sequential operations, using flow charts.

• Simon announces that computers can ‘think’.

• Symbol processing languages are introduced allowing some success at theorem proving, problem solving, etc.

• Minsky and Pappert kill off Perceptrons.

• Cognitive psychologists distinguish between algorithm and hardware.

• Neisser deems physiology to be only of ‘peripheral interest’

• Psychologists investigate mental processes as sequences of discrete stages.

Page 4: Psychology 209 Stanford University Jan 7, 2013
Page 5: Psychology 209 Stanford University Jan 7, 2013

Ubiquity of the Constraint SatisfactionProblem

• In sentence processing– I saw the grand canyon flying to New York– I saw the sheep grazing in the field

• In comprehension– Margie was sitting on the front steps when she heard the

familiar jingle of the “Good Humor” truck. She remembered her birthday money and ran into the house.

• In reaching, grasping, typing…

Page 6: Psychology 209 Stanford University Jan 7, 2013
Page 7: Psychology 209 Stanford University Jan 7, 2013

Graded and variable nature of neuronal responses

Page 8: Psychology 209 Stanford University Jan 7, 2013

Lateral Inhibition in Eye of Limulus

(Horseshoe Crab)

Page 9: Psychology 209 Stanford University Jan 7, 2013

The Interactive Activation Model

Page 10: Psychology 209 Stanford University Jan 7, 2013

Interactive activation and probabilistic computation

• Rumelhart’s first effort to understand context effects in perception was formulated in explicitly probabilistic models.

• Although he abandoned this formulation in favor of a neural network formulation, we will see that neural network models very similar to the IA model can be understood in explicit probabilistic terms.

• Likewise, neural network models can be related to other sorts of models, including the drift-diffusion model of decision making and exemplar models of categorization and memory.

• One of the goals of the course this year will be to explore these linkages more fully.

Page 11: Psychology 209 Stanford University Jan 7, 2013

Synaptic Transmission and Learning• Learning may occur by

changing the strengths of connections.

• Addition and deletion of synapses, as well as larger changes in dendritic and axonal arbors, also occur in response to experience.

• [Recent evidence suggests that neurons may be added under certain circumstances.]

Pre Post

Page 12: Psychology 209 Stanford University Jan 7, 2013

Connection-based learning creates implicit knowledge

• Connection adjustment affects processing, not necessarily conscious awareness.

• But not all learning is implicit.

• Connection based learning can also be used to reinstate patterns of activation or to ‘auto-associate’ some elements of a pattern with other elements.

• Perhaps we are aware of the patterns, but not of the connections that support their activation.

Page 13: Psychology 209 Stanford University Jan 7, 2013

Cognitive Neuropsychology (1970’s)

• Geshwind’s disconnection syndromes:– Conduction Aphasia

• Patient can understand and produce spoken language but cannot repeat sentences or nonwords

– Alexia without Agraphia• Deep and surface dyslexia (1970’s):

– Deep dyslexics can’t read non-words (e.g. VINT), make semantic errors in reading words (PEACH -> ‘apricot’)

– Surface dyslexics can read non-words, and regular words (e.g. MINT) but often regularize exceptions (PINT).

• Work leads to ‘box-and-arrow’ models, reminiscent of flow-charts

Page 14: Psychology 209 Stanford University Jan 7, 2013
Page 15: Psychology 209 Stanford University Jan 7, 2013

Graceful Degradation in Neuropsychology

• Patient deficits are seldom all or none

– This is true both at the task and at the item level.

– Performance is slower, more errorful, and requires more contextual support.

• And error patterns are far from random:– Visual and semantic errors in

deep dyslexia suggest degradation, rather than loss of a module or disconnection

– Regularization errors depend on a word’s frequency, and how many other exceptions there are that are like it

• Effects of lesions to units and connections in distributed connectionist models nicely capture these features of neuropsychological deficits.

Page 16: Psychology 209 Stanford University Jan 7, 2013

Core Principles of Parallel Distributed Processing

• Processing occurs via interactions among neuron-like processing units via weighted connections.

• A representation is a pattern of activation.

• The knowledge is in the connections.

• Learning occurs through gradual connection adjustment, driven by experience.

• Learning affects both representation and processing.

H I N T

/h/ /i/ /n/ /t/

Page 17: Psychology 209 Stanford University Jan 7, 2013

Implications of this approach• Knowledge that is otherwise represented in explicit form is inherently implicit

in PDP:– Rules– Propositions– Lexical entries…

• None of these things are represented as such in a connectionist/PDP models.

• Knowledge that others have claimed must be innate and pre-specified domain-by-domain often turns out to be learnable within the PDP approach.

• Thus the approach provides an alternative to other ways of looking at many aspects of knowledge-dependent cognition and development.

• While the approach allows for structure (e.g. in the organization and interconnection of processing modules; structured similarity relations among patterns of activation), processing is generally far more distributed, representation is less explicit, and causal attribution becomes more complex.

Page 18: Psychology 209 Stanford University Jan 7, 2013

In short…

• Models that link human cognition to the underlying neural mechanisms of the brain simultaneously provide alternatives to earlier ways of understanding processing, learning, and representation at a cognitive level.

Page 19: Psychology 209 Stanford University Jan 7, 2013

The PDP Approach…

• Attempts to explain human cognition as an emergent consequence of neural processes.– Global outcomes, local processes

• Forms a natural bridge between cognitive science on the one hand and neuroscience on the other.

• Is an ongoing process of exploration.• Depends critically on computational modeling

and mathematical analysis.

Page 20: Psychology 209 Stanford University Jan 7, 2013

Beyond PDP• Since the PDP work began, several new approaches and

communities have arisen– NIPS/Machine Learning Community– Computational Neuroscience Community– Bayesian Approaches in Cognitive Science and Cognitive Neuroscience

• Many of the models we consider belong more to these communities than to what might be called ‘Classic PDP’

• Much of my own work now involves either– Constucting models at the interface between PDP and other approaches– Attempting to understand the relationship between PDP models and models

formulated in other frameworks, including Bayesian approaches.

• A good fraction of the course material will cover work of this type, and links between such work and PDP models.

Page 21: Psychology 209 Stanford University Jan 7, 2013

This course…• Invites you to join the ongoing exploration of

human cognition using PDP models and related approaches to mind, brain, and computation.

• Focuses ultimately on human cognition and the underlying neural mechanisms, rather than abstract computational theory or artificial intelligence.

• Includes exercises that provides an introduction to the modeling process and its mathematical foundations, preparing you to join the ongoing exploration.

Page 22: Psychology 209 Stanford University Jan 7, 2013

Assignment for WednesdayRead:

• * McClelland, J. L. (2013). Bayesian inference, generative models, and probabilistic computations in interactive neural networks. Draft, Jan. 6. 2013, Department of Psychology, Stanford University. Pages 1-28.

and for a primer on real neurons:

• † Kolb, B. and Whishaw, I. Q. (1980). Physiological organization of the nervous system. Chapter 2 of Fundamentals of Human Neuropsychology (pp. 31-42). San Francisco: Freeman.

We will discuss connectionist units and their properties in relation both to Bayesian computations and physiology of real neurons.