Linguistic Neuroscience:
Extending Perceptual Neuroscience to Language
Ling 411 – 12
“Linguistic Neuroscience”?
Applying the findings of perceptual neuroscience to language
Perceptual neuroscience as in Mountcastle’s 1998 book Mountcastle doesn’t say anything about language But his findings can be applied
Findings relating to columns(Mountcastle, Perceptual Neuroscience, 1998)
The column is the fundamental module of perceptual systems • probably also of motor systems
This columnar structure is found in all mammals that have been investigated
The theory is confirmed by detailed studies of visual, auditory, and somatosensory perception in living cat and monkey brains
Adjacency and the Proximity Principle
Neighboring areas for closely related functions• The closer the function the closer the proximity
Consequences• Members of same category will be in same area
Why? • Same category because similar functions
• Competitors will be neighbors in the same area Why?
• Neighbors in same area have same general function along with additional differentiating function
• They compete w.r.t. the differentiating function
Inhibitory connections Based on Mountcastle (1998)
Columnar specificity is maintained by pericolumnar inhibition (190)
• Activity in one column can suppress that in its immediate neighbors (191)
Inhibitory cells can also inhibit other inhibitory cells (193)
Inhibitory cells can connect to axons of other cells (“axoaxonal connections”)
Large basket cells send myelinated projections as far as 1-2 mm horizontally (193)
Extrapolation to Language?
Our knowledge of cortical columns comes mostly from studies of perception in cats, monkeys, and rats
Such studies haven’t been done for language• Cats and monkeys don’t have language• That kind of neurosurgical experiment isn’t done
on human beings Are they relevant to language anyway?
• Relevant if language uses similar cortical structures• Relevant if linguistic functions are like perceptual
functions
Perception and Language
Why haven’t such studies been done for language?1. That kind of neurosurgical experiment isn’t done on
human beings2. Cats and monkeys don’t have language
Are they relevant to language anyway?1. Relevant if language uses similar cortical structures2. Relevant if linguistic functions are like perceptual
functions
Relevance to Language
These studies of perception are relevant if• Perceptual structure and functions are
basically the same across modalities Including associative areas (higher-level)
• Linguistic comprehension is basically a perceptual process
Objection
Cats and monkeys don’t have language Language (as we know it) is a unique human faculty Therefore language must have unique properties of its
structural representation in the cortex Answer: Yes, language is different, but
• The differences are a consequence not of different (local) structure but differences of connectivity
• The neurocognitive network does not have different kinds of structure for different kinds of information Rather, different connectivities
Justifying extrapolation
Hypothesis: Extrapolation of findings about cortical columns can be extended to • humans• linguistic and conceptual structures
Why? • Summary of the argument
Cortical structure, viewed locally, is • uniform across mammalian species • uniform across different cortical regions• Exceptions in primary visual and primary auditory areas
Different cortical regions have different functions • because of differences in connectivity• not because of differences in structure
Essence of the argument
Cortical structure and function, locally, are essentially the same in humans as in cats and monkeys and rats
Moreover, in humans,• The regions that support language have the same
structure locally as other cortical regions
Uniformity of cortical function
Claim:• Locally, all cortical processing is the same• The apparent differences of function are
consequences of differences in larger-scale connectivity
Conclusion (if the claim is supported):• Understanding language, even at higher
levels, is basically a perceptual process
Argument for local uniformity of representation
Different types of cortical information• Perceptual• Conceptual• Grammatical• Phonological
How are they different? Two possibilities
1. They differ in their structural form2. They differ based on their connections
Claim: Possibility #2 is the correct one • The “connectionist claim”
Support for the connectionist claim
Lines and nodes (i.e., columns) are approximately the same all over
Uniformity of cortical structure• Same kinds of columnar structure• Same kinds of neurons• Same kinds of connections
Conclusion: Different areas have different functions because of what they are connected to
Linguistic Information in the Cortex
Problem: Linguistic information is usually described symbolically
In the symbolic mode of description, different kinds of linguistic information appear to have different kinds of structure• Phonology• Morphology
Regular and irregular inflections• Syntax• Semantics
Claim: If the information is viewed as connectional instead of symbolic, it turns out to have a high degree of uniformity
Uniformity of cortical structure
Six layers throughout, with similar structure Columns throughout Same neuron types everywhere – pyramidal
most frequent, spiny stellate in layer IV, etc. Inhibitory and excitatory connections throughout Same neurotransmitters everywhere
• Excitatory: glutamate• Inhibitory: GABA
But: What about the different Brodmann areas?1. The differences are relatively minor2. They may be based on experience
Structural Uniformity?A closer look
Differences are found at lower levels• Primary sensory areas have specialized structures • These are called heterotypical areas• Properties of columns depend on afferent inflow
More uniformity in higher-level areas • “Homotypical” areas (i.e., same type)• Relatively uniform structure• Makes them flexible, adaptable• Properties depend on intracortical processing• Different homotypical areas differ not because of
their structures but because of their connections
A heterotypical area: Visual motion perception
An area in the posterior bank of the superior temporal sulcus of a macaque monkey (“V-5”)
Albright et al. 1984
400-500 μ
Auditory areas in a cat’s cortex(Heterotypical)
AAF – Anterior auditory fieldA1 – Primary auditory field PAF – Posterior auditory fieldVPAF – Ventral posterior auditory field
A1
Exceptions: Diversity in cortical function
Perception vs. production• Back brain vs. front brain
Sharpness of contrast • Phonology and morphology require sharp contrasts• Conceptual categories have fuzzy definitions
Left vs. right hemisphere • RH may have..
Larger minicolumns Less lateral inhibition
Uniformity in LH Associative Areas
The associative areas are homotypical The structure that subserves language
understanding is the same as perceptual structure• Columns of similar structure• With similar kinds of connections
Claim: Understanding language is the same process as perception• To support this claim, must look more closely
at cortical function• Subclaim: Locally, all columns function alike
Primary areas and higher-level areas (LH)
These are homotypical
The uniformity of information?
Different types of cortical information• Perceptual• Conceptual• Grammatical• Phonological
How are they different? Two possibilities
1. They differ in their form of representation2. They differ based on their connections
Claim: Possibility #2 is the correct one • The “connectivity claim”
Uniformity of cortical function
If cortical function is uniform across mammals and across different cortical areas, then the findings presented by Mountcastle can be extended to language
Claims:• Locally, all cortical processing is the same• The apparent differences of function are
consequences of differences in larger-scale connectivity
Conclusion (if the claim is supported):• Understanding language, even at higher levels, is
basically a perceptual process
Testing the claim
Claim:• The apparent differences of function are consequences
of differences in larger-scale connectivity To test, we need to understand cortical function That means we have to better understand the function of
the cortical column
Columns do not store symbols!
They only• Receive activation• Maintain activation• Inhibit competitors• Transmit activation
Important consequence:• We have linguistic information represented
in the cortex without the use of symbols• It’s all in the connectivity
The Challenge:• How?• This claim goes against most of the history
of linguistics
Operation of the Network
The linguistic system operates as distributed processing of multiple individual components – cortical columns
Columnar Functions • Integration: A column is activated if it receives enough
activation from other columns Can be activated to varying degrees Can keep activation alive for a period of time
• Broadcasting: An activated column transmits activation to other columns
Exitatory – contribution to higher level Inhibitory – dampens competition at same level
Columns do not store symbols!
Why the usual approach won’t work
Let us suppose that words are stored in some kind of symbolic form
What form? If written, there has to be..
• something in there that can read them• something in there that can write them• something in there that can move them around, from
one place to another• something in there to compare them with forms
entering the brain as it hears someone speaking – otherwise, how can an incoming word be recognized?
Why the usual approach won’t work (cont’d)
If not written, then represented in some other medium Doesn’t solve the problem You still need whatever kind of sensory detectors can
sense the symbols in whatever medium you choose Plus means of performing all those other operations
Compare imagery
Visual images• Little pictures?• If so, what is in there to see them?
Auditory images• Little sounds vibrating in the brain?• If so, what is in there to hear them?
There has to be another way!
There must be another way
Visual imagery (e.g. of your grandmother)• Reactivation of some of the same nodes and
connections that operate when actually seeing her Auditory imagery (e.g. of a tune)
• Reactivation of some of the same nodes and connections that operate in actually hearing it
Another way, for language
A syllable• Activation of the nodes and connections
needed to recognize or produce it A word
• Activation of the nodes and connections needed to recognize it
A syntactic construction• Activation of the nodes and connections
needed to recognize or produce it
The postulation of objects as something different from the terms of relationships is a superfluous axiom and consequently a metaphysical hypothesis from which linguistic science will have to be freed.
Louis Hjelmslev Prolegomena to a Theory of Language
(1943: 61)
Hjelmslev’s view
Columnar Functions: Integration and Broadcasting
Integration: A column is activated if it receives enough activation from • Other columns • Thalamus
Can be activated to varying degrees Can keep activation alive for a period of time Broadcasting: An activated column transmits
activation to other columns• Exitatory• Inhibitory
Learning: adjustment of connection strengths and thresholds
Integration and Broadcasting
Broadcasting• To multiple locations• In parallel
Integration
Integration and Broadcasting
Integration
Broadcasting
Wow, I got activated!
Now I’ll tell my friends!
What matters is not ‘what’ but ‘where’
What distinguishes one kind of information from another is what it is connected to
Lines and nodes are approximately the same all over Hence, uniformity of cortical structure
• Same kinds of columnar structure• Same kinds of neurons• Same kinds of connections
Different areas have different functions because of what they are connected to
Operations in neurocognitive networks
Activation moves along lines and through nodes• Integration • Broadcasting
Connection strengths are variable• A connection becomes stronger with repeated
successful use• A stronger connection can carry greater activation
Can language be accounted for by such simple operations?
Phonology Words and their meanings Syntax and morphology Conceptual relationships
Sequence
In language, sequence is very important• Word order• Order of phonological elements in syllables• Etc.
Also important in many non-linguistic areas• Dancing• Eating a meal
Can cortical columns handle sequences?
Lasting activation in minicolumn
Subcorticallocations
Connections to neighboring columns not shown
Cell Types
Pyramidal
Spiny Stellate
Inhibitory
Recurrent axon branches keep activation alive in the column –Until is is turned off by inhibitory cell
Notation for lasting activation
> Thick border for a node that stays active for a relatively long time > Thin border for a node
that stays active for a relatively short time
Recognizing items in sequence
This link stays active
a b
Node c is satisfied by activation from both a and b If satisfied it sends activation to output connections Node a keeps itself active for a whileSuppose that node b is activated after node a Then c will recognize the sequence ab
c
This node recognizes the sequence ab
Recognizing stop consonants
Consider stop consonants, e.g. t, d At the time of closure
• For voiceless stops there is no sound to hear• For voiced stops, very little sound
The stops are identified by transitions • To following vowel• From preceding vowel
Demisyllables [di, de, da, du]
F1 and F2For [a]
It is unlikely that [d] is represented as a unit in perception
Recognizing a syllable and its demisyllables
dim
di- -im
Cardinal node for dim
Functional subweb for dim
Auditory features of [di-]Auditory features of [-im]
Just labels
Another syllable and its demisyllables
bil
bi- -il
Cardinal node for bill
Subweb for bill
Multiple connections of -il
bil hil kil
bi- -il
Bill hill mill kill etc.
One and the same /-il/ in all of them
Multiple connections of -il
bil hil kil
bi- -il
Bill hill mill kill etc.
Similarly for multiple connections of bi- bit, bib, bid, etc.
Multiple connections of -il
bil hil kil
bi- -il
Bill hill mill kill etc.
To lower level nodes, for phonological features
Syntactic Recognition – same principle
This link stays active
a b
Let node a represent Noun Phrases (Subject) and let b represent Predicates (Verb Phrases etc.)Then c represents Clauses: the sequence ab
c
This node recognizes the sequence ab
Syntactic Recognition: higher-level perception
This link stays active
a b
The whole process is one of recognition, just as at lower levels (e.g., phonological recognition)Same structures, different connections
c
This node recognizes the sequence ab
Perhaps All of linguistic structure is relational?
Remains to be shown for• Syntax (beyond the essence: recognizing sequence)• Regular and irregular inflection• Lexical structure
If it can be shown, then: The whole of linguistic structure is a connectionist system
• No symbols – it’s all relationships Good thing, since that is exactly the kind of system that
the cortex is built to represent and to operate with
Deductions from findings about cortical columns
If linguistic structure is purely relational, as seems likely, then the properties of cortical structure identified earlier also apply to language:
Property I: Intra-column uniformity of function Property II: Cortical topography Property III: Nodal specificity Property IV: Adjacency Property V: Extension of II-IV to larger columns Property VI: Competition
Deductions from findings about cortical columns
If linguistic structure is purely relational, as seems likely, then the properties of cortical structure identified earlier also apply to language:
Property I: Intra-column uniformity of function• Therefore, the column behaves as a single unit
A node of the linguistic network Property II: Cortical topography Property III: Nodal specificity Property IV: Adjacency Property V: Extension of II-IV to larger columns Property VI: Competition
Deductions from findings about cortical columns
If linguistic structure is purely relational, as seems likely, then the properties of cortical structure identified earlier also apply to language:
Property I: Intra-column uniformity of function Property II: Cortical topography
• Linguistic structure as a two-dimensional array of nodes
Property III: Nodal specificity Property IV: Adjacency Property V: Extension of II-IV to larger columns Property VI: Competition
Deductions from findings about cortical columns
If linguistic structure is purely relational, as seems likely, then the properties of cortical structure identified earlier also apply to language:
Property I: Intra-column uniformity of function Property II: Cortical topography Property III: Nodal specificity
• Every linguistic node has a specific function Property IV: Adjacency Property V: Extension of II-IV to larger columns Property VI: Competition
Deductions from findings about cortical columns
If linguistic structure is purely relational, as seems likely, then the properties of cortical structure identified earlier also apply to language:
Property I: Intra-column uniformity of function Property II: Cortical topography Property III: Nodal specificity Property IV: Adjacency
• Adjacent linguistic nodes have similar linguistic functions For example, nodes for stop consonants
Property V: Extension of II-IV to larger columns Property VI: Competition
Deductions from findings about cortical columns
If linguistic structure is purely relational, as seems likely, then the properties of cortical structure identified earlier also apply to language:
Property I: Intra-column uniformity of function Property II: Cortical topography Property III: Nodal specificity Property IV: Adjacency Property V: Extension of II-IV to larger columns
• Linguisic categories in neighboring cortical areas
Property VI: Competition
Deductions from findings about cortical columns
If linguistic structure is purely relational, as seems likely, then the properties of cortical structure identified earlier also apply to language:
Property I: Intra-column uniformity of function Property II: Cortical topography Property III: Nodal specificity Property IV: Adjacency Property V: Extension of II-IV to larger columns Property VI: Competition
• Contiguous linguistic nodes are in competition E.g. , stop consonants
Operation of the Network
The linguistic system operates as distributed processing of multiple individual components – cortical columns
Columnar Functions • Integration: A column is activated if it receives enough
activation from other columns Can be activated to varying degrees Can keep activation alive for a period of time
• Broadcasting: An activated column transmits activation to other columns
Exitatory – contribution to higher level Inhibitory – dampens competition at same level
Columns do not store symbols!
Review
end