Introduction, Goals, & Questions

Neural Bases of Language (UG)

Neurolinguistics (G)

Course website:http://www.psych.nyu.edu/pylkkanen/NeurolinguisticsFall2016

Instructors:

Prof. Liina Pylkkänen TA: Esti Blanco-Elorrieta

[email protected] [email protected]

Office hours by appointment Office hours: Mon 9-10am

!  Cognition: human intelligence in all its forms"  Forms of intelligence:"  Reasoning Planning Decision making Memory "  Problem solving Abstract thinking Language Learning

…!  History:

"  Term “cognitive science” coined in the early 1970’s. "  Cognitive Science Society founded in 1979.

!  Mission statement was to be highly interdisciplinary, combining methods from"  Psychology"  Linguistics"  Philosophy"  Computer science"  Neuroscience (--> Cognitive Neuroscience)

!  Study of human cognitive abilities in the context neighboring cognitive abilities.

Cognitive science

!  The study of the neural bases of cognition!  Cognitive Neuroscience Society founded in 1994!  Basically: a (blurry) line is typically drawn between very low

level sensory neuroscience and cognitive neuroscience. " E.g., the level of representation at which you can perform

some type of categorization on a stimulus (like figuring out that a noise is a dog barking) counts as cognitive neuroscience whereas the early pre-thalamic processing of the auditory stimulus would generally count as sensory neuroscience.

Cognitive neuroscience

!  But: A major finding of cognitive neuroscience is that the brain is a massively predictive top-down modulator of “low-level” processing; i.e., higher level processes systematically affect lower level computations due to expectations based on prior knowledge/experience.

Top-down processing

!  Example: "  Complete the following sentence

#  Yesterday, I saw Max’s _______ .#  ‘Max’s’ sets up an expectation for a noun.

"  Complete the following sentence#  When the lights turned off, the room became ______ .#  This context sets up an expectation for a specific adjective, dark.

"  In both cases, activity in the visual cortex is reduced at ~100ms if the predicted category/item is in fact encountered (as compared to a situation in which an unpredicted category/item is encountered).

"  This is very early, given that the signal does not arrive in visual cortex much before 100ms.

!  Lesson for cognitive neuroscience: There is no a priori definition of what counts as “cognitive” for the brain – high level “cognitive” factors modulate lower level perception in many ways and thus we cannot a priori exclude sensory brain regions or early time intervals as irrelevant for cognition.

Top-down processing

!  Conscious = aware !  No: a vast amount of language processing is unconscious, automatic, unstoppable.!  Example:

"  Words are complex entities, decomposing to smaller parts, so called morphemes: neuro-scien-tist (3 morphemes, at least)

morpheme = the smallest meaning bearing unit of language"  Q: At what stage in processing do words get decomposed into their constituents? "  A: At relatively early levels of visual perception, based simply on the form

properties of the word (i.e., blind to semantics)."  Consequence: your visual system makes mistakes since lots of words look

complex even though they aren’t: corner, brother …"  The meaning-blind decomposition happens at an unconscious level."  Evidence: masked priming

BROTHER (30ms, unconsc) BROTH (until button press) faster thanBROTHEL (30ms, unconsc) BROTH (until button press)

Cognitive = Conscious?

$  The recognition of the basic building blocks of language is a fundamental process in language comprehension and happens early and unconsciously.

$  Cognition does not sit above consciousness.$  Rather, understanding which linguistic computations

require consciousness is a fascinating question.

Cognitive = Conscious?

! Not:" Language as a social construct.

!  Generally, when we talk about languages, such as English, we talk about them as social constructs.

!  Although you and the person sitting next to you may both speak “English,” you do not have the same cognitive ability of language anymore than you have the same visual system or liver.

Language as a cognitive ability

!  When we talk about language as a cognitive ability, there is no such notion as “English”; rather English is multiple linguistic capacities (living in individual brains) that are sufficiently similar such that when these capacities are put to use, the speakers are able to recover from sensory stimulation the mental model that the interlocutor had in mind when generating the auditory output.


!  Yes:" The mental representations and computations that

underlie humans’ ability to produce and comprehend language.


!  What are those mental representations and computations like?

!  Are the mental representations and computations specific to language, or shared with other cognitive abilities? (Modularity)

!  What are the neural bases of linguistic mental representations and computations? (Cognitive neuroscience)

Questions

!  A complete understanding of how sound relates to meaning

" both in terms of the speaker’s subconscious knowledge about what constitutes a well-formed sound-meaning mapping

" and in terms of uncovering meaning from sound in real-time comprehension and executing the motor commands necessary to externalize meaning in language production.

Ultimate goal

“mushrooms are an edible fungus”

continuous speech signal


Spectrogram: Representation of sound amplitude across different frequencies over time.

http://home.cc.umanitoba.ca/~robh/archives/arc0610.html



“In general, if x is a mushroom, then x is edible and x is a fungus”

?




sound categories




sound categories mushroom s are an edi able fungus morphemes





mushrooms are an edible fungus “words”





mushrooms are an edible fungus “words”

syntax

mushroom s are an edi able fungus

mushrooms are an edible fungus


sound categories morphemes

“words”

syntax

semantics



sound categories morphemes

“words”

syntax

semantics


!  Study of speech sounds: Phonetics

!  The meanings of linguistic expressions - Semantics

!  Study of the abstract categories of speech sounds: Phonology

!  Study of the internal structure of words - Morphology

!  Study of the internal structure of sentences - Syntax

! Have some idea of what you’re looking for, i.e., have a theory of what language is. " Equipped with a theory of the internal architecture of language,

we can design experiments that vary the properties of linguistic expressions at different levels of representation. Then we can ask what brain activity is affected by the manipulations. This is classic experimental psychology with brain measurements added.

" Further, if we are equipped with a theory of the internal architecture of language, we can also also conduct more naturalistic experiments in which a stimulus such as a narrative is tagged word by word for its linguistic properties. We can then ask what neural activity correlates with each such property.

How should we go about “finding” language in the brain?

!  Cognitive neuroscience of language needs to sit on the shoulders of theoretical linguistics (the field that gives you theories about what language is).

!  However, much of it hasn’t… for example, the early language tasks used in neuroimaging were mostly designed by physicists, who were trying to come up with some cognitive tasks to assess whether their machines could detect more abstract processing. These tasks have had a huge impact on our field since the findings were big news and thus inspired much follow-up experimentation. " Example: the verb generation task

How should we go about “finding” language in the brain?

The verb generation task

SCISSORS

Generate an associated verb (overtly or covertly)

CAT

Generate an associated verb (overtly or covertly)

!  In order to localize language, what brain regions are active during the verb generation task as opposed to when subjects are just staring at a cross?

+

Q: What aspect of language processing does this represent?

The verb generation task

!  In order to localize language, what brain regions are active during the verb generation task as opposed to when subjects are just staring at a cross?

•  Xiong et al. (Human Brain Mapping 6:42–58 (1998)):•  Left inferior frontal gyrus (BAs 45-47)•  Broca’s area (BAs 44/6)•  Left superior temporal gyrus (BA 22)•  Cingulate gyrus (BAs 32 and 24)•  Inferior temporal gyrus (BA 37)•  Occipital gyri (BAs 18 & 19)•  Basal ganglia•  Thalamus•  Insula•  Cerebellum

An early example of a more theoretically grounded study: Neville et al.

! Question: Are syntactic and semantic processing equivalent?

! More specific question: Is ungrammaticality equivalent to semantic anomaly?

!  Independent variable: type of anomaly! Dependent variable: ERP difference between

anomalous sentences and control sentences at critical word

An early example of a more theoretically grounded study: Neville et al.

!  Semantic anomaly:

!  Phrase structure violation:

!$  Negativity at ~400ms after the onset of the anomaly.

$  Negativity at ~125ms after the onset of the violation.

!  Tells us something about the time course of processing different aspects of linguistic representations. Specifically, results like these gave rise to so-called “syntax-first” hypotheses of language processing.

In this course

!  You’ll learn about the structure of language such that we can talk about experiments varying the properties of linguistic expressions at different levels representations.

!  You’ll mostly see examples of brain research that IS grounded in linguistic theory.

!  We’ll work from sound to meaning, with some special topics inserted (like sign language, or genetic language disorders).

!  In addition to classic findings, you’ll get some idea of current debates and hot topics.

Documents

Introduction, Goals, & Questions