SLHS 1301 The Physics and Biology of Spoken LanguageThe Physics and Biology of Spoken Language

Yang Zhang, Ph.D.

6:00 – 7:15 pm T. Th.20 Shevlin Hall20 Shevlin HallSummer, 2011

(June 13 – August 5 2011)(June 13 – August 5, 2011)

Speech-Language-Hearing Sciences

1

University of Minnesota

Overview 4 credits: lectures (discussions) + lab

2 liberal education requirements h i l i i h l b» a physical science course with laboratory

» a writing intensive course

3 goals» Introduction to the SLHS field» Introduction to the empirical methods

2» Introduction to technology in speech and hearing

Structure60% lecture, 40% discussion & Exercises3 ( h 25%) + l b k (25%)3 exams (each 25%) + lab work (25%)Lecture Instructor: Yang Zhang, PhDhttp://www.tc.umn.edu/~zhang470

Teaching assistant:Sharon Miller, PhD studentg ,– Labs– Grades– Reviews– Office hours for individual feedbacks and Q/As.

3

Course Web Site: http://www.tc.umn.edu/~zhang470

MaterialsTextbook

» Denes P B and Pinson E N (1993 Second» Denes, P.B. and Pinson, E.N. (1993, Second Edition). The Speech Chain: The Physics and Biology of Spoken Language. New York: W. H.Biology of Spoken Language. New York: W. H. Freeman and Company.

Laboratory ManualLaboratory Manual» available online (updated on weekly basis).Lecture Materials Study Guides ReviewsLecture Materials, Study Guides, Reviews

» available online (updated on weekly basis).

4

Chapter 1 Lecture Contents

1. Introduction to speech, language, and hearing sciences p , g g , g

2. Why is speech so important?

3. What is the speech chain? How do we study it?

4. Discussion - The making of a human-machine dialogue system

5 Practice Test5. Practice Test

5

1. Introduction

The field» studies speech and language development, anatomy and

physiology of speech and hearing, speech perception and production speech language and hearingand production, speech, language and hearing pathologies, and basic evaluation, treatment and management procedures. g p

» Video - Speech pathologyp p gy- Audiology

http://video.google.com/videoplay?docid=-

6

p g g p y5508745086763024203&q=speech+pathologist&hl=en

2. Why is speech so important? Some reasons» communicate with each other, share experiences,communicate with each other, share experiences,

exchange ideas, and transmit knowledge.» think abstractly, reflect on past history and plan for the

future.» express our emotions and identities.

ffi i d i f i f i h» efficient and primary way of information exchange.» many great inventions.

7

Comparison with Alternative FormsSpeech vs. written language» Efficiency

–135 words/minute (normal speech rate)–655 words/minute (World record)

–40 words/minute (average touch typing)–216 words/minute (World record)

Johann Gutenberg» Convenience

» For the sake of record

Johann Gutenberg (1400 - 1468)

» For the sake of record–Influence by dialects, accents, speaking habits, etc.–Vulnerability to distortions, masking, and

8

y ginterference

Comparison with Alternative FormsSpeech vs. sign language» Temporal structure (auditory system)» Temporal structure (auditory system)» Spatial structure (visual system)

Li i ti i ti» Linguistic organization – Phonetics, phonology, syntax, semantics, pragmatics

B i i ti» Brain organization – Hemisphere specialization

» Language acquisition» Language acquisition – Genetics or environment? (Nature vs. Nurture Debate)– Is speech unique to humans? (Sign language?)

9

– Is speech unique to humans? (Sign language?)

The Talking Apes and ChimpanzeesMeet Washoe» http://www.friendsofwashoe.org/p g

A conversation with Koko» http://www.koko.org/

10

Man vs. Machine

11

3. The Speech Chain

12

Linguistic Level

Linguistic Entities

» Phonemes» Syllables» WordsWords» Phrases» Sentence» Discourse

What are the three major questions in linguistics?» structure» acquisition» use

13

Physiological LevelNeural & muscular structures (CNS, PNS)

1. Auditory system (sensory)» Outer ear, middle ear, inner ear (cochlea) » Auditory nerve – spiral gangalion – trapzoid body –

inferior colliculus – superior olive – medial geniculate body – primary auditory cortex – association auditory cortex

14

2. Speech production system (motor)» Diaphragm rib cage lungs trachea larynx pharynx» Diaphragm, rib cage, lungs, trachea, larynx, pharynx,

oral cavity, nasal cavity» Voicing: vocal folds arytenoid thyroid cricoid» Voicing: vocal folds, arytenoid, thyroid, cricoid, …» Articulators: tongue, velum, teeth, jaw, lips …

3 Vi l t3. Visual system4. Tactile system

15

Acoustic LevelSound source and mediumProperties of soundProperties of sound» frequency» intensity» intensity» phase

d ti» durationSpeech acoustics» Waveform, spectrum, spectrogram» Acoustic features of speech

16

Speech as a Code SystemThe communication system must have a» SOURCE (TRANSMITTER encoding)» SOURCE (TRANSMITTER, encoding)

» TRANSMITTING MEDIUM(COMMUNICATION CHANNEL)

DESTINATION (RECEIVER d di )» DESTINATION (RECEIVER, decoding)

17

How do we study speech and language?» Diachronic vs synchronic» Diachronic vs. synchronic» Prescriptive vs. descriptive

B h i l h i l i l» Behavioral vs. neurophysiological

Extra links on the world wide web» http://www.asha.org/» http://asa.aip.org/» http://apu.sfn.org/

18» http://www.cogneurosociety.org/

19

Practice Test1 . Speech communication consists of a chain of

events linking the speaker’s brain with theevents linking the speaker s brain with the listener’s brain, which can be described by the following three levels of scientific analyses:following three levels of scientific analyses:

a) linguistic, physiological, and acousticb) h i l bi l i l d ditb) physical, biological and auditoryc) sensory, motor, and linguisticd) l l d did) neural, muscular, and auditory

20

2. The physiologic level of the speech chain consists ofof ____ .a) muscular activity necessary to generate speech soundsb) l ti it t t h db) neural activity necessary to generate speech sounds

arrangement and ordering of speech soundsc) muscular and neural activity necessary to receive andc) muscular and neural activity necessary to receive and

generate speech soundsd) linguistic level and acoustic leveld) linguistic level and acoustic level.

21

3. Why is speech/language so important? Because we primarily rely on speech and language towe primarily rely on speech and language to

a) exchange ideas and transmit knowledge efficientlyb) ti d id titb) express our emotions and identityc) think abstractly d) ll f h bd) all of the above

22

4. Which of the following is NOT considered as a defining feature of spoken language?eatu e o spo e a guage?a) Spoken language uses symbols to represent objects or ideas.b) While word-object associations were mostly arbitrary initially, ) j y y y

they became conventional and stable in daily use.c) Spoken language can be further analyzed at different levels

h h ti d h l d l l h itsuch as phonetics and phonology, and every level has its own rules or constraints.

d) The arrangement and rearrangement of word elements into ) g gphrases and sentences are arbitrary; therefore, there can be an infinite number of sentences.

23

5. Speech variability is a big challenge in spoken language technology Which of the followinglanguage technology. Which of the following contributes to speech variability?

a) speaker (age gender emotional mood etc )a) speaker (age, gender, emotional mood, etc.)b) environmental noise) li i ti t t d h tc) linguistic context and speech rate

d) All of the above

24

6. An idealized communication system is composed ofof

a) source (transmitter), medium (communication channel) and destination (receiver)channel), and destination (receiver).

b) encoder, decoder and billions of neurons.c) speaker microphone and electric powerc) speaker, microphone, and electric power.d) consonants, vowels, and semivowels.

25

Chapter 2p

LINGUISTIC ORGANIZATIONLINGUISTIC ORGANIZATIONPhoneticsPhonologyPhonologyMorphology

SyntaxSemanticsSemanticsPragmatics

26

Consider the analogy of language being a symphonic musical scoresymphonic musical score» The rhythm, melody, harmony, intensity patterns,

etc for the orchestra are charted on the scoreetc. for the orchestra are charted on the score» The score identifies when each orchestra member

should play.should play. » Who interprets the musical score?

The conductor who has license to add variationsThe conductor, who has license to add variations from the score to supply his/her signature to the piece

27

p

Each talker is in a sense a conductor but as aEach talker is, in a sense, a conductor -- but as a language user, the talker, like the conductor, will be constrained by certain rules that can be ycalled LINGUISTIC RULESWhen operating within those rules, the talker-p g ,conductor specifies what, when, and how the “linguistic orchestra will play.”

28

The Organization of Language

At this introductory level we will confine ourAt this introductory level, we will confine our discussion to phonology: phonemes formed into wordsinto wordsThe message that the talker intends to convey

t fi t b d i li i timust first be arranged in a proper linguistic form -- the talker must select the appropriate

d d t t th t twords and sentences to convey the content or idea

29

The Organization of Language(Cont’d)(Cont’d)

The units of language are SYMBOLS thatThe units of language are SYMBOLS that represent objects, concepts, ideas, and so forth» Words are symbols:» Words are symbols:

– “Chair” is the symbol for an object or a position occupied

– “Distraught” is a symbol for a state of mind» Language is a system that consists of these symbols

-- and RULES FOR ORGANIZING THE-- and RULES FOR ORGANIZING THE SYMBOLS INTO SEQUENCES THAT WILL ENABLE THE TALKER TO EXPRESS

30

THOUGHTS, IDEAS, AND SO FORTH

The Organization of Language(Cont’d)(Cont d)

Words comprise a sequence of even smaller p qlinguistic units called PHONEMES (speech sounds))Consider “heed,” “hid,” and “had”» In this example the initial and final elements» In this example, the initial and final elements

(phonemes) are “identical”» The three words are distinguished by the middle» The three words are distinguished by the middle

elements (a vowel phoneme)

31

The PhonemeThe phoneme is the basic, or fundamental, linguistic unitlinguistic unit» The phoneme, when existing alone, usually

symbolizes nothing » But in relation to other phonemes, words

(combinations of phonemes) can be distinguished from one anotherfrom one another

» /p/ symbolizes nothing -- but in combination with “ it”, we can distinguish “pit” from “bit”_ it , we can distinguish pit from bit

» We will confine our discussion to the phonemes of English

32

The Phoneme (Cont’d)

Phonemes can be divided into one of two groups: 15 vowels and diphthongs, and 24 consonants» As we shall learn later, the vowels can be

further divided into:– MONOPHTHONGS (pure vowels, e.g. /ɒ/– as in “bomb” and /Ʋ/ as in “book”) – DIPHTHONGS (a combination of two vowels

into one phoneme e g /aƲ/ as in “cow”

33

into one phoneme, e.g., /aƲ/ as in cow

The Phoneme (Cont’d)» The consonants can be further divided into:

– STOPS (PLOSIVES) (e.g., /p/)STOPS (PLOSIVES) (e.g., /p/)– FRICATIVES (e.g., /s/)– AFFRICATES (e.g., / t∫ / as in “chew”)– NASALS (e.g., /m/)– APPROXIMANTS (e.g., /r/)

34

The Phoneme (Cont’d)

Recall that the independent phoneme does notRecall that the independent phoneme does not necessarily symbolize anything.We can, however, elevate certain phonemes to aWe can, however, elevate certain phonemes to a status of meaning» /s/ might mean “boo” or “hiss”g» /ʃ/ might mean “be quiet”

How, then, do we form meaningful symbols?How, then, do we form meaningful symbols?» The phoneme can be combined with other

phonemes to compose a larger unit, the

35

p o e es o co pose ge u , eSYLLABLE

The Syllable and Word

As the textbook indicates, most of us have an intuitiveAs the textbook indicates, most of us have an intuitive sense of what a syllable is, even though scholars sometimes disagree on a formal definition» For example how many syllables are there in each» For example, how many syllables are there in each

word of the following nonsensical sentence?» “Small boat with a picture has become”

1 1 1 1 2 1 2In this case, the syllable is a unit of spoken language that consists of a vowel alone (/ /) or of a vowel (orthat consists of a vowel alone (/ /), or of a vowel (or diphthong) with one or more consonants (/h z/ as in “has”)

36

The Syllable and Word (Cont’d)

However, as we learned earlier, there are restrictions on which h b bi d d h h b bi dphonemes can be combined and how they can be combined» If there were no restrictions, and if, for

computational convenience we could produce 50computational convenience, we could produce 50 phonemes, we could form

– 2,450 two-phoneme syllables– ? three-phoneme syllables

Because of constraints, the English language probably , g g g p ycomprises less than 2,000 syllables, and most words in English comprise a sequence of 2-5 phonemes and only 1 3 syllables

37

1-3 syllables

The Syllable and Word (Cont’d)One prominent restriction» One of the nasals /ŋ/ cannot be in the syllable» One of the nasals /ŋ/ cannot be in the syllable-

initial position of a word in EnglishThe WORD is a larger linguistic unit thatThe WORD is a larger linguistic unit that ordinarily consists of a sequence of phonemes combined into one or more syllablescombined into one or more syllables

38

The Syllable and Word (Cont’d)

Fi ll d b bi d i t till lFinally, words can be combined into still longer linguistic units such as PHRASES or SENTENCESSENTENCES The structure of sentences can be described by the GRAMMAR of a language, which includes: PHONOLOGY, MORPHOLOGY, SYNTAX, AND SEMANTICSTh t hi h ill b i t d d iThose concepts, which will be introduced in more advanced coursework.

39

Summary Write down the key concepts we have learned in Chapters 1 and 2 and your own definitions forChapters 1 and 2 and your own definitions for these concepts.

40