Acoustics of Speech Lecture 4 Spoken Language Processing Prof.
Andrew Rosenberg
Slide 3
Overview What is in a speech signal? Defining cues to phonetic
segments and intonation. Techniques to extract these cues. 1
Slide 4
Phone Recognition Goal: Distinguishing One Phoneme from
AnotherAutomatically ASR: Did the caller say I want to fly to
Newark or I want to fly to New York? Forensic Linguistics: Did that
person say Kill him or Bill him What evidence is available in the
speech signal? How accurately and reliably can we extract it? What
qualities make this difficult? easy? 2
Slide 5
Prosody and Intonation How things are said is sometimes
critical and often useful for understanding Forensic Linguistics:
Kill him! vs. Kill him? TTS: Travelling from Boston? vs. Travelling
from Boston. What information do we need to extract from/generate
in the speech signal? What tools do we have to do this? 3
Slide 6
Speech Features What cues are important? Spectral Features
Fundamental Frequency (pitch) Amplitude/energy (loudness) Timing
(pauses, rate) Voice Quality How do we extract these? Digital
Signal Processing Tools and Algorithms Praat Wavesurfer Xwaves
4
Slide 7
Sound Production Pressure fluctuations in the air caused by a
voice, musical instrument, a car horn etc. Sound waves propagate
through material air, but also solids, etc. Cause eardrum
(tympanum) to vibrate Auditory system translates this into neural
impulses Brain interprets these as sound Represent sounds as change
in pressure over time 5
Slide 8
How loud are sounds? EventPressure (Pa)dB Absolute silence200
Whisper20020 Quiet office2K40 Conversation20K60 Bus200K80
Subway2M100 Thunder20M120 *Hearing Damage*200M140 6
Slide 9
Voiced Sounds are (mostly) Periodic Simple Periodic Waves (sine
waves) defined by Frequency: how often does the pattern repeat per
time unit Cycle: one repetition Period: duration of a cycle
Frequency: #cycles per time unit (usually second) Frequency in
Hertz (Hz): cycles per second or 1 / period E.g. 400 Hz = 1/0.0025
(a cycle has a period of 0.0025 seconds; 400 cycles complete in a
second) Zero crossing: where the waveform crosses the x- axis
7
Slide 10
Voiced Sounds are (mostly) Periodic Simple Periodic Waves (sine
waves) defined by Amplitude: peak deviation of pressure from normal
atmospheric pressure Phase: timing of a waveform relative to a
reference point 8
Slide 11
Phase Differences 9
Slide 12
Complex Periodic Waves Cyclic but composed of multiple sine
waves Fundamental Frequency (F0): rate at which the largest pattern
repeats and its harmonics Also GCD of component frequencies
Harmonics: rate of shorter patterns Any complex waveform can be
analyzed into its component sine waves with their frequencies,
amplitudes and phases (Fourier theorem in 2 lectures) 10
Slide 13
2 sine wave -> 1 complex wave 11
Slide 14
4 sine waves -> 1 complex wave 12
Slide 15
Power Spectra and Spectrograms Frequency components of a
complex waveform represened in the power spectrum. Plots frequency
and amplitude of each component sine wave Adding temporal dimension
-> Spectrogram Obtained via Fast Fourier Transform (FFT), Linear
Predictive Coding (LPC) Useful for analysis, coding and synthesis.
13
Slide 16
Example Power spectrum 14
http://clas.mq.edu.au/acoustics/speech_spectra/fft_lpc_settings.html
Australian male /i:/ from heed FFT analysis window 12.8ms
Slide 17
Example Spectrogram 15
Slide 18
Terms Spectral Slice: plots the amplitude at each frequency
Spectrograms: plots amplitude and frequency over time Harmonics:
components of a complex waveform that are multiples of the
fundamental frequency (F0) Formants: frequency bands that are most
amplified in speech. 16
Slide 19
Aperiodic Waveforms Waveforms with random or non-repeating
patterns Random aperiodic waveforms: white noise Flat spectrum:
equal amplitude for all frequency components. Transients: sudden
bursts of pressure (clicks, pops, lip smacks, door slams, etc.)
Flat spectrum at a single impulse Voiceless consonants 17
Slide 20
Speech Waveforms Lungs plus vocal fold vibration is filtered by
resonance of the vocal tract to produce complex, periodic
waveforms. Pitch range, mean, max: cycles per sec of lowest
frequency periodic component of a signal = Fundamental frequency
(F0) Loudness RMS amplitude Intensity: in dB where P0 is a
reference atmospheric pressure 18
Slide 21
Collecting speech for analysis? Recording conditions A quiet
office, a sound booth, an anechoic chamber Microphones convert
sound into electrical current oscillations of air pressure are
converted to oscillations of current Analog devices (e.g. tape
recorders) store these as a continuous signal Digital devices (e.g.
DAT, computers) convert to a digital signal (digitizing) 19
Slide 22
Digital Sound Representation A microphone is a mechanical
eardrum, capable of measuring change in air pressure over time.
Digital recording converts analog (smoothly continuous) changes in
air pressure over time to a digital signal. The digital
representation: measures the pressure at a fixed time interval
sampling rate represents pressure as an integral value bit depth
The analog to digital conversion results in a loss of information.
20
Slide 23
Waveform Name 21
Slide 24
Analog to Digital Conversion Quantization or Discretization
22
Slide 25
Analog to Digital Conversion Quantization or Discretization
23
Slide 26
Analog to Digital Conversion Quantization or Discretization
24
Slide 27
Analog to Digital Conversion Quantization or Discretization
25
Slide 28
Analog to Digital Conversion Bit depth impact 16bit sound CD
Quality 8bit sound Sampling rate impact 44.1kHz 16kHz 8kHz 4kHz
26
Slide 29
Nyquist Rate At least 2 samples per cycle are necessary to
capture the periodicity of a waveform at a given frequency 100Hz
needs 200 samples per sec Nyquist Frequency or Nyquist Rate Highest
frequency that can be captured with a given sampling rate 8kHz
sampling rate (Telephone speech) can capture frequencies up to 4kHz
27
Slide 30
Sampling/storage trade off Human hearing: ~20kHz top frequency
Should we store 40kHz samples? Telephone speech 300-4kHz (8kHz
sampling) But some speech sounds, (e.g., fricatives, stops) have
energy above 4kHz Peter, Teeter, Dieter 44kHz (CD quality) vs.
16-22kHz Usually good enough to study speech, amplitude, duration,
pitch, etc. Golden Ears. 28
Slide 31
Filtering Acoustic filters block out certain frequencies of
sounds Low-pass filter blocks high frequency components High-pass
filter blocks low frequencies Band-pass filter blocks both high and
low, around a band Reject band (what to block) vs. pass band (what
to let through) What if the frequencies fo two sounds overlap?
Source Separation 29
Slide 32
Estimating pitch Pitch Tracking: Estimate F0 over time as a
function of vocal fold vibration How? Autocorrelation approach A
periodic waveform is correlated with itself, since one period looks
like another Find the period by finding the lag (offset) between
two windows of the signal where the correlation of the windows is
highest Lag duration, T, is one period of the the waveform F0 is
the inverse: 1/T 30
Slide 33
Pitch Issues Microprosody effects of consonants (e.g. /v/)
Creaky voice -> no pitch track, or noisy estimate Errors to
watch for: Halving: shortest lag calculated is too long, by one or
more cycles. Since the estimated lag is too long, the pitch is too
low (underestimation) of pitch Doubling: shortest lag is too short.
Second half of the cycle is similar to the first Estimates a short
lag, counts too many cycles per second (overestimation) of pitch
31
Slide 34
Pitch Doubling and Halving 32 Halving Error Doubling Error
Slide 35
Next Class Speech Recognition Overview Reading: J&M 9.1,
9.2, 5.5 33