2018 FallCTP431: Music and Audio Computing
Sound Synthesis (Part 1)
Graduate School of Culture Technology, KAISTJuhan Nam
Outlines
• Signal model (analog / digital) – Part 1- Additive Synthesis- Subtractive Synthesis- Modulation Synthesis- Distortion Synthesis
• Sample model (digital) – Part 2- Sampling Synthesis- Granular Synthesis- Concatenative Synthesis
• Physical model (digital) – Part 2- Digital Waveguide Model
Signal Model
• Modeling the patterns of musical tones using elementary waveforms- Time domain: ADSR- Frequency domain: spectrum
• Types of signal models- Additive synthesis: a set of sine waveforms - Subtractive synthesis: sawtooth, square waveforms + filters- Frequency modulation synthesis: a pair of sine waveforms - Distortion synthesis: sine waveforms + nonlinear units
• These techniques date back to the analog age
Additive Synthesis
• Synthesize sounds by adding multiple sine oscillators- Also called Fourier synthesis
OSC
OSC
OSC
.
.
.
Amp (Env)
Amp (Env)
Amp (Env)
.
.
.
+
Telharmonium
• Additive synthesizer using electro-magnetic “tone wheels” (Cahill, 1897)- Transmitted through telephone lines - Subscription only but the the business failed
Tone wheel
Hammond Organ
• Drawbars- Control the levels of individual tonewheels
Theremin
• A sinusoidal tone generator- Two antennas are remotely controlled to adjust pitch and volume
Theremin ( by Léon Theremin, 1928)
Theremin (Clara Rockmore)https://www.youtube.com/watch?v=pSzTPGlNa5U
Sound Examples
• Web Audio Demo- http://femurdesign.com/theremin/- http://www.venlabsla.com/x/additive/additive.html- http://codepen.io/anon/pen/jPGJMK
• Examples (instruments)- Kurzweil K150- https://soundcloud.com/rosst/sets/kurzweil-k150-fs-additive
- Kawai K5, K5000
Subtractive Synthesis
• Synthesize sounds by filtering wide-band oscillators- Source-Filter model
5 10 15 20−60
−50
−40
−30
−20
−10
0
10
20
Frequency (kHz)
Mag
nitu
de (d
B)
5 10 15 20−60
−50
−40
−30
−20
−10
0
10
20
Frequency (kHz)
Mag
nitu
de (d
B)
0 0.5 1 1.5 2 2.5x 104
−60
−50
−40
−30
−20
−10
0
10
20
Frequency (kHz)
Mag
nitu
de (d
B)
Source Filter Filtered Source
FilterOscillators Amp
50 52 54 56 58 60−0.4
−0.2
0
0.2
0.4
time−milliseconds
amplitude
Moog Synthesizers
MiniMoog (1970)
Moog Synthesizers
• Architecture
Envelope
Envelope LFO
Wheels Slides PedalPhysical Control
FilterOscillators Amp
Keyboard
Audio Path
SoftControl
Parameter = offset + depth*control (e.g. filter cut-off
frequency)(static value) (dynamic value)
Parameter Parameter Parameter
“Switched-On-Bach” by Wendy Carlos(1968)
Oscillators
• Classic waveforms
• Modulation- Pulse width modulation- Hard-sync- More rich harmonics
0 50 100 150 200−2
0
2
5 10 15 20−60−40−20
020
Frequency (kHz)
Mag
nitu
de (d
B)
−6dB/oct
0 50 100 150 200−1
0
1
5 10 15 20−60−40−20
020
Frequency (kHz)M
agni
tude
(dB)
−6dB/oct
0 50 100 150 200−2
0
2
5 10 15 20−60−40−20
020
Frequency (kHz)
Mag
nitu
de (d
B)
−12dB/oct
Sawtooth TriangularSquare
Amp Envelop Generator
• Amplitude envelope generation- ADSR curve: attack, decay, sustain and release- Each state has a pair of time and target level
Note On Note Off
Attack Decay Sustain
Release
Amplitude(dB)
Examples
• Web Audio Demos- http://www.google.com/doodles/robert-moogs-78th-birthday- http://webaudiodemos.appspot.com/midi-synth/index.html- http://aikelab.net/websynth/- http://nicroto.github.io/viktor/
• Example Sounds- SuperSaw- Leads- Pad- MoogBass- 8-Bit sounds: https://www.youtube.com/watch?v=tf0-Rrm9dI0- TR-808: https://www.youtube.com/watch?v=YeZZk2czG1c
Modulation Synthesis
• Modulation is originally from communication theory- Carrier: channel signal, e.g., radio or TV channel - Modulator: information signal, e.g., voice, video
• Types of modulation synthesis- Amplitude modulation (or ring modulation)- Frequency modulation
• Decreasing the frequency of carrier to hearing range can be used to synthesize sound - Generate new sinusoidal components- Modulation is non-linear processing
Ring Modulation / Amplitude Modulation
• Change the amplitude of one source with another source- Slow change: tremolo- Fast change: generate a new tone
OSC
OSC
(1+ am (t))Ac cos(2π fct)am (t)Ac cos(2π fct)
Amplitude Modulation
xCarrier
Modulator
OSC
OSC
xCarrier
Modulator
+
Ring Modulation
Ring Modulation / Amplitude Modulation
• Frequency domain - Expressed in terms of its sideband frequencies- The sum and difference of the two frequencies are obtained according to
trigonometric identity- If the modulator is a non-sinusoidal tone, a mirrored-spectrum with regard
to the carrier frequency is obtained
fc+fmfcfc-fm
am (t) = Am sin(2π fmt))
carriersidebandsideband
Examples
• Tone generation- SawtoothOsc x SineOsc- https://www.youtube.com/watch?v=yw7_WQmrzuk
• Ring modulation is often used as an audio effect- http://webaudio.prototyping.bbc.co.uk/ring-modulator/
Frequency Modulation
• Change the frequency of one source with another source- Slow change: vibrato- Fast change: generate a new (and rich) tone- Invented by John Chowning in 1973 à Yamaha DX7
Ac cos(2π fct +β sin(2π fmt))
β =Amfm
Index of modulationOSC
OSC
Carrier
Modulator
frequency
Frequency Modulation
• Frequency Domain- Expressed in terms of its sideband frequencies- Their amplitudes are determined by the Bessel function- The sidebands below 0 Hz or above the Nyquist frequency are folded
y(t) = Ac Jk (k=−∞
k=−∞
∑ β)cos(2π ( fc + kfm )t)
fc+fmfc fc+2fm fc+3fmfc-fmfc-2fmfc-3fm
carriersideband1sideband1
sideband2sideband2sideband3sideband3
Frequency Modulation
• Bessel Function
Jk (β) =(−1)n (β
2)k+2n
n!(n+ k)!n=0
∞
∑
0 50 100 150 200 250 300 350−0.5
0
0.5
1
beta
J_(k
)
CarrierSideband 1Sideband 2Sideband 3Sideband 4
The Effect of Modulation Index
0 500 1000 1500 2000−1
0
1
Time (Sample)
Ampl
itude
0.2 0.4 0.6 0.8 1−60−40−20
020
Frequency (kHz)
Mag
nitu
de (d
B) Beta = 0
0 500 1000 1500 2000−1
0
1
Time (Sample)
Ampl
itude
0.2 0.4 0.6 0.8 1−60−40−20
020
Frequency (kHz)
Mag
nitu
de (d
B) Beta = 1
0 500 1000 1500 2000−1
0
1
Time (Sample)
Ampl
itude
0.2 0.4 0.6 0.8 1−60−40−20
020
Frequency (kHz)
Mag
nitu
de (d
B) Beta = 10
0 500 1000 1500 2000−1
0
1
Time (Sample)Am
plitu
de
0.2 0.4 0.6 0.8 1−60−40−20
020
Frequency (kHz)
Mag
nitu
de (d
B) Beta = 20
fc = 500, fm = 50
FM Synthesizer
Yamaha DX7 (1983)
Examples
• Web Audio Demo- http://www.taktech.org/takm/WebFMSynth/
• Sound Examples- Bell- Wood - Brass- Electric Piano- Vibraphone
Non-linear Synthesis (wave-shaping)
• Generate a rich sound spectrum by distorting sine waveforms using non-linear transfer functions
• Also called “distortion synthesis”
0 50 100 150 200−1
0
1
Time (Sample)
Am
plitu
de
5 10 15 20−60−40−20
020
Frequency (kHz)
Mag
nitu
de (d
B)
0 50 100 150 200−1
0
1
Time (Sample)
Am
plitu
de
5 10 15 20−60−40−20
020
Frequency (kHz)
Mag
nitu
de (d
B)
−1 −0.5 0 0.5 1−1
−0.5
0
0.5
1
Time (Sample)
Ampl
itude
x’=gx: g correspond to the “gain” of the distortion
Distortion Transfer Function
• Examples of transfer function: y = f(x)- y = 1.5x’ – 0.5x’3
- y = x’/(1+|x’|)- y = sin(x’)- Chebyshev polynomial: Tk+1(x) = 2xTk(x)-Tk-1(x)
T0(x)=1, T1(x)=x, T2(x)=2x2-1, T2(x)=4x3-3x