ESE 250: Digital Audio Basics Week 4 February 5, 2013 The Frequency Domain 1ESE 250 - Spring'13 DeHon, Kod, Kadric, Wilson-Shah

1

ESE 250:Digital Audio Basics

Week 4February 5, 2013

The Frequency Domain

ESE 250 - Spring'13 DeHon, Kod, Kadric, Wilson-Shah

2

Course Map


3

Musical Representation

• With this compact notation– Could communicate a sound to pianist– Much more compact than 44KHz time-sample

amplitudes (fewer bits to represent)– Represent frequencies


4

Physics review

• Consider a sound signal that varies as a sine wave in time:

• What is the relationship between and ?– rad/s, Hz


)sin()( tAts

ff 2

)2sin()( ftAts

Tf /1

5

Time-domain Digital representation

• Week2: Quantization in TIME and VALUE

• Question: Other ways to represent this signal?


6

Another-domain representation?

• Consider a sound signal that varies as a sine wave in time:

• Note that a sine wave is periodic, this implies redundancy that we could exploit to better compress it (week3 – compression)

• At least we could save memory by representing only one period. But can we do even better?

• Question: What information do we need to represent the signal?– Amplitude, frequency, phase

• Question: How much memory to store each of these?ESE 250 - Spring'13 DeHon, Kod, Kadric, Wilson-Shah

)2sin()( ftAts

7

Another-domain representation?

• Remember week 2 lab:• Inputs:

– Amplitude: 1– Frequency: various values, 220Hz, etc…– Phase: 0– Sampling rate: 48,000 kHz

• Output: Periodic sine function sampled in time, with 48,000 samples for 1 second of sound

• Question: Why did we need to create the large LVM files? Why not just store the little information we started with?


8

Storage advantage

• Assume each of amplitude, frequency and phase are stored as 16bit integers.

• How much memory to store the sine in lab2?– Only storing the info above?– With an LVM file containing 48,000 samples?

• Percent savings?


9

Frequency-domain

• T = π, A = 3: s(t) = A*sin(2π*f*t) = 3*sin(2*t)


10

Frequency-domain• Of course, not all signals are this simple• For example, consider s(t) = sin(2t) + sin(t)/2• Question: What will the frequency

representation look like?


11ESE 250 - Spring'13 DeHon, Kod, Kadric, Wilson-Shah

Frequency-domain• How about the time-domain?

– Plot sin(2t)– Plot sin(t)/2– Sum: sin(2t)+ sin(t)/2

• Notice how it was easier to plot the frequency domain representation

12

Frequency-domain

• Another example

• What function is this?


13

Frequency-domain

• So far, we have seen how a signal written as a sum of sines of different frequencies can have a frequency domain representation.

• Each sine component is more or less important depending on its coefficient

• But can any arbitrary signal be represented as a sum of sines?– No. But the idea has potential, let’s explore it!


14

More background

• Fourier series:– Any periodic signal can

be represented as a sum of simple periodic functions: sin and cos

sin(nt) and cos(nt)For n = 1, 2, 3, …These are called the

harmonics of the signal Jean Baptiste Joseph Fourier, wikipedia


15

Fourier Series – more formallyGiven a (well-behaved) periodic function f(t), wecan always write it as:

Where:

These two equations give us the and coefficients of the Fourier series based on the input function.

These coefficients form the frequency domain representation We will not prove the equations If you’re interested: Office Hours, ESE 325, Math 312, …

N

nnn

o ntbntaa

tf1

)]sin()cos([2

)( 0N

dtnttfan )cos()(

1

dtnttfbn )sin()(

1

0N

1N


na nb


Fourier Series – Why does it work?

• The cos(nx) and sin(nx) functions form an orthogonal basis: they allow us to represent any periodic signal by taking a linear combination of the basis components without interfering with one another

17

Orthogonal basis• Choose a basis• Order the vectors in it• All vectors in the space can be represented as sequences

of coordinates, or ordered coefficients of the basis vectors• Example: 3D space

– Basis: [i j k]– Linear combination: xi + yj + zk– Coordinate representation: [x y z]

• Example: Quadratic polynomials– Basis: [x2 x 1]– Linear combination: ax2 + bx + c– Coordinate representation: [a b c]


18

Fourier Series

• Consider the xy-plane:

• We can address a point by its two coordinates (x,y)• The blue point is located at (10, 3)

• In Fourier series, we have a basis with infinite dimension, so we sum up over an infinite number of harmonics

• Harmonics are 1, cos(t), cos(2t), cos(3t), …, sin(t), sin(2t), …

0 5 100

5

10


19

Example of another orthogonal basis

• Although we have an infinite number of basis elements, we don’t need to use all of them

• The components with the largest coefficients are the most significant

• The more components we add, the closer to the function we get.• i.e.: As ,

Examples followN 0error



Fourier Series – Sawtooth wave

(falstad.com/fourier/)


Fourier Series – Sawtooth wave

(falstad.com/fourier/)

22

Interlude


23

Where Are We Heading After Today?• Week 2

– Received signal is• discrete-time-stamped• quantized

– q = PCM[ r ]

= quantL [SampleTs[r] ]

• Week 3– Quantized Signal is Coded– c =code[ q ]

• Week 4 – Sampled signal

• not coded directly • First linearly

transformed into frequency domain

– Q = DFT[ q ] [Painter & Spanias. Proc.IEEE, 88(4):451–512, 2000]

q

Sample Code Store/Transmit Decode Producer(t) p(t)

Generic Digital Signal Processor

q c

c

Q

Psychoacoustic Audio Coder


24

What now?

• In the first half of the lecture we have introduced the idea of frequency domain

• In the second half of the lecture, we will see how to extend the idea of Fourier Series to the class of signals we are interested in:

• Not necessarily periodic and quantized


25

Frequency-domain

• Fourier Series deal with periodic signals• Are all signals periodic?• Is your favorite mp3 song periodic?• What do we do with these?


26

Frequency-domain

• Certainly, we could loop a song and make the signal periodic, but this is not a good solution

• Mathematicians have extended the notion of Fourier Series into that of Fourier Transformsby making the period of the signal go to infinity: T


27

Discrete Fourier Transforms• Fourier Transforms are nice, but we want to store and process

our signals with computers• We therefore extend Fourier Transforms into Discrete

Fourier Transforms, or DFT• The signal is now discrete: • The signal contains N samples: • Remember Euler’s formula:

1

0

/2)(N

n

Nkninkn exXxDFT

1

0

/21 1)(

N

k

Nkniknk eX

NxXDFT


nxtx )(

10 Nn

The frequency domain signal also hasN-1 elements:

The time domain signal can be reconstructed from

kX

nx

kX

10 Nk

)sin()cos( titeit

28

Discrete Fourier Transforms• A DFT transforms N samples of a signal in time domain into

a (periodic) frequency representation with N samples• So we don’t have to deal with real signals anymore• We work with sampled signals (quantized in time), and the

frequency representation we get is also quantized in time!


1

0

/2)(N

n

Nkninkn exXxDFT

(sepwww.stanford.edu/oldsep/hale/FftLab.html)

29

Discrete Fourier Transforms

• A smaller sampling period-> More points to represent the signal

larger N-> More harmonics used in DFT

N harmonics-> Smaller error compared to actual

analog signal we capture/produce• DFTs are extensively used in practice, since

computers can handle themESE 250 - Spring'13 DeHon, Kod, Kadric, Wilson-Shah

30

Discrete Fourier Transform exercise

• Complete the table below using the DFT equation shown above


1

0

/2)(N

n

Nkninkn exXxDFT

1

0

1

0

)2

sin()2

cos(N

nn

N

nnk N

knxi

N

knxX

(The bottom is redundant because the DFT of a real signal is symmetrical)7 samples -> 7 harmonics

31

• Approximate Reconstruction– Always achievable

• A signal sampled in time can be approximated arbitrarily closely from the time-sampled values

• With a DFT, each sample gives us knowledge of one harmonic

• Each harmonic is a component used in the reconstruction of the signal

• The more harmonics we use, the better the reconstruction

Approximating the Sampled Signal

{ Cos[0t], Sin[1t], Cos[1t] , Sin[2t], Cos[2t] , Sin[3t], Cos[3t] }


32

Usually Computed, Not “Solved”7 Samples; 7 Harmonics 11 Samples 15 Samples; 15 Harmonics; 11 Harmonics

DFT DFTDFT


ESE 250 – Spring’10 DeHon & Koditschek 33

Yet Another Sampled (Real) Signal

t

v

Measured Data:

Sampled Signal:

34

• Approximate Reconstruction– although always achievable– may require a lot of samples – to get good performance

Sometimes time is better than frequency

Some Signals Dislike Some Harmonics

15 Samples & Harmonics

21 Samples & Harmonics

31 Samples & HarmonicsESE 250 - Spring'13 DeHon, Kod, Kadric, Wilson-Shah

35

Saving resources

• However, N can get very large• e.g., with a sampling rate of 48,000Hz• How big is N for a 4min song?• How many operations does this translate to?

– To compute one frequency component?– To compute all N frequency components?

• This is not practical. Instead, we use a window of values to which we apply the transform– Typical size: …, 512, 1024, 2048, …


36

A Window Operation

• In the example below, we traverse the signal but only look at 64 samples at a time


Time

Freq

(sepwww.stanford.edu/oldsep/hale/FftLab.html)

37

A Window Operation• The following shows a DFT (on the right) applied to a sinusoidal signal (on

the left). The DFT is only applied to part of the signal, the window, which is shown in the middle

• When we increase the window size, the information of the DFT is more accurate


8-point DFT

(http://math.mit.edu/mathlets/mathlets/discrete-fourier-transform/)

16-point DFT32-point DFT

38

Moonlight Sonata in Time and Frequency


39

Big ideas• A signal in time-domain can also be represented in the frequency domain.

– X(f) = Transform(x(t))• This representation may be more efficient:

– For storage– For signal processing– For understanding the wave (e.g. can cut-off some frequencies)

• We go from a signal represented by values in time to one represented by coefficients of harmonics

• Fourier Series, for continuous periodic signals• FT: Fourier Transform, for continuous signals• DFT: Discrete Fourier Transform, for discrete signals

– DFTs are extensively used in practice– Finite amount of data, easy to implement for computers


40

Admin

• Lab 3 due Wednesday (tomorrow)• On Thursday, Lab 4:

– You will be looking at signals in the frequency domain in LabView

– You will create your own synthesizer!


41

Announcement

• Hackathon and bootcamp– If you are interested

• The bootcamp will be this Saturday, noon to six, in Detkin lab.

• The Hackathon will begin around 5-6 the weekend of the 15-17th


Documents

ESE 250: Digital Audio Basics Week 4 February 5, 2013 The Frequency Domain 1ESE 250 - Spring'13 DeHon, Kod, Kadric, Wilson-Shah