1

CS 177Week 8: Audio Processing

2

Sound

Light and sound are both transmitted in waves

3

Frequency

The human ear can hear between about 12 Hz and 20,000 Hz

The higher the frequency of the wave, the higher the frequency of the note

Note that the A an octaveabove concert A (A440) has twice the frequency

Each half-step is an increase in the frequency by a factor of about 1.06

Note Frequency

A 440B 493.88C 523.25D 587.33E 659.26F 698.46G 783.99A 880

4

Example of frequency change We can take a sound:

And reproduce that sound at double the frequency (speeding it up):

Notice that we have to add twice as much information to have the sound fill the same amount of time

5

Amplitude

The amplitude of a wave is the distance to its peak (measured by its y-value)

In sound, amplitude is a measure of volume

The larger the amplitude, the louder the sound

Amplitude

6

Example of amplitude change We can take a sound:

And make the sound with half the amplitude:

The frequency is exactly the same, but the sound is half as loud

7

Real sounds

Something that looks like a smooth sine wave is called a pure tone

No real instruments play anything like that

Even the purest real sound has overtones and harmonics

Real sound is the result of many messy waves added together:

8

Digital Sampling

On a computer, we cannot record a wave form directly

As usual, we have to figure out a way to store a wave as a series of numbers

We are going to use these numbers to approximate the heights of the wave at various points

9

Sample rate

Hertz (Hz) is a unit that means a number of times per second

We are going to break down the wave into lots of slices

We are going to have 44,100 slices in a second

Thus, we are slicing at 44,100 Hz

10

Sample values

We slice up a wave and record the height of the wave

Each height value is called a sample

By getting 44,100 samples per second, we get a pretty accurate picture of the wave

11

Sample format

There are many different formats for sampling audio

In our system, each sample will be recorded as a double

The minimum value of a sample will be -1.0 and the maximum value of a sample is 1.0

A series of samples with value 0.0 represents silence

Our samples will be stored in an array

12

Purpose of the StdAudio class Audio data on computers is sometimes stored

in a WAV file A WAV file is much simpler than an MP3

because it has no compression Even so, it contains two channels (for stereo)

and can have many different sample rates and formats for recording sound

The StdAudio class lets you read and write a WAV file easily and always deal with a single array of sound, sampled at 44,100 Hz

13

StdAudio methods Everything you’d want to do with sound:

To do interesting things, you have to manipulate the array of samples

Make sure you have StdAudio.java in your directory before trying to use it

Method Usestatic double[] read(String file) Read a WAV file into an array of doublesstatic void save(String file, double[] input)

Save an array of doubles (samples) into a WAV file

static void play(String file) Play a WAV file

static void play(double[] input) Play an array of doubles (samples)

14

StdAudio example Let’s load a file into an array:

If the song has these samples:

Perhaps sample will contain:

String file = “song.wav”;double[] sample = StdAudio.read(file);

-.9 -.7 -.6 -.4 -.2 -.1 .1 .2 .3 .4 .5 .6 .6 .5 .4 .3 .2 0 -.2 -.4

15

StdAudio example

With the audio samples loaded into the array named sample, we can play them as follows:

StdAudio.play(sample);

16

Generating sound with StdAudio

Or, we could generate sound from scratch with StdAudio

This example from the book creates 1 second of the pitch A440:

StdAudio.SAMPLE_RATE is 44100

double[] sound = new double[StdAudio.SAMPLE_RATE + 1];for( int i = 0; i < sound.length; i++ )

sound[i] = Math.sin(2 * Math.PI * i * 440 / StdAudio.SAMPLE_RATE);

StdAudio.play(sound);

17

PlayThatTune.java

The book provides a short program called Play That Tune (pp. 150 and 205) that will generate a sequence of notes according to an input file

It’s a sort of software synthesizer If you know the notes for a song

you’d like to play and their durations, you can create a file to play it with PlayThatTune.java

java PlayThatTune < song.txt

18

Play That Tune file format As you can see, the file is just a list of

numbers giving pitch and duration Although ugly, this form of bookkeeping

takes up virtually no space compared to WAV files

Drawbacks: You can only play one note at a time All notes have a pure tone with a little bit of

harmonics It takes forever to type out an input file No dynamics (volume control)

19

Sound waves

Remember, sound is a wave We are going to start playing

with the waveform now to get different effects

Phase is a property of a wave that can be determined by its starting position

You can think of phase as the alignment of the wave

20

Inverting phase

What happens if we invert a sound wave?

For example, we turn this wave:

Into this wave:

21

Inverting phase in code

How would we invert the phase of a wave in code?

And what does that crazy, upside-down sound sound like?

Exactly the same The human ear is not sensitive to phase

double[] sound = StdAudio.read( file );for( int i = 0; i < sound.length; i++ ) sound[i] = -sound[i]; StdAudio.play(sound);

22

Concerns with phase Phase is still worth knowing about If you hear two copies of the same sound, but

one is 180° out of phase, the sounds will cancel each other out

This can happen if you hook up one of your stereo speakers backwards

23

Reversing a sound

To reverse a sound, we simply send the wave form backwards

For example, we take this sound:

And turn it into this sound:

24

Reversing a sound in code How would we reverse a sound in code?

What does it sound like? Sort of like a bizarre foreign language

double[] sound = StdAudio.read( file );int start = 0; int end = sound.length - 1;while( start < end ) {

double temp = sound[start];sound[start] = sound[end];sound[end] = temp;start++;end--;

}StdAudio.play(sound);

25

Amplitude Recall that the amplitude of a wave is the

distance to its peak (measured by its y-value)

In sound, amplitude is a measure of volume The larger the amplitude, the louder the

sound

Amplitude

26

Example of amplitude change We can take a sound:

And make the same sound with half the amplitude:

The frequency is exactly the same, but the sound is half as loud

27

Reducing volume in code How would we half the volume of a sound in code?

This is half the amplitude. Is this exactly half the volume?

No… this stuff gets complicated Different frequencies with the same amplitude are

actually perceived as different loudnesses But, it’s close enough

double[] sound = StdAudio.read( file );for( int i = 0; i < sound.length; i++ ) sound[i] = 0.5*sound[i];StdAudio.play(sound);

28

Increasing volume in code How would we double the volume of a sound in code? Be careful! Remember that the max is 1.0 and the min is -1.0 If you go outside of that range, you should limit the

valuesdouble[] sound = StdAudio.read( file );for( int i = 0; i < sound.length; i++ ) {sound[i] = 2*sound[i];if( sound[i] > 1.0 )sound[i] = 1.0;else if( sound[i] < -1.0 )sound[i] = -1.0;

}StdAudio.play(sound);

29

Speeding up sound

We can take a sound:

And shrink that sound in half the time (while also doubling its frequency):

30

Speeding up a sound in code How would we double the speed of a

sound in code?

We are just taking every other sample We are throwing out half the information

double[] sound = StdAudio.read( file );double[] speed = new double[sound.length/2]; for( int i = 0; i < speed.length; i++ ) speed[i] = sound[2*i];StdAudio.play(speed);

31

Slowing down sound

We can take a sound:

And stretch that sound to double the time (while also cutting its frequency in half):

32

Slowing down a sound in code How would we stretch that sound to double in code?

Often times, this can sound terrible We are doubling the length of the sound, but we

have no extra information We are just filling in holes in the samples with

copies

double[] sound = StdAudio.read( file );double[] slow = new double[sound.length*2]; for( int i = 0; i < slow.length; i++ ) slow[i] = sound[i/2];StdAudio.play(slow);

33

Speeding up or slowing down non-integer amounts Integers are easy You can apply the same ideas to non-

integer speed-ups and slow-downs Smarter algorithms, like the ones

used by professionals, may do some averaging or other tricks to retain sound quality

34

Adding noise To add noise to a waveform, add small

random numbers to the wave Make sure that the random numbers are both

positive and negative (with a mean of 0) You can turn this wave:

Into this wave:

35

Removing noise How can we turn this wave…

Into this wave?

That is much harder!