Infinity-project.org © 2005-2008 The Institute for Engineering Education at SMU. All Rights Reserved Engineering Education for today’s classroom. Unit

infinity-project.org© 2005-2008 The Institute for Engineering Education at SMU. All Rights Reserved

Engineering Education for today’s classroom.

Unit 1: Sound Mixing, Demixing and Spatial Audio



1.1 Sound Mixing and Demixing

When two or more sound waves are created near each other, such as two musical instruments playing a tune, the sound waves combine together to make one sound wave.

Your brain tries to imagine what could have possibly made the sound.

It is hard to make an artificial or electronic device, such as microphones connected to a digital computer or processor, do the same thing.



Sound Mixing and DemixingAudio Examples



Building A Sound Separation System

These sounds were recorded using actual microphones

Can we process the recordings to pull out individual voices? YES!



Mission: Visioneer It Video(Show this to your students!)



1.2 The Math Behind Demixing Sounds

• Suppose we have a speech sound = Speech• And, noise sound that we call = Noise• A microphone signal picks up 50% of Speech and

50% of Noise”Mic1 = (0.5 x Speech) + (0.5 x Noise)

More realistic:Mic 1 = (0.4 x Speech) + (0.6 x Noise)

Mic 2 = (0.3 x Speech) + 0.7 x Noise)Question: Why are the microphone mixtures

different?



Two Ears Might be Better than One

Idea: Add the microphone signals togetherMic1 + Mic2 = (0.4 x Speech) + (0.6 x Noise)

+(0.3 x Speech) + 0.7 x Noise)= (0.4 + 0.3) x Speech + (0.6 + 0.7) x

Noise= (0.7 x Speech) + (1.3 x Noise)

Concept: When the number in front of one sound is bigger, it is louder than the other sound

In practice, only relative levels matter, so we can turn the amounts into percentages again

0.5 x (Mic1+Mic2) = (0.35 x Speech) + (0.65 x Noise)



Two Ears Might be Better than One

How Do We Cancel the Noise? Idea: Subtract microphone signals

Mic1 - Mic2 = (0.4 x Speech) + (0.6 x Noise)

- [(0.3 x Speech) + 0.7 x Noise)]

= (0.4 - 0.3) x Speech + (0.6 - 0.7) x Noise

= (0.1 x Speech) - (0.1 x Noise) Ears ignore negative signs: - (0.1 x Noise) sounds like

(0.1 x Noise) Turn fractions into percentages again: 0.1 + 0.1 = 0.2, so

5 (Mic1 – Mic2) = (0.5 x Speech) – (0.5 x Noise) How can we do better than this?



Size of Signals and Volume

Multiplying Signals by a Constant

If

Mic1 = (0.4 x Speech) + (0.6 x Noise),

then

2 x Mic1 = (0.8 x Speech) + (1.2 x Noise)

would be “twice as loud” as Mic1.



Separating the Speech Signal from Noise

Combine the ideas of adding and subtracting signals with changing the volumes of signals.Suppose we take 0.9 times Mic2 and subtracted it from Mic1. We would get

Mic1 - (0.9 x Mic2) = (0.4 x Speech) + (0.6 x Noise) - 0.9 x [(0.3 x Speech) + (0.7 x Noise)]

= (0.13 x Speech) - (0.03 x Noise)The speech is now more than 4 times louder than the

noise (0.13/0.03) = 4.333…



Separating the Speech Signal from Noise

Refining the result (advanced):Suppose we take 0.86 times Mic2 and subtracted it from Mic1. We would get

Mic1 - (0.86 x Mic2) = (0.4 x Speech) + (0.6 x Noise) - 0.86 x [(0.3 x Speech) + (0.7 x Noise)]

= (0.4 x Speech) - (0.258 x Speech) + (0.6 x Noise) - (0.602 x Noise)

= (0.142 x Speech) - (0.002 x Noise) The speech is now 71 times louder than the noise.

(0.142/0.002 = 71)



Separating Speech and Noise in the Lab

Designing the system - two ways1. Using calculations as we have up to now

2. Turn the number in front of Mic2 into a control and playing with the control until the noise goes away.

Let’s try the second method using the LabVIEW labs:

01ListenToOriginalSpeechandNoise

02ListenToMixedSpeechandNoise

03SeparateSpeechandNoiseSlider



01ListenToOriginalSpeechandNoise



02ListenToMixedSpeechandNoise






Worksheet Exercise 1.1

For the following exercises, letMic1 = (0.2 x Speech) + (0.8 x Noise)Mic2 = (0.3 x Speech) + (0.7 x Noise)

Calculate the following signals.1. Mic1 + Mic2 = (_________ x Speech) ____ (_________ x Noise)2. Mic1 - Mic2 = (_________ x Speech) ____ (_________ x Noise)3. (2 x Mic1) = (_________ x Speech) ____ (_________ x Noise)4. (0.7 x Mic1) = (_________ x Speech) ____ (_________ x Noise)5. (1.5 x Mic2) = (_________ x Speech) ____ (_________ x Noise)Question: In which of the above cases is (a) Speech louder than

Noise? (b) Speech softer than Noise? (c) Speech and Noise of the same volume?

.




For the following microphone signal pairs, find the volume needed forthe Mic2 signal that, when subtracted from the Mic1 signal, removesthe noise signal completely.1. Mic1 = (0.3 x Speech) + (0.7 x Noise)

Mic2 = (0.5 x Speech) + (0.5 x Noise)2. Mic1 = (0.6 x Speech) + (0.4 x Noise)

Mic2 = (0.2 x Speech) + (0.8 x Noise)

3. Mic1 = (0.1 x Speech) + (0.4 x Noise)Mic2 = (0.5 x Speech) + (0.6 x Noise)




Separating Speech and Noise Using Math

Recall: Designing the system - two ways1. Using calculations

2. Playing with the control on Mic2 until the noise is gone.

Let’s try the first method using the numbers we’ve just calculated using the lab:







1.3 Designing a Two-Talker Speech Separation System

• Suppose we have two speech sounds Speech1 and Speech2, and we record two different mixtures.

• How does the problem change?– Most important: Which speech signal is the one we want to

hear?– Maybe we need a system that has two outputs:

• Separated Speech1• Separated Speech2

– So, we need a system that calculates two different combinations of Mic1 and Mic2

– We will choose later which sound to which we will listen. – For now, we’ll use a single combination and change the slider

to hear the talker we want.



Differences Between Noise and Speech

• A lot of noises are always “on”, making sound every second

• But speech is impulsive or intermittent. Sometimes there’s no talker speaking. How does this fact change the design process?– It doesn’t change anything about the math, but it

does make the “playing with the slider” design method harder. Try it with the labs:

04ListenToOriginalSpeech205ListenToMixedSpeech206SeparateSpeech2Slider



04ListenToOriginalSpeech2



05ListenToMixedSpeech2



06SeparateSpeech2Slider



The Math Behind Two-Talker Speech Separation

• Suppose Mic1 = (0.4 x Speech1) + (0.6 x Speech2) Mic2 = (0.5 x Speech1) + (0.5 x Speech2)

• Separating out Speech1Mic1 - (1.2 x Mic2) = (0.4 x Speech1) + (0.6 x Speech2)

- 1.2 x [(0.5 x Speech1) + (0.5 x Speech2)]= (0.4 - (1.2 x 0.5)) x Speech1 + (0.6 - (1.2 x 0.5)) x Speech2= - (0.2 x Speech1) + (0.0 x Speech 2)= - (0.2 x Speech1)



The Math Behind Two-Talker Speech Separation

• Separating out Speech2Mic1 - (0.8 x Mic2) = (0.4 x Speech1) + (0.6 x Speech2)

- 0.8 x [(0.5 x Speech1) + (0.5 x Speech2)]= (0.4 - (0.8 x 0.5)) x Speech1 + (0.6 - (0.8 x 0.5)) x Speech2= (0.0 x Speech1) + (0.2 x Speech 2)= (0.2 x Speech2)

• Important Idea: The two separation systems are identical except for the value multiplying Mic2.

• Aside: Engineers re-use system designs all the time - so long as they own them (i.e. patents)




For the following microphone signal pairs, a) Find the volume needed for the Mic2 signal that, when subtracted

from the Mic1 signal, removes Speech2 completely.b) Find the volume needed for the Mic2 signal that, when subtracted

from the Mic1 signal, removes Speech2 completely.





Designing a Two-Talker Speech Separation System in the Lab

• Now, we will use our math skills to build a two-talker speech separation system without sliders.

• Our goal: For the two mixtures, – Calculate the right amount of Mic2 in two ways so

that we get Speech1 alone and Speech2 alone. – We’ll use a button control to toggle between the two

outputs.

07SeparateSpeech2System



07SeparateSpeech2System



Important Exercises for Lab 8

• Make sure you have the students type in different amounts of Speech1 and Speech2 into the mixtures (to simulate different room conditions)

• Have them calculate what the right amounts of Mic2 should be to get Speech1 and Speech2 at the output of the system

• Discuss with them some of the problems– Sometimes, the volume of the output is too low to hear - need

to “turn up the volume” after separation– Is there ever a situation where separation isn’t possible? Yes,

if Mic1 and Mic2 have identical percentages of Speech1 and Speech2 in them. Then, they are the same microphone!



Epilogue: Spatial Audio

• When watching movies with surround sound, the sound “moves” with the visual action all around the room.

• A sound designer is someone who creates the multichannel audio signals to make this sound movement happen.

• How does the sound designer design these sounds? Using math, of course!– Two math tools: Volume (what we have been

using) and Delay (time shift - see Unit 2).



Spatial Audio

(1.0 x Speech1) (0.0 x Speech 1)+ (0.0 x Speech2) + (1.0 x

Speech2)

(0.5 x Speech1) (0.5 x Speech 1)+ (0.2 x Speech2) + (0.8 x

Speech2)

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.






are needed to see this p icture.




are needed to see this p icture.


are needed to see this picture. QuickTime™ and aTIFF (LZW) decompressor




Unit 2: Echo, Reverberation and Sound Effects



2.1 The Speed of Sound

Sound does not travel (nearly) instantaneously like light does. It takes time for sound to go from one place to another.

If air were visible, here’s what a sound wave might look like “frozen” in time:

Light areas: Not many air particles

Dark areas: Lots of air particles

Movement: Moves to right over time - like ripples in a pond



Movement of Sound Waves

Sound waves are caused by changes in air density - the number of air particles in a particular space

These are very small changes - ears hears sound well!

Sound movement is similar to wave movement in a spring.

(Look at each row of the figure as a frame of a movie going upwards.)



Traveling Wave in a Spring















Push

Here->



Speed of Sound in Different Materials

Medium Meters/sec Feet/sec Miles/hour

• Air 331 1086 740

• Hydrogen 1270 4167 2841

• Carbon Dioxide 258 846 577

• Water 1450 4757 3244

• Glass 5500 18000 12300

• Granite 6000 19700 13500

UNITS: Distance/Time (“distance over time”)



How to Measure the Speed of Sound in Air

Place yourself far away from a large, flat object, like the side of a building.

Clap your hands, and start a stopwatch. Wait until you hear the sound come back - the echo. Since the distance traveled is twice the distance from you to the

wall, calculate

Speed of Sound = (2 x Distance to Wall)/(Time Traveled) Problems: (1) It is hard to start and stop the stopwatch

accurately. (2) There may be other reflections from other buildings.

Can’t we do this better with a microphone?




A. For each of the distances, calculate how long it would take for a sound of a handclap to travel from your hands to a wall that is the distance away and back again.1. 5 meters2. 300 feet3. 0.25 miles

B. Suppose you are 100 feet away from a large building. It takes 185 milliseconds for an echo of a handclap sound to return to you. What is your measurement of the speed of sound?

C. How far away from a wall would you have to be to create an echo in air with the following time delays?1. 0.1 seconds2. 1.1 seconds3. 5 seconds



2.2 Delay and Amplitude of an Echo

When sound bounces of a surface and comes back to us in a distinct reflection that we can hear, we call the sound an echo.

Two numbers describe this echo:1. Echo Delay - the amount of time it takes for the

echo to get to us.2. Echo Amplitude - the size of the echo signal

relative to the original sound that produced it. We can measure the delay and the amplitude

of an echo in LabVIEW.



Plot of Original Sound



Plot of Original Sound With Echo



Calculating the Echo Delay

1. Find some easily recognized point of the first part of the sound, like its largest value or when the sound first starts.

2. Determine the time that this point occurs. For example, the start of the first part of the sound happens at time T_start = 0.52 seconds.

3. Find the time where this feature repeats. For example, when the sound repeats, the start of the repeated sound happens at time T_repeat = 0.87 seconds.

4. Subtract the second value from the first one. In this case, 0.87 – 0.52 = 0.35 seconds. This is the echo delay.



Calculating the Echo Delay Using A Transparency

1. Put the transparency over the graph. Draw a straight line for the zero value of the graph.

2. For the first part of the sound, trace out its shape on the transparency.

3. Mark the axis to the left of some feature of this sound using regular units (such as every 0.1 seconds).

4. Now, slide the transparency over so that the trace that you just drew falls on top of the second part of the sound--the part that repeats.

5. Look at the first part of the sound. Your markings should now tell you exactly what the time delay is--in this case, 0.35 seconds.



Echo Amplitude

When sound bounces off of something, not all of it is reflected back. Some of the sound is lost (absorbed).

The echo signal is softer and therefore smaller in amplitude (or height on a graph) than the original sound.

We can calculate the amount of the echo amplitude as

Echo Amplitude = (Volume of Reflected Signal) /

(Volume of Original Signal)

Question 1: What is the Echo Amplitude of the echo in the previous plot?

Question 2: How did you calculate it?



Measuring Echo Delay and Echo Amplitude in the Lab

Let’s use LabVIEW to simulate echoes and measure their amplitudes and delays09SimplePCEchoDelay

• Use the Esc key to stop the worksheet.• It takes some timing, but it should be possible

to stop the worksheet when both the original sound and the echo are on the same graph.

• Calculate the delay and amplitude using the methods we’ve described.



08SimplePCEchoDelay




A. The following T_start and T_repeat times are measured for a sound with echo. Find the echo delay time.

1) T_start = 0.22 seconds. T_repeat = 0.32 seconds. T_delay = ______

2) T_start = 0.37 seconds. T_repeat = 0.4 seconds.

T_delay = ______

B. The following volumes are measured for a sound with echo. Find the amplitude of the echo.

1) Volume of Original Signal = 0.7, Volume of Reflected Signal = 0.4

Echo Amplitude = ______

2) Volume of Original Signal = 1.5, Volume of Reflected Signal = 1.32

Echo Amplitude = ______



Worksheet Exercise 2.2 (cont.)C. For the following signal, measure and calculate the echo delay and the echo

amplitude.

T_start = _____ T_repeat = _____ T_delay =_____

Volume of Original Signal = _____ Volume of Reflected Signal =_____

Echo Amplitude =_____



2.3 Recreating Echo

Most sounds that are made for music and movies have echo that is artificially-made.

The math tools for this are identical to those we have used in the labs to simulate echo.

We can use our knowledge of physical space to calculate what the delay ought to be for a particular space.

• Example: Sports Stadium: 300 m from one side to anothero What should be the echo delay in seconds?o How do you calculate it?

09SimplePCEchoSpeech10SimplePCEchoStadium



09SimplePCEchoSpeech



10SimplePCEchoStadium



2.4 Reverberation - Repeating Echo

When sound bounces off of two opposing walls, a repeating echo is created. The sound slowly decays away as the amplitude decreases after each bounce.

When the distances are short, we don’t hear the distinct echoes, but we still do hear a change - the sound is somehow better to us.

This effect is called reverberation. We can use LabVIEW to simulate reverberation

effects. This takes more number-crunching, so we will use the DSP board.

11DSPReverberation



11DSPReverberation



Hints on Audio Laboratories

You might want to demonstrate the lab to the students before having them use it themselves.

Headphones are really helpful here. Ask questions of the students to test their

understanding of what is going on. Their tendency will be to “just play with the controls,” so get them back to calculating numbers to verify what they see and hear.



System Design: The Gosney Speaker



Speakers and Microphones

• Transducer : something that converts energy from one form to another– Speaker : electricity sound (acoustic)– Microphone : sound electricity

• Most inexpensive transducers use electricity and/or magnetism to work.



Loudspeaker Parts

• Cut-away view of a typical loudspeaker



The Physics of Loudspeakers

• Force = (Magnetic Flux) x (Current) – causes the wire to

move with small changes in electricity

• To get a higher force, we turn the wire into a coil

• Then, we attach a rigid cone to the coil

Magnetic Flux B

Current I

B

I

S

S

N



The Gosney Speaker

• Let’s Build This!

Plastic Salad Bowl

Wires

Coil

Button Magnet

Tab that attaches cone to bowl

Paper Cone



Wrap-Up Discussion

We have covered the basics of the sound engineering curriculum

The key idea: Let the application motivate the math and science• Your students will want to make things happen • The key is to show them that they can achieve what

they want with technology if they understand it well• Push the responsibility of the design to them as much

as possible so that they own the design



Thank You!• Universities and Organizations Involved in the Infinity Project

– Created by Institute for Engineering Education at SMU– Other participating universities include Georgia Institute of Technology, Rose-

Hulman Institute of Technology, Rice University, Texas A&M University, University of Texas at Austin, University of Texas at El Paso, Prairie View A&M, University of Michigan, Santa Clara, University of Houston, and many others.

• Corporate – Government Sponsors and Entities– Texas Instruments– Department of Education – Communities Foundation of Texas– National Instruments– National Science Foundation– Tyco, Compaq, Microsoft– And others

Documents

Infinity-project.org © 2005-2008 The Institute for Engineering Education at SMU. All Rights Reserved Engineering Education for today’s classroom. Unit