Upload
tashya-jimenez
View
45
Download
4
Embed Size (px)
DESCRIPTION
Audio design for interactive systems. Camille Goudeseune Integrated Systems Laboratory, Beckman Institute. Outline. Sound libraries music instruments Digital Audio music theory Roles of sound composition. high-level multiplatform (linux+windows at least) sound libraries. Java Sound API - PowerPoint PPT Presentation
Citation preview
Audio design forinteractive systems
Camille Goudeseune
Integrated Systems Laboratory,
Beckman Institute
Outline
• Sound libraries– music instruments
• Digital Audio– music theory
• Roles of sound– composition
high-level multiplatform (linux+windows at least) sound
libraries• Java Sound API
• VSS
• FMOD
• SDL
• SEAL
• Housemarque
Java Sound API
• Painless for Java. Very basic. Avoid if you're ambitious. http://java.sun.com/products/java-media/sound/
VSS Virtual Sound Server
• www.isl.uiuc.edu/software/software.html
• the 500-pound gorilla if you need stuff like 8-channel output or linux/irix/windows compatibility
• steep learning curve for advanced stuff
FMOD
• Syzygy (the Cube) uses it for sound.
• CPU-miserly
• finite but featureful API
• www.fmod.org
SDL
• large user community
• open source.
• Good if you want to integrate sound and graphics tightly.
• www.libsdl.org
SEAL Synthetic Audio Library
• many C compilers supported
• Wide support for hardware acceleration
• www.sonicspot.com/sealsdk/sealsdk.html
Housemarque
• Multichannel with certain PC soundcards
• www.s2.org/hmqaudio/
low-level sound libraries: linux
• OSS, www.linux.org.uk/OSS/– Most distros include the free basic version– ~$30 for fancy multichannel soundcard drivers
• ALSA, www.alsa-project.org – religiously open-source alternative
• 0.5.10 is the stable version• 0.9.0 development version is fine too
(part of the upcoming 2.5 linux kernel).
low-level sound libraries: windows
• MMIO– simple, universal– waveOutGetDevCaps(), waveOutWrite(), ...
• DirectSound – part of DirectX– LPDIRECTSOUNDBUFFER,
CreateSoundBuffer(), Lock(), Unlock(), ...– “faster” but more awkward; use a wrapper
Basics of Digital Audio
• File formats• 16-bit or 8-bit?
– These days, 8-bit is embarrassing.Pro gear uses 32!
• Stereo or mono?– Panning mono is faster + simpler than stereo.
• 8 kHz? 44 kHz?• .WAV, .AIFF, .AU, .MP3• converters: “sox” and dozens of others
Basics of Digital Audio
• Two tradeoffs
CompressionRatio
CPU Load
Sound Quality
Latency
CPU Load
Basics of Digital Audio
• Debugging
• Gaps in sound excessive CPU use
• stuttering CPU starvation(CPU fast enough but poorly scheduled)
• Different from graphics! The “frames per second” can’t degrade if the CPU is taxed.
Basics of Digital Audio
• Debugging
• Electric-guitar distortion clippingToo quiet: hissToo loud: clippingJust right: almost clipping– For every stage in the audio pipeline,
both software and hardware.Every place you can set the volume level!
Where to get sounds
• Buy: fx + music libraries
• Build: record it yourself
• Build: synthesize it yourself– adjust an existing synth patch, a little or a lot
• Steal: websearch– 8-bit might still suffice while prototyping
What to do with the sounds,once you have them
• common roles– alerts; acknowledgements; ambience
• sound vs. image
• speech vs. non-speech
• synchronization with visual events
• combining sounds
• tips for spatializing
Short and subtle is best!
• “Graphical excellence gives to the viewerthe greatest number of ideasin the shortest timewith the least inkin the smallest space.”
(Edward Tufte, The Visual Display of
Quantitative Information) • This applies to alerts, acks, and ambience.• Clutter is worse with sound: no earlids!
Sound or image for a message?
• Use sound if:• Simple• Short• Standalone• Temporal• Demands immediate
action
• Use visuals if:• Complex• Long• Referred to later• Spatial• Can deal with it later
Speech vs. non-speech
• Main advantage: precise, simple! But...
• Carries extra connotations
• Tone of voice, imprecise word choice, can confuse or distract
• Non-speech doesn’t interfere with conversation
Synchronizing to other things
• Often 100 msec is accurate enough,when passively observing a sonic anda visual event
• Sync with input action might needto be tighter, 30 or even only 4 msec.
Moving a probe through a dataset
• Trigger sounds on button-click, or when crossing thresholds
• Or, play a continuous sound.
• Vector-valued data is trickier.
• Nominal data: one sound per name.
• Scope of probe: point, ball, shell.– Auto-size based on probe speed.– Granulation
Navigating a VR world
• Dense world– azimuth: click rate varies with turning rate– altitude: high/low beeps
(rate varies with climb rate)– speed: vehicle-engine metaphor
• Large world: quiet continuous ambiences localized to individual parts of the world.– Play only the nearest two or three.
Combining sounds
• It can get only so loud before clipping.
• Spatial separation (“panning”).
• Temporal separation (one sound at a time).– A party of drunks / a good dinner conversation.
• Frequency separation.
• Each layer gets its own tempo.– Heavily layered techno or orchestral music.
Spatialized sound
• Steady tones are worst.Bird chirps are best (they should know).– wide frequency band, complex attack
• Loudspeaker array
• Headphones with HRTF
• Motion-tracked headphones– In the real world you move your head slightly to
tell where a sound comes from.