Computer Science & MusicSean Bechhofer

University of Manchester !

@seanbechhofer !

Animation 14 Awards Day, July 2014

Computer Science isn't just about

Computers

Name That Tune!

Name That Tune!

Name That Tune!

Name That Tune!

Name That Tune!

Track 1

Track 2

Cool! How does it work?

How would you describe

me?

Tall

48

48

Scottish

Tall

Tall

48

Scottish

Computer Scientist

48

Scottish

Computer Scientist

Guitarist

Tall

48

Scottish

Computer Scientist

Guitarist

Tall

Manchester (City)

Tall

48

Scottish

Computer Scientist

Guitarist

Scuba DiverManchester (City)

48

Scottish

Computer Scientist

Guitarist

Tall

Manchester (City)Scuba Diver

Tall48ScottishComputer ScientistGuitarist

Scuba Diver

Features

Manchester (City)

These features allow us to identify people. We can do the same with sound or music

Feature ExtractionWe can analyse a piece of music to find out

what sounds are happening at different times

!

Low frequencies: e.g. Bass Mid frequencies: e.g. Vocals

High frequencies: e.g. Cymbals This gives us a Spectrogram

Signature

Computer Music Files

mp3?

“Motion Picture Experts Group Phase

1 Audio Layer III”

Let's stick with mp3...

Sound = Waves

Turning waves into numbers

-4, 0, 4, 7, 6, 3, 0, -1, 1, 3, 4, 3, -1, -5, -8, -7, -5, -1, 0, -2, -4, -5, -4

Numbers!

0000000 49 44 33 03 00 00 00 04 1b 55 54 4c 45 4e 00 00!

0000010 00 0f 40 00 01 ff fe 33 00 30 00 35 00 30 00 31!

0000020 00 33 00 41 50 49 43 00 01 02 9e 00 00 00 69 6d!

0000030 61 67 65 2f 6a 70 65 67 00 03 00 ff d8 ff e0 00!

0000040 10 4a 46 49 46 00 01 01 00 00 01 00 01 00 00 ff!

0000050 db 00 43 00 05 03 04 04 04 03 05 04 04 04 05 05!

0000060 05 06 07 0c 08 07 07 07 07 0f 0b 0b 09 0c 11 0f!

0000070 12 12 11 0f 11 11 13 16 1c 17 13 14 1a 15 11 11!

0000080 18 21 18 1a 1d 1d 1f 1f 1f 13 17 22 24 22 1e 24!

0000090 1c 1e 1f 1e ff db 00 43 01 05 05 05 07 06 07 0e!

00000a0 08 08 0e 1e 14 11 14 1e 1e 1e 1e 1e 1e 1e 1e 1e!

Lots of Numbers!

Over 44,000 samples per second for high quality audio

That's 31.5 MB for a 3-minute song Only 250 songs on an 8Gb music

player.

Lots of Numbers!

Over 44,000 samples per second for high quality audio

That's 31.5 MB for a 3-minute song Only 250 songs on an 8Gb music

player.

Throw some numbers away!

High Quality

Low Quality

What's in the picture?

Another volunteer!

Lossy Encoding

My nm s Sn nd m Cmptr Scntst t th nvrsty f Mnchstr. !

lk t ply th gtr. lk t g scb dvng nd shrks r my fvrt nmls.

111 letters

Lossy Encoding

My name is Sean and I'm a Computer Scientist at the University of Manchester. !

I like to play the guitar. I like to go scuba diving and sharks are my favourite animals.

170 letters

Lossy Encoding

Original = 170 letters, New = 111 letters !

65% smaller! !

Imagine if we had to pay £1 per letter! !So this is “cheaper”, but we might need

some extra effort to read it

Lossy Encoding

ae i ea a I a oue iei a e Uiei o aee. I ie o a e uia. I ie o o ua ii a a ae aouie aia.

53 letters

Lossy Encoding

My name is Sean and I'm a Computer Scientist at the University of Manchester. !

I like to play the guitar. I like to go scuba diving and sharks are my favourite animals.

170 letters

Good

FastCheap

Trade Offs: Pick Any Two!

Everyday Encoding

r u gng to the gig?

I <3 daft punk!

not if i c u first :-)

yeah shld be gr8

me 2! c u l8r

lol ;-)

How high can you hear?

How high?

1.Everybody stand up! 2.You will hear a tone play. 3.When you can no longer hear the tone, sit down!

This is an experiment, not a competition — no cheating!

Spot the Difference!

Perceptual Encoding

Fewer Numbers

0000000 49 44 33 03 00 00 00 04 1b 55 54 4c 45 4e 00 00!

0000010 00 0f 40 00 01 ff fe 33 00 30 00 35 00 30 00 31!

0000020 00 33 00 41 50 49 43 00 01 02 9e 00 00 00 69 6d!

0000030 61 67 65 2f 6a 70 65 67 00 03 00 ff d8 ff e0 00!

0000040 10 4a 46 49 46 00 01 01 00 00 01 00 01 00 00 ff!

0000050 db 00 43 00 05 03 04 04 04 03 05 04 04 04 05 05!

0000060 05 06 07 0c 08 07 07 07 07 0f 0b 0b 09 0c 11 0f!

0000070 12 12 11 0f 11 11 13 16 1c 17 13 14 1a 15 11 11!

0000080 18 21 18 1a 1d 1d 1f 1f 1f 13 17 22 24 22 1e 24!

0000090 1c 1e 1f 1e ff db 00 43 01 05 05 05 07 06 07 0e!

00000a0 08 08 0e 1e 14 11 14 1e 1e 1e 1e 1e 1e 1e 1e 1e!

High Quality

mp3

“It's hard to make predictions, particularly about the future.”

Niels Bohr (1885-1962)

Mark Zuckerberg

2004

2005

2006

2007

2008

2009

2010

2011

2012

2013

0 350 700 1050 1400

1230

1056

845

608

360

145

58

12

6

1

Million Monthly users

If you liked that, you'll

like this!

Listens

Popularity

One Direction

Metallica

Daft Punk

Popularity

One Direction

Metallica

Daft Punk 3.4 million listeners

2.5 million listeners

0.7 million listeners

Popularity

One Direction

Metallica

Daft Punk

233 million listens 2.5 million listeners173 million listens 3.4 million listeners36 million listens 0.7 million listeners

?

This works for lots of stuff!

Inside &

Outside

Mona Lisa Leonardo da Vinci

1503-1505 Louvre Museum

Who's like me?

Friends?

Patterns

Patterns

Patterns

Computer Science isn't just about Computers

We can't predict what the impact of new technologies will be

Sharks are great!

Eat lots of broccoli!

Things to Remember

Thank you!

The best way to predict the future is to invent it

— Alan Kay

Image Credits• Daft Punk: https://www.flickr.com/photos/caesarsebastian/1031975612/ • Red Army: https://www.flickr.com/photos/patsemchism/345981835 • Constellation: http://commons.wikimedia.org/wiki/File:Ursa_major_constellation_map.png • Great Wave: Public domain, Katsushika Hokusai (葛飾北斎)

• Etihad: http://en.wikipedia.org/wiki/File:City_Of_Manc_Stadium.jpg • Hammerhead Shark: https://www.flickr.com/photos/barrypeters/4242623174/ • Oceanic White Tip: https://www.flickr.com/photos/michaelaston/2062803551 • Ear: https://www.flickr.com/photos/tbisaacs/3911558890/ • Fraunhofer: http://www.mp3-history.com/ • Niels Bohr: http://www.clipartsfree.net/clipart/89-niels-bohr-clipart.html • Mark Zuckerberg: http://www.fotopedia.com/items/flickr-2326586665 • Cassette Tapes: https://www.flickr.com/photos/cass_ette/4571540478 • One Direction: http://en.wikipedia.org/wiki/File:One_Direction_at_the_Logies_Awards_2012.jpg • Metallica: http://en.wikipedia.org/wiki/File:Metallica_London_2008-09-15_Kirk_and_James.jpg • CD Rack: http://en.wikipedia.org/wiki/File:CD_rack.JPG • Books: https://www.flickr.com/photos/meeli/2854849909 • Beans: http://en.wikipedia.org/wiki/File:No_name_sans_nom_beans_feves.jpg • Video Games: http://commons.wikimedia.org/wiki/File:Videogameretaildisplay.jpg • Broccoli: http://commons.wikimedia.org/wiki/File:Basket_of_broccoli_in_Singapore_market.jpg • Mona Lisa: Public Domain • Other Images: Sean Bechhofer