28
Unlocking Data Trapped in Audio & Video Files Paul Murphy [email protected] @prmurphy

Unlocking Data Trapped in Audio & Video Files

Embed Size (px)

Citation preview

Page 1: Unlocking Data Trapped in  Audio & Video Files

Unlocking Data Trapped in

Audio & Video Files

Paul [email protected]@prmurphy

Page 2: Unlocking Data Trapped in  Audio & Video Files
Page 3: Unlocking Data Trapped in  Audio & Video Files

Córdoba, Argentina

Page 4: Unlocking Data Trapped in  Audio & Video Files

Milan, ItalyParis, France

Page 5: Unlocking Data Trapped in  Audio & Video Files

London, UK

Page 6: Unlocking Data Trapped in  Audio & Video Files

So what’s wrong?

Page 7: Unlocking Data Trapped in  Audio & Video Files

Context

• My accent• The way I look• Bio (maybe)

Page 8: Unlocking Data Trapped in  Audio & Video Files

My Background (why am I here?)

Banking » Telephony » Analytics

When things important, they’re oral

Oral means no trace

Except that that’s no longer true

Page 9: Unlocking Data Trapped in  Audio & Video Files

Why am I excited?

• It’s a hard problem• It has to be solved (trends!)• I founded a company to do that

Page 10: Unlocking Data Trapped in  Audio & Video Files

Trends

• End of the Gutenberg Pause• Humans communicate through sound &

images• Text is an optimization • Text is dead

• Massive amounts of data are now being stored in A/V files

Page 11: Unlocking Data Trapped in  Audio & Video Files

Where are we today?

• Lots of tools for manipulating text• Almost no tools for manipulating

audio & video

Page 12: Unlocking Data Trapped in  Audio & Video Files

Where are we today?

• Lots of tools for manipulating text• Almost no tools for manipulating

audio & video

Page 13: Unlocking Data Trapped in  Audio & Video Files

How can we compute on A/V?

• Transcription• Annotation

Page 14: Unlocking Data Trapped in  Audio & Video Files

Turks are great!

• Human transcription (APIs)• Manual annotation

Page 15: Unlocking Data Trapped in  Audio & Video Files

Can those be automated?

• Transcription• Annotation

» ASR» Artificial vision

Of course! (or I wouldn’t be here)

Page 16: Unlocking Data Trapped in  Audio & Video Files

ASRs

• AT&T• IBM Watson• Vocapia• Speechmatics• …

Page 17: Unlocking Data Trapped in  Audio & Video Files

Vision

• Clarifai• Orbeus• Image Vision Labs• Face++• …

Page 18: Unlocking Data Trapped in  Audio & Video Files

State of the Art

• Last 4 slides

Page 19: Unlocking Data Trapped in  Audio & Video Files

Context (input)

• Telephony vs. wideband• Conversation vs. voicemail• Music vs. speech• English vs. Spanish

Better & better data

Page 20: Unlocking Data Trapped in  Audio & Video Files

Context (output)

• Gender?• Identity?• Relationship?• Emotion?• Location?

Need audio.

Transcripts & annotations aren’t enough.

Page 21: Unlocking Data Trapped in  Audio & Video Files

Recap…why does context matter?

• Context helps us extract more & better data

• Context is data

We compute on data

Page 22: Unlocking Data Trapped in  Audio & Video Files

Recap…where are we today

• Trends• Technology• Context

Page 23: Unlocking Data Trapped in  Audio & Video Files

Recap…where are we today

Page 24: Unlocking Data Trapped in  Audio & Video Files

Recap…where are we today

Page 25: Unlocking Data Trapped in  Audio & Video Files

Recap…where are we today

Page 26: Unlocking Data Trapped in  Audio & Video Files

Recap…where are we today

Page 27: Unlocking Data Trapped in  Audio & Video Files

Paul [email protected]@prmurphy

Thank you!

Page 28: Unlocking Data Trapped in  Audio & Video Files

Any Questions?PS. We’re hiring!

Paul [email protected]@prmurphy