Embed Size (px)
Citation preview
THE YEAR OF VOICE
LIBOR FX ScandalBanks face
Multi-Billion $ finesAmazon Alexa SIRI(?)
As almost 50% of all corporate data will have a voice component within 5 years, either as audio or video, all companies, but particularly banks and insurance companies, need to get a handle not just on where this data is being held, but what is being said in it, and also who is saying it.
20152016?2017!
AUDIENCE PARTICIPATION
HOW OFTEN DO YOU USE A VOICE ASSISTANT?
0% 10% 20% 30% 40% 50% 60% 70% 80% 90%
Daily
Weekly
Monthly
Never
Results taken from a survey on 5th October 2017 of 1500 people across Europe
Of the people with a smart phone how many use their integrated voice assistant (e.g. Siri, Cortana):
HOW OFTEN DO YOU USE A VOICE ASSISTANT?
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50%
Daily
Weekly
Monthly
Results taken from a survey on 5th October 2017 of 1500 people across Europe
Of the people with an Alexa home assistant how often do they use it:
IT’S A DOUBLE WHAMMY
Where?
GDPRMiFID II
What?Who?
CLOUD SECURITY
Where is your voice stored?
Your voice could be used for any number of the following:
Use (edit) your voice recordings to impersonate you
Learn about you
→ Your identity, gender, nationality (accent), emotional state..
Track you from uploads / communications of voice recordings
WHERE
ENCRYPTED SPEECH PROCESSING
Privacy preserving encrypted phonetic search of speech dataC Glackin, G Chollet, N Dugan, N Cannings, J Wall, S Tahir, IG RayIEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2017.
A New Secure and Lightweight Searchable Encryption Scheme over Encrypted Cloud DataS Tahir, S Ruj, Y Rahulamathavan, M Rajarajan, C GlackinIEEE Transactions on Emerging Topics in Computing, 2017.
AES Encryption (Public key)Powered by machine learning
Powered by GPU
WHERE
DEEP SPIKING NEURAL NETWORKS FOR SPEECH ENHANCEMENT
Recurrent lateral inhibitory spiking networks for speech enhancementJ Wall, C Glackin, N Cannings, G Chollet, N DuganInternational Joint Conference on Neural Networks (IJCNN), pp. 1023-1028, 2016.
TECHNICAL
CONVOLUTIONAL NEURAL NETWORKS FOR ACOUSTIC MODELLING
TIMIT Speech Corpus
1.4M spectrograms for the training set
Sliding window used for timing
4 to 5 phones in each 0.256 second window
61 Phoneme Classes ?
- Beaten the current NTIMIT. State of the art! - Beaten the current NTIMIT. State of the art! - Beaten the current NTIMIT. State of the
TECHNICAL
TECHNICAL
HOW FAST?
10
30
50
80
100
0
20
40
60
80
100
120
Times Real Time
WHAT
UNDERSTANDING
100x Realtime using P5000
WHAT
TELEFONICA/O2
But this is just the beginning: Voice data is generated not only in the organisation, but externally, maybe as YouTube content.
One area commonly forgotten is mobile telephony. MiFID II now places a strong requirement not just on recording calls made from a regulated organisation premises, but their mobile calls as well.
Intelligent Voice are working with Telefonica/O2 to capture, index and analyse mobile phone calls, and introduce them as part of a compliance and monitoring workflow for MiFID II .
WHAT
CREDIBILITYWHAT IS WRONG WITH THESE STATEMENTS?
“Woke up at 7:30. Had a shower. Made breakfast and read the newspaper. At 8:30, drove to work.”
“We should have done a better job.”
“That’s their way of doing things.”
“You’d better ask them.”
Alleged robbery victim: “The man asked for my money.”
“He told me not to look at him. He said he would shoot me if I screamed.”
WHAT
CREDIBILITY INDICATORSPronouns: Omission, Improper use, Higher rates of third person plural pronounced person plural
pronounsComplexity: Parameters such as number of letters/syllables per word, higher word count, higher
rate of pausesSpeaking verbs: Strong tone (told, demanded, telling), soft tone (said, asked, stated, saying) – tone
changesTempo: Slow tempo (indicator of cognitive load), fast tempo (indicator of arousal and
negative effects)Pitch: Higher pitch/lower voice quality at specific times are indications of fraudulent related
utterancesSpecific Words: Explainers (so, since therefore, because…)
These are just a few of the indicators of suspicious language
WHAT
CREDIBILITY NETWORKVoice Activity
Detection
i-vector diarization
What happened next?
He told me not to look at him. He said he would shoot me if…
INTERVIEWER
CALLER
… He told me not to look at him . He said … EmbeddingLSTM
LSTM
Strong tone
Weak tone
followed by
Inspired by recurrent networks for named entity recognition and part of speech taggingWe can use bi-directional recurrent networks to attach credibility tags to the speech transcriptionBi-directionality is important for contextNetwork can tag explainers, changes in tone, pronouns etc.
GPU-accelerated RNN-based
Speech to Text
WHAT
SPEAKER IDENTIFICATION
RASTA SOX MATLAB PYTHON RASTA 12
Dialect identification via images and DIGITSNIST evaluation of 500 hours and 20 dialects
WHO
NIST EVALUATION
Preliminary Results
0 50 100
English-Portuguese-Brazilian
Spanish-Spanish-European
Chinese-Min_DongArabic
Chinese-CantoneseArabic-Egyptian
English-BritishSpanish-Caribbean
Slavic-RussianArabic-Maghrebi
Chinese-MandarinArabic-Iraqi
English-American
Chinese-WuSlavic-Polish
French-Haitian
Arabic-Leventine
French-West_African
WHO
WHO
CELEBRITY SOUND A LIKE
https://celebsoundalike.com/
Tweet your results to @intelligentvox
WHO
