04/22/2304/22/23 11

Speech Database/Tool SystemSpeech Database/Tool SystemAnd Preliminary Accent study.And Preliminary Accent study.

Dr. Charles Tappert a.k.a (Project Dr. Charles Tappert a.k.a (Project Manager)Manager)

Arthur Phidd, DPS a.k.a (The Client)Arthur Phidd, DPS a.k.a (The Client)Padmashree ThimmappaPadmashree Thimmappa

Shankar VijayakumarShankar VijayakumarRichard SautherRichard Sauther

May 6th 2005

04/22/2304/22/23 22

OverviewOverview

Create a tool & database for collecting speech Create a tool & database for collecting speech samples and data mining processing.samples and data mining processing.

Ability to record new voice files.Ability to record new voice files. Upload these files on to the server.Upload these files on to the server. Retrieve these files when needed.Retrieve these files when needed. Play the files.Play the files. Analyze the files using Pronunciation Affinity Matrix Analyze the files using Pronunciation Affinity Matrix

(PAM) to determine the possible accent.(PAM) to determine the possible accent. Analyze the files for further research using Analyze the files for further research using

available Speech Filing System (SFS) to decompose available Speech Filing System (SFS) to decompose spectrograms into data elements for data miningspectrograms into data elements for data mining..

04/22/2304/22/23 33

Specification of Spectrographic ToolsSpecification of Spectrographic Tools

Ability to perform spectral analysis of the speech Ability to perform spectral analysis of the speech signal.signal.

Segment a portion of signal from the background Segment a portion of signal from the background noise.noise.

Ability to view and store various voice data and Ability to view and store various voice data and functionality for research purposes.functionality for research purposes.

Spectrographic tool that provides access to the Spectrographic tool that provides access to the actual numerical data (e.g., the energy in a actual numerical data (e.g., the energy in a particular frequency band in a particular time particular frequency band in a particular time interval) that can be processed later in an interval) that can be processed later in an application.application.

04/22/2304/22/23 44

Human Computer InteractionHuman Computer Interaction User fills in the demographic information.User fills in the demographic information. He can upload his voice file.He can upload his voice file. He can play back any voice file stored in the database He can play back any voice file stored in the database

as indicated by the drop down list.as indicated by the drop down list. He can hear the voice file and choose values for He can hear the voice file and choose values for

certain key words in the voice file for various accents.certain key words in the voice file for various accents. The best chosen accent is recognized and displayed.The best chosen accent is recognized and displayed. Voice owner information is displayed for comparison.Voice owner information is displayed for comparison. For further analysis of voice file, he can download the For further analysis of voice file, he can download the

Speech Filing system installer, install it and run the Speech Filing system installer, install it and run the voice file to get various types of spectrograms and voice file to get various types of spectrograms and other voice data.other voice data.

04/22/2304/22/23 55

Screen shotsScreen shotsParticipation form

Choices are “Academic” or “Natural”.

04/22/2304/22/23 66

Once submitted…Once submitted…

04/22/2304/22/23 77

Classification using Pronunciation Affinity Matrix Classification using Pronunciation Affinity Matrix (PAM)(PAM)

The letter “V” is a “B”Pronunciation in Spanish

Vowel preceeding “ry” ending is typically dropped.

In the various Asian dialects“T” & “TH” are commonly

Pronunced as “D”

04/22/2304/22/23 88

Accent determined with reference to values chosenAccent determined with reference to values chosen

File upload index

04/22/2304/22/23 99

Analyze voice files using SFSAnalyze voice files using SFS

04/22/2304/22/23 1010

Spectrogram generated using Speech Filing System.Spectrogram generated using Speech Filing System.

04/22/2304/22/23 1111

Actual Voice data retrieved from the spectrogram

04/22/2304/22/23 1212

Smooth Fundamental Frequency track

04/22/2304/22/23 1313

Noise Analysis data

04/22/2304/22/23 1414

Users of this ApplicationUsers of this Application

This application will be mainly used for This application will be mainly used for experimental research in areas such as :experimental research in areas such as :

1.Speech Recognition and Accent determination.1.Speech Recognition and Accent determination.2.Voice Biometric Studies. 2.Voice Biometric Studies. 3.Speaker Authentication applications.3.Speaker Authentication applications.

04/22/2304/22/23 1515

Research Next StepsResearch Next Steps

1.1. Build up a sizeable corpus of voice samples across the four Build up a sizeable corpus of voice samples across the four pronunciation nationalities in the PAM matrix.pronunciation nationalities in the PAM matrix.

2.2. Identify examiners that come from the same cross section Identify examiners that come from the same cross section of nationalities found in PAMof nationalities found in PAM

3.3. Perform more identification exams to validate the Perform more identification exams to validate the effectiveness of the selected words/phraseeffectiveness of the selected words/phrase

4.4. Create a data-mart of the numerical equivalent of the Create a data-mart of the numerical equivalent of the spectrograms of each voice sample in the corpus.spectrograms of each voice sample in the corpus.

5.5. Select a data mining classification algorithm to effectively Select a data mining classification algorithm to effectively classify the accents. classify the accents. (maybe focus on the correlation (maybe focus on the correlation between energy levels, specific words, and accents or between energy levels, specific words, and accents or stress patterns and accents)stress patterns and accents)

04/22/2304/22/23 1616

Thank you!