Upload
subzero-cool
View
226
Download
0
Embed Size (px)
Citation preview
8/2/2019 An Application That Talks and Listens [Duites]
1/51
AN APPLICATION THAT TALKS AND LISTENS(MANIPULATING TEXT FOR MOBILE SHORT MESSAGING SYSTEM)
DEEP XAVIER T. DUITES
SUBMITTED TO THE FACULTY OF THE COLLEGE OF COMPUTER STUDIESCEBU INSTITUTE OF TECHNOLOGY
IN PARTIAL FULFILLMENT OF THE REQUIREMENTSFOR THE DEGREE OF
MASTER IN INFORMATION TECHNOLOGY
OCTOBER 2008
8/2/2019 An Application That Talks and Listens [Duites]
2/51
ii
ABSTRACT
Text-To-Speech and Speech-To-Text (TTS&STT) application was a tool that
converts SMS text (also known as SMS Language, or Textese, or TxtSpk
1
) into
speech and vice versa. It used the English language as its base language. The
conversion process used an SMS dictionary of words which were stored in a
database file, to contrast every SMS text to an English word or an English word
to an SMS text.
TTS&STT was implemented using Microsoft .NET technology and Microsoft
Speech Application's Programming Interface (SAPI). SAPI aided in the
conversion processes of both speech recognition and text to speech. The
conversion technology was built-in of the Speech API and would not be part of
the development of this project.
Aside from speech recognition and reading out of text (from a text file, WAV file,
or text typed in the user interface) in which the TTS&STT application was
capable of, it could also produce a converted SMS text into a text file or into a
WAV file. With these types of outputs produced by TTS&STT application, it could
become usable to other applications that rely on speech recognition and text to
speech.
1 SMS Languagehttp://wapedia.mobi/en/SMS_language
8/2/2019 An Application That Talks and Listens [Duites]
3/51
iii
ACKNOWLEDGMENT
I would like to express my sincerest thanks to the following for extending their
support since the start until the completion of this work:
My God, the Almighty Father, who has always been my source of life,
wisdom and will. For without Him, I could not have completed this work.
My wife, Gilda, for being there always to give encouragement and love.
My son, Marcus, for being there to give the smile that fills my heart with so
much joy.
My daughter, Margaret, for being there to give the sweetest smile that fills
my day with so much love and affection.
My dean, Prof. Cherry Lyn C. Sta. Romana, for having been one of the
persons who encouraged me to finishing this work.
My adviser, Prof. Larmie T. Santos-Feliscuzo, for always has been very
supportive of me.
My mentors in the graduate studies, for their consideration, patience, and
understanding.
My colleagues in the College of Computer Studies, for their words of
encouragement and motivation.
My classmates in the graduate studies, for their shared time and thoughts.
8/2/2019 An Application That Talks and Listens [Duites]
4/51
iv
TABLE OF CONTENTS
1. INTRODUCTION .............................................................................................. 1
1.1 Background of the Study ............................................................................ 1
1.2 Objectives of the Study .............................................................................. 2
1.3 Significance of the Study ............................................................................ 3
1.4 Scope and Limitations ................................................................................ 3
2. REVIEW OF RELATED LITERATURE ............................................................. 5
2.1 Yap: Voice-To-Text Translation on Your Cell ............................................. 5
2.2 Jott: Speech-to-Text Mobile Interface ........................................................ 5
2.3 NMS Communications Voice SMS ............................................................ 6
2.4 SpinVox ...................................................................................................... 7
2.5 IBM ViaVoice.............................................................................................. 8
2.6 Microsoft Voice Command ......................................................................... 8
3. MATERIALS AND METHODOLOGIES .......................................................... 10
3.1 Project Budget and Materials ................................................................... 10
3.2 Work Schedule ......................................................................................... 11
3.3 Implementation Plan ................................................................................ 11
4. THEORITICAL FRAMEWORK ....................................................................... 12
4.1 TTS&STT Application ............................................................................... 12
8/2/2019 An Application That Talks and Listens [Duites]
5/51
v
4.1.1 Speech-To-Text Interface................................................................ 12
4.1.2 Text-To-Speech Interface................................................................ 14
5. CONCLUSIONS AND RECOMMENDATIONS ............................................... 16
APPENDIX A: USERS MANUAL ..................................................................... A-1
A.1The Installer Package and the Installation Instructions .......................... A-1
Installation Instructions: .......................................................................... A-1
A.2Getting Started ....................................................................................... A-2
A.3Using the Speech-To-Text Interface ...................................................... A-4
Steps in Interacting with the Speech-To-Text Application: ..................... A-8
A.4Using the Text-To-Speech Interface ...................................................... A-8
Steps in Interacting with the Text-To-Speech Application: ................... A-11
A.5Using the SMS Dictionary Interface ..................................................... A-12
APPENDIX B: PROGRAM LISTINGS
BIBLIOGRAPHY
8/2/2019 An Application That Talks and Listens [Duites]
6/51
vi
LIST OF FIGURES
Figure 1: The basic steps to convert speech to text ............................................ 12
Figure 2: STT's System Architecture .................................................................. 13
Figure 3: The basic steps to convert text to speech ............................................ 14
Figure 4: TTS' System Architecture .................................................................... 15
Figure 5: Contents of the SpeechUiSetup folder ............................................... A-1
Figure 6: Contents of the Debug folder ............................................................. A-1
Figure 7: Contents of the SpeechUiSetup folder in Program Files .................... A-2
Figure 8: TTS&STT's main graphical user interface ......................................... A-2
Figure 9: The LaunchASR shortcut button ....................................................... A-4
Figure 10: The STT's user interface .................................................................. A-5
Figure 11: The Launch TTS shortcut button ..................................................... A-8
Figure 12: The TTS user interface ................................................................... A-9
Figure 13: View menu of the STT application ................................................. A-12
Figure 14: View menu of the TTS application ................................................. A-12
Figure 15: The SMS Dictionary interface ........................................................ A-12
8/2/2019 An Application That Talks and Listens [Duites]
7/51
vii
LIST OF TABLES
Table 1: Materials and Costs .............................................................................. 10
Table 2: Gantt Chart ........................................................................................... 11
Table 3: Main GUI's menu ................................................................................ A-3
Table 4: Menu items of the drop-down File menu ............................................. A-4
Table 5: Menu item of the drop-down Help menu ............................................. A-4
Table 6: The STT's main menu ......................................................................... A-6
Table 7: Menu items of the drop-down File menu of the STT application ......... A-7
Table 8: Menu items of the drop-down Edit menu of the STT application ......... A-7
Table 9: Menu item of the drop-down View menu of the STT application ......... A-7
Table 10: Menu item of the drop-down Help menu of the STT application ....... A-7
Table 11: The TTS main menu ...................................................................... A-10
Table 12: Menu items of the drop-down File menu of the TTS application ..... A-11
Table 13: Menu item of the drop-down View menu of the TTS application ..... A-11
Table 14: Menu item of the drop-down Help menu of the TTS application ..... A-11
8/2/2019 An Application That Talks and Listens [Duites]
8/51
1. INTRODUCTION
1.1 Background of the Study
For the past several decades, designers have processed speech for a wide
variety of applications ranging from mobile communications to automatic
reading machines. Speech recognition reduces the overhead caused by
alternate communication methods. Speech has not been used much in the
field of electronics and computers due to the complexity and variety of
speech signals and sounds. However, with modern processes, algorithms,
and methods we can process speech signals easily and recognize the text.
Voice recognition software (VRS), also known as speech-recognition,
automatic speech recognition, ASR or natural language recognition
software, converts your voice to text on a computer. In essence, what this
means is that you can create text files without typing. When you speak
through a microphone (most voice-recognition software includes this
accessory), the software translates the sounds into written words. It is
initially time consuming, but accuracy and ease of use have now improved
so much that it may finally be worth the investment of time and money.
Many first-generation voice-recognition packages used discrete speech
technology, which meant you had to pause between words in order for the
computer to understand them. The latest generation uses continuous
speech technology, which allows you to speak more naturally. All require an
enrollment process, during which you sit at the computer and read sample
8/2/2019 An Application That Talks and Listens [Duites]
9/51
2
text out loud to help train the speech recognition software to understand
your voice patterns. While most voice-recognition software users want a
large vocabulary in their software package, the larger the vocabulary, the
more time-consuming the training process. Thats because many words
sound similar, and you need to train the recognition system to understand
how you pronounce them.
1.2 Objectives of the Study
This study aims to develop an application software that is capable of
performing the following:
Convert speech or voice data into SMS text.
Convert SMS text into voice data.
In addition, the following specific objectives must also be achieved during
the project development and completion:
To create a desktop application that is a precursor to a mobile
application (since no mobile technology as of this project is
developed that is available publicly that is capable of doing the
abovementioned objectives.)
To design an application interface that will convert voice or speech
data into SMS text.
To design an application interface that will convert SMS text into
voice or speech data.
To be able to implement the designed system using Microsoft Visual
Basic programming of the .NET technology.
8/2/2019 An Application That Talks and Listens [Duites]
10/51
3
To come up with an application that handles both speech-to-text and
text-to-speech.
1.3 Significance of the Study
In the innovation of wireless devices these days, mobile communications is
greatly a part of it. In mobile communications, text messaging (SMS) is so
prevalent. In reality, text messaging has never come too easy; we have to
press a lot of buttons in our cell phones just to create a single text message
but though to some people it is not much of an agony but to the many, it is.
So, this study aims to develop a system that makes text messaging a little
easier a system that will transcribe voice or speech data into text message
and vice versa. By this process, text messaging may have been a
convenience to the many.
1.4 Scope and Limitations
The study is only focused to developing a system that is capable of
converting voice or speech data into a text message and vice versa. The
following will be the main focus of the development of the study: the task to
design the interfaces for the conversion processes; the conversion
mechanisms for both speech-to-text and text-to-speech; and the definition of
the SMS dictionary of words.
The following lists the limitations of this study:
The software cannot convert a saved or recorded audio message
into a text message.
8/2/2019 An Application That Talks and Listens [Duites]
11/51
4
The software cannot extract text messages from a recorded or
saved audio file.
The software cannot speak out other formats of audio files but WAV
files.
The software cannot convert a recognized word into an SMS text if
the recognized word is not present in the defined SMS dictionary.
The software is not capable of recognizing words of other languages
but the English language.
The software is not capable of converting the recognized word into
another language but the English language to form a text message.
8/2/2019 An Application That Talks and Listens [Duites]
12/51
5
2. REVIEW OF RELATED LITERATURE
The following numbered literatures serve as my basis for the development of this
system; for these literatures are the closest applications to this developed system
that are by-products of speech technologies used today.
2.1 Yap: Voice-To-Text Translation on Your Cell
The system lets you send text messages just by talking into your phone.
The software also allows you to query web services like Google, Wikipedia,
or YouTube with nothing but your voice. It also provides voice-to-text
translation services for mobile phones. Users can say anything they like and
Yap will send a text copy to anyone of their contacts. The service is
completely automated so you wont have intermediary Yap employees
listening to your messages, typing them and then sending them out. They
also have a text messaging application call Yap9 that allows you to keep in
touch with friends, family, and co-workers. Users can also use the
application to instantly query mobile web services just by talking. They can
search Google, Wikipedia, Yahoo, and YouTube, or interact with Facebook
without using their phones miniature keyboards.2
2.2 Jott: Speech-to-Text Mobile Interface
Jott is an application from Jott Networks Inc. accessible via the phone, via
web browser, text messaging, email, downloaded software, links created by
2 Kochanov, Ilya Yap: Voice-To-Text Translation On Your Cellhttp://www.crunchgear.com/2007/09/17/yap-voice-to-text-translation-on-your-cell/http://www.techcrunch.com/2007/09/17/techcrunch-40-session-2-mobile-communications/
8/2/2019 An Application That Talks and Listens [Duites]
13/51
6
3rd parties, and 3rd party applications which incorporate the Jott Service via
Application Programming Interfaces. Jott Networks operates a voice to text
service that makes staying organized and in touch easy. Jott allows
consumers to easily and safely send emails and text messages, set
reminders, organize lists, and post to web services with their voice.3
2.3 NMS Communications Voice SMS
Voice SMS is a fast way to send a short message to another mobile
subscriber. In that sense its similar to an SMS text message. But Voice
SMS is much easier to use. There are no keystrokes to compose the
message, you just talk. When someone sends you a Voice SMS, you
receive an SMS text message saying You have a Voice SMS from
**someone in your phonebook**. Click here to listen to your message. One
click and youre listening to the message. In some implementations, it says
Dial *0* to listen to your message. Thats four clicks with the Send key,
but its equally easy for users.
There are additional advantages for Voice SMS over conventional text SMS.
The text user interface on mobile phones is great if you use a European
language, Kanji, or another widely practiced language. But once you get
beyond the top twenty languages, there is little or no support for text
messaging. And of course, text messages are of no use to people who are
illiterate. Yet they easily learn to use Voice SMS.
3 Platzek, Dirk Jott: Speech-to-Text Mobile Interfacehttp://jott.com/default.aspxhttp://www.wunschfeld.net/blog/2007/03/speech-to-text-mobile-interface.html
8/2/2019 An Application That Talks and Listens [Duites]
14/51
7
Voice SMS is perfect for when you want to communicate but dont need or
dont want a live conversationfor example when you think the other party
is likely to be asleep or in a noisy environment. Voice SMS is a convenient
way to give them some information or initiate a non-real time conversation
where the other party can delay their response until they are ready.4
2.4 SpinVox
SpinVox captures spoken messages and cleverly converts them into text. It
then delivers your message to a destination of your choice inbox, blog,
wall or space. The captured spoken words will be fed into a Voice Message
Conversion System, known as D2 (the Brain), and spits them out as text
content. So D2s pretty smart. Hes bound to be, as hes a combination of
artificial intelligence, voice recognition and natural linguistics. But he also
learns from us humans. He learns all the time about how we speak, and
what we say, from the mundane to the ridiculous. And even the smartest
machines need a helping hand if they are to stay clever: help to understand
and digest new words or phrases to ensure that he is converting what you
mean to say. Over the past four years D2 has been chomping through our
words, converting millions of messages from millions of different voices and
accents, in English, French, Spanish and German. But now hes onto his
main course and he wants to feast on your words to make him bigger and
stronger.5
4 Turner, Brough Voice SMSCreating a Service People Will Adopthttp://www.nmscommunications.com/News/NL/TIN/Jan2007/VoiceSMS.htm
5 Whats SpinVox All About Then?
8/2/2019 An Application That Talks and Listens [Duites]
15/51
8
2.5 IBM ViaVoice
The system allows email and web navigation via voice command, meaning,
you can use your voice to create, manage, and send email, chat on the
Internet, command your browser, launch URLs and surf the Web. It has
transcription support for digital handheld recorders. It has a new IBM
speech engine with improved background noise adaptation which can result
in greater dictation and voice command accuracy. It has over 300,000
vocabulary and backup dictionary words and you can add customized
addresses, names, acronyms, terms and colloquialisms to the vocabulary.
6
2.6 Microsoft Voice Command
Voice Command transforms your Windows Mobile smart phone into your
own virtual personal assistant, letting you use your voice to look up
contacts, make phone calls, get calendar information, and play and control
your music, as well as start programs. Voice Command makes it easier and
more convenient than ever to take your digital lifestyle with you wherever
you go.
Because Voice Command has state-of-the-art speech technology, you will
never have to prerecord important phone numbers or use difficult
commands to access the information that you need. Simply select one of
the many commands that is most natural to you and let Voice Command do
http://www.spinvox.com/homepage.htmlhttp://www.spinvox.com/how_it_works.html
6 IBM Corporation IBM ViaVoice Advanced Edition Release 10http://www.nuance.com/viavoice/advanced/
8/2/2019 An Application That Talks and Listens [Duites]
16/51
9
the rest. You can even ask, "What can I say?" at any time to help find an
appropriate command.7
7 Microsoft Voice Command 1.6
http://www.microsoft.com/windowsmobile/voicecommand/features.mspx
8/2/2019 An Application That Talks and Listens [Duites]
17/51
10
3. MATERIALS AND METHODOLOGIES
3.1 Project Budget and Materials
Shown below is the table of itemized list of tangible and intangible materials
needed to develop and realize the system. Also shown are the costs of each
respective material (in pesos) as of this writing:
Material Cost
a. Personal Computer (1 unit) 32,000.00b. Microsoft Windows Vista 7,500.00
c. Microsoft Office 2007 6,500.00
d. Microsoft .NET Framework 3.0 (free, available for downloadat microsoft.com)
0.00
e. Microsoft Visual Studio 2008 Express Edition (free, availablefor download at microsoft.com)
0.00
f. Standard headset (with microphone) 200.00
g. Speakers 300.00
Total Budget Cost 46,500.00Table 1: Materials and Costs
8/2/2019 An Application That Talks and Listens [Duites]
18/51
11
3.2 Work Schedule
The Gantt chart below shows the timeline of activities that is to be observed
during the development of this system:
June2008
July2008
Aug2008
Sept2008
Oct2008
System Analysis andDesign
Coding
Testing
Maintenance
DocumentationTable 2: Gantt Chart
3.3 Implementation Plan
This application software will be deployed in a standalone desktop with the
software requirements based on Table 1: Materials and Costs. The speech-to-
text interface of the application will require a standard headset for speech input.
The text-to-speech interface of the application will require a set of speakers for
the audio output. The SMS dictionary of words will be stored into a database in
which the user can add, update and delete words.
8/2/2019 An Application That Talks and Listens [Duites]
19/51
12
4. THEORITICAL FRAMEWORK
4.1 TTS&STT Application
The TTS&STT application is designed and implemented using object-
oriented approach. It uses the Microsoft Visual Basic .NET
of the
Microsoft .NET Framework as the programming language, Speech
Application Programming Interface (SAPI) for speech recognition and
synthesis, and Microsoft Access 2003 as its database tool.
4.1.1 Speech-To-Text Interface
The Speech-To-Text interface of the TTS&STT application handles the
speech recognition capability using the Speech APIs speech recognition
engine. The speech recognition process of the speech-to-text interface is
based on the following steps, as shown in Figure 1.
Figure 1: The basic steps to convert speech to text
Voice Input
Text Output
SR En ine SS En ine
SAPI
8/2/2019 An Application That Talks and Listens [Duites]
20/51
13
The conversion of speech to text follows the basic steps as shown in
Figure 1. The detailed conversion of speech to text is shown in Figure 2.
Figure 2: STT's System Architecture
In this conversion process, the user utters a word using a standard
microphone and SAPI will recognize the word. Once recognized by SAPI,
the recognized word will passed to the text parser and the text parser will
Voice Input
SR En ine SS En ine
SAPI
Recognized Text
Text Parser
Text Output
SMS text Database
SMS text manipulation
Add word
Delete word
Update word
Sort
8/2/2019 An Application That Talks and Listens [Duites]
21/51
14
compare the recognized word into the SMS text dictionary and outputs the
word.
4.1.2 Text-To-Speech Interface
The Text-To-Speech interface of the TTS&STT application handles the
text to speech conversion or the so called "speech synthesis" using the
Speech API's speech synthesis engine. The speech synthesis process of
the text-to-speech interface is based on the following steps, as shown in
Figure 3.
Figure 3: The basic steps to convert text to speech
The conversion of text to speech follows the basic steps as shown in
Figure 3. The detailed conversion of text to speech is shown in Figure 4.
Text Input
Voice Output
SR En ine SS En ine
SAPI
8/2/2019 An Application That Talks and Listens [Duites]
22/51
15
Figure 4: TTS' System Architecture
In this conversion process, the text input will parsed by the text parser by
comparing each word of the text input to the SMS text dictionary. Once
parsed, each word of the parsed text will be analyzed by SAPI using the
speech synthesis engine and outputs it as voice.
Text Input
SR En ine SS En ine
SAPI
Parsed Text
Text Parser
Voice Output
SMS text Database
SMS text manipulation
Add word
Delete word
Update word
Sort
8/2/2019 An Application That Talks and Listens [Duites]
23/51
16
5. CONCLUSIONS AND RECOMMENDATIONS
The Speech API (Applications Programming Interface) for mobile applications
has been used by a number of developers today. But it has been exclusively
developed and distributed to mobile companies and its of no avail to anyone who
might want to use the technology. The IDE (Integrated Development
Environment) and other related tools for developing such applications are also
exclusively distributed.
As of the development of this project, the public API for developing speech
application for mobile is not yet available. Despite the limited resource and
technology, this project was still pursued. The speech-to-text and text-to-speech
interfaces that are supposed to be implemented in a mobile environment are
developed instead in a desktop environment.
Due to the unavailability of public API for developing speech application for
mobile, the project developer was not able to implement it in the intended target
environment which is the mobile environment. Interested developers may
implement the following suggestions for its improvement and suitability:
1. The speech-to-text and text-to-speech interfaces should be implemented
in a mobile environment that supports speech technology.
8/2/2019 An Application That Talks and Listens [Duites]
24/51
17
2. The Speech API for mobile applications to be used should support both
speech recognition and voice output.
3. The SMS dictionary of words should be well defined so that the messages
it will create will be understandable and reliable.
4. The speech recognition process should be given attention and more
training since it requires a so-much-like English or American accent for the
engine to recognize a word. More so, constant training with your voice can
improve the recognition capability of the engine.
8/2/2019 An Application That Talks and Listens [Duites]
25/51
A-1
APPENDIX A: USERS MANUAL
A.1 The Installer Package and the Installation Instructions
The contents of the installation folder SpeechUiSetup are shown inFigure 5.
Figure 5: Contents of the SpeechUiSetup folder
The Debug folder, as shown in Figure 5, has the setup program to beused for the installation. The contents of the Debug folder are shown inFigure 6.
Figure 6: Contents of the Debug folder
Installation Instructions:
1. Run the setup.exe orSpeechUiSetup.exe file that is found inside theDebug folder in the installation folder (as shown in Figures 5 & 6). Thiswill install the TTS&STT (Text-To-Speech & Speech-To-Text) programinto the host PC. (This step assumes that the Microsoft .NET 3.0 orabove has been installed into the host PC already.)
2. After running the setup program, an installation wizard will appear which will guide you through a successful installation process. Just
leave the default settings as is and proceed with the installation.
3. After successful installation, you need to proceed to the final instructionwhich is written inside the file READ_ME.txt, located in the defaultdirectory C:\Program Files\DeepQuest Corp\SpeechUiSetup. Thecontents of the directory are shown in Figure 7.
8/2/2019 An Application That Talks and Listens [Duites]
26/51
A-2
Figure 7: Contents of the SpeechUiSetup folder in Program Files
4. After doing the instruction inside READ_ME.txt file, you can double-click on SpeechUI.exe file (make sure not the XML Configuration file)to run the program.
A.2 Getting Started
Assuming the TTS&STTprogram is not running, run it by double-clicking theSpeechUI.exe file that is found inside the SpeechUiSetup folder (as shown in
Figure 7). After a few seconds, the TTS&STTs main graphical user interface(GUI) will appear on screen, as shown in Figure 8.
Figure 8: TTS&STT's main graphical user interface
The descriptions of the labeled parts of the main GUI are the following:
Menu bar contains the drop-down menus used for launching the differentfeatures of the program.
Menu bar
Shortcut buttons
Minimize and Close control buttons
8/2/2019 An Application That Talks and Listens [Duites]
27/51
A-3
Shortcut buttons one-click controls for easy access of the main features ofthe program.
Minimize and Close control buttons the minimize button is used to obviouslyminimize the window if the user decides to get back to the running programlater, and the close button is used to completely terminate the runningapplication.
The following tabulations give details of the different menus provided in theprograms main GUI:
Menu Name Description When Activated When Deactivated
File A drop-down menu; itcontains the menuitems which are themain features of the
program
(always activated) (always activated)
Help A drop-down menu; itcontains the menuitem that launches adialog box thatdisplays theprograms version,the copyright year,the author and thebrief description ofthe program.
(always activated) (always activated)
Table 3: Main GUI's menu
Menu Name Description When Activated When Deactivated
Launch ASR It opens the Speech-To-Text interfacewhich allows the userto use speech to textconversion.
(always activated) (always activated)
Launch TTS It opens the Text-To-Speech interfacewhich allows the userto use text to speechconversion.
(always activated) (always activated)
8/2/2019 An Application That Talks and Listens [Duites]
28/51
A-4
Menu Name Description When Activated When Deactivated
Exit It terminates therunning program.
(always activated) (always activated)
Table 4: Menu items of the drop-down File menu
Menu Name Description When Activated When Deactivated
About It opens the dialogbox that displays theprograms version,the copyright year,the author and thebrief description ofthe program.
(always activated) (always activated)
Table 5: Menu item of the drop-down Help menu
A.3 Using the Speech-To-Text Interface
The Speech-To-Text (STT) interface can be launched by accessing the Filemenu then choosing Launch ASRor by clicking on the Launch ASRshortcutbutton of the main GUI, as shown in Figure 9 below.
Figure 9: The Launch ASRshortcut button
Using either of the two options to launch STT, the STT graphical user interfacewill appear on screen, as shown in Figure 10.
Launch ASR button
8/2/2019 An Application That Talks and Listens [Duites]
29/51
A-5
Figure 10: The STT's user interface
The descriptions of the labeled parts of the Speech-To-Text GUI are thefollowing:
Menu bar contains the advance and basic operations of the STT interface.
Recognized text area contains the recognized text or words. (If the text areacontains [Recognized text will be placed here.], which is the default text,then no recognized text yet has been detected by the recognition engine.)
Parsed text area contains the parsed text or words. (If the text area contains[Parsed text will be placed here.], which is the default text, then no parsingof the recognized text has been done yet.)
Minimize and Close control buttons the minimize button is used to obviouslyminimize the window if the user decides to get back to STT program later,and the close button is used to terminate the STT application and return tothe main GUI.
Status bar indicates the status of the recognition engine.
The following tabulations give details of the different menus provided in theSpeech-To-Text GUI:
Menu Name Description When Activated When Deactivated
File A drop-down menu; itcontains the items for
(always activated) (always activated)
Recognized text area
Parsed text area
Menu bar
Status bar
Minimize and Close control buttons
8/2/2019 An Application That Talks and Listens [Duites]
30/51
A-6
Menu Name Description When Activated When Deactivated
loading a grammar,saving to file andclosing the interface.
Edit A drop-down menu; itcontains the items forparsing therecognized text toSMS text, clearingthe textboxes
(always activated) (always activated)
View A drop-down menu; itcontains the item thatwill display the SMSdictionary of words.
(always activated) (always activated)
Help A drop-down menu; itcontains the menuitem that launches adialog box thatdisplays theapplication name, thecopyright year andthe author.
(always activated) (always activated)
Table 6: The STT's main menu
Menu Name Description When Activated When Deactivated
Load DefaultGrammar
It loads the defaultgrammar supportedby the Speech APIinstalled in the hostPC.
(always activated) (always activated)
LoadExternal
Grammar
(not implemented no support yet as of
this writing)
(not implemented no support yet as of
this writing)
(not implemented no support yet as of
this writing)
Save To File A submenu thatcontains the menuitems for saving therecognized text or
(always activated) (always activated)
8/2/2019 An Application That Talks and Listens [Duites]
31/51
A-7
Menu Name Description When Activated When Deactivated
the parsed text into aWAV file.
Close It terminates the STT
application andreturns to the mainapplication.
(always activated) (always activated)
Table 7: Menu items of the drop-down File menu of the STT application
Menu Name Description When Activated When Deactivated
Parse toTxtSpk
It parses/converts therecognized text to
SMS text based onthe SMS dictionary ofwords.
(always activated) (always activated)
EmptyTextboxes
It clears the contentsof the two textboxes.
(always activated) (always activated)
Table 8: Menu items of the drop-down Edit menu of the STT application
Menu Name Description When Activated When Deactivated
Dictionary It opens the interfaceof the SMS dictionaryof words, as shownin Figure 15.
(always activated) (always activated)
Table 9: Menu item of the drop-down View menu of the STT application
Menu Name Description When Activated When Deactivated
About It opens a dialog box
that displays thename of theapplication, thecopyright year andthe name of theauthor
(always activated) (always activated)
Table 10: Menu item of the drop-down Help menu of the STT application
8/2/2019 An Application That Talks and Listens [Duites]
32/51
A-8
Steps in Interacting with the Speech-To-Text Application:
1. The user needs to load the default grammar (installed in the host PC andsupported by the Speech API) by clicking on FileLoad Default Grammar.
2. The user can now utter the words he or she wishes to be recognized (by thespeech recognition engine used in the STT application) using a standardmicrophone.
3. After the uttered words are recognized, the user can parse or convert therecognized text to SMS text based on the database of SMS words.
4. After parsing or conversion, the user can optionally save the recognized textto a WAV file by clicking on FileSave To File Recognized Text, or theuser can optionally save the parsed text to a WAV file by clicking onFileSave To FileParsed Text.
A.4 Using the Text-To-Speech Interface
The Text-To-Speech (TTS) interface can be launched by accessing the Filemenu then choosing Launch TTS or by clicking on the Launch TTS shortcutbutton of the main GUI, as shown in Figure 11 below.
Figure 11: The Launch TTS shortcut button
Using either of the two options to launch the TTS application, the TTS graphicaluser interface will appear on screen, as shown in Figure 12.
Launch TTS button
8/2/2019 An Application That Talks and Listens [Duites]
33/51
A-9
Figure 12: The TTS user interface
The descriptions of the labeled parts of the Text-To-Speech GUI are thefollowing:
Menu bar contains the advance and basic operations of the TTS interface.
Text entry area contains the text or words typed by the user or extractedfrom a text file which are to be spoken by the speech engine.
Parsed text area contains the parsed text or words that have been spokenby the speech engine.
Speech control buttons controls the speech engine by making it speak,stop, pause and reset.
Voice settings sets the supported voices (if there are other voices installed),format for the audio output, adjusts the speaking rate (speed) and the volume.
Minimize and Close control buttons the minimize button is used to obviouslyminimize the window if the user decides to get back to TTS application later,and the close button is used to terminate the TTS application and return to
the main GUI.
Status bar indicates the status of the speech engine.
The following tabulations give details of the different menus provided in theSpeech-To-Text GUI:
Text entry area
Parsed text area
Speech control buttons
Voice settings
Menu bar
Status bar
Minimize and Close control buttons
8/2/2019 An Application That Talks and Listens [Duites]
34/51
A-10
Menu Name Description When Activated When Deactivated
File A drop-down menu; itcontains the menuitems that allows theapplication to speakfrom a WAV file orfrom a text file.
(always activated) (always activated)
View A drop-down menu; itcontains the menuitem that will displaythe SMS dictionary ofwords.
(always activated) (always activated)
Help A drop-down menu; itcontains the menu
item that launches adialog box thatdisplays theapplication name, thecopyright year andthe author.
(always activated) (always activated)
Table 11: The TTS main menu
Menu Name Description When Activated When Deactivated
Speak fromText File
It opens the OpenDialog Box to let theuser choose a textfile.
(always activated) (always activated)
Speak fromWave File
It opens the OpenDialog Box to let theuser choose a WAVfile.
(always activated) (always activated)
Save to
Wave File
It opens the Save As
Dialog Box to let theuser save the audioequivalent of theparsed text to a WAVfile.
(always activated) (always activated)
Close It terminates the TTS (always activated) (always activated)
8/2/2019 An Application That Talks and Listens [Duites]
35/51
A-11
Menu Name Description When Activated When Deactivated
application andreturns to the mainapplication.
Table 12: Menu items of the drop-down File menu of the TTS application
Menu Name Description When Activated When Deactivated
Dictionary It opens the interfaceof the SMS dictionaryof words, as shownin Figure 15.
(always activated) (always activated)
Table 13: Menu item of the drop-down View menu of the TTS application
Menu Name Description When Activated When Deactivated
About It opens a dialog boxthat displays thename of theapplication, thecopyright year andthe name of theauthor
(always activated) (always activated)
Table 14: Menu item of the drop-down Help menu of the TTS application
Steps in Interacting with the Text-To-Speech Application:
1. The user needs to type a word or set of words in the text entry area or theuser opens a text file (click on FileSpeak from Text File) or opens a WAVfile (click on FileSpeak from Wave File).
2. The user can now adjust the voice settings, as shown in Figure 8, but leavingit as is will do the job perfectly.
3. After adjustments were made in the voice settings (if any), the user needs toclick on the Speak button in the Speech control buttons group, as shown inFigure 8. Note that if the user decided to let the application speak from aWAV file, then it is not necessary for the user to click on the Speak buttonsince this option will let the application speak automatically.
4. Optionally, the user can save the typed words in the text entry area into aWAV file if he or she wishes to.
8/2/2019 An Application That Talks and Listens [Duites]
36/51
A-12
A.5 Using the SMS Dictionary Interface
The SMS Dictionary interface can be launched using the ViewDictionary menuof both the Speech-To-Text and Text-To-Speech applications, as shown inFigures 13 and 14 respectively.
Figure 13: View menu of the STT
applicationFigure 14: View menu of the TTS
application
Using either of the options to launch the SMS Dictionary, the SMS Dictionary
interface will appear on screen as shown in Figure 15.
Figure 15: The SMS Dictionary interface
In this interface, the user can add, update, delete word or words, or sort (in
ascending or descending manner) the words in the dictionary.
8/2/2019 An Application That Talks and Listens [Duites]
37/51
B-1
APPENDIX B: PROGRAM LISTINGS
----------------------------------------------------------------------------------------------------------------------Imports System.Data.OleDb
Imports System.Speech.RecognitionPublic Class frmASR
Dim SpRecognizer As New SpeechRecognitionEnginePrivate Sub mnuAbout_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles
mnuAbout.ClickMsgBox("Speech Recognition application." & Chr(13) & "Copyright 2008" & Chr(13) & "Deepquest
Corp.", MsgBoxStyle.OkOnly + MsgBoxStyle.Information, "ASR")End Sub
Private Sub mnuDictionary_Click(ByVal sender As System.Object, ByVal e As System.EventArgs)Handles mnuDictionary.Click
frmDictionary.Show()End Sub
Private Sub mnuClose_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) HandlesmnuClose.Click
Me.Close()frmSpeechEngineMain.Show()
End SubPrivate Sub frmASR_FormClosed(ByVal sender As Object, ByVal e As
System.Windows.Forms.FormClosedEventArgs) Handles Me.FormClosedfrmDictionary.Close()SpRecognizer.Dispose()SpRecognizer = NothingfrmSpeechEngineMain.Show()
End Sub
Private Sub mnuParseToTxtSpk_Click(ByVal sender As System.Object, ByVal e As System.EventArgs)Handles mnuParseToTxtSpk.Click
Dim p_Str As String = txtRecoString.TexttxtRecoString.Text = TextParser(p_Str)
End SubPrivate Function TextParser(ByVal p_Str As String) As String
Dim n_Str As String = String.EmptyDim t_Str() As StringDim d_Str As String = " " & vbCrLf & "." & "," & "?" & "!"Dim d_Chars() As Char = d_Str.ToCharArray
'Tokenize the stringt_Str = p_Str.Split(d_Chars, StringSplitOptions.None)'Compare every string if it's in the databaseFor Each s As String In t_Str
n_Str = n_Str & " " & ParseToDigiWord(s)NextReturn n_Str.Trim
End FunctionPrivate Function ParseToDigiWord(ByVal f_Str As String) As String
'Connect to the database
Dim objConnection As OleDbConnection = ConnectToDatabase()
'Search if the string is in the dictionaryDim strSQLSearch As String = "SELECT digiWord FROM Words WHERE realWord = '" & f_Str & "'"Dim objDataAdapter As New OleDbDataAdapter(strSQLSearch, objConnection)Dim objDataTable As New DataTable("mWords")objDataAdapter.Fill(objDataTable)
If objDataTable.Rows.Count > 0 Thenf_Str = objDataTable.Rows(0)("digiWord").ToString
End If
8/2/2019 An Application That Talks and Listens [Duites]
38/51
B-2
objDataAdapter.Dispose()objDataAdapter = NothingobjDataTable.Dispose()objDataTable = NothingobjConnection.Close()objConnection = Nothing
Return f_StrEnd Function
Private Sub frmASR_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) HandlesMyBase.Load
Dim grammar As Grammar = DefaultGrammar()SpRecognizer.LoadGrammarAsync(grammar)
SpRecognizer.SetInputToDefaultAudioDevice()SpRecognizer.RecognizeAsync()
AddHandler SpRecognizer.SpeechDetected, AddressOf SpeechDetectedEventHandlerAddHandler SpRecognizer.SpeechRecognized, AddressOf SpeechRecognizedEventHandlerAddHandler SpRecognizer.SpeechRecognitionRejected, AddressOf
SpeechRecognitionRejectedEventHandlerAddHandler SpRecognizer.RecognizeCompleted, AddressOf SpeechRecognizeCompletedEventHandler
End SubPrivate Sub SpeechRecognizedEventHandler(ByVal sender As Object, ByVal e As
SpeechRecognizedEventArgs)Dim result As RecognitionResult = e.ResultDim phrase As String = result.Text
'Update the recognized text window's stringtxtRecoString.Text = txtRecoString.Text & phrase & " "tssRecoText.Text = "Recognized"
'Update the parsed text window's stringtxtParseText.Text = txtParseText.Text & ParseToDigiWord(phrase) & " "
End SubPrivate Sub SpeechRecognitionRejectedEventHandler(ByVal sender As Object, ByVal e As
SpeechRecognitionRejectedEventArgs)tssRecoText.Text = "Rejected"
End Sub
Private Sub SpeechDetectedEventHandler(ByVal sender As Object, ByVal e AsSpeechDetectedEventArgs)
tssRecoText.Text = "Ready"End SubPrivate Sub SpeechRecognizeCompletedEventHandler(ByVal sender As Object, ByVal e As
RecognizeCompletedEventArgs)SpRecognizer.RecognizeAsync()
End Sub
Private Sub mnuLoadDefaultGrammar_Click(ByVal sender As System.Object, ByVal e AsSystem.EventArgs) Handles mnuLoadDefaultGrammar.Click
Dim grammar As Grammar = DefaultGrammar()
SpRecognizer.UnloadAllGrammars()SpRecognizer.LoadGrammar(grammar)
End Sub
Private Function DefaultGrammar() As Grammar'Connect to the databaseDim objConnection As OleDbConnection = ConnectToDatabase()
'Extract all word pairs in the dictionaryDim strSQL As String = "SELECT realWord FROM Words"Dim objDataAdapter As New OleDbDataAdapter(strSQL, objConnection)
8/2/2019 An Application That Talks and Listens [Duites]
39/51
B-3
Dim objDataTable As New DataTable("mWords")objDataAdapter.Fill(objDataTable)
Dim words As New ChoicesDim gBuilder As New GrammarBuilder
For Each row As DataRow In objDataTable.Rowswords.Add(row.Item("realWord"))
Next
gBuilder.Append(words)
objDataAdapter.Dispose()objDataAdapter = NothingobjDataTable.Dispose()objDataTable = NothingobjConnection.Close()objConnection = Nothing
Dim grammar As New Grammar(gBuilder)Return grammar
End Function
Private Function ConnectToDatabase() As OleDbConnectionDim strConnectionString As String = "Provider=Microsoft.Jet.OLEDB.4.0;" & _
"Data Source=C:\Data\TextWords.mdb;"Dim objConnection As New OleDbConnection(strConnectionString)
TryobjConnection.Open()
Catch oleDbException As OleDbExceptionMsgBox(oleDbException.Message, MsgBoxStyle.OkOnly + MsgBoxStyle.Critical, "Error " &
oleDbException.ErrorCode)Catch invalidOperationException As InvalidOperationException
MsgBox(invalidOperationException.Message, MsgBoxStyle.OkOnly + MsgBoxStyle.Critical, "Error")End TryReturn objConnection
End Function
Private Sub txtRecoString_DoubleClick(ByVal sender As Object, ByVal e As System.EventArgs) HandlestxtRecoString.DoubleClick
txtRecoString.Text = String.EmptytxtParseText.Text = String.Empty
End Sub
Private Sub txtParseText_DoubleClick(ByVal sender As System.Object, ByVal e As System.EventArgs)Handles txtParseText.DoubleClick
txtParseText.Text = String.EmptyEnd Sub
Private Sub mnuEmptyTextboxes_Click(ByVal sender As System.Object, ByVal e As System.EventArgs)Handles mnuEmptyTextboxes.Click
txtRecoString.Text = String.EmptytxtParseText.Text = String.Empty
End Sub
Private Sub mnuSaveRecognizedText_Click(ByVal sender As System.Object, ByVal e AsSystem.EventArgs) Handles mnuSaveRecognizedText.Click
If String.IsNullOrEmpty(txtRecoString.Text) ThenMsgBox("Cannot save an empty text to a file.", MsgBoxStyle.OkOnly + MsgBoxStyle.Exclamation,
"Empty")Exit Sub
End If
Dim SaveAsDialog As New SaveFileDialogSaveAsDialog.Filter = "Text files (*.txt)|*.txt"
8/2/2019 An Application That Talks and Listens [Duites]
40/51
B-4
If SaveAsDialog.ShowDialog = DialogResult.OK Then'Write string to fileFileOpen(1, SaveAsDialog.FileName, OpenMode.Output)PrintLine(1, txtRecoString.Text)FileClose(1)
End If
SaveAsDialog.Dispose()SaveAsDialog = Nothing
tssRecoText.Text = "Saved"End Sub
Private Sub mnuSaveParsedText_Click(ByVal sender As System.Object, ByVal e As System.EventArgs)Handles mnuSaveParsedText.Click
If String.IsNullOrEmpty(txtParseText.Text) ThenMsgBox("Cannot save an empty text to a file.", MsgBoxStyle.OkOnly + MsgBoxStyle.Exclamation,
"Empty")Exit Sub
End If
Dim SaveAsDialog As New SaveFileDialogSaveAsDialog.Filter = "Text files (*.txt)|*.txt"
If SaveAsDialog.ShowDialog = DialogResult.OK Then'Write string to fileFileOpen(1, SaveAsDialog.FileName, OpenMode.Output)PrintLine(1, txtParseText.Text)FileClose(1)
End If
SaveAsDialog.Dispose()SaveAsDialog = Nothing
tssRecoText.Text = "Saved"End Sub
End Class----------------------------------------------------------------------------------------------------------------------
Imports System.Data.OleDbImports System.Speech.SynthesisImports System.Speech.AudioFormatPublic Class frmTTS
Dim SSynthesizer As New SpeechSynthesizerPrivate Sub mnuAbout_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles
mnuAbout.ClickMsgBox("Text-To-Speech application." & Chr(13) & "Copyright 2008" & Chr(13) & "Deepquest
Corp.", MsgBoxStyle.OkOnly + MsgBoxStyle.Information, "TTS")End Sub
Private Sub frmTTS_FormClosed(ByVal sender As Object, ByVal e AsSystem.Windows.Forms.FormClosedEventArgs) Handles Me.FormClosed
frmSpeechEngineMain.Show()SSynthesizer.Dispose()SSynthesizer = Nothing
frmDictionary.Close()End Sub
Private Sub frmTTS_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) HandlesMyBase.Load
'Populate the contents of the voice combo boxFor Each voice As InstalledVoice In SSynthesizer.GetInstalledVoices()
cbVoice.Items.Add(voice.VoiceInfo.Name)NextcbVoice.SelectedIndex = 0
8/2/2019 An Application That Talks and Listens [Duites]
41/51
B-5
'Populate the contents of the format combo boxDim hrtz As New ArrayListDim type As New ArrayListhrtz.Add("8kHz")hrtz.Add("11kHz")hrtz.Add("12kHz")hrtz.Add("16kHz")hrtz.Add("22kHz")
hrtz.Add("24kHz")hrtz.Add("32kHz")hrtz.Add("44kHz")hrtz.Add("48kHz")type.Add("8 Bit Mono")type.Add("8 Bit Stereo")type.Add("16 Bit Mono")type.Add("16 Bit Stereo")For Each f As String In hrtz
For Each t As String In typecbFormat.Items.Add(f & " " & t)
NextNextcbFormat.SelectedIndex = 18
'Add an event handler to notify if speaking is completedAddHandler SSynthesizer.SpeakCompleted, AddressOf SpeakCompletedEventHandler
'Add an event handler to notify if speaking is startedAddHandler SSynthesizer.SpeakStarted, AddressOf SpeakStartedEventHandler
End Sub
Private Sub btnSpeak_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) HandlesbtnSpeak.Click
Dim p_Str As String = TextParser(txtTextWindow.Text)Try
btnSpeak.Enabled = FalsebtnStop.Enabled = TruebtnPause.Enabled = TrueSSynthesizer.SelectVoice(cbVoice.Text)SSynthesizer.SetOutputToDefaultAudioDevice()
SSynthesizer.Volume = trVolume.ValueSSynthesizer.Rate = trRate.ValueSSynthesizer.SpeakAsync(p_Str)txtRealSpoken.Text = p_Str
Catch ex As ExceptionMsgBox(ex.Message, MsgBoxStyle.OkOnly + MsgBoxStyle.Critical, "Error")
End TryEnd SubPrivate Sub btnStop_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles
btnStop.ClickSSynthesizer.SpeakAsyncCancelAll()btnStop.Enabled = FalsebtnPause.Enabled = FalsebtnSpeak.Enabled = True
End Sub
Private Sub btnPause_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) HandlesbtnPause.Click
If btnPause.Text = "Pause" ThenSSynthesizer.Pause()btnPause.Text = "Resume"
ElseSSynthesizer.Resume()btnPause.Text = "Pause"
End IfEnd Sub
8/2/2019 An Application That Talks and Listens [Duites]
42/51
B-6
Private Sub mnuClose_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) HandlesmnuClose.Click
Me.Close()End Sub
Private Sub btnReset_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) HandlesbtnReset.Click
'Reset buttons' status
SSynthesizer.SpeakAsyncCancelAll()btnStop.Enabled = FalsebtnPause.Enabled = FalsebtnSpeak.Enabled = True'Reset voice, rate, volume and format valuescbVoice.SelectedIndex = 0trRate.Value = 0trVolume.Value = 100cbFormat.SelectedIndex = 18'Clear textboxtxtRealSpoken.Text = String.Empty
End Sub
Private Sub mnuDictionary_Click(ByVal sender As System.Object, ByVal e As System.EventArgs)Handles mnuDictionary.Click
frmDictionary.Show()End Sub
Private Sub mnuSpeakTextFile_Click(ByVal sender As System.Object, ByVal e As System.EventArgs)Handles mnuSpeakTextFile.Click
Dim f_Str As String = String.Empty, f_pStr As String = String.EmptyDim TTSOpenFileDialog As New OpenFileDialogTTSOpenFileDialog.Filter = "Text files (*.txt)|*.txt"If TTSOpenFileDialog.ShowDialog() = DialogResult.OK Then
If TTSOpenFileDialog.FileName String.Empty ThenTry
FileOpen(1, TTSOpenFileDialog.FileName, OpenMode.Input)Do Until EOF(1)
f_pStr = LineInput(1)f_Str = f_Str & f_pStr & vbCrLf
Loop
txtTextWindow.Text = f_StrCatch ex As Exception
MsgBox(ex.Message, MsgBoxStyle.OkOnly + MsgBoxStyle.Critical, "Error")Finally
FileClose(1)End Try
End IfEnd IfTTSOpenFileDialog.Dispose()TTSOpenFileDialog = Nothing
End Sub
Private Function TextParser(ByVal p_Str As String) As StringDim n_Str As String = String.EmptyDim t_Str() As StringDim d_Str As String = " " & vbCrLf & "." & "," & "?" & "!"
Dim d_Chars() As Char = d_Str.ToCharArray'Tokenize the stringt_Str = p_Str.Split(d_Chars, StringSplitOptions.None)'Compare every string if it's in the databaseFor Each s As String In t_Str
n_Str = n_Str & ParseToRealWord(s) & " "NextReturn n_Str
End Function
Private Function ParseToRealWord(ByVal f_Str As String) As String
8/2/2019 An Application That Talks and Listens [Duites]
43/51
B-7
'Connect to the databaseDim strConnectionString As String = "Provider=Microsoft.Jet.OLEDB.4.0;" & _
"Data Source=C:\Data\TextWords.mdb;"Dim objConnection As New OleDbConnection(strConnectionString)
'Open the databaseTry
objConnection.Open()
Catch oleDbException As OleDbExceptionMsgBox(oleDbException.Message, MsgBoxStyle.OkOnly + MsgBoxStyle.Critical, "Error " &
oleDbException.ErrorCode)Catch invalidOperationException As InvalidOperationException
MsgBox(invalidOperationException.Message, MsgBoxStyle.OkOnly + MsgBoxStyle.Critical, "Error")End Try
'Search if the string is in the dictionaryDim strSQLSearch As String = "SELECT realWord FROM Words WHERE digiWord = '" & f_Str & "'"Dim objDataAdapter As New OleDbDataAdapter(strSQLSearch, objConnection)Dim objDataTable As New DataTable("mWords")objDataAdapter.Fill(objDataTable)
If objDataTable.Rows.Count > 0 Thenf_Str = objDataTable.Rows(0)("realWord").ToString
End If
objDataAdapter.Dispose()objDataAdapter = NothingobjDataTable.Dispose()objDataTable = NothingobjConnection.Close()objConnection = Nothing
Return f_StrEnd Function
Private Sub mnuSaveToWaveFile_Click(ByVal sender As System.Object, ByVal e As System.EventArgs)Handles mnuSaveToWaveFile.Click
If String.IsNullOrEmpty(txtTextWindow.Text) ThenMsgBox("Cannot save an empty text to a wave file.", MsgBoxStyle.OkOnly +
MsgBoxStyle.Exclamation, "Empty")Exit Sub
End If
Dim p_Str As String = TextParser(txtTextWindow.Text)Dim TTSSaveFileDialog As New SaveFileDialog
TTSSaveFileDialog.Filter = "Wave files (*.wav)|*.wav"If TTSSaveFileDialog.ShowDialog() = DialogResult.OK Then
SSynthesizer.SetOutputToWaveFile(TTSSaveFileDialog.FileName)SSynthesizer.SpeakAsync(p_Str)
End If
TTSSaveFileDialog.Dispose()TTSSaveFileDialog = Nothing
End Sub
Private Sub mnuSpeakWaveFile_Click(ByVal sender As System.Object, ByVal e As System.EventArgs)Handles mnuSpeakWaveFile.Click
Dim TTSOpenFileDialog As New OpenFileDialogTTSOpenFileDialog.Filter = "Wave files (*.wav)|*.wav"
If TTSOpenFileDialog.ShowDialog() = DialogResult.OK Then
txtTextWindow.Text = "[Enter the text you wish spoken here.]"txtRealSpoken.Text = String.Empty
8/2/2019 An Application That Talks and Listens [Duites]
44/51
B-8
txtTextWindow.Enabled = FalsetxtRealSpoken.Enabled = False
'Disable | Enable control buttonsbtnSpeak.Enabled = FalsebtnStop.Enabled = TruebtnPause.Enabled = True
'Create a prompt to handle the wave fileDim promptBuilder As New PromptBuilderpromptBuilder.AppendAudio(TTSOpenFileDialog.FileName)SSynthesizer.SpeakAsync(promptBuilder)
End If
TTSOpenFileDialog.Dispose()TTSOpenFileDialog = Nothing
End Sub
Private Sub SpeakCompletedEventHandler(ByVal sender As Object, ByVal e AsSpeakCompletedEventArgs)
btnSpeak.Enabled = TruebtnStop.Enabled = FalsebtnPause.Enabled = FalsetxtTextWindow.Enabled = TruetxtRealSpoken.Enabled = TruetssStatus.Text = "Ready"
End Sub
Private Sub SpeakStartedEventHandler(ByVal sender As Object, ByVal e As SpeakStartedEventArgs)tssStatus.Text = "Speaking"
End Sub
Private Sub txtTextWindow_DoubleClick(ByVal sender As Object, ByVal e As System.EventArgs)Handles txtTextWindow.DoubleClick
txtTextWindow.Text = String.EmptyEnd Sub
Private Sub txtRealSpoken_DoubleClick(ByVal sender As Object, ByVal e As System.EventArgs) HandlestxtRealSpoken.DoubleClick
txtRealSpoken.Text = String.EmptyEnd Sub
End Class----------------------------------------------------------------------------------------------------------------------Imports System.Data.OleDb
Public Class frmDictionary
Private Sub frmDictionary_Load(ByVal sender As System.Object, ByVal e As System.EventArgs)Handles MyBase.Load
PopulateListView()End Sub
Private Sub mnuClose_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) HandlesmnuClose.Click
Me.Close()
End Sub
Private Sub PopulateListView()Dim wordCount As Integer = 0Dim objConnection As OleDbConnection = ConnectToDatabase()
If objConnection.State = ConnectionState.Closed ThenExit Sub
End If
Dim strSQL As String = "SELECT * FROM Words"
8/2/2019 An Application That Talks and Listens [Duites]
45/51
B-9
Dim objDataAdapter As New OleDbDataAdapter(strSQL, objConnection)Dim objDataTable As New DataTable("mWords")objDataAdapter.Fill(objDataTable)
'Populate the ListView controllvwWords.Items.Clear()Dim listItem As ListViewItemDim objDataRow As DataRow
For Each objDataRow In objDataTable.RowslistItem = lvwWords.Items.Add(objDataRow.Item("realWord"))listItem.SubItems.Add(objDataRow.Item("digiWord"))wordCount += 1
Next
tssWordCount.Text = wordCount
objDataAdapter.Dispose()objDataAdapter = NothingobjDataTable.Dispose()objDataTable = Nothing
objConnection.Close()objConnection.Dispose()objConnection = Nothing
End Sub
Private Function ConnectToDatabase() As OleDbConnectionDim strConnectionString As String = "Provider=Microsoft.Jet.OLEDB.4.0;" & _
"Data Source=C:\Data\TextWords.mdb;"Dim objConnection As New OleDbConnection(strConnectionString)Try
objConnection.Open()Catch oleDbException As OleDbException
MsgBox(oleDbException.Message, MsgBoxStyle.OkOnly + MsgBoxStyle.Critical, "Error " &oleDbException.ErrorCode)
Catch invalidOperationException As InvalidOperationExceptionMsgBox(invalidOperationException.Message, MsgBoxStyle.OkOnly + MsgBoxStyle.Critical, "Error")
End TryReturn objConnection
End Function
Private Sub mnuDelete_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) HandlesmnuDelete.Click
Dim r_Word As StringDim d_Word As String'Get the data from the selected item in the listviewIf lvwWords.SelectedItems.Count 0 Then
r_Word = lvwWords.SelectedItems(0).Textd_Word = lvwWords.SelectedItems(0).SubItems(1).Text
ElseMsgBox("No item is selected.", MsgBoxStyle.OkOnly + MsgBoxStyle.Exclamation, "No Selection")Exit Sub
End If
Dim objConnection As OleDbConnection = ConnectToDatabase()
If objConnection.State = ConnectionState.Closed ThenExit Sub
End IfDim strSQL As String = "DELETE FROM Words WHERE realWord = '" & r_Word & "' AND digiWord =
'" & d_Word & "'"Dim objCommand As New OleDbCommandobjCommand.Connection = objConnectionobjCommand.CommandText = strSQLobjCommand.CommandType = CommandType.TextDim m_Row As Integer = objCommand.ExecuteNonQuery
8/2/2019 An Application That Talks and Listens [Duites]
46/51
B-10
If m_Row = 1 ThenPopulateListView()MsgBox("Word is successfully deleted" & ControlChars.CrLf & "in the dictionary.",
MsgBoxStyle.OkOnly + MsgBoxStyle.Information, "Deletion")Else
MsgBox("Cannot find the word in the dictionary.", MsgBoxStyle.OkOnly + MsgBoxStyle.Critical,"Not Found")
End If
objCommand.Dispose()objCommand = NothingobjConnection.Close()objConnection.Dispose()objConnection = Nothing
End Sub
Private Sub mnuUpdate_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) HandlesmnuUpdate.Click
Dim r_Word As StringDim d_Word As String'Get the data from the selected item in the listviewIf lvwWords.SelectedItems.Count 0 Then
r_Word = lvwWords.SelectedItems(0).Textd_Word = lvwWords.SelectedItems(0).SubItems(1).Text
ElseMsgBox("No item is selected.", MsgBoxStyle.OkOnly + MsgBoxStyle.Exclamation, "No Selection")Exit Sub
End If
frmUpdateWordPair.txtRealWord.Text = r_WordfrmUpdateWordPair.txtDigiWord.Text = d_WordfrmUpdateWordPair.Show()Me.Close()
End Sub
Private Sub mnuAdd_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) HandlesmnuAdd.Click
Me.Close()frmAddWordPair.Show()
End Sub
Private Sub mnuSortAtoZ_Click(ByVal sender As System.Object, ByVal e As System.EventArgs)Handles mnuSortAtoZ.Click
Dim objConnection As OleDbConnection = ConnectToDatabase()If objConnection.State = ConnectionState.Closed Then
Exit SubEnd IfDim strSQL As String = "SELECT * FROM Words ORDER BY realWord"Dim objDataAdapter As New OleDbDataAdapter(strSQL, objConnection)Dim objDataTable As New DataTable("mWords")objDataAdapter.Fill(objDataTable)
'Populate the ListView controllvwWords.Items.Clear()Dim listItem As ListViewItem
Dim objDataRow As DataRowFor Each objDataRow In objDataTable.Rows
listItem = lvwWords.Items.Add(objDataRow.Item("realWord"))listItem.SubItems.Add(objDataRow.Item("digiWord"))
Next
objDataAdapter.Dispose()objDataAdapter = NothingobjDataTable.Dispose()objDataTable = Nothing
8/2/2019 An Application That Talks and Listens [Duites]
47/51
B-11
objConnection.Close()objConnection.Dispose()objConnection = Nothing
End Sub
Private Sub mnuSortZtoA_Click(ByVal sender As System.Object, ByVal e As System.EventArgs)Handles mnuSortZtoA.Click
Dim objConnection As OleDbConnection = ConnectToDatabase()
If objConnection.State = ConnectionState.Closed ThenExit Sub
End IfDim strSQL As String = "SELECT * FROM Words ORDER BY realWord DESC"Dim objDataAdapter As New OleDbDataAdapter(strSQL, objConnection)Dim objDataTable As New DataTable("mWords")objDataAdapter.Fill(objDataTable)
'Populate the ListView controllvwWords.Items.Clear()Dim listItem As ListViewItemDim objDataRow As DataRowFor Each objDataRow In objDataTable.Rows
listItem = lvwWords.Items.Add(objDataRow.Item("realWord"))listItem.SubItems.Add(objDataRow.Item("digiWord"))
Next
objDataAdapter.Dispose()objDataAdapter = NothingobjDataTable.Dispose()objDataTable = Nothing
objConnection.Close()objConnection.Dispose()objConnection = Nothing
End SubEnd Class----------------------------------------------------------------------------------------------------------------------Imports System.Data.OleDbPublic Class frmAddWordPair
Private Sub btnCancel_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) HandlesbtnCancel.Click
Me.Close()frmDictionary.Show()
End Sub
Private Sub btnSave_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) HandlesbtnSave.Click
Dim r_Word As String = txtRealWord.TextDim d_Word As String = txtDigiWord.Text
If String.IsNullOrEmpty(r_Word) Or String.IsNullOrEmpty(d_Word) ThenMsgBox("Cannot append with an empty string.", MsgBoxStyle.OkOnly + MsgBoxStyle.Critical,
"Empty")
Exit SubEnd If
Dim strConnectionString As String = "Provider=Microsoft.Jet.OLEDB.4.0;" & _"Data Source=C:\Data\TextWords.mdb;"
Dim objConnection As New OleDbConnection(strConnectionString)Try
objConnection.Open()Catch oleDbException As OleDbException
MsgBox(oleDbException.Message, MsgBoxStyle.OkOnly + MsgBoxStyle.Critical, "Error " &oleDbException.ErrorCode)
8/2/2019 An Application That Talks and Listens [Duites]
48/51
B-12
Catch invalidOperationException As InvalidOperationExceptionMsgBox(invalidOperationException.Message, MsgBoxStyle.OkOnly + MsgBoxStyle.Critical, "Error")
End Try
'Search if r_Word in the database already since we'do not allow duplicates on our dataDim strSQLSearch As String = "SELECT realWord FROM Words WHERE realWord = '" & r_Word & "'"Dim objDataAdapter As New OleDbDataAdapter(strSQLSearch, objConnection)
Dim objDataTable As New DataTable("mWords")objDataAdapter.Fill(objDataTable)
'Append the word pairDim strSQL As String = "INSERT INTO Words VALUES('" & r_Word & "', '" & d_Word & "')"Dim objCommand As New OleDbCommandobjCommand.Connection = objConnectionobjCommand.CommandText = strSQLobjCommand.CommandType = CommandType.TextDim m_Row As Integer = objCommand.ExecuteNonQuery
If m_Row = 1 ThenMsgBox("Word pair is successfully added" & ControlChars.CrLf & "into the dictionary.",
MsgBoxStyle.OkOnly + MsgBoxStyle.Information, "Append")End If
objCommand.Dispose()objCommand = NothingobjConnection.Close()objConnection.Dispose()objConnection = Nothing
Me.Close()frmDictionary.Show()
End SubEnd Class----------------------------------------------------------------------------------------------------------------------Imports System.Data.OleDbPublic Class frmUpdateWordPair
Private Sub btnCancel_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles
btnCancel.ClickMe.Close()frmDictionary.Show()
End Sub
Private Sub btnSave_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) HandlesbtnSave.Click
Dim r_Word As String = txtRealWord.TextDim d_Word As String = txtDigiWord.Text
If String.IsNullOrEmpty(d_Word) ThenMsgBox("Cannot update with an empty string.", MsgBoxStyle.OkOnly + MsgBoxStyle.Critical,
"Empty")Exit Sub
End If
Dim strConnectionString As String = "Provider=Microsoft.Jet.OLEDB.4.0;" & _"Data Source=C:\Data\TextWords.mdb;"
Dim objConnection As New OleDbConnection(strConnectionString)Try
objConnection.Open()Catch oleDbException As OleDbException
MsgBox(oleDbException.Message, MsgBoxStyle.OkOnly + MsgBoxStyle.Critical, "Error " &oleDbException.ErrorCode)
Catch invalidOperationException As InvalidOperationExceptionMsgBox(invalidOperationException.Message, MsgBoxStyle.OkOnly + MsgBoxStyle.Critical, "Error")
End Try
8/2/2019 An Application That Talks and Listens [Duites]
49/51
B-13
Dim strSQL As String = "UPDATE Words SET digiWord = '" & d_Word & _"' WHERE realWord = '" & r_Word & "' AND digiWord = '" & d_Word & "'"
Dim objCommand As New OleDbCommandobjCommand.Connection = objConnectionobjCommand.CommandText = strSQLobjCommand.CommandType = CommandType.TextDim m_Row As Integer = objCommand.ExecuteNonQuery
If m_Row = 1 ThenMsgBox("Word is successfully updated" & ControlChars.CrLf & "in the dictionary.",
MsgBoxStyle.OkOnly + MsgBoxStyle.Information, "Update")Else
MsgBox("Cannot find the word in the dictionary.", MsgBoxStyle.OkOnly + MsgBoxStyle.Critical,"Not Found")
End If
objCommand.Dispose()objCommand = NothingobjConnection.Close()objConnection.Dispose()objConnection = Nothing
Me.Close()frmDictionary.Show()
End SubEnd Class----------------------------------------------------------------------------------------------------------------------Public NotInheritable Class AboutBox
Private Sub AboutBox1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) HandlesMyBase.Load
Me.LabelProductName.Text = "Speech Engine"Me.LabelVersion.Text = "Version 1.0"Me.LabelCopyright.Text = My.Application.Info.CopyrightMe.LabelCompanyName.Text = "Deepquest Corp."Me.TextBoxDescription.Text = "An application that demonstrates speech recognition and text-to-
speech."End Sub
Private Sub OKButton_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) HandlesOKButton.Click
Me.Close()End Sub
End Class----------------------------------------------------------------------------------------------------------------------Public Class frmSpeechEngineMain
Private Sub mExit_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) HandlesmExit.Click
EndEnd Sub
Private Sub mAbout_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) HandlesmAbout.Click
AboutBox.Show()End SubPrivate Sub mLaunchTTS_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles
mLaunchTTS.ClickMe.Hide()frmTTS.Show()
End SubPrivate Sub mLaunchASR_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles
mLaunchASR.ClickMe.Hide()frmASR.Show()
8/2/2019 An Application That Talks and Listens [Duites]
50/51
B-14
End SubPrivate Sub pbASR_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles
pbASR.ClickMe.Hide()frmASR.Show()
End Sub
Private Sub pbTTS_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles
pbTTS.ClickMe.Hide()frmTTS.Show()
End SubEnd Class------------------------------------------------------------------------------------------------------------------
8/2/2019 An Application That Talks and Listens [Duites]
51/51
BIBLIOGRAPHY
Dunn, Michael D., Pro Microsoft Speech Server 2007: Developing SpeechEnabled Applications with .NET, Berkeley, California: Apress, 2007
Halvorson, Michael, Microsoft Visual Basic
2008, Redmond, Washington:Microsoft Press, 2008
Barker, F. Scott, Database Programming with Visual Basic
.NET andADO.NET: Tips, Tutorials, and Code, New York: Sams Publishing, 2003
Dobson, Rick, Programming Microsoft Visual Basic .NET for MicrosoftAccess Databases, Redmond, Washington: Microsoft Press, 2003
Willis, Thearon, Beginning Visual Basic2005 Databases, Indianapolis,Indiana: Wiley Publishing, Inc., 2006
Pelland, Patrice, Microsoft Visual Studio2005 Express Edition: Build AProgram Now, Redmond, Washington: Microsoft Press, 2006
Kochanov, Ilya, Yap: Voice-To-Text Translation On Your Cell,http://www.crunchgear.com/2007/09/17/yap-voice-to-text-translation-on-your-cell/
Platzek, Dirk, Jott: Speech-to-Text Mobile Interface,http://www.wunschfeld.net/blog/2007/03/speech-to-text-mobile-interface.html
Turner, Brough, Voice SMSCreating a Service People Will Adopt,http://www.nmscommunications.com/News/NL/TIN/Jan2007/VoiceSMS.htm
Wapedia.com, SMS Language, 2006http://wapedia.mobi/en/SMS_language
Wikipedia.com, SMS Language, Common Abbreviationshttp://en.wikipedia.org/wiki/SMS_language
TextingSlang.com, Common Abbreviations
http://www.textingslang.com/