VOIP Colloquium SIP SPEECH PHONE

VOIP Colloquium

SIP SPEECH PHONE

Mathieu BenoitSai Nithin Singh C.Supervisor: Prof. Carol Davids

VOIP NEW ARCHITECTURE : IMS

IP LAYERIP LAYER

IP Multimedia Subsystemdefined by

3GPP and 3GPP2 standards.

Convergence of voice and data services!

VOIP TECHNOLOGY : CONVERGENCE???

New age in telecommunications :

ØØNEW TECHNOLOGY (Wireless, VoIP,…)NEW TECHNOLOGY (Wireless, VoIP,…)

ØØNEW APPLICATIONS (Tele banking, Tele medicine,…)NEW APPLICATIONS (Tele banking, Tele medicine,…)

ØØNEW POSSIBILITIES NEW POSSIBILITIES

ØØNEW PROMISESNEW PROMISES

ØØNEW HOPES…For disabled peopleNEW HOPES…For disabled people

VoIP- A step forward for the hearing impaired

VoIP offers us a unique opportunity to significantly improve communication :

• Not only for those of us who can hear, but also for the audibly challenged.

• A unique opportunity to bridge the gap that has long-existed between PSTNtext phone users in different countries.

• Finally, have a 21st Century Phone offering services with no barriers.

Existing solution in the market …

nTTY (Text Telephone)nTDD (Telecommunications Device for the Deaf)

nTelephone Relay Assistance Center Highly trained professionals take calls from TDDs and relay the messages by telephone to hearing people or take telephone calls from hearing people and relay the messages via TDD to the hearing impaired e.g.: The 1-800-743-3333 provide the relay service for Sprint

n VOIP : Few services not really efficientOnly few concepts using video or IM messaging…

Hearing Hearing ImpairedImpaired

Hearing Enabled Hearing Enabled Soft / Hard PhoneSoft / Hard PhoneINTERNETINTERNET

or PSTNor PSTN

NO VOICETEXT

Old problem… new solution.

How VoIP Can Connect these two?

SIPSIP--PhonePhoneSpeechSpeech

Synthesizer//Synthesizer//RecognizerRecognizer

VOICETEXT

SIP SPEECH PHONE: Text To Speech

SIP PROXY SIP PROXY SERVERSERVER


SynthesizerSynthesizer////RecognizerRecognizer


TEXT VOICE

Hearing Enabled Hearing Enabled Soft / Hard PhoneSoft / Hard Phone

Hello?

INTERNETINTERNET// PSTN// PSTN

SIP SPEECH PHONE: Speech To Text

SIP PROXY SIP PROXY SERVERSERVER




TEXT VOICE

INTERNETINTERNETPSTNPSTN

Hearing Enabled Hearing Enabled Soft / Hard PhoneSoft / Hard Phone

Yes!

VoIP Lab configuration

VoIP Lab configuration

SOFT PHONE 1SOFT PHONE 1

•• Windows PlatformWindows Platform

•• Java JDKJava JDK

•• Java Media FrameworkJava Media Framework

•• IDE EclipseIDE Eclipse

•• EtherealEthereal

SIP PROXYSIP PROXY

•• Linux PlatformLinux Platform


•• IM ManagerIM Manager


SOFT PHONE 2SOFT PHONE 2

•• Windows PlatformWindows Platform


•• Java Media FrameworkJava Media Framework

•• IDE EclipseIDE Eclipse


Software Engineering

DEVELOPMENT METHODOLOGY:

-Software prototype ( Fast Development and validation tests)

“Proof of Concept”“Proof of Concept”

- Programming Language : Java

SPECIFICATION (based on RFC) :

• RFC 3351 - User Requirements for the Session Initiation Protocol (SIP) in Support of Deaf, Hard of Hearing and Speech-impaired Individuals

•RFC 3261- SIP: Session Initiation Protocol

• RFC 3550 - RTP: A Transport Protocol for Real-Time Applications

•RFC 3428 - Session Initiation Protocol (SIP) Extension for Instant Messaging

USERUSER

SPEECHSPEECHAPPLICATIONAPPLICATION

CALL/SESSIONCALL/SESSIONMANAGERMANAGER

MEDIAMEDIAPROCESSINGPROCESSING

SIPSIPINFRASTRUCTUREINFRASTRUCTURE

Project Architecture

SIP Architecture

SIPSIPPHONEPHONE

• A SIP Soft Phone written in JAVA with voice and instant messaging client

•Open Source

• Jain-Sip-Applet-Phone – based on JAIN SIP APIs

• Use Java Media Framework (JMF).

• SIP user-agent with audio support RTP

SIPSIPPROXYPROXY

SERVERSERVER

• A SIP Proxy Server written entirely in the Java

• Open Source

• JAIN-SIP Proxy Server – based on JAIN SIP APIs

• Presence Server capability

• Registrations uploading

• Trace viewer

SPEECH ARCHITECTURE

SPEECHSYNTHETIZER

SPEECHSPEECHSYNTHETIZERSYNTHETIZER

• A speech synthesizer written entirely in the Java

• Open Source

• FreeTTS 1.2 – based on Java Speech APIs

• A medium quality, unlimited domain, 16kHz diphonevoice, called kevin16

SPEECHRECOGNIZER

SPEECHSPEECHRECOGNIZERRECOGNIZER

• A speech recognizer written entirely in the Java

• Open Source

• Sphinx-4 – based on Java Speech APIs

• A Small Vocabulary with approximately 100 words

THE BUILDOUT CHALLENGE 1 : INTEGRATION

SPEECHRECOGNIZER

CODE

SPEECHSPEECHRECOGNIZERRECOGNIZER

CODECODE

SIPSIPPROXYPROXY

SERVERSERVER

Libraries,Configuration,

…

SPEECHSYNTHETIZER

CODE

SPEECHSPEECHSYNTHETIZERSYNTHETIZER

CODECODE


…

SIPSIPPHONEPHONECODECODE


…



THE BUILDOUT CHALLENGE 2: STREAMS

SIPSIPPROXYPROXY

SERVERSERVER





RTP STREAM AUDIO

SIP signa

l

<IM M

essag

e>SIP signal

<IM Message>

Proxy and Speech Engine Features

3 different voices:• an 8khz diphone, male, US English voice • a 16khz diphone, male US English voice • a 16khz limited domain, male US English voice

Phone Features

Manage contact with Add/Remove

Action

Menu• Configuration• Register• Unregister• Exit

Display the list of contacts

Phone Features

Send an IMUsing Speech

Synthesis

Make a phone CallAudio/RTP

Speech recognitionPush-to-talk

Chat BoxDisplay the

conversation

Conclusion

STATE OF THE PROJECT:STATE OF THE PROJECT:

SIP Speech Phone is working in his first version

Speech synthesis and recognition are functional

Integration of complete

Handling different stream and push audio synthesis in real stream call (in progress)

FUTURE IMPROVEMENT:FUTURE IMPROVEMENT:

Utilization of continuous speech recognition

Extend the vocabulary

Add more voices

Add Video

Use of Avatar ( Language of signs)

Thank youfor your

attention.

Documents

VOIP Colloquium SIP SPEECH PHONE