VoiceXML: A Field Evaluation By: Kristy Bradnum Supervisor: Peter Clayton Presented in partial fulfilment of the CS Honours Project

VoiceXML:A Field Evaluation

By: Kristy BradnumSupervisor: Peter Clayton

Presented in partial fulfilment of the CS Honours Project

------------ VoiceXML: A Field Evaluation ------------ Kristy Bradnum – Computer Science Honours 2004

Overview Objective of Research Background Aims & Motivation Methodology

Tools Results Conclusions Questions


Objective of Research

My project in a nutshell:An evaluation of VoiceXML 2.0, using a range

of platforms, looking specifically at it’s maturity as a technology and it’s status as an industry standard.

Objective of Research


Background

Overview of Speech Technology Overview of VoiceXML

HistoryRole

Background


Overview of Speech Technology

Conversion between spoken word and binary Output

Previously: pre-recorded prompts Today: speech synthesis (TTS)

Input Previously: DTMF (pressing keys on the phone) Today: speech recognition (ASR)

Background >> Overview of Speech Technology


History of VoiceXML

AT&T Bell’s PhoneWeb project Lucent’s Phone Markup Language Motorola’s VoxML IBM’s SpeechML

VoiceXMLversion 2.0full W3C recommendation16 March 2004

Background >> Overview of VoiceXML


Role of VoiceXMLBackground >> Overview of VoiceXML


Motivation 2002

Mya Anderson’s Field Investigation of VoiceXML 1.0

New technology Unstable Unsuccessful

Now VoiceXML 2.0 = W3C standard Nortel: maturity increasing, widely accepted Jackson: already mature

Project Aims & Motivation


Project Aims

Investigate these claims Examine:

maturity of VoiceXML 2.0 as a technology it’s status as an industry standard

Project Aims & Motivation


Methodology

Approach Tools

WebSphereOptimTalkBeVocal Café

Analysis Cross-Platform Analysis

Methodology


Approach

Iterative approachSet goalEvaluate outcomesDetermine next goal

ROSS prototypeRelevant to RhodesProduct secondary to investigation

Methodology >> Approach

Inadequate


Tools

3 approaches [Seth]:BuyRentBuild

3 environments [Beasley et al]:HostedSimulatedWeb-based

Methodology >> Tools


Project Tools

WebSphere Studio Application Developerwith Voice Toolkit

OptimTalk 0.9.1 BeVocal Café 2.5

Methodology >> Tools


WebSphere

IBM’s WebSphere Studio Application Developer with Voice Toolkit plug-in

“Buy” approach Voice Toolkit also includes other features:

CCXML developer NLU model maintenance

Call Flow Builder Grammar developer Pronunciation Builder

but Version problems

Methodology >> Tools >> WebSphere


OptimTalk

Simple VoiceXML platform Desktop standalone development environment Set of libraries interpret W3C SIF markup

languages Tailored towards research Command line application Requirements: microphone and speakers

Methodology >> Tools >> OptimTalk


BeVocal Cafe

Good background Web-based development environment Hosted platform “Rent” approach Tools:

VoiceXML CheckerVocal Scripter

Methodology >> Tools >> BeVocal Café


Analysis

Each platform studied separately 15 examples for OptimTalk

Basic + Blackjack

10 projects for BeVocal Café Miller’s “10 Projects to Voice-Enable Your Web Site”

Methodology >> Analysis


Cross-Platform Analysis

BeVocal Café’s projects in OptimTalk OptimTalk’s examples in the Café Run amendments through original platform

Methodology >> Cross-Platform Analysis


Results

Platform IndependenceOptimTalkBeVocal Café

Grammars Design Considerations Platform Certification

Results


Platform Independence

Environments: steep learning curve Extensible tag set

Limits platform independenceProprietary extensionsSome features added – some left out

Example code usually worked

Results >> Platform Independence


OptimTalkResults >> Platform Independence >> OptimTalk


Results >> Platform Independence >> OptimTalk

OptimTalk


OptimTalk

Speech recognition component erratic Built-in grammars not supported in OptimTalk

type attribute of <field> Boolean = yes / no grammar Number = ?

No <say-as> Phone numbers (from database)

Results >> Platform Independence >> OptimTalk


BeVocal Café

More mature Very confusing error messages Many proprietary extensions

Results >> Platform Independence >> BeVocal Café



Grammars

Built-in grammars not supported in OptimTalk

OptimTalk’s grammar not supported by BeVocal

W3C passes responsibility to SIFPlatforms should support ABNF of SRGS

Results >> Grammars


Design Considerations

No long menus Pronunciation

Different voices for TTS ROSS by LH Michael vs Microsoft Sam

Be careful with ASR Ties in with grammar

No break in <enumerate> in OptimTalkSo lists run together

Results >> Design Considerations


Platform Certification

VoiceXML Forum’s Platform Certification ProgramTest Suite v1.0 ≈ 700 test programsTo check compliancy with VoiceXML 2.03 platforms passed (in September)

NVP VoxPilot Open Media Platform VoiceGenie Platform

Results >> Platform Certification


Conclusions

Learnt a lot about Speech technology Language does seem fairly mature now

Fewer extensionsMore complete as a standard

Still not quite stable

Conclusions


Conclusions

Give platforms time to catch up Still hurdles in development

Especially in South Africa Possibly better for commercial enterprises

Not for research But improving all the time

Conclusions


Future work

Windows vs Linux comparison

Look at W3C Speech Interface Framework as a whole

VoiceXML 2.1 is on it’s way

Conclusion >> Future Works


QuestionsQuestions ???

Documents

VoiceXML: A Field Evaluation By: Kristy Bradnum Supervisor: Peter Clayton Presented in partial fulfilment of the CS Honours Project