VoiceXML:

Forms, Menus, Grammars, Form Interpretation Algorithm

Acknowledgements

Prof. Mctear, Natural Language Processing, http://www.infj.ulst.ac.uk/nlp/index.html, University of Ulster.

Overview

Forms Form items Form attributes

Fields Field Attributes Prompts

Menus Form Interpretation Algorithm

Forms

Interpreting a form item (e.g. field) Select and play one or more prompts Collect user input Throw events (e.g. noinput) Interpret <filled> actions (procedural logic that executes when

input item variables are assigned)

Form attributes id name of the form e.g. <form id=“mainmenu”>

reference to another form <goto next=#validate”> (validate is the name of a form)

scope scope of the form’s grammars e.g.<scope = “dialog”>

Form Items

Input Items

<field> input using speech or DTMF

<record> records user’s input as audio clip

<transfer> transfers user to another telephone number

<object> invokes a platform-specific object e.g. voice authentication program

<subdialog> invokes another dialog

Control Items

<block> procedural statements for prompting and computation

<initial> controls initial interaction in a mixed initiative form

Example: A form for getting student information

C: Say students, courses, or reports

H: Students.C: What is the student’s

name?H: John Scott.C: What is the student’s id?H: 654556.C: What information do you

require?H: Address

The system prompts the user for input in order to fill in the required fields.

The user responds to the system’s prompts.

The system accepts the responses if they match the grammars for each field.

Fields

The form for the previous example consists of 4 fields: choice, student_name, student_id, info

Each field has a prompt and a grammar Each field declares a variable which usually has the

value of ‘undefined’ when the field is first interpreted Field attributes

name e.g. <field name=“choice”>

expr the initial value of the form item variable

cond a conditional expression e.g. to activate a field only when another field has been filled

type e.g. <field name=“student_id” type=“digits”>

Headers (see next slide)

<form id="mainmenu" scope="dialog"> <field name="choice" modal="false"> <prompt version="1.0"> Welcome … To begin say one of the following: students, courses, reports

</prompt> <grammar type="application/srgs+xml" root="choice" version="1.0"

mode="voice"><rule id="choice" scope="public"><one-of><item> students </item><item> courses </item><item> reports </item></one-of></rule></grammar>

</field> </form> </vxml>

studentsystem1a.vxmlExample

HeadersIBM Voice Toolkit:

<?xml version="1.0" encoding="iso-8859-1"?>

<!DOCTYPE vxml PUBLIC "-//W3C//DTD VOICEXML 2.0//EN" "vxml20-1115.dtd">

<vxml version="2.0" xml:lang="en-GB" xmlns="http://www.w3.org/2001/vxml">

<meta name="GENERATOR" content="Voice Toolkit for WebSphere Studio" />

Bevocal:

<?xml version="1.0" ?>

<!DOCTYPE vxml

PUBLIC "-//BeVocal Inc//VoiceXML 2.0//EN"

"http://cafe.bevocal.com/libraries/dtd/vxml2-0-bevocal.dtd">

<vxml version="2.0" xmlns="http://www.w3.org/2001/vxml“ xml:lang="en-US">

Tutorial Exercises

Exercise 1: Load and run the file ‘studentsystem1a.vxml’. The system will prompt you to say one of: students, courses, details.

Exercise 2: Create a file called ‘studentdetails.vxml’. The file should have fields to elicit a student's name and the name of a course. You will need to construct grammars in each field containing a few names of students and courses.

<prompt> (1)

Bargein If an implementation platform supports bargein, the

application author can specify whether a user can interrupt, or "bargein" on, a prompt using speech or DTMF input

To disable “bargein”:<prompt bargein = “false>

Timeout Specifies the interval of silence between prompt and user

input e.g.<prompt timeout = “30s”>

Default timeout value in Voice Toolkit is 7s. Timeout can be specified for individual prompts or set as a

property at a higher level e.g. in the root document

<prompt> (2)

Count Used for prompts that change with each attempt to elicit input

e.g. to provide more help<prompt count=“1”> what is the student’s id? </prompt>

<prompt count=“2”> please say the student’s id. It consists of 6 digits </prompt>

Conditional prompt Specifies that a prompt is only to be spoken if the condition is

true

<prompt cond=“now &lt; 12"> Good morning </prompt>

<prompt cond=“now &gt;= 19"> Good evening </prompt>

Menus

A <menu> is a shorthand version of a form with a single field that prompts the user to make a choice and then transitions to another place (e.g. another dialog) based on that choice

Menu attributesid e.g. <menu id=“mainmenu”>scope grammar scope of the menu dtmf when set to true, first nine choices are

given the implicit values ‘1’, ‘2’, etc (unless explicitly specified)

accept default “exact” – defines the exact phrase to be recognised“approximate” – allows approximate

recognition

Menu example

C: Welcome to the Student System main menu

Please choose one of the following options:

You can say student details, course details, or records

H: Student details.

This example uses a menu to present a choice to the user.

The user selects one of the options and the application moves to the next dialogue state.

<?xml version="1.0" encoding="iso-8859-1"?>

<!DOCTYPE vxml PUBLIC "-//W3C//DTD VOICEXML 2.0//EN" "vxml20-1115.dtd">

<vxml xmlns="http://www.w3.org/2001/vxml" xml:lang="en-GB" version="2.0">

<menu id="mainmenu" scope="dialog" accept="exact" dtmf="false">

<prompt version="1.0"> To begin say one of the following: </prompt>

<choice next=“studentdetails.vxml"> students </choice> <choice next=“coursedetails.vxml"> courses </choice><!-- remaining choices --></menu> </vxml>

examplemenu1.vxml

Choice

Can specify a speech grammar – may be automatically generated

Can specify a DTMF grammar Can use contents to form the <enumerate> prompt

string Specifies an event to be thrown or URI to go to when

the choice is selected

<choice next=“reports.vxml"> reports </choice>

Where to go to Grammar for this choice

DTMF in Menus

DTMF only

<menu>

<property name=“selected” value=“dtmf” />

<prompt> For reports press 1, etc </prompt>

<choice dtmf=“1” next=“reports.vxml” />

</menu>

DTMF or speech

<menu><prompt> For reports press 1 or say ‘reports’ </prompt>

<choice dtmf = "1" next=“reports.vxml">reports </choice>

</menu>  

State transitions and branching

<form>

<!– ‘choice’ field as in ‘studentsystem1a.vxml -->

<filled><if cond="choice == 'students'"><goto next="studentdetails.vxml"/><elseif cond="choice == 'courses'"/><goto next="coursedetails.vxml"/><else/> <goto next="reports.vxml"/></if></filled>

</form>

comment

<filled> elementExecutable code whenfields are filled ConditionalsRequires documents tobe specified

Tutorial Exercises

Exercise 3: Save the file ‘studentsystem1a.vxml as studentsystem1b.vxml. Insert the additional code after the end of the field </field> and before the end of the form </form>. Note: To make this example work, it will be necessary to

create dummy files for the options for the transitions specified in the <filled> element.

Exercise 4: Load and run the file ‘examplemenu1.vxml’

Form Interpretation Algorithm

Verification – verifying that the recognition result corresponds to what the user said boolean builtin grammar

Variables referencing variables declaring variables

The Form Interpretation Algorithm (FIA)

viewdetails.vxml<form id="student_details" scope="dialog"><!– field to collect student_name -->

<field name="confirm" type="boolean" modal="false"><prompt version="1.0">you want details on <value expr="student_name"/> </prompt><filled><if cond="confirm"><prompt version="1.0"> looking up details on <value

expr="student_name"/> </prompt><else/>let's try again<clear namelist="student_name confirm"/></if></filled></field></form>

Referencing a variable

you want details on <value expr="student_name"/>

“student_name“ is the name of a field the expression retrieves the value of the field, i.e. the

value that was recognised by the system this value is not necessarily the value that the user

spoke e.g. if the input was misrecognised

More on Variables<var> - declare a variable and possibly assign an initial value to it

<block>     <var name="status"/>     <!--  Declare the variable status, but do not assign an initial value to it.-->     <var name="amount" expr = "13"/>     <!-- Declare the variable amount and assign an initial value of 13 to amount. -->

</block>

<assign> – assign a value to an existing variable

<block>     <var name="status"/>     <assign name = "status" expr = "completed"/>      <var name="amount" expr = "13"/>     <assign name = "amount" expr = "amount + 1"/>

</block>

Variables: More examples

<var name = "revised_balance" expr = "initial_balance + use_fee"/><!-- revised_balance is declared a variable whose value is calculated when the variable is declared -->

<var name = "days_of_the_week" expr="new Array()"/><!-- days_of_the_week is declared to be an array with undefined values -->

<var name = "days_of_the_weekend" expr = "new Array ('Saturday', 'Sunday') " /><!-- days_of_the_weekend is declared to be an array and is initialized with string values Saturday and Sunday -->

Tutorial Exercises

Exercise 4: Load and run the file ‘viewdetails.vxml’. In response to the prompt 'Please say the student name',

you can say one of: John, David, Rosemary, Jennifer. In response to the prompt asking for confirmation, you can

say either 'yes' or 'no'. Try both options.

Exercise 5: Adding verification to studentdetails.vxml Using the file viewdetails.vxml as a basis, save the file

‘studentdetails.vxml’ as ‘studentdetails_confirm.vxml’. Amend the file so that the system confirms the values elicited (studentname, coursename). If the user confirms, then the application terminates, otherwise it goes back to elicit the values again.

Form Interpretation Algorithm

The form interpretation algorithm (FIA) determines the order of execution in a VoiceXML form or menu

The FIA cycles through all of the fields in the form, asking the caller to supply values for each unfilled field. 

The FIA provides the default sequence of processing fields within a form; and thus hides the procedural aspects of form processing the from the dialog designer.  

Sometimes the fields of a form are “reset.”  In this case, the FIA will select the reset field during the next cycle so all fields are filled eventually.   

Form Interpretation Algorithm (FIA)

Amount

Source

Destination

Date

Confirm

Amount

Source

Destination

Date

Each field has a prompt anda grammar (for user input)The field variable is initiallyset to ‘undefined’

When a field has been completedthe field variable is set to ‘true’.The Form Interpretation Algorithmskips fields that are ‘true’

If a field value is disconfirmedthe variable is re-set to ‘undefined’

Confirm

Amount

Source

Destination

Date

Amount and Destination were disconfirmed

Their values are re-set to ‘undefined’

The FIA visits the ‘Amount’ and‘Destination’ fields again until theirvalues are set to ‘true’

FIA (continued)

FIA: More detail

Each variable (form-level <var> elements and form item variables) is initialised to ‘undefined’

The first form item whose guard condition is false (undefined) is chosen to be visited.

If no guard condition is false, then the FIA does an implicit <exit> operation.

<block> element form item variable set to true (cannot be visited again) content evaluated no input is collected

FIA in operation

Field

Name

Course

Level

Date

Before Mid Name disconfirmed

undefined

undefined

undefined

undefined undefined

undefined

true

true

undefined

true

true

true

More detail

Note that the FIA selects one item on each iteration e.g. given the 4 fields above without any conditions on the fields:

Iteration By-passed item Selected item1 None name2 name course3 name course level4 name course level date5 course level date name6 name course level date none: FIA exits

<field name="confirm" > <grammar type="application/srgs+xml" root = "yes_no“ version=“1.0”> <rule id = "yes_no" scope = "public"> <one-of> <item>yes</item> <item>no</item> </one-of> </rule> </grammar> <prompt> You want details on <value expr=“student_name"/> </prompt> </field>

<filled> <if cond = "confirm == 'no'" >

<prompt> Let's try again </prompt> <clear namelist = “student_name confirm"/>

<!-- Because the variables have been cleared, the Form Interpretation Algorithm processes the form again -->

<else/> <prompt> Let's continue</prompt> </if>

</filled>

FIA example

The <filled> element

Defines how to process input-form-item variables after they have values e.g. to check that the input is correct or to verify it with the

user (example: arrival time not earlier than departure time)

If the checking involves only the value of a single field, the <filled> element is positioned within the <field> element. 

If the check involves more than a single input-form-item variable, the <filled> element is placed within the <form> element.

Example: Validating the input

<form id=“get_details">

<!– The system has collected values for ‘source’ and ‘destination’ and checks that these are different

-->

<filled><if cond = “source == destination"> <prompt> the destination and source accounts must be different</prompt><clear namelist = “source destination"/></if>

</filled>

</form>

<if>

<if cond = "confirm == 'no'" > <prompt> let's try again </prompt> <clear namelist = “student_name confirm"/>

<else/> <prompt> Let's continue</prompt>

</if>

“confirm” is the name of a field The condition checks if the value of “confirm” is ‘no’ <clear> re-sets variables to ‘undefined’ namelist specifies which variables are to be re-set to

‘undefined’, otherwise all variables within the current form are re-set

Tutorial Exercise

Exercise 5

1. Save the file ‘viewdetails.vxml’ as ‘newviewdetails.vxml’

2. Delete the line <clear namelist = "student_name confirm" /> and see what happens.

3. Insert the line but delete the value ‘confirm’.

4. Try again, this time with ‘confirm’ included and ‘student_name’ deleted.

Exercise 6: Asking which item to confirm

Scenario: 1. User has supplied values for studentname and

coursename

2. System verifies and user says ‘no’

3. System asks: which do you want to change ‘the student name or the course name?’

<field name="change"><prompt>which item do you wish to change: the student name or the course name?</prompt><!– insert grammar for each of the 2 parameters -->

<filled><if cond="change == ‘student name'"><clear namelist=“student_name confirm"/><elseif cond="change == ‘course name'" /><!– etc. --></if></filled>

<field name=“confirm” >

<prompt> so you want to details on <value expr=“student_name"/> taking <value expr=“course_name"/>

</prompt>

<!– grammar for ‘yes/no’ -->

<filled>

<if cond="confirm==‘no’">

<goto nextitem = "change" />

<else />

<assign name="change" expr="true" />

</if>

</filled>

</field>

<field name="change">

<prompt>

which item do you wish to change: the student name or the course name?

</prompt>

<!– insert grammar for each of the 2 parameters -->

The line <assign name=“change” expr = “true” /> makes the value of “change” be ‘true’ – this ensures that the field will not be visited by the FIA.