Towards Conversational Human Computer Interaction

Dept. of Computer Science University of Rochester Rochester, NY 14627

By: James F. Allen, Donna K. Byron, Myroslava Dzikovska George Ferguson, Lucian Galescu, Amanda Stent

Present By: Nourhan Younis

MOIS 466- Human Computer Interaction Mini-Project 1

What is a dialogue system?

A computer intended to converse with a human, with a coherent structure.

Dialogue System

Limit communication Increasing efficiency

of service Making speech more

predictable Easier recognition

and language processing

Human Conversation

Basing HCI on human conversation

More interaction and complexity

BUT People claim it won’t

be as effective as graphic user interfaces (GUIs)

Conversational Interfaces Vs. GUIs

Limitations of Graphic User Interfaces:

The size of the device used Some tasks requires the use

of hands and eyes More flexible Reduces the training time The provide a set of

operations with many complex, sequential tasks

Adds high level commands with many options

Requires lots of training

Considerations

It can’t totally replace GUIs because: The user will have to remember a long

set of verbal commands to perform a task (ex. Menus)

The Conversational inter- faces, however, would provide the opportunity for the user to state what he/she wants to do in his/her own terms, just as he/she would do to another person, and the system takes care of the complexity.

Conversation Interfaces Conversational Interaction will not depend on a

series of commandsBut Interaction involves:

Defining and discussing tasks Exploring ways to perform the task Collaborating to get it done

All interactions are contextually interpreted with respect to the interactions performed to this point, allowing the system to anticipate the user’s needs and provide responses that best further the user’s goals

Dialogue Task Complexity

Hypotheses H1: Practical Dialogues

Conversational competence required for practical dialogues, although still complex, is significantly simpler to achieve than general human conversational competence.

H2: The Domain IndependenceWithin the genre of practical dialogue, the bulk of the complexity in the language interpretation and dialogue management are independent of the task being performed

A Fragment of Practical Dialogue

Challenges For Practical Dialogue

1. Parsing language in practical dialogues

2. Integrating dialogue and task performance

3. Intention recognition

4. Mixed initiative dialogue

1.Parsing Language In Practical Dialogues Need to produce a detailed semantic

representation the captures what the user meant by the utterance Put limitations on the different meanings of the words Encode restrictions to make sure words are in the

right context

Spoken language is not sentence based, rather a single utterance may realize a sequence of communicative acts called Speech Acts The grammar used is an act description rather than

sentence structures

Speech Acts

EXAMPLE: “OK, let’s do that then

send a truck to Avon” OK: an

acknowledgement Let’s do that: an

acceptance Send a truck to

Avon: a request

2. Integrating Dialogue and Task Performance We need dialogue systems that can be

adapted to any practical tasks Requires an AGENT BASED FRAMEWORK

Dialogue Systems:▪ Knows about the tasks that needs to be accomplished ▪ The generic system is specialized to the particular

domain by integrating domain specific information Broker : link between the dialogue system and

backend Backend: a set of agents providing services

The Agent Based Architecture

Abstract Problem Solving and Domain-Specific Task Models The key to this enterprise is the development of an

abstract problem solving model with: ▪ Objectives▪ Solutions▪ Resources▪ Situations

As utterances can mean different things in different domains:

▪ A domain specific task model: provides mapping from the abstract problem solving model to the operations in a particular domain by specifying what things count as objectives, solutions, resources, and situations

3. Intention Recognition

Determining what the user wants to say by saying the utterance

Example: Emergency System that will be used to evacuate people from an island in the face of a hurricane

Example

Intention Recognition (Cntd) That example shows

different interpretations of those utterances

Reasoning used, requires information about the task to identify which makes sense rationally in the current situation

The challenge here is to match the intentions with the context of the the question asked

4. Mixed Initiative Dialogue Practical human dialogue involves mixed

initiative interaction i.e. dynamic exchange of control of the dialogue flow, increasing efficiency, effectiveness; enabling both participants'’ needs to be met

For systems that ask questions it can work in simple tasks, like long distance dialing

For complex tasks: you may often need to get

through irrelevant interactions before getting the information you need.

Limitations of Frame Based System The user has to supply information for

the system, until there is sufficient information for the system to perform the task

BUT The user might not know what

information he might still needs to supplySo Currently system provide limited mixed

initiative interactions

Conclusion There are still lots of technical issues

remaining to overcome to increase the efficiency and effectiveness of the dialogue based user interfaces.

Thank You