AQUAINT June 2002 Workshop June 2002 Just-in-Time Interactive Question Answering Sanda Harabagiu: PI Language Computer Corporation

June 2002 AQUAINT June 2002 Workshop

Just-in-Time Interactive Question Answering

Sanda Harabagiu: PI

Language Computer CorporationLanguage Computer Corporation

AQUAINT June 2002 Workshop - JITIQA

Just-in-Time Interactive Question Answering

People Sanda Harabagiu, PI Dan Moldovan Larry Daffner John Williams


Objective of Research Project

Address the interactive aspect of Q&A systems

design and implement a dialog shell that can be used with any Q&A system

Extension of work in factual Q&A with dialog capabilities already tested in call center systems


Novelty

Usage of just-in-time citations of other analyst’s similar Q&A sessions: Just-in-time citations are provided by Just-in-Time

Information Seeking Agents (JITISA) JITISA: intelligent agents that proactively search and

retrieve information that might be useful without requiring any action from the professional analyst Non-intrusive software agents that monitor the dialog

contexts generated when different agents interrogate Q&A systems


Tasks Proposed


Tasks Proposed


Building a Dialog Manager

Wizard-of-Oz Experiments Extract the features from the Wizard-of-Oz Data Implement the dialog manager with a

strategy that optimizes the features


Wizard of Oz Experiments

Year 1: Open-Curtain experiments Main focus: gauge the acceptability of the system-user

interaction. General assumption = no actually system exists,

develop a detailed conception of the system such that the responses to the user’s questions are known

Year 2: Study annoyance factors


Our assumption

Build a “transition network” of questions, answers and follow-up questions Completely unrestricted dialog on the user’s side

The user asks any questions Be able to refine the transition network by

allowing different turns


Open-Curtain Wizard-of-Oz experiment

Start with a complex question (which eventually does not return any acceptable answer)

User1

InformationSatisfaction

User2

DifferentPerspectives

User1

TemporalPerspectives

Q1

Q2

Q3

Q2

Q1

Q3

Q1

Q2 Q3 Qn

10AM 11AM 2PM




Semantic Representations for Interactive Q/A

Example: Why is there so much violence on the West Bank now ?

Processing: Syntactic Parse- binary dependencies

SemanticRepresentation

CAUSEviolence

West Bank

now


Novel Representation

Frame-like template:Event: template

slots:event: violenceentity: agent ?? patient ?? location: West BankMotivation: the answertime: now May 2, 2002

The same template can be generated by another reformulation of the original question:

“What are the causes of the current violence in the West Bank ?”

information comingfrom the answer

now



Ideal Case

Create a complex “transition network” containing all possible follow-up questions and all possible dialog paths

Q2

Q1

Q3



Template River Flow

Question: What are the causes of the current violence in the West Bank?

STATEEVENT = Template Name Event: violence Entity: Israel Location: West Bank Time: now

Answer: Large parts of the West Bank and Gaza remain under Israeli military occupation.

New slot: Motivation: Israeli occupation of West Bank Filler is a paraphrase of the answer

Template (Question)


Follow-up Question

Possibilities:

1) -same event

-a new slot

2) Relation-slot from Template

-other entity/event

-…

Possible Motivator: Elaboration


Color Code

1) Same slot, Same filler yellow blue

2) Same slot, Different filler red

River of colors



How are templates generated?

2-Phase Process First, templates are proposed by a linguist

novice, having a clear goal. (Larry) Second, they are refined by John and Sanda.

Phase I: Larry develops an algorithm of identifying fillers for each template.Sources: Question Parse, Semantic Form

Answer Parse, Semantic Form


Phase II

Generate template slots automatically (John Williams)Sources: Question Parse, Semantic Form

Answer Parse, Semantic FormDialog annotations, NE RecognizerDialog motivators and Context

modeling


Template Generation

Larry has created several types of templates:

STATE

INVOLVEMENT CONTRAST

EVENT RELATION ATTITUDE

ATTRIBUTES-definition questions

MOTIVATION-causality questions


Example

RELATION Templates:Generated by:

What is Ariel Sharon’s connection to the Palestinians?How do Israel’s Likud and Labor parties differ on peace with Palestine? ATTITUDE Templates:What is Ariel Sharon’s position on Palestine?Why is Jerusalem important to Palestinians?What were Ehud Barak’s views on peace with the Palestinians?Why did the Lebanese Phalange hate the Palestinians?How do the Hashemites view the Palestinian Authority?


Research Challenges

How many template kinds are sufficient? How should they be organized? What is the relationship between template

kinds, question classes and dialog context?


Novelties

1) Open-Domain Questions

2) No pre-defined task

-Task needs to be “guessed” as the dialog evolves

3) A general template algebra needs to be implemented to:

a) Deal with templates created in an ad-hoc manner

b) Help define the template motivators

c) New set of motivators (e.g. elaboration, tangent)


Phase III and IV

Phase III: Use two analysts to conduct a dialog starting with the same first question and the same general goal. Study the completeness and efficiency of the dialog motivators as well as the possible overlap.

Phase IV: Allow mixed initiative.


Enhance the Q/A Accuracy

Definition questions Causality questions Causality-Effect/Involvement questions Attitude questions


Thank You !Thank You !

Documents

AQUAINT June 2002 Workshop June 2002 Just-in-Time Interactive Question Answering Sanda Harabagiu: PI Language Computer Corporation