Dialogue systems at KTH

The August project

• Part of the Stockholm Cultural Capital of Europe '98 program

• Swedish spoken dialogue system with an animated agent

• Named after the 19th century author August Strindberg

• Built with speech technology components developed at TMH

• Designed to be easily expanded and reconfigured

• Communicates using synthetic speech, facial expressions, maps, tables and a thought balloon

• Used to collect spontaneous speech data

Key features of the August system

The topics of the August system

• Stockholm (events and general information)

• Yellow pages

• KTH and speech technology

• August Strindberg

• Greetings and social utterances

• Comments about the system capabilities and the discourse

A sample video of the system environment

There are 24,000 islands in the Stockholm archipelago

Can you see me?We are now in the center of Stockholm

Hello AugustHello!How are you today?Excellent as always!What can I ask about?I can tell you about

restaurants in StockholmCan you recommend a

restaurant in the neighborhood

There are over 700 restaurants in Stockholm

Is there one close by?You must specify a streetAre there any restaurants

on Blekingegatan?The information is shown

on the mapWhere are we?But we are here!Where?I did not understand that!Where are we?But we are here!Are we in Kulturhuset?

My life cannot be measured in

terms of days and years!

My life cannot be measured in

terms of days and years!

The August system

What is your name?I call myself Strindberg, but

I don’t really have a surname

When were you born?Strindberg was born in 1849What do you do for a living?I can answer questions

about Strindberg, KTH and

Stockholm

How many people live in Stockholm?

Over a million people livein the Stockholm area

Do you like it here?People who live in glass

housesshould not throw stones

Yes, that was a smart thing to say!

I come from the department

of Speech, Music and Hearing

The Royal Institute of Technology!

The information is shownon the mapThank you!You are welcome!Good bye!

Perhaps we will meet soon again!

Yes, it might be that we

will!

Strindberg was married three times!

Strindberg was married three times!

The AdApt system

• Multimodal conversational dialogue system

• System features an animated agent and an interactive map in an apartment domain

• Developed by researchers from Centre for Speech Technology (KTH) and Telia Research

(Linda Bell, Johan Boye, Rolf Carlson, Jens Edlund, David House, Joakim Gustafson and Mats Wirén)

The Adapt user interface

Animated faces are appearing everywhereThey offer us advice on stock exchange and there are virtual news anchors

None of them understand what we say At the Centre for Speech Technology at the Royal Institute of Technology

researchers are developing animated agentsthat will be able to engage in dialogues The animated agent Urban will be able to pursue a freer sort of conversation

My name is Urban. What can I do for you?Well, what can you do?

I can tell you about available apartments in downtown Stockholm

I'd like to look at two-room apartments on the south side of town with a balcony and a fireplace

I have information about five such apartments, and I now show their locations on the map

Thank you very muchThis is fun, but I could just as well access this information through a text-based interfaceWhat is the purpose of speaking computer?The same information can be accessed,

but perhaps more efficientlyThe system can also compare different apartmentsI think speaking computers will be important in the future You don't need the keyboardDisabled people can use them

and inexperienced computer usersWhat are the difficulties involved?Our goal is to build a system that can

handle a more complex sort of dialogueA lot of problems remain to be solved...... many things in the dialogue can go wrong

misunderstandings occurWe are now collecting data to see

how people speak to such a computer...to try to improve the performance of future systemsDo people speak about anything just because

it is a face they are talking to? We get a sense of trust, since we believe the

animated agent has human capabilitiesThis is something we have observed in our data -

people like to socialize with Urban and they enjoy talking to him about this and that.

And once they have been developed, what do you think their functions will be?

They will be used for a wide range of thingsThey can be used as personal assistants and there is a commercial potential

We will probably see them selling things. When can we meet Urban again

Stockholm Technical museum, where he will be part of an exhibition shown this summerUrban will make a guest appearance thereAnd you can talk to himDo you know what the next part

of the program will be about?Yes, about magnifying glasses that the

Vikings used a thousand years ago

The AdApt project presented in a popular science programbroadcasted on Swedish national television, May 2000

Hello my name is Urban and I can tell you about nice apartments in downtown Stockholm

Hi Urban..eh..are there any available apartments .. around here

How many rooms do you wantI want a three room apartment..eh..it would be nice with an older one

There are five apartments in the shaded areaWell this apartment on Nybrogatan, can youtell me a little bit more about it

The blue apartment was built in 1920. It has a tiled stove and a parquette floor

Alright and..eh..the yellow apartmentThe yellow building was constructed in 1883Does this apartment also have tiled stoves?I’m sorry but I can’t answer that question