Natural Language Agents

Turing’s Prediction

1950 Turing predicted that in about fifty years "an average interrogator will not have more than a 70 percent chance of making the right identification after five minutes of questioning".

Watson

How does Watson win? http://www.youtube.com/watch?v=d_yXV22O6n4

Watch a sample round: http://www.youtube.com/watch?v=WFR3lOm_xhE

From Day 1 of the real match: http://www.youtube.com/watch?v=seNkjYyG3gI

Introduction: http://www.youtube.com/watch?v=FC3IryWr4c8

IBM’s site: http://www-03.ibm.com/innovation/us/watson/what-is-watson/index.html

Bad Final Jeopardy: http://www.youtube.com/watch?v=mwkoabTl3vM&feature=relmfu

Machine Translation

Austin Police are trying to find the person responsible for robbing a bank in Downtown Austin.

El policía de Austin está intentando encontrar a la persona responsable de robar un banco en Austin céntrica.

The police of Austin is trying to find the responsible person to rob a bank in centric Austin.

A Babelfish example from about 2004:

Machine Translation

Austin Police are trying to find the person responsible for robbing a bank in Downtown Austin.

De Policía de Austin están tratando de encontrar a la persona responsable de robar un banco en el centro de Austin.

Austin Police are trying to find the person responsible for robbing a bank in downtown Austin.

http://translate.google.com/translate_t?hl=en#

A Google Translate example in 2008:

Machine Translation – Another Old One

A Florida teen charged with hiring an undercover policeman to shoot and kill his mother instructed the purported hitman not to damage the family television during the attack, police said on Thursday.

Un adolescente de la Florida cargado con emplear a un policía de la cubierta interior para tirar y para matar a su madre mandó a hitman pretendida para no dañar la televisión de la familia durante el ataque, limpia dicho el jueves.

An adolescent of Florida loaded with using a police of the inner cover to throw and to kill his mother commanded to hitman tried not to damage the television of the family during the attack, clean said Thursday.

Machine Translation – 2008

A Florida teen charged with hiring an undercover policeman to shoot and kill his mother instructed the purported hitman not to damage the family television during the attack, police said on Thursday.

Un adolescente de Florida acusado de contratar a un policía encubierto para disparar y matar a su madre encargó a la supuesta Hitman no dañar a la familia de televisión durante el ataque, la policía dijo el jueves.

A Florida teenager accused of hiring an undercover police to shoot and kill his mother instructed the purported Hitman no harm to the family television during the attack, police said on Thursday.

Machine Translation – 2009

A Florida teen charged with hiring an undercover policeman to shoot and kill his mother instructed the purported hitman not to damage the family television during the attack, police said on Thursday.

Un adolescente de Florida acusado de contratar a un policía encubierto para disparar y matar a su madre instrucciones al presunto asesino a sueldo de no dañar el televisor de la familia durante el ataque, dijo la policía el jueves.

A Florida teenager accused of hiring an undercover policeman to shoot and kill his mother instructed the alleged murderer paid not to damage the family television during the attack, police said Thursday.

Machine Translation – 2010

A Florida teen charged with hiring an undercover policeman to shoot and kill his mother instructed the purported hitman not to damage the family television during the attack, police said on Thursday.

Un adolescente de Florida acusado de contratar a un policía encubierto para disparar y matar a su madre pidió al presunto asesino a sueldo de no dañar la televisión de la familia durante el ataque, dijo la policía el jueves.

A Florida teenager accused of hiring an undercover policeman to shoot and kill his mother asked the alleged murderer paid not to damage the family television during the attack, police said Thursday.

Machine Translation – 2012

A Florida teen charged with hiring an undercover policeman to shoot and kill his mother instructed the purported hitman not to damage the family television during the attack, police said on Thursday.

Un adolescente de Florida acusado de contratar a un policía encubierto para disparar y matar a su madre instruyó al presunto sicario de no dañar el televisor de la familia durante el ataque, dijo la policía el jueves.

A Florida teenager accused of hiring an undercover police officer to shoot and kill his mother instructed the alleged hitman not to damage the family television during the attack, police said on Thursday.

Machine Translation - 2011

The estimated death toll from Japan's disasters climbed past 10,000 Sunday as authorities raced to combat the threat of multiple nuclear reactor meltdowns and hundreds of thousands of people struggled to find food and water. The prime minister said it was the nation's worst crisis since World War II.

El número de muertos por los desastres de Japón subió últimos 10.000 el domingo como las autoridades se apresuraron a combatir la amenaza de múltiples fusiones de reactores nucleares y cientos de miles de personas lucharon para encontrar comida y agua. El primer ministro dijo que era peor crisis del país desde la Segunda Guerra Mundial.

The death toll from Japan's disaster climbed past 10,000 on Sunday as authorities scrambled to combat the threat of multiple mergers of nuclear reactors and hundreds of thousands of people struggled to find food and water. The prime minister said it was worse crisis in the country since the Second World War.

Machine Translation - 2011

The estimated death toll from Japan's disasters climbed past 10,000 Sunday as authorities raced to combat the threat of multiple nuclear reactor meltdowns and hundreds of thousands of people struggled to find food and water. The prime minister said it was the nation's worst crisis since World War II.

当局は、複数の原 子炉のメルトダウンと食料と水を見つけるのに苦労何十万もの人々の脅威と戦うためにレースと しての日本の災害からの推定死亡者数は過去 1 万日 曜日に登った。首相は、第二次世界大戦以来、国 家の最悪の危機であると言いました。

Officials estimated the death toll from the disaster of Japan as a race to combat the threat of hundreds of thousands of people struggling to find food and water to more than one nuclear reactor meltdown Sunday climbed past 10,000 . The prime minister since World War II, said the nation's worst crisis.

Machine Translation - 2012

The estimated death toll from Japan's disasters climbed past 10,000 Sunday as authorities raced to combat the threat of multiple nuclear reactor meltdowns and hundreds of thousands of people struggled to find food and water. The prime minister said it was the nation's worst crisis since World War II.

当局は、複数の原子炉のメルトダウンと食べ物と水を見つけるのに苦労し、何十万もの人々の脅威に対抗するためにレースとしての日本の災害からの推定死者数は万を超えて日曜日に登った。首相はそれが第二次世界大戦以来の国家の最悪の危機であると言いました。

Authorities, the number of deaths estimated from the disaster in Japan as the race Sunday more than million to struggle to find food and water and the meltdown of a nuclear reactor, plurality, to counter the threat of hundreds of thousands of people I went up to the. Prime Minister said that it is the crisis of the nation's worst since World War II.

TranslationParty

http://translationparty.com/

A federal appeals court ruled on Tuesday that regulators had limited power over Web traffic under current law. The decision will allow Internet service companies to block or slow specific sites and charge video sites like YouTube to deliver their content faster to users.

連邦控訴裁判所は 27 日、規制当局は、 Web トラフィッ ク過電流法の下で限られた力を持って判決を下した。決定インターネットサービス会社がブロックする、または特定のサイトや YouTube のような無料動画 サイトを遅らせる faster ユーザーにコンテンツを提供するようになります。

A federal appeals court on April 27, regulators, Web has ruled under a limited power over current traffic laws. The decision to block Internet services company, or specific sites and free video sites like YouTube to delay the faster will now provide content to the user.

April 27, regulators, Web, has limited power under the control of traffic control in the current federal appeals court. Decisions, Internet, and a specific site or YouTube, the delay can be blocked immediately Bideokontentsusaito as the current user is free.

Going Both Ways

Notice that both of these applications require that we process language in two directions:

•Understanding

•Generation

But also notice that it is possible to do well without going through any meaning representation.

When Meaning is Critical

English:

Put the kid’s cereal on the bottom shelves.

Java

import java.util.ArrayList;public class GroceryStore{ private int[][][] shelves; private ArrayList products; public void placeProducts(String productFile) { FileReader r = new FileReader(productFile); GroceryItemFactory factory = new GroceryItemFactory(); while(r.hasNext()) products.add( factory.createItem(r.readNext())); ThreeDLoc startLoc; GroceryItem temp; for(itemNum = 0; itemNum < products.size(); itemNum++) { temp = (GroceryItem)(products.get(itemNum)) startLoc = temp.getPlacement(this); shelves[startLoc.getX()][startLoc.getY()][startLoc.getY()]= tempgetIDNum(); } }}

Java, Continued

public class ChildrensCereal extends GroceryItem{ private static final int PREFERRED_X = -1; private static final int PREFERRED_Y = 0; private static final int PREFERRED_Z = 0; public ThreeDLoc getPlacement(GroceryStore store) { ThreeDLoc result = new ThreeDLoc(); result.setX(store.find(this)); result.setY(PREFERRED_Y); result.setZ(PREFERRED_Z); return result; }}

It’s All about Mapping

What Are We Going to Map to?

English: Do you know how much it rains in Austin?

Months

Month

Days

The database:

RainfallByStation

year

month

station

rainfall

Stations

station

City

English: What is the average rainfall, in Austin, in months with 30 days?

SQL:

SELECT Avg(RainfallByStation.rainfall) AS AvgOfrainfall FROM Stations INNER JOIN (Months INNER JOIN RainfallByStation ON Months.Month =

RainfallByStation.month) ON Stations.station =

RainfallByStation.stationHAVING (((Stations.City)="Austin") AND ((Months.Days)=30));

Doing it Without Meaning

WikiAnswers

How much does it rain in Austin?

Ambiguity – the Core Problem

• Time flies like an arrow.

● Chris likes the girl with the cat.

Ambiguity – the Core Problem

● Chris likes the girl with the cat.

● Chris shot the bear with a rifle.

Ambiguity – the Core Problem

● Chris likes the girl with the cat.

● Chris shot the bear with a rifle.

Ambiguity – the Core Problem

● Chris likes the girl with the cat.

● Chris shot the bear with a rifle.

● Chris shot the bear with a rifle.

Ambiguity – the Core Problem

Ambiguity – the Core Problem

• I saw a Boeing 747 flying to New York.

Ambiguity – the Core Problem

• I saw the Grand Canyon flying to New York.

Ambiguity – the Core Problem

• You can have potatoes or rice and beans.

or

Ambiguity – the Core Problem

• You can have potatoes or rice and beans.

andor

Ambiguity – the Core Problem

• You can have potatoes or rice and salad.

andor

Language Requires Knowledge

Beijing Stops Construction for Olympics

Headlines

Language Requires Knowledge

Beijing Stops Construction for Olympics

Hospital Pays $100,000 to Overdose Victim’s Family

Headlines

Language Requires Knowledge

Beijing Stops Construction for Olympics

Florida Reporter Completes Sentence

Hospital Pays $100,000 to Overdose Victim’s Family

TV ads boost eating of obese children.

Headlines

Olive oil

Semantics: The Meaning of Phrases

Semantics is (mostly) compositional.

(oil (made-from olives))

Occasionally It Isn’t

Olive oil

But Usually It Is

Peanut oil

(oil (made-from peanuts))

Another One

Coconut oil

(oil (made-from coconut))

Another One

Sesame oil

(oil (made-from sesame))

But What About This One?

Baby oil

But What About This One?

Baby oil

(oil (used-on baby))

And Another One

Cooking oil

(oil (used-for cooking))

Idioms Don’t Work This Way

• I’m going to give her a piece of my mind.

• He bent over backwards to make the sale.

• I’m going to brush up on my Spanish.

Discourse Coherence - Pronouns

• My cat saw a bird out the window.

It batted at it.

It had almost completely frosted over but …

Coherence

Winnie doesn’t like melted ice cream. He always dreads August.

* Winnie doesn’t like melted ice cream. He always dreads January.

Winnie wanted to go to the store. He went to find Christopher Robin.

* Winnie wanted to go to the store. He started leafing through a new book.

Winnie walked into the room. Christopher Robin looked up and smiled.

* Winnie walked into the room. The earth rotates around the sun.

Coherence

Winnie doesn’t like melted ice cream. He always dreads August.

* Winnie doesn’t like melted ice cream. He always dreads January.

Winnie wanted to go to the store. He went to find Christopher Robin.

Winnie wanted to go to the store. He started leafing through a new book instead.

Winnie walked into the room. Christopher Robin looked up and smiled.

* Winnie walked into the room. The earth rotates around the sun.

Conversational Postulates and Scalar Implicature

A: Have you done the first math assignment yet?

B: I’m going to go buy the book tomorrow.

Another Example of Scalar Implicature

A: When did you get home last night?

B: I was in bed by midnight.

Conversational Postulates and Inference

A: Joe doesn't seem to have a girl-friend these days.

B: He's been going to Dallas a lot lately.

Conversational Postulates and Inference

A: Let’s go to the movies tonight.

B: I have to study for an exam.

Conversational Postulates and Inference

Reviewer of new book:

It is well-bound and free of typographical errors.

Conversational Postulates and Inference

A: What do you think of my new dress?

B: It’s interesting.

Conversational Postulates and Illocutionary Force

• Do you know what time it is?

Conversational Postulates and Illocutionary Force

• Do you know what time it is?

• What time is it?

Conversational Postulates and Illocutionary Force

• Do you know what time it is?

• What time is it?

• I’m freezing.

Conversational Postulates and Illocutionary Force

• Do you know what time it is?

• What time is it?

• I’m freezing.

• Get up and go close the window.

Conversational Postulates and Illocutionary Force

• Do you know what time it is?

• What time is it?

• I’m freezing.

• Get up and go close the window.

Politeness

Sometimes it is Very Subtle

A Bangkok dry cleaner asks its customers to, “Drop your trousers here for best results.”

Sometimes it is Very Subtle

A Bangkok dry cleaner asks its customers to, “Drop your trousers here for best results.”

A Norwegian cocktail lounge asks, “Ladies are requested not to have children in the bar.”

Sometimes it is Very Subtle

A Bangkok dry cleaner asks its customers to, “Drop your trousers here for best results.”

A Norwegian cocktail lounge asks, “Ladies are requested not to have children in the bar.”

In the window of a Barcelona travel agency, “Go away.”

Sometimes it is Very Subtle

A Bangkok dry cleaner asks its customers to, “Drop your trousers here for best results.”

A Norwegian cocktail lounge asks, “Ladies are requested not to have children in the bar.”

In the window of a Barcelona travel agency, “Go away.”

In a laundry in Rome, “Ladies, leave your clothes here and spend the afternoon having a good time.”

Sometimes it is Very Subtle

A Bangkok dry cleaner asks its customers to, “Drop your trousers here for best results.”

A Norwegian cocktail lounge asks, “Ladies are requested not to have children in the bar.”

In the window of a Barcelona travel agency, “Go away.”

In a laundry in Rome, “Ladies, leave your clothes here and spend the afternoon having a good time.”

Underpaid guards in a Budapest zoo, “Please do not feed the animals. If you have any suitable food, give it to the guards on duty.”

Spoken Language

0 2 4 6 8 10 12

x 104

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

The dis crete

Fourier

trans form of a real sig nal

val ued

is con ju gate

sym me

tric

Speech Understanding

http://www.nuance.com/talk/

Text to Speech

http://www.research.att.com/~ttsweb/tts/demo.php

Text-To Speech

http://www.naturalreaders.com/?gclid=CJW_85uCwZUCFQEuxwoddiru0g

The horse raced past the barn fell down.

A Recent Loebner Prize Winner

> Why do you think people want to go to the moon?

Jabberwacky: You don't. Plastic and metal, perhaps. No flesh.

> Can you imagine life before plastic?

Jabberwacky: Can you?

Sure. Milk came in glass bottles.

Jabberwacky: You can come here whenever you want. I will welcome you.