View
1.053
Download
12
Category
Preview:
DESCRIPTION
The main goal of this thesis is to study how different levels of knowledge stored in ontologies can be used to facilitate the creation of new coaching dialogue systems capable of domain reasoning. We take McGuinness' ontologies spectrum - and related work - to support the division of a cooking ontology into increasingly complex knowledge levels.The hypothesis being tested is whether ontologies can be used to enrich a coaching spoken dialogue system and be used in it in such way that the system can abstract the source of domain-specific knowledge - related to the tasks being coached - focusing only on the dialogue phenomena. The integration of ontological knowledge should be done with few architecture adaptions to the dialogue system so that when adding a new domain - a new class of tasks - minor changes in special modules are sufficient. Software engineering techniques for systems integration, like extension-based modularization and design patterns, are to be considered and extensively used to minimize the number of architecture adaptions.
Citation preview
Cooking an ontology-based coaching dialogue system
— based on my PhD proposal —
Joana Paulo PardalQuality and Usability ColloquiumDecember 7th, 2009Berlin, Germany
Monday, December 7, 2009
Who am I?Monday, December 7, 2009
Researcher, PhD Student and LecturerRESEARCHER:
- Spoken Language Systems Laboratory (L2F) @ INESC-ID, Lisbon
PHD STUDENT: - IST, Technical University of Lisbon- Ontology-based (spoken) Dialogue Systems
LECTURER:- IST, Technical University of Lisbon
Monday, December 7, 2009
Monday, December 7, 2009
cooking a coaching
ontology-based (spoken) dialogue systems
Monday, December 7, 2009
Cooking
Monday, December 7, 2009
House of the future
Monday, December 7, 2009
Monday, December 7, 2009
Part of our daily life
Monday, December 7, 2009
With a lot of inspiration
Monday, December 7, 2009
And interested people
Monday, December 7, 2009
Eyes and hands busy
Monday, December 7, 2009
Dialogue Systems Monday, December 7, 2009
Typical Architecture
back-endsystem
DialogueManagement
Speech Recognition
Natural Language
Understanding
Natural Language
Generation
Speech Synthesis
User
Monday, December 7, 2009
Typical Architecture
acoustic models
grammar
back-endsystem
DialogueManagement
Speech Recognition
Natural Language
Understanding
Natural Language
Generation
Speech Synthesis
User
lexicon
grammar
lexicon
grammar
prosody
coded speech
Monday, December 7, 2009
IO Manager Dialogue Manager
Se
rvic
e M
an
ag
er
JV
oic
eX
ML &
JS
pe
ech
AP
I 2
.0 in
terf
ace Parser
Interpretation
Manager
Behavioral
Agent
Generation
Manager
Surface
Generation
Discourse Context
Ta
sk
Ma
na
ge
r
Dia
log
ue
Ma
na
ge
r In
terf
ace
De
vice
Co
re
Access
Manager
Device
Manager
Core
Business
De
vice
Co
reD
evi
ce
Co
re
QA@L2F
Data
base
User
ASR(Audimus)
GUI
TTS(DIXI)
FaceEvent Manager
Microphone
Array
DigA L2F INESC-ID
Monday, December 7, 2009
TRIPS Univ Rochester
Monday, December 7, 2009
!"
#$%&'()*+,-&./0$12%02,/%
3405%67
!"#$%&'(&
)#*+,&
-.#"&/0
1&234#5
!"#$%&%$#'#()"!"#$%&'
*('+),-.'"',$%()*$%+,)-
/$"$%'#()"(#.$//)
0$1),"(#()")01&#.$(*$(
23"#4$5(52),,&#!$
!"#$%'1#()"-.,%)!#,,#
8)62$9&:6;16%!"#$%&
<%0=;9&:6;16%!"$'%()&
Olympus Carnegie Mellon
Monday, December 7, 2009
Practicaldialogue
Monday, December 7, 2009
http://flickr.com/photos/carpathian/1656738696/in/photostream/
Practical Dialogue HypothesisThe conversational competence required for practical dialogues, while still complex, is significantly simpler to achieve than general human conversational competence.
Monday, December 7, 2009
Domain-Independence HypothesisWithin the genre of practical dialogue, the bulk of the complexity in the language interpretation and dialogue management is independent of the task being performed.
Monday, December 7, 2009
Types of dialogue systems:- Information providing- Appliances control- Planning & scheduling- Tutoring...
Monday, December 7, 2009
Coaching
Tehran Times Sports Desk
Monday, December 7, 2009
Simple but funcional dialogue systems
Monday, December 7, 2009
How different levels of knowledge stored in domain-specific ontologies can be used to facilitate the creation
of new coaching dialogue systems capable of domain reasoning?
http://www.flickr.com/photos/jhhwild/1781000505/Monday, December 7, 2009
short introduction to
ONTOLOGIES
Monday, December 7, 2009
on·tol·o·gy
ORIGIN early 18th cent.:from modern Latin ontologia, from Greek ONT- ‘being’ -LOGY.
(philosophy) the branch of metaphysics that addresses the nature or essential characteristics of being and of things that exist; the study of being in the capacity of being.
(computer science) a structure of concepts or entities within a domain, organized by relationships; a system model.
Definition adapted from http://en.wiktionary.org/wiki/ontology
Monday, December 7, 2009
is a description of the existing concepts and relationships in a given domain
“An ontology is a specification of a conceptualization.”
[Gruber, 1993]
What is an Ontology?
Monday, December 7, 2009
Monday, December 7, 2009
Tim Berners-Lee’s Semantic WebMachine processable (not natural language), for data, for the future.
http://www.w3.org/2000/Talks/1206-xml2k-tbl/slide10-0.html
Monday, December 7, 2009
machine processable and
interoperable systems
http://flickr.com/photos/ul_marga/755378645/Monday, December 7, 2009
sharing knowledge
http://flickr.com/photos/kgoldenchild/2223459035/Monday, December 7, 2009
easier communicationhttp://flickr.com/photos/hchalkley/92120879/
Monday, December 7, 2009
easier integrationhttp://flickr.com/photos/stevehopson/1794895311
Monday, December 7, 2009
Data organizationwhen a large amount is available
http://flickr.com/photos/shetha/1561376494/Monday, December 7, 2009
Ontologiesin dialogue systems
Monday, December 7, 2009
Ontology in a dialog system
acoustic models
grammar
back-endsystem
DialogueManagement
Speech Recognition
Natural Language
Understanding
Natural Language
Generation
Speech Synthesis
User
lexicon
grammar
lexicon
grammar
prosody
coded speech
Domain Ontology
Monday, December 7, 2009
!"#"$%&''
()#*+)*"''
,-."/'
01*$)2'2-."/3''
4)3".'-#'
&-$5+3'.)6)'
("7%&-#'
8$-5"$'
0-+#39'
:6&;'
<#6-/-*='
<#6-/-*='
,)55%#*'
>+/"3'
?8@'A&63'
(B3'(B3'
C-$.3'C-$.3'
@5""&D''
>"&-*#%E"$'
8)$3"$'
@5""&D''
@=#6D"3%3'
@+$F)&"''
!"#"$)G-#'
H-2)%#1@5"&%I&'
?-25-#"#63'
J#6"$5$"6)G-#'
,)#)*"$!H%3&-+$3"''
@6)6"'
!$)22)$'
TRIPS Univ Rochester
Monday, December 7, 2009
McGuinness’ spectrum
Monday, December 7, 2009
My work so far...
Monday, December 7, 2009
OntoChef a cooking ontology
joint work with R. Ribeiro, F. Batista, N. Mamede and P. VazMonday, December 7, 2009
Recipe: Chocolate Cake
Ingredients
Procedure
4 oz of sweet dark chocolate
5 eggs
...
separate egg whites and egg yolks
with an electric mixer whisk the egg
whites until forming stiff peaks
put egg whites in a container
...
video file
...
Main goal
Monday, December 7, 2009
Competency QuestionsRecipes:
- How do I make recipe R? - What are the quantities to make recipe R for 4 persons?
Actions: - How do I do A1?
Times: - Which recipes that take less than 10 minutes to do?
Food: - Which recipes have food item F1, but not F2? - Which recipes have as main ingredient food item F1?
Utensils: - Which utensils are used in recipe R1? - Which recipes can be made using the microwave?
Equivalencies: - How many liters is a cup?
Monday, December 7, 2009
OntoChefcooking domain model
Monday, December 7, 2009
Food
Actions
Utensils
Equivalences
Recipes Units &
Measures
several modules
Monday, December 7, 2009
Formalization overview
Monday, December 7, 2009
Resulting Ontology4 main (reusable) modules:
- Food- Utensils- Actions- Recipes
3 auxiliary (reusable) modules: - Measures- Equivalences- Types of dishes
Metrics: - 1151 classes- 92 slots
- 52 relations between classes- 311 instances
Monday, December 7, 2009
OntoChefbuilding methodology
Monday, December 7, 2009
Main activities: - Knowledge acquisition- Conceptualization- Formalization- Evaluation
Monday, December 7, 2009
separate teams for bulk work
Food
Actions
Utensils
Equivalences
Recipes Units &
Measures
Monday, December 7, 2009
weekly brainstorm sessionsMonday, December 7, 2009
hardest: reach agreementMonday, December 7, 2009
OntoChefknowledge sources
Monday, December 7, 2009
Monday, December 7, 2009
professional websites
Web Site #Recipes #Photos
www.vaqueiro.pt 3136 2790
www.continente.pt 467 467
sabores.sapo.pt 2760 1038
comezainas.clix.pt 1646 0
www.entrepratos.com 189 51
Total: 8198 4346Recipes collected from professional websites.
Monday, December 7, 2009
Web Site #Recipes
paladares.do.sapo.pt 329
receitasdeculinaria.blogs.sapo.pt 10
culinaria.no.sapo.pt 236
Total: 575Recipes collected from personal blogs.
personal blogs
Monday, December 7, 2009
CMU’s Recipe Database<recipe version="1.1"><line><originaltext>1 (18 ounce) package small chocolate chip cookies</originaltext><annotation>create_ing(ing0, "1 (18 ounce) package small chocolate chip cookies")</annotation>
</line><line><originaltext>1 cup hot fudge topping</originaltext><annotation>create_ing(ing2, "1 cup hot fudge topping")</annotation>
</line>...<line><originaltext>Spread 3/4 cup fudge topping over crust. </originaltext><annotation>separate(ing2, ing12, "3/4 cup fudge", ing13, "1/4 cup fudge", "")</annotation>
</line>...
Monday, December 7, 2009
USENET Cookbook.RH MOD.RECIPES-SOURCE CHOC-CAKE-1 D "20 May 86" 1986.RZ "CHOCOLATE FANTASY CAKE" "A no-bake chocolate cake"This is a very simple, quick, no-bake cake..IH "Serves 6\-8".SH! ! CAKE.IG "2\(12 cups" "cream" "600 ml".IG "3 cups" "milk" "750 ml".IG "3 pkgs" "instant chocolate pudding".PH.SK 1Arrange a layer of biscuits at the bottom of a large baking pan.....SH RATING.I Difficulty:easy..I Time:10 minutes preparation, several hours chilling..I Precision:measure the ingredients..WRMike Trachtman, Weizmann Institute, Rehovot, Israel, mike@wisdom.BITNET
Monday, December 7, 2009
OntoChefconceptualization
Monday, December 7, 2009
OntoChefdatabase population
joint work with T. Machado and N. MamedeMonday, December 7, 2009
Manual input
Monday, December 7, 2009
Recipes
(in text)
Pro
ces
s
Internet
e-Books
Cookbooks
Database(ontology intances)
OntoChef(cooking ontology)
Ontology-basedsemi-automatic inputthrough natural language processing
Monday, December 7, 2009
OntoChefevaluation
Monday, December 7, 2009
OntoCheffuture work
Monday, December 7, 2009
additional languagesMonday, December 7, 2009
Knowledge sourcesBook’s Cover Book’s Bibliographic Reference
English:“Ingredients”, by Loukie Werle, Ullmann Publishing, 2009, 384 pp.
Portuguese:“Ingredientes”, by Loukie Werle, Könemann, 2009, 384 pp.
English:“The Cooks' Bible”, by Le Cordon Bleu, Carroll & Brown Publishers, 2005, 352 pp.
Portuguese:“O Livro das Técnicas Culinárias”, by E. Treuille & J. Wright, Livros e Livros, 2006, 352 p.
“Dicionário Visual Português/Inglês”, Civilização Editora, 2006, 360 pgs.“Dicionário Visual Português/Espanhol”, Civilização Editora, 2006, 360 pgs.“Dicionário Visual Português/Françês”, Civilização Editora, 2006, 360 pgs.“Dicionário Visual Português/Italiano”, Civilização Editora, 2006, 360 pgs.“Dicionário Visual Português/Alemão”, Civilização Editora, 2006, 360 pgs.
Monday, December 7, 2009
Collected informationLanguages # pairs PT-EN
Portuguese — English Ingredients 21125927Portuguese — English Actions 1658 5927
Portuguese — English 21575927
Portuguese — Spanish 2157Portuguese — French 2157Portuguese — Italian 2157Portuguese — German 2157
Total: 14555Monday, December 7, 2009
wine information
Monday, December 7, 2009
Cooking Coach an ontology-based
coaching dialogue systems
Monday, December 7, 2009
Recipe description
Monday, December 7, 2009
Monday, December 7, 2009
Available ‘tasks’
- next- previous- repeat- how? (down)- continue (up)
Monday, December 7, 2009
Cooking Corpus
Monday, December 7, 2009
Monday, December 7, 2009
Corpus annotation
Food
Actions
Utensils
Equivalences
Recipes Units &
Measures
Monday, December 7, 2009
Cooking Coach next steps...
Monday, December 7, 2009
Catalog
/ ID
Terms /
Glossary
General
Logic
Complexity
with automated Reasoningwithout automated Reasoning
ThesaurusTerm
Hierarchy
Formal
Taxonomy
Formal
Instance
Frames
(props)
Value
Restrictions
Description
Logics
Explore semantic spectrum
Monday, December 7, 2009
Reasoning
‘Caracóis’ (snails): broiled animal, a cake with snail shape
Monday, December 7, 2009
Tackle different users
Monday, December 7, 2009
Different coaching styles
Monday, December 7, 2009
Pri
or
exp
eri
en
ce
wit
h
the
Dia
log
Syste
m
Knowledge of the Domain
(Cooking prior experience)
none some a lot
no
ne
so
me
a lo
t
System’s evaluation
Monday, December 7, 2009
MIT’s Counter
Monday, December 7, 2009
DFKI’s Smart Kitchen
Monday, December 7, 2009
Coaching another tasks
Monday, December 7, 2009
Questions
http://flickr.com/photos/jazz_dalek/1845705965/Monday, December 7, 2009
Thank you!
http://flickr.com/photos/42304632@N00/232004616/Monday, December 7, 2009
Joana Paulo Pardal
joana.paulo.pardal@gmail.com
ladydoctorbug.blogspot.comwww.l2f.inesc-id.pt ⁄!joana
Monday, December 7, 2009
Cooking an ontology-based coaching dialogue system
Joana Paulo PardalQuality and Usability ColloquiumDecember 7th, 2009Berlin, Germany
Monday, December 7, 2009
Recommended