Embed Size (px)
Citation preview
Psychology 1B, January 13th, 2015
Conversational Informatics
Toyoaki NishidaKyoto University
Copyright © 2014, Toyoaki Nishida, Atsushi Nakazawa, Yoshimasa Ohmoto, Yasser Mohammad, At ,Inc. All Rights Reserved.
What is AI?What is AI?
Example
→ Enthiran (The Robot), 2010
→ Her, 2013
AI in the moviesYear Title Who Wikipedia
1968 2001: A Space Odyssey HAL9000 http://ja.wikipedia.org/wiki/2001%E5%B9%B4%E5%AE%87%E5%AE%99%E3%81%AE%E6%97%85
1977 Star Wars C‐3PO, R2‐D2 http://ja.wikipedia.org/wiki/%E3%82%B9%E3%82%BF%E3%83%BC%E3%83%BB%E3%82%A6%E3%82%A9%E3%83%BC%E3%82%BA%E3%83%BB%E3%82%B7%E3%83%AA%E3%83%BC%E3%82%BA
1982 Blade Runner Replicants http://ja.wikipedia.org/wiki/%E3%83%96%E3%83%AC%E3%83%BC%E3%83%89%E3%83%A9%E3%83%B3%E3%83%8A%E3%83%BC
1984 The Terminator The terminator http://ja.wikipedia.org/wiki/%E3%82%BF%E3%83%BC%E3%83%9F%E3%83%8D%E3%83%BC%E3%82%BF%E3%83%BC_(%E6%98%A0%E7%94%BB)
1987 RoboCop RoboCop (cyborg) http://ja.wikipedia.org/wiki/%E3%83%AD%E3%83%9C%E3%82%B3%E3%83%83%E3%83%97
1993 War Games WOPR: War Operation Plan Response
http://ja.wikipedia.org/wiki/%E3%82%A6%E3%82%A9%E3%83%BC%E3%83%BB%E3%82%B2%E3%83%BC%E3%83%A0_(%E6%98%A0%E7%94%BB)
1994 Disclosure Angel http://ja.wikipedia.org/wiki/%E3%83%87%E3%82%A3%E3%82%B9%E3%82%AF%E3%83%AD%E3%83%BC%E3%82%B8%E3%83%A3%E3%83%BC_(%E6%98%A0%E7%94%BB)
1998 Bicentennial Man Andrew, a new NDR‐114 robot http://ja.wikipedia.org/wiki/%E3%82%A2%E3%83%B3%E3%83%89%E3%83%AA%E3%83%A5%E3%83%BCNDR114
1999 The Matrix Computer http://ja.wikipedia.org/wiki/%E3%83%9E%E3%83%88%E3%83%AA%E3%83%83%E3%82%AF%E3%82%B9_(%E6%98%A0%E7%94%BB)
2001 A.I. Artificial Intelligence David , achild Mecha http://ja.wikipedia.org/wiki/A.I.
2002 Minority Report Insect robots / user interface http://ja.wikipedia.org/wiki/%E3%83%9E%E3%82%A4%E3%83%8E%E3%83%AA%E3%83%86%E3%82%A3%E3%83%BB%E3%83%AA%E3%83%9D%E3%83%BC%E3%83%88
2004 I, ROBOT V.I.K.I . , Sonny http://ja.wikipedia.org/wiki/%E3%82%A2%E3%82%A4,%E3%83%AD%E3%83%9C%E3%83%83%E3%83%88
2009 ATOM Astro boy Astro boy http://ja.wikipedia.org/wiki/ATOM_(%E6%98%A0%E7%94%BB)
2009 Avatar (tele‐existence) http://ja.wikipedia.org/wiki/%E3%82%A2%E3%83%90%E3%82%BF%E3%83%BC
2009 Surrogate (tele‐existence) http://ja.wikipedia.org/wiki/%E3%82%B5%E3%83%AD%E3%82%B2%E3%83%BC%E3%83%88_(%E6%98%A0%E7%94%BB)
2013 Her Secretary / Lover http://www.herthemovie.com/#/home
2014 Transcendence (Technical singularity) http://ja.wikipedia.org/wiki/%E3%83%88%E3%83%A9%E3%83%B3%E3%82%BB%E3%83%B3%E3%83%87%E3%83%B3%E3%82%B9
[Nishida 2012b]
Year AI ICT1940～ 1936: Turing Machine, 1947: von Neumann Computer, 1948: Information
Theory, by C. Shannon and W. Weaver, 1948: Cybernetics by Wiener
1950～ 1952‐62: Checker program by A.Samuel1956: Dartmouth Conference 1957: FORTRAN by J.Backus
1960～ 1961: Symbolic Integration program SAINT by J.Slagle1962: Perceptron by F.Rosenblatt1966: The ALPAC report against Machine Translation by R. Pierce1967: Formula Manipulation System Macsyma by J.Moses1967: Dendral for Mass Spectrum Analysis by E.Feigenbaum
1961: Mathematical theory of Packet Networks by L. Kleinrock1963: Interactive Computer Graphics by I.Sutherland
1968: Mouse and Bitmap display for oN Line System (NLS) by D.C.Engelbart1969: ARPA‐net
1970～ 1971: Natural Language Dialogue System SHRDLU, by T.Winograd1973: Combinatorial Explosion problem pointed out in The Lighthill report1974: MYCIN by T.ShortliffeMid 1970’s: Prial Sketch and Visual Perceptron by D.Marr1976: Automated Mathematician (AM) by D.Lenat1979: Autonomous Vehicle Stanford Cart by H.Moravec
1970: ALOHAnet1970: Relational Database Theory by E.F.Codd1972: Theory of NP‐completeness by S.Cook and R.KarpMid 1970’s: Alto Machine by A.Kay and A.Goldberg1976: Ethernet1979: Spreadsheet Program Visicalc by D.Bricklin
1980～ 1982: Fifth Generation Computer Project1984: The CYC Project by D.LenatMid 1980’s: Back‐propagation algorithm was widely used1985: the Cybernetic Artist Aaron by H.Cohen1986: Subsumption Architecture by R.Brooks1989: An Autonomous Vehicle ALVINN by D.Pomerleau
1982:TCP/IP Protocol by B.Kahn and V.CerfMid 1980’s: First Wireless Tag Products1987: UUNET started the Commercial UUCP Network Connection Service1988: Internet worm (Morris Worm)1989: World Wide Web by T.Berners‐Lee1989: The number of hosts on the Internet has exceeded 100,000.
1990～ 1990: Genetic Programming by J.R.KozaEarly1990’s: TD‐Gammon by G.TesauroMid 1990’s: Data Mining Technology1997: DeepBlue defeated the World Chess Champion G.Kasparov1997: The First Robocup by H.Kitano1999: Robot pets became commercially available
1992: The number of hosts on the Internet has exceeded 1,000,000.1994: Shopping malls on the Internet1994: W3C was founded by T. Berners‐Lee1997: Google Search1998: XML1.0（eXtensible Markup Language) by W3C1998: PayPal
2000～ 2000: Honda Asimo
2004: The Mars Exploration Rovers (Spirit & Opportunity)
2001: Wikipedia.2003: Skype / iTunes store2004: Facebook2005: YouTube / Google Earth2006: Twitter2007: Google Street View
2010～ 2010: Google Driverless Car / Kinect2011: IBM Watson Jeopardy defeated two of the greatest champions2012: Siri
History of AI research in contrast with ICT
Year AI ICT1940～ 1936: Turing Machine, 1947: von Neumann Computer, 1948: Information
Theory, by C. Shannon and W. Weaver, 1948: Cybernetics by Wiener
1950～ 1952‐62: Checker program by A.Samuel1956: Dartmouth Conference 1957: FORTRAN by J.Backus
1960～ 1961: Symbolic Integration program SAINT by J.Slagle1962: Perceptron by F.Rosenblatt1966: The ALPAC report against Machine Translation by R. Pierce1967: Formula Manipulation System Macsyma by J.Moses1967: Dendral for Mass Spectrum Analysis by E.Feigenbaum
1961: Mathematical theory of Packet Networks by L. Kleinrock1963: Interactive Computer Graphics by I.Sutherland
1968: Mouse and Bitmap display for oN Line System (NLS) by D.C.Engelbart1969: ARPA‐net
1970～ 1971: Natural Language Dialogue System SHRDLU, by T.Winograd1973: Combinatorial Explosion problem pointed out in The Lighthill report1974: MYCIN by T.ShortliffeMid 1970’s: Prial Sketch and Visual Perceptron by D.Marr1976: Automated Mathematician (AM) by D.Lenat1979: Autonomous Vehicle Stanford Cart by H.Moravec
1970: ALOHAnet1970: Relational Database Theory by E.F.Codd1972: Theory of NP‐completeness by S.Cook and R.KarpMid 1970’s: Alto Machine by A.Kay and A.Goldberg1976: Ethernet1979: Spreadsheet Program Visicalc by D.Bricklin
1980～ 1982: Fifth Generation Computer Project1984: The CYC Project by D.LenatMid 1980’s: Back‐propagation algorithm was widely used1985: the Cybernetic Artist Aaron by H.Cohen1986: Subsumption Architecture by R.Brooks1989: An Autonomous Vehicle ALVINN by D.Pomerleau
1982:TCP/IP Protocol by B.Kahn and V.CerfMid 1980’s: First Wireless Tag Products1987: UUNET started the Commercial UUCP Network Connection Service1988: Internet worm (Morris Worm)1989: World Wide Web by T.Berners‐Lee1989: The number of hosts on the Internet has exceeded 100,000.
1990～ 1990: Genetic Programming by J.R.KozaEarly1990’s: TD‐Gammon by G.TesauroMid 1990’s: Data Mining Technology1997: DeepBlue defeated the World Chess Champion G.Kasparov1997: The First Robocup by H.Kitano1999: Robot pets became commercially available
1992: The number of hosts on the Internet has exceeded 1,000,000.1994: Shopping malls on the Internet1994: W3C was founded by T. Berners‐Lee1997: Google Search1998: XML1.0（eXtensible Markup Language) by W3C1998: PayPal
2000～ 2000: Honda Asimo
2004: The Mars Exploration Rovers (Spirit & Opportunity)
2001: Wikipedia.2003: Skype / iTunes store2004: Facebook2005: YouTube / Google Earth2006: Twitter2007: Google Street View
2010～ 2010: Google Driverless Car / Kinect2011: IBM Watson Jeopardy defeated two of the greatest champions2012: Siri
History of AI research in contrast with ICT
Exponential Growth- Moore’s Law- Expanding Internet World
Universal Turing Machine- from Hardware to Software
Generic Computing Engines- from Software to Data
Service Copying- Machine Learning / Data Mining
Symbolic Computing- Knowledge-based Systems
Heuristics- Intelligence as Clever Computing
Embodied & Network Intelligence- Like human and society
Successful Topics of AI
Large-scale Search Knowledge-based Systems Language, Speech, Vision Planning Machine Learning and Data Mining Using AI in Creating Works of Art
AARON (1985)
The Mars Exploration Rovers (2003)
Experiments in Musical Intelligence (1987)
DeepBlue (1997)http://www.research.ibm.com/deepblue/
http://marsrovers.jpl.nasa.gov/mission/spacecraft_surface_rover.html
http://www.kurzweilcyberart.com/
http://arts.ucsc.edu/faculty/cope/http://artsites.ucsc.edu/faculty/cope/mp3page.htm
IBM Watson (2011)http://www-03.ibm.com/innovation/us/watson/index.shtml
ALVINN （An Autonomous Land Vehicle in a Neural Network） on Navlab (1989)http://dl.acm.org/citation.cfm?id=89891
Recent Trends
1997: Deep Blue defeated G. Kasparov… Source: IBM Deep Blue, wikipedia
1997: Official opening of Robocup (H. Kitano)… Source: robocup.org, robocup.or.jp, Wikipedia
1997: Mars Pathfinder… Source: NASA Mars Pathfinfer, Wikipedia
1999: Robot pet SONY AIBO… Source: SONY AIBO, Wikipedia
1999: OpenCV Project… Source: OpenCVWiki，wikipedia
2000: Honda ASIMO… Source: Honda ASIMO, Wikipedia
2004: Mars Exploration Rovers… Source: NASA‐JPL
2010: Google Driverless Car… Source: Wikipedia, Sebastian Thrun’s home page, 3P
2010: Kinect… Source: xbox.com, Wikipedia
2011: IBM Watson defeated two Jeopardy! champions… Source: IBM Watson, Wikipedia
2011: iPhone Siri… Source: Apple, Wikipedia
2011: Google Voice Search… Source: Google
2012: Zen Takemiya Masaki with 4 stones … Source: Sig ECS, UEC2012: Google Glass Project
… Source: Project Glass2012: NTT DOCOMO’s Shabette Concier
… NTT DOCOMOOther： Smarter than You Think (New York Times)
Landmarks
‐ From philosophy to science‐technology
‐ Toy problems to the real world
‐ Powerful tools
‐ Grand challenges and competitions
Human society supported by ICT
Information and Communication TechnologyComputers, The Internet, MR，HCI, CSCW，AI，…
Community
Association
Society
Support
Corporation
Service consumers
Service producers
Conventional Society
[Nishida 2014]
Agents
Corporation
Service consumers
Service producers
Technology Society
[Nishida 2014]
CorporationPeople
AI Society
[Nishida 2014]
Dark side of super intelligenceDark side of super intelligence
Technology abuse
Responsibility flaw
Moral in crisis
Over‐dependency on technology
Technical Singularity
End of Work
Traditional AI Wow!Wow!
Wow!Wow!
Wow!
Wow!Wow!
Wow!
Wow!Wow!
Wow!
Wow!Wow!
Wow!Wow!
Wow!
Wow!Wow!
Wow!
Wow!Wow!
Wow!
[Nishida 2012]
The ability to understand others’ emotions and/or perspectives and, often, to resonate with others’ emotional states.
An affective response that is identical, or very similar, to what the other person is feeling or might be expected to feel given the context: a response stemming from an understanding of another’s emotional state or condition.
or …
[Eisenberg 2010]
Empathy
Towards Empathic Agents
• Service• Presence• Empathy
DeepBlue Entertain with a game
IBM Watson Entertain with a game
Siri Conversation partner
(AI that can pass entrance exam) Effective and affective tutor
Traditional AI：High competence
Future AI：High empathy
Communicative Intelligence for Bridging People and CI
[Nishida‐Nakazawa‐Ohmoto‐Mohammad 2014]
Communicative Intelligence
Super Intelligence People
History of conversational systems development
t1990 2000 201019801970
Natural language dialogue systems
Speech dialogue systems
Multi‐modal dialogue systems
Embodied Conversational Agents / Intelligent Virtual Human
Story Understanding systems
Conversational Systems
Transactional systems
Interactional systems
Affective Computing
Cognitive systems
Natural language question answering systems
The Knowledge Navigator
[Nishida‐Nakazawa‐Ohmoto‐Mohammad 2014]
At a marketplace
Under cherry blossom
At a seminar room
Conversation is everywhere in our life
Challenge: A robot that can participate in conversation
Long‐term goal
Not just conversational but also empathic
Towards empathic agents
Eye gaze
Hand gesture PostureFacial expression
AskingNegotiating
Proposing
ConvivialitySocial networksTrust
Conversation is a complex business
Self
Internal theater
Interactive
Social
Inter cultural
Perception
Sensors
Embodiment
Motors
cognition
Language
Memory
Imagination
Consciousness
Learning skills
Evolutional system
Mind
JudgmentEmpathic
The Architecture of Empathic Agent?
Our approach: Building a Primordial Soup of Conversation
Content provisionTraining data
Acquired content
Conversation management‐ Conversation environment management‐ Sensing human behaviors‐ Controlling agents
Conversation environment #1
Conversation environment #2
Conversation environment #3
Partially mechanized humans & Partially humanized machines live together and converse with each other.
A road to empathic agents
[Nishida‐Nakazawa‐Ohmoto‐Mohammad 2014]
Empathy
Sharing hypothesis
Common ground Conversational intelligence
Primordial soup of conversation
Engagement
Conversational Informatics
Application
Platform Evaluation
Content production Model building
Analysis
Theory
Measurement
Conversational interactions
Applied Intelligence Information Processing GroupCognitive DesignWe aim at designing the expressions, functions, control, and interactions of artifacts based on a physio‐cognitive approach.
Interaction Measurement, Analysis and Modeling We aim at uncovering principles of verbal and nonverbal interactions that people engage everyday as a part of intellectual activities and developing a suite of technologies to help people benefit from conversational interactions.
Augmented Conversation EnvironmentWe aim at building a smart environment that integrates conversation in physical and virtual spaces.
Conversation in the physical environment Conversation in the virtual environment
Learning by imitation
Designing high‐level cognitive interaction
Reconstructing 3D conversation scene
ICIE: Immersive Collaborative Interaction Environment
Multiparty conversation recorder
Eye tracker
Eye tracker (wearable)
Polygraph Optical motion capture systems
Cluster computer
IMADE: Interaction Measurement, Analysis, and Design Environment
SmartInFill
Annotation tool
Conversation corpus Virtualization of physical space
[Nishida‐Nakazawa‐Ohmoto‐Mohammad 2014]
IMADE: a real‐world interaction measurement, analysis and design environment
[Yano 2012]
3D conversation capture – over the shoulder
Corneal Imaging CameraScene cameraEye camera
• Lightweight and versatile system • Appl.: Google Glass like HMD, unconstrained setups
Corneal Image Feature MatchingProblem:Local feature correspondence + RANSAC does not work due to large noise in eye images
Approach:1. Formulate problem as registration of 3D
spherical light maps of eye and scene image2. Single point algorithm for robust alignment
Non‐intrusive Eye Gaze Tracking (EGT) by corneal imaging
[Nitschke 2013]
[Nishida‐Nakazawa‐Ohmoto‐Mohammad 2014]
Experimental settings for capturing human behaviors and physiological indices
[Nishida-Nakazawa-Ohmoto-Mohammad 2014]
ICIE – immersive collaborative interaction environment
Immersive Interaction Environment
[Nitschke 2013]https://www.youtube.com/watch?v=V‐9SKpcMrzk
Projecting the real world into the virtual world
[Mori 2011]
[Mori 2012]
Virtualized physical world
Immersive WOZ environment
[Ohmoto, Ohashi, Saiga]
Feedback generationMotion mappingUser motion sensingHead recognitionGesture recognitionFace modelHuman body model
WOZ operating environment
WOZ operatorTele-operated robot
The conversation place
Experimental setting for studying human‐robot interaction
[Nishida‐Nakazawa‐Ohmoto‐Mohammad 2014]
[Nishida‐Nakazawa‐Ohmoto‐Mohammad 2014]
iCorpusStudio
Collaborative annotation system
[Nishida‐Nakazawa‐Ohmoto‐Mohammad 2014]
Learning by imitation
[Mohammad 2009]
Constrained Motif Discovery: • Given a time series X(t)
find recurring patterns of length between L1 and L2 using distance function D subject to the constraint P(t), where P(t) is an estimation of the probability that a motif occurrence exists near time step t.
[Mohammad 2009]
P(t)
unlikely
likely
Change point discovery
Learning by imitation
Future
Change angle
GH
Past Futuret
;...; 1H t seq t n seq t 1 ;...;G t seq t seq t n
1
1
1
ˆ
f
f
Ti i i
l
i ii
l
ii
s t t t
csx
c
t
TtVtStUtH )()()()( Find optimal lP
ggT uutGtG )()(Find optimal lF
11and,)( jjjFg
ii liut
fTll
Tll
i litUUtUUt ,)()()(
)()()()()(ˆ)(~ tttttxtx PFPF
Learning by imitation
Robust Singular Spectrum Transform
[Mohammad 2009]
Learning by imitation
[Mohammad 2009]
Fluid Learning
[Mohammad 2013]
Immersive games
[Nishida‐Nakazawa‐Ohmoto‐Mohammad 2014]
[Lala 2013]
Virtual Basketball
Tacit social signals
Intentions are dynamic and tacit
[Nishida‐Nakazawa‐Ohmoto‐Mohammad 2014]
Facilitative agent that can track dynamic and tacit intentions
Synthetic Evidential Study
Structured collection of {story, background,
critique}
Reuse
Componentize Interpretation archive
Theatrical role play
Agent play
Critical discussions
[Nishida et al 2015]
1. ICT, AI in particular, is giving a deep impact on human and society.2. Conversations for empathic relations among people and machines.3. Why do people converse with each other? ‐> As a part of social
interaction, for creating and maintaining stories, for fun.4. The complexity of conversations: ‐> Multi‐level, multi‐layered, polysemy,
polymorphism, coordination of multi‐modality.5. People are not only proficient in expressing ideas but also skillful in
interpreting utterances, thereby they learn from each other.6. Building empathic agents helps people effectively communicate with
each other by conversations.7. Conversational informatics addresses science and engineering of
communicative intelligence.8. Conversational intelligence can be counted as an important branch of
artificial intelligence.9. We will draw on a data‐intensive approach. 10. Conversation quantization as a foundation of data‐intensive approach
to conversational informatics.
Summary
Further readings
• Toyoaki Nishida. Information and Communication Technology for augmenting human and society potential : The Role of artificial intelligence. Journal of Information Processing and Management, Vol. 57 (2014) No. 8 P 517-530 (in Japanese)http://dx.doi.org/10.1241/johokanri.57.517
• Toyoaki Nishida: Towards Mutual Dependency between Empathy and Technology, 25th anniversary volume, AI & Society, Volume 28, Issue 3, pp 277-287, 2013.http://dx.doi.org/10.1007/s00146-012-0403-5
• Toyoaki Nishida. Artificial intelligence research in the second half century. Journal of Information Processing and Management Vol. 55 (2012) No. 7 P 461-471 (in Japanese)http://dx.doi.org/10.1241/johokanri.55.461
• Toyoaki Nishida. The Best of AI in Japan – Prologue. AI Magazine 33(2): 108-111, 2012.http://www.aaai.org/ojs/index.php/aimagazine/article/view/2358/2288
• Rifkin, Jeremy. The End of Work: The Decline of the Global Labor Force and the Dawn of the Post-Market Era. G. P. Putnam’s Sons, 1995.
• Kurzweil, Ray. The Singularity is Near: When Humans Transcend Biology. Viking Penguin, 2005.
• Brynjolfsson, Erik; McAfee Andrew. Race Against the Machine. Digital Frontier Press, 2011.• Gratton, Lynda. The Shift: The Future of Work is Already Here. Harper Collins Pub., 2011.• Cowen, Tyler. Average is Over: Powering America beyond the Age of the Great Stagnation.
Dutton. 2013.• Turkle, Sherry. Alone Together: Why We Expect More from Technology and Less from
Each Other. Basic Books, 2011.
For more details …
http://www.springer.com/engineering/signals/book/978‐4‐431‐55039‐6
![Turing-Post Relativized Computability and Interactive ...soare/Turing/shagrir.pdfTuring’s [1936] automatic machine (a-machine), now called simply a Turing machine, gave one such](https://img.pdfslide.net/doc/110x75/60ad846317ffd9574071b4bf/turing-post-relativized-computability-and-interactive-soareturingshagrirpdf.jpg)
