Embed Size (px)
Citation preview
Frameworks for Human Assist with Ubiquitous Agents
2004/12/17China Agent School
Hideyuki NakashimaFuture University - Hakodate
Background
• Information technologies are advanced to the point where it becomes feasible to assist ordinary people enjoy the full capability of information processing powers.– Although the Internet opened up huge
possibilities for daily use, users still have to learn a lot of special concept to use it.
Background 2
• With the help of various sensing technologies, it is time to design arich, keyboard-less interface and provide an ubiquitous computing environment for ordinary people in everyday life. Ubiquitous agent technology plays essential role here.
Agent Concept is a Key to Ubiquitous Computing
• Human-sensor interface• Access control
Overview of the tutorial• Location based communication• Agent-based network architecture• Dialog system
– Situated dialog– Information integration
• Application– Cyber Assist– Urban information infrastructure– RoboCup soccer/rescue
Introduction
TICKET
Take the next train.Change to A line at B station.The fare is YY yen.
I want to go to XX station.
Good Old DaysGood Old Days
券売機券売機 券売機
●●●●
××××
△△△
○○○○○
▲▲▲▲
120
560
340
780
900
□□□ 210
※※※※ 400
※●※●※
×○×
△■△■
○□○□
▲×▲×▲
120
560
340
780
900
□◎□◎ 210
■※■ 400
●●●●
××××
△△△
○○○○○
▲▲▲▲
120
560
340
780
900
□□□ 210
※※※※ 400
※●※●※
×○×
△■△■
○□○□
▲×▲×▲
120
560
340
780
900
□◎□◎ 210
■※■ 400
券売機
・・・・・
Wrong Use of“IT”
Wrong Use of“IT”
Please go through..
The next train will leave from platformNo 2 to your right.
TURN RIGHT TURN RIGHT
speaker
Connection to Internet
Time table server
Seat reservation system
My-button
LED display
Station master computer
Station LAN
TokyoRF tag
Unawareable environmentUnawareable environmentIR communicator
Position sensor
Ubiquitous Computing• Omnipresence of computing power• Human centered services
– The opposite direction from virtual reality where the reality is reconstructed in the digital world
– Computers come out to the real world• Gadgets are hidden
– calm computing– disappearing computer– oxygen– unawarable computers
MIT Oxygen Project
• Goal– Human collaboration aid– Human interface (spoken, visual)– Software, network– Intelligent space, portable device
• Approach– pervasive– embedded– nomadic– eternal
User Technologies:Speech and visionAutomationIndividualized knowledge accessCollaboration
User Technologies:Speech and visionAutomationIndividualized knowledge accessCollaboration
H21
The Oxygen System
System:HW and SWArchitecture
System:HW and SWArchitecture
Handhelddevices
Embeddeddevices
Devices, actuators, sensors
E21
N21
Migrate: An Architecture for Vertical Mobility
• Allows efficient and seamless “vertical” mobility across different wireless networks
• Enables connections to be separated from, and move between, IP addresses– Uses dynamic updates to track
host location– Uses connection migration to
retain TCP connections
• Key Abstraction– Session rather than Connection
Campus-Area Packet Radio
Regional/Area“wireless cable”
Metro-Area
In-Building & In-Room
Before:People had to learn to use machines
Computers in the Digital WorldComputers in the Digital World
Internet schoolInternet schoolPC schoolPC school
People in the Real WorldPeople in the Real World
After:Grounding is the key technology
Digital WorldDigital World
groundinggrounding
Real WorldReal World
From WWW to HNM
• Wherever, Whenever, Whoever→ Here, Now, Me (situatedness)
• At the same time:Just as much as you want to know→ intelligent information filter
• And,Just as much as you want to be known→ privacy
Location based communication
Ubiquitous computing environment in the near future
• Sensors– Vision, motion, sound, vibration, stress,
position, temperature …• Communication infrastructure
– Radio, wired (electric/optical)• Processing infrastructure
– CPU server, external display, auxiliary input
Human Centered Infrastructure
ID-based communication
ID-based communication
Location-based communication
Location-based communication
Broadcasting satellite３６０００ｋｍ～７８０ｋｍ
Broadcasting satellite３６０００ｋｍ～７８０ｋｍ
High-altitude aeroplane/ship２０ｋｍ
High-altitude aeroplane/ship２０ｋｍ
broadcastbroadcast
broadcastbroadcast
Sensor networkSensor network
Communication Methods
Local 1-10 meters global
Magnetic cardMagnetic cardICIC--cardcard
NetworksNetworksPhonesPhones
BroadbandHigh-speed
LowLow--powerpowerShortShort--rangerange
LocationLocation--basedbased
IDID--basedbased
Seamless connection to the Internet
ID-based communication
Location-based communication
Advantages of Location Based Communication
<x, y, z, t>• Location is one of the essential keys for
grounding the digital (logical) world to the real world.– Situated information processing and services
• Interaction without ID– Possibility of PRIVACY protection– For better human interface– Application to emergency situations
Various location-based communication(addressing) methods
Logical addressing“passenger at 12C”
Physical addressing“you”
(cricket)*(mobile phones)*
(cricket)*(mobile phones)* carrier pigeoncarrier pigeon
(bluetooth)*DSRC
Self-Positioning
(bluetooth)*DSRC
CoBITi-lidar
CoBITi-lidar
Positioning by server
*positioning only
Security vs. Privacy vs. Convenience
securitysecurity privacyprivacy
convenienceconvenienceThey basically contradict each other with some exceptions
Trade-offs between Convenience and Privacy
• Convenience– By providing personal information
• Social security number• Credit card number• “cookies”
• Privacy– Anonymity– Cash ← using physical property
Trade-offs between Security and Convenience
• Security– Key to doors– Security checks
• Convenience– Free entry
Trade-offs between Security and Privacy
• Security– Surveillance cameras– Monitoring– ID check– Restricted entry
• Privacy– Anonymity– Freedom of action
Needs technology for privacy protection
amountofservice
not zero
privacy protection
Dynamic/Voluntary Control of the Balancing Point is Necessary
• Trade-off between the amount of services and the amount of personal information
• Let the user set the point!
Privacy Security
Problem
Tally method for protection of personal data
Hospital B
Hospital A
Data transfer
Human transfer
Tally Method: is it possible?
• Requirements:– Accessible only when both the server and the client
are (physically) present.– Ability to protect data from malicious server
• Simple encryption does not work:– It becomes plain information once decoded.
• Probably unsolvable within digital world– Must use physical properties?
Agent-based network architecture
Ubiquitous Computing
How do we access and control devices
Intuitive User Interfacethat uses real world propertiessuch as location(point and click)
Why Agents?
• Aids for human cognition– interface
• Resource control– cognitive resources– devices
• multiple users and• multiple devices
Cognitive Resource Control
• SmartKom example:– cognitive resource of human
• when running terse• when walking• when standing verbose
・Maintains who reserved which time slot・Controls remote access・The reserver has the access right
homeVCR
toilet
bathliving
air conditionerthermometer
thermometer
・The access right belongs to who reserved・Access right can be delegated・The device may act as a mediator
homeserver
RFID tag無線タグ
RFID tag
Son
父
×
×
○
！
town
OK Dad, go ahead...
Access Control of Home AppliancesBased on Location and Priority
My son has a reservation. Let’s ask him.
someone is using the air conditioner・change reservation
or, delegate authority
・Deny remote access when someone is near-by
・communication center for the family
Agent Architecture CONSORTS(by AIST)
• Grounding agents to the physical world using location information– Translation between
X-Y coordinates and human recognizable names
InferenceEngine
PersonalAgent
ServiceAgentsMobile
User 1MobileUser n
Communicationand SensingDevices
Physical WorldContentServer
DeviceWrapperAgents
Agent Communication
APIPhysical Link
SpatialContent DB
Spatio-Temporal Reasoner
Camera, UWBLaser Radar, GPS
Wireless LANBluetooth, etc.
User ModelManager
SpatialContainer Graph
Physically Grounding Mechanism
Agent communicationCONSORTS API
physical link
InferenceEngines
Personal Agents
ServiceAgentsUser1 Usern
Communication and censoring
devices
Physical WorldContentServer
Devicewrapper
Spatial information database- sensor-based geometrical representation- ontology- location information
Spatio-Temporal Reasoning Agent
Camera, Wireless LANCoBIT, etc.
User modelmanager
Physical Grounding
Web Services
Spatio-temporal Inference Agent
spatio-temporal map(Segment Representation)
Inference engine:spatio-temporal relations and their changes (distance, phase, connection etc.)
device wrapper agents
personal agents,service agents
request, reply, inform
time
sensor data
Spatial informationdatabase
interpretation
Sensor information at location (x,y) mapped onto a map
Geometric representation of sensor information
Museum
1st Floor
Entrance
3rd Floor2nd Floor
Room CRoom A Room B
Painting BPainting A Painting C
Hierarchical representation of physical space
Painting D Painting D
Communication of CONSORTS Agents
• Uses FIPA Agent Communication Language– Foundation for Intelligent Physical Agents– http://www.fipa.org/– Separation of message content and speech act（e.g.
inform, request, subscribe）
• FIPA SL Content Language – predicate logic in S-expression– domain ontology for agents
• FIPA Interaction Protocol– Request (Request, Accept, Refuse), Subscribe etc.
Example of a FIPA-ACL message：Inform
• Sensor-wrapper-agent1 reports Spatio-temporal reasoner that John is at location x=120, y=100.
( inform :sender Sensor-wrapper-agent1:receiver Spatio-temporal reasoner:content (Located John (location (x 120) (y 100))):in-reply-to message1:language FIPA-SL:ontology location-ontology
)
Ubiquitous Computing withFIPA-Agents
• Direct connections to sensors are not good• Agents mediate communication of sensor information• Agentcities Network
• http://www.agentcities.net/
– EU centered worldwide research network based on FIPA standard
– 134 platforms• EPFL, BT, Motorola， etc.
– Tokyo.Agentcities.Net (CARC)• http://www.carc.aist.go.jp/agentcities/
A Snapshot of Our Museum Information Support System
Main Window (Museum Map)
Information Window
Message WindowMessage Window
Human Icon: Current Location
Yellow Area: Service Zones
Blue Line: Users’ Trajectories
UserUser
LAN card LAN cardLAN card
wirelesswireless--LANLAN
Web Service
LAN card
Web ServiceHTTPHTTP
SOAP SOAP
CONSORTS
AgentCities Networks
monitor wrapper agent
Spatio-temporal inference agent
FIPA-ACL
Wireless LAN wrapper agent
FIPA-ACL
access point access point
Web browserWeb browser
User-location tracking on the network
FirewallFirewallreal worldreal world
UBKit: Software construction tool kits (by AIST)
UBKit middleware
infrared
SOAP DNS
network
real world
IEEE802.11 BluetoothMicroservers
voice positioning
Micro servers + Optical Communication forIntegrated Remote Control System
positioning system
Ultra small low energy micro server(Linux) coin size
radio
voice recognition
sensor networkinfrared
Lightweight middleware (service directory, SOAP)
sensorgateway
UBKit (Ubiquity Building Toolkit)
Integrated control of all home applianceswith intuitive interface
Self configuring plug and play network
An Integrated Remote Control
pointing by IRand/or commandby voice
room sensing- temperature, brightness
user modelingaccess priority controlprivacy and security control
Integrated action betweeninformation retrieval andappliance control(search for a program andthen record/view it)
My Button
home server(s)
Dialog System
Situated dialogInformation integration
Mutual belief is an illusion“Roger for Roger for Roger”
• P is a mutual belief iff– Bap– Bbp– BaBbp– BbBap– BaBbBap– ….
• Any sequence of one way communication cannot establish it
“Roger for Roger”
“Roger”
“attack together tomorrow”
ＢＡ
Ｃ
Situation Theory
• A situation s supports an infon is |= i
• Example of time representationJapan|=<<time, 2, pm>>GMT|=<<time, 7, am>>world|=<<time, 2, pm, JST>>world|=<<time, 7, am, GMT>>
Gricean Maxim• The maxim of Quality
– Do not say what you believe to be false– Do not say that for which you lack adequate
evidence • The maxim of Quantity
– Make your contribution as informative as is required– Do not make your contribution more informative than
is required • The maxim of Relevance
– Make your contributions relevant • The maxim of Manner
– Be perspicuous, and specifically: 1. Avoid obscurity, 2. Avoid ambiguity, 3. Be brief, 4. Be orderly
Annotation as Grounding
Content(Real World)
SemanticWeb
AnnotationAnnotation
Semantics forComputers
Sharing semanticsbetween computers
and human(Grounding)
Linguistic Content Processing
• Interactive (semi-automatic) semantic search– NOT a keyword matching– Uses semantic structure of documents and query
• Content preparation with semi-automatic annotations– Content preparation is expensive
• But only once• Enhancement with authoring tools
Multimodal Content Processing• Grounding linguistic content to the physical
world– annotation with location– interaction through CoBIT
• User modeling– record of movement– record of inquiries
• Summarization and interactive presentation
Application
Cyber AssistUrban information infrastructure
RoboCup soccer/rescue
ApplicationCyber Assist (http://www.carc.aist.go.jp/)
Urban information infrastructureRoboCup soccer/rescue
Grounding is the key technology
Real WorldReal World
groundinggroundingCybernetics!
Digital WorldDigital World
The goal:Grounding the digital world to the real world
for “here, now, and to me” serviceReal world
informationinformation
Digital worldDigital world
goodsgoods
humanhumanSemantic WebSemantic Web
Ubiquitous ComputingUbiquitous Computing
Aimulet™(MyButton)• One button
– Do what I want here, now.
– No button?Activation by
brain wave?
• The ultimate man-machine interface？– Intelligent
personal agent– Sharing
situations
command button
fingerprint recognizer
reflectors photo detecto
microphone
speaker
Positive Characteristics of Optical Communication
• Unidirectional– location + direction control
• Interference free (almost)• Matching with intuitive definition of the
space– Radio wave invades to the next room and/or
upper and lower floors
Optical Bi-directional Communication
server
cameraLED
terminal
•Download•voice
•power supply
•Upload•position•direction•motion•(ID)
Aimulet™ V1: Tobiot™(CoBIT)• Compact Battery-less Information Terminal• Terminal: solar cell + reflector + earphone• Tobiot is a trademark of Cyber Assist One Inc.
CoBIT (Compact Battery-less Information Terminal）
LED emitter
Biased sound signal
Ear phone
Solar cell
Corner-reflectingsurface
Camera with visible-light-cut filter
Amplifier
Infra-red LEDs
Track bright points in dark image
CoBIT’s position,direction,sign
Application to Markets青森産のリンゴです。
一個200円です。これ買いますか？
ありがとうございます。
アラスカ産のシャケです。
これを買いますか？
となりにはサンマがあります。
The Image of the Target System
Environment
User
device
Intelligent Content
Sensors and actuators
Situationrecognition
Query/orders
information
request
Information retrievalInformation processing
Voicerecognition
Semantic query
Photo/radio communication
vision
Multiagent technology
Tokyo
ＷｉｒｅｌｅｓｓＬＡＮ
Ｍｙ-Ｂｕｔｔｏｎ
i-lidar
camera
intelligent contentsemantic web
voicecommand/query
voiceinformation
ＩＲ
The next train will leave from platformNo 2 to your right.
ＣｏＢＩＴ
Application
Cyber AssistUrban information infrastructure
RoboCup soccer/rescue
Mass Support
• Information support for the whole city• Examples
– Global Car Navigation• Optimal route planning using the current positions and
destinations of all cars in the area• Coordinated signal control
– Theme Park total coordination– On-line Government
• Direct democracy with the help of discussion and decision support system
Positioning Infrastructures• Outdoor
– GPS– PHS/mobile phones
• Indoor– ultrasonic (bat, crocket etc.)– Wireless LAN
• Both– terrestrial magnetism sensor– Infra Red beacon– Radio Frequency ID tag– Ultra Wide Band
Supporting Groups/Mass of People
• Reducing queues in town– global car navigation– traffic control– theme parks
• Transportation support– goods– work
• New government– decision making– discussion support
Bus on Call
• no fixed route• no fixed timetable• Within Japan,
operational only in Nakamura City, Kochi Pref. (the picture)
• Our simulation showed much more possibility large cities
Multiagent Simulation on the Feasibility of Bus-on-Call
• Assuming there are means to– report the current position of the customer– report the destination of the customer– report the ETA of the bus– acknowledge the customer’s decision– check if the customer gets on board
Simulation Resultexpected time to the destination / no. of demand
• more bus for more passenger
• average waiting time decreases as the demand increases
Global Car Navigation
• The current car navigation devices has– location of each car– destination of each car
• But card do not communicate each other…
ST: shortest time routeInput： vehicle density in each link
Route choice： the route minimizing the amount of expected travel time from the current position to the destination
chooses the route to minimize the amount of expected travel timerecomputation at every intersection.
A driver using the ST calculates expected travel time of each link.
origin
?
destination
RIS: route information sharing
Input： prospective traffic volume of each link that Route Information Server calculates based on gathered routes of drivers using the RIS.
Route choice： a route to minimize projected traffic volume.
??
DRA
?
?
Choose the route minimizing the amount of expected travel time.
Calculates projected traffic volume of each link.
Re-choose the route minimizing the amount of projected traffic volume
recomputation at every intersection.
The route
prospective traffic volume
Drivers using the RIS
Drivers using the RIS
Route Information Server
Route Information Server
Typical Transition of Expected Travel Time and Projected Traffic Volume
Link 1～8 (innermost ring)
Transition of expected travel time on link 1～8 (innermost ring)
•Case1 SD:ST:RIS = 0：1.0：0(All drivers use the ST.)
•Case2 SD:ST:RIS = 0：0：1.0(All drivers use the RIS.)
step
prospective traffic volume on link 1～8
step
Expected travel time on link 1～8
step
Expected travel time on link 1～8Vehicles concentrate on one link.
Vehicles diffuse to entire network.
Application
Urban information infrastructureCyber Assist
RoboCup soccer/rescue
Soccer as a MA problem• Hierarchical agents
– individual, formation, team, match• No director
– dynamic coordination, role change• Real time planning• Failure• Many communication methods
– observation, eye contact, predetermined formation
Special project for earthquake disaster mitigation in urban areas
(Ministry of Education and Science)• Ad hoc network
for emergency communications
• Multiagent simulation for– evacuation– rescue
Real-time dynamic ad-hoc network for large-scale disasters (earthquake)
– Effective in congested areas
– Alternative to other communication infrastructures
• First 10-30 min.– Connection to
other lines when possible
Large-scale Integrated Simulation System: needs MA technology
• Large scale disaster = complex phenomena– Integration of specialists’ knowledge in many areas– Integration of multiple single-purpose simulators
Beyond Engineering
• No complete specification– “safe”, “secure, “convenient”, etc.
• No optimization axis– in disaster rescue
• No a priori evaluation criteria– urgent packets passing other packets
• OPEN SYSTEM
Grounding the Simulation
KernelGIS
Agents
Viewers
ComponentSimulators
Portable devices Robots
Real-worldsensors
Rescue HQ
Rescue team
Total damage by number of agents
0
100
200
300
400
500
600
700
800
1 2 3 4 5 6 7 8 9
Total
Ext
number of agents
number of burnt down house
Efficiency of rescue per agent
0
500
1000
1500
2000
2500
3000
3500
1 2 3 4 5 6 7 8 9
Efficiency
Efficiency：Ei(n)=C/(damage x n)
Summary
• Multiagent concept is the key to ubiquitous computing– The core architecture– Ubiquitous agents for human support– Many MA applications
Future problems
• Use of timing information• Management of right of users
– privileged class?
![Microcontainers, Microservices, Microservers? Less [Linux] is more!](https://img.pdfslide.net/doc/110x75/58f0d4aa1a28ab39538b4593/microcontainers-microservices-microservers-less-linux-is-more.jpg)
