Embed Size (px)
Composing Devices
Prasun Dewan
Department of Computer Science University of North Carolina
2
Issues raised by UPnP
Scaling problem of multicast-based discovery Auto Shut off problem
Simple-minded search Lack of access and concurrency control No programmer-defined types High programming and maintenance costs Manual, procedural service composition Device-independent UI
3
Declarative Specifications Defined by declarations Higher-level than procedural
HTML vs. Toolkits User addresses what rather than how.
Not as general. Domain specific HTML vs. Toolkits
Form-based user interfaces No graphics, incremental feedback.
Composition domains for declarative specifications?
4
Database based composition
Queries over multiple devices For each rainfall sensor, average rainfall For each sensor in Tompkin county,
current rainfall For next 5 hrs, every 30 minutes, rainfall
in Tompkin county
5
Query KindsDevice Queries Historical
For each rainfall sensor, average rainfall Snapshot
For each sensor in Tompkin county, current rainfall
Long-running For next 5 hrs, every 30 minutes, rainfall in
Tompkin county New kind of query
Defining device databases and queries?
6
Cougar Device Model Embedded vs. Attached Stationary vs. Mobile Strongly vs. Intermittently connected Local area vs. wide area Device database work focuses on stationary
devices Sensors vs. arbitrary devices?
7
Cougar Device Operations Operation Model
acquire, store, and process data may trigger action in physical world return result
Synchronous operation returns result immediately
Asynchronous operation result(s) later
abnormal rainfall as event
Intermittently connected device only asynchronous operations possible device not guaranteed to be connected when
operation invoked
8
Defining Device DatabaseDevice DBMS vs. Traditional
Relational DBMS Device vs. Data collection
Computed vs. stored values
Distributed information sources Data needed not available
locally May not even be available
remotely for intermittent Long-running queries.
Not modelled by traditional DBMS
Solution Base device relations
One record for each device
Virtual Relations Records partitioned
over distributed nodes Includes results of
device operations Extended query
language over virtual relations
9
Base Relations
Collection of devices of a particular type.
one record per device attributes
device id X coordinate Y coordinate
ID X Y
10
Virtual RelationsPer device function
Attribute for each function
argument result global timestamp of
result device id
New record added for each new result
Append-only relation Each device
contributes to part of relation
a1 Valam TSID
f(a1,…, am ): T
11
ExampleRFSensor One function
getRainfallLevel() : int Base relation
RFSensors Virtual relation
VRFSensorsGetRainfallLevel
Value TS Device ID
RFSensors
VRFSensorsGetRainfallLevel
Device ID X Y
12
For next four hours, retrieve every 30 seconds, rainfall level of each sensor if it is greater than 50mm.
Query Q:
SELECT VR.value
FROM RFSensors R, VRFSensorsGetRainfallLevel VR
WHERE R.ID = VR.ID AND VR.value > 50 AND
AND $every(30)
Run Q for 4 hours
200 devices
R cardinality = 200
VR cardinality = 480
Long running queries
join selection
AND R.X = ...
IN ...
Why Join?
13
Execution Strategies
R Materialized VR
Q
No local knowledge
R VR VR VR
VR.value > 50R.ID = VR.ID
Val every 3O min
Local rate info, remote Q
R VR VR VR
if R.ID = VR.ID
VR.value > 50
Val every 3O min
Local Join, Remote Selection
R VR VR VR
R.ID = VR.ID
if VR.value > 50
Val every 3O min
Local Selection, Remote Join
14
Performance Metrics
Traditional Throughput Response time
Long query running time implementation independent
Sensor specific Resource usage
network, power Reaction time
Production to consumption time
15
Power Usage Components
CPU Memory access Sending message Sending Nbytes
Cost in joules = Wcpu*CPU + Wram*RAM + Wmsg*Msg + Wbytes*NBytes
16
Centralized Warehouse
No local knowledge Single location
monitors all sensors Queries are sent to
site
Works for historical queries
Wastes resources for long running queries Irrelevant sites Higher rate than
necessary What to monitor for long
running queries Camera direction?
Centralization Workload Bottleneck
17
Distributed Device Database All sensors together
form a distributed device database system
Individual nodes sense on demand do part of query
processing
Better resource utilization
Know what to monitor
Historical? based on some
long-running query
18
Remote Query Evaluation
All sensors send data
No relations sent to sensors
R VR VR VR
VR.value > 50R.ID = VR.ID
Val every 3O min
Local rate info, remote Q
19
Local Join
R VR VR VR
if R.ID = VR.ID
VR.value > 50
Val every 3O min
Local Join, Remote Selection
Only joined sensors send value
Whole relation send to device communication
overhead
20
Local Selection Only selected
sensors send data Whole relation not
send to device best approach
R VR VR VR
R.ID = VR.ID
if VR.value > 50
Val every 3O min
Local Selection, Remote Join
21
Composition Techniques Query approach
Data of sensors aggregated together Alternatives
Operations of devices aggregated Results of one device affect operations of
another
22
Automatic Composition Domains ICrafter and Local Work
Device pipes Camera/Scanner to printer
Proxy based connection compression
Adapter based connection transcoding
Composite commands DoAll
• Turn on all VCRs
Do Related• Turn on projector• Dim lights
23
ICrafter ApproachSystem selects
composer based on interfaces of components and registered interface patterns
DataP
roducer
DataC
onsumer PowerSwitchPowerSwitch
DataProducer, DataConsumer ProducerConsumerPipe
Interface Pattern “Generic” Composer
PowerSwitch PowerSwitchAll
24
Selecting Composer
Pattern described using “regular expressions” single producer consumer multiple producer consumers
Client requests composer for a set of devices Returns composer for each subset of devices
with matching pattern Composer provides user-interface for invoking
composite commands Switch on all lights Drag producer to consumer or vice versa
25
ICrafter: A Service Framework for
Ubiquitous Computing
Environments Shankar Ponnekanti, Brian Lee, Armando Fox, Pat Hanrahan, Terry Winograd (Stanford Univ.)
26
Aggregation
May seem individual UI’s can be combined. However ….
27
UI(S1+S2) != UI(S1)+UI(S2)
Good for individual operations Clumsy for compound operation: snap-and-
display
28
UI(S1+S2) != UI(S1) + UI(S2)
Transfer = snap and display in one click
29
Where We Stand
Suppose n services, m appliances and w workspaces O( F(n) × m × w ) UI’s F(n) is some combinatorial function of n
Existing ad-hoc interaction systems: Jini, UPnP, Hodes et al (Mobicom97, USITS99), Roman et al (WMCSA00)
Mostly focus on appliance heterogeneity
30
Generators: Handling Aggregation
Generators for multiple services Eg. {Camera, Display}, {Camera,
Display, Display} Generators for service patterns
Eg. {Camera, Display+}, {Projector* } Generators for service interface
patterns Eg. {DataProducer, DataDisplayer+}
31
Simplified Example
Request for {Camera, Display} Matches Camera generator, Display generator, and
{DataProducer, DataDisplayer} generator
32
Simplified Example
Aggregated using panels
33
Outline
What? Why? How?
34
Bootstrapping
35
Future Work
Authentication and synchronization Aggregation: more intelligent generator
set selection: Eg. Eliminate a {DataProducer,
DataDisplayer} generator if a {Camera, Display} generator exists
36
Interactive Workspaces Project Info
Some software already available Major release (including ICrafter) due
this month For software and other info:
http://iwork.stanford.edu/
37
Problems of ICrafter
Must manually connect devices select from a list drag and drop Automatic continuous like Unix pipes?
Equivalent of standard input and output? Interface/Class-based composition
rather than programming pattern Getter/setter
38
Problems of Interface-based Approach
Type of Data in Interfaces? Generic Object
Truly generic composer possible One composer for all producer/consumer pairs
• Rather than camera/printer, temperature sensor/aggregator pairs
Errors possible• Temperature piped to printer
Programmer-defined Type checking possible Need programmer-defined composers.
39
Problems of Interface-based Approach
Number of DoAll Composers At least one for each interface
Number of Interfaces Implemented? One of for every operation
power, play can share composers for different devices
• can turn on all cameras and VCRs
too many interfaces and associated composers One of each service
limited sharing of composers need one for each combination of device types
40
Summary of Problems of Interface-based Approach
Explosion of # of composers In producer/consumer connections
for each consumer/producer data type In do-all composition
if interface = operation• for every operation
if interface = service• for every service combination with overlapping operations
No automation really supported in composing Automation only in looking up and
“composing”composers No support for proxy and adapters.
41
Alternate: Pattern-based approach
Pattern-based vs. interface- based protocols Pattern-based
must follow conventions
42
DoAll Composition
capture()
on()on()
capture()
Do all: on() Do all: capture()
on()
43
Method-based doAll algorithm Find operation headers of all component devices For each operation header
create a doAll operation that invokes device-specific implementation of header on each device providing it.
Accidentally combining unrelated commands with same signature foo(), compute(), doIt(), mutate()
Not combining related commands with different signatures power() powerVCR() and on()
non combinable commands() serialize()
44
Overriding Default doAll()
doAll on():
vcr.powerVCR(), tv.powerTV() and camera.turnOn()
Access methods
on = power.Device.()
serialize = serialize()
Disable doALL
serialize()
Access methods
on = turnOn()
serialize = serialize()
Disable doALL
serialize()
45
doAll Access-Method Algorithm
Find access-method headers of all component devices For each header
create a doAll operation that invokes an implementation of header on each device providing it that does not disable its compositions
46
Producer-Consumer Composition Much harder problem
how to regularize producers and consumers? Enumerate and Generalize Examples
Continuous vs. discrete through composer Unix pipes of continuous data vs. ICrafter connection of discrete
data Continuous direct connection
push-pull based continuous proxy based connection
push-pull based continuous adapter based connection
push-pull based Direct connection vs. connection through composer
47
interface PictureProducer {
Image capture();
...
}
interface PictureConsumer {
void print (Image p);
…
}
Discrete: Single producer, single consumer
Connect (camera, printer)
new operation: printAndCapture
48
interface PictureProducer {
Image capture();
...
}
interface PictureConsumer {
void print (Image p);
…
}
Discrete: Multiple producer, single consumer
Connect (cameras, printer)
new operation: captureAllAndPrint
49
interface PictureProducer {
Image capture();
...
}
interface PictureConsumer {
void print (Image p);
…
}
Discrete: Single producer, multiple consumers
Connect (camera, printers)
new operation: captureAndPrintAll
50
interface PictureProducer {
Image capture();
void setPictureDestination(PictureConsumer c);
void sendToPictureDestination();
...
}
interface PictureConsumer {
void print (Image p);
…
}
Continuous Push-based Connection
Connect (camera, printer):
camera.setPictureDestination(printer);
51
interface PictureProducer {
void capture();
void addPictureDestination(PictureConsumer c);
void sendToPictureDestinations();
...
}
interface PictureConsumer {
void print (…);
…
}
Pushing to multiple destinations
Connect (camera, printer):
camera.addPictureDestination(printer);
52
interface PictureConsumer {
void setPictureSource(PictureProducer c);
void getFromPictureSource();
...
}
interface PictureProducer {
void capture (…);
…
}
Direct Pull-based Continuous Connection
connect(camera, printer):
printer.setPictureSource(camera);
53
interface PictureConsumer {
void addPictureSource(PictureProducer c);
void getFromSources();
...
}
interface PictureProducer {
void capture (…);
…
}
Pulling from multiple sources
connect(camera, printer):
printer.addPictureSource(camera);
54
interface PictureConsumer {
void print();
...
}
interface PictureProducer {
void addPictureDestination(PictureConsumer c)
void capture (…);
}
Proxy based connection
interface PictureCompresser extends PictureConsumer {
void addPictureDestination(PictureConsumer c);
void compress();
...
}
registerProxy(ACompressor)
connect(camera, printer):
compresser new ACompresser();
compresser.addPictureDestination(printer);
camera.addPictureDestination(printer);
55
interface PictureConsumer {
void addPictureSource(PictureProducer c);
void getFromSources();
...
}
interface PictureProducer {
void capture (…);
…
}
Proxy-based pull connection
interface PictureCompresser extends PictureProducer {
void addPictureSource(PictureProducer c);
void compress();
...
}
registerProxy(ACompressor)
connect(camera, printer):
compresser new ACompresser();
compresser.addPictureSource(camera);
printer.addPictureSource(camera);
56
interface PictureConsumer {
void print();
...
}
interface PictureProducer {
void addPictureDestination(PictureConsumer c)
void capture (…);
}
Proxy vs. Adapter: Proxy
interface PictureCompresser extends PictureConsumer {
void addPictureDestination(PictureConsumer c);
void compress();
...
}
registerProxy(ACompressor)
connect(camera, printer):
compresser new ACompresser();
compresser.addPictureDestination(printer);
camera.addPictureDestination(printer);
57
interface BWPictureConsumer {
void print();
...
}
interface ColorPictureProducer {
void addPictureDestination(ColorPictureConsumer c)
void capture (…);
}
Proxy vs. Adapter: Adapter
interface PictureTranscoder extends ColorPictureConsumer {
void addPictureDestination(BWPictureConsumer c);
void transcode();
...
}
connect(camera, printer):
transcoder new APictureTranscoder();
camera.addPictureDestination(transcoder);
transcoder.addPictureDestination(printer);
58
Implementation of Continuous Connection
Motivated by composition of components of Collaboration Infrastructure called Collaboration Bus
Direct, Proxy based, and Adapter based continuous connections
Integration of registration/discovery and composition
Based on notion of facets (like pins on a chip)
Munson and Sunku
59
Facets public void add<FacetName>(<FacetType>) {...} [public remove<FacetName>(<FacetType>) {...}]
[public Enumeration enumerate<FacetName>() {...}]public void set<FacetName> (<FacetType>) {...}
[public <FacetType> get<FacetName> () {...}]public <FacetType> <facetName>;
FacetName (FacetType)
objectAssign (source, facet, destination)
source.addFacet(destination)
source.setFacet(destination)
source.facet = destination
60
Bi Directional Connect
connect (camera, pictureDestination, printer, pictureSource)
assign (camera, pictureDestination, printer) assign( printer, pictureSource, camera);
PictureDestination (PictureConsumer)
camera
printer
PictureSource(PictureProducer)
61
Uni Directional Connect
connect (camera, null, printer, PictureSource) assign (printer, pictureSource, camera)
camera
printer
PictureSource(PictureProducer)
62
Uni Directional Connect
connect (camera, pictureDestination, printer, null)
camera.setPictureDestination(printer)
PictureDestination (PictureConsumer)
camera
printer
63
Multiple Communication Primitives
May wish to pass reference to Object Socket Multicast channels
Pattern getReferenceFor<FacetName>() If this method exists
return value used instead of object defining the facet
64
Multiple Communication Primitives
connect (camera, PictureDestination, printer, PictureSource) assign (camera, PictureDestination,
printer.getReferenceForPictureSource()) assign( printer, PictureSource,
camera.getReferenceForPictureDestination());
PictureDestination (PictureConsumer)
camera
printer
PictureSource(PictureProducer)
65
Integrating connection and discovery
Connect (printer, PictureDestination) Integrate discovery and connection Discover an object whose type can be assigned to facet
Discovered objects must have been registered Registries distributed and connected to each other
Must now find peer facet public Facet get<FacetName>DestinationFacet().
Make bi-directional connect if such a facet exists, else unidirectional connect
66
Integrating registration and connection
Register(camera) For each auto connect facet, f
Connect (camera, f) Facet is auto connect if
member of AutoConnectFacetNames property Or if class of object
Implements AutoConnectAll interface
67
Proxy-based ConnectPictureDestination (PictureConsumer)
camera
printer
PictureSource(PictureProducer)
PictureDestination(PictureCosnumer)
Proxy
PictureSource(PictureProducer)
68
Proxy-based connectregisterProxy(p, “Proxy/facet1/type1/facet2/type2”)
connect (s, fs, d, fd)p = findProxy(“Proxy/fd/typeOf(s)]/fs/typeOf(d)”)
If (p != null)
connect (s, fs, p, fd), connect (p, fs, d, fd)
else if not (direct_connect(s, fs, d, fd))
adapterConnect(s, fs, d, fd)
;
69
Adapting the TypePictureDestination (ColorPictureConsumer)
camera
printer
PictureSource(BWPictureProducer)
PictureDestination
(BWPictureConsumer)Adapter
PictureSource
(ColorPictureProducer)
70
Adapting the FacetColorPictureDestination
(ColorPictureConsumer)
camera
printer
BWPictureSource
(BWPictureProducer)
BWPictureDestination
(BWPictureConsumer)Adapter
ColorPictureSource
(ColorPictureProducer)
connect (camera, ColorPictureDestination, printer, BWPictureSource) Cannot derive adapter facet names from
connect.
71
Unidirectinal: Adapting the FacetColorPictureDestination
(ColorPictureProducer)
camera
printer
BWPictureDestination
(BWPictureProducer)Adapter
connect (camera, ColorPictureDestination, printer, BWPictureSource) Can derive adapter from connect, but not
opposite facet
72
Adapting the TyperegisterProxy(p, “Proxy/facet1/type1/facet2/type2”)
adapterConnect (s, fs, d, fd)p = findProxy(“Proxy/fd/*/fs/*”)
connect (s, fs, p, fd), connect (p, fs, d, fd)
73
Adapting the FacetregisterProxy(p,
“Proxy/facet1/type1/facet2/type2/pfacet1/pfacet2”)
adapterConnect (s, fs, d, fd)
p = findProxy(“Proxy/fd/*/fs/*/pf1/pf2”)
connect (s, fs, p, pf1), connect (p, pf2, d, fd)
74
Creating proxiesregisterProxy(p,
“Proxy/facet1/type1/facet2/type2/pfacet1/pfacet2”)
registerProxy(pClass, “Proxy/facet1/type1/facet2/type2/pfacet1/pfacet2”)pClass instantiated automatically
registerProxy(pFactory, “Proxy/facet1/type1/facet2/type2/pfacet1/pfacet2”)
special method in pFactory to create proxy
75
Hybrid of Discrete and Continuous
Example set time of newly turned on vcr to that of clock
composite command: replicate clock time system calls getter method for clock property and
then setter method for vcr property when vcr service registered
Can generalize to arbitrary properties synchronize TV channels in a bar
user enters synchronize channel mode composer listens for channel properties and
synchronizes them Roussev’s coupler research valid here
76
Domain-specific Device Composition
Example Compose multiple devices, possibly of
different users, so that they work together to show a multimedia web presentation.
Websplitter Based on XML
77
WebSplitter: A Partial-View XML Framework For
Multi-Device Collaborative Web
BrowsingProfessor Rick HanDepartment of Computer ScienceUniversity of Colorado at [email protected]
78
Outline
A High-Level View of Pervasive Computing
WebSplitter Motivations Per-Device Partial Views Per-User Partial Views
WebSplitter’s XML Framework Demo Prototype
79
Motivation: Enabling A Pervasive Computing
World
80
WebSplitter Motivation #1: Generate Partial Views Per
Device Scenario: Web presentation in lecture hall controlled
by a wireless PDA
Lecturer
• Each device receives partial view• device’s capability• intended function
• Leverage other dev’s
• Composite view
Lecturer
81
WebSplitter Motivation #2: Partial-View Collaborative
Presentations Scenario: Online Web presentation to
audience – what you see is ~ what I see
Lecturer
Participant 2
Participant 1
82
Partial Views Per User WebSplitting generates subsets of a Web page
for each user, i.e. partial views
• Partial views limit access to components– To hide information– To prevent misdirection of a presentation
• Partial view scenarios:– Teacher giving presentation to students
(distributed)• Customized learning based on individual skill
levels, i.e. (teacher, pupil1, pupil2, …), not merely (teacher, students)
– Adversarial collaboration, e.g. lawyers
83
WebSplitter: Joint Multi-User Multi-Device Presentations
Scenario: Web presentation to online audience and a lecture hall via a remote control PDA
Lecturer
Lecturer
Participant 2
Participant 1
84
Multipleusers
MultipledevicesLecturer
Web
Server
SessionPresenter
WebSplitter
WebSplitter’s Architecture
WebSplitter Middleware
PolicyFile
---------------------------------------------
85
XML Framework:How does WebSplitter
use XML? Web pages are authored in XML
Tags correspond to components on Web page
Easy to manipulate tags & groups of tags – nested hierarchy
use XSL to filter/insert/delete tags “Policy file” implements partial views
defines tag mapping policies: which users and which devices have access to which XML tags
Written in XML External annotation file containing
authorization
86
Example: Splitting an XML Web page
<?xml version="1.0" encoding="UTF-8"?>
<sound xml:link="simple" href="music.wav"/>
<head>
<title>The online Shopping company</title>
<logo xml:link="simple" href="logo.gif"/>
<description>The most reliable and the best prices online</description>
</head>
WebSplitter
<HTML>
<BODY>
<center>
<table width="100%" cellpadding="0" cellspacing="0">
<tr>The online Shopping company</tr>
<tr><IMG SRC="logo.gif"></tr>
<tr>The most reliable and the best prices online</tr>
</table>
</center>
</BODY>
</HTML>
Title, Description, and GIF Logo
HTML to PocketPC
<?xml version="1.0" encoding="UTF-8"?>
<sound xml:link="simple" href="music.wav"/>
<head>
<logo xml:link="simple" href="logo.gif"/>
</head>
WAV Music and GIF Logo
XML to laptop
87
XML Policy File:Filtering Tags by User
Groups Define user privilege
groups <cmdb:group name=“ “
password=“ “ input_permitted=“ “>
Each privilege group receives a subset of tags <cmdb:taglist>
<cmdb:group name = ”lecturer” password = ”YYYYY” input_permitted = “yes”><cmdb:taglist>
presentation, head, title, nav_bar, slide, picture
</cmdb:taglist></cmdb:group>
<cmdb:group name = ”students” password = ”ZZZZZ” input_permitted = “no”><cmdb:taglist>
presentation, slide, picture
</cmdb:taglist></cmdb:group>
88
XML Policy File:Filtering Tags by Devices
Within a group, map subsets of tags to devices <cmdb:device
name=“ “> Standardizing
names
<cmdb:group name = ”lecturer” …<cmdb:device name = “IE||
Netscape”><cmdb:taglist> presentation,
head, title, nav_bar, slides, picture
</cmdb:taglist></cmdb:device><cmdb:device name =
“PocketIE”><cmdb:taglist> presentation,
head, title, nav_bar, notes </cmdb:taglist>
</cmdb:device></cmdb:group>
<cmdb:group name = ”students” ……………</cmdb:group>
89
XML Policy File: Filtering Tags By User &
By Device Define user
privilege groups Each privilege
group receives a subset of tags
Within a group, map subsets of tags to devices
<cmdb:group name = ”lecturer” …<cmdb:device name = “IE||
Netscape”><cmdb:taglist> presentation,
head, title, nav_bar, slides, picture
</cmdb:taglist></cmdb:device><cmdb:device name =
“PocketIE”><cmdb:taglist> presentation,
head, title, nav_bar, notes </cmdb:taglist>
</cmdb:device></cmdb:group>
<cmdb:group name = ”students” ……………</cmdb:group>
90
Implementation: WebSplitter Demo
Bluetooth
Bluetooth LAN Access
Point
EudoraWebLaptop2
WebSplitterproxy
Ethernet LAN
Laptop1
Web Server
XML Source & Policy File
2 privilege groups in policy file
91
WebSplitter Prototype Demonstrates:
Splitting to different browsers on different devices XML source Web page XML policy file
2 privilege groups Client-side component
on Palm, WinCE, Netscape
Static service discovery
92
Issues raised by WebSplitter
Integrate with Producer-Consumer Notion Splitting an output ~ connection to
multiple consumers Splitting an input ~ connection to multiple
producers Automatic vs. Declarative
93
Composition Techniques
Fully automatic doAll command pipes false positives/negatives possible
Semi-automatic disabling/enabling doAll composition Specifying proxies
Declarative WebSplitter XML file
94
Semi-Automatic vs. Declarative Fine line between semi-automatic and declarative Simple parameters (forms) rather than
specification language. Semi-automatic overrides fully automatic
