Network Computing Laboratory A programming framework for Stream
Synthesizing Service Slide 2 Network Computing Laboratory | 2 Korea
Advanced Institute of Science and Technology Outline Motivation
expected applications Scenario An example application Challenges of
application development Approach What to do Slide 3 Network
Computing Laboratory | 3 Korea Advanced Institute of Science and
Technology Motivation Stream Synthesizing Services (S 3 ) Receives
data streams from heterogeneous sources (context, rate..)
Continuously runs over the streaming data Includes complex logic
that requires operations beyond declarative queries E.g.
Location-based services (LBS), ITS, Continuously generated as a
form of streams Stream Synthesizing Services Stream Synthesizing
Services RFID GPS Sensor Net Mobile devices Slide 4 Network
Computing Laboratory | 4 Korea Advanced Institute of Science and
Technology Scenario Scenario: A Command Control System
Context-aware target prioritization & fire control CCS
Locations of adversary objects Radar, sensor net, sonar Armor
status, Vehicle condition, Health condition, Location Monitor
Command Slide 5 Network Computing Laboratory | 5 Korea Advanced
Institute of Science and Technology At each level of the hierarchy
A unique algorithm for target partitioning & fire allocation is
implemented Target partitioning & fire allocation changes as
Positions of objects change Data : Location stream, Processing :
speed & orientation detection Available fire power Data :
Physical condition of weapons & soldiers Data : biomedical
data, Processing : expert system for diagnosis Scenario Monitor
Command Slide 6 Network Computing Laboratory | 6 Korea Advanced
Institute of Science and Technology Challenges of S 3
development(1/3) Control flow management is inherently complex
Threaded model, message-passing model are both inappropriate for
the development Thread? Control flow of a tuple is not sequential
(condition waiting .) E.g. Tuple/Time windows are placed between
program modules Thread logic condition checking . (Example)
Message-Passing? An event-handler may be triggered by a synthesized
event Synthesized event: A packaged event formed with events from
multiple sources N N ? Enabling Flexible control flow expression is
required Slide 7 Network Computing Laboratory | 7 Korea Advanced
Institute of Science and Technology Synthesized stream Multiple
source stream input stream .. ?? Slide 8 Network Computing
Laboratory | 8 Korea Advanced Institute of Science and Technology..
weather traffic smoothingaggregation Traffic status S A R A A M
aggregation Black box Slide 9 Network Computing Laboratory | 9
Korea Advanced Institute of Science and Technology Challenges of S
3 development (2/3) Before running the logic, complex processing of
raw data streams is required Since values and data rates change
dynamically over time Operations on data from a single source
Buffering: due to the fluctuating data rate Windowing: logics
require aggregated data often (time, tuple window) Filtering:
logics require selected data according to values Operations on data
from multiple sources Scheduling data update: app semantic,
performance reason Updating constraints Temporal constraint, ratio
constraint( ), semantic constraints Logic Slide 10 Network
Computing Laboratory | 10 Korea Advanced Institute of Science and
Technology Challenges of S 3 development (3/3) Error inspection and
diagnosis is hard due to the long running feature and dynamicity of
data Many factors affect the execution of the services Data rate
Data values Running time Due to the time factor, it becomes harder
to find an error & verify the correction of the error A program
may have state which is built over time It may take some time to
browse the program state which leads to error E.g. a service may
start to misbehave after 15 hours of execution Unpredictable, Thus
hard to handle exceptional situations Slide 11 Network Computing
Laboratory | 11 Korea Advanced Institute of Science and Technology
Stream Synthesizing Services (S 3 ) Stream Synthesizing Services
Receives data from multiple sources, exhibits multiple merging
stages, and then generates final outputs Data from each source goes
through multiple processing steps As time passes (the steps may
require different time window) As more data arrives (the steps may
require different tuple window) Modules may require data from other
sources (or modules) Slide 12 Network Computing Laboratory | 12
Korea Advanced Institute of Science and Technology Approach Provide
a programming model and an execution framework to facilitate the
development of S 3 s Let developers to concentrate on service logic
by Delegating primitive data processing tasks to the framework
Interface which abstracts input data streams Hiding complex
execution flow Event-based programming Enabling easy verification
of the logic Emulating data sources Marking certain moment of
execution (?) Allow execution speed control (?) Programming
Framework Programming model Runtime Error diagnosis Slide 13
Network Computing Laboratory | 13 Korea Advanced Institute of
Science and Technology The waterwheel model Waterwheel: logic
segment which is triggered as stream arrives A stream creates event
& event triggers the logic A service is composed of waterwheels
Each waterwheel have different input streams Flexible event
definition & management The constructed waterwheels run as
streams are pushed from the sources Logic service Logic Slide 14
Network Computing Laboratory | 14 Korea Advanced Institute of
Science and Technology What to do Scenario refinement Realizing the
programming model Design the interface which is used for input
abstraction Define operations & constraints on multiple sources
Define events & define how they could be expressed Define how
the waterwheels can be organized into a program Design the runtime
Data management Supporting operations Connection between
waterwheels (outputs to inputs) Waterwheel scheduling Evaluating
events of waterwheels Programming Framework Programming model
Runtime Error diagnosis Slide 15 Network Computing Laboratory | 15
Korea Advanced Institute of Science and Technology What to do
Supporting error diagnosis Refine possible requirements Case study
Build one or two services Programming Framework Programming model
Runtime Error diagnosis Slide 16 Network Computing Laboratory | 16
Korea Advanced Institute of Science and Technology Scenario fuel
Engine temp Tire pressure My position Values from sensors Mobility
Map Orientation, speed Aim target Scenario: Military logistics
applications Guide an object (e.g. tank) strategically using
context information Vehicle status diagnosis Position of external
objects Raw stream pre-processing Complex execution flow Error
inspection & diagnosis Black box
