Low-Cost Approximate and Adaptive Monitoring Techniques for the Internet of Things

3/9/17 1Demetris Trihinas 1

Low-Cost Approximate & Adaptive Monitoring Techniques for the Internet of Things

Demetris [email protected]

PhD Candidate Department of Computer Science

University of Cyprus


Cloud Computing

Big Data Systems and Analytics

Internet Computing and IoT

Online Social Networks

Activity Areas

Current EU-Funded Projects

http://linc.ucy.ac.cy/




The Internet of Things

The physical world is now becoming an information system

Exchanging continuous data streams with other network-enabled devices,

systems and humans…

Physical (battery-powered) and network-enabled devices with smart processing capabilities…


Why is IoT Just Now Becoming a Reality?Nano Bio-Sensors and

Printed ElectronicsCommunication

ProtocolsData-Mining

Learning Algorithms

- Distributed closed neighbors

- Outlier/event detection

- Pattern matching

- Stream processing

- Learning algorithms

- ….“The Emerging Internet of Things Market Perspective: A Survey”, Perera et al., IEEE Trans. Computing, 2015

“Internet of Things (IoT): A Vision, Architectural Elements, and Future Directions”, J. Gubbi and R. Buyya, FGCS, 2013

The Building Blocks of the Internet of Things


IoT in AgriculturePrecision Farming

Detect crop condition and spread nutrients to parts of field with need

Cattle as a ServiceDetect optimum breading

period, monitor health and dairy protection

“Connected Cattle: Wearables are Changing the Dairy Industry”, Kathy Pretz, IEEE Magazine: the Institute, May 2016.


IoT in Medicine and Health Tracking

Precision MedicinePill-shaped cameras traversing

human body

“Fitbit data led doctors to shock a patient’s heart”, Steven Dent, Engadget, Apr 2016.

WearablesBiosignal and activity tracking


IoT in Intelligent Transportation SystemsSmart Traffic Flow

In Toronto travel time was reduced by 26%

Smart ParkingIn California carbon emissions were reduced by 730 tons per

15 block radius

“Can smart traffic lights ease Toronto’s road congestion?”, Joseph Hall, Toronto Star, Jun 2014.


The “Big Data” in IoT[IDC, Jun 2014]

[Cisco, Apr 2011]

IoT Monitoring Data 2% of digital universe in 2012 Projections for >12% in 2020

[Gartner, Mar 2015]

IP Traffic 1.6ZB in 2018, 57% from IoT Devices

[Cisco, Apr 2014]


Big Data and the Cloud


IoT and Cloud Computing

Driving Cloud Adoption for IoT Developers

- Simplified resource management

- low capital expenses to get started [IoT Developer Survey, Eclipse Foundation, 2016]

67% of production-ready

IoT services reside in the

cloud [State of IoT, Embarcadero, 2016]


So…is Cloud Monitoring Really Ready for the Internet of Things?


Monitoring Topologies

Scalability and Single points of failure

Monitoring servers closer to the root are overloaded

with relay costs preventing scalability

“JCatascopia: Monitoring Elastically Adaptive Applications in the Cloud”, D. Trihinas, G. Pallis and M. D. Dikaiakos, IEEE/ACM CCGrid 2014


Monitoring as a Service (MaaS)• Eases management of the monitoring infrastructure

• Service providers enable multi-tenant and scalable

monitoring over the internet

• Pay-as-you-use business model (although usually hourly)

Monitoring libraries for metric collection from

popular languages and frameworks

REST API to insert/extract

monitoring data

Shared data warehouse with services to query, process and

view analytic insights

custom app-specific metricsmanaged by cloud provider


Augmenting IoT with the Cloud

“

app

The Harsh Reality

Network latencies, bandwidth limitations,

pricing (in/out cloud traffic is billed),

constant location awareness

Datacenter

Interesting Articles“The cloud is not enough: Saving IoT from the cloud”, B. Zhang et al., Usenix HotCloud 2015“Taking the internet to the next physical level”, V. Cerf and M. Senges, IEEE Computer, 2016

The IoT Developer Perspective


The Cost of Monitoring

Based on AWS pricing scheme (May 2015)

for a 3-tier (video) streaming application

$0.12/GB network traffic

“Monitoring Elastically Adaptive Multi-Cloud Services”, D. Trihinas, G. Pallis and M.D. Dikaiakos, IEEE TCC 2016


P2P Monitoring• Monitored elements form Gossip Overlay Network to propagate monitoring

info -> ease coordination and query computation

• Gossip denotes periodic and pairwise propagation of the current state between

monitored peers“Monitoring Elastically Adaptive Multi-Cloud Services”, D. Trihinas, G. Pallis and M.D. Dikaiakos, IEEE TCC 2016

“Self managing monitoring for highly elastic large scale cloud deployments”, S. Ward et al., ACM DIDC 2014


Edge Computing

Init

Sense

Act

Sleep

Receive

Decide

Transmit

A term coined to reflect data processing and decision-making on “smart”

devices that sit at the edge of IoT networks

Init

Sense

Transmit

Sleep

Interesting Articles“The promise of edge computing”, W. Shi and S. Dustdar, IEEE Computer, 2016

“The swarm at the edge of the cloud”, E. Lee et al., IEEE Design & Test, Jun 2014

Simple Sensing

Edge Computing

<10ms

Big Data Traffic

>100ms

ProcessingNo Processing


Augmenting IoT with the CloudChallenge 1 – Taming data volume and data velocity with limited processing and network capabilities in the IoT/Edge realm

Challenge 2 - IoT devices are usually battery-powered which means intense processing leads to less battery-life

Processing and data dissemination are the main energy drains in embedded

and mobile devicesInteresting Article

“Practical data prediction for real-world wireless sensor networks”, U. Raza and T. Palpanas, IEEE TKDE, 2015


Augmenting IoT with the Cloud

“Taking the internet to the next physical level”, V. Cerf and M. Senges, IEEE Computer, 2016

Challenge 3 – Security and data privacy

October 2016, DDOS Attack


Formulating the Challenges as Research Questions


Preliminaries• A monitoring stream published by a monitoring source is a large stochastic

sequence of i.i.d datapoints, denoted as , where and

• A datapoint is a tuple described by a unique identifier for the monitoring

source , a timestamp and a value

• A datapoint may have a set of other attributes (e.g., location) although for

brevity we will omit other attributes without loss of generality

𝑑𝑖

𝑑𝑖+1

Monitoring Stream M


Periodic Sampling• The process of triggering the collection mechanism of a monitored source

every time units such that the datapoint is collected at time

𝑑𝑖

𝑑𝑖+1

Metric Stream M sampled every T = 1s

Compute resources and energy are wasted while generating large data volumes at a high velocity

Metric Stream M sampled every T = 10s

Sudden events and significant insights are missed


More processing (e.g., peak detection

for hr monitoring) Driving Peripherals(e.g., accelerometer,

leds)

Interesting Articles“Energy-harvesting wearables for activity-aware services”, S. Khalifa et al., IEEE Internet Computing, 2015

“Taking the internet to the next physical level”, V. Cerf and M. Senges, IEEE Computer, 2016

Emulated behavior of wearable device on Raspberry Pi with ARM processor

(1 core 32MHz, 128MB RAM)

Periodic Sampling with Physical Sensing

OS Monitoring


Periodic Sampling in an Energy Perspective

Lithium battery (35mAh) State of Charge (SoC) projection for a wearable

device with step, calorie and heartrate monitoring

Interesting Article“A universal state-of-charge algorithm for batteries”, B. Xiao et al., ACM DAC, 2010


Periodic Dissemination

Monitoring Source

Receiving Entity

• The process of triggering the network controller of a monitored source every

time units such that the datapoint is disseminated at time to interested

receiving entities

No data loss

• Aggregation-based policy dissemination:

• Withhold triggering dissemination until a window of datapoints is collected

• Withhold dissemination for fixed period of time and disseminate all datapoints collected until

then


Periodic Dissemination Monitoring Cost

𝜇𝑠+𝛽𝑠∙ χ +ε s

per message costper χ byte cost

datapoint d (t,v)

-> message compression

Interesting Article“Real-Time Data Analytics in Sensor Networks”, T. Palpanas, ACM Sensors, 2013

energy for state transitions

Example Values (Mica2)

mJ

mJ mJ/byte


Low-Cost Approximate and Adaptive Monitoring


Preliminaries• Approximate Techniques?

• Decision mechanisms based on estimation models capturing and predicting

monitoring stream runtime evolution within certain accuracy guarantees

• Adaptive?

• Adapt monitoring source properties based on the predicaments of the

estimation models

• Low-Cost?

• The costs (time, resources) of applying adaptive monitoring techniques are

(much) less than leaving the monitoring process as is

• We are interested in techniques running in-place and inexpensively


Adaptive Sampling• Dynamically adapt the sampling period based on some estimation model ,

containing runtime information of the monitoring stream evolution

• How large of an adjustment is required, “ideally”, depends on some evaluation

(distance) metric denoting the ability of the model to (correctly) estimate and

follow metric stream evolution

𝑑𝑖𝑑𝑖+1

𝑇 𝑖+1

Metric Stream dynamically sampledMetric Stream with


Adaptive Sampling

𝑑𝑖𝑑𝑖+1

𝑇 𝑖+1

Metric Stream dynamically sampledMetric Stream with

Find the max to collect based on an estimation of the monitoring stream

evolution , such that differs from less than a user-defined imprecision

value ()


Adaptive Dissemination• Dynamically adapt dissemination rate by applying approximation

techniques to sensed datapoints to reduce communication overhead

• By suppressing consecutive datapoints from dissemination with

“little” change in their metric values

• How much is change is considered as “little” depends on:

• The approximation technique

• Accuracy guarantees given by the user and denoting the probability

(confidence δ) with which sensed datapoints are approximated


Adaptive Dissemination (Model-Based)

• Monitoring source maintains runtime estimation model capturing monitoring

stream evolution and variability

• At the time interval instead of metric values, the model is disseminated

• Receiving entities predict the IoT device state from given model assuming

subsequent k-datapoints can be approximated within the given guarantees

Metric Stream Approximated Metric Stream


Adaptive Dissemination (Model-Based)

• Monitoring source withholds further dissemination interacting with receiver

only when shifts in monitoring stream value distribution render model as

inconsistent with the actual IoT device state

• If model parameterization is inconsistent, at this point, it must be updated

decision criteria?decision function?

Metric Stream Approximated Metric Stream


Metric Filtering• A form of adaptive dissemination: suppress latest datapoint(s)

dissemination if their values have not “changed” since last reported

• Metric stream suppression but… no model -> no forecasting

• Fixed filter ranges assume monitoring stream value distribution will not

change in in the future (no guarantees any values will be filtered at all)

if (curValue [prevValue – R, prevValue + R ])

filter(curValue)

Interesting Article“Monitoring, aggregation and filtering for efficient management of virtual networks”, S. Clayman, IEEE NOMS, 2011


Adaptive Filtering• Dynamically adjust the filter range based on the current variability of the

monitoring stream, denoted as Metric Stream with adaptive RangeMetric Stream

𝑑𝑖𝑑𝑖+1

𝑅𝑖+1𝑅𝑖

• Find the max filter range for such that differs from less than a user-defined

imprecision value based on the variability of the metric stream


AdaM


The Adaptive Monitoring Framework

μProcessor

μOS

Memory

Low-Cost Approximate Stream Estimation

AdaptiveSampling

AdaptiveFiltering

SENSING

UNIT

metricupdates

API

updated periodicityestimation confidence

AdaptiveDissemination

model updates & buffer content when

shift detected

updated filter rangestream variability

filtered metric stream

AdaMNETWORK

UNIT

ModelBase

stream evolution and online stats

Datapoint Buffer

IoT Device

metric stream

compressedmetric stream


Low-Cost Approximate Estimation Model

• Three (3) phase approach to approximate monitoring stream

evolution and variability by exploiting knowledge (hidden) in

the monitoring stream

Step 1Estimate Monitoring

Stream Evolution and label datapoints

as “(un)expected”

Step 2Detect if Gradual

Trends Exist in Monitoring Stream

Evolution

Step 3Test if Seasonality

Enrichment is Beneficial to Stream

Evolution



Phase 1: Update runtime monitoring stream evolution

• Probabilistic Exponential Weighted Moving Average (PEWMA) to estimate

from and the standard deviation :

• Datapoints labelled as “expected” if estimation lands in prediction intervals

determined from user confidence guarantees or “unexpected” otherwise

• “Unexpected” datapoints are stored in local buffer for dissemination

Looks like an exponential moving average, right?

But weighting is probabilistically applied!



EWMA after spikes overestimates

subsequent values

EWMA slow to acknowledge spikes

With probabilistic reasoning each datapoint will contribute to the

estimation process depending on it’s p-value

Why use a Probabilistic EWMA?


AdaM: Adaptive Sampling and Filtering

“AdaM: Adaptive Monitoring Framework for Sampling and Filtering on IoT Devices”, D. Trihinas, G. Pallis and M.D. Dikaiakos, IEEE BigData 2015“[Low-Cost Adaptive Monitoring Techniques for the Internet of Things”, D. Trihinas, G. Pallis and M.D. Dikaiakos, IEEE TSC 2016

μProcessor

μOS

Memory


AdaptiveSampling

AdaptiveFiltering

SENSING

UNIT

metricupdates

API




shift detected



AdaMNETWORK

UNIT

ModelBase


Datapoint Buffer

IoT Device

metric stream



Adaptive Sampling• After updating estimation model, we compute current confidence of

approach based on the actual and estimated standard deviation

• Next, we update sampling period based on the current confidence and

the user-defined imprecision γ (e.g. 10% tolerance to errors)

𝑐 𝑖=1− ¿ �̂� 𝑖−𝜎 𝑖∨¿𝜎 𝑖

¿

… the more “confident” the algorithm is, the larger the

outputted sampling period can be…

λ is an aggressiveness

multiplier (default λ=1)


Adaptive Filtering• Compute current variability of the metric stream using moving Fano

Factor which is an indicator of the stream value dispersion

• We then compare to the user-provided maximum tolerable imprecision

guarantees in attempt to widen filter range

𝐹 𝑖=𝜎 𝑖2

𝜇𝑖

…a low (due to ) indicates a currently in-dispersed data

stream which means low variability in the metric stream…


AdaM in action!Steps Heartrate

Calories

94% accuracy!

96% accuracy!

91% accuracy!


6x less processing! 4x less network traffic!

4x less energy usage!

+ AdaM

3 - 5 days

7 - 8 days

AdaM in action!


Memory TraceJava Sorting Benchmark

CPU TraceCarnegie Mellon RainMon Project

Disk I/O TraceCarnegie Mellon RainMon Project

TCP Port Monitoring TraceCyber Defence SANS Tech Institute

Step TraceFitbit Charge HR Wearable

Heartrate TraceFitbit Charge HR Wearable

More datasets!


Balance between efficiency and accuracy!

Energy Consumption

CPU Cycles Network Traffic

Error (MAPE)

FAST [L. Fan et al., SIGMOD 2014] L-SIP [E Gaura et al., IEEE Trans. on Sensors, 2013]


Dublin Smart City Intelligent Transportation Service (Dublin ITS)

.

.

.

.

.

.

1000 Buses* with GPS tracking

sending updates to ITS with 16

params (e.g. busID, location,

current route delay)

*Real data from Jan. 2014

Apache Kafka queuing

service on x-large VM

(16VCPU, 16GB RAM,

100GB Disk)

Apache Spark cluster with 5 workers on large VMs

(8VCPU, 8GB RAM, 40GB Disk)

Alerts ITS operators when more than 10 buses in a

Dublin city area, per 5min window, are reporting delays

over 1 standard deviation from their weekly average

AdaM… integrated in big data streaming service!


Spark total delay (processing + scheduling) for T=1, 5, 10 intervals and

AdaM with max imprecision γ = 0.15



AdaM achieves a >85% accuracy in major Dublin areas



AdaM: Adaptive Dissemination

“ADMin: Adaptive Monitoring Dissemination for the Internet of Things”, D. Trihinas, G. Pallis and M.D. Dikaiakos, IEEE INFOCOM, 2017

μProcessor

μOS

Memory


AdaptiveSampling

AdaptiveFiltering

SENSING

UNIT

metricupdates

API




shift detected



AdaMNETWORK

UNIT

ModelBase


Datapoint Buffer

IoT Device

metric stream



AdaM: Adaptive Dissemination

“ADMin: Adaptive Monitoring Dissemination for the Internet of Things”, D. Trihinas, G. Pallis and M.D. Dikaiakos, IEEE INFOCOM, 2017

NetworkUnit

ADMin

Seasonality Enrichment

Shift Detection

API

ModelBase

Approximate Stream

Estimation

Remote seasonal periodicity

updatetest if enrichment

is beneficial?

metric updates

update stream

evolution and trend

seasonal enrichment of

stream evolution

compressed metric stream

Seasonal Periodicity Detection

The ADMin plugin for

AdaM Don’t disseminate

datapoint values...

Instead disseminate

model updates!



Step 2: Detect over time gradual trends in monitoring stream to reduce

“lagging” effects in monitoring stream evolution estimation

�̂� 𝑖+𝑘∨𝑖❑←𝜇𝑖+𝑘 𝑋 𝑖

Upward trend

Downward trend

• Holt’s Trend Method used to bring moving average to

appropriate value base

• Improve forecasting from 1-step ahead (moving

average) predictions to k-datapoint values



Step 3: Test if seasonality enrichment is beneficial to estimation process

and update low-cost approximate model accordinglyseasonal

Seasonal with exponential trend

Seasonal with damped trend

• Tendency of the metric stream to exhibit behavior that

repeats itself every L periods (e.g., hourly)

• Seasonal effects highly evident in IoT data (e.g., human

biosignals, environmental data)

PV Panel Production Air Temperature



• Holt Winter’s Method used to estimate seasonal contribution

• Forecasting k-subsequent datapoints with trend and seasonality

• However… perfect seasonal behavior is rarely observed in real-life

systems PV Panel Production

Considering day before hourly average (Si-L) will lead to overestimation



• Online testing (T-Test) to determine if seasonal contribution beneficial

to estimation or not

• Detecting optimal seasonality cycle (L) is an open research challenge

especially when different cycles exist in monitoring stream

• Approximate runtime seasonal periodicity detection

• ComCube Framework (Matsubara et al., WWW, 2016): lightweight tensor-based

and parameter-free framework for near-optimal seasonal periodicity detection

“Non-linear mining of competing local activities ”, Y. Matsubara, Y. Sakurai and C. Faloutsos, WWW 2016


Detecting Shifts in a Monitoring Stream

• Cumulative Sum (CUSUM) to detect shifts in monitoring stream value

distribution which render estimation model as inconsistent

ts

prior shift

after shift𝑐 𝑖=ln

𝑃 (𝑀𝑖 ,𝜇′ ′ )𝑃 (𝑀𝑖 ,𝜇

′ )

where ε magnitude of change

1. log-likelihood ratio 2. Detecting shifts in mean

However, not known beforehand but… estimation model model provides us with an approximate


Detecting Shifts in a Monitoring Stream3. Online CUSUM

4. Decision Function

Challenge 1: Linear time… Can be used

instead of ?

ts ti

ti may greatly from ts differ in

cases of gradual trends

𝑖𝑓 𝐺𝑖 ,{𝑙𝑜𝑤 ,h h𝑖𝑔 }>h❑→h𝑠 𝑖𝑓𝑡 𝑑𝑒𝑡𝑒𝑐𝑡𝑒𝑑

h is measured in standard

deviation units5. Actual Shift Time

the time the CUSUM detects

the shifts


Detecting Shifts in a Monitoring Stream

ts ti

Trend and seasonality knowledge provide model with greater accuracy () and to adapt to

unexpected, abrupt and and volatile changes is monitoring stream

Challenge 2: CUSUM threshold h static and

sensitive when stream variability is low

( thus triggering… false alarms

Adapt CUSUM sensitivity by adapting

h after dissemination triggered and

restrict h with hmin


ADMin Evaluation• ADMin in 3 configurations

• No seasonality enrichment (ADMin)

• Static seasonality enrichment - previous day hourly average (ADMin_S1)

• Dynamic seasonality enrichment - ComCube integration (ADMin_S2)

• Under comparison frameworks

• LANCE [Werner et al., ACM SenSys 2011]

• G-SIP [Gaura et al., IEEE Trans. on Sensors 2013]

• ADWIN [Bifet et al., SIAM 2010]

All three framework parameters configured to output best results


Photovoltaic Panel Current (IDC) Production

2 Weeks of data collected every 1 second

Data reduction: 87% -- Accuracy: 93%

Weather Station Air Temperature (oC)

2 Weeks of data collected every 1 second


Wearable Human Heartrate (bpm)

2 Weeks of data collected every 1 minute


ADMin in Action!


Shift Detection Evaluation

ground truth(offline PELT algorithm)

false alarms

correctly detected

shifts

Air Temperature Dataset Wearable Dataset

Shift Detection Delay


Energy Consumption

Data Volume Reduction

1O interval

aggregation

ADWIN large energy consumption due to

complexity and absence of data reduction

scheme

LANCE large energy consumption due to enabling

dissemination x2 more than ADMin (false alarms) and

downloading each time all available datapoints

G-SIP large energy consumption due to shift

detection sensitivity (only EWMA based) which

enables false alarm disseminations

Overhead Evaluation


Facebook DB Cluster Adaptive Monitoring

“Inside the Social Network’s (Datacenter) Network”, A. Roy et al., ACM SIGCOMM, 2015


• Edge-mining on IoT devices is both resource and energy

intensive

• Big data streaming engines struggle to cope as the volume

and velocity of IoT-generated data keep increasing

• Adapts the monitoring intensity based on current metric

evolution and variability

• Reduces processing, network traffic and energy

consumption on IoT devices and the IoT network

• Achieves a balance between efficiency and accuracy

The AdaM Framework

Conclusion


http://linc.ucy.ac.cy/AdaM

http://linc.ucy.ac.cy/AdaM


Low-Cost Approximate & Adaptive Monitoring Techniques for the Internet of Things

Demetris [email protected]

PhD Candidate Department of Computer Science

University of Cyprus

Technology

Low-Cost Approximate and Adaptive Monitoring Techniques for the Internet of Things