Event Processing Applied to Streams of TV Channel Zaps and Sensor Middleware with Virtualization

Introduction SenseWrap EventCaster Television Statistics Paradigm Comparison AdScorer

Event Processing Applied toStreams of TV Channel Zaps and

Sensor Middleware with VirtualizationPhD Defense

Pal Evensen

Department of Electrical Engineering and Computer ScienceUniversity of Stavanger, Norway

April 23rd, 2013


OutlineIntroduction

BackgroundProject ContextResearch Contributions

SenseWrapSenseWrap

EventCasterSystem ArchitectureEsper and EPL

Television StatisticsBackgroundImplementationStatisticsDeployment

Paradigm ComparisonPerformanceComplexityConclusions

AdScorerOverviewExperimentsLive Scoring ResultsSummary


From this


To this


Request/Reply

Client Server

Request

Reply


To this


and this


Publish/Subscribe

Publisher Subscriber

PublisherSubscriber

Subscriber

Subscriber

Event

Service

Notificationpublish

notify

subscribe/unsubscribe


IS-Home

Altibox at home

INTERNETFrom 10 to 400

Mbit/s, both ways

DIGITAL TVChoose between150 TV channels, film rental and more.

ALARM SERVICESAlarm monitoringcentre. Direct alarm connection to the firebrigade.

MOBILE PHONE SERVICESSame rate, no matter who you call

IP TELEPHONYNo charges to Landlines in Scandinavia

TvFILM RENTALChoose from 1000 titles Now in HD too

FOOTBALLThe whole Premier League and Tippeligaen (Norwegian football pools league)

NEWSUp to date local news

TV-GUIDEA complete overviewfor your TV evening


Research Contributions

Two main topics:

1. Sensors in smart homes• Middleware for hiding the heterogeneity of devices

2. Event processing applied to TV channel zaps• Architecture for efficient processing of high volumes of events• Evaluating programming models for stateful event processing


OutlineIntroduction


SenseWrapSenseWrap






Motivation

Sensorsand

actuators


SenseWrap Features

• IP-enabling sensor devices

• Uniform interface to hetereogeneous devices

• A “blueprint” for developers of smart home applications

• Automatic network configuration and service discovery


SenseWrap Architecture

VirtualSensor

Physical sensors

Sensewrap

VirtualSensor

Sensor protocol (ZigBee, Bluetooth, etc)

zeroconf

Driver Driver

Gateway

Service

UDP TCP

Service

TCP

Service

HTTP

Client Client Client


OutlineIntroduction


SenseWrapSenseWrap






Event-Driven Architecture

Event ProcessingNetwork

EventProcessingAgents

Event ConsumersEvent Producers


Components

• Esper

• HornetQ

• Custom Java glue


Event Flow and Interfaces

EventProcessor

Adapter

EventPublisher

EventProcessor

EventPublisher

Message bus

Producer Core

Event

EPL

EPLUpdateListener


Event Processing Language (EPL)

• Declarative query language derived from SQL

• Operates on stream data as opposed to relational data

• Looks for event patterns that matches the query, andproduces an output event

• Includes additional operators, such as sliding windows

• Part of the Esper framework (Open Source)


Continuous Queries


Sliding Windows

win:time(10 sec)

time0 10 20

now

e1 e2 e3 e4 e5

1 select ∗ from ChannelWin.win:time(10 sec)


Event Patterns

Simple

AEvent

Complex

CEvent

Simple

BEvent

1 select ∗ from pattern [every a=A −> b=B]


OutlineIntroduction


SenseWrapSenseWrap






Current State of TV Viewer Statistics

• Sample size is 0.045 % of Norwegian television households

• Only 0.022 % of American households are sampled

• Data collection requires specialized equipment

• Transferred once a day


Motivation

• IP-based Set Top Boxes (STBs) allows for more accuratestatistics

• Ads can be evaluated on an individual basis

• Additional behavioral markers such as mute and volumechanges allows for better understanding


The Future of Media Measurement

• Current measurement methods for TV not in line with newmodels of media consumption

• Gartner: The online/offline division of media will be replacedby measured/unmeasured by 2015

• STB data increasingly being used to augment traditional TVratings


Altibox Deployment of IPTV

• Over 320,000 STBs deployed in Norway and Denmark


Our Scenario

• Two-way communication

• Immediate transfer of zap, mute, hdmi and volume events


Observed Events

• Channel change event (also called a zap event)

• HDMI status event: TV set on/off

• STB audio on/off event (mute)

• STB volume change event


Number of zap events/day over a 15-day period

31 1 2 3 4 5 6 7 8 9 1011121314

2

2.2

2.4

2.6

·106

Avg=2212097


Zapping Activity

0 5 10 15 200

0.5

1

1.5

·105

Time (hrs)

30 Minutes15 Minutes5 Minutes


EPL Viewer Statistics

Esper Engine

Zaps ChannelWinchannelName,viewersselect(...)

channelName,viewers

1 select channelName, viewers2 from ChannelWin3 output snapshot every 1 sec


Tru

de

/18

Sy

kk

el:

Gir

o d

`It

ali

a 1

4.

eta

pp

e

Sp

ort

& S

pil

l

Re

ise

må

l

Lø

rda

gsm

ag

asi

ne

tS

po

rtsn

yh

ete

ne

Væ

ret

Vin

n p

å m

inu

tte

t 9

:12

Ny

he

ten

e

Go

d k

ve

ld N

org

e

Sp

ort

sny

he

ten

e

Fil

m:

Mo

rd p

å k

red

itt

Sy

kk

el:

Gir

o d

`It

ali

a -

op

psu

mm

eri

ng

Fa

rme

n 3

0:3

0

Ny

he

ten

e

Ak

va

rie

t

TV

2 h

jelp

er

de

g 1

1

0

10000

20000

30000

40000

50000

60000

12:00 14:00 16:00 18:00 20:00 22:00

Vie

wer

s

Time

Viewers for TV2 Norge + TV2 HD, Saturday 21.05.2011, Separated by Zip Codes

Total viewers0001-39994000-59996000-79998000-9999


30000

35000

40000

45000

50000

55000

60000

65000

70000

10:00 12:00 14:00 16:00 18:00 20:00 22:00

See

re

Tidspunkt

Seerstatistikk for TV2 Norge og NRK1, Lørdag 21.05.2011

TV2 Norge + HDNRK1 + HD


Annoyance Detection

• Aimed at detecting ads/programs that is causing viewers tochange channel

• Triggers an output event on rapid drop in viewers on aparticular channel

• Sliding window algorithm, continously comparing viewernumber with the last minute average


Annoyance Detector

Annoyance detector

ZapSnap.win.time(1 min)

channelName,viewers,avg(viewers)

avg(viewers)

having(...)

channelName,viewers

1 select channelName, viewers, avg(viewers)2 from ChannelWin(viewers > 2000).win:time(1 min)3 group by channelName4 having viewers < avg(viewers) ∗ 0.85


Deployment Details

• Processes over 40,000 events/min at peak hours

• Deployed in the Altibox network across four servers:

ZapCollector Messagebus

DatabaseCore

+Manager


CPU load - ZapCollector


CPU load - Message bus


CPU load - EventCaster instance


OutlineIntroduction


SenseWrapSenseWrap






EPL and Java Implementations


Experimental Setup for Performace Evaluation

• 2,117,897 zap events (one day)

• RHEL6,Quad Core 2.4GHz CPU, 14GB RAM

• Processing capacity tested by loading events from:• Memory• Disk• Network: UDP and HornetQ

• Memory consumption tested by loading events from:• Memory• Disk


Processing capacity

Memory Disk UDP HornetQ

0

2

4

6

8·105

Eve

nts

/sec

Esper Java


Memory consumption

From memory From disk

600

800

1,000

1,200

1,400

1,600 1,570

1,401

590 577

Mem

ory

use

d(M

B)

Esper Java


Performance Evaluation - Observations

• The Java implementation outperforms the Esper version

• Can be accredited to abstraction overhead imposed by theEsper engine

• Performance margin decrease as network overhead isintroduced, but is still significant

• UDP benchmarks offer 45% higher throughput with the Javaversion


Halstead’s Metrics

• Metrics that describe the complexity and effort required towrite a program

• Sees a program as a series of tokens, classified as eitheroperands or operators


Complexity Metrics

0 100 200 300 400

Vocabulary (n1+n2)

Program length (N1+N2)

Total operators (N1)

Total operands (N2)

Unique operators (n1)

Unique operands (n2)

Lines of code

(Java)(EPL)

Viewer Statistics

Annoyance Detector

Parsing and Query Setup

Utility Functions


Complexity Evaluation - Observations

• The EPL implementation scores slightly better for the viewerstatistics application

• EPL scores significantly better for the annoyance detector

• Initial effort was similar for both applications

• Expanding EPL applications is significantly easier than Java


Summary

• The Java implementation is the most performant

• The Esper implementation is easier to expand

• Can be attributed to different degrees of abstraction


OutlineIntroduction


SenseWrapSenseWrap






Deployment

TV Network

AdDetectorMute

AdSuccess-Evaluator

STBs

Inp

ut ad

apters

Stats

Ad start

Channel zap events

Filterin

g, tran

sformation

Zap

Ad success score

Historical stats

Processing

QueueingInput


Defining an advertisement as a context in EPL

Listing 1: Context declaration

1 create context AdBreakCtx as2 initiated by3 AdIdentified(begin=true) as ad4 terminated by5 AdIdentified(detectId=ad.detectId, begin=false) as endAd

Listing 2: Populating the context

1 context AdBreakCtx2 create window STBsnapshots.win:keepall() as STBWin3 insert where4 channel in (context.ad.channel) and hdmi=true and mute=false


Experimental Setup for AdScorer Evaluation

• 1.5 hour of prime time TV was recorded

• Advertisement times were manually recorded

• 23 days of STB data was sent through the system in order toreach a correct state

• Logged STB data and AdIdentified-events was then pushed tothe system

• Simulation run


Viewer numbers

Figure: Viewership (in thousands) for the three largest channels, NRK1,TV2, and TVN.


Viewer numbers

12

13

14

15

16

17

18

19

20

21

S E S E S E

TVN

Figure: Viewership (in thousands) for a single channel (TVN).


Retained Viewers


Gained Viewers


Raw output for a single ad

1 Event: {2 AdId: Mitsubishi ASX,3 channel: TV2 Norge,4 startTime: 22:55:12,5 stopTime: 22:55:32,6 viewersBegin: 83846,7 retained: 79244,8 IAR: 94.51,9 mutes: 141,

10 viewersLost: 4602,11 average volume: 49.6412 }


Summary

• AdScorer is capable of delivering an unprecedented level ofdetail, not possible through the current measurement regime

• The results indicate that our implementation is capable ofscoring advertisements on multiple channels simultaneously innear real-time with consistent results


Thank You for listening!

Technology

Event Processing Applied to Streams of TV Channel Zaps and Sensor Middleware with Virtualization