05/03/2023 1

EventShopReal-Time Macro Situation Recognition

from Heterogeneous Streams

Siripen PongpaichetUCI ISG Talk02/14/2014

05/03/2023 2

Web

Location BasedMobile Applications

Ongoing Archived Database System

satelliteCloudresources

Environmental Sensor Devices

Internet of Things

Social Media

Billions of geo-location and

time based devices

Social Life Network

Real-timeInformation sharing

&decision making

Experts

People

Governmental Agencies

Situations

[Jain 2011] Social Life Network

05/03/2023 3

Examples of (Specific) System in SLN approach

one-touch SOS

Emergency SituationDaily Situation

Social Life NetworkConnect People to real-world Resources

effectively, efficiently, and promptly in given Situations.

EventShop : Global Situation Detection

Situation Recognition

Evolving Global Situation

….

Data Ingestion

and aggregation

Database Systems

Satellite

Environmental Sensor Devices

Social Network

Internet of Things

05/03/2023 4

00

Need- Resource Matcher

Recommendation Engine

Actionable Information

Resources

Needs

Personal Situation

Recognition

Personal EventShop: Personal Situation Detection

Evolving Personal Situation

Data Ingestion

Wearable Sensors

Calendar

Location….

Dat

a So

urce

s

05/03/2023 5

History of EventShop• Building as part of SLN framework• Environment and visualization tool for analyzing

heterogeneous data streams in macro scale• Help non (CS) technical experts in various domains to easily

conduct experiments for detecting real-world situations• Representing geo-spatial data in grid structure called E-mage• Generic set of operators for detecting situations• Pioneers: Vivek Singh (MIT), Mingyan Gao (Google)

6

EventShop UI

11/13/2013

Example Notification / Alerts:

You are currently in the area where there is a high chance of flooding,

these are available shelters within 10 miles around you.Space

Time Situation

Resources

People

05/03/2023 7

Current State and Next Steps

• Enhance EventShop Architecture• Collaboration Research (with NICT): – Sticker 3D visualization tool, – EventWarehouse

• Multi Granularity E-mage• Predictive Analytics• SLN Use Case

05/03/2023

OutputIngestor

Data Source Parser

Data Adapter

Emage Generator

(+resolution mapper)

Processing

EvShop Storage

Query Parser

Query Rewriter

Event Stream Processing Executor

Action Parser

Register Data Source Register Continuous Query

Situation

Emage

Visualization (Dashboard)

Actuator Communication

Action Control

Event Property & Other Information

(e.g., spatio-temporal pattern)

ᴨ

ᴨµ

Data Access Manager

Live StreamArchived Stream

Situation Stream

EventShop Architecture

Physical Data Source (e.g., sensor

streams, geo-image streams)

Logical Data Source

(e.g., preprocessing data streams, social

media streams)

Raw Event

9

From Heterogeneous Data to Situation Recognition in EventShop 2.0

11/13/2013

05/03/2023 10

EvS Input ManagerExternal Event Preprocessing

(EvWarehouse)

Real-Time Sensor Streamse.g., Cloud Satellite Pictures,

Gridding Data

Real-Time Sensor Streamse.g., Wind Speed, Traffic Flow

Real-Time Sensors

Event Model Wrapper 1D STT to Emage

Event Model Wrapper 2D

Data Adapter Emage Generator

Emage

Emage Factory

STT

Emage

Raw Social Media Streamse.g., Twitter, News RSS Feed

Near Real-Time Sensors

Event Model Wrapper

STT to Emage

Data Adapter Emage Generator

Emage

Emage Factory

STT

Topic Event Detection

Abnormal Event

Detection

Raw Sensor Streamse.g., PM2.5 data

“EventModel” Streamse.g., suddenly change

of data trend within time window

Emage Store

STT Store

Metadata Store

EventSource Parser Interface

ES internal storage(Optional)

RealTime Emage Streams

NearRealTime Emage Streams

Processing Manager

ES Descriptor

ES Control (Start/Stop/View ES)

Users Input

Data/Events Flow

ThemeAdapterTypeSourceURLTimeWindowParameters

InitialResolutionAggregationFunc

Metadata

05/03/2023 11

Stream Processing Engine

Operators Manager

Built-in Operators

User-Defined Operators

ᴨ

ᴨµ

Data Access

ᴨ

ᴨµ

Data Access

ᴨ

ᴨµ

Data Access

Input Manager

Event Stream Executor

Operators Nodes

Storage

AsterixDB, SciDB, MongoDB

Emage Store

STT Store

Metadata Store

Query Parser Interface

Query Descriptor

Query Control (Start/Stop/ View)

Real-time/ near real-time Emage Streams

Archived Emage Streams Situation Streams

Emage Interpolation

Function

Emage Conversion

Final Resolution,Interpolation Func

Parameter Operators

Operators

Store Parameters Retrieve Parameters

Query Rewriter

Execution Plan

05/03/2023 12

Current State and Next Steps

• Enhance EventShop Architecture• Collaboration Research (with NICT): – Sticker 3D visualization tool, – EventWarehouse

• Multi Granularity E-mage• Predictive Analytics• SLN Use Case

05/03/2023 13

Sticker & EventWarehouseNICT

05/03/2023 14

EvShop and EvWarehouse Interface

1. Retrieve EventModel stream – Option1: EvShop periodically sends request to EvWH

to access new events stored in EventModel table (MPQL)

– Option 2: EvWH pushes new events to EvShop (listener)

2. Access EventModel stream’s metadata3. Create new EventModel Stream

05/03/2023 15

Example of MPQLSELECT MIN(observation),MAX(observation),SUM(observation), AVG(observation)FROM LiveERestflCO2SensorGROUP BYTIME('2013-10-01T00:00:00','2013-10-02T00:00:00', 12 HOUR ),SPACE( 130.0,30.0,140.0,40.0, 5,5 )

SELECT observation FROM STREAM LiveERestflCO2Sensor

05/03/2023 16

Current State and Next Steps

• Enhance EventShop Architecture• Collaboration Research (with NICT): – Sticker 3D visualization tool, – EventWarehouse

• Multi Granularity E-mage• Predictive Analytics• SLN Use Case

05/03/2023 17

Multi Granularity E-mage

• data is created and collected in different forms• different sensors cover different sized spaces,

produce data at different rates• data is produced and consumed at different

spatial, temporal, and symbolic granularities

05/03/2023 18

Pyramid of E-mage ResolutionLevel Stel Size

1 78 km2 39 km3 19.6 km4 9.8 km5 4.9 km6 2.4 km7 1.2 km8 611 m9 306 m

10 153 m11 76 m12 39 m13 19 m14 10 m15 5 m16 2.4 m17 1.2 m18 60 cm19 30 cm20 15 cm

Inspired by the most popular service like Google Maps, Bing Maps, and OGC WMTS

They provide the standard of the granularity level of the world map

05/03/2023 19

Multi Granularity E-mage

Timet1 t2 t3 t4

Spac

e

DS1: update every 10 minsDS2: update every 5 minsDS3: update every 30 mins

The situation model is processed every 10 mins

E-mage spatial transformation are categorized into two main types 1) Coarse2Fine: nearest-neighbor interpolation, linear interpolation,

bilinear interpolation, and split uniform. 2) Fine2Coarse: summation, maximum value, minimum value, average,

majority.

05/03/2023 20

Multi Granularity E-mage

• How to dynamically adjust appropriate granularity?– Guarantee the quality of the results– Data error propagation• Uncertainty of data stream, data loss during data

conversion, etc.– Source selection

05/03/2023 21

Rasterization Errors Prediction

• The regression model depicts the relationships between rasterization errors and their affecting factors– Equal area conversion (EAC) algorithm is used for rasterization of vector

polygons– Rasterization errors calculated from Error Evaluation Method Based on

Grid Cells (EEM-BGC)

– The factors includes both the complexity of polygons perimeter index (e.g., density of arcs length (DA) and density of polygon (DP)) and the size of gird cells (SG).

Relative area error = Area before conversion – Area after conversion

Area Before conversion

)ln(456.931.0418.0499.58 SGDPDAE

For vector data of county level boundary of Beijing

[Liao 2012] Error Prediction for Vector to Raster Conversion Based on Map Load and Cell Size

05/03/2023 22

Current State and Next Steps

• Enhance EventShop Architecture• Collaboration Research (with NICT): – Sticker 3D visualization tool, – EventWarehouse

• Multi Granularity Emage• Predictive Analytics• SLN Use Case

05/03/2023 23

Predictive AnalyticsSituation An actionable abstraction of observed or extrapolated spatio-temporal characteristics

- Ish Rishabh

05/03/2023 24

Current State and Next Steps

• Enhance EventShop Architecture• Collaboration Research (with NICT): – Sticker 3D visualization tool, – EventWarehouse

• Multi Granularity Emage• Predictive Analytics• SLN Use Case

05/03/2023 25

Calendar PESi

FMB (Individual’s Feeling)Accelerometer

Location

Fitness Data(Nike, Fitbit) Data

Ingestion & Aggregation

Heart RateLocation (Move)

Food Log

FMB (People’s Feeling, Location)

ESOzoneCO2SO2PM 2.5

Pollen (Tree, Grass)

Air Quality Index

Data Ingestion & Aggregation

Social Media (News, Tweets)

Weather

Macro Situation Recognition

Predictive Analytics

PersonalSituation Recognition

Persona

Asthma Allergy App Server

Data Collection

Mac

ro S

ituati

onPe

rson

al S

ituati

on

Need and Resources Recommendation

SLN Use Case

05/03/2023 26

Please Stay Tuned! Open Source (Next week)