Upload
manuel-martin
View
101
Download
0
Embed Size (px)
Citation preview
Effects of change propagation resulting from adaptive preprocessing in multicomponent predictive systems
Manuel Martín Salvador, Marcin Budka, Bogdan Gabrys{msalvador,mbudka,bgabrys}@bournemouth.ac.uk
Data Science Institute. Bournemouth University
KES-2016, York, UKSeptember 7th, 2016
Outline1. Prologue2. Introduction to MCPS3. Motivation4. Reactive adaptation of MCPS5. Experiments6. Conclusion
Butterfly effectSmall causes can have large effects
— Edward Lorenz (1917 - 2008)
Source: GloWings
Change propagationControlled change management in a system
CC by TheGiantVermin
Data streams
“Infinite” number of records
Continuously arriving to the system at different or same rates
Can be stationary or evolving
Data streams
Examples:
● Sensors in manufacturing industry● Traffic monitoring sensors● Event logs in websites● Transactions in the financial sector
“Infinite” number of records
Continuously arriving to the system at different or same rates
Can be stationary or evolving
A single engine of Airbus A320 has more than 1000 sensors
generating 10GB/s!!
Data Stream
Data stream learning for online prediction
PredictiveModel
Online Supervised Learning Algorithm
Predictions
True labels
t+k
t
Data Stream
Data stream learning for online prediction
PredictiveModel
PredictionsPreprocessing Postprocessing
Multicomponent Predictive System (MCPS)
MCPS composition
Manual● WEKA● RapidMiner● Knime● IBM SPSS
Automatic● Auto-WEKA (Bayesian optimisation)● Auto-sklearn (Bayesian optimisation + Meta-learning)● TPOT (Genetic programming)● e-Lico IDA (Ontologies + Planning)
Example of WEKA workflow
Formalising MCPS
otoken(data) i
place
transition
Well-handled and Acyclic Workflow Petri net (WA-WF-net)MCPS = (P, T, F)
Formalising MCPS
o predictioni
place
transition
Well-handled and Acyclic Workflow Petri net (WA-WF-net)MCPS = (P, T, F)
“Automatic composition and optimisation of multicomponent predictive systems” @ IEEE TNNLS (under review) http://bit.ly/automatic-mcps-tnnls
Formalising MCPS
Classifier
o
Replace missing values
Dimensionality reduction
Outlier handling
token(data) i
place
transition
Well-handled and Acyclic Workflow Petri net (WA-WF-net)MCPS = (P, T, F)
“Automatic composition and optimisation of multicomponent predictive systems” @ IEEE TNNLS (under review) http://bit.ly/automatic-mcps-tnnls
Need of model adaptation
Streaming error (mean over last 10 samples)SYN dataset with GFMM classifier
GFMMZ-Score PCA Min-Max
Wrongly classified
Need of preprocessing adaptation
Streaming error (mean over last 10 samples)SYN dataset with GFMM classifier
GFMMZ-Score PCA Min-Max
Wrongly classified (out of [0,1])
New hyperboxes
Main strategies for MCPS adaptation
Adaptation strategies GLOBAL LOCAL
Re-composition Full Partial
Hyperparameter optimisation (keep components) Full Partial
Parameterisation (keep components and hyperparameters) Full Partial
Main strategies for MCPS adaptation
Adaptation strategies GLOBAL LOCAL
Re-composition Full Partial
Hyperparameter optimisation (keep components) Full Partial
Parameterisation (keep components and hyperparameters) Full Partial
“Adapting Multicomponent Predictive Systems using Hybrid Adaptation Strategies with Auto-WEKA in Process Industry” @ AutoML / ICML 2016 http://bit.ly/adapting-mcps-paper
This work!
Need of change propagation
Streaming error (mean over last 10 samples)SYN dataset with GFMM classifier
GFMMZ-Score PCA Min-Max
Inconsistent hyperboxesdue to a different input space
Reactive adaptation of MCPSGFMMZ-Score PCA Min-Max
Time
i p1 p2 p3 o
data
meta-data
[-3.1, 2.7]
x1 = 3.6
[-3.1, 3.6]
Reactive adaptation of MCPSGFMMZ-Score PCA Min-Max
Time
i p1 p2 p3 o
data
meta-dataprediction
[-3.1, 2.7]
x1 = 3.6
[-3.1, 3.6]
Updating a component: GFMM
0 1
1
0
(-3.1) (2.7)x1
x2
0 1
1
0
(-3.1) (3.6)x1
x2
Hyperboxes are mapped to the new
input space
Experiments
Name # Attr # Class Type
SYN 2 2 Synthetic
ELEC 7 2 Real
COVERTYPE 54 7 Real
GAS 128 6 Real
Datasets Scenarios
Id Adap.Model
Adap.Prepro.
ChangePropagation
#1 No No No
#2 Yes No No
#3 Yes Yes No
#4 Yes Yes YesFirst 200 samples for initial training, rest 400 for testing and online learning
GFMMZ-Score PCA Min-Max
Conclusion
Only model adaptation may not be enough to cope with evolving data streams, adaptive preprocessing should be considered.
However, “blind” adaptation of components can result in inconsistent models or even in a system crash.
Local adaptation of a component may require adapting further components. Therefore, a system must be reactive and propagate changes.
The definition of MCPS has been extended to support change propagation using a new token for meta-data in a coloured Petri net (cMCPS).
Future work
Large study to measure the actual cost of adaptation.
Open questions:
● How to handle propagation requiring changes of the Petri net structure?● How to handle transformations in systems with nonlinear components?● How to order components to reduce the cost of adaptation?● Can a meta-data token be removed at an early stage instead of being fully
propagated?
Thanks!
Paper: http://bit.ly/change-propagation-mcps
Slides: http://www.slideshare.net/draxus
Manuel <[email protected]>
@draxus