Francisco Serdio, Edwin Lughofer, Kurt Pichler, Thomas Buchegger, Markus Pichler, Hajrudin Efendic

francisco.serdio@jku.at http://www.flll.jku.at/staff/francisco

Multivariate Fault Detection using Vector Autoregressive Moving Average

and Orthogonal Transformation in Residual Space

Francisco Serdio Fernández

Department of Knowledge-Based Mathematical Systems

Johannes Kepler University

Linz - Austria

{francisco.serdio,edwin.lughofer}@jku.at http://www.flll.jku.at/staff/{francisco,lughofer}Francisco Serdio, Dr. Edwin Lughofer

Index

• IFAC Technical Committee SAFEPROCESS» Fault detection» Fault

• Residual Based Approach» Recall Main Idea» Graphical Explanation

{francisco.serdio,edwin.lughofer}@jku.at http://www.flll.jku.at/staff/{francisco,lughofer}Francisco Serdio, Dr. Edwin Lughofer

Index

• Orthogonal transformations» Principal Components Analysis (PCA)

» Mathematical Formulation» Meaning» How to use

» Partial Least Squares (PLS)» Mathematical Formulation» Meaning» How to use

{francisco.serdio,edwin.lughofer}@jku.at http://www.flll.jku.at/staff/{francisco,lughofer}Francisco Serdio, Dr. Edwin Lughofer

Index

• Vector Autoregressive Moving Average (VARMA)» Motivation ARMA» Differences with ARMA» How to use

• Soft Computing: Sparse Fuzzy Inference Systems (SparseFIS)

• Overall picture: PCA/PLS + SparseFIS + VARMA• Dynamic Residual Analysis• Results as ROC curves• Conclusions & Outlook

{francisco.serdio,edwin.lughofer}@jku.at http://www.flll.jku.at/staff/{francisco,lughofer}Francisco Serdio, Dr. Edwin Lughofer

IFAC Technical Committee SAFEPROCESS

• Fault detection» Determination of faults present in a system and

the time of detection• Fault

» Unpermitted deviation of at least one characteristic property or variable of the system from acceptable / usual / standard behaviour

{francisco.serdio,edwin.lughofer}@jku.at http://www.flll.jku.at/staff/{francisco,lughofer}Francisco Serdio, Dr. Edwin Lughofer

Main Idea of Residual-Based Approach

Fault No Fault!, but non-smooth pattern of signal

Joint Channel Space (smooth dependency)

Increasing the dimensionality of the joint channel space decreases the likelihood that a fault is affected in all channels with same intensity and direction!

{francisco.serdio,edwin.lughofer}@jku.at http://www.flll.jku.at/staff/{francisco,lughofer}Francisco Serdio, Dr. Edwin Lughofer

Orthogonal Transformations

• Principal Components Analysis (PCA)» Vector space transformation » Identifies the most meaningful basis to re-express the

original space

» Preserves maximum variance in minimum number of dimensions filter out the noise / irrelevant information

{francisco.serdio,edwin.lughofer}@jku.at http://www.flll.jku.at/staff/{francisco,lughofer}Francisco Serdio, Dr. Edwin Lughofer

Orthogonal Transformations

• Partial Least Squares (PLS)» Also known as Projection to Latent Structures» As PCA, also a vector space transformation » Reduces the dimensionality of the input and target

variables by projecting them to the directions maximizing the covariance between target and input variables filter out the noise / irrelevant information

{francisco.serdio,edwin.lughofer}@jku.at http://www.flll.jku.at/staff/{francisco,lughofer}Francisco Serdio, Dr. Edwin Lughofer

Orthogonal Transformations

• What is the deal ?» Apply the orthogonal transformation» Train a model on top of the new (transformed)

space

{francisco.serdio,edwin.lughofer}@jku.at http://www.flll.jku.at/staff/{francisco,lughofer}Francisco Serdio, Dr. Edwin Lughofer

Vector Autoregressive Moving Average (VARMA)

• Motivation: Arma» AutoRegressive Moving Average model» Predicts a channel using its own history

» Autoregressive

» Own history means some (chosen) past values» Lag operator, also known as Backshift operator

• Differences with Arma» The lags belong to other channels» Predicts a channel using other channels and other

channels’ history» Vector Autoregressive

{francisco.serdio,edwin.lughofer}@jku.at http://www.flll.jku.at/staff/{francisco,lughofer}Francisco Serdio, Dr. Edwin Lughofer

Vector Autoregressive Moving Average (VARMA)

• What is the deal ?» Span the dataset introducing lags» Model over the spanned dataset including lags

{francisco.serdio,edwin.lughofer}@jku.at http://www.flll.jku.at/staff/{francisco,lughofer}Francisco Serdio, Dr. Edwin Lughofer

Soft Computing: SparseFIS

• Sparse Fuzzy Inference System (SparseFIS)» Top down fuzzy modeling approach applying numerical

sparsity constraints optimization, out-weighting unimportant rules and parameters

» Employs iterative VQ, projected gradient descent and Semi-Smooth Newton

{francisco.serdio,edwin.lughofer}@jku.at http://www.flll.jku.at/staff/{francisco,lughofer}Francisco Serdio, Dr. Edwin Lughofer

Orthogonal Transformations, Soft Computing and Vector Autoregressive Moving Average

{francisco.serdio,edwin.lughofer}@jku.at http://www.flll.jku.at/staff/{francisco,lughofer}Francisco Serdio, Dr. Edwin Lughofer

Orthogonal Transformations, Soft Computing and Vector Autoregressive Moving Average

{francisco.serdio,edwin.lughofer}@jku.at http://www.flll.jku.at/staff/{francisco,lughofer}Francisco Serdio, Dr. Edwin Lughofer

Dynamic Residual Analysis (On-line)

Normalized Residual for ith model: Confidence Band

Incremental/Decremental Tolerance Band

{francisco.serdio,edwin.lughofer}@jku.at http://www.flll.jku.at/staff/{francisco,lughofer}Francisco Serdio, Dr. Edwin Lughofer

ROC curves: LR & SPF, with PCA & PLS

{francisco.serdio,edwin.lughofer}@jku.at http://www.flll.jku.at/staff/{francisco,lughofer}Francisco Serdio, Dr. Edwin Lughofer

ROC curves: LR vs PCR vs PCR+Lags

{francisco.serdio,edwin.lughofer}@jku.at http://www.flll.jku.at/staff/{francisco,lughofer}Francisco Serdio, Dr. Edwin Lughofer

Conclusions

• PCA before training» The expansion of the datasets with the lags produces

no clear improvement in fault detection capabilities, and the VARMA models can be ignored in this case

• PLS before training» When the datasets are transformed using PLS, VARMA

models help to improve the ROC curves, and therefore the fault detection capabilities

{francisco.serdio,edwin.lughofer}@jku.at http://www.flll.jku.at/staff/{francisco,lughofer}Francisco Serdio, Dr. Edwin Lughofer

Outlook

• Deeper analysis of results» Enforce the results with statistical tests

• Work in the Fault Identification and Fault Isolation domain» Create a confidence measure accompanying

the ROC curve » Use the deformation of the model when a fault

appears» Analyze the gradients of the inputs

» Compare gradients with detections

» Goal: Determine how true a detection is

{francisco.serdio,edwin.lughofer}@jku.at http://www.flll.jku.at/staff/{francisco,lughofer}Francisco Serdio, Dr. Edwin Lughofer

Thanks a lot for your attention!