Outline
• Introduction
• Anomaly detection in data
• Negative Selection Algorithm (NSA)
• Application examples
• Conclusions
What are Anomalies?
• Anomaly is a pattern in the data that does not conform to the expected behavior
• Anomaly is also referred to as outliers, exceptions, peculiarities, surprise, etc.
• Anomalies translate to significant (often critical) real life entities– Credit card fraud– Cyber intrusions
Simple Example
• N1 and N2 are regions of normal behavior
• Points o1 and o2 are anomalies
• Points in region O3 are anomalies
X
Y
N1
N2
o1
o2
O3
Simple Example
Simple Example
Real World Anomalies
• Credit Card Fraud– An abnormally high purchase made on a
credit card
• Cyber Intrusions– A web server involved in ftp traffic
Applications of Anomaly Detection
• Network intrusion detection• Insurance / Credit card fraud detection• Healthcare informatics / Medical diagnostics• Industrial damage detection• Image processing / Video surveillance • Novel topic detection in text mining• …
Key Challenges
• Defining a representative normal region is challenging
• The boundary between normal and outlying behavior is often not precise
• The exact notion of an outlier is different for different application domains
• Data might contain noise• Normal behavior keeps evolving
Artificial Immune Systems (AIS)
• Artificial Immune Systems (AIS) are an emerging kind of soft computing methods– Inspired by natural immune systems– Features of pattern recognition, anomaly detection, data
analysis, machine learning, etc• Negative Selection Algorithm (NSA) is an important
partner of AIS– Maturation of T cells and self/nonself discrimination– Developed by Forrest in 1994
• NSA is applied to deal with anomaly detection in data
Biological Information Processing Systems
SystemGenetic
SystemEndocrine
SystemBrain
SystemImmune
Natural Immune System
Natural Immune System
X X
Pathogens
Biochemical barriers
Skin
Innate immune response
Adaptive immune response
Lymphocytes
How Does Natural Immune System Work?
Negative Selection Algorithm (NSA)
• Immune system (B and T cells) is capable of distinguishing self from nonself– Negative censoring of T cells in thymus
• Negative Selection Algorithm (NSA) mimics mechanism of immune system– 1. Define self samples (representative samples)– 2. Generation of detectors (binary and real-valued)– 3. Negative selection of detectors– 4. Employment of detectors in anomaly detection
Negative Selection Algorithm (NSA)
Generation of NSA Detectors
Negative Selection Algorithm (NSA)
Anomaly Detection Using NSA
Self and Nonself Samples
Self Samples
Nonself Samples
Self and Nonself Coverage in NSA
Self and Nonself Samples
0 0.2 0.4 0.6 0.8 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Two Examples of Random Detector Groups
0 0.2 0.4 0.6 0.8 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 0.2 0.4 0.6 0.8 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Two Examples of Optimized Detector Groups (Gao, 2004)
0 0.2 0.4 0.6 0.8 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 0.2 0.4 0.6 0.8 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Distribution of Fisher’s iris Data in Sepal Length-Sepal Width Dimensions
4 4.5 5 5.5 6 6.5 7 7.5 82
2.5
3
3.5
4
4.5
Sepal Length
Sep
al W
idth
setosa
versicolorvirginica
Distribution of Fisher’s iris Data in Petal Length-Petal Width Dimensions
0 1 2 3 4 5 6 70
0.5
1
1.5
2
2.5
Petal Length
Pet
al W
idth
setosa
versicolorvirginica
Detector Generation in NSA (Gao, 2006)
Anomaly Detection in Fisher’s iris Data with NSA Detectors
Anomaly Detection in Chaotic Time Series
• Mackey-Glass chaotic time series
controls the behaviors of Mackey-Glass time series– and
• Anomaly detection rate can be improved by neural networks-based NSA
)()(1
)()( tbx
tx
taxtx
c
1730
Mackey-Glass Time Series
30
17
Fresh Data
Anomaly Detection in Mackey-Glass Time Series Using Adaptive NSA
Before Training
After Training
57M 9L%86
31M 1L%97
NSA-based Motor Fault Detection
• Monitoring of the working conditions of the running motors is necessary in maintaining their normal status
• Anomaly in the feature signals acquired from the faulty motors is caused by faults
• Motor fault detection is converted to a typical problem of anomaly detection
Normal, Abnormal, and Faulty Feature Signals of Motors
Normal Feature Signal
Abnormal Feature Signal
Faulty Feature Signal
NSA in Motor Fault Detection
Feature Signals from Healthy Motors
Feature Signals from Operating Motors
Signal Preprocessing
Signal Preprocessing
Detector Generation
Detectors Anomaly Detection
Fault Detection
Fault Detection PhaseDetector Generation Phase
NSA-based Motor Fault Detection
•Anomaly in feature signals is caused by faults–Healthy feature signals: self–Faulty feature signals: nonself
•Neural networks are combined with NSA• NSA detectors are built up on the structure of BP
neural networks• Neural networks training algorithm is applied
Neural Networks-based NSA (Gao, 2010)
1x 2x Nx
Nw
1v 2v Nv
y
f f f
2w1w
E
3Layer
2Layer
1Layer
2 iii wxd
N
iiii
N
iii wxvdvy
1
2
1
)(f)f(
yE
Training of Neural Networks-based NSA
),( Weights vw
)E
(E
rror
Mat
chin
gv
w,
)E( vw,
),( ** vw
Margin Training Strategy of NSA Detectors
• Case1: (for faulty plant feature signals only): if , detectors are trained using normal BP learning algorithm to decrease E; otherwise, no training is employed
• Case 2 (for healthy plant feature signals only): if , detectors are trained using ‘positive’ learning algorithm to increase E; otherwise, no training is employed
E0
0 E
Margin Training of NSA Detectors in Fault Detection
y
Detectors
Out
puts
Det
ecto
r IIRegion Training
IRegion Training
Inner Raceway Fault Detection of Bearings
• Bearings are important components in rotating machinery
• Defect on the inner raceway is a common but typical fault of bearings
• Fault detection is based on vibration signals of bearings– A sensor mounted on eight-ball bearings with a motor rotation
speed at 1,782 rpm
Inner Raceway Fault of Bearings
Features Signals of Healthy and Faulty Bearings
Healthy Bearings
Faulty Bearings
Fault Detection Rate of Neural Networks-based NSA
27M
Before Training
7L
%79
15M
After Training
0L
%100
Motor Fault Detection using NSA (Gao, 2012)• Two kinds of motor faults are considered here
– Rotor fault– Stator fault
• Stator current signals are used as feature signals
• Both healthy and faulty motors are running with/without varying loads
• Fault detection rate is %100
BA
B
Feature Signals for Motor Fault Detection
0.5 0.55 0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95 1-150
-100
-50
0
50
100
150
Time in Seconds
Sta
tor
Cur
rent
Healthy Motor
Feature Signals for Motor Fault Detection
0.5 0.55 0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95 1-200
-150
-100
-50
0
50
100
150
Time in Seconds
Sta
tor
Cur
rent
Broken Rotor
Feature Signals for Motor Fault Detection
0.5 0.55 0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95 1-150
-100
-50
0
50
100
150
Time in Seconds
Sta
tor
Cur
rent
Broken Stator
Motor Fault Detection Results
0 50 100 150 200 250 300 350 400 450 5000
1
2
3
4
5
6
7
8
Number of Signal Windows
Num
ber
of A
ctiv
ated
Det
ecto
rs
Healthy Motor
Motor Fault Detection Results
0 50 100 150 200 250 300 350 400 450 5000
1
2
3
4
5
6
7
8
Number of Signal Windows
Num
ber
of A
ctiv
ated
Det
ecto
rs
Broken Rotor
Motor Fault Detection Result
0 50 100 150 200 250 300 350 400 450 5000
1
2
3
4
5
6
7
8
Number of Signal Windows
Num
ber
of A
ctiv
ated
Det
ecto
rs
Broken Stator
Detection Results of Rotor and Stator Faults
Current Signals of Healthy Motor withFour Different Loads
0 5000 10000
-100
0
100
Time in Seconds
Rot
or C
urre
nt
(a)
0 5000 10000-400
-200
0
200
400
Time in Seconds
Rot
or C
urre
nt
(b)
0 5000 10000-100
-50
0
50
100
Time in Seconds
Rot
or C
urre
nt
(c)
0 5000 10000-100
-50
0
50
100
Time in Seconds
Rot
or C
urre
nt
(d)
Current Signals of Faulty Motor withFour Different Loads
0 5000 10000-200
-100
0
100
200
Time in Seconds
Rot
or C
urre
nt
(a)
0 5000 10000
-500
0
500
Time in Seconds
Rot
or C
urre
nt
(b)
0 5000 10000
-100
0
100
Time in Seconds
Rot
or C
urre
nt
(c)
0 5000 10000
-200
0
200
Time in Seconds
Rot
or C
urre
nt
(d)
Numbers of Activated NSA Detectorsfor Healthy Motor
0 100 200 300 400 5000
5
10
Number of Signal Windows
Num
ber
of A
ctiv
ated
Det
ecto
rs (a)
0 100 200 300 400 5000
5
10
Number of Signal Windows
Num
ber
of A
ctiv
ated
Det
ecto
rs (b)
0 100 200 300 400 5000
5
10
15
20
Number of Signal Windows
Num
ber
of A
ctiv
ated
Det
ecto
rs (c)
0 100 200 300 400 5000
10
20
30
40
Number of Signal Windows
Num
ber
of A
ctiv
ated
Det
ecto
rs (c)
Healthy Motor
Numbers of Activated NSA Detectorsfor Faulty Motor
0 100 200 300 400 5000
5
10
Number of Signal Windows
Num
ber
of A
ctiv
ated
Det
ecto
rs (a)
0 100 200 300 400 5000
5
10
Number of Signal Windows
Num
ber
of A
ctiv
ated
Det
ecto
rs (b)
0 100 200 300 400 5000
5
10
15
20
Number of Signal Windows
Num
ber
of A
ctiv
ated
Det
ecto
rs (c)
0 100 200 300 400 5000
10
20
30
40
Number of Signal Windows
Num
ber
of A
ctiv
ated
Det
ecto
rs (c)
Faulty Motor
Fault Detection Rates of Faulty Motorswith Different Loads (Gao, 2013)
Conclusions
• Anomaly detection in data is an important topic• NSA can be used for anomaly detection based
on only the normal data• A few application examples have demonstrated
the effectiveness of the NSA in anomaly detection
• Performance comparisons need to be made between the NSA and other anomaly detection methods, e.g., Support Vector Machine (SVM)
Recommended
