Salil presentation 11.07

Anomaly Detection- SALIL NAVGIRE

Introduction• problem of finding patterns in data that

do not conform to expected behavior• covers diverse disciplines from statistics,

machine learning, data mining, information theory, spectral theory

Applications• Intrusion detection- detection of malicious activity• Host based – OS call traces• Network based – packet level traces

• Fraud detection - detection of criminal activities in commercial organizations• Credit card fraud detection• Insurance Claim Fraud Detection• Insider trading detection

• Industrial damage detection• Anomaly detection in data• Anomaly detection in sensor networks

Challenges• Defining normal region• Sometimes malicious agent adapt

themselves to appear as normal observation• Different techniques for different

application domain• Availability of labeled data for training• Sometimes noise is similar to anomaly

and difficult to distinguish

Different aspects of detection techniques• Nature of input data• Types of Anomaly• Point Anomalies• Contextual Anomalies• Collective Anomalies

• Data Labels• Supervised anomaly detection• Semi-Supervised anomaly detection• Unsupervised anomaly detection

• Output • Scores• Labels

Anomaly detection

techniques

Classification

Nearest Neighbor Clustering Spectral Informatio

n theoretic Statistical Time Series

Anomaly Detection Techniques

• Classification• Neural network based• Bayesian Network based• Support Vector Machine based• Rule based

• Nearest Neighbor• KNN• Relative density

• Clustering• K means• SOM

• Statistical• Parametric• Gaussian model based• Regression model based• Mixture of parametric distributions based

• Non-parametric• Histogram based• Kernel function based

• Spectral• Dimensionality reduction