SWARM INTELLIGENCE IN

DATA MINING

Written by Crina Grosan, Ajith Abraham & Monica Chis

Presented by Megan Rose Bryant

INTRODUCTION

We will cover the following

Biological motivation of some theoretical concepts of swarm intelligenceParticle Swarm OptimizationAnt Colony Optimization

Basic Data Mining Terminologies Implementation with Swarm Intellegince Insect Swarm

BIOLOGICAL COLLECTIVE

BEHAVIOR

BIOLOGICAL COLLECTIVE BEHAVIOR Swarm behavior can be seen in bird flocks, fish schools, and insects.

Group behaviors of some organisms are so integrated that they appear to move as a coherent entity.

These behaviors are the influence behind swarm optimization.

Swarm (“School”) of Fish

MAIN PRINCIPLES OF COLLECTIVE BEHAVIOR Homogeneity: every organism in swarm has the same behavior model. No leader.

Locality: motion is influenced only by nearest members.

Collision Avoidance: avoids collision with nearby members.

Velocity Matching: attempt to match velocity of nearby members.

Flock Centering: attempt to stay close to nearby members.

TYPES OF COLLECTIVE DYNAMICAL BEHAVIOR Swarm: an aggregate with cohesion, but low level of parallel alignment among members.

Torus: individuals perpetually rotate around an empty core. Direction of rotation is random.

Dynamic Parallel Group: individuals are polarized and move as a coherent group, but group form and density fluctuate.

Highly Parallel Group: much more static in terms of exchange of spatial positions. Form and density variety is minimal.

SWARMS AND ARTIFICIAL LIFE

SWARMS AND ARTIFICIAL LIFE Collective behavior algorithms have been applied to a variety of well-known algorithms including:

Traveling Salesman Problem

Quadratic Assignment Problem

Graph Problems

Clustering

Data mining

etc.

TSP Point Set of Argentina

PARTICLE SWARM OPTIMIZATION (PSO)

PSO is a population based search algorithm.

Initialized with a population of random solutions (called ‘particles’).

Each particle has an associated velocity.

Particles fly through space with dynamically adjusted velocities according to historical behaviors.

Particles fly towards better and better search area over time.

Swarm of Starlings

BIOLOGICAL INTUITION FOR PSO Imagine the following: you are a bird in a flock of birds that is searching for a single French fry in a McDonald’s parking lot (the search area).

You don’t know where the fry is, but you do know how far the food is and the position of all flock members.

What is the best strategy to find the French fry?

An effective strategy is to follow the bird closest to the food.

Flock of Birds

PSO ALGORITHM

PSO learns from this scenario and uses it to solve optimization problems.

Each bird is a particle (a single solution).

Each bird has a fitness value and a velocity.

Birds fly through the search space by following the bird nearest the food thus far.

Political Cartoon

ANT COLONIES OPTIMIZATION Now imagine that you are an ant among a colony of ants. When searching for food, you begin by searching the area closest to the nest in a random fashion.

As you go, you leave behind a pheromone trail to tell your ant friends what you have found.

When you find food, you use these pheromones to let everyone know how much there is and its quality.

Pheromone Trail

DATA MINING

DATA MINING

Data mining is the application of specific algorithms for extracting patterns from data.

Historically, this application had been given many names including knowledge extraction, information discovery, and data pattern processing.

Swarm Optimization can be very helpful in this process of Knowledge Discovery.

STEPS OF KNOWLEDGE DISCOVERY1. Developing understanding of

the domain, prior knowledge, and the goal.

2. Creating a target data set.

3. Data cleaning and preprocessing.

4. Data reduction and projection.

5. Matching the goals with a particular data mining method.

Success requires continual growth

SWARM INTELLIGENCE AND

KNOWLEDGE DISCOVERY

SWARM INTELLIGENCE & KNOWLEDGE DISCOVERY Data mining and swarm optimization can be used together to form a method that often leads to very good results.

PSO methods have been used successful in pattern recognition, image processing, and unsupervised classification & image segmentation.

PSO IN DATA MINING

Particle Swarm Optimization has been used in several data mining algorithms including the following:

Visual Data Mining

Recommender Systems

Classification Tasks

etc.

PSO can often be employed when other implementations would be too large or too costly.

Recommender System

ANT COLONY OPTIMIZATION AND

DATA MINING

ANT COLONY OPTIMIZATION IN DATA MINING Ant Colony Optimization has been used with great success in clustering.

Modeled after real ant behaviors, the computer ants ‘pick up’ data and move it to other areas with similar data.

Several species have been studied to model different behaviors.

Cluster Analysis

CONCLUSIONS

CONCLUSIONS

Biological behaviors can inform efficient optimization techniques such as Particle Swarm Optimization and Ant Colony Optimization.

These optimization techniques have a variety of applications in Data Mining.

They can often be employed when other techniques prove too costly. Zerg Rush, Starcraft II