1

Efficient Pattern Matching Algorithm for Memory Architecture

Author:Cheng-Hung Lin and Shih-Chieh ChangPublisher:IEEE TRANSACTIONS ON VERY LARGE SCALE INTEGRATION (VLSI) SYSTEMS 2009Presenter:Han-Chen ChenDate:2010/03/03

2

Introduction In order to accommodate the increasing number of attack

patterns and meet the throughput requirement of networks, a successful network intrusion detection system must have a memory-efficient pattern-matching algorithm and hardware design.

By merging traditional AC algorithm states and constructing a new traversal mechanism on merged FSM, we can achieves 21% of memory reduction.

3

Review of AC Algorithm

Pattern1 : bcdf

Pattern2 : pcdg

input Next state

failure Match vector

State0 b 1 0 00

State0 p 5 0 00

State1 c 2 0 00

State2 d 3 0 00

State3 f 4 0 01

State5 c 6 0 00

State6 d 7 0 00

State7 g 8 0 10

In a traditional AC state machine, a final state stores the corresponding match vector which is one-hot encoded.

4

Merge States Problem

Input string : pcdf causes false positive results

Pattern1 : bcdf

Pattern2 : pcdg

Merge similar transition

If we can memorize the precedent state entering the merged states, we can differentiate all merged states.

00 00 00 01

000000

00

10

5

Data structure

Pattern1 : bcdfPattern2 : pcdg

Match vector ->pathVec_ifFinal

PathVec : record the path

ifFinal : whether the state is final

Only add an additional bit to each state.

6

Merge_FSM Definition: Two states are defined as pseudo-equivalent states if they have identical input transitions, identical failure transitions, and identical ifFinal bit, but different next states.

State 2 and state 6 are similar.State 3 and state 7 are similar.

11_001_0 01_0 01_0 01_1

10_0 10_0 10_010_1

7

Construction of State Traversal Machine (1/2)

1. The construction of valid transition, failure transition, pathVec, and ifFinal functions.

2. Merging pseudo-equivalent states.

Pattern 1: abcdef Pattern 2: apcdeg Pattern 3: awcdeh

Add pattern : apcdeg

8

Construction of State Traversal Machine (2/2)

Add pattern : awcdeh

Merge similar states

9

State Traversal Mechanism on a Merge_FSM (1/3)

In addition, we need a register, called preReg, to trace the precedent pathVec in each state.

The width of preReg is equal to the width of pathVec. Each bit of the preReg also corresponds to a string pattern.

10

State Traversal Mechanism on a Merge_FSM (2/3)

Input string : p c d f

Pattern1 : bcdf

Pattern2 : pcdg

11

State Traversal Mechanism on a Merge_FSM (3/3)

Input string : p c d g

Pattern1 : bcdf

Pattern2 : pcdg

12

Loop Back in Merged States(1/2)

Pattern : abcdef and wdebcg

Input string “abcdebcdef” will be mistaken as a match of the pattern “abcdef”

13

Loop Back in Merged States(2/2)

Patterns:abcdefghijklm and abcwsghidefxyklm

1. Using the longest common substring algorithm, we can extract all of the common substrings of these two patterns such as “abc”, “def”, “ghi” and “klm”.

2. Then, we label the substrings “abc”, “def”, “ghi”, and “klm” as α, β, γ, δ.“abcdefghijklm” is labeled as “αβγδ ” while the sequence of substrings in “abcwsghidefxyklm” is labeled as “αγβδ ”.

3. LCS result : αβδ or αγδ.we can merge the subsequences of α (“abc”), β (“def”) andδ (“klm”) or the subsequences of α (“abc”), γ(“ghi”), and δ (“klm”)

4. Output result

14

Hardware Architecture

32 patterns & 1024 transitions (states)

All preReg bits are 0 or n_valid is 1 ns_sel =0

Not all preReg bits are 0 and n_valid is 0 ns_sel =1

ns_ctrl :

15

Performance (1/3)

16

Performance (2/3)

17

Performance (3/3)

18

Thanks for your

listening