String searching

String Searching AlgorithmsString Searching AlgorithmsProblem DescriptionProblem Description

Given two strings P and T over the same alphabet ,determine whether P occurs as a substring in T (orfind in which position(s) P occurs as a substring in T).

The strings P and T are called pattern and target respectively.

[Adapted from G.Plaxton]

String Searching AlgorithmsString Searching AlgorithmsSome applicationsSome applications

[Adapted from K.Wayne]

String Searching AlgorithmsString Searching AlgorithmsTrivial Approach - AlgorithmTrivial Approach - Algorithm

SimpleMatcher(string P, string T)

n length[T]

m length[P]

for s 0 to n m do

if P[1...m] = T[s+1 ... s+m] then

print s

T(n,m) = (n m + 1) m (1) = (n m)

String Searching AlgorithmsString Searching AlgorithmsRabin-Karp Algorithm - IdeaRabin-Karp Algorithm - Idea

Pattern - P[1m]Target - T[1n]

p = P[m] + 10 P[m–1] + 100 P[m–2] + + 10m P[1]

for s = 0 to n – m:

ts = T[s+m] + 10 T[s+m–1] + 100 T[s+m–2] + + 10m T[s+1]

P matches T at position i if and only if p = ti

String Searching AlgorithmsString Searching AlgorithmsRabin-Karp Algorithm - IdeaRabin-Karp Algorithm - Idea

p = P[m] + 10 P[m–1] + 100 P[m–2] + + 10m P[1]

p = P[m] + 10 (P[m–1] + 10 (P[m–2] + + 10 (P[2] + 10 P[1]))))

t0 = T[m] + 10 T[m–1] + 100 T[m–2] + + 10m T[1]

t0 = T[m] + 10 (T[m–1] + 10 (T[m–2] + + 10 (T[2] + 10 T[1]))))

ts+1 = 10 (ts – 10m–1 T[s+1]) + T[s+m+1]

String Searching AlgorithmsString Searching AlgorithmsRabin-Karp Algorithm - Example Rabin-Karp Algorithm - Example 11

T = 289462340372392345P = 234

p = 234

t = 289, 894, 946, 462, 623, 234, 340, 403, 37, 372, 723, 239, 392, 923, 234, 345

String Searching AlgorithmsString Searching AlgorithmsRabin-Karp Algorithm - ProblemRabin-Karp Algorithm - Problem

What to do, if p is too large to be stored as integer data type?

(The simplest) solution:

Use p mod q and ti mod q, instead of p and ti

If p mod q ti mod q no match is possible at position i

If p mod q = ti mod q we have to check for match explicitly

String Searching AlgorithmsString Searching AlgorithmsRabin-Karp Algorithm - Rabin-Karp Algorithm - AlgorithmAlgorithmRabinKarpMatcher(string P, string T, integer d, integer q)

n length[T]; m length[P]h dm–1 mod qp 0; t0 0

for i 1 to m dop (d p + P[i]) mod qt0 (d t0 + T[i]) mod q

for s 0 to n – 1 doif p = ts then

if P[1...m] = T[s+1 ... s+m] thenprint s

if s < n – 1 thents (d (ts – T[s+1] h) + T[s+m+1]) mod q

String Searching AlgorithmsString Searching AlgorithmsRabin-Karp Algorithm - Example Rabin-Karp Algorithm - Example 22

T = 289462340372392345P = 234q = 5

p = 234p mod q = 4

t = 289, 894, 946, 462, 623, 234, 340, 403, 37, 372, 723, 239, 392, 923, 234, 345

t mod q = 4, 4, 1, 2, 3, 4, 0, 3, 2, 2, 3, 4, 2, 3, 4, 0

String Searching AlgorithmsString Searching AlgorithmsRabin-Karp Algorithm - Rabin-Karp Algorithm - generalizationgeneralization

Instead of calculating numbers mod q, we can use an arbitraryhash function

String Searching AlgorithmsString Searching AlgorithmsRabin-Karp Algorithm - Rabin-Karp Algorithm - ComplexityComplexity

[Adapted from T.Ralphs]

String Searching AlgorithmsString Searching AlgorithmsRabin-Karp Algorithm - Rabin-Karp Algorithm - ComplexityComplexity

Worst case:

T(n,m) = (n m + 1) m (1) = (n m)

Average case:

number of correct matches - vnumber of incorrect matches - n/q

T(n,m) = (n + m) + (m(v + n/q))

If v is small and m q, then T(n,m) = (n + m)

String Searching AlgorithmsString Searching AlgorithmsTwo dimensional pattern Two dimensional pattern matchingmatching

[Adapted from M.Crochemore,T.Lecroq]

String Searching AlgorithmsString Searching AlgorithmsTwo dimensional pattern Two dimensional pattern matchingmatching

[Adapted from M.Crochemore,T.Lecroq]

String Searching AlgorithmsString Searching AlgorithmsKnuth-Morris-Pratt Algorithm - Knuth-Morris-Pratt Algorithm - IdeaIdea

[Adapted from A.Cawsey]

String Searching AlgorithmsString Searching AlgorithmsKnuth-Morris-Pratt Algorithm - Knuth-Morris-Pratt Algorithm - Some historySome history

[Adapted from K.Wayne]

String Searching AlgorithmsString Searching AlgorithmsKnuth-Morris-Pratt Algorithm - Knuth-Morris-Pratt Algorithm - IdeaIdeaT = gadji beri bimba glandridiP = gadjama

g a d j i b e r i b i m b a g l a n d r i d i

g a d j a m a


g a d j a m a

String Searching AlgorithmsString Searching AlgorithmsKnuth-Morris-Pratt Algorithm - Knuth-Morris-Pratt Algorithm - IdeaIdeaT = gadjama gramma beridaP = gaga

g a d j a m a g r a m m a b e r i d a

g a g a

g a d j a m a g r a m m a b e r i d a

g a g a

String Searching AlgorithmsString Searching AlgorithmsKnuth-Morris-Pratt Algorithm - Knuth-Morris-Pratt Algorithm - IdeaIdea

For each position q = 1, , m in P compute the number of positions by which pattern can be advanced, if a mismatch has been previously detected in q-th position.

String Searching AlgorithmsString Searching AlgorithmsKMP - AlgorithmKMP - Algorithm

KnuthMorrisPrattMatcher(string P, string T)n length[T]m length[P] PrefixFunction(P)q 0for i 1 to n do

while q > 0 & P[q+1] T[i] do q [q]

if P[q+1] = T[i] thenq q + 1

if q = m then print i m q [q]

String Searching AlgorithmsString Searching AlgorithmsKMP - Prefix FunctionKMP - Prefix Function

A << B - A is prefix of B, e.g. ab << abacae

A >> B - A is suffix of B, e.g. ae >> abacae

Ps - initial substring of P of length s

sP - terminal substring of P of length s

Prefix function:

: {1 ,2, , m} {0, 1, 2, , m–1}

[q] = max {k : k < q & Pk >> Pq}

String Searching AlgorithmsString Searching AlgorithmsKMP - Prefix FunctionKMP - Prefix Function

[q] = max {k : k < q & Pk >> Pq}

[q] is the length of the longest prefix of P that is a propersuffix of Pq.

If a mismatch is detected at position q, then patterncan be advanced by q – [q] positions.

String Searching AlgorithmsString Searching AlgorithmsKMP - Prefix Function - ExampleKMP - Prefix Function - Example

[q] = max {k : k < q & Pk >> Pq}


P = abracadabra

= 0,0,0,1,0,1,0,1,2,3,4

String Searching AlgorithmsString Searching AlgorithmsKMP - Prefix Function - KMP - Prefix Function - AlgorithmAlgorithmPrefixFunction(string P)

m length[P][1] 0 k 0for q 2 to m do

while k > 0 & P[k+1] P[q] do k [k]

if P[k+1] = P[q] thenk k + 1

[q] kreturn

String Searching AlgorithmsString Searching AlgorithmsKMP - ComplexityKMP - Complexity




if q = m then print i m q [q]

(n) times

In worst case (m) times

Thus T(n,m) = O(nm)...

String Searching AlgorithmsString Searching AlgorithmsKMP - Complexity KMP - Complexity

T(n,m) = TP(m) + n TWhile(m) = O(n m)?

• q value are increased at most n times

• always q 0

Thus, q can not be decreased more than n times, i.e.while loop can be executed no more than n times.

T(n,m) = TP(m) + n TWhile(m) = TP(m) + (n)




if q = m then print i m

q [q]

String Searching AlgorithmsString Searching AlgorithmsKMP - Prefix Function - KMP - Prefix Function - CorrectnessCorrectness

[q] = max {k : k < q & Pk >> Pq}


We define : 0[q] = q, i+1[q] = [i[q]]

*[q] = {q, [q], 2[q], , t[q] = 0}


0[q] = q, i+1[q] = [i[q]]

*[q] = {q, [q], 2[q], , t[q] = 0}

Lemma

Let P be a pattern of length m with prefix function .Then, for q = 1,2, , m we have *[q] = {k : Pk >> Pq}


Lemma

Let P be a pattern of length m with prefix function .For q = 1,2, , m, if [q] > 0, then [q] – 1 *[q–1].


Corollary

Let P be a pattern of length m with prefix function .For q = 2, , m:

For q = 2, , m we define Eq–1 *[q–1] by

Eq–1 = {k : k *[q–1] & P[k+1] = P[q]}

[q] =0, if Eq–1 =

1 + max{k Eq–1}, if Eq–1

String Searching AlgorithmsString Searching AlgorithmsKMP - Prefix Function - KMP - Prefix Function - CorrectnessCorrectnessWe consecutively compute [1], [2], , [m]

[1] = 0

For k > 1:

if P[k] = P[[k–1] + 1], then [k] = [k–1] + 1,

else, if P[k] = P[[k–2] + 1], then [k] = [[k–1]] + 1,

else, if P[k] = P[[k–3] + 1], then [k] = [[[k–1]]] + 1,

String Searching AlgorithmsString Searching AlgorithmsKMP - Prefix Function - KMP - Prefix Function - Complexity Complexity

TP(m) = const + m TWhile(m) = O(m2)

• k value are increased at most n times

• always k 0

Thus, k can not be decreased more than n times, i.e.while loop can be executed no more than n times.

TP(m) = const + m TWhile(m) = (m)

PrefixFunction(string P)m length[P][1] 0 k 0for q 2 to m do



[q] kreturn

String Searching AlgorithmsString Searching AlgorithmsKMP - Complexity KMP - Complexity

T(n,m) = TP(m) + n TWhile(m) = TP(m) + (n) =(m) + (n) =(m + n)

PrefixFunction(string P)m length[P][1] 0 k 0for q 2 to m do



[q] kreturn




if q = m then print i m

q [q]

String Searching AlgorithmsString Searching AlgorithmsBoyer-Moore Algorithm - Idea 1Boyer-Moore Algorithm - Idea 1

T = gadji beri bimba glandridiP = lonni


l o n n i


l o n n i

Bad character heuristic

String Searching AlgorithmsString Searching AlgorithmsBoyer-Moore Algorithm - Idea 2Boyer-Moore Algorithm - Idea 2

T = gadji beri bimba glandridiP = ajiji


a j i j i


a j i j i

Good suffix heuristic

String Searching AlgorithmsString Searching AlgorithmsBoyer-Moore - Bad Character Boyer-Moore - Bad Character FunctionFunction

Bad character function:

: {0,1,2, , m}

[s] = max {k : P[k] = s} (if such k exists)

[s] = 0 (otherwise)[Adapted from M.Goodrich, R.Tamassia]

String Searching AlgorithmsString Searching AlgorithmsBoyer-Moore - Bad Character Boyer-Moore - Bad Character FunctionFunction

BadCharacterFunction(string P, set )m length[P]for a do

[a] 0for j 1 to m do

[P[j]] jreturn

TB(m,||) = (m + ||)

String Searching AlgorithmsString Searching AlgorithmsBoyer-Moore - Suffix FunctionBoyer-Moore - Suffix Function

Suffix function:

: {1, 2, , m} {1, 2, , m}

[j] = m – max {k : k < m & jP >> Pk PK >> jP}

[Adapted from R.Lee, C.Lu]








SuffixFunction(string P)m length[P] PrefixFunction(P) P’ Reverse(P); ’ PrefixFunction(P’)for j 0 to m do

[j] m – [m]for l 1 to m do

j m – ’[l]if [j] > l – ’[l] then

[j] l – ’[l]return

TS(m) = (m)

String Searching AlgorithmsString Searching AlgorithmsBoyer-Moore Algorithm - Boyer-Moore Algorithm - AlgorithmAlgorithmBoyerMooreMatcher(string P, string T, set )

n length[T]m length[P] LastOccurenceFunction(P,m, ) GoodSuffixFunction(P,m)s 0while s n m do

j mwhile j > 0 & P[j] = T[s + j] do

j j 1if j = 0 then

print ss s + [0]

else s s + max( [j], j [T[s + j]])

String Searching AlgorithmsString Searching AlgorithmsBoyer-Moore Algorithm - Boyer-Moore Algorithm - ComplexityComplexity

TB(m,||) = (m + ||)

TS(m) = (m)

T(n,m,||) = TB(m,||) + TS(m) + n TWhile(m) =

= (m + ||) + (m) + O(n m) = O(|| + n m)?

It can be shown that

T(n,m,||) = (|| + n + m)


It can be shown that:

T(n,m,||) = (|| + n m) using only bad character rule

T(n,m,||) = (|| + n + m) using only good suffix rule, ifthe pattern does not occur in text

T(n,m,||) = (|| + n m) using only good suffix rule, ifthe pattern does occur in text


With Galil's modification:

T(n,m,||) = (|| + n + m) using only good suffix rule

There is also a similar Apostolico-Giancarlo algorithm that achieves (|| + n + m) time bound (which is much easier toprove)

On average the number of character comparisons is n/m (for large ||)

String Searching AlgorithmsString Searching AlgorithmsAlgorithms - Complexity Algorithms - Complexity comparisoncomparison

[Adapted from H.Løvengreen]

String Searching AlgorithmsString Searching AlgorithmsAlgorithms - Efficiency Algorithms - Efficiency comparisoncomparison

n=5000

[Adapted from I.Spence]

String Searching AlgorithmsString Searching AlgorithmsComplexity - Lower BoundComplexity - Lower Bound

Theorem (Rivest)

Any string searching algorithm has worst-case time complexity

T(n,m) = (m + n)

String Searching AlgorithmsString Searching AlgorithmsSuffix Trees - The problemSuffix Trees - The problem

Theorem (Rivest)

Any string searching algorithm has worst-case time complexity

T(n,m) = (m + n)

Despite this, we probably can do better!(Well, for slightly different problem...)

[Adapted from P.Kilpeläinen]

String Searching AlgorithmsString Searching AlgorithmsSuffix TreesSuffix Trees


String Searching AlgorithmsString Searching AlgorithmsSuffix TreesSuffix Trees


String Searching AlgorithmsString Searching AlgorithmsSuffix Trees - ExampleSuffix Trees - Example


String Searching AlgorithmsString Searching AlgorithmsSuffix Trees - Do they always Suffix Trees - Do they always exist?exist?


String Searching AlgorithmsString Searching AlgorithmsSuffix Trees - Application to Suffix Trees - Application to string matchingstring matching


String Searching AlgorithmsString Searching AlgorithmsSuffix Trees - ConstructionSuffix Trees - Construction


String Searching AlgorithmsString Searching AlgorithmsSuffix Trees - ConstructionSuffix Trees - Construction


String Searching AlgorithmsString Searching AlgorithmsSuffix Trees - Construction - Suffix Trees - Construction - ExampleExample






String Searching AlgorithmsString Searching AlgorithmsSuffix Trees - Construction - Suffix Trees - Construction - ComplexityComplexity


String Searching AlgorithmsString Searching AlgorithmsSuffix Trees - Compact Suffix Trees - Compact representationrepresentation


String Searching AlgorithmsString Searching AlgorithmsSuffix Trees - Compact Suffix Trees - Compact representation - Examplerepresentation - Example


String Searching AlgorithmsString Searching AlgorithmsSuffix Trees - Some historySuffix Trees - Some history


String Searching AlgorithmsString Searching AlgorithmsSuffix Trees - Ukkonen's Suffix Trees - Ukkonen's algorithmalgorithm


String Searching AlgorithmsString Searching AlgorithmsSuffix Trees - Implicit treesSuffix Trees - Implicit trees






String Searching AlgorithmsString Searching AlgorithmsSuffix Trees - String pathsSuffix Trees - String paths


String Searching AlgorithmsString Searching AlgorithmsSuffix Trees - Ukkonen's Suffix Trees - Ukkonen's algorithmalgorithm


String Searching AlgorithmsString Searching AlgorithmsSuffix Trees - ExtensionsSuffix Trees - Extensions


String Searching AlgorithmsString Searching AlgorithmsSuffix Trees - ExtensionsSuffix Trees - Extensions


String Searching AlgorithmsString Searching AlgorithmsSuffix Trees - Extensions - Suffix Trees - Extensions - ExampleExample


String Searching AlgorithmsString Searching AlgorithmsSuffix Trees - Ukkonen's Suffix Trees - Ukkonen's algorithm - Complexityalgorithm - Complexity






String Searching AlgorithmsString Searching AlgorithmsSuffix Trees - Suffix linksSuffix Trees - Suffix links






String Searching AlgorithmsString Searching AlgorithmsSuffix Trees - Speeding upSuffix Trees - Speeding up


















String Searching AlgorithmsString Searching AlgorithmsSuffix Trees - Eliminating Suffix Trees - Eliminating extensionsextensions


String Searching AlgorithmsString Searching AlgorithmsSuffix Trees - Single phase Suffix Trees - Single phase algorithmalgorithm








Engineering

String searching