1

FingerprintFingerprint

2

Verifying set equalityVerifying set equality

String Matching – Rabin-Karp AlgorithmString Matching – Rabin-Karp Algorithm

3

Verifying set equalityVerifying set equality

4

Verifying set equalityVerifying set equality

5

Verifying set equalityVerifying set equality

6

Verifying set equalityVerifying set equality

7

Fingerprinting Fingerprinting

8

FingerprintingFingerprinting

9

Fingerprinting ComputationFingerprinting Computation

10

Fingerprinting ComputationFingerprinting Computation

Horner’s Rule

11

ProtocolProtocol

12

Prime Number qPrime Number q

13

False PositiveFalse Positive

14

Prime DivisorsPrime Divisors

15

Density of PrimesDensity of Primes

16

Density of PrimesDensity of Primes

(x) = número de primos menores ou iguais a x(x) = número de primos menores ou iguais a x (13) = 6(13) = 6– Primos < = do que 13 = 2, 3, 5 , 7, 11 e 13Primos < = do que 13 = 2, 3, 5 , 7, 11 e 13

O valor de O valor de não muda até chegarmos ao não muda até chegarmos ao próximo primo.próximo primo. (13) = (13) = (14) = (14) = (15) = (15) = (16) (16) – Ou seja, Ou seja, aumenta em salto de 1, mas o intervalo aumenta em salto de 1, mas o intervalo

entre esses saltos é irregularentre esses saltos é irregular

17

Density of PrimesDensity of Primes

EEsses intervalos tornam-se cada vez maiores, isto sses intervalos tornam-se cada vez maiores, isto é, a chance de um inteiro escolhido ao acaso ser é, a chance de um inteiro escolhido ao acaso ser

primo diminui quando avançamos para os primo diminui quando avançamos para os números maiores. números maiores.

PERGUNTA:PERGUNTA: O valor de O valor de não poderia ser aproximado por alguma não poderia ser aproximado por alguma

função conhecida?função conhecida?

18

Density of PrimesDensity of Primes

Para um valor elevado de x, Para um valor elevado de x,

(x) ~(x) ~ x/ ln x x/ ln x . .

Ou seja, Ou seja,

lim lim (x)(x) = 1 = 1x x x/ln x x/ln x

19

Sample SpaceSample Space

20

Probability of a bad primeProbability of a bad prime

21

Final Protocol PropertiesFinal Protocol Properties

22

String MatchingString Matching

23

String MatchingString Matching

Many applications

–While using editor/word processor/browser

–Login name & password checking

–Virus detection

–Header analysis in data communications

–DNA sequence analysis

24

Naïve O(nm) algorithmNaïve O(nm) algorithm

25

Rabin-Karp AlgorithmRabin-Karp Algorithm

26

FingerprintingFingerprinting

27

Fingerprinting functionFingerprinting function

28

Fingerprinting Fingerprinting computationcomputation

The only expensive operation

29

False Positives?False Positives?

30

Sample SpaceSample Space

31

False PositivesFalse Positives

32

FingerprintingFingerprinting

33

Primality testingPrimality testing A natural number A natural number nn is prime iff the only is prime iff the only

natural numbers dividing natural numbers dividing nn are are 11 and and nn

34

Primality testingPrimality testing A natural number A natural number nn is prime iff the only is prime iff the only

natural numbers dividing natural numbers dividing nn are are 11 and and nn The following are prime: The following are prime: 22,, 3 3,, 5 5,, 7 7,, 11 11,, 13 13,,

35

Primality testingPrimality testing A natural number A natural number nn is prime iff the only is prime iff the only

natural numbers dividing natural numbers dividing nn are are 11 and and nn The following are prime: The following are prime: 22,, 3 3,, 5 5,, 7 7,, 11 11,, 13 13, ,

……aand so are nd so are 12997091299709,,

1548586315485863,,

2280176348922801763489, …, …

36

Primality testingPrimality testing A natural number A natural number nn is prime iff the only is prime iff the only

natural numbers dividing natural numbers dividing nn are are 11 and and nn The following are prime: The following are prime: 22,, 3 3,, 5 5,, 7 7,, 11 11,, 13 13, ,

……aand so are nd so are 12997091299709,,

1548586315485863,,

2280176348922801763489, …, …

There is an infinite number of prime numbersThere is an infinite number of prime numbers

37

Primality testingPrimality testingThere is an infinite number of prime numbersThere is an infinite number of prime numbersProof: Let us suppose the number of primes is Finite.

38

Primality testingPrimality testingThere is an infinite number of prime numbersThere is an infinite number of prime numbersProof: Let us suppose the number of primes is Finite.

Let p1, p2, … pk be all primes.

Let n = p1 p2 … pk+1,

39

Primality testingPrimality testingThere is an infinite number of prime numbersThere is an infinite number of prime numbersProof: Let us suppose the number of primes is Finite.

Let p1, p2, … pk be all primes.

Let n = p1 p2 … pk+1,

n must be composite.

40

Primality testingPrimality testingThere is an infinite number of prime numbersThere is an infinite number of prime numbersProof: Let us suppose the number of primes is Finite.

Let p1, p2, … pk be all primes.

Let n = p1 p2 … pk+1,

n must be composite.

there exists a prime p s.t. p | n (fund theo. arithmetic), and p cannot be any of the p1, p2, … pk

41

Primality testingPrimality testingThere is an infinite number of prime numbersThere is an infinite number of prime numbersProof: Let us suppose the number of primes is Finite.

Let p1, p2, … pk be all primes.

Let n = p1 p2 … pk+1,

n must be composite.

there exists a prime p s.t. p | n (fund theo. arithmetic), and p cannot be any of the p1, p2, … pk

Therefore, p1, … pk were not all the prime numbers.

42

Some questions?Some questions? Is Is 22101101-1-1==25353012004564588029934064107512535301200456458802993406410751

prime?prime? How do we check whether a number is How do we check whether a number is

prime?prime? How do we generate huge prime numbers?How do we generate huge prime numbers? Why do we care?Why do we care?

43

Some questions?Some questions? Is Is 22101101-1-1==25353012004564588029934064107512535301200456458802993406410751

prime?prime? How do we check whether a number is How do we check whether a number is

prime?prime? How do we generate huge prime numbers?How do we generate huge prime numbers? Why do we care?Why do we care?

44

Some questions?Some questions? Is Is 22101101-1-1==25353012004564588029934064107512535301200456458802993406410751

prime?prime? How do we check whether a number is How do we check whether a number is

prime?prime? How do we generate huge primeHow do we generate huge prime numbersnumbers?? Why do we care?Why do we care?

45

Some questions?Some questions? Is Is 22101101-1-1==25353012004564588029934064107512535301200456458802993406410751

prime?prime? How do we check whether a number is How do we check whether a number is

prime?prime? How do we generate huge primeHow do we generate huge prime numbersnumbers?? Why do we care?Why do we care?

46

Naïve solution: Finding Naïve solution: Finding the the smallest divisorsmallest divisor

of nof n

– For i=2,..., n do For i=2,..., n do Divide n by i until Divide n by i until n mod i = 0n mod i = 0

Check if i is a divisor of n for some i = 2, ..., n

47

An improvementAn improvement

Check if i is a divisor of n for some i = 2, ..., n

48

An improvementAn improvement

Check if i is a divisor of n for some i = 2, ..., n

Why can we do that?

49

Theorem:Theorem: Composit numbers have a divisor Composit numbers have a divisor bellow their square root bellow their square root

50

Theorem:Theorem: Composit numbers have a divisor Composit numbers have a divisor bellow their square root bellow their square root

Proof Idea: n composite n = ab, 0 < a b < n

51

Theorem:Theorem: Composit numbers have a divisor Composit numbers have a divisor bellow their square root bellow their square root

Proof Idea: n composite n = ab, 0 < a b < n

a sqrt(n)

52

Theorem:Theorem: Composit numbers have a divisor Composit numbers have a divisor bellow their square root bellow their square root

Proof Idea: n composite n = ab, 0 < a b < n

a sqrt(n)

Otherwise, we obtain ab > n (contradiction!!)

53

Is there a more efficient Is there a more efficient way of checking primality?way of checking primality?

54

Is there a more efficient Is there a more efficient way of checking primality?way of checking primality?

Yes! At least if we are willing to accept

a tiny probability of error.

55

We can prove that a number is not We can prove that a number is not primeprime withoutwithout explicitly finding a explicitly finding a divisordivisor of it of it

Is there a more efficient Is there a more efficient way of checking primality?way of checking primality?

Yes! At least if we are willing to accept

a tiny probability of error.

56

We can prove that a number is not We can prove that a number is not primeprime withoutwithout explicitly finding a explicitly finding a divisordivisor of it of it

Is there a more efficient Is there a more efficient way of checking primality?way of checking primality?

Yes! At least if we are willing to accept

a tiny probability of error.

RANDOMNESS IS USEFUL IN COMPUTATION

57

The The FermatFermat Primality Test Primality Test

FermatFermat’s little theorem’s little theorem::If If pp is a prime and p does not divide the is a prime and p does not divide the

inteintegger er aa, then:, then:

a a p-1p-11(mod p1(mod p))

58

Suppose that ra e sa have are the same modulo p, then we have r = s (mod p) Contradiction!!

Aa, 2a, 3a, ..., (p-1)a quando divididos por p possuem restos diferentes:1, 2, ..., p-1

Proof: • List the first p-1 positive multiple of a:a, 2a, 3a, 4a, ..., (p-1) a

The The FermatFermat Primality Test Primality Test

FermatFermat’s little theorem’s little theorem::If If pp is a prime and p does not divide the is a prime and p does not divide the

inteintegger er aa, then:, then:

a a p-1p-11(mod p1(mod p))

59

Suppose that ra and sa are the same modulo p, then we have r = s (mod p) Contradiction!!

Aa, 2a, 3a, ..., (p-1)a quando divididos por p possuem restos diferentes:1, 2, ..., p-1

Proof: • List the first p-1 positive multiple of a:a, 2a, 3a, 4a, ..., (p-1) a

The The FermatFermat Primality Test Primality Test

FermatFermat’s little theorem’s little theorem::If If pp is a prime and p does not divide the is a prime and p does not divide the

inteintegger er aa, then:, then:

a a p-1p-11(mod p1(mod p))

60

Suppose that ra and sa are the same modulo p, then we have r = s (mod p) Contradiction!!

Aa, 2a, 3a, ..., (p-1)a when divided by p have the different reminders:1, 2, ..., p-1

Proof: • List the first p-1 positive multiple of a:a, 2a, 3a, 4a, ..., (p-1) a

The The FermatFermat Primality Test Primality Test

FermatFermat’s little theorem’s little theorem::If If pp is a prime and p does not divide the is a prime and p does not divide the

inteintegger er aa, then:, then:

a a p-1p-11(mod p1(mod p))

61

a(p-1)(p-1)! = (p-1)! (mod p)

Proof: a. 2a. 3a. ... . (p-1)a 1. 2. 3. ... . (p-1) (mod p)

The The FermatFermat Primality Test Primality Test

FermatFermat’s little theorem’s little theorem::If If pp is a prime and p does not divide the is a prime and p does not divide the

inteintegger er aa, then:, then:

a a p-1p-11(mod p1(mod p))

62

a(p-1)(p-1)! = (p-1)! (mod p)

Dividing by (p-1)! we get the result

Proof: a. 2a. 3a. ... . (p-1)a 1. 2. 3. ... . (p-1) (mod p)

The The FermatFermat Primality Test Primality Test

FermatFermat’s little theorem’s little theorem::If If pp is a prime and p does not divide the is a prime and p does not divide the

inteintegger er aa, then:, then:

a a p-1p-11(mod p1(mod p))

63

A Corollary:A Corollary:

If If pp is a prime then, for any integer is a prime then, for any integer a:a:

aapp a (mod p) a (mod p)

64

A Corollary:A Corollary:

If If pp is a prime then, for any integer is a prime then, for any integer a:a:

aapp a (mod p) a (mod p)

The result is trivial if p divides a: a(ap-1 – 1) 0 (mod p)

If a does not divide a, then we need only multiply the congruence in Fermat´s little theorem by a

to complete the proof

65

A Corollary:A Corollary:

If If pp is a prime then, for any integer is a prime then, for any integer a:a:

aapp a (mod p) a (mod p)

The result is trivial if p divides a: a(ap-1 – 1) 0 (mod p)

If p does not divide a, then we need only multiply the congruence in Fermat´s little theorem by a

to complete the proof

66

Corollary:Corollary:

If If aan n ≠≠a a (mod n)(mod n) , , for some for some aa, , then then nn is not a prime! is not a prime!

Such an a is a Such an a is a witnesswitness to the to the compositeness of compositeness of nn..

The Fermat Test:

Do 100 times:

Pick a random 1<a<n and compute an (mod n).

If an a (mod n), then n is not a prime.

If all 100 tests passed, declare n to be a prime.

69

Is the Fermat test correct?Is the Fermat test correct?

If the Fermat test says that a number If the Fermat test says that a number nn is composite, is composite,then the number then the number nn is indeed a composite number. is indeed a composite number.

If If n n is a prime number, the Fermat test will always say that is a prime number, the Fermat test will always say that nn is is prime. prime.

But,

Can the Fermat test say that a composite Can the Fermat test say that a composite number is prime?number is prime?

What is the What is the probabilityprobability that this will happen? that this will happen?

70

Carmichael numbersCarmichael numbersA composite number n is a Carmichael number

iff an a (mod n) for every integer a.

The first Carmichael numbers are: 561, 1105, 1729, 2465, 2821, 6601, 8911, 10585, 15841, …

On Carmichael numbers, the Fermat test is always wrong!

Carmichael numbers are fairly rare.

71

Theorem: (Rabin ’77) If n is a composite number that is not a Carmichael number, then at least half of the numbers between 1 and n are witnesses to the compositeness of n.

72

Theorem: (Rabin ’77) If n is a composite number that is not a Carmichael number, then at least half of the numbers between 1 and n are witnesses to the compositeness of n.

Proof: Consider Z*n = {1, 2, ..., n-1}

Let B={x / x Z*n and xn-1 1 (mod n)}

We are going to show that B is subgroup of Z*n

For this:

1. 1 B2. x1, x2 B x1. x2 B3. x B x-1 B

73

Theorem: (Rabin ’77) If n is a composite number that is not a Carmichael number, then at least half of the numbers between 1 and n are witnesses to the compositeness of n.

Proof: Consider Z*n = {1, 2, ..., n-1}

Let B={x / x Z*n and xn-1 1 (mod n)}

We are going to show that B is subgroup of Z*n

For this:

1. 1 B : 1n-1 1 (mod n)2. x1, x2 B x1. x2 B3. x B x-1 B

74

Theorem: (Rabin ’77) If n is a composite number that is not a Carmichael number, then at least half of the numbers between 1 and n are witnesses to the compositeness of n.

Proof: Consider Z*n = {1, 2, ..., n-1}

Let B={x / x Z*n and xn-1 1 (mod n)}

We are going to show that B is subgroup of Z*n

For this:

1. 1 B : 1n-1 1 (mod n)2. x1, x2 B x1. x2 B3. x B x-1 B

(x1)n-1 1 (mod n)(x2)n-1 1 (mod n)

(x1.x2)n-1 1 (mod n)

75

Theorem: (Rabin ’77) If n is a composite number that is not a Carmichael number, then at least half of the numbers between 1 and n are witnesses to the compositeness of n.

Proof: Consider Z*n = {1, 2, ..., n-1}

Let B={x / x Z*n and xn-1 1 (mod n)}

We are going to show that B is subgroup of Z*n

For this:

1. 1 B : 1n-1 1 (mod n)2. x1, x2 B x1. x2 B3. x B x-1 B

(1)n-1 1 (mod n)(x.x-1)n-1 1 (mod n)

(x-1)n-1 1 (mod n)

76

Theorem: (Rabin ’77) If n is a composite number that is not a Carmichael number, then at least half of the numbers between 1 and n are witnesses to the compositeness of n.

Proof: Pr(xn-1 1 (mod n)) = Pr(x B)

It can be proved that 1 B and n-1 B and therefore, |B| 2Since the order of a subgroup divides the subgroup we have that

|B| |Z*n | / 2

Pr(x B) 1/2

77

Theorem: (Rabin ’77) If n is a composite number that is not a Carmichael number, then at least half of the numbers between 1 and n are witnesses to the compositeness of n.

Corollary: Let n be a composite number that is not a Carmichael number. If we pick a random number a, 1<a<n, then a is a witness with a probability of at least a 1/2 !

78

““Correctness” of the Correctness” of the Fermat testFermat test

If If nn is prime, the Fermat test is always is prime, the Fermat test is always rightright.. If If nn is a Carmichael number, is a Carmichael number,

the Fermat test is always the Fermat test is always wrong!wrong! If If nn is composite number that is not a Carmichael number, the Fermat test is wrong with a probability of is composite number that is not a Carmichael number, the Fermat test is wrong with a probability of

at most at most 22-100-100

Is an error probability of 2-100 acceptable?Yes!

79

The Rabin-Miller testThe Rabin-Miller test

A fairly simple modification of the Fermat test that A fairly simple modification of the Fermat test that is correct with a probability of at least 1-2is correct with a probability of at least 1-2-100-100 also on also on Carmichael numbers.Carmichael numbers.

Will not be covered in this course.Will not be covered in this course.

80

A probabilistic algorithmA probabilistic algorithmAn algorithm that uses random choices but

outputs the correct result, with high probability,for every input!

Randomness is a very useful algorithmic tool.

Up to 2002, there were no efficient deterministic primality testing algorithms.

In 2002, Agarwal, Kayal and Saxena found a fast deterministic primality testing algorithm.

81

Finding large prime Finding large prime numbersnumbers

The prime number Theorem:

The number of prime numbers smaller than n is asymptotically n / ln n.

Thus, for every number n, there is “likely” to be a prime number between n and n + ln n.

To find a prime number roughly the size of n, simply test n, n+2, n+4, … for primality.

82

Primality testing versus Primality testing versus FactoringFactoring

Fast primality testing algorithms determine that a number n is composite without Fast primality testing algorithms determine that a number n is composite without finding any of its factors.finding any of its factors.

No efficient factoring algorithms are known.No efficient factoring algorithms are known. Factoring a number is believed to be a much harder task.Factoring a number is believed to be a much harder task.

Primality testing - EasyFactoring - Hard

But, factoring is not that hard on a quantum computer!