Sublinear Algorithms Lecture 23: April 20 10312013 …

Sublinear AlgorithmsSublinear Algorithms

Lecture 23: April 20

1 0 3 1 2 0 1 3 …

Matrix Multiplication

Matrix multiplication: Given two nxn matrices A,B, compute AB.

Straightfoward: O(n3) algorithm.

Sophisticated: O(n2.376) algorithm

Matrix multiplication verification: Given A,B,C, check if AB=C.

Would it be easier?


Matrix multiplication verification: Given A,B,C, check if AB=C.

Freivald’s technique: there is an O(n2) time randomized algorithm

This technique assumes that the entries are field elements, say, rational numbers.

Idea: check some random places!

Can we just take some row of A and some column of B?


1

Can we just take some row of A and some column of B?

1

Each operation just takes linear time, i.e. O(n) time.

But the probability of catching it is too low, i.e. O(1/n2).

pick a random vector


Idea: Need to do something more global, i.e. every entry is involved.

Freivald’s technique:

Then, compute x=Br, y=Ax=ABr, z=Cr.

Check if y=z.

Obviously, if AB=C, then y=z.

This procedure can be done in O(n2) time.

Theorem: if AB≠C, then Pr[ABr=Cr] ≤ 1/2.


Proof: Let D=AB-C; so D is not the all-zeroes matrix.

We wish to bound the probability that y=z,

or equivalently, the probability that Dr=0.

Let say the first row D(1) of D has a non-zero entry.

And let D(1) = {d1,d2,d3,…,dk,0,0,…,0}.

What is the probability that D(1)·r is non-zero?

This probability would give a lower bound that we catch y≠z.


What is the probability that D(1)·r is non-zero?

Recall that D(1) = {d1,d2,d3,…,dk,0,0,…,0}.

This is non-zero if and only if:

if and only if:

r1 are uniformly chosen from two values (0 or 1),

the probability that D(1)·r is non-zero is at least

1/2.Hence, we have at least 1/2 probability to catch it if AB≠C.

Verifying Polynomial Identities

Given three polynomials P1(x), P2(x), and P3(x), verify that P1 P2 = P3.

Fast Fourier Transform: O(n log(n)) time if P1,P2 are of degree n.

Idea: just pick a random point y and test if P1(y) P2(y) = P3(y).

Can we test it in linear time?

Consider Q(x)=P1(x)P2(x) – P3(x).It is a degree 2n polynomial, and has at most 2n roots.

If we choose randomly from, say, 4n elements and test it,

then with probability at least 1/2 we can catch it.

Verifying Perfect Matching

Given a bipartite graph B=(U,V,E), verify if B has a perfect matching.

Consider the following matrix:

=

if there is an edge between u(i) and v(j)

otherwise

Theorem: B has a perfect matching if and only if det(A) ≠ 0.



Since each variable x(i,j) occurs at most once in A,there can be no cancellation of the terms in the summation.

Therefore the determinant is not identically zero if and only ifthere is a permutation πfor which the corresponding term is non-zero.

That is, each entry is non-zero, and hence a perfect matching!



So, now we “only” need to check if det(A) ≠ 0.

Idea: Substitute random points and evaluate the polynomial.

Observation: det(A) is a (multi-variable) polynomial

Once the values are substituted, the problem reduces to

compute a determinant, which can be done in O(n2.376) time.

Deterministic running time = O(m√n) = O(n2.5) time.


Idea: Substitute random points and evaluate the polynomial.

What is the probability that it works?

(Schwartz-Zippel Theorem): Let Q(x1,…,xn) in F[x1,…xn] bea multivariate polynomial of total degree d. Fix any finite set S of F,and let r1,…,rn be chosen independently and uniformly at random from S.Then,

In our application, d = number of vertices on one side.

Choose a field of size at least 2n, say modulo a small prime number, will do.

Verifying Equality of Strings

Suppose that Alice maintains a large database of

information. Bob maintains a second copy of the

database. Periodically, they want to compare their

databases for consistency. Any deterministic consistency check requires to transmit all n bits.

Can we do better?

Let the bits of Alice be (a1,…,an) and the bits of Bob be (b1,…,bn).

Check random bits?

Not efficient enough if there is only one error.

Need some more “global” method.


Idea: create “fingerprints” for the two databases.

Interpret the data as n-bit integers a and b, by defining

Define the fingerprinting function as

where p is a prime number.

Algorithm: pick a random prime, test if F(a) = F(b).


Algorithm: pick a random prime, test if F(a) = F(b).

Bits sent: O(log(p)) instead of n.To be efficient, want p to be small.

What is the probability of making mistake?

Consider c = |a-b|. How many primes can divide c?

Since c ≤ 2n, the number of distinct prime divisor of c is at most n, why?


There are at most n primes which divide c = |a-b|.

Now we want to choose a number z,

large enough so that there are enough primes smaller than z,

small enough so that the communication is efficient.

Bits sent: O(log(p)) instead of n.To be efficient, want p to be small.

Given z, by the Prime Number Theorem,there are about z/ln(z) primes smaller than z.


Fix z.

Probability of making mistake ≤ n/(z/ln(z)) = nln(z)/z.

Bits need to be sent ≤ O(log(z))

A good choice: set z = tn log(tn) for a large t.

Probability of making mistake ≤ O(1/t).

Bits need to be sent ≤ O(log(t) + log(n))

Set t=n would be good.

Pattern Matching

Given a text X = x1 x2 … xn, and a pattern Y = y1 y2 … ym,check if there is a j so that xj … x(j+m-1) = y1 … ym.

There is a straightforward O(mn) algorithm.

There is an optimal O(m+n) deterministic algorithm.

We now show a very simple O(m+n) randomized algorithm.

Idea: fingerprinting.

Pattern Matching

Write X(j) for xj … x(j+m-1).

Goal: to check if Y=X(j) for some j.

Idea: fingerprinting.

Algorithm: compare F(Y) and F(X(j)), say “match” if they are equal

Fix a number z, choose a random prime p smaller than z.

There could be false match, but this probability is at most

Pattern Matching

There are n positions, so the probability that we make a false match

Choose This is at most O(1/n).

What is the running time?

So, the fingerprints can be updated in O(1) time, and hence O(n+m) time.

Interactive Proof System

Counting 3-SAT: Given a boolean formula,

check if the number of satisfying assignments is equal to k.

3-SAT: Given a boolean formula,

check if there is a satisfying assignment.

Reversi: Given a broad configuration,

check if the black player has a winning strategy.

All have an interactive proof system.

Implication: even being foolish doesn’t mean

someone can fool you easily!

Interactive Proof System

Key technique: arithmetization

Counting 3-SAT: Given a boolean formula,

check if the number of satisfying assignments is equal to k.

And then use the previous technique or picking random element to evaluate…

Probabilistic Checkable Proofs

3-SAT: Given a boolean formula,

check if this formula is satisfiable.

If this is satisfiable, you can convince me by showing a truth assignment,

then I can check in linear time whether it indeed satisfies the formula.

Can we do it in sublinear time?

PCP theorem: There is a proof protocol with the following property:

I only need to read 3 bits of the proof and

if you are honest, then I always trust you;

if you are lying, then I can catch you lying with probability at least ½!!

Has deep implication in hardness of approximation!

about this course

Focus: analysis

Exact algorithm: running time

Approximation algorithm: performance guarantee

Randomized algorithm: probability

Techniques:

-Combinatorial approaches

-Linear programming

-Probabilities

-Algebraic technique?

-Topological method?????

Feedback?

Documents

Sublinear Algorithms Lecture 23: April 20 10312013 …