Random Permutation and its Applications

Random PermutationHiring Problem

Smallest enclosing circleRandomized incremental construction

Routing in Hypercube

.. Random Permutation and its Applications

Randomized Algorithms

Random Permutation and its Applications

Randomized Algorithms Random Permutation




.. Random Permutation

Algorithm RandomPermutation(A)1. ▷ Produce a R.P. of items stored in an array A[1..n]2. for i← 1 to n3. do r← random(i,n)4. Swap(A[i],A[r])

Question: Does the “algorithm” work if if we replace random(i, n)in line 3 by random(1, n)?





Problem DefinitionDeterministic AlgorithmRandomized AlgorithmAnalysis

.. Hiring Problem

Suppose you are doing a project, for which you need the bestassistant you can get.

You contact an employment agency that promises to send yousome candidates, one per day, for interviewing.

Since you really want to have the best assistant available, youdecide on the following strategy:Strategy: whenever you interview a candidate and thecandidate turns out to be better than your current assistant,you fire your current assistant and hire the new candidate.






.. Deterministic Algorithm

Algorithm HireAssistant1. current-assist ← nil2. for i← 1 to n3. do interview candidate i4. if candidate i is better than current-assist5. then current-assist ← candidate i

Cost: You have to pay f ·n in the worse case where f is the cost ofchanging your assistant and n is the number of candidates.






.. Randomized Algorithm

Algorithm HireAssistant-Randomized1. Compute a random permutation of the candidates2. current-assist ← nil3. for i← 1 to n4. do interview candidate i5. if candidate i is better than current-assist6. then current-assist ← candidate i






.. Analysis

Let Xi be an I.R.V. which is 1 iff the candidate i is better thancandidate 1,2, . . . , i−1.

E(cost) = E(n

∑i=1

cost to be paid for i-th candidate)

=n

∑i=1

E(cost to be paid for i-th candidate)

=n

∑i=1

E(f ·Xi) = fn

∑i=1

E(Xi)

= fn

∑i=1

Pr(Xi = 1) = fn

∑i=1

1i

≈ f ·n lnn





Facility locationProperties of the smallest enclosing circleRandomized incremental constructionA more restricted problemA yet more restricted problemEfficiency analysis

.. Facility location

Given a set of houses and farmsin an isolated area. Can we placea helicopter ambulance post sothat each house and farm can bereached within 15 minutes?

Where should we place anantenna so that a number oflocations have maximumreception?






.. Facility location in geometric terms

Given a set of points in the plane.Is there any point that is within acertain distance of these points?

Where do we place a point thatminimizes the maximum distanceto a set of points?






.. Facility location in geometric terms

Given a set of points in theplane, compute the smallestenclosing circle






.. Smallest enclosing circle

Observation: It must pass throughsome points, or else it cannot besmallest

Take any circle that encloses thepoints, and reduce its radiusuntil it contains a point p

Move center towards p whilereducing the radius further, untilthe circle contains anotherpoint q







Move center on the bisector of pand q towards their midpoint,until:(i) the circle contains a thirdpoint, or(ii) the center reaches themidpoint of p and q







Question: Does the “algorithm” ofthe previous slide work?







Observe: A smallest enclosing circlehas (at least) three points on itsboundary, or only two in which casethey are diametrally opposite

Question: What is the extraproperty when there are three pointson the boundary?






.. Randomized incremental construction

Construction by randomized incremental construction

incremental construction: Add points one by one andmaintain the solution so far

randomized: Use a random order to add the points






.. Adding a point

Let p1, . . . ,pn be the points in random order

Let Ci be the smallest enclosing circle for p1, . . . ,pi

Suppose we know Ci−1 and we want to add pi

If pi is inside Ci−1, then Ci = Ci−1

If pi is outside Ci−1, then Ci will have pi on its boundary






.. Adding a point

Ci−1

pi

Ci−1

pi






.. Adding a point

Question: Suppose we remembered not only Ci−1, but alsothe two or three points defining it. It looks like if pi is outsideCi−1, the new circle Ci is defined by pi and some points thatdefined Ci−1. Why is this false?






.. Adding a point






.. Adding a point

How do we find the smallestenclosing circle of p1 . . . ,pi−1 with pi

on the boundary?

We study the new(!) geometricproblem of computing the smallestenclosing circle with a given point pon its boundary

p






.. Smallest enclosing circle with point

Given a set P of points and onespecial point p, determine thesmallest enclosing circle of P thatmust have p on the boundary

Question: How do we solve it? p






.. Randomized incremental construction

Construction by randomized incremental construction

incremental construction: Add points one by one andmaintain the solution so far

randomized: Use a random order to add the points






.. Adding a point

Let p1, . . . ,pi−1 be the points in random order

Let C′j be the smallest enclosing circle for p1, . . . ,pj (j≤ i−1)and with p on the boundary

Suppose we know C′j−1 and we want to add pj

If pj is inside C′j−1, then C′j = C′j−1

If pj is outside C′j−1, then C′j will have pj on its boundary(and also p of course!)






.. Adding a point

C ′j−1

pj

C ′j−1

pj

p p






.. Adding a point

How do we find the smallestenclosing circle of p1 . . . ,pj−1 with pand pj on the boundary?

We study the new(!) geometricproblem of computing the smallestenclosing circle with two given pointson its boundary

pq






.. Smallest enclosing circle with two points

Given a set P of points and twospecial points p and q, determine thesmallest enclosing circle of P thatmust have p and q on the boundary

Question: How do we solve it? pq






.. Two points known

p

q q

p






.. Two points known

p

q q

p






.. Algorithm: two points known

Assume w.lo.g. that p and q lie on a vertical line. Let ℓ be theline through p and q and let ℓ′ be their bisector

For all points left of ℓ, find the one that, together with p andq, defines a circle whose center is leftmost → pl

For all points right of ℓ, find the one that, together with pand q, defines a circle whose center is rightmost → pr

Decide if C(p,q,pl) or C(p,q,pr) or C(p,q) is the smallestenclosing circle






.. Two points known

p

q q

p p

qq

p

pl pr

C(p, q, pr)

C(p, q, pl)






.. Analysis: two points known

Smallest enclosing circle for n pointswith two points already known takesO(n) time, worst case

pq






.. Algorithm: one point known

Use a random order for p1, . . . ,pn; start withC1 = C(p,p1)

for j← 2 to n doIf pj in or on Cj−1 then Cj = Cj−1; otherwise, solvesmallest enclosing circle for p1, . . . ,pj−1 with two pointsknown (p and pj)

C ′j−1

pj

C ′j−1

pj

p p






.. Analysis: one point known

If only one point is known, we used randomized incrementalconstruction, so we need an expected time analysis

C ′j−1

pj

C ′j−1

pj

p p







Backwards analysis: Consider the situation after adding pj,so we have computed Cj

p Cjp Cj







The probability that the j-th addition was expensive is thesame as the probability that the smallest enclosing circlechanges (decreases in size) if we remove a random point fromthe j points

p Cjp Cj







This probability is 2/j in the left situation and 1/j in the rightsituation

p Cjp Cj







The expected time for the j-th addition of a point is

j−2j·Θ(1)+

2j·Θ(j) = O(1)

or

j−1j·Θ(1)+

1j·Θ(j) = O(1)

The expected running time of the algorithm for n points is:

Θ(n)+n

∑j=2

Θ(1) = Θ(n)







Smallest enclosing circle for n pointswith one point already known takesΘ(n) time, expected

p






.. Algorithm: smallest enclosing circle

Use a random order for p1, . . . ,pn; start withC2 = C(p1,p2)

for i← 3 to n doIf pi in or on Ci−1 then Ci = Ci−1; otherwise, solvesmallest enclosing circle for p1, . . . ,pi−1 with one pointknown (pi)

Ci−1

pi

Ci−1

pi






.. Analysis: smallest enclosing circle

For smallest enclosing circle, we used randomized incrementalconstruction, so we need an expected time analysis

Ci−1

pi

Ci−1

pi







Backwards analysis: Consider the situation after adding pi,so we have computed Ci

Ci Ci







The probability that the i-th addition was expensive is thesame as the probability that the smallest enclosing circlechanges (decreases in size) if we remove a random point fromthe i points

Ci Ci







This probability is 3/i in the left situation and 2/i in the rightsituation

Ci Ci







The expected time for the i-th addition of a point is

i−3i·Θ(1)+

3i·Θ(i) = O(1)

or

i−2i·Θ(1)+

2i·Θ(i) = O(1)

The expected running time of the algorithm for n points is:

Θ(n)+n

∑i=3

Θ(1) = Θ(n)






.. Result: smallest enclosing circle

Theorem The smallest enclosingcircle for n points in plane can becomputed in O(n) expected time





ConditionsDiameter and closest pairWidthMore examples

.. When does it work?

Randomized incremental construction algorithms of this sort(compute an ‘optimal’ thing) work if:

The test whether the next input object violates thecurrent optimum must be possible and fast

If the next input object violates the current optimum,finding the new optimum must be an easier problem thanthe general problem

The thing must already be defined by O(1) of the inputobjects

Ultimately: the analysis must work out






.. Diameter, closest pair

Diameter: Given a set of n points inthe plane, compute the two pointsfurthest apart

Closest pair: Given a set of n pointsin the plane, compute the two pointsclosest together






.. Width

Width: Given a set of n points in theplane, compute the smallest distancebetween two parallel lines thatcontain the points (narrowest strip)






.. Rotating callipers

The width can be computed using the rotating callipersalgorithm

Compute the convex hull

Find the highest and lowest point on it; they define twohorizontal lines that enclose the points

Rotate the lines together while proceeding along theconvex hull










































.. Width

Property: The width is alwaysdetermined by three points of the set

Theorem: The rotating callipersalgorithm determines the width (andthe diameter) in O(n logn) time






.. Width by RIC?

Property: The width is alwaysdetermined by three points of the set

We can maintain the two linesdefining the width to have a fast testfor violation






.. Adding a point

Question: How about adding apoint? If the new point lies insidethe narrowest strip we are fine, butwhat if it lies outside?






.. Adding a point






.. Adding a point






.. Width

A good reason to be very suspiciousof randomized incrementalconstruction as a working approachis non-uniqueness of a solution






.. Minimum bounding box

Question: Can we compute theminimum axis-parallel bounding boxby randomized incrementalconstruction?






.. Minimum bounding box

Yes, in O(n) expected time

. . . but a normal incrementalalgorithm does it in O(n) worst casetime






.. Lowest point in circles

Problem 1: Given n disks in theplane, can we compute the lowestpoint in their common intersectionefficiently by randomized incrementalconstruction?

Problem 2: Given n disks in theplane, can we compute the lowestpoint in their union efficiently byrandomized incrementalconstruction?






.. Red-blue separation

Problem: Given a set of n red andblue points in the plane, can wedecide efficiently if they have aseparating line?






.. One-guardable polygons

Problem: Given a simple polygonwith n vertices, can we decideefficiently if one guard is enough?






.. One-guardable polygons

It can easily happen that aproblem is an instance of linearprogramming

Then don’t devise a newalgorithm, just explain how totransform it, and show that it iscorrect (that your problem isreally solved that way)





HypercubeRoutingBit Fixing StrategyRandomized AlgorithmAnalysis

.. Hypercube Topology

Consider a parallel machine with n processors, labeled0, ...,n−1.The processors are linked together in a network, so that theycan communicate. However, not every pair of processors has alink between them, which means that some processors canonly communicate via other processors.Hypercube topology: the number of processors, n, is a powerof 2 and there is a (bidirectional) link between processors iand j if and only if the binary representation of i and j differ inexactly one bit.A hypercube with n = 8 processors has the following links:

(0,1),(0,2),(0,4),(1,3),(1,5),(2,3),

(2,6),(3,7),(4,5),(4,6),(5,7),(6,7)Randomized Algorithms Random Permutation





.. Rourting

Suppose each processor i wants to send a message mess(i) tosome other processor dest(i).Consider the case of permutation routing, where the setdest(1), · · · , dest(n) of destinations forms a permutation of1, · · · ,n.Sending these messages is done in rounds. In each round, aprocessor can send at most one message to each of itsneighbors.

Each processor i has a collection of logn buffers, one for eachoutgoing link. Denote the buffer for the link (i, j) by Bi(j).Buffer Bi(j) will store messages that processor i needs toforward to its neighbor j, but that have to wait because i alsoneeds to forward other messages to j.






.. Routing Algorithm

Algorithm RoutingStrategy(i)1. ▷ Send phase;2. for each outgoing link (i, j)3. do Select a message from Bi(j) and send it along link (i, j)4. ▷ Receive phase;5. for each outgoing link (i, j)6. do Store that message in Bi(j), where j is the next

processor on its route.






.. Routing Goal

Main question is how the routes are chosen so that # roundsneeded before every message has arrived at its destination isas small as possible.

To reach this goal, we would like the routes to be short andtoo many routes do not use the same links in the network(buffers of the congested links have to store many messages)

Now how does a processor i select from its buffers Bi(j) whichmessages to send in each round? Note that a processor i doesnot necessarily know the destinations dest(j) of the messagesof the other processors.

Hence, the route-planning algorithm should be oblivious: eachprocessor should determine route(mess(i)) based only on i anddest(i), not on any dest(j) for j = i.






.. Bit Fixing Strategy

Suppose we have already constructed some initial portion ofroute(mess(i)) and let j be the last processor on this initialportion.

To determine the next processor j′ on the route we look at thebits in the binary representation of dest(i) from left to right,and determine the leftmost bit b that is different from thecorresponding bit of the binary representation of j.We then take j′ such that its binary representation is the sameas that of j, except for the bit b.if n = 32 and processor 01101 wants to send a message to00110, then the route would be

01101→ 00101→ 00111→ 00110






.. Bit Fixing Strategy: Properties

The strategy is pretty simple and it is oblivious.

It has the advantage that a processor does not need to includethe whole route for mess(i) into the message header.

Based on dest(i) and its own processor number j, theprocessor j knows where to send the message mess(i) to.

Unfortunately, the number of rounds can be fairly large: thereare sets of destinations on which this routing strategy needsΩ(

√n/ logn) rounds.

One can prove that for any deterministic routing strategy thereis a set of destinations that will require Ω(

√n/ logn)rounds.






.. Bit Fixing Strategy: Bad Example

Consider the vertex i with all bits to be 1, and its neighborvertex j, who has a bit 0 at the first bit of the second half.

Now consider the permutation routing problem that for pointswho has logn/4 1s in the first logn/2 positions, the last bit ofthe first half is 1, and all 1s in the second logn/2 positions.

Those nodes want to send message to the nodes whose firstlogn/2 positions are 1 and second logn/2 positions haslogn/4 1s and the first bit of the second half is 0.

Thus we have

# of messages passing i-j=

(logn/2logn/4

)≈ 2logn/2√

logn=

√n

logn

This implies an Ω(√

n/ logn) complexity in the worst case.






.. Randomized Routing Algorithm

Algorithm RandomizedRoutingStrategy(i)1. ri← Random(0, n-1)2. Construct a route from processor i to processor ri using the

bit-fixing strategy, and construct a route from processor ri toprocessor dest(i) using the bit-fixing strategy. Letroute(mess(i)) be the concatenation of these two routes.






.. Analysis

For simplicity we will slightly change the algorithm so that itconsists of two phases: in the first phase all messages mess(i)will be routed from i to ri, and in the second phase allmessages will be routed from ri to dest(i).

Thus the messages wait at their intermediate destination untilall messages have arrived at their intermediate destination.

We will analyze the number of rounds needed in the firstphase. By symmetry the same analysis holds for the secondphase.

From now on, we let route(i) denote the route from i to ri, asprescribed by the bit-fixing strategy.






.. Analysis

Lemma. Let Si be the set of messages whose route uses at leastone of the links in route(i). Then the delay incurred by mess(i) inphase 1 is at most |Si|. In other words, mess(i) reaches itsintermediate destination ri after at most |route(i)|+ |Si| rounds,where |route(i)| is the length of the path route(i).






.. Analysis

Consider indicator random variables Xij

Xij =

1 if j = i and route(i) and route(j) share at least one link;0 otherwise.

Now fix some processor i. By the previous lemma, mess(i) willreach ri after at most

|route(i)|+ ∑0≤j<n

Xij

steps.






.. Analysis

E(|route(i)|+∑0≤j<n Xij) = E(|route(i)|)+E(∑0≤j<n Xij)

E(|route(i)|) = logn/2

We have to bound E(∑0≤j<n Xij)

Item 2 tells us the expected total number of links used over allroutes is (n/2) logn. On the other hand, the total number oflinks in the hypercube is n logn. By symmetry, all links in thehypercube have the same expected number of routes passingthrough them. Thus, this expected is equal to (n/2) logn

n logn = 12






.. Analysis

This implies that if we look at a single link on route(i) thenwe expect less than 1/2 other routes to use this link.

Since |route(i)| ≤ logn, we get

E( ∑0≤j<n

Xij)≤ |route(i)|/2≤ (logn)/2

This is good news: message mess(i) is expected to arrive atits intermediate destination ri within logn rounds. But we arenot there yet. We want to achieve that all messages arrive attheir destination quickly. To argue this, we need ahigh-probability bound on the delay of a message.






.. Analysis

To get such a bound we note that for fixed i, the randomvariables Xij are independent. This is true because theintermediate destinations rj are chosen independently.

Each Xij is 1 or 0 with a certain probability, and the Xij areindependent. So,we can apply the Chernoff bound.

Pr(X > (1+δ )µ)≤ (eδ

(1+δ )1+δ )µ

where X = ∑0≤j<n Xij and µ = E(X).






.. Analysis

Plugging in δ = e lognµ −1, and using that 0≤ µ ≤ (logn)/2

one can deduce that

Pr(X > e logn)≤ ee logn−µ µe logn

(e logn)e logn ≤ee logn((logn)/2)e logn

(e logn)e logn

= (1/2)e logn =1ne ≤

1n2

This together with the union inequlity implies that withprobability at least 1−1/n all messages arrive with a delay ofat most e logn steps.


Documents

Random Permutation and its Applications