Upload
others
View
5
Download
0
Embed Size (px)
Citation preview
Randomized Algorithms CS648
Lecture 18
Randomized Incremental Construction
(part II)
1
RANDOMIZED INCREMENTAL CONSTRUCTION
2
Randomized Incremental Construction
• Permute the elements of input randomly uniformly.
• Build the structure incrementally.
• Keep some data structure to perform 𝒊th iteration efficiently.
• Use Backward analysis to analyze the expected running time.
3
Randomized Incremental Construction
• Convex Hull of a set of points
• Trapezoidal decomposition of a set of segments.
• Convex polytope of a set of half-planes
• Smallest sphere enclosing a set of points.
• Linear programming in finite dimensions.
4
Trapezoidal decomposition
5
O((𝒏 + 𝒎) 𝐥𝐨𝐠 𝒏 ) time O(𝒎 + 𝒏 𝐥𝐨𝐠 𝒏 ) time
PROBLEM 3 CONVEX HULL OF POINTS
6
Convex hull of Points
Problem definition:
Given 𝒏 points in a plane, compute a convex polygon of smallest area that encloses all the points.
7
Convex hull of Points
Deterministic algorithm:
• O(𝒏 𝐥𝐨𝐠 𝒏) time algorithm.
• Many algorithms exist:
Grahams Scan, Jarvis’s march, divide and conquer,…
Randomized algorithm:
• O(𝒏 𝐥𝐨𝐠 𝒏) time algorithm.
• Based on Randomized Incremental Construction.
• Generalizable to higher dimensions.
8
Randomized algorithm for convex hull
Convex-hull-algorithm(𝑷) { Let <𝒑𝟏,𝒑𝟐,…,𝒑𝒏 > be a uniformly random permutation of 𝑷;
𝑯 triangle(𝒑𝟏,𝒑𝟐, 𝒑𝟑);
For 𝒊 = 𝟒 to 𝒏 do
insert 𝒑𝒊 and update 𝑯
return 𝑯;
}
9
A simple exercise from geometry
Exercise: Given a line L and two points p and q,
determine whether the points lie on the same/different sides of L.
10
L
p
q
q
𝒚 = 𝒎𝒙 + 𝒄
11
Conflict graph : a powerful data structure
𝒄𝟒
𝒄𝟏
𝒄𝟐
𝒄𝟑
𝒄𝟓
𝒄𝟔
𝒄𝟕
𝒄𝟖
cones 𝒏 − 𝒊 points 𝒄𝟐
𝒄𝟑
𝒄𝟒
𝒄𝟓
𝒄𝟔
𝒄𝟕
𝒄𝟖
𝒄𝟏
12
Before entering the for loop
13
𝒄𝟏
𝒄𝟐
𝒄𝟑
𝒄𝟏
𝒄𝟐
𝒄𝟑
𝒏 − 𝟑 points cones
INSERTING 𝒊TH POINT
14
𝒊th iteration
15
𝒄𝟏
𝒄𝟑
𝒄𝟒
𝒄𝟓
𝒄𝟔
𝒄𝟕
𝒄𝟖
𝒄𝟐
𝒊th iteration
16
𝒄𝟏
𝒄𝟐
𝒄𝟑
𝒄𝟒
𝒄𝟓
𝒄𝟔
𝒄𝟕
𝒄𝟖
𝒄𝟐
𝒄𝟑
𝒄𝟒
𝒄𝟓
𝒄𝟔
𝒄𝟕
𝒄𝟖
𝒏 − 𝒊 + 𝟏 points cones 𝒄𝟏
𝒊th iteration
17
𝒄𝟏
𝒄𝟐
𝒄𝟑
𝒄𝟒
𝒄𝟓
𝒄𝟔
𝒄𝟕
𝒄𝟖
𝒄𝟐
𝒄𝟑
𝒄𝟒
𝒄𝟓
𝒄𝟔
𝒄𝟕
𝒄𝟖
𝒏 − 𝒊 + 𝟏 points cones 𝒄𝟏
𝒊th iteration
18
𝒄𝟏
𝒄𝟐
𝒄𝟑
𝒄𝟒
𝒄𝟓
𝒄𝟔
𝒄𝟕
𝒄𝟖
𝒄𝟐
𝒄𝟑
𝒄𝟒
𝒄𝟓
𝒄𝟔
𝒄𝟕
𝒄𝟖
𝒏 − 𝒊 + 𝟏 points cones 𝒄𝟏
𝒊th iteration
19
𝒄𝟏
𝒄𝟐
𝒄𝟑
𝒄𝟒
𝒄𝟓
𝒄𝟔
𝒄𝟕
𝒄𝟖
𝒄𝟐
𝒄𝟑
𝒄𝟒
𝒄𝟓
𝒄𝟔
𝒄𝟕
𝒄𝟖
𝒏 − 𝒊 points cones 𝒄𝟏
𝒊th iteration
20
𝒄𝟏
𝒄𝟐
𝒄𝟑
𝒄𝟒
𝒄𝟓
𝒄𝟔
𝒄𝟕
𝒄𝟖
𝒄𝟐
𝒄𝟑
𝒄𝟒
𝒄𝟓
𝒄𝟔
𝒄𝟕
𝒄𝟖
𝒏 − 𝒊 points cones 𝒄𝟏
𝒊th iteration
21
𝒄𝟏
𝒄𝟐
𝒄𝟑
𝒄𝟒
𝒄𝟓
𝒄𝟔
𝒄𝟕
𝒄𝟖
𝒄𝟐
𝒄𝟑
𝒄𝟒
𝒄𝟓
𝒄𝟔
𝒄𝟕
𝒄𝟖
𝒏 − 𝒊 points cones 𝒄𝟏
𝒊th iteration
22
𝒄𝟏
𝒄𝟐
𝒄𝟑
𝒄𝟒
𝒄𝟓
𝒄𝟔
𝒄𝟕
𝒄𝟖
𝒄𝟐
𝒄𝟑
𝒄𝟒
𝒄𝟓
𝒄𝟔
𝒄𝟕
𝒄𝟖
𝒏 − 𝒊 points cones 𝒄𝟏
𝒊th iteration
23
𝒄𝟏
𝒄𝟐
𝒄𝟑
𝒄𝟒
𝒄𝟓
𝒄𝟔
𝒄𝟕
𝒄𝟖
𝒄𝟐
𝒄𝟑
𝒄𝟒
𝒄𝟓
𝒄𝟔
𝒄𝟕
𝒄𝟖
𝒏 − 𝒊 points cones 𝒄𝟏
𝒄′
𝒄"
𝒊th iteration
24
𝒄𝟏
𝒄′
𝒄"
𝒄𝟔
𝒄𝟕
𝒄𝟖
𝒄′
𝒄"
𝒄𝟔
𝒄𝟕
𝒄𝟖
𝒏 − 𝒊 points cones 𝒄𝟏
Running time of 𝒊th iteration
Running time of 𝒊th iteration is of the order of
• Number of edges of convex hull that are destroyed
• Number of new edges of convex hull that are created
• Number of points in the two adjacent cones that get created
Question: What is the max. number of new edges created in an iteration ?
Answer: 2
Number of edges created during the algorithm = O(𝒏)
Since every edge destroyed was once created, so
Total number of edges destroyed < Total number of edges created
25
Total time for 𝒏 iterations = O(𝒏)
𝑿𝒊
Backward analysis of 𝒊th iteration
𝒂 : a subset of 𝒊 points from 𝑷.
𝓔𝒂: first 𝒊 points of 𝑷 are some permutation of 𝒂
𝐄,𝑿𝒊 𝓔𝒂 = ??
26
Convex hull at the end of 𝒊th iteration
Backward analysis of 𝒊th iteration
𝒂 : a subset of 𝒊 points from 𝑷.
𝓔𝒂: first 𝒊 points of 𝑷 are some permutation of 𝒂
𝐄,𝑿𝒊 𝓔𝒂 = ??
27
𝒄𝟏 𝒄𝟐
𝒄𝟑
𝒄𝟒
𝒄𝟓
𝒄𝟔
𝒄𝟕
𝒄𝟖 𝒑𝟏
𝒑𝟐
𝒑𝟑
𝒑𝟒
𝒑𝟓
𝒑𝟔
𝒑𝟕
𝒑𝟖
Backward analysis of 𝒊th iteration
𝒂 : a subset of 𝒊 points from 𝑷.
𝓔𝒂: first 𝒊 points of 𝑷 are some permutation of 𝒂
𝐄,𝑿𝒊 𝓔𝒂 = ??
28
𝒄𝟏 𝒄𝟐
𝒄𝟑
𝒄𝟒
𝒄𝟓
𝒄𝟔
𝒄𝟕
𝒄𝟖 𝒑𝟏
𝒑𝟐
𝒑𝟑
𝒑𝟒
𝒑𝟓
𝒑𝟔
𝒑𝟕
𝒑𝟖
𝒏𝟏 + 𝒏𝟐
𝒏𝟐 + 𝒏𝟑
𝒏𝟑 + 𝒏𝟒
𝒏𝟒 + 𝒏𝟓
Backward analysis of 𝒊th iteration
𝒂 : a subset of 𝒊 points from 𝑷.
𝓔𝒂: first 𝒊 points of 𝑷 are some permutation of 𝒂
𝐄,𝑿𝒊 𝓔𝒂 = ??
29
𝒄𝟏 𝒄𝟐
𝒄𝟑
𝒄𝟒
𝒄𝟓
𝒄𝟔
𝒄𝟕
𝒄𝟖 𝒑𝟏
𝒑𝟐
𝒑𝟑
𝒑𝟒
𝒑𝟓
𝒑𝟔
𝒑𝟕
𝒑𝟖
𝒏𝟏 + 𝒏𝟐
𝒏𝟐 + 𝒏𝟑
𝒏𝟑 + 𝒏𝟒
𝒏𝟒 + 𝒏𝟓
𝒏𝟕 + 𝒏𝟖
𝒏𝟖 + 𝒏𝟏
𝟏
𝒊
𝟏
𝒊
𝟏
𝒊
𝟏
𝒊
𝟏
𝒊
𝟏
𝒊
+
+
+
+
+
Backward analysis of 𝒊th iteration
𝒂 : a subset of 𝒊 points from 𝑷.
𝓔𝒂: first 𝒊 points of 𝑷 are some permutation of 𝒂
𝐄,𝑿𝒊 𝓔𝒂 = ??
Calculating 𝐄,𝑿𝒊- :
Let 𝑺𝒊 be the set of all subsets of
𝑷 of size 𝒊.
𝐄,𝑿𝒊- =
= 𝟐(𝒏−𝒊)
𝒊∙ 𝐏(𝓔𝒂) 𝒂∈𝑺𝒊
30
𝒄𝟏 𝒄𝟐
𝒄𝟑
𝒄𝟒
𝒄𝟓
𝒄𝟔
𝒄𝟕
𝒄𝟖 𝒑𝟏
𝒑𝟐
𝒑𝟑
𝒑𝟒
𝒑𝟓
𝒑𝟔
𝒑𝟕
𝒑𝟖 𝟐(𝒏 − 𝒊)
𝒊
= 𝟐(𝒏−𝒊)
𝒊
𝐄,𝑿𝒊 𝓔𝒂 ∙ 𝐏(𝓔𝒂)
𝒂∈𝑺𝒊
Running time of the algorithm
Expected running time of ith iteration
= 𝐄,𝑿𝒊- + O(1)
= O(𝒏−𝒊
𝒊)
Expected running time of the algorithm = O(𝒏 𝐥𝐨𝐠 𝒏 )
Theorem:
There is an O(𝒏 𝐥𝐨𝐠 𝒏 ) time Las Vegas algorithm for computing convex hull.
31
USING BACKWARD ANALYSIS FOR MISCELLANEOUS APPLICATIONS
32
PROBLEM 1 SMALLEST ENCLOSING CIRCLE
33
Smallest Enclosing Circle
34
Smallest Enclosing Circle
Question:
Suppose we sample 𝒌 points randomly uniformly from a set of 𝒏 points,
what is the expected no. of points that remain outside the smallest circle
enclosing the sample?
35
For 𝒌= 𝒏
𝟐, the answer is < 𝟏𝟎
PROBLEM 2 SMALLEST LENGTH INTERVAL
36
0 1
Sampling points from a unit interval
Question: Suppose we select 𝒌 points randomly uniformly from interval [0,1],
what is expected length of the smallest sub-interval ?
• for 𝒌 = 1, it is ??
• for 𝒌 = 2, it is ??
• General solution : ??
This bound can be derived using two methods.
• relationship between uniform distribution and exponential distribution.
• Backward analysis.
𝟏
𝟒
𝟏
𝟗
𝟏
𝒌 + 𝟏 𝟐
PROBLEM 3 MINIMUM SPANNING TREE
38
Minimum spanning tree
39
b
a c
d
h x
y
u
v
18
7
1
19
22
10
3 12
3
15
11
5
16
17
13
4 2
9
6
Minimum spanning tree
40
b
a c
d
h x
y
u
v
18
7
1
19
22
10
3 12
3
15
11
5
16
17
13
4 2
9
6
Light Edge
Definition: Let 𝒂 ⊂ 𝑬.
An edge 𝒆 ∈ 𝑬\𝒂 is said to be light with respect to 𝒂 if
Question:
If 𝒂 ⊂𝑟 𝑬 and |𝒂|= 𝒌,
how many edges from 𝑬\𝒂 are light with respect to 𝒂 on expectation ?
Answer: ??
41
b
a c
d
h x
y
u
v
18
7
1
19
22
10
3 12
3
15
11
5
16
17
13
4 2
9
6
MST(*𝒆+ ∪ 𝒂) ≠ MST(𝒂)
<𝒏
𝒌(𝒎 − 𝒌)
Smallest Enclosing Circle Let 𝑪 be smallest enclosing circle for set 𝑷 of point.
Facts from high school geometry
• 𝑪 is defined by 2 or 3 points lying on its boundary.
42
Smallest Enclosing Circle Let 𝑪 be smallest enclosing circle for set 𝑷 of point.
Facts from high school geometry
• 𝑪 is defined by 2 or 3 points lying on its boundary.
43
Not the smallest enclosing circle Since the defining points form an obtuse triangle.
Smallest Enclosing Circle Let 𝑪 be smallest enclosing circle for set 𝑷 of point.
Facts from high school geometry
• 𝑪 is defined by 2 or 3 points lying on its boundary.
44
This is the smallest enclosing Circle since the defining points form an acute angle triangle.
Smallest Enclosing Circle Let 𝑪 be smallest enclosing circle for set 𝑷 of point.
Facts from high school geometry
• 𝑪 is defined by 2 or 3 points lying on its boundary.
Lemma 1: If 𝒑 lies outside 𝑪,
𝒑 must be one of the defining point of smallest enclosing circle of 𝑷 ∪ *𝒑+ 45
Smallest Enclosing Circle Randomized Incremental Construction
Smallest-Enclosing-Circle(𝑷) { Let <𝒑𝟏,𝒑𝟐,…,𝒑𝒏 > be a uniformly random permutation of 𝑷;
𝑪 Circle(𝒑𝟏,𝒑𝟐);
For 𝒊 = 𝟑 to 𝒏 do
if ( 𝒑𝒊 lies outside 𝑪 ) update 𝑪;
return 𝑯;
}
Question: How many times will 𝑪 be updated ?
𝑿𝒊 = 𝟏 if 𝑪 is udated in ith iteration𝟎 otherwise
46
?
Smallest Enclosing Circle Question: What is P[𝑿𝒊 = 𝟏] ?
47
Smallest Enclosing Circle at the end of 𝒊th iteration
Smallest Enclosing Circle Question: What is P[𝑿𝒊 = 𝟏] ?
Use it to solve the given problem.
48
≤𝟑
𝒊 Using Lemma 1
PROBLEM 3 MINIMUM SPANNING TREE
49
Minimum spanning tree
50
b
a c
d
h x
y
u
v
18
7
1
19
22
10
3 12
3
15
11
5
16
17
13
4 2
9
6
Minimum spanning tree
Algorithms:
• Prim’s algorithm
• Kruskal’s algorithm
• Boruvka’s algorithm
51
b
a c
d
h x
y
u
v
18
7
1
19
22
10
3 12
3
15
11
5
16
17
13
4 2
9
6
Less known but it is the first algorithm for MST
Minimum spanning tree
𝑮 = (𝑽, 𝑬) : undirected graph with weights on edges
𝒏 = |𝑽|,
𝒎 = |𝑬|.
Deterministic algorithms:
Prim’s algorithm
1. O((𝒎 + 𝒏) log 𝒏) using Binary heap
2. O(𝒎 + 𝒏 log 𝒏) using Fibonacci heap
Best deterministic algorithm:
O(𝒎 + 𝒏 𝐥𝐨𝐠∗ 𝒏) bound
– Too complicated to design and analyze
– Fails to beat Prim’s algorithm using Binary heap
52
Minimum spanning tree
𝑮 = (𝑽, 𝑬) : undirected graph with weights on edges
𝒏 = |𝑽|,
𝒎 = |𝑬|.
Randomized algorithm:
Karger-Klein-Tarjan’s algorithm [1995]
1. Las Vegas algorithm
2. O(𝒎 + 𝒏) expected time
This algorithm uses
• Random sampling
• MST verification algorithm
• Boruvka’s algorithm
• Elementary data structure 53
We will discuss this milestone result in the next class.