Upload
clinton-morris
View
216
Download
0
Tags:
Embed Size (px)
Citation preview
1
More on Intractability
Knapsack Problem
Wednesday, August 5th
2
Outline For Today
1. Knapsack Pseudo-Poly-time DP 1
2. Knapsack Pseudo-Poly-time DP 2
3. Knapsack Greedy ½-Approximation Alg
4. Knapsack Fully Poly-time Approx. Scheme (FPTAS)
Recap: Classes P, NP
3
Given a computational problem C
P: C is ∈ P (polynomial-time solvable) if ∃an
algorithm solving C with O(nk) run-time, for some
constant k.
where n is the input length in bits
NP (or brute-force solvable): C ∈ NP if:
1. Correct solutions have polynomial length.
2. Claimed solutions are verifiable in poly-time.
Recap: NP-completeness
4
NP
C1
C*: NP-complete
C3 C4C5
C6
Ck
C2
C* is as hard as any NP problem!
Recap: History of NP-completeness
5
NP
C2
SAT
C3 C4C5
Ck
C6
C1
K1
K2… K20
K21
1971: Cook-Levin1972: Karp
Recap: History of NP-completeness
6
NP
C2
SAT
C3 C4C5
Ck
C6
C1
K1
K2… K20
K21
NP-complete
Since 1972
Recap: Showing C** is NP-complete
7
NP
C2
C*: NP-complete
C3 C4C5
Ck
C6
C*
*
If we can solve C** efficiently => we solve C*
efficiently
=> we solve all NP problems efficiently
C** is NP-
complete!
Recap: Two Very Important Skills
8
1. Recognizing NP-complete problems.
2. Learning The Right Methods of Approaching
NP-complete Problems
Recap: Approaching NP-complete Problems
9
Option 1: Focus to special-case inputs.
Option 2: Find an approximate answer.
Option 3: Be exponential time but better than
brute-force search.
Option 4: Heuristics: fast algorithms that are not
always correct (or even approximate)
Option 5: Mix some of these options
10
Outline For Today
1. Knapsack Pseudo-Poly-time DP 1
2. Knapsack Pseudo-Poly-time DP 2
3. Knapsack Greedy ½-Approximation Alg
4. Knapsack Fully Poly-time Approx. Scheme (FPTAS)
11
Knapsack (Sec 6.4, 11.8)
Input:
n items
values for items v1, …, vn ≥ 0
sizes for items w1, …, wn ≥ 0
knapsack capacity W ≥ 0
Output: subset S ⊆ 1, 2, …, n items s.t.Fact: Knapsack is NP-
complete.3SAT≤p KNAPSACK
SUBSET-SUM ≤p
KNAPSACK
12
Knapsack Examplev1 = 2.2
w1 = 1.5
v2 = 4w2 = 3
v4 = 3w4 = 4.6
W=7.8
v3 = 2
w3 = 3
13
Knapsack Examplev1 = 2.2
w1 = 1.5
v2 = 4w2 = 3
v4 = 3w4 = 4.6
W=7.8
v3 = 2
w3 = 3
v2 = 4w2 = 3
v3 = 2w3 = 3
v1 = 2.2
w1 = 1.5
OPT = 4+2+2.2=8.2
14
Restrict wi and W to be Integers
Input:
n items
values for items v1, …, vn ≥ 0
sizes for items w1, …, wn ≥ 0 & wi are **INTEGERS**
knapsack capacity W ≥ 0 & W is an **INTEGER**
Output: subset S ⊆ 1, 2, …, n items s.t.
Recap: Recipe of a DP Algorithms
15
1. Identify small # of subproblems
2. Quickly + correctly solve “larger” subproblems
given solutions to smaller ones
3. After solving all subproblems, can quickly
compute final solution
16
Knapsack DP Algorithm 1
Order the n items in arbitrary order: 1, 2, …, n.
Consider the optimal solution S*
A Claim that Doesn’t Require A Proof:
(1) n ∉ S* or (2) n ∈ S*
17
Case 1: n ∉ S*
Q: What can we assert about S* for items 1,
…, n-1?
A: S* is opt. for items 1, …, n-1 and capacity
W.
Proof: Assume ∃better S** w/ cap. W for 1, …,
n-1
S** is feasible for 1, …, n and better than
S*
Which would contradict S*’s optimality
Q.E.D.
18
Case 2: n ∈ S*
Q: What can we assert about S*-n for items
1, …, n-1?
A: S*-n is opt. for items 1, …, n-1 and cap.
W-wn.
Pf: Assume ∃better S** w/ cap ≤ W-wn for 1,
…, n-1
S** ∪n has capacity ≤ W
S** ∪n is feasible for 1, …, n and better
than S*
Which would contradict S*’s optimality
Q.E.D.
19
What Are The Subproblems?
K(i, c): opt. knapsack for the first i items and cap
c.
Q: How many subproblems are there?
A: n*W
K(i, c) =
max
K(i-1, c)
K(i-1, c - w_i) +
vi
Knapsack DP Algorithm 1 Pseudocode
procedure DP-Knapsack-1(n, W): Base Cases: A[0][i] = 0
for i = 1,2,…,n: for c = 1, …, W:
A[i][c] = maxA[i-1][c], A[i-1][c-wi]+vi
return A[n][W]20
K(i, c) =
max
K(i-1, c)
K(i-1, c - w_i)
Run-time
21
Runtime: O(nW)
Brute-Force Search: Ω(2n)
Observation: This is polynomial in n and W.
Q: Why does this not prove P=NP?
A: B/c we’re still exponential in input size.
Input
22
Input size: # bits (key strokes) to represent the
problem
n weights, values (n * (log of max weight and value))
capacity => log(W) bits.
Note: W is exponential in log(W)
w1 v1 w2 v2 … … wn vn W
Pseudo-polynomial Time Algorithm
23
An algorithm that’s polynomial in the
numeric values of the inputs but not the
# bits to represent it.
Ex: O(nW) is pseudo-polynomial
Interpretation: If we fix W to an integer
value
=> Knapsack is tractable
Called “Fixed-Parameter Tractable”
Problem
Summary of Knapsack DP Alg 1
24
1. Took Knapsack, which is NP-complete.
2. Restricted to inputs with integer wi and W
3. Got a pseudo-poly-time algorithm DP
algorithm (exponential but better than brute-
force search)
4. Further fixing W yields a full poly-time
algorithm
Recap: Approaching NP-complete Problems
25
Option 1: Focus to special-case inputs.
Option 2: Find an approximate answer.
Option 3: Be exponential time but better than
brute-force search.
Option 4: Heuristics: fast algorithms that are not
always correct (or even approximate)
Option 5: Mix some of these options
Important Note To Keep In Mind
26
When attacking NP-complete problems,
if you stick with always correct algorithms
(i.e. you don’t pick option 2 and try to
approximate)
you necessarily have to be exponential time.
However can get non-trivial speed-ups
if some aspect of the problem is
small.
27
Outline For Today
1. Knapsack Pseudo-Poly-time DP 1
2. Knapsack Pseudo-Poly-time DP 2
3. Knapsack Greedy ½-Approximation Alg
4. Knapsack Fully Poly-time Approx. Scheme (FPTAS)
28
Now Restrict vi to be Integers
Input:
n items
values for items v1, …, vn ≥ 0 & vi are **INTEGERS**
let v* = maxi vi , and V = v1 + v2 + … + vn ≤ nv*
sizes for items w1, …, wn ≥ 0
knapsack capacity W ≥ 0
Output: subset S ⊆ 1, 2, …, n items s.t.
29
A Different DP Algorithm
DP Algorithm 1 asked:
What is the maximum value we can pack into
at most X capacity given the first k items?
We can also ask:
What is the minimum capacity needed to
pack
at least Y value into the knapsack given the
first k items?
Subproblems For Approach 2
30
M(i, v): min capacity needed to pack value v from
the first i items.
Q: How many subproblems are there?
A: n*V≤n(nv*)=n2v*
Runtime is O(n2v*)
M(i, v) =
min
M(i-1,
v)M(i-1, v – v_i) + wi
Knapsack DP Algorithm 2 Pseudocode
procedure DP-Knapsack-2(n, V = v1 + … + vn):
Base Cases: A[0][0] = 0, A[0][j] = +∞ for i = 1,2,…,n:
for v = 1, …, V:A[i][v] = minA[i-1][v], A[i-1]
[v-vi]+wireturn max v s.t. A[n][v] ≤ W
31
M(i, v) =
min
M(i-1,
v)M(i-1, v - v_i) + wi
Knapsack DP Algorithm Output
32
1 2 3 … n
1 … … … … … … 3
2 … … … … … …
3 … … … … … …
… … … … … … …
… … … … … … …
… … … … … … …
… … … … … … …
V … … … … … …
Min capacity needed to pack a value of 1
Knapsack DP Algorithm Output
33
1 2 3 … n
1 … … … … … … 3
2 … … … … … … 3
3 … … … … … …
… … … … … … …
… … … … … … …
… … … … … … …
… … … … … … …
V … … … … … …
Min capacity needed to pack a value of 2
Knapsack DP Algorithm Output
34
1 2 3 … n
1 … … … … … … 3
2 … … … … … … 3
3 … … … … … … 5
… … … … … … … …
45 … … … … … … W-1
… … … … … … …
… … … … … … …
V … … … … … …
Min capacity needed to pack a value of 45
Knapsack DP Algorithm Output
35
1 2 3 … n
1 … … … … … … 3
2 … … … … … … 3
3 … … … … … … 5
… … … … … … … …
45 … … … … … … W-1
46 … … … … … … W+3
… … … … … … …
V … … … … … …
Min capacity needed to pack a value of 46
Knapsack DP Algorithm Output
36
1 2 3 … n
1 … … … … … … 3
2 … … … … … … 3
3 … … … … … … 5
… … … … … … … …
45 … … … … … … W-1
46 … … … … … … W+3
… … … … … … … …
V … … … … … …
Min capacity needed to pack a value of V
3 Exp-time Correct Knapsack Algorithms
37
Brute-Force Search
DP1 DP2
Ω(2n) O(nW) O(n2v*)
Takeaway 1: There is good & bad exponential
times
Takeaway 2: We can get non-trivial speed-
ups if some aspect of the problem is small.
38
Outline For Today
1. Knapsack Pseudo-Poly-time DP 1
2. Knapsack Pseudo-Poly-time DP 2
3. Knapsack Greedy ½-Approximation Alg
4. Knapsack Fully Poly-time Approx. Scheme (FPTAS)
39
Back To Original Knapsack
Input:
n items
values for items v1, …, vn ≥ 0
sizes for items w1, …, wn ≥ 0
knapsack capacity W ≥ 0
Output: subset S ⊆ 1, 2, …, n items s.t.
Option 2:
Now we’ll give up
correctness.
Recall Scheduling Problem Input: Each jobi has length li AND weight wi
l1, w1Job 1
l2, w2Job 2
ln,wnJob n
…
Output: A schedule of the jobs on a processor
s.t:
is minimum over all possible n! schedules.
weighted completion time of
job i
41
A Possible Greedy Approach
Similar to Greedy Weighted Scheduling (Lecture 5)
If weights are the same, put higher value items first
If values are the same, put lighter items first
Greedy Algorithm:
1. Combine vi and wi into a single score vi / wi:
2. Sort items in increasing combined score
Assume w.l.o.g.: v1/w1 ≥ v2/w2 ≥ … ≥ vn/wn
3. Pack until can’t pack anymore
42
Example 1v1 = 5w1 =
1
v2 = 4
w2 = 2
v3 = 3
w3 = 3
W=
5
43
Example 1
v1 = 5w1 =
1
v2 = 4
w2 = 2
v3 = 3
w3 = 3
W=
5
44
Example 1
v1 = 5w1 =
1
v2 = 4
w2 = 2
v3 = 3
w3 = 3
3rd item does not fit
Output: 4 + 5 = 9
Found the optimal
Q: What can go
wrong?
W=
5
45
Bad Examplev1 = 2w1 =
1
v2=100
w2=100
W=100
46
Bad Example
v1 = 2w1 =
1
v2=100
w2=100
Output: 2
Optimal is 100
Can be
arbitrarily far
from optimal.
W=100
47
Simple Fix Fixed-Greedy Algorithm:
1. Combine vi and wi into a single score vi / wi:
2. Sort items in increasing combined score
Assume w.l.o.g.: v1/w1 ≥ v2/w2 ≥ … ≥ vn/wn
3. Pack until can’t pack anymore
4. Return Y = max step 3, max-value item
(assume all items have weight ≤ W)
Claim: Y is a ½-approximation to OPT
Runtime: O(nlogn)
48
Thought Experiment
Assume greedy packed first k items
And only x% of item k+1 fits into our
knapsack
Suppose we can slice x% of k
Fill in the rest of out knapsack
And get x% of vk.
49
Example of Slicing
v1 = 2w1 =
1
v2=100
w2=100
W=100
Slice 99%
of item 2
50
Example of Slicing
v1 = 2w1 =
1
v2=1w2=1
W=100
v2=99
w2=99
Claim: This
“Fractional”
Solution is better
than OPT.
51
Proof Sketch (Fill in the Gaps As Exercise)
Same
Take arbitrary solution S
Assume they differ in l units
Greedy picks those l units
from highest value per
weight (plus might have
extra stuff)
value(l units of greedy) ≥
value(l units of S)
Greedy ≥ S
Greedy
Fractional
Any
Solution
S
Same
Different
Different
52
Proof of ½-Approximation
Assume greedy packs first k items and gets stuck.
Fixed-Greedy returns:
maxA=first k, or max-value item
≥ maxA=first k, B=value of k+1st item
Fractional-Greedy: A + fraction of B
maxA, B >= ½ (A+B)
Fixed-Greedy ≥ ½ Fractional-Greedy ≥ ½ OPT
Q.E.D
53
Outline For Today
1. Knapsack Pseudo-Poly-time DP 1
2. Knapsack Pseudo-Poly-time DP 2
3. Knapsack Greedy ½-Approximation Alg
4. Knapsack Fully Poly-time Approx. Scheme (FPTAS)
54
Back To Integer Values Input:
n items
values for items v1, …, vn ≥ 0 (**INTEGERS**)
let v* = maxi vi , and V = v1 + v2 + … + vn ≤
nv*
sizes for items w1, …, wn ≥ 0
knapsack capacity W ≥ 0
Output: subset S ⊆ 1, 2, …, n items s.t.
55
Very Ambitious Goal
KnapsackAlgorithm
I: (wi, vi, W)
accuracy ε (say 0.01)
≥ (1-ε)*OPT
(1-ε)-approx
**Catch: Run-time will grow as ε
decreases**Upshot
Instead of I solve incorrect input I’(wi, ṽi, W)
But solve I’ optimally with DP2
Argue opt for I’ is (1-ε) close to opt for I
56
Recall DP2
Pseudo poly-time DP Algorithm
Run-time: O(n2v*)
Performs badly only when values are very
large.Items Values
1 1005380100001
2 1421480174001
3 5801342740012
4 9925141920410
… …
Items Values
1 10053801
2 14214801
3 58013427
4 99251419
… …
Idea: Remove Low Order Bits.
57
Rounding Algorithm
Q1: How small should m be to satisfy ε
accuracy?
Q2: What is the running time of the algorithm
once we pick the the appropriate m?
procedure Knapsack-FPTAS(wi, vi, W, ε):
round each vi down to nearest mult. of
m=f(ε)
let ṽi = rounded(vi)/m,
return DP2(wi, ṽi, W)
58
Accuracy Analysis (1)
vixi=mṽi ≤ vixi=mṽi ≤ vi ≤ yi=m(ṽi + 1)
(1)(2)
Let S be our solution, and S* is optimal
solution
Hint: Weights were NOT perturbed.
Goal:Goal is to say we’re not that far from OPT.
59
Accuracy Analysis (2)
Goal:
by (1)
by (2)
xi’s
yi’s
60
Accuracy Analysis (3)
error term
This inequality is true for any m.
Q: What should m be so that we’re ε off?
A: Pick m s.t.
61
Accuracy Analysis (4)
Picking satisfies the inequality.
62
Run-time Analysis
∀elements i, we have:
DP2’s run-time was: O(n2v*)
When inputs are ṽi each ṽi ≤ n/ε
So run-time is: O(n3/ε)
63
Key Takeaway
We took an NP-complete problem.
We are approximating it to arbitrary
precision.
And in poly-time for each precision rate!
64
Next Week
More Algorithms For Intractability
&
What’s Beyond CS 161