Parameterized Complexity Hans Bodlaender IPA Basiscursus Algoritmiek

Parameterized Complexity

Hans Bodlaender

IPA Basiscursus Algoritmiek

Parameterized Complexity2

Today

• First session:– Parameterized complexity: what is it about– FPT algorithms: branching– FPT algorithms: color coding– Hardness proofs

• Second session:– FPT algorithms: Kernelisation – FPT algorithms: Iterative compression

• Third session:– Treewidth

1Introduction

Parameterized complexity:

What is it about?


Fixed Parameter Complexity

• Many problems have a parameter

• Analysis what happens to problem when some parameter is small


Motivation

• In many applications, some number can be assumed to be small– Time of algorithm can be exponential in this

small number, but should be polynomial in usual size of problem

– Or something is fixed


Parameterized problem

• Given: Graph G, integer k, …

• Parameter: k

• Question: Does G have a ??? of size at least (at most) k?– Examples: vertex cover, independent

set, coloring, …


Examples of parameterized problems

(1)Graph Coloring

Given: Graph G, integer k

Parameter: k

Question: Is there a vertex coloring of G with k colors? (I.e., c: V {1, 2, …, k} with for all {v,w} E: c(v) c(w)?)

• NP-complete, even when k=3.


Clique

• Subset W of the vertices in a graph, such that each pair has an edge



(2)Clique


Parameter: k

Question: Is there a clique in G of size at least k?

• Solvable in O(nk) time with simple algorithm. Complicated algorithm gives O(n2k/3). Seems to require (nf(k)) time…


Vertex cover

• Set of vertices W V with for all {x,y} E: x W or y W.

• Vertex Cover problem:– Given G, find vertex cover of minimum size



(3)Vertex cover


Parameter: k

Question: Is there a vertex cover of G of size at most k?

• Solvable in O(2k (n+m)) time


Three types of complexity

• When the parameter is fixed– Still NP-complete (k-coloring, take k=3)– O(f(k) nc) (nf(k))


Fixed parameter complexity theory

• To distinguish between behavior:O( f(k) * nc)( nf(k))

• Proposed by Downey and Fellows.


Parameterized problems

• Instances of the form (x,k)– I.e., we have a second parameter

• Decision problem (subset of {0,1}* x N )


Fixed parameter tractable problems

• FPT is the class of problems with an algorithm that solves instances of the form (x,k) in time p(|x|)*f(k), for polynomial p and some function f.


Hard problems

• Complexity classes– W[1] W[2] … W[i] … W[P]– Defined in terms of Boolean circuits– Problems hard for W[1] or larger class are

assumed not to be in FPT• Compare with P / NP


Examples of hard problems

• Clique and Independent Set are W[1]-complete

• Dominating Set is W[2]-complete• Version of Satisfiability is W[1]-complete

Given: set of clauses, k

Parameter: k

Question: can we set (at most) k variables to true, and al others to false, and make all clauses true?


So what is parameterized complexity about?

• Given a parameterized problem• Establish that it is in FPT

– And then design an algorithm that is as fast as possible• Or show that it is hard for W[1] or “higher”

– Try to find a polynomial time algorithm for fixed parameter

• Or even show that it is NP-complete for fixed parameters– Solve it with different techniques (exact or

approximation)


FPT techniques

• Branching algorithms– Bounded search trees– Greedy localization– Color coding

• Kernelisation (local rules, global rules)• Induction

– Iterative compression– Extremal method

• Win/Win– Well quasi ordering– Structures (e.g., treewidth)

FollowingSloper/Telle

taxonomy

FollowingSloper/Telle

taxonomy


Disclaimer

• Here: focus on techniques, not on best known results

• Time arguments often too crude• Also: parameterized problems are considered with

something else as parameter– Pair of integers, string, …

– Assumed to be fixed

– Can be mapped to integer

2

FPT Algorithmic techniques:

Branching


Branching

• More or less classic technique of search tree

• Counting argument helps to keep size of search tree small

• At each step, we recursively create a number of subproblems: answer is yes, if and only if at least one subproblem has yes as answer


Vertex cover

• First example: vertex cover

• Select an edge {v,w}. Note that a solution includes v or it includes w

• If we include v, we can ignore all edges incident to v in subproblems

v w


Branching algorithm for Vertex Cover

• Recursive procedure VC(Graph G, int k)• VC(G=(V,E), k)

– If G has no edges, then return true– If k == 0, then return false– Select an edge {v,w} E

– Compute G’ = G [V – v] (remove v and incident edges)

– Compute G” = G [V – w] (remove w and incident edges)

– Return VC(G’,k – 1) or VC(G”,k – 1)


Analysis of algorithm

• Correctness– Either v or w must belong to an optimal VC

• Time analysis– At most 2.2k recursive calls– Each recursive call costs O(n+m) time– O(2k (n+m)) time: FPT


Independent Set on planar graphs

Given: a planar graph G=(V,E), integer kParameter: kQuestion: Does G have an independent set with at

least k vertices, i.e., a set W of size at least k with for all v, w V: {v,w}

• NP-complete• Easy to see that it is FPT by kernelisation…• Here: O(6k n) algorithm


The red vertices form an independent set


Branching

• Each planar graph has a vertex of degree at most 5• Take vertex v of minimum degree, say with

neighbors w1, …, wr, r at most 5• A maximum size independent set contains v or one

of its neighbors– Selecting a vertex is equivalent to removing it and its

neighbors and decreasing k by one

• Create at most 6 subproblems, one for each x {v, w1, …, wr}. In each, we set k = k – 1, and remove x and its neighbors


v

w2

w1


Closest string

Given: k strings s1, …,sk each of length L, integer d

Parameter: dQuestion: is there a string s with Hamming

distance at most d to each of s1, …,sk

• Application in molecular biology• Here: FPT algorithm• (Gramm and Niedermeier, 2002)


Subproblems

• Subproblems have form– Candidate string s– Additional integer parameter r– We look for a solution to original problem, with

additional condition:• Hamming distance at most r to s

• Start with s = s1 and r=d (= original problem)


Branching step

• Choose an sj with Hamming distance > d to s

• If Hamming distance of sj to s is larger than d+r: NO

• For all positions i where sj differs from s– Solve subproblem with

• s changed at position i to value sj (i)

• r = r – 1

• Note: we find a solution, if and only one of these subproblems has a solution


Example

• Strings 01113, 02223, 01221, d=3– First position in solution will be a 0– First subproblem (01113, 2)– Creates three subproblems

• (02113, 1)

• (01213, 1)

• (01123, 1)


Time analysis

• Recursion depth d• At each level, we branch at most at d + r 2d

positions• So, number of recursive steps at most d2d+1

• Each step can be done in polynomial time: O(kdL) • Total time is O(d2d+1 . kdL)• Speed up possible by more clever branching and

by kernelisation


More clever branching

• Choose an sj with Hamming distance > d to s• If Hamming distance of si to s is larger than d+r:

NO• Choose arbitrarily d+1 positions where sj differs

from s– Solve subproblem with

• s changed at position i to value sj (j)• r = r – 1

• Note: still correct, and running time can be made O(kL + kd dd)


Technique

• Try to find a branching rule that– Decreases the parameter– Splits in a bounded number of subcases

• YES, if and only if YES in at least one subcase

3

FPT Algorithmic techniques

Color coding


Color coding

• Interesting algorithmic technique to give fast FPT algorithms

• As example: • Long Path

Given: Graph G=(V,E), integer kParameter: kQuestion: is there a simple path in G with at least

k vertices?


Problem on colored graphs

• Given: graph G=(V,E), for each vertex v a color in {1,2, … , k}

• Question: Is there a simple path in G with k vertices of different colors?

– Note: vertices with the same colors may be adjacent.

• Can be solved in O(2k (nm)) time using dynamic programming

• Used as subroutine…


DP

• Tabulate:– (S,v): S is a set of colors, v a vertex, such that

there is a path using vertices with colors in S, and ending in v

– Using Dynamic Programming, we can tabulate all such pairs, and thus decide if the requested path exists


A randomized variant

• For each vertex v, guess a color in {1, 2, …, k}• Check if there is a path of length k with only

vertices with different colors• Note:

– If there is a path of length k, we find one with positive chance ( 2k /k!)

– We can do this check in O(2k nm) time

– Repeat the check many times to get good probability for finding the path


Derandomization

• Instead of randomly selecting a coloring, check all colorings from a k-perfect family of hash functions

• A k-perfect family of hash functions on a set V is a collection H of functions V {1, …,k}, such that for each set W V, |W|=k, there exists an h H with h(W) = {1, …,k}

• Algorithm:– Generate a k-perfect family of hash functions – For each function in the set, check for the properly

colored path of length k


Existence and generation of k-perfect families of hash

functions• Alon et al. (1995): collection with O(ck log n)

functions exist and can be deterministically generated

• Hence: O((2c)k nm log n) algorithm for Longest Path– Generate all elements of k-perfect family of hash

functions, and for each, solve the colored version with dynamic programming

• Refined techniques exist

4

Hardness proofs


Remember Cook’s theorem

• NP-completeness helps to distinguish between decision problems for which we have a polynomial time algorithm, and those for which we expect no such algorithm exists

• NP-hard; NP-completeness; reductions• Cook’s theorem: `first’ NP-complete problem;

used to prove others to be NP-complete• Similar theory for parameterized problems by

Downey and Fellows


Classes

• FPT W[1] W[2] W[3] … W[i] … W[P]

• Theoretical reasons to believe that hierarchy is strict


Parameterized m-reduction

• Let L, L’ be parameterized problems.• A standard parameterized m-reduction transforms

an input (I,k) of L to an input (f(I,k), g(k)) of L’– (I,k) L if and only if (f(I,k), g(k)) L’ – f uses time p(|I|)* h(k) for a polynomial p, and some

function h

• Note: time may be exponential or worse in k• Note: the parameter only depends on parameter,

not on rest of the input


A Complete Problem

• Classes W[1], … are defined in terms of circuits (definition skipped here)

• Short Turing Machine AcceptanceGiven: A non-deterministic Turing machine M, input x,

integer kParameter: kQuestion: Does M accept x in a computation with at most

k steps?• Short Turing Machine Acceptance is

W[1]-complete (compare Cook)• Note: easily solvable in O(nk+c) time


More complete problems for W[1]

• Weighted q-CNF SatisfiabilityGiven: Boolean formula in CNF, such that each

clause has at most q literals, integer k

Parameter: k

Question: Can we satisfy the formula by making at most k literals true?

• For each fixed q > 1, Weighted q-CNF Satisfiability is complete for W[1].


Hard problems

• Independent Set, Clique: W[1]-complete

• Dominating Set: W[2]-complete

• Longest Common Subsequence III: W[1]-complete (complex reduction to Clique)Given: set of k strings S1, …, Sk, integer m

Parameter: k, m

Question: is there a string S of length m that is a subsequence of each string Si, 1 i k?


Example of reduction

Precedence constrained K-processor schedulingInstance: set of tasks T, each taking 1 unit of time, partial

order < on tasks, deadline D, number of processors K

Parameter: K

Question: can we carry out the tasks on K processors, such that

• If task1 < task2, then task1 is carried out before task2

• At most one task per time step per processor

• All tasks finished at most at time D


Transform from Dominating Set

• Let G=(V,E), k be instance of DS

• Write n = |V|, c = n2, D = knc + 2n.

• Take the following tasks and precedences:

• Floor: D tasks in “series”:

1 2 3 … … … … D-2 D-1 D


Floor gadgets

• For all j of the form j = n-1+ ac + bn (0 a < kn, 1 b n), take a task that must happen on time j (parallel to the jth floor vertex)

1 2 … j-1 j j+1 … … D-1 D


Selector paths

• We take k paths of length D-n+1• Each models a vertex from the dominating set• To some vertices on the path, we also take parallel

vertices:– If {vi, vj} E, and i j, then place a vertex parallel to

the n-1+ac+in-jth vertex for all a, 0 a < kn

1 … … … … … … … … D-n-1…


Lemma and Theorem

• Lemma: We can schedule this set of tasks with deadline D and 2k processors, if and only if G has a dominating set of size at most k

• Theorem: Precedence constrained k-processor scheduling is W[2]-hard

• Note: size of instance must depend in polynomial way on size of G (and hence on k < |V|)

• It is allowed to use transformations where new parameter is exponential in old parameter


Conclusions

• Fixed parameter proofs: method of showing that a problem probably has no FPT-algorithms

• Often complicated proof • But not always

5

FPT Algorithmic techniques

Kernelisation


Kernelisation

• Preprocessing rules reduce starting instance to one of size f(k)– Should work in polynomial time

• Then use any algorithm to solve problem on kernel

• Time will be p(n) + g(f(k))


Simple example:Independent Set on planar

graphs• A planar graph has an independent set of at

least n/4 vertices– 4-color the graph, and take the set of vertices

with the most frequent color

• Kernelisation algorithm– If n 4k, then return YES– Else return G


Vertex cover: observations that helps for kernelisation• If v has degree at least k+1, then v belongs

to each vertex cover in G of size at most k.– If v is not in the vertex cover, then all its

neighbors are in the vertex cover.

• If all vertices have degree at most k, then a vertex cover has at least m/k vertices.– (m=|E|). Any vertex covers at most k edges.


Kernelisation for Vertex Cover

H = G; ( S = ; )

While there is a vertex v in H of degree at least k+1 doRemove v and its incident edges from H

k = k – 1; ( S = S + v ;)

If k < 0 then return falseIf H has at least k2+1 edges, then return falseSolve vertex cover on (H,k) with some algorithm


Time

• Kernelisation step can be done in O(n+m) time

• After kernelisation, we must solve the problem on a graph with at most k2 edges, e.g., with branching this gives:– O( n + m + 2k k2) time– O( kn + 2k k2) time can be obtained by noting

that there is no solution when m > kn.


Maximum Satisfiability

Given: Boolean formula in conjunctive normal form; integer k

Parameter: k

Question: Is there a truth assignment that satisfies at least k clauses?

• Denote: number of clauses: C


Reducing the number of clauses

• If C 2k, then answer is YES– Look at arbitrary truth assignment, and the

mirror truth assignment where we flip each value

– Each clause is satisfied in one of these two assignment

– So, one assignment satisfies at least half of all clauses


Bounding number of long clauses

• Long clause: has at least k literals• Short clause: has at most k-1 literals• Let L be number of long clauses• If L k: answer is YES

– Select in each of the first k long clause a literal, whose complement is not yet selected

– Set these all to true– k long clauses are satisfied


Reducing to only short clauses

• If less than k long clauses– Make new instance, with only the short clauses and k

set to k-L

– There is a truth assignment that satisfies at least k-L short clauses, if and only if there is a truth assignment that satisfies at least k clauses

• =>: choose for each satisfied short clause a variable that makes the clause true. We may change all other variables, and can choose for each long clause another variable that makes it true

• <=: trivial


An O(k2) kernel for Maximum Satisfiability

• If at least 2k clauses: return YES

• If at least k long clauses: return YES

• Else: remove all L long clauses, and set k=k-L


Formal definition of kernelisation

• Let P be a parameterized problem. (Each input of the form (I,k).) A reduction to a problem kernel is an algorithm, that transforms A inputs of P to inputs of P, such that– (I,k)) P , if and only if A(I,k) P for all (I,k)– If A(I,k) = (I’,k’), then k’ f(k), and |I’| g(k)

for some functions f, g– A uses time, polynomial in |I| and k


Kernelisation implies FPT and vice versa

• If P has a reduction to a problem kernel, and is decidable, then P is in FPT:– Use transformation– Solve remaining instance– Uses polynomial time, plus T(g(|I|)) time

• If P is in FPT, then P has a (perhaps trivial) reduction to a problem kernel– Suppose we have an f(k) nc algorithm– If |I| > f(k), solve problem exactly: this is O(nc+1) time

• Formality: take small yes or no-instance afterwards– Otherwise, take original instance as kernel


Improved kernelisation for Vertex Cover

• Nemhauser/Trotter: kernel for VC of size at most 2k

• ILP formulation of Vertex Cover:– Minimize v Vxv

• For all {v,w} E: xv + xw 1

• For all v V: xv {0,1}


Relaxation

• Solve relaxation:– Minimize v Vxv

• For all {v,w} E: xv + xw 1

• For all v V: 0 xv 1

• Define– C0 = {v V| xv

– V0 = {v V| xv

– I0 = {v V| xv


Theorem and reduction

• Theorem: There is a minimum size vertex cover S with C0 S and S I0 = .

• Reduction rule: remove all vertices in C0 and I0 and set k = k - | C0 |.– Safe

• New, equivalent instance: (G[V0], k - | C0 |)• If |V0| > 2k, relaxation, and hence also ILP has

optimal solution > k, answer NO.• We have a kernel with at most 2k vertices.


Example to explain techniques

• Convex recoloring

• Techniques:– Annotation– Rules that either

• Decide

• Make the instance smaller

• Improve structural insight


Connected tree coloring

• Tree• Each vertex has a

color• Coloring is connected,

if each set of vertices with the same color is connected


Connected recoloring

• Given: vertex-colored tree T

• Task: give some vertices a different color, such that we obtain a connected coloring.– With as few as possible

recolored vertices


Two recolored vertices


A problem on evolution trees

• Tree models evolutionary ancestry of species• Colors model values for some attributes• Often: only once evolutionary change to specific

value is made– Set of species with a specific value forms connected

part of tree

• Correct data: connected coloring of tree• Incorrect data: how to change as few as possible

values to obtain consistent tree?


Problem definition

Convex tree recoloringGiven: Tree T=(V,E), color c(v) for each vertex v,

integer k

Parameter: k

Question: can we change the color of at most k vertices such that for each color, all vertices with this color form a subtree?


Results

• For this problem: B et al. O(n2) size kernel

• Here: polynomial kernel– Reducing number of vertices per color– Reducing number of colors


• Trick: annotation: some vertices get a fixed color• Annotation: solving a “different” (generalized)

problem, with some additional conditions for objects in problem

• At end: undoing annotation• Simple example: if v is incident to k+2 vertices

with color C, then v must have color C

Fixing colors

v

Fixing colors improves structural

insight

Fixing colors improves structural

insight


Color fixing rule

• Suppose the trees of T-v have a1 a2 … ar vertices with color c, and a1 + … + ar-1 k+1.

• Then in any solution: v gets color c.– Otherwise we must recolor

the vertices with color c in all but one of the subtrees of T-v: at least k+1 recolorings.

• Give v fixed color c; possibly decrease k by 1

v k=4


Analysis

• If there are at least 2k + 3 vertices with color C, then we can fix a vertex with color C


One fixed color vertex per color

• If v and w have the same color C, both fixed, then– Fix the color of all vertices on the path from v to w to

C, possibly decreasing k, or rejecting if there is a vertex with a different fixed color on the path

• If v and w are adjacent, and have the same fixed color C, then contract the edge {v,w}

• After these rules: each color C has at most one vertex with fixed color C


Subtrees for a fixed color vertex

• Suppose v has a fixed color C. Look at the subtrees T-v

• If one of these has at least 2k+3 vertices with color C, we can fix a vertex with color C in this subtree, and reduce T

• So, to bound the number of vertices with color C, we should make sure that v has small degree


Getting rid of uninteresting subtrees

• Uninteresting subtree: No color in the subtree is broken, except perhaps C, and the vertices with color C are connected and incident to v, v has fixed color C

• Rule: remove all vertices in an uninteresting subtree

v

Red, yellow, green donot appear elsewhere in T


Bounding degree of fixed color vertices

• If v has degree at least 2k+1, a fixed color, and each subtree of T-v is interesting, then return NO– Each color change can “help” at most 2 subtrees of T-v

• Bounds number of vertices per color to O(k2) after reductions– If more than 2k+2 vertices, we can fix a vertex with

color C– One fixed color C vertex– Each subtree has at most 2k+2 vertices with color C– At most 2k subtrees in T-v


Types of colors

• Broken colors: the vertices with color C are not connected

• Unbroken colors• Rule: If there are more than 2k broken colors,

return NO– Each color change can fix at most 2 colors: the old and

new color of the recolored vertex

• Consequence: there are at most 2k broken colors


Broken colors• A color is broken, if it is not connected in the

given coloring.• If there are more than 2k broken colors, there is no

solution.– Each recoloring can correct at most two colors (the old

and the new color of the vertex) but not more.

– So: Assume there are at most 2k broken colors.

v

v


Bounding number of colors

• A color is interesting, if it is– Broken, or– Appears on a path of length at most k between two

vertices with the same broken color

• We never need to recolor vertices that are not interesting

• Rule: remove all vertices whose color is not interesting

• Gives a forest


Rough analysis

• O(k) broken colors, each with O(k2) vertices• O(k3) possible paths in tree, so O(k4) interesting

colors• O(k6) vertices in reduced instance • Working harder gives better bound (quadratic)• We are not yet finished: going back to original

problem– From forest to tree– Without fixed colors


From forest to tree

• Take one new color, and one vertex v* with this color, fixed.

• Make v* incident to a vertex with this color at each subtree

Like this (click)


From forest to tree

• Take one new color, and one vertex v* with this color, fixed.

• Make v* incident to a vertex with this color at each subtree


Getting rid of annotations

• If v with color C is annotated (fixed color), then add k+2 leaves with color C, incident to v and remove the annotation

• Still O(k6) size


Lessons

• Reduction rules that– Make the graph smaller and transform Yes-instances to

Yes-instances and No-instances to No-instances

– Or improve structural map: insight in instance

• Different rules ensure that different aspects of the resulting instance are small

• Annotation• Analysis of problem and sizes

6

Iterative compression


Idea of Iterative Compression

• Take small subset of input• Repeat until we have a solution on entire

input– Solve problem on small subset

• We have a solution from the previous round, and this often helps!

– If solution > k, return NO– Add tiny part of input to subset of input, e.g.,

add one additional vertex


• Feedback vertex set problem

• Recent (Chen et al. 2007) result, combining different techniques

Example


Feedback Vertex Set

Instance: graph G=(V,E)

Parameter: integer k

Question: Is there a set of at most k vertices W, such that G-W is a forest?

• Known in FPT• Here: recent algorithm O(5k p(n)) time algorithm• Can be done in O(5k kn) or less with kernelisation


Iterative compression technique

• Number vertices v1, v2, …, vn

• Let X = {v1, v2, …, vk}

• for i = k+1 to n do• Add vi to X

– Note: X is a FVS of size at most k+1 of {v1, v2, …, vi}

• Call a subroutine that either– Finds (with help of X) a feedback vertex set Y of size at

most k in {v1, v2, …, vi}; set X = Y OR

– Determines that Y does not exist; stop, return NO


Compression subroutine

• Given: graph G, FVS X of size k + 1

• Question: find if existing FVS of size k– Is subroutine of main algorithm

for all subsets S of X doDetermine if there is a FVS of size at most k that

contains all vertices in S and no vertex in X – S


Yet a deeper subroutine

• Given: Graph G, FVS X of size k+1, set S• Question: find if existing a FVS of size k containing all

vertices in S and no vertex from X – S 1. Remove all vertices in S from G2. Mark all vertices in X – S3. If marked cycles contain a cycle, then return NO4. While marked vertices are adjacent, contract them5. Set k = k - |S|. If k < 0, then return NO6. If G is a forest, then return YES; S7. …


Subroutine continued

7. If an unmarked vertex v has at least two edges to marked vertices

• If these edges are parallel, i.e., to the same neighbor, then v must be in a FVS (we have a cycle with v the only unmarked vertex)

• Put v in S, set k = k – 1 and recurse

• Else recurse twice: • Put v in S, set k = k – 1 and recurse

• Mark v, contract v with all marked neighbors and recurse– The number of marked vertices is one smaller


Other case

8. Choose an unmarked vertex v that has at most one unmarked neighbor (a leaf in G[V-X])

By step 7, it also has at most one marked neighbor

• If v is a leaf in G, then remove v• If v has degree 2, then remove v and connect

its neighbors


Analysis

• Precise analysis gives O*(5k) subproblems in total• Imprecise: 2k subsets S• Only branching step:

– k is decreased by one, or

– Number of marked vertices is decreased by one

• Initially: number of marked vertices + k is at most 2k

• Bounded by 2k.22k = 8k

7

Conclusions


Conclusions

• Fixed parameter tractability gives interesting new insights on combinatorial problems

• Here: examples from (mainly) graphs and logic

• Can be applied to different fields

Documents

Parameterized Complexity Hans Bodlaender IPA Basiscursus Algoritmiek