View
222
Download
2
Category
Preview:
Citation preview
1
Computational Complexity(Ctnd.)
ORF 523Lecture 15
Instructor: Amir Ali AhmadiTA: G. Hall
Spring 2015
2
Open questions in complexity for optimization
3
Open questions in complexity for optimization
4
Open questions in complexity for optimization
5
Open questions in complexity for optimization
6
Convexity
This talk:-- Given a multivariate polynomial, can we efficiently decide if it is convex?
-- Given a basic semialgebraic set, can we efficiently decide if it is a convex set?
“In fact the great watershed in optimization isn't between linearity and nonlinearity, but convexity and nonconvexity.”
Rockafellar, ’93:
But how easy is it to distinguish between convexity and nonconvexity?
7
Data separationwith convex sublevel sets
Convex envelope approximation
Convex fitting
Search over Convex Polynomials: Applications
Other Applications:
- Convex Lyapunov functions in robust control
- Norms defined by polynomials
8
Complexity of deciding convexityInput to the problem: an ordered list of the coefficients (all rational)
Degree d odd: trivial
d=1, always convex
d>1 and odd, never convex
d=2, i.e., p(x)=xTQx+qTx+c : poly time
check if Q is PSD
d=4, first interesting case
Question of N. Z. Shor
9
What we are going to prove:
Thm: Deciding convexity of quartic polynomials is (strongly) NP-hard. This is true even if the polynomials are restricted to be homogeneous (i.e., forms).
10
Sequence of reductions
STABLE SET
Nonconvex QP
Nonnegativity of biquadratic forms
Convexity of quartic forms
CLIQUE
Any problem in NP
Circuit SAT
SAT 3SAT
Cook-Levin Theorem
Saw in class
Flip edges
Motzkin-Strauss Thm.
Next
Next
(straightforward, see [DPV])
(straightforward, see [DPV])
Any problem in NP Circuit SAT (review)
11
The Cook-Levin theorem.
In a way a very deep theorem.
At the same time almost a tautology.
An example of a problem in NP: STABLE SET
12
Sequence of reductions
STABLE SET
Nonconvex QP
Nonnegativity of biquadratic forms
Convexity of quartic forms
CLIQUE
Any problem in NP
Circuit SAT
SAT 3SAT
Cook-Levin Theorem
Saw in class
Flip edges
Motzkin-Strauss Thm.
Next
Next
(straightforward, see [DPV])
(straightforward, see [DPV])
13
NP-hardness of deciding convexity of quartics
Reduction from problem of deciding “nonnegativity of biquadratic forms”
Thm: Deciding convexity of quartic forms is NP-hard.
Biquadratic form:
These are degree-4 polynomials of very special structure.
So our previous proof that testing nonnegativity of quartic (homogeneous) polynomials is NP-hard does not suffice…
14
CLIQUEBiquadratic Nonnegativity
15
Why this problem? And why aren’t we done?
Biquadratic form:
Can write any biquadratic form as
where is a matrix whose entries are quadratic forms
Example: with
This is exactly how the Hessian of a quartic form looks like!
So why aren’t we done?
16
Why aren’t we done?
The Hessian has very special structure!
above is not a valid Hessian:
17
From biquadratic forms to biquadratic Hessian forms
We give a constructive procedure to go from any biquadratic form
to a biquadratic Hessian form
by doubling the number of variables, such that:
In fact, we construct the polynomial f(x,y) that has H(x,y) as its Hessian directly
Let’s see this construction…
18
The main reductionThm: Given any biquadratic form
Let Let
Let
Then
19
Observation on the Hessian of a biquadratic form
20
Proof of correctness of the reductionStart with
Let Let
Claim:
21
Reduction on an instance
A 6x6 Hessian with quadratic form entries
22
Questions for you
Why does the result imply testing convexity is NP-hard also for degree-6 polynomials? (and degree-8, etc.)
(Hint: add an independent variable)
Why does the result imply testing convexity of basic semialgebraic sets (i.e., sets defined by polynomial inequalities) is NP-hard?
(Hint: epigraphs)
23
Other notions of interest in optimizationStrong convexity
“Hessian uniformly bounded away from zero”
Appears e.g. in convergence analysis of Newton-type methods
Strict convexity
“curve strictly below the line”
Guarantees uniqueness of optimal solution
Convexity
Pseudoconvexity
“Relaxation of first order characterization of convexity”
Any point where gradient vanishes is a global min
Quasiconvexity
“Convexity of sublevel sets”
Deciding convexity of basic semialgebraic sets
24
Summary of complexity results
25
QuasiconvexityA multivariate polynomial p(x)=p(x1,…,xn ) is quasiconvex if all its sublevel sets
are convex.
Convexity Quasiconvexity
(converse fails)
Deciding quasiconvexity of polynomials of even degree 4 or larger is (strongly) NP-hard
Quasiconvexity of odd degree polynomials can be decided in polynomial time
26
Quasiconvexity of even degree forms
Proof outline:
A homogeneous quasiconvex polynomial is nonnegative (why?)
The unit sublevel sets of p(x) and p(x)1/d are the same convex set
p(x)1/d is the Minkowski (semi)-norm defined by this convex set and hence a convex function
A convex nonnegative function raised to a power d larger than one remains convex (why?)
Corollaries:-- Deciding quasiconvexity is NP-hard
-- Deciding convexity of basic semialgebraic sets is NP-hard
Lemma: A homogeneous polynomial p(x) of even degree d is quasiconvex if and only if it is convex.
27
Proof:
Show super level sets must also be convex sets
Only convex set whose complement is also convex is a halfspace
Quasiconvexity of odd degree polynomials
This representation can be checked in polynomial time
Thm: The sublevel sets of a quasiconvex polynomial p(x) of odd degree are halfspaces.
Thm: A polynomial p(x) of odd degree d is quasiconvex iffit can be written as
As we’ve seen several times by now…
28
The boundary between the tractable and the intractable is very delicate!
Testing quasiconvexity of odd vs. even degree polynomials
Mincut vs. Maxcut
Testing matrix positive semidefiniteness versus matrix copositivity
Robust stability of interval polynomials versus robust stability of interval matrices
Testing primality versus finding the factors (?)
Many many others…
Let’s recall what we are achieving by an NP-hardness proof
29[Garey, Johnson]
NP-complete problems are everywhere…
30
Today we have thousands of NP-complete problems. In all areas of science and engineering.
The domino effect
31
All NP-complete problems reduce to each other!
If you solve one in polynomial time, you solve ALL in polynomial time!
The $1M question!
32
• Most people believe the answer is NO!• Philosophical reason: If a proof of the Goldbach conjecture were to fly from
the sky, we could certainly efficiently verify it. But should this imply that we can find this proof efficiently? P=NP would imply the answer is yes.
Nevertheless, there are believers too…
33
• Over 100 wrong proofs have appeared so far (in both directions)! See http://www.win.tue.nl/~gwoegi/P-versus-NP.htm
Coping with NP-completeness
34
Solving special cases exactly
Approximation algorithms – suboptimal solutions with worst-case guarantees
At least certifying gap to optimality on an instance-by-instance basis Can easily be done by finding bounds using convex optimization (think e.g., SDP for STABLE SET)
Challenge whether worst-case analysis is the right notion But propose rigorous alternatives
Average-case analysis, provable behavior on random instances, etc.
Heuristics OK if you are truly solving a real-life problem
Easy for practitioners to become overly optimistic about their performance
On what fraction of the input instances do you think a poly-time heuristic should fail? Just a few contrived examples that theoreticians can construct?
Constant fraction?
Linearly many?
Polynomially many?
Results in complexity theory from the 80s and 90s: even the last answer would imply P=NP
Main messages…
35
Computational complexity theory beautifully classifies many problems of optimization theory as easy or hard
At the most basic level, when faced with a new optimization problem, you should try to prove membership to the class P or establish NP-hardness.
After that, you can still study the complexity of your problem in further detail (e.g., quality/hardness of approximation)
Complexity theory is important to the theory of convex optimizationArguably the most rigorous tool we have for proving that certain problems do not have an efficient convex reformulation
The boundary between tractable and intractable problems is very delicate:
MINCUT vs. MAXCUT, PSD-ness vs. copositivity, LP vs. IP, ...
P=NP?
Maybe one of you guys will tell us one day.
References
36
- NP-hardness of testing convexity: http://web.mit.edu/~a_a_a/Public/Publications/convexity_nphard.pdf
- NP-hardness of testing nonnegativity of biquadratic forms: http://www.math.ucsd.edu/~njw/PUBLICPAPERS/LNQY_july02.pdf
- Survey on performance of heuristic algorithms: http://arxiv.org/abs/1210.8099v1
- [DPV08] S. Dasgupta, C. Papadimitriou, and U. Vazirani. Algorithms. McGraw Hill, 2008.
- [GJ79] D.S. Johnson and M. Garey. Computers and Intractability: a guide to the theory of NP-completeness, 1979.
Recommended