Upload
virginia-patrick
View
224
Download
0
Tags:
Embed Size (px)
Citation preview
Discrete Optimization Lecture #1
Today:
Reading Assignments1. Chapter 1 and the Appendix of [Pas82]2. Chapter 1 of [GaJ79]
Outline:1. Course Overview
» A taxonomy of optimization problems» Course introduction» Requirements and schedule
2. Some Basics of Optimization» Local and global optima» Feasibility» Convexity» Convex Programming
3. Algorithms and Complexity» Problems, algorithms, and Complexity» Polynomial time algorithms» Intractability» NP-Complete Problems
§I.1 Course Overview
• Introduction to Optimization Problems Ingredients of an optimization problem
• A set of independents on the values of variables or parameters• Condition or restrictions on the values of variables• Criterion or objection find the best solution
• A Standard Mathematical Form max F(x)x sSubject to (x)=0 i=1,…,m (x)≤0 j=1,…,r• Classifications(1) By the time factor
static
dynamic
jghi
Subject to X(t)=
a ≤ u(t) ≤ b 0 ≤ x(t) ≤ R
(2) By the nature of variables
S
=> nonlinear programming => discrete(combinatorial) optimization =>mixed integer programming
I R
0t du
In
Rn
C(t)
Grass Price Factor
Example 2: Example 2:
養馬問題 max P(x())- c(t)E(u(t))d x(t)
t f tt f0
t
(3) By the nature of problem functions
Properties of F(x) Properties of g(x)&h(x)
Function of a single variable Linear function
No Constraints
Sum of squares of linear functions
Simple bounds
Quadratic functions Linear functions
Sum of squares of nonlinear functions
Sparse linear functions
Smooth function Smooth nonlinear functions
Sparse nonlinear function Non-smooth nonlinear functions
Non-smooth function
In this course , we will consider problems with discrete variables.
• Example: The Traveling Salesman Problem (TSP)
Cities: , , , Distance between cities d( , )Find the shortest path that goes through every city once and only once and back to the starting city.Q: Why is this a discrete optimization problem?
c1 c2
c2
c1
c3
c3
c4
c4ci c j
Course Introduction
Nonlinear programming
Convex programming
Linear programming Integer
programming
.
• Linear Programming minimize c’ x x subject to A1 x=b1
A2 X≤ b2
Fact : Optimal solution happens at a vertex discrete (combinatorial) nature LP serves as a bridge between the continuous and discrete
optimization
x1
x2 c
Rn
• Optimization on Networks Example: The general minimum-cost flow problem minimize subject to - = bi i=1,2…,n [Flow balance] ≤ ≤ [Flow balance] LP with special constraint structure efficient solution techniques available by exploiting such a structure
Variations: shortest path problem minimum spanning tree maximum flow Minimum-cost flow
i
j ijc ijx
j ijx k
kix
ijl ijx iju
b1 1 3
4
6
7
2 7 8
x13c13
[ , ]l13 u13
Emphases:(1)Exploiting special problem structure(2)Applications in EE&CS(3)Distributed or parallel algorithms(4)Foundations for problems with nonlinear objectives or other
discrete optimization problems
Integer Programming basic techniques for general problems NP-completeness TSP Knapsack Problems min s. t.
Scheduling Problems
Simulated Annealing an approach to global optimization
xp i
n
ii
1
ni
v
x
xw
i
i
n
ii
,...,2,1
,...1,01
§I.2. Basics of Optimization
• Definition of on Optimization Problem
J= i.e. a mapping A: F ->
• An instance of an optimization: (F,A)• Feasibility : x is feasible e.g.
minFx x arg
0x
min)(xg
R1
Fx
mjx
nixxF
gh
j
i
,...,1,0
,...,1,0
Fx
• Neighborhood: Given an optimization problem with instances(F,A), a
neighborhood is a mapping N: F -> example: (a) In LP with FC , we can define a neighborhood
2F
Rn
xyandybyAyxN ,0,)(
Q: what to do with discrete cases? Example:
2-change of ={g: and g can be obtained from f by
removing two edges from the tour and then replacing them with two edges}
c1
c2
c3 c4
Ff)(
2fN fg
• Local and Global Optima Example: continuous case
A and C are local minima B is the global minima
Q: How to define them for discrete optimization?
F(x)
0 a x
A
BC
Definition : Local Optima Given an instance (F,A) of an optimization problem, and a neighborhood N, is locally optimal with respect to N if A(f)≤A(g) example: A best TSP tour in may not be the solution
tour. Definition : Global Optima If and is locally optimal w.r.t a neigborhood N, it
is then also locally optimal to any other neighborhood f is globally optimal N is exact.
Q: How to check with respect to all N? example: In TSP, is not exact but is for an n-city
problem.
Ff fNg
N 2
Ff
N
N 2 N n
Discrete Optimization Lecture #2
Last time: Course overviewSome basics of optimization
Today: Reading Assignments: 1. Chapter 1 of [GAJ79] 2. Sections 2.1~2.3 of [PaS82] 3. Sections 2.4,2.5,3.1~3.8 of [Lue84].
Outline: 1. Some basics of optimization (cont.)
convexity & Convex Programming 2. Algorithms and Complexity
Problems, algorithms, and complexity Polynomial time algorithms Intractability NP-Complete Problems
3. Basic properties of Linear Programs From of LP Basic feasible solutions
Geometry of LP 4. The simplex method Homework#1 Due:
I.3 Convexity and Convex Programming
Why are we so interested in convex functions and convex sets? min
(1) Globally speaking, If J, F are convex minimum points are global minimum(2) Local convexity local minimumDefinition: Convex Set is convex iff and Q: Facts about convex sets
(1) A hyperplane = is a convex set(2) A half space ≤ is a convex set
(3) Intersection of convex sets is convex Union of convex set?
(4) Contraction of a convex set is convex
(5) An empty set is convex
Fx )(xJ
Rn
cS ,Sx Sy
Syx )1( 10
cT x
cT x
z
z
Definition Convex Functionsf:
If
Then f is convex• Facts about convex functions
(1) A quadratic function is convex if Q≥0(2) The linear extrapolation(approximation) at a point underestimates a convex functioni.e. assume
RRn 1
Rn
yx ,
yfxfyxf 11
10
cxbxQxT
x
xxxxcf 2121,,,
xxxxx Tfff12111
x
xf 2
xxxfxT
f1211
x1 x2
(3) f is convex iff is positive semi-definite over
proof: read by yourself(4) Linear combination ( positive coeff.) of convex functions is convex (5) The level set is convex for all c if f is convex
Min
s.t. is a convex programming problem if f. g. and are convex.
cf2 xf2
x
cxfxxc
,
x xf
0xg
Properties of Convex Minimization(a) s= arg min is a convex set Proof: are two optimal solutions by def. of min.
x xf
xxtt
21
fxxttt
ff 21
ffxxxxtttttt
fff 2121
11
Sxx tt21
1