A Polyhedral Approach to Cardinality Constrained Optimization

Ismael Regis de Farias Jr. and Ming ZhaoUniversity at Buffalo, SUNY

Summary

• Problem definition

• Relation to previous work

• Simple bound inequalities

• Further research

Problem Definition

Given c1n , Amn , bm1 , and ln1 , un1 ≥ 0, find xn1 that:

maximizes

cx

subject to

Ax b,

−l ≤ x ≤ u,

and at most k variables are nonzero

Motivation

• Portfolio selection

• Feature selection in data mining

Polyhedral approach

Derive within a branch-and-cut scheme strong

inequalities valid for:

Pi = conv {x Rn : jN aij xj bi , −l ≤ x ≤ u,

and at most k variables are nonzero},

i {1, …, m}, to use as cutting planes in the

branch-and-cut

Previous work

• Bienstock (1996): critical set inequalities

• de Farias and Nemhauser (2003): cover inequalities.

However, the present case is more general and

the polyhedral structure is much richer …

Example

Let P = conv {x [−1, 1]6 : 6 x1 + 4 x2 + 3 x3 + 2 x4 + x5 + x6 6 and at most 3 variables are nonzero}. The following inequalities define facets of P:• 6 x1 + 4 x2 + 3 x3 + 2 x4 + x5 + x6 6• 4 x2 + 3 x3 + 2 x4 + x5 + x6 6• 4 x2 + 3 x3 + 2 x4 + x5 6• 4 x2 + 3 x3 + 2 x4 + x6 6• 4 x2 + 2 x4 + x5 + x6 6• 4 x2 + 2 x4 + x6 6• 4 x2 + 2 x4 + x5 6

To take advantage of previous work

… first, we scale and translate the variables,

i.e. P = conv {x [0, 1]n : jN aj xj b and xj

βj , j N, for at most k variables}, and

second, we consider the pieces of P

The pieces are defined as follows …

Proposition Let W N, XW = {x Rn : xj

βj j W and xj ≤ βj j N − W}, and PW = P ∩ XW . Then, PW = conv (S ∩ XW), where S = {x [0, 1]n : jN aj xj b and xj βj , j N, for at most k variables}. �

For each piece …

i.e. for a given W, we change the variables

as:

• yj ← (xj – βj) / (1 – βj), j W

• yj ← (βj – xj) / βj , j N − W

Example

P = conv {x [0, 1]2 : 6 x1 + 4 x2 7, and x1

=

½ or x2 = ½}.

½

½

1

1x1

x2

6 x1 + 4 x2 = 7

Example

P = conv {x [0, 1]2 : 6 x1 + 4 x2 7, and x1 =

½ or x2 = ½}.

½

½

1

1x1

x2

PN

P{1}

P{2}

P

Example

P = conv {x [0, 1]2 : 6 x1 + 4 x2 7, and x1 =

½ or x2 = ½}.

½

½

1

1x1

x2

3 y1 + 2 y2 2−3 y1 + 2 y2 2

3 y1 − 2 y2 2−3 y1 − 2 y2 2

at most 1 nonzero

at most 1 nonzeroat most 1 nonzero

at most 1 nonzero

When aj 0 and b > 0 …

Proposition The inequality jN xj k is facet-defining iff an−k + …+ an−1 b and a1 + an−k+2 + …+ an b. �

Proposition When an−k + …+ an−1 b and a1 + an−k+2 + …+ an > b, the inequality a1x1 +2≤j≤n−k−1 max {aj , Δ} xj +Δ n−k≤j≤n xj ≤ k Δ defines a facet of P, where Δ = (b − n−k−2≤i≤n

ai). �

It then follows that …

Proposition The inequality:

jW (xj – βj)/(1 – βj)–jN−W (xj – βj)/βj k is valid W N, and it is facet-defining “under certain conditions”. �

In the same way …

Proposition The inequality:

a1(x1 – β1)/(1 – β1) +2≤j≤n−k−1, jW max {aj ,

Δ}(xj – βj)/(1 – βj)+2≤j≤n−k−1, jN−W max {aj,

Δ} (xj – βj)/βj + Δ n−k≤j≤n , jW (xj – βj)/(1 – βj) + Δ n−k≤j≤n, jN−W (xj – βj)/βj ≤k Δ defines a facet of P “under certain conditions”. �

Example

P = conv {x [0, 1]2 : 6 x1 + 4 x2 7, and x1 =

½ or x2 = ½}.

½

1

1x1

x2

x1 + x2 3/2

x1 − x2 1/2x1 + x2 ≥ 1/2

(y1 + y2 1 and 3 y1 + 2 y2 2)

(y1 + y2 1 and 3 y1 − 2 y2 2)

−x1 + x2 1/2

(y1 + y2 1 and −3 y1 − 2 y2 2)

(y1 + y2 1 and −3 y1 + 2 y2 2)

Critical sets and covers

• By fixing, at 0 or 1, variables with positive or negative coefficients, we can obtain implied critical sets or cover inequalities that define facets in the projected polytope.

• Then, by lifting the fixed variables, we obtain strong inequalities valid for P

Example

Let P = conv {x [0,1]5: 6x1 + 4x2 − 3x3 − 2x4

+ x5 6 and at most 2 variables are positive}.

Fix x3 = 1 and x4 = 0. The inequality:

6x1 + 4x2 + 3x5 9

defines a facet of P ∩ {x [0,1]5: x3 = 1 and

x4 = 0}.

Simple bound inequalities

Let P = conv {x [0,1]4: 6x1 − 4x2 + 3x3 − x4

3 and at most 2 variables are positive}. Fix x3 = x4

= 0. Then, x1 1 defines a facet of P ∩ {x [0,1]4:

x3 = x4 = 0}. Lifting with respect to x4, we obtain x1 +

α x4 ≤ 1, which gives α = ⅓. Lifting now with

respect to x3, we obtain 3x1 + α x3 + x4 ≤ 3, which

gives α = 2, and so 3x1 + 2 x3 + x4 ≤ 3.

Additional results

• Two families of lifted cover inequalities

• Two families of inequalities derived from simple bounds

• Necessary and sufficient condition for “pieces of a facet” to be a facet

Further Research

• Separation routines and computational testing

• Inequalities derived from intersection of knapsacks

• Special results for feature selection in data mining