Matrix Completion Problemsusers.uoa.gr/~wvkarag/files/wvk-asb-mcp.pdf · Completion Problems Summary References Introduction Some De nitions Partial matrix Any matrix A ∈ F m ×n

Outline

Matrix Completion Problems

William Âáóßëåéïò Êáñáãåþñãïò

ÌÐËÁ, ÅÊÐÁ

Áëãüñéèìïé óôçí ÄïìéêÞ ÂéïðëçñïöïñéêÞ8 Áðñéëßïõ, 2008

William Vassilis Karageorgos, A.M.: 200609 Matrix Completion Problems

Outline

Outline

1 Introduction

2 Completion ProblemsPositive (de�nite) MatricesEuclidian Distance MatricesCompletely Positive MatricesContraction MatricesRank Completions

3 Summary


IntroductionCompletion Problems

SummaryReferences

Introduction

Some De�nitions

Partial matrixAny matrix A ∈ Fm×n with unspeci�ed entries, i.e. only a subset

S ⊆ A of its positions is speci�ed

Matrix completion & completion problemsA matrix completion is the complement CAS of S over A

Completion problems deal with whether it is possible to �nd such a

complement, which meets certaint criteria, in the form of matrix

properties

Patternbipartite graph G(V1 + V2;E ) : V1 = {i |i = 1; :::;m};V2 = {j |j =

1; :::;n};E = {{i ; j}|aij ∈ S ⊆ A}



SummaryReferences

Introduction

Some De�nitions

Matrix properties (criteria) we will be concerned with:

Positive (de�nite) matrices(∀x ∈ Fm(〈x |Ax 〉 ≥ 0) ⇒ A � 0) ⇔ (A =

∑i ëi |i〉〈i | = BBT )

Distance matrices (Euclidian)D ∈ Rn×n : ∃u1; :::; uk∀i ; j ≤ n(dij = ‖ui − uj‖2)Completely positive matrices A = BBT , where B is nonnegative

i.e. bij ≥ 0

Contraction matrices〈Ax |Ax 〉 = ë〈x |x 〉 ⇒ ë ≤ 1

Matrix rankThe maximum number of linearly independant columns/rows



SummaryReferences

Introduction

Remarks

For positive (de�nite) and distance n × n matrices, we usuallyassume aii ∈ S

A � 0⇒ A = A† ∵ A = B + iC for any A, whereB = B†;C = C †

Inheritance structureAny principal submatrix of A must share the same propertieswith AGives rise to combinatorial aspect of completion problems



SummaryReferences

Positive (de�nite) MatricesEuclidian Distance MatricesCompletely Positive MatricesContraction MatricesRank Completions

Outline

1 Introduction


3 Summary



SummaryReferences


Positive (semi)de�nite Completion Problem (PSD)

The (PSD) problem is an instance of the more generalSemide�nite Programming Problem (P):

Given Hermitian matrices Ai and scalars bi ,

where i = 1; :::;m, decide whether

X � 0; 〈Aj |X 〉 = bj

is feasible

which in turn can be viewed as a generalization of the LinearProgramming problem (LP)

maximize cT x ;Ax ≤ b; x � 0



SummaryReferences


Complexity of (P)

Still open question, but if NP 6= coNP , then neitherNP-complete nor coNP-complete

Can be solved in P with arbitrary precision by usingcombinatorial means such as the ellipsoid and interior-pointmethods, for �xed m



SummaryReferences


PSD Solvability Methods

PSD actually a convex optimization problem

Ellipsoid methodEnclose the minimizer of a convex function in a sequence of volume

decreasing ellipsoids

Interior-point methodDe�ne a self-concordant fuction f : R → R : |f ′′′(x )| ≤ 2f ′′(x )

3

2 to

encode the convex set and traverse the feasibility region



SummaryReferences


PSD Solvability Preconditions I

To be positive (de�nite), matrix A must also be partialpositive (de�nite)

Partial positive matrix A has positive completion i� pattern Gis a chordal graph (C4 �ind G)

If G is not chordal, then the minimum �ll-in is the number ofedges that need to be added in order to make it so

Finding the minimum �ll-in is NP-hard, but if it is known to beequal to m, then (P) can be solved in P (m linear constraints)



SummaryReferences


PSD Solvability Preconditions II

If a partial m × n matrix A with diagonal entries equal to 1has a positive (de�nite) completion, then the associated vector

x :=1

ðarccos aij

satis�es ∑e∈F

xe −∑

e∈C\F

xe ≤ |F | − 1

where F ⊆ C ;C cycle in G ; |F | oddConversely, if a pattern G satis�es the above relation, then theaccording matrix has a positive (de�nite) complement i� Gdoes not contain a homeomorph of K4 as an induced subgraph

Polynomial complexity for x ∈ Q, but generally unknown



SummaryReferences


PSD Solvability Preconditions III

Partial matrix A has a positive (de�nite) completion i�∑i∈V

aiixii +∑

i 6=j ;ij∈Eaij xij ≥ 0;∀X ∈ PG

where PG is the cone graph associated to G

PG := {X = [xij ]|X � 0; xij = 0;∀i 6= j ; ij ∈ E}

One needs only check those X's extremal in PG

If we de�ne order(G) := min{rank(X )|X extremal in PG},then rank 1 graphs are exactly the chordal graphs

Complexity unknown for higher orders



SummaryReferences


Outline

1 Introduction


3 Summary



SummaryReferences


Euclidian Distance Matrix Completion Problem (EDM)

(EDM): Given partial symmetric matrix A with pattern G

A ∈ Rn×n : aii = 0

G = {V ;E};V = {i |i = 1; :::;n}

determine whether there exist a realization of A, i.e. vectors

u1; :::; un ∈ Rk; k ≥ 1 : ∀{i ; j} ∈ E (aij = ‖ui − uj ‖2)

(EDMk): Graph Realization Problem. Variation of (EDM) for�xed k



SummaryReferences


EDM Complexity

(EDM) can be reduced to (PSD)

Given symmetric n × n matrix D = [aij ]; aii = 0, D is adistance matrix i� n − 1× n − 1 matrix X

X = [xij ]; xij :=1

2(din + djn − dij )

is positiveD has a realization in kd-space i� rank(X ) ≤ k

Therefore, (EDM) solved with arbitrary precision in P

Exact complexity of (EDM) unknown, but (EDMk) isNP-complete for k = 1 and NP-hard for k ≥ 2



SummaryReferences


EDM Solvability

The same preconditions that apply to (PSD) also apply

(EDMk) for k ≤ 3, can be reformulated into an optimizationproblem, by minimizing a cost function such as

f : Rn×k → R; f (w) :=∑i ;j

(‖ui − uj ‖2 − aij )2

where w := (u1; :::; un); ui ∈ Rk

Since at least NP-hard, special programming techniques suchas \tabu", \pattern searching" and of course \D&C" are used



SummaryReferences


Outline

1 Introduction


3 Summary



SummaryReferences


Completely Positive Matrice Completion Problem

Matrix A is doubly nonnegative if A � 0 and A is nonnegative

A completely positive ⇒ A doubly nonnegative

Actually a con�nement of (PSD) over R+

Problem solvable if K4 �ind G



SummaryReferences


Outline

1 Introduction


3 Summary



SummaryReferences


Contraction Matrice Completion Problem

Reduction to (PSD)Matrix A is a contraction matrix i� A � 0, where

A :=

(In A

A† Im

)Solvable i� one of the below preconditions hold

G does not contain an induced matching of size 2(nonseparable bipartite)~G is chordal, where ~G is the pattern of ~A



SummaryReferences


Outline

1 Introduction


3 Summary



SummaryReferences


Determining the rank of a completion

If minimum and maximum ranks are known then we canconstruct completions of any rank in between, by starting o�one of the former and changing entries one at a time

Maximum rank MR(A) computed in P

Minimum rank mr(A) hard to compute



SummaryReferences


Maximum rank

Equivalent to computing generic rank of A, where unspeci�edvalues are concidered variables

For X ⊆ V1 ∪ V2, X is a cover of G i�AX := [aij ]; i ∈ V1 \ X ; j ∈ V2 \ X is a fully specifedsubmatrix

MR(A) = rank(AX ) + |X |, where X in the minimum cover ofG

Solvable in P by greedy algorithm which perturbs an arbitrarycompletion



SummaryReferences


Minimum rank

Generally depends on the actual values aij

Which are the graphs G for which mr(A) depends only on therank of submatrices, like with MR(A)? (rank determinedgraphs)

Conjecture: bipartite chordal graphs (C6 �ind G) are rankdetermined. Proven for nonseparable bipartite



SummaryReferences


Minimum rank

Matrix A has nonseparable pattern G i� it is of the\triangular" form

A =

(B ?C D

)Then,

mr(A) = rank

(B

C

)+ rank

(C D

)− rank(C )



SummaryReferences

Summary

Summary

All problems are closely related to each other

Graph theory is an important tool

Complexity dependant upon the �eld over which the matricesare de�ned

Generally hard problems, but in most cases approximatelysolvable in P

Constructive algorithms for approximate solutions



SummaryReferences

References

M. Laurent.Matrix Completion Problems.The Encyclopedia of Optimization, vol. III, pages 221{229.

M. Laurent.Polynomial instances of the positive semide�nite and euclideandistance matrix completion problems.SIAM Journal on Matrix Analysis and its Applications,22:874{894, 2000.


AppendixNotationPartial Matrices

Appendix Outline

4 AppendixNotationPartial Matrices



Dirac Notation: bra & ket

Ket: A \ket" |x 〉 is an element of a Hilbert space(V ;F ;+; ·) ≡ VBra: A \bra" 〈x | is an element of the dual spaceV∗ ≡ (V ∗;L(V ;F );+; ·) of a Hilbert space V



Dirac Notation: braket

Inner product (\braket"): For x ; y ∈ V , x · y ≡ 〈x |y〉 ∈ Fe.g.

For V an in�nite dimensional Hilbert space over a �eld F offunctions öi , 〈ök |ö`〉 ≡

∫ö∗k (t)ö`(t)dt

For V a �nite dimensional Hilbert space over a �eld Fn×1 ofvectors

〈x |y〉 ≡nXi=1

x∗i yi =

ˆx∗1 : : : x∗n

˜ 264 y1.

.

.

yn

375 = (X ∗)TY ≡ X†Y

For V a �nite dimensional Hilbert space over the �eld Fn×n ofmatrices

〈x |y〉 ≡ tr(X †Y ) =n∑i=1

(X †Y )ii =n∑i=1

n∑j=1

x∗ij yji



Dirac Notation: braket viewpoints

Analytical viewpoint: The braket is properly de�ned as abilinear mapping 〈·|·〉 : V∗ × V → F

Geometric viewpoint: If |x 〉 and |y〉 reside on di�erentsubspaces of space V, and |x 〉 is unitary, then the braket 〈x |y〉consists of a projection of |y〉 onto the subspace spanned by|x 〉



Dirac Notation: linear operators

If A is a linear operator acting on a Hilbert space V, then

〈x |A|y〉 ≡ 〈x |Ay〉 ∈ F

〈x |A†|y〉 ≡ 〈Ax |y〉 ∈ F

where |Ay〉 ∈ V is a new ket resulting from operating on |y〉with A, and mutatis mutandis

Outer product: Any linear operator A acting on V can beexpressed as A = |x 〉〈y | in terms of the mapping

|·〉〈·| : (V ∗;L(F ;V ))× (V ;F ) → F × F



Dirac Notation: linear operator viewpoints

Matrix viewpoint: When working with a �eld F n×1 of vectors,then A = |x 〉〈x | is the matrix product of an n × 1 matrix witha 1× n matrix, thus A is an n × n matrix

Geometric viewpoint: Operators of the form |x 〉〈x | consist ofprojection operators onto the subspace spanned by |x 〉



Partial Matrices as Linear Partial Functions

Any matrix A ∈ Fm×n can be viewed of as a linear function

fv ;u : (V ;Fm) → (U ;F n)

uniquely de�ned by the choise of bases v ; u for spaces V;Uaccordingly

Similarily, a partial matrix can be viewed of as a partialfunction

öv ;u : (V ;Fm) * (U ;F n) ≡ öv ;u : (V ;Fm) → (U ;F n∪{⊥})

where \bottom" ⊥ is an abstract value devoid of meaning


Documents

Matrix Completion Problemsusers.uoa.gr/~wvkarag/files/wvk-asb-mcp.pdf · Completion Problems Summary References Introduction Some De nitions Partial matrix Any matrix A ∈ F m ×n