半正定規劃

國立交通大學

應用數學系

碩士論文

半正定規劃

Semidefinite Programming Problems

研究生：葉彬

指導教授：翁志文 教授

半正定規劃

Semidefinite Programming Problems

研究生：葉彬

指導教授：翁志文 教授

Student: Bin Yeh

Advisor: Chih-Wen Weng

國立交通大學 應用數學系

碩士論文

A Thesis

Submitted to Department of Applied Mathematics College of Science

National Chiao Tung University in Partial Fulfillment of the Requirements

for the Degree of Master

in

Applied Mathematics

June 2010

Hsinchu, Taiwan, Republic of China

半正定規劃

研究生：葉彬

指導教授：翁志文

國立交通大學應用數學系碩士班

摘要

在半正定規劃的問題中，我們要求一些對稱矩陣的彷射組合必須是半正 定，在這樣的限制下試圖將目標線性函數最小化。這些限制未必是線性， 但它們具有中凸的性質故半正定規劃是一種中凸規劃。在這篇論文中我們 探討了一些半正定規劃的基本性質與基礎理論並給出證明。

Semidefinite Programming Problems

Student: Bin Yeh

Advisor: Chih-Wen Weng

Department of Applied Mathematics

National Chiao Tung University

Abstract

In semidefinite programming problems one minimizes a linear func-tion subject to some constraints which requires an affine combinafunc-tion of symmetric matrices to be positive semidefinite. The constraints may not be linear but it is convex so semidefinite programming prob-lems are convex optimization probprob-lems. In this paper we give some basic properties and fundamental theorems with their proofs regrading semidefinite programming problems.

誌謝

這一路上最感謝我的指導教授翁志文老師，因為我的不成熟而給他添了 很多麻煩，實在是非常的對不起老師，在這邊要跟他說一聲對不起與謝謝 您。 兩年多來還有其他很多很照顧我的教授們，包括傅恆霖老師、白啟光老 師、黃大原老師、陳秋媛老師、王夏聲老師，真的非常感謝他們的指導與 照顧。 我還要感謝我同師門的學長鎬文和喻培，當然還有其他給我許多幫助的學 長姐，像是芳婷學姐、敏筠學姐、惠蘭學姐、國元學長、智懷學長，他們 給了我很多的幫助。 也非常謝謝眾多好同學們，永潔、光祥、文昱、易萱、宜君、舜婷、玉雯、 慧棻，以及研究室的夥伴們：育生、建文、思綸、健峰、思賢，還有我在 求學過程中認識的許多學弟妹：侖欣、彥伶、權益、湧昇、昇華、育慈、 定國、冠緯、閔豪、瑩晏、瑜堯、紹鵬、昌翰，謝謝你們陪我渡過快樂的 時光。 最後我要感謝我的家人，爸爸，媽媽與弟弟。無論是什麼情況，他們總是 支持著我、鼓勵著我，萬分的感謝你們。 要感謝的人很多，很幸運我能得到這麼多人的幫助。感恩之心難以言喻， 就僅在此聊表感激之情，沒有各位就沒有今日的我。

Contents

1 Introduction 1 2 Semidefinite programing problems 2 3 Dual problem 5 4 A few lemmas 8 5 Property of Semidefinite programming Problems 21 6 References 24

1 Introduction

This paper is based on the survey paper by Vandenberghe and Boyd [1] and some details, mostly proofs, which are omitted in that survey. In this paper we focus on fundamental theory about semidefinite programming problem, rather than its application.

Semidefinite programming (SDP) is an optimization problem concerned with the optimization of a linear objective function over the intersection of the cone of positive semidefinite matrices. Linear programming, a well-used mathematical model which is used to obtain the best outcome with some certain restriction in the form of linear equations, is actually a special case of semidefinite programming problem. There are many different types of algorithms to solve semidefinite programming problem and these algorithms are capable of getting the result in polynomial time.

There are a lot of applications of semidefinite programming. In opera-tions research and combinatorial optimization many problems are modeled as semidefinite programming problem so they can be well approximated. For example, Goemans and Williamson found an algorithm to obtain maximum cut using semidefinite programming [3].

In section 2 we give some basic definition about semidefinite programming problem and an example for better understanding. In section 3 the maybe most important property of semidefinite programming problem, duality, is discussed. In the further sections some more lemmas and propositions are

given and eventually in section 5 a critical theorem is proved.

Some proofs in this paper are based on the lecture note written by L´aszl´o Lov´asz [2] and the book by Abraham Berman and Naomi Shaked-Monderer [4] and the note by Konstantin Aslanidi [5]. The main idea of proof of lemma 4.13 is provided by Renato Paes Leme.

2

Semidefinite programing problems

Throughout, we useRn _{to denote the set of column vectors of size n and}

Rn×n_{to denote the set of n×n symmetric matrices over R. For a real matrix}

M , we use M ≥ 0 when M has nonnegative entries, and use M ≥p 0 when

M is symmetric and positive semidefinite, i.e. zT_{M z ≥ 0 for all z ∈ R}n_.

Similarly, we will denote positive definite by M >p 0. The following is the

general form of a semidefinite programming problem.

Problem 2.1. (SDPn(c, F0, F1, . . . , Fm))

Given a column vector c ∈ Rm _{and m + 1 n} × n symmetric matrices

F0, F1, . . . , Fm ∈ Rn×n. Find min x c T_x for x = [x1, x2, . . . , xm]T ∈ Rm subject to F0 + m ∑ i=1 xiFi ≥p 0.

Give a SDPn(c, F0, F1, . . . , Fm) and let F (x) = F0+

∑m

i=1xiFi. A vector

x∈ Rm_{is said to be feasible if F (x)≥}

called the feasible region of the problem. A vector xoptin the feasible region

is called an optimal point if cT_x

opt reaches the minimum of cTx among all

feasible points x. In this case the value cT_x

opt is called the minimum of the

problem. The number

inf

x c T

x∈ R ∪ {−∞},

where the infimum is taking for feasible x, is called the infimum of the prob-lem. If the infimum is not equal to the minimum, then SDPn(c, F0, F1, . . . , Fm)

is said to be infeasible. The following is a simple example of semidefinite programming problem. Example 2.2. Consider SDP2(   0 1   ,   0 1 1 0   ,   1 0 0 0   ,   0 0 0 1  ). The feasible region is

     x1 x2   |   0 1 1 0   + x1   1 0 0 0   + x2   0 0 0 1   ≥p 0    =      x1 x2   |   x1 1 1 x2   ≥p 0   . Note that   x1 1 1 x2 

 is semidefinite if and only if both x₁ and x1x2− 1

are nonnegative, i.e. x1 ≥ 0 and x1x2− 1 ≥ 0. Thus the feasible region is

     x1 x2   |x1 ≥ 0, x1x2 ≥ 1   . (1)

The minimum of (0, 1)(x1, x2)T = x2 does not exist in the feasible region since

x2 tends to zero as x1 tends to infinity. However the infimum of this problem,

obtained by above discussion, is zero. Thus, this example is infeasible. The following is the general form of a linear programming problem, which we will show to be a special case of semidefinite programming problem.

Problem 2.3. (LPPm(c, b, A))

Given b, c_{∈ R}m _{and a symmetric matrix A∈ R}m×m_{. Find}

min

x c T

x

subject to all x ∈ Rm _satisfying

Ax + b≥ 0.

Note that a linear programming problem is a special case of semidefinite programming problem. In fact by setting F0 = diag(b), Fi = diag(Ai) for

i = 1, . . . , m, where Ai is the ith column of A, the LPPm(c, b, A) becomes

the SDPn(c, F0, F1, . . . , Fm).

Similar to the semidefinite programming problem, in LPPm(c, b, A), a

vector x _{∈ R}m _{is said to be feasible if Ax + b ≥ 0 and the set {x ∈}

Rm | Ax + b ≥ 0} is called the feasible region. A vector x

opt in the feasible

region is called an optimal point if cT_x

opt reaches the minimum of cTx for

all feasible points x.

We now show Example 2.2 is no way to be interpreted as a linear program-ming problem. To the contrary assume Example 2.2 is also a LPm(c, b, A).

Then from the definition m = 2 and A is a 2 _{× 2 matrix. Let V denote} the nullspace of A. Then referring to (1) as the feasible region Ω := {x ∈ R2 | Ax + b ≥ 0} also as a feasible region of LP

m(c, b, A) we must have

Ω + V ⊆ Ω. This only happens when V = 0, i.e. A is invertible. But then the feasible region of LPm(c, b, A) is {x ∈ R2 | x ≥ −A−1b}, which is clearly

not to be Ω, a contradiction.

3

Dual problem

To solve a classical optimization problem, the problem and its dual

prob-lem play an important role. The following probprob-lem is the dual of SDPn(c, F0, F1, . . . , Fm).

Problem 3.1. (SDP∗_n(c, F0, F1, . . . , Fm))

Given c_{∈ R}m_{and m+1 symmetric matrices F}

0, F1, . . . , Fm ∈ Rn×n. Find

max

Z −tr(F0Z)

subject to all symmetric n_{× n matrices Z with}

Z ≥p 0,

tr(FiZ) = ci for 1≤ i ≤ m, (2)

where tr(M ) is the trace of M .

Give a SDP∗_n(c, F0, F1, . . . , Fm), a symmetric matrix Z is said to be

fea-sible if Z ≥p 0 and tr(FiZ) = ci for 1 ≤ i ≤ m. The set {Z ∈ Rn×n|Z ≥p

problem. A symmetric matrix Zopt in the feasible region is called an

opti-mal point if−tr(F0Z) reaches the maximum of−tr(F0Z) among all feasible

points Z. In this case the value cT_x

opt is called the maximum of the

prob-lem. The number

sup

Z −tr(F

0Z)∈ R ∪ {∞},

where the supremum is taking for feasible Z, is called the supremum of the

problem. If the supremum is not equal to the maximum, then SDP∗_n(c, F0, F1, . . . , Fm)

is said to be infeasible. The original semidefinite programming problem will be referred as the primal problem.

In Section 5, we will show that the supremum of SDP∗_n(c, F0, F1, . . . , Fm)

is no larger than the infimum of SDPn(c, F0, F1, . . . , Fm). This explains their

dual relation. Here we give an example of dual semidefinite programming problem.

Example 3.2. The dual problem of

SDP2(   0 1   ,   0 1 1 0   ,   1 0 0 0   ,   0 0 0 1  ) in Example 2.2 is SDP∗₂(   0 1   ,   0 1 1 0   ,   1 0 0 0   ,   0 0 0 1  ). That is to maximize −tr(   0 1 1 0   Z)

for all 2_{× 2 symmetric matrices Z subject to} tr(   1 0 0 0   Z) = 0, tr(   0 0 0 1   Z) = 1, and Z ≥p 0. Let Z =   Z11 Z12 Z21 Z22 

. The first condition says

0 = tr(   1 0 0 0     Z11 Z12 Z21 Z22  ) = tr(   Z11 Z12 0 0  ) = Z11,

and the second condition implies

1 = tr(   0 0 0 1     Z11 Z12 Z21 Z22  ) = T r(   0 0 Z21 Z22  ) = Z₂₂.

Thus we only need to consider matrices of the form 

 0 Z12

Z21 1

 . The third condition Z _≥p 0 is equivalent to

( x1 x2 ) Z   x1 x2   ≥ 0 for all x1, x2 ∈ R. As 0≤ ( x1 x2 ) Z   x1 x2   =( x2Z21 x1Z12+ x2 )  x1 x2   = x1x2(Z21+ Z12) + x22 = 2x1x2Z12+ x22,

Z ≥p 0 is equivalent to Z12= 0. Then our feasible region is

     0 Z12 Z21 1   |Z21 = Z12 = 0   .

The goal is to maximize −tr(   0 1 1 0     Z11 Z12 Z21 Z22  ) = −tr(   Z21 Z22 Z11 Z12  ) = −(Z21+ Z12),

which is always zero in the feasible region. Then the maximum of

SDP∗₂(   0 1   ,   0 1 1 0   ,   1 0 0 0   ,   0 0 0 1  )

is 0 which, as we showed before, is also the infimum but not the minimum of

SDP2(   0 1   ,   0 1 1 0   ,   1 0 0 0   ,   0 0 0 1  ).

The general theory of relation between a primal problem and its dual problem will be given the section 5.

4

A few lemmas

Before proceeding, we introduce some basic properties of positive semidef-inite matrices. Positive semidefinite matrices have some great properties in the way of geometry space. To see this we need some definition first. A

convex cone C is a set of vectors in a vector space such that (i) for any

vector v in C, rv is also in C for any r > 0, and (ii) for any two vectors v,

v′ ∈ C, v + v′ ∈ C.

For any convex cone C _{⊆ R}n×n_{, the polar cone C}∗ _{is defined by}

where A_{· B is the inner product for matrices, defined as}

A· B = tr(AtB) = ∑

1≤i,j≤n

Ai,jBi,j.

Note that a polar cone is also a convex cone, which is trivial to prove by applying the definition of convex cone.

Lemma 4.1. A positive multiple of a positive semidefinite symmetric matrix

is still positive semidefinite, and sum of two positive semidefinite symmetric matrices is still positive semidefinite.

Proof. Let symmetric matrices A, B ∈ Rn×n _{be positive semidefinite and}

r > 0. Then for any vector x∈ Rn, we have xT(rA)x = r(xTAx)≥ 0

and

xT(A + B)x = (xTAx) + (xTBx)≥ 0,

which finished the proof.

The above lemma shows that the set of all symmetric positive semidefinite matrices Pn is actually a convex cone inRn×n. For next lemma, we will use

Hadamard product of matrices; that is, for two matrices A, B _{∈ R}n×n_{, the}

Hadamard product A_{◦ B is defined as}

(A◦ B)ij = AijBij

for all 1_{≤ i, j ≤ n.}

A property of Hadamard product will be given later. To prove the prop-erty we first show the following proposition.

Proposition 4.2. Let A be a symmetric n_{×n real matrix. Then A is positive}

semidefinite if and only if there exists an n×n matrix B such that A = BBT_.

Proof. (_⇐)

For all x∈ Rn_,

xTAx = xT(BBT)x = (BTx)T(BTx) = (BTx)· (BTx)≥ 0.

(⇒)

Since A is symmetric, it follows that A is normal and therefore A is unitary similar to a diagonal matrix, i.e. P APT _{= D for some n× n matrix P with}

PT_{P = I and some diagonal matrix D. Then A = P}T_{DP . Note since A}

and D are similar, they have same eigenvalues λ1, λ2, . . . , λn and they are

all nonnegative because A is positive semidefinite. Then we may assume

D = diag(λ1, λ2, . . . , λn). Now let

D′ =         √ λ1 0 √ λ2 . .. 0 √λn         , thus A = PTDP = PTD′D′P = (PTD′)(PTD′)T.

Corollary 4.3. Let A be symmetric n_{× n real matrix. Then A is positive}

semidefinite if and only if there exist vectors v1, v2, ..., vn ∈ Rn such that

Proof. (_⇒)

By last lemma A = BBT _{for some n× n matrix B. Let v}

i be the ith column

vector of B for 1 _{≤ i ≤ n. Then} (A)ij = (BBT)ij = n ∑ h=1 (vh)i(vh)j = n ∑ h=1 (vhvhT)ij = ( n ∑ h=1 vhvhT)ij. (_⇐)

Let A = ∑n_i=1vivTi . Let B =

( v1 v2 . . . vn ) . Then (A)ij = ( n ∑ h=1 vhvhT)ij = n ∑ h=1 (vhvhT)ij = n ∑ h=1 (vh)i(vh)j = (BBT)ij.

Then by last proposition, A is positive semidefinite.

Corollary 4.4. The Hadamard product of two positive semidefinite matrices

is still positive semidefinite.

Proof. Let A =∑k_i=1vivTi and B =

∑l j=1wjw T j, then A◦ B = k,l ∑ i=1,j=1 (vi◦ wj)(vi◦ wj)T.

Therefore by last corollary A_{◦ B is positive semidefinite.} Now the lemma mentioned earlier can be proved.

Lemma 4.5. The polar cone of Pn is itself, i.e. Pn∗ = Pn.

Proof. Let symmetric matrices A, B _{∈ R}n×n _{be positive semidefinite. Note}

tr(AB) = n ∑ i=1 (AB)ii= n ∑ i=1 ∑ 1≤j≤n AijBji = n ∑ i=1 ∑ 1≤j≤n AjiBji = eT(A◦ B)e

where e _{∈ R}n _{is the all-1 vector. Then by corollary 4.4, A◦ B is still positive}

semidefinite and then eT_{(A◦ B)e ≥ 0. Therefore, P}∗ n ⊇ Pn.

On the other hand, suppose A_{∈ Pn}∗. For any column vector x_{∈ R}n_{, the}

n× n symmetric matrix xxT is positive semidefinite since

yT(xxT)y = (xTy)T(xTy) = ( n ∑ i=1 xiyi)( n ∑ i=1 xiyi) = ( n ∑ i=1 xiyi)2 ≥ 0

for any y_{∈ R}n_{. Thus}

xAxT = tr(x(AxT)) = tr(AxTx)≥ 0

for any x∈ Rn_{. Then A}∈ P n.

So the polar cone of Pn is itself, which leads to the next proposition.

Proposition 4.6. A symmetric matrix A is positive semidefinite if and only

if tr(AB)≥ 0 for all symmetric positive semidefinite matrix B.

We immediately have the following corollary.

Corollary 4.7. Let A,B ∈ Rn×n _{be symmetric matrices. If A,B} ≥

p 0, then

tr(AB)≥ 0.

Proposition 4.8. Let A _{∈ P}n and B ∈ Rn×n be a positive definite matrix.

Then tr(AB) = 0 if and only if A = 0.

Proof. It’s clearly tr(0B) = 0. On the other hand, since A ≥p 0 by taking

ei, i.e. the vector with 1 at the ith entry and zero for the rest, we have

Furthermore there exists a n_{× n matrix P such that P}t_{P = P P}t _{= I and}

A′ = PtAP = diag(λ1, ..., λn) is diagonal. Note A′ is still positive

semidefi-nite. Then

0 = tr(AB) = tr(PtP AB) = tr(PtABP ) = tr(PtAP PtBP )

= tr(A′PtBP ) = ∑ 1≤i≤n λi(PtBP )ii = ∑ 1≤i≤n λi(etiP t_{BP e} i) = ∑ 1≤i≤n λi((P ei)tB(P ei))≥ 0

Since (P ei)tB(P ei) > 0 for all 1 ≤ i ≤ n, it follows that λi = 0 for all

1≤ i ≤ n, A′ = 0. Therefore A = 0.

Now we need something else to prove next lemma. A set C ⊆ Rn _is

convex if

(1− t)x + ty ∈ C for all x, y ∈ C, t ∈ [0, 1]. Let _|x0| =

√

x· x for all x ∈ Rn_{, where x · x is the inner product of x and}

x. For any C ⊆ Rn_{, let cl(C) =}{x|x ∈ Rn _{and for any r > 0, there is a y}∈

C such that |x − y| < r}. A subset M of Rn is called an affine set if M =

{x|Bx = b} for some B ̸= 0, B ∈ Rn×n _{and b∈ R}n

A hyperplane inRn _{is a subset which can be written as {x|x ∈ R}n_{, x· b =}

β} for some β ∈ R and b ̸= 0, b ∈ Rn_{. A hyperplane separates R}n _{into two}

parts _{{x|x ∈ R}n_{, x· b ≥ β} and {x|x ∈ R}n_{, x· b < β}. The sets C}

1 and C2 are

separated if there exist a hyperplane H such that C1 and C2 are in different

Proposition 4.9. Let C _{⊂ R}n _{be a convex set and x}

0 ∈ Rn with x0 ∈ cl(C)./

Then there is a hyperplane H separates C and x0.

Proof. Since _Rn _{is compact, we have}

|y∗_{− x}

0| = inf

y∈C|y − x0|

for some y∗ ∈ cl(C). Note |y∗− x0| ̸= 0 since x0 ∈ cl(C). Let/

h(x) = 1 |y∗_{− x0|}2(2(x· (y ∗_{− x} 0)) +|x0|2− |y∗|2) Then h(x0) = 1 |y∗ _{− x0|}2(2(x0· y ∗_{)− 2(x} 0· x0) +|x0|2− |y∗|2) = 1 |y∗_{− x0|}2(2(x0· y ∗_{)− (x} 0· x0)− |y∗|2) = 1 |y∗_{− x0|}2(2(x0· y ∗_{)− (x} 0· x0)− (y∗· y∗)) = −1 |y∗_{− x0|}2(x0− y ∗_{)· (x} 0− y∗) = −1 And h(y∗) = 1 |y∗_{− x0|}2(2(y ∗ _{· (y}∗_{− x} 0)) +|x0|2− |y∗|2) = 1 |y∗_{− x0|}2(2(y ∗_{· y}∗_{)− 2(y}∗_{· x} 0) +|x0|2 − |y∗|2) = 1 |y∗_{− x0|}2((y ∗_{· y}∗_{)− 2(y}∗_{· x} 0) + (x0· x0)) = 1 |y∗_{− x0|}2(y ∗_{− x} 0)· (y∗− x0) = 1

Thus the hyperplane _{{x|h(x) = 0} separates C and x}0 since C is convex.

Proposition 4.10. Let C ⊆ Rn_{be a convex set. Let M} ⊆ Rn _{be a nonempty}

affine set with C _{∩ M = ∅. Then there is an hyperplane H containing M} such that C is contained in one of the two parts that H separates Rn _into.

Proof. By induction on dim(M ). First let S be a subspace with M = S + a

for some a ∈ Rn_{. If dim(S) = n− 1 then M itself is such a hyperplane, so}

we are done.

Suppose it holds for all S with dimension larger than k < n− 1 for some

k. When dim(S) = k− 1, we have dim(S⊥) ≥ 2 and therefore contains a subspace T of dimension 2. The set C− M = {x − y|x ∈ C, y ∈ M} does not contain 0, so we can find a subset L_{⊂ T of dimension 1 such that 0 ∈ L and}

L∩(C −M) = ∅. We now can add the basis of L into the basis of S obtaining

a new subspace S′. Then by induction hypothesis, there is a hyperplane H containing S′ and C is contained in one of two parts that H separates Rn

into. Thus we are done.

Proposition 4.11. Let C1, C2 ⊆ Rnbe nonempty convex sets with C1∩C2 =

∅. Then they are separated.

Proof. Taking C = C1− C2 = {x − y|x ∈ C1, y ∈ C2} and M = {0} in last

proposition. Thus there is a hyperplane H contains 0 and H _{∩ C = ∅. Let}

H = {x|x ∈ Rn, x· b = 0} for some b ∈ Rn. By using −b to replace b if necessary, we have inf x∈Cx· b ≥ 0, sup x∈C x· b > 0. Then 0≤ inf x∈Cx· b = infx1∈C1 x1· b − sup x2∈C1 x2· b, that is inf x1∈C1 x1· b ≥ sup x2∈C1 x2· b.

Then C1 and C2 are separated by hyperplane

H′ ={x|x · b = inf

x1∈C1

x1· b}.

The next lemma is well-known as the semidefinite version of Farkas’ Lemma, which is a similar theorem regarding linear programming.

Lemma 4.12 (Homogenous Version). Let A1, A2, . . ., Am be symmetric

matrices in Rn×n_{. Then the system}

x1A1+ . . . + xmAm >p 0

has no solution in x1, x2, . . ., xm if and only if there exists a symmetric

matrix Y _{̸= 0 such that A}i· Y = tr(AiY ) = 0 for all 1≤ i ≤ m and Y ≥p 0.

Proof. (_⇒)

Since the system has no solution, we have that

{∑

i

xiAi|xi ∈ R} ∩ int(Pn) = ∅

where int(Pn) is the interior of Pn. Moreover, by directly checking the

def-inition, {∑_ixiAi|xi ∈ R} is a convex cone and therefore a convex set. We

can consider Rn×n _asRn2

, thus by last lemma, there is a hyperplane

H ={x|x ∈ Rn×n, x· Y = β}

separates _{∑_ixiAi|xi ∈ R} and Pnfor some Y ∈ Rn×n and β∈ R, which we

may assume

β = inf

P∈Pn

by the definition of hyperplane. Furthermore, we may assume inf

P∈Pn

P · Y ≥ 0

by replacing Y with −Y if necessary. But since Pn is a convex cone, for any

x∈ Pn, rx∈ Pn for all r > 0. By taking r small enough we have

inf

P∈Pn

P · Y = 0

Then our hyperplane

H ={x|x · Y = 0}

implies that (∑_ixiAi)· Y ≤ 0 and P · Y > 0 for all P ∈ int(Pn). Now for a

fixed 1_{≤ i ≤ m, let x}i = 1 and xj = 0 for all i̸= j. Then

0≥ (∑

i

xiAi)· Y = Ai· Y.

We can also let xi =−1 and xj = 0 for all i ̸= j. Then

0≥ (∑

i

xiAi)· Y = −Ai· Y.

These two inequalities show that Ai· Y = 0 for all 1 ≤ i ≤ m. On the other

hand, P _{· Y ≥ 0 for all P ∈ P}n. Then by proposition 4.6, Y ≥p 0.

(_⇐)

Suppose there is a solution of x1A1+ . . . + xmAm >p 0. Then

(∑ i xiAi)· Y = ∑ i xi(Ai· Y ) = 0 By proposition 4.8, Y = 0, a contradiction.

Lemma 4.13 (Nonhomogeneous Version). Let A1, A2, . . ., Am, B be

sym-metric matrices in Rn×n_{. Then the system}

x1A1+ . . . + xmAm− B >p 0

has no solution in x1, x2, . . ., xm if and only if there exists a symmetric

matrix Y _{̸= 0 such that tr(A}iY ) = 0 for all 1 ≤ i ≤ m, tr(BY ) ≥ 0 and

Y ≥p 0.

Proof. This is done by applying last lemma to following matrices

  A1 0 0 0   ,   A2 0 0 0   , ...,   Ak 0 0 0   ,   −B 0 0 1   . We claim that if the system

x1   A1 0 0 0  +x₂   A2 0 0 0  +...+x_k   Ak 0 0 0  +x_k+1   −B 0 0 1   >_p 0

has a solution, then so does the system

x1A1 + . . . + xmAm− B >p 0.

To prove this claim, suppose v1, ..., vk, vk+1 is a solution of

x1   A1 0 0 0  +x2   A2 0 0 0  +...+xk   Ak 0 0 0  +xk+1   −B 0 0 1   >p 0. Let M = ( ∑ 1≤i≤k vi   Ai 0 0 0  ) + v_k+1   −B 0 0 1   . Then M >p 0. Thus 0 < et_k+1M ek+1 = vk+1.

Let ui = _vvi

k+1 for all 1≤ i ≤ k + 1, and

M′ = ( ∑ 1≤i≤k ui   Ai 0 0 0  ) +   −B 0 0 1   = 1 vk+1 M

which is still positive definite. Note that all principal submatrices of a positive definite matrix is again positive definite. Therefore the following principal submatrix of M′ (∑ 1≤i≤k uiAi)− B >p 0. Thus u1, u2, ..., uk is a solution of x1A1 + . . . + xmAm− B >p 0. (⇒)

The claim we proved is equivalent to that if

x1A1+ . . . + xmAm− B >p 0

has no solution, then either the system

x1   A1 0 0 0  +x2   A2 0 0 0  +...+xk   Ak 0 0 0  +xk+1   −B 0 0 1   >p 0.

So by previous lemma, there exists Y _{∈ R}(n+1)×(n+1) _{such that Y ≥}

p 0 and

  Ai 0

0 0 

 · Y = 0 for all 1 ≤ i ≤ k, and   −B 0 0 1   · Y = 0. Let Y =   Y′ y yT _y 0  

for some y _{∈ R}n_{, y}

0 ∈ R, then

Ai· Y′ = 0 for all 1≤ i ≤ k.

Since Y ≥p 0, we take en+1 and we have

0≤ eT_n+1Y en+1 = y0. Together with 0 =   −B 0 0 1   · Y = −BY′_{+ y} 0, we conclude that BY′ = y0 ≥ 0.

Note Y′ ≥p 0 since Y′ is a principal submatrix of Y ≥p 0.

(⇐)

By assumption there exists Y ̸= 0 such that Ai·Y = tr(AiY ) = 0 for all 1 ≤

i≤ m, Y ≥p 0 and tr(BY ) ≥ 0. Let

Z =   Y 0 0 B· Y   .

Suppose there is a solution of x1A1 + . . . + xmAm− B >p 0. Then so does

the system x1   A1 0 0 0  +x₂   A2 0 0 0  +...+x_k   Ak 0 0 0  +x_k+1   −B 0 0 1   >_p 0. But Z ·   A1 0 0 0   =   Y 0 0 B· Y   ·   A1 0 0 0   = 0 for all 1 ≤ i ≤ m,

and Z·   −B 0 0 1   =   Y 0 0 B· Y   ·   −B 0 0 1   = −B · Y + B · Y = 0. Note Z _≥p 0. However, since the system

x1   A1 0 0 0  +x2   A2 0 0 0  +...+xk   Ak 0 0 0  +xk+1   −B 0 0 1   >p 0

has solution, such Z should not exist by last lemma, a contradiction.

5 Property of Semidefinite programming

Prob-lems

A SDPn(c, F0, F1, . . . , Fm) is said to have feasible solution if there exists

x with F (x) ≥p 0, and is said to be strictly feasible if F (x) >p 0 for some

x in the feasible region, where as before F (x) = F0+

∑m

i=1xiFi. In this case

x is called a strictly feasible solution. Similarly SDP∗n(c, F0, F1, . . . , Fm)

is said to have feasible solution if there is some Z in the feasible region, and Z is said to be strictly feasible if Z >p 0. In this case Z is called a

strictly feasible solution.

Theorem 5.1. Let p∗ and d∗ be the infimum of SDPn(c, F0, F1, . . . , Fm) and

supremum of SDP∗n(c, F0, F1, . . . , Fm) respectively, and assume p∗, d∗ <∞.

Then p∗ _{≥ d}∗. Moreover, suppose either of the following conditions (i)-(ii) holds.

(i) The primal problem is strictly feasible.

(ii) The dual problem is strictly feasible.

Then p∗ = d∗.

Proof. Let x be a vector in the feasible region of primal problem and Z be a

symmetric matrix in the feasible region of its dual problem. Then referring to (2) and by corollary 4.7, cTx + tr(ZF0) = m ∑ i=1 tr(ZFixi) + tr(ZF0) = tr(ZF (x))≥ 0. Thus cT_{x≥ −tr(ZF} 0) which shows p∗ ≥ d∗.

Now the system

cTx < p∗ F (x)≥p 0

has no solution x _{∈ R}m _{by the definition of p}∗_{. Therefore if we define the}

matrices F₀′ =   p∗ 0 0 F0   and F′ i =   −ci 0 0 Fi   for 1 ≤ i ≤ m,

F₀′ + x1F1′ + . . . + xmFm′ >p 0 has no solution in Rn. Thus by lemma 4.13

there is a positive semidefinite matrix Y ̸= 0 such that

tr(F₀′Y )≥ 0 and tr(F_i′Y ) = 0 for 1≤ i ≤ m. By letting Y =   y00 y yT _Z 

we obtain

tr(F0Z)≥ y00p∗ and tr(FiZ) = y00ci for 1≤ i ≤ m.

We claim that y00̸= 0. Suppose not. Then

tr(F0Z)≥ 0 and tr(FiZ) = 0 for 1≤ i ≤ m.

Therefore by lemma 4.13, the existence of Z implies F0 + x1F1 + . . . +

xmFm >p 0 has no solution inRn, contradicts to the hypothesis that primal

problem is strictly feasible. Thus, y00 ̸= 0.

Now y00̸= 0 and since Y is positive semidefinite, y00> 0. By scaling we may

assume y00= 1. But then Z satisfies tr(ZF0)≥ y00p∗ = p∗ and d∗ ≥ tr(ZF0)

by definite of d∗, thus d∗ _{≥ p}∗. Together with first part of the proof we have

p∗ = d∗.

The case that condition (ii) holds is similar to prove. Therefore the proof is complete.

6 References

[1] Lieven Vandenberghe and Stephen Boyd, Semidefinite Programming, SIAM Review, Vol. 38, No. 1. (Mar., 1996), pp. 49-95.

[2] L´aszl´o Lov´asz, Semidefinite programs and combinatorial optimization, lecture notes, 1995-2001.

[3] Michel X. Goemans and David P. Williamson, Improved approximation

algorithms for maximum cut and satisfiability problems using semidefi-nite programming, J. ACM, Vol. 42, pp. 1115-1145, 1995.

[4] Abraham Berman and Naomi Shaked-Monderer, Completely positive

matrices, World Scientific Publishing Company, 2003.

[5] Konstantin Aslanidi, Notes on Quantitative Analysis in Finance, Avail-able: http://www.opentradingsystem.com, 2007.