Smoothing methods for Second-Order Cone Programs/Complementarity Problems

Programs/Complementarity Problems

Paul Tseng

Mathematics, University of Washington Seattle

Kyungpook National University, Daegu February 2008

Abstract

This is a talk given at Pukyong National University.

Talk Outline

• Second-Order Cone (SOC) Program and Complementarity Problem

• Unconstrained Diff. Min. Reformulation

• Numerical Experience

• SOCP from Dist. Geometry Optim

Convex SOCP

min g(x) s.t. Ax = b

x ∈ K

A ∈ <^m×n, b ∈ <^m

g : <ⁿ → <, convex, twice cont. diff.

K = Kⁿ¹ × · · · × K^np K^{ni def}= n

x_i = h

x_1i x_2i

i ∈ < × <ⁿⁱ⁻¹ : kx_2ik₂ ≤ x_1io

Special cases? LP, SOCP,...

SOC K

Suff. Optim. Conditions

x ∈ K, y ∈ K, x^Ty = 0, Ax = b, y = ∇g(x) − A^Tζ_d

⇐⇒

x ∈ K, y ∈ K, x^Ty = 0, x = F (ζ), y = G(ζ) with F (ζ) = d + (I − A^T(AA^T)⁻¹A)ζ

G(ζ) = ∇g(F (ζ)) − A^T(AA^T)⁻¹Aζ (Ad = b)

SOCCP

Find ζ ∈ <ⁿ satisfying

x ∈ K, y ∈ K, x^Ty = 0, x = F (ζ), y = G(ζ) F, G : <ⁿ → <ⁿ smooth

∇F (ζ), −∇G(ζ) column-monotone ∀ζ ∈ <ⁿ, i.e.,

∇F (ζ)u − ∇G(ζ)v = 0 ⇒ u^Tv ≥ 0

Special cases? convex SOCP, monotone NCP,...

How to solve SOCCP?

For LP, simplex methods and interior-point methods.

For SOCP, interior-point methods.

For convex SOCP and column-monotone SOCCP?

Interior-point methods not amenable to warm start. Non-interior methods?

Nonsmooth Eq. Reformulation

x_i · y_i ^def= h

x_1i x_2i

i · h

y_1i y_2i

i

= h

x^T_i y_i x_1iy_2i+y_1ix_2i

i

(Jordan product assoc. with Kⁿⁱ)

φ_FB(x, y) ^def= h

(x²_i + y_i²)^1/2 − x_i − y_ii^p

i=1

Fact (Fukushima,Luo,T ’02):

φ_FB(x, y) = 0 ⇐⇒ x ∈ K, y ∈ K, x^Ty = 0

Thus, SOCCP is equivalent to

φ_FB(F (ζ), G(ζ)) = 0

φ_FB is strongly semismooth (Sun,Sun ’03)

Unconstr. Smooth Min. Reformulation

min f_FB(ζ) ^def= kφ_FB(F (ζ), G(ζ))k²

2

F, G smooth and ∇F (ζ), −∇G(ζ) column-monotone ∀ζ ∈ <ⁿ (e.g., LP, SOCP, convex SOCP, monotone NCP)

For monotone NCP (K = <ⁿ₊),

f_FB is smooth, and ∇f_FB(ζ) = 0 ⇐⇒ ζ is a soln

(Geiger,Kanzow ’96)

The same holds for SOCCP. (J.-S. Chen,T ’04)

Advantage? Any gradient method for unconstrained diff. min. (e.g., CG, BFGS, L-BFGS) can be used to find ∇f_FB(ζ) = 0.

Numerical Experience on Convex SOCP

x = F (ζ) = d + (I − P )ζ y = G(ζ) = ∇g(F (ζ)) − P ζ

with P = A^T(AA^T)⁻¹A, Ad = b. ^(Solve min kAx − bk to find d)

• Implement in Matlab CG-PR, BFGS, L-BFGS (memory=5) to minimize f_FB(ζ), using Armijo stepsize rule, with ζ^init = 0. Stop when

max{f_FB(ζ), |x^Ty|} ≤ accur.

• Let ψ_FB(x, y) ^def= kφ_FB(x, y)k²

2. Then f_FB(ζ) = ψ_FB(x, y)

∇f_FB(ζ) = (I − P )∇_xψ_FB(x, y) − P ∇_yψ_FB(x, y)

Compute P ζ using Cholesky factorization of AA^T or using preconditioned CG. Compute ψ_FB(x, y) and ∇ψ_FB(x, y) within Fortran Mex files.

DIMACS Challenge SOCPs

• Problem names and statistics:

nb (m = 123, n = 2383, K = (K³)⁷⁹³ × <⁴₊)

nb-L2 (m = 123, n = 4195, K = K¹⁶⁷⁷ × (K³)⁸³⁸ × <⁴₊)

nb-L2-bessel (m = 123, n = 2641, K = K¹²³ × (K³)⁸³⁸ × <⁴₊)

Compare iters/cpu(sec)/accuracy with Sedumi 1.05 (Sturm ’01), which implements a predictor-corrector interior-point method.

Problem SeDuMi (pars.eps=1e-5) L-BFGS-Chol (accur=1e-5)

Name iter/cpu iter/cpu

nb 19/7.6 1042/16.5

nb-L2 11/11.1 330/9.2

nb-L2-bessel 11/5.3 108/1.7

Table 1: (cpu times are in sec on an HP DL360 workstation, running Matlab 6.1)

Regularized Sum-of-Norms Problems

min_w≥0 PM

i=1 kA_iw − b_ik₂ + h(w),

A_i ∼ U[−1, 1]^mi×`, b_i ∼ U[−5, 5]^mi, m_i ∼ U{2, 3, ..., r} (r ≥ 2).

h(w) = 1^Tw + ¹₃kwk³₃ (cubic reg.) Reformulate as a convex SOCP:

minimize PM

i=1 z_i + h(w)

subject to A_iw + s_i = b_i, (z_i, s_i) ∈ K^mi+1, _i=1,...,M, w ∈ <^`₊.

Problem BFGS-Chol CG-PR-Chol L-BFGS-Chol

`, M, r (m, n) iter/cpu iter/cpu iter/cpu 500,10,10 (56,566) 352/24.6 1703/6.6 497/2.4 500,50,10 (283,833) 546/85.1 3173/69.0 700/12.4 500,10,50 (246,756) 272/36.3 1290/23.0 371/5.6

Table 2: (cpu times are in sec on an HP DL360 workstation, running Matlab 6.5.1, with accur=1e-3)

Smoothing Newton Step

φ^µ

FB(x, y) ^def= (x² + y² + µ²e)^1/2 − x − y

with e = (1, 0, .., 0

| {z }

n₁

, ..., 1, 0, .., 0

| {z }

n_p

)^T, µ > 0 (Fukushima,Luo,T ’02)

Given ζ, choose µ > 0 and solve

∇φ^µ

FB(F (ζ), G(ζ))^T∆ζ = −φ_FB(F (ζ), G(ζ)) Use ∆ζ to accelerate convergence.

This requires more work per iteration. Use it judiciously.

Observations

For our unconstrained smooth merit function approach:

Advantage:

• Less work/iteration, simpler matrix computation than interior-point methods.

• Applicable to convex SOCP and column-monotone SOCCP.

• Useful for warm start?

Drawback:

• Many more iters. than interior-point methods.

• Lower solution accuracy.

SOCP from Dist. Geometry Optim

n pts in <^d (d = 2, 3).

Know x_m+1, ..., x_n and Eucl. dist. estimate for pairs of ‘neighboring’ pts d_ij > 0 ∀(i, j) ∈ A ⊆ {1, ..., n} × {1, ..., n}.

Estimate x₁, ..., x_m.

Problem (nonconvex):

x₁min,...,x_m

X

(i,j)∈A

kx_i − x_jk²₂ − d²_ij 

• Objective function is nonconvex.

6. .

_

• Problem is NP-hard (reduction from PARTITION). ⁶_{_}

• Use a convex (SDP, SOCP) relaxation. SOCP is much easier to solve than SDP.

SOCP Relaxation

υ_opt = min

x₁,...,x_m,y_ij

X

(i,j)∈A

y_ij − d²_ij 

s.t. y_ij = kx_i − x_jk²₂ ∀(i, j) ∈ A

Relax “=” to “≥” constraint:

υ_socp = min

x₁,...,x_m,y_ij

X

(i,j)∈A

y_ij − d²_ij 

s.t. y_ij ≥ kx_i − x_jk²₂ ∀(i, j) ∈ A

y ≥ kxk²₂ ⇐⇒ y + 1 ≥ k(y − 1, 2x)k₂

(also Doherty,Pister,El Ghaoui ’03)

Simulation Results

Uniformly generate x˜₁, ..., ˜x_n in [−.5, .5]², m = 0.9n

˜

x_i and x˜_j are neighbor if dist< .06. Set d_ij = k˜x_i − ˜x_jk₂ (Biswas, Ye ’03)

True soln

(m = 900, n = 1000)

SOCP soln found by IP method or smoothing method

Lastly...

Thank you for coming!

6. .

^