VARIATIONAL ANALYSIS OF CIRCULAR CONE PROGRAMS
JINCHUAN ZHOU^{1}, JEIN-SHAH CHEN^{2} and BORIS S. MORDUKHOVICH^{3}

Abstract. This paper conducts variational analysis of circular programs, which form a new class of optimization problems in nonsymmetric conic programming important for optimization theory and its applications. First we derive explicit formulas in terms of the initial problem data to calculate various generalized derivatives/coderivatives of the projection operator associated with the circular cone.

Then we apply generalized differentiation and other tools of variational analysis to establish complete characterizations of full and tilt stability of locally optimal solutions to parameterized circular programs.

Keywords. variational analysis, optimization, generalized differentiation, conic programming, circular cone, second-order cone, projection operator, full and tilt stability

AMS subject classifications. 90C30, 90C31, 49J52

### 1 Introduction

The circular cone [5, 34] is a pointed, closed, convex cone having hyperspherical sections orthogonal to its
axis of revolution about which the cone is invariant to rotation. Let its half-aperture angle be θ ∈ (0,^{π}_{2}).

Then the n-dimensional circular cone denoted by L_{θ}can be expressed as follows (see Figure 1):

Lθ: =x = (x1, x_{2}) ∈ IR × IR^{n−1}

kxk cos θ ≤ x1

=x = (x1, x_{2}) ∈ IR × IR^{n−1}

kx_{2}k ≤ x_{1}tan θ . (1.1)
When θ = 45^{◦}, the circular cone reduces to the well-known second-order cone (SOC for short, also

(a) 0 < θ < 45^{◦} (b) θ = 45^{◦} (c) 45^{◦}< θ < 90^{◦}

Figure 1: The graphs of circular cones.

1Department of Mathematics, School of Sciences, Shandong University of Technology. Zibo 255049, P.R. China (jinchuanzhou@163.com). Research of this author was partly supported by the National Natural Science Foundation of China under grant 11101248, 11271233 and by the Shandong Province Natural Science Foundation under grant ZR2010AQ026, ZR2012AM016.

2Department of Mathematics, National Taiwan Normal University, Taipei 11677, Taiwan (jschen@math.ntnu.edu.tw).

Research of this author was supported by Ministry of Science and Technology, Taiwan.

3Department of Mathematics, Wayne State University, Detroit, MI 48202 and King Fahd University of Petroleum and Minerals, Dhahran 31261, Saudi Arabia (boris@math.wayne.edu). Research of this author was partly supported by the USA National Science Foundation under grant DMS-1007132.

known as the Lorentz cone and the ice-cream cone) given by
K^{n}: =x = (x1, x_{2}) ∈ IR × IR^{n−1}

kx2k ≤ x1

=x = (x1, x_{2}) ∈ IR × IR^{n−1}

kxk cos 45^{◦}≤ x_{1} . (1.2)
Concerning SOC, for any vector x = (x1, x2) ∈ IR × IR^{n−1} we can decompose it as

x = λ_{1}(x)u^{(1)}_{x} + λ_{2}(x)u^{(2)}_{x} , (1.3)
where λ1(x), λ2(x) and u^{(1)}x , u^{(2)}x are the spectral values and the associated spectral vectors of x relative
to K^{n} defined by, respectively,

λi(x) : = x1+ (−1)^{i}kx2k,

u^{(i)}_{x} : =

1 2

1, (−1)^{i} x2

kx2k

if x26= 0,

1 2

1, (−1)^{i}w

if x2= 0, i = 1, 2,

with w being any unit vector in IR^{n−1}. If x26= 0, decomposition (1.3) is unique. Using this decomposi-
tion, for any f : IR → IR we consider [3, 4] the vector function associated with K^{n}, n ≥ 1 by

f^{soc}(x) := f (λ_{1}(x))u^{(1)}_{x} + f (λ_{2}(x))u^{(2)}_{x} , x = (x_{1}, x_{2}) ∈ IR × IR^{n−1}. (1.4)
If f is defined only on some subset of IR, then f^{soc} is defined on the corresponding subset of IR^{n}.
Definition (1.4) is unambiguous whether x_{2}6= 0 or x2= 0.

Note that circular cone systems described by (1.1) with θ 6= 45^{◦} naturally arises in many real-life
engineering problems. In particular, we refer the reader to the recent paper [11] and the bibliographies
therein to the important class of optimal grasping manipulation problems for multi-fingered robots in
which the grasping force of the i-th finger is subject to a contact friction constraint given by

(ui2, ui3)

≤ µui1, (1.5)

where ui1is the normal force of the i-th finger, ui2 and ui3 are the friction forces of the i-th finger, k · k is the 2-norm, and µ is the friction coefficient; see Figure 2.

Figure 2: The grasping force forms a circular cone where α = tan^{−1}µ < 45^{◦}.

It is easy to see that (1.5) is a circular cone constraint corresponding to the description ui= (ui1, ui2, ui3) ∈
Lθ in (1.1) with the angle θ = tan^{−1}µ < 45^{◦}.

Observe that a possible way to deal with circular cone constraints is to scale L_{θ}as SOC by
Lθ= A^{−1}K^{n} and K^{n}= AL_{θ} with A =tan θ 0

0 I

, (1.6)

which is justified in [34, Theorem 2.1]. However, this approach may not be acceptable from both theoret- ical and numerical viewpoints. Indeed, the “scaling” step can cause undesirable numerical performance due to round-off errors in computers, which has been confirmed by experiments. Furthermore, we will see in what follows that applying (1.6) does not help to obtain some major results of the paper while being useful in deriving the other ones.

Optimization problems with both SOC and circular cone constraints belong to a broad and important class in modern optimization theory known as conic or cone-constrained programming; see, e.g., [1, 2, 22]

and the references therein. However, the main difference between circular cone constraints and those given by SOC and most of the other constraint systems in conic programming is that the circular cone Lθ is non-self-dual, i.e., nonsymmetric, which makes its study more challenging and rather limited.

In contrast to symmetric conic programming, we are not familiar with a variety of publications de-
voted to their nonsymmetric counterparts. Referring the reader to [14, 26, 32, 33] and the bibliographies
therein, observe that there is no any unified way to handle nonsymmetric cone constraints, and each
study uses certain specific features of the nonsymmetric cones under consideration. The previous papers
[34, 35] concerning the circular cone show that some properties holding in the SOC framework can be
extended to the circular cone setting. At the same time, some other SOC properties fail to be satisfied
for the general nonsymmetric circular cone, where the angle θ 6= 45^{◦} plays a crucial role; see [36].

This paper is mainly devoted to two major interrelated issues of variational analysis and optimization for problems involving circular cone constraints. Our first goal is to calculate, entirely in terms of the initial circular cone data, some generalized differential constructions of variational analysis that have been proven to be important for various aspects of optimization. Namely, we derive explicit formulas to calculate generalized differential constructions for the (metric) projection operator associated with the general circular cone that are known as the B-subdifferential, directional derivative, graphical deriva- tive, regular derivative, regular coderivative, and (limiting) coderivative. Except the B-subdifferential and the (regular and limiting) coderivatives, the results obtained are new even for the symmetric SOC case. The obtained calculations allow us, in particular, to prove the strong semismoothness of the pro- jection operator onto the circular cone, which is important for many applications including those to numerical optimization. Furthermore, we establish new relationships between these generalized differen- tial constructions for the projection operator onto the circular cone and the metric projection onto the orthogonal spaces to the spectral vectors in the circular cone representation.

The second major goal of this paper is to completely characterize the notions of tilt stability and full stability of mathematical programs with circular cone constraints. These fundamental stability concepts were introduced in optimization theory by Rockafellar and his collaborators [12, 28] and then have been intensively studied by many researchers, especially in the recent years, for various classes of optimization problems; see, e.g., [2, 7, 8, 13, 17, 18, 19, 20, 21, 22, 23, 24, 25, 28] and the references therein. The construction of the second-order subdifferential/generalized Hessian in the sense of Mordukhovich [15]

(i.e., the coderivative of the first-order subgradient mapping) plays a crucial role in the characterization of tilt and full stability obtained in the literature. In this paper we establish, by using the obtained second-order calculations and the recent results of [23], complete characterizations of full and tilt stability for locally optimal solutions to mathematical programs with circular cone constraints expressed entirely in terms of the initial program data via certain second-order growth and strong sufficient optimality conditions under appropriate constraint qualifications.

The rest of the paper is organized as follows. In Section 2 we recall and briefly discuss the generalized differential constructions of variational analysis employed in deriving the main results of this paper.

Section 3 is devoted to calculating the generalized derivatives listed above for the projection operator onto the circular cone. In Section 4 we represent these generalized differential constructions for the

aforementioned projection operator via the orthogonal projections generated by the spectral vectors of the circular cone. Finally, Section 5 applies the second-order subdifferential of the indicator function associated with the circular cone and related to the above coderivative calculations to establish complete characterizations of full and tilt stability of mathematical programs with circular cone constraints.

Throughout the paper we use the standard notation and terminology of variational analysis; see, e.g.,
[16, 31]. Given a set-valued mapping/multifunction F : R^{n} →→R^{m}, recall that the constructions

Lim sup

x→¯x

F (x) :=n

y ∈ R^{m}

∃ sequences xk→ ¯x, yk → y such that yk ∈ F (xk) for all k ∈ IN := {1, 2, . . .}o

,

(1.7)

Lim inf

x→¯x F (x) :=n

y ∈ R^{m}

for any xk→ ¯x, ∃ yk → y such that

yk ∈ F (xk) for all k ∈ INo (1.8)
are known as the (Painlev´e-Kuratowski) outer limit and inner limit of F as x → ¯x, respectively. For a
set Ω ⊂ R^{n}, the symbol x→ ¯^{Ω} x signifies that x → Ω with x ∈ Ω.

### 2 Tools of Variational Analysis

In this section we briefly review those tools of generalized differentiation in variational analysis, which are widely used in the subsequent sections. We start with geometric notions.

Given a set Ω ⊂ IR^{n} locally closed around x ∈ Ω, the (Bouligand-Severi) tangent/contingent cone to
Ω at ¯x ∈ Ω is defined by

TΩ(¯x) := Lim sup

t↓0

Ω − ¯x t =n

d ∈ IR^{n}

∃tk ↓ 0, dk→ d with ¯x + tkdk ∈ Ωo

(2.1) via the outer limit (1.7), while the (Clarke) regular tangent cone to Ω at ¯x ∈ Ω is given by

Tb_{Ω}(¯x) := Lim inf

x→¯^{Ω}x

T_{Ω}(x) (2.2)

via the inner limit (1.8). The (Fr´echet) regular normal cone to Ω at ¯x ∈ Ω is NbΩ(¯x) :=n

z ∈ IR^{n}

hz, x − ¯xi ≤ o kx − ¯xk

for all x ∈ Ωo

, (2.3)

and the (Mordukhovich, limiting) normal cone to Ω at ¯x ∈ Ω can be equivalently defined by NΩ(¯x) := Lim sup

x→¯^{Ω}x

NbΩ(x) = Lim sup

x→¯x

nconex − ΠΩ(x)o

, (2.4)

where Π_{Ω} denotes the (Euclidean) projection operator onto Ω, and where “cone” stands for the conic
(may not be convex) hull of the set in question.

Consider next a set-valued mapping H : IR^{n}⇒ IR^{m}with its graph and domain given by
gph H :=(x, y) ∈ IR^{n}× IR^{m}

y ∈ H(x)

and dom H :=x ∈ IR^{n}

H(x) 6= ∅ , respectively. The graphical derivative of H at (¯x, ¯y) ∈ gph H is defined by

DH(¯x, ¯y)(w) :=z ∈ IR^{m}

(w, z) ∈ TgphH(¯x, ¯y) , w ∈ R^{n}, (2.5)
via the tangent cone (2.1), while the (limiting) coderivative is defined via the normal cone (2.4) by

D^{∗}H(¯x, ¯y)(y^{∗}) :=x^{∗}∈ IR^{n}

(x^{∗}, −y^{∗}) ∈ NgphH(¯x, ¯y) , y^{∗}∈ R^{m}, (2.6)

where we drop ¯y in the derivative/coderivative notion if H is single-valued at ¯x. Similarly, the regular derivative and the regular coderivative of H at (¯x, ¯y) is defined via, respectively, (2.2) and (2.3) by

DH(¯b x, ¯y)(w) :=z ∈ IR^{m}

(w, z) ∈ bTgphH(¯x, ¯y) , w ∈ R^{n}, (2.7)
Db^{∗}H(¯x, ¯y)(y^{∗}) :=x^{∗}∈ IR^{n}

(x^{∗}, −y^{∗}) ∈ bNgphH(¯x, ¯y) , y^{∗}∈ R^{m}. (2.8)
Now let f : IR^{n} → R := (−∞, ∞] be an extended-real-valued function finite at ¯x ∈ R^{n}. To define the
second-order subdifferential construction needed in what follows, we proceed in the way of [15, 16] and
begin with the first-order (limiting) subdifferential of f at ¯x given by

∂f (¯x) :=v ∈ R^{n}

(v, −1) ∈ N_{epif} x, f (¯¯ x)

(2.9)
via the normal cone (2.4) of the epigraph {(x, µ) ∈ R^{n}× R| µ ≥ f(x)} of f. Observe the representation
NΩ(¯x) = ∂δΩ(¯x) the normal cone (2.4) via the subdifferential (2.9) of the indicator function δΩ(x) of Ω
equal to 0 if x ∈ Ω and ∞ otherwise. The second-order subdifferential (or generalized Hessian) of f at

¯

x relative to ¯y ∈ ∂f (¯x) is defined as the coderivative (2.6) of the first-order subdifferential (2.9) by

∂^{2}f (¯x, ¯y)(u) := (D^{∗}∂f )(¯x, ¯y)(u), u ∈ IR^{n}. (2.10)
Finally in this section, consider a single-valued mapping F : IR^{n}→ IR^{m}locally Lipschitzian around ¯x
and recall that F is almost everywhere differentiable in a neighborhood of ¯x with the derivative ∇F (x)
by the classical Rademacher theorem; see [31]. Then the B-subdifferential of F at ¯x is defined by

∂BF (¯x) :=

xlimk→x∇F (xk)

F is differentiable at xk

. (2.11)

Recall also that F is directionally differentiable at ¯x if the limit
F^{0}(x; h) := lim

t→0^{+}

F (x + th) − F (x)

t exists for all h ∈ IR^{n}. (2.12)

Having this, F is said to be semismooth at ¯x if F is locally Lipschitzian around ¯x, directionally differen- tiable at this point, and satisfies the relationship

V h − F^{0}(x; h) = o khk

for any V ∈ co ∂BF (x + h) as h → 0. (2.13) Furthermore, F is ρ-order semismooth at x with 0 < ρ < ∞ if (2.13) is replaced above by

V h − F^{0}(x; h) = O khk^{1+ρ}

for any V ∈ co ∂BF (x + h) as h → 0. (2.14) The case of ρ = 1 in (2.13) corresponds to strongly semismooth mappings.

### 3 Generalized Differentiation of the Projection Operator onto the Circular Cone

In this section we derive precise formulas for calculating the above generalized derivatives of the projec-
tion operator onto the circular cone (1.1). First we recall the following spectral decomposition from [34,
Theorem 3.1] of any vector x = (x1, x2) ∈ IR × IR^{n−1}relative to the circular cone Lθ:

x = λ_{1}(x)u^{1}_{x}+ λ_{2}(x)u^{2}_{x}, (3.1)

where the spectral values λ1(x) and λ2(x) are defined by

λ1(x) := x1− kx2kctanθ, λ2(x) := x1+ kx2k tan θ, (3.2)

and where the spectral vectors u^{1}_{x}and u^{2}_{x}are written as

u^{1}_{x}:= 1
1 + ctan^{2}θ

1 0

0 ctanθ

1

−¯x_{2}

, u^{2}_{x}:= 1
1 + tan^{2}θ

1 0

0 tan θ

1

¯
x_{2}

(3.3)

with ¯x_{2}:= x_{2}/kx_{2}k if x_{2}6= 0 and ¯x_{2}equal to any unit vector w ∈ IR^{n−1}otherwise. Given any f : R → R
we construct the vector function

f^{L}^{θ}(x) := f λ1(x)u^{1}_{x}+ f λ2(x)u^{2}_{x}. (3.4)
associated with circular cone. It follows from [34] that the projection Π_{L}_{θ}(x) of x onto Lθ, which is a
single-valued and Lipschitzian operator, corresponds to f (t) := (t)+= max{t, 0} in (3.4), i.e., we have

Π_{L}_{θ}(x) = x_{1}− kx2kctanθ

+u^{1}_{x}+ x_{1}+ kx_{2}k tan θ

+u^{2}_{x}. (3.5)

Our first result in this section provides a complete calculation of the B-subdifferential (2.11) of the projection operator (3.5) entirely in terms of the initial data of the general circular cone (1.1). This result is widely used in what follows.

Lemma 3.1 (calculating the B-subdifferential of the projection operator). For any x ∈ IR^{n}
with the spectral decomposition (3.1), the B-subdifferential of the projection operator ΠL_{θ} is calculated
as follows:

(a) If λ1(x)λ2(x) 6= 0, then Π_{L}_{θ} is differentiable at x and ∂B(Π_{L}_{θ})(x) = {∇Π_{L}_{θ}(x)}.

(b) If λ1(x) = 0 and λ2(x) > 0, then

∂B(ΠL_{θ})(x) =

I, I + 1

tan θ + ctanθ

− tan θ x¯2

¯

x2 −ctanθ¯x2x¯^{T}_{2}

.

(c) If λ1(x) < 0 and λ2(x) = 0, then

∂B(Π_{L}_{θ})(x) =

0, 1

tan θ + ctanθ

ctanθ x¯^{T}_{2}

¯

x2 tan θ ¯x2x¯^{T}_{2}

.

(d) If λ_{1}(x) = λ_{2}(x) = 0, then

∂B(Π_{L}_{θ})(x)

=

( 1

tan θ + ctanθ

"ctanθ w^{T}

w

tan θ + ctanθ aI +h

tan θ − a(ctanθ + tan θ)i
ww^{T}

#

a ∈ [0, 1]

kwk = 1 )

[ 0, I

.

Proof. In case (a) the function f (t) = (t)_{+} is differentiable at λ_{i}(x) for i = 1, 2. Hence it follows from
[35, Theorem 2.3] that Π_{L}_{θ} is also differentiable at x. Furthermore, in this case we have by (3.5) that

Π_{L}_{θ}(x) =

x if λ_{1}(x) > 0 and λ_{2}(x) > 0,
0 if λ_{1}(x) < 0 and λ_{2}(x) < 0,
(x1+ kx2k tan θ)u^{2}_{x} if λ1(x) < 0 and λ2(x) > 0.

In particular, kx2k 6= 0 when λ1(x) < 0 and λ2(x) > 0, and thus ∇kx2k = ¯x2. This gives us

∂BΠL_{θ}(x) =∇ΠL_{θ}(x) ,

where the derivative of Π_{L}_{θ} at x is calculated by

∇ΠL_{θ}(x) =

I if λ1(x) > 0 and λ2(x) > 0, 0 if λ1(x) < 0 and λ2(x) < 0, 1

tan θ + ctanθ

ctanθ x¯^{T}_{2}

¯ x2

x1+ kx2k tan θ

kx2k I − x1

kx2kx¯2x¯^{T}_{2}

if λ_{1}(x) < 0 and λ_{2}(x) > 0.

(3.6)
In case (b) we have kx_{2}k 6= 0, and so it follows from [35, Theorem 3.1] that

∂_{B} f^{L}^{θ}(x) =

ξ %¯x^{T}_{2}

%¯x2 aI + (η − a)¯x2x¯^{T}_{2}

a = f λ_{2}(x) − f λ1(x)
λ2(x) − λ1(x)
ξ − %ctanθ ∈ ∂Bf (λ1(x))
ξ + % tan θ ∈ ∂_{B}f (λ_{2}(x))
η = ξ − %(ctanθ − tan θ)

.

This implies by the obvious calculation

∂_{B}(t)_{+}=

1 for t > 0, {0, 1} for t = 0, 0 for t < 0 that the B-subdifferential of the projection operator is represented as

∂B(Π_{L}_{θ})(x) =

ξ %¯x^{T}_{2}

%¯x_{2} aI + (η − a)¯x_{2}¯x^{T}_{2}

a = 1 ξ − %ctanθ ∈ {0, 1}

ξ + % tan θ = 1 η = ξ − %(ctanθ − tan θ)

. (3.7)

Analyzing (3.7) in the case of ξ − %ctanθ = 1 and ξ + % tan θ = 1 shows that ξ = 1, % = 0, and η = 1.

Hence (3.7) reduces in this case to I. For ξ − %ctanθ = 0 we know that ξ = ctanθ

tan θ + ctanθ, % = 1

tan θ + ctanθ, and η = tan θ tan θ + ctanθ, and so equation (3.7) in this case takes the form of

ctanθ tan θ + ctanθ

1

tan θ + ctanθx¯^{T}_{2}
1

tan θ + ctanθx¯_{2} I +

tan θ

tan θ + ctanθ − 1

¯
x_{2}x¯^{T}_{2}

= I + 1

tan θ + ctanθ

− tan θ x¯^{T}_{2}

¯

x2 −ctanθ¯x2x¯^{T}_{2}

,

which gives us the B-subdifferential representation

∂B ΠL_{θ}(x) =

I, I + 1

tan θ + ctanθ

− tan θ x¯_{2}

¯

x2 −ctanθ¯x2x¯^{T}_{2}

.

In case (c) we also have x26= 0. Similarly to case (b), it is not hard to verify that

∂B ΠL_{θ}(x) =

0, 1

tan θ + ctanθ

ctanθ x¯^{T}_{2}

¯

x2 tan θ ¯x2x¯^{T}_{2}

.

It remains to consider case (d) when x = 0. Then the result of [35, Theorem 3.4] tells us that

∂B ΠL_{θ}(x)

=

ξ %w^{T}

%w aI + (η − a)ww^{T}

either a = ξ ∈ {0, 1}, % = 0 or a ∈ [0, 1]

ξ − %ctanθ = 0 ξ + % tan θ = 1

η = ξ − %(ctanθ − tan θ) kwk = 1

=

( 1

tan θ + ctanθ

"ctanθ w^{T}

w

tan θ + ctanθ aI +

tan θ − a(ctanθ + tan θ)
ww^{T}

#

a ∈ [0, 1]

kwk = 1 )

[ (

0, I )

,

which thus completes the proof of the lemma. 2

Our next goals is to verify the directional differentiability of the projection operator (3.5) and derive
formulas for calculating its directional derivative (2.12). Observe to this end that the result of [35,
Theorem 2.2] tells us that the vector function f^{L}^{θ} from (3.4) is directionally differentiable at x provided
that f is directionally differentiable at λi(x) for i = 1, 2. Moreover, for x2= 0 we have

f^{L}^{θ}^{0}

(x; h) = 1

1 + ctan^{2}θf^{0}

x1; h1− kh2kctanθ

1 0

0 ctanθ

1

−¯h_{2}

+ 1

1 + tan^{2}θf^{0}

x1; h1+ kh2k tan θ

1 0

0 tan θ

1

¯h2

(3.8)

= f^{0}

x1; h1− kh2kctanθ

u^{1}_{h}+ f^{0}

x1; h1+ kh2k tan θ
u^{2}_{h}.
On the other hand, for x26= 0 we denote

Mx_{2} :=

0 0

0 I − x2x^{T}_{2}
kx2k^{2}

and arrive at the following relationships:

f^{L}^{θ}^{0}
(x; h)

= 1

1 + ctan^{2}θf^{0}

λ1(x); h1−x^{T}_{2}h2

kx2kctanθ

1 0 0 ctanθ

1

−¯x_{2}

− ctanθ
1 + ctan^{2}θ

f (λ1(x))

kx2k Mx_{2}h (3.9)

+ 1

1 + tan^{2}θf^{0}

λ2(x); h1+x^{T}_{2}h2

kx2k tan θ

1 0 0 tan θ

1

¯
x_{2}

+ tan θ
1 + tan^{2}θ

f (λ2(x))
kx2k M_{x}_{2}h

= f^{0}

λ1(x); h1−x^{T}_{2}h2

kx2kctanθ

u^{1}_{x}+ f^{0}

λ2(x); h1+x^{T}_{2}h2

kx2k tan θ

u^{2}_{x}+f (λ2(x)) − f (λ1(x))
λ2(x) − λ1(x) Mx_{2}h.

This leads us to calculate the directional derivative (2.12) of the projection operator (3.5).

Lemma 3.2 (calculating the directional derivative of the projection operator). The projector
operator (3.5) is directionally differentiable at any point x ∈ IR^{n} with the spectral decomposition (3.1),
and its directional derivative at x in any direction h ∈ R^{n} is calculated as follows:

(a) If λ_{1}(x)λ_{2}(x) 6= 0, then Π^{0}_{L}

θ(x; h) = ∇Π_{L}_{θ}(x)h.

(b) If λ1(x) = 0 and λ2(x) > 0, then Π^{0}_{L}_{θ}(x; h) = h − (1 + ctan^{2}θ) (u^{1}_{x})^{T}h

−u^{1}_{x}.

(c) If λ_{1}(x) < 0 and λ_{2}(x) = 0, then Π^{0}_{L}

θ(x; h) = (1 + tan^{2}θ) (u^{2}_{x})^{T}h

+u^{2}_{x}.
(d) If λ1(x) = λ2(x) = 0, then Π^{0}_{L}

θ(x; h) = Π_{L}_{θ}(h).

Proof. The directional differentiability of (3.5) at x follows from the discussions above. Moreover, in
case (a), corresponding to f (t) = (t)_{+} in (3.4), we get the differentiability of Π_{L}_{θ} at this point, and
hence Π^{0}_{L}

θ(x; h) = ∇Π_{L}_{θ}(x)h for all h ∈ R^{n}.

In case (b) we have x26= 0. It follows from (3.9) that
Π^{0}_{L}_{θ}(x; h) =

h1−x^{T}_{2}h_{2}
kx2kctanθ

+

u^{1}_{x}+

h1+x^{T}_{2}h_{2}
kx2k tan θ

u^{2}_{x}+ Mx_{2}h

= (1 + ctan^{2}θ) (u^{1}_{x})^{T}h

+u^{1}_{x}+ h +

tan^{2}θ
1 + tan^{2}θ

−h1+x^{T}_{2}h_{2}
kx2kctanθ

tan θ
1 + tan^{2}θ

h_{1}+x^{T}_{2}h_{2}
kx2k tan θ

−x^{T}_{2}h_{2}
kx2k

¯
x_{2}

= (1 + ctan^{2}θ) (u^{1}_{x})^{T}h

+u^{1}_{x}+ h −

tan^{2}θ
1 + tan^{2}θ

h_{1}−x^{T}_{2}h_{2}
kx2kctanθ

tan^{2}θ

1 + tan^{2}θ

h1− ¯x^{T}_{2}h2ctanθ

− ctanθ¯x2

= (1 + ctan^{2}θ) (u^{1}_{x})^{T}h

+u^{1}_{x}+ h − tan^{2}θ

1 + tan^{2}θ(1 + ctan^{2}θ) (u^{1}_{x})^{T}h1 0
0 ctanθ

1

−¯x_{2}

= (1 + ctan^{2}θ) (u^{1}_{x})^{T}h

+u^{1}_{x}+ h − (1 + ctan^{2}θ) (u^{1}_{x})^{T}hu^{1}_{x}

= h − (1 + ctan^{2}θ) (u^{1}_{x})^{T}h

−u^{1}_{x},

where the representations t = (t)_{+}+ (t)_{−} for all t ∈ IR is used together with
tan θ

1 + tan^{2}θ

h1+x^{T}_{2}h2

kx2k tan θ

−x^{T}_{2}h2

kx2k

= tan θ

1 + tan^{2}θ

h1+x^{T}_{2}h2

kx_{2}k tan θ − 1 + tan^{2}θ
tan θ

x^{T}_{2}h2

kx_{2}k

= tan θ

1 + tan^{2}θ

h1−x^{T}_{2}h2

kx2kctanθ

, and
(u^{1}_{x})^{T}h

+− (u^{1}_{x})^{T}h = − (u^{1}_{x})^{T}h

−.

In case (c) we employ (3.9) again to get the conclusion claimed. The final case (d) yields x = 0, and hence representation (3.8) gives us the equalities

Π^{0}_{L}

θ(x; h) = (h1− kh2kctanθ)_{+}u^{1}_{h}+ (h1+ kh2k tan θ)_{+}u^{2}_{h}= Π_{L}_{θ}(h),
which therefore complete the proof of the lemma. 2

The following theorem uses the previous considerations to establish the strongly semismoothness
property of the projection operator Π_{L}_{θ}. It has been well recognized the importance of this property
of Lipschitzian mappings in many aspects of variational analysis and optimization; in particular, to
establish the quadratic rate of convergence of the so-called semismooth Newton method; see [9, 30].

Theorem 3.3 (strong semismoothness of the projection operator). The projection operator ΠL_{θ}

in (3.5) is strongly semismooth over R^{n}.

Proof. The proof is inspired by [10, Proposition 4.5]. Note first that the directional differentiability
of the Lipschitz continuous projection operator ΠL_{θ} from Lemma 3.2, and thus it remains to show that
representation (2.14) holds for it with ρ = 1.

To verify our claim, deduce from the proof of Lemma 3.1 that

Π_{L}_{θ}(x) =

x if λ1(x) ≥ 0 and λ2(x) ≥ 0,

1

1 + tan^{2}θ(x1+ kx2k tan θ)1 0
0 tan θ

1

¯
x_{2}

if λ1(x) < 0 and λ2(x) > 0,

0 if λ_{1}(x) ≤ 0 and λ_{2}(x) ≤ 0.

(3.10)

Then we split the subsequent proof into two cases: x_{2}6= 0 and x_{2}= 0.

Case 1. When x_{2}6= 0, we can easily observe that in all the formulas from (3.10) corresponding to this
case, the projection operator Π_{L}_{θ} is a piecewise C^{2}-smooth mapping whose strong semismoothness is
well known in optimization [29]. It verifies the claim in this case.

Case 2. For x2 = 0, suppose first that x1 6= 0. Then λi(x) = x1 6= 0, i = 1, 2. Since λi(y) is
Lipschitz continuous by [35, Lemma 2.1], we get from (3.10) that ΠL_{θ}(y) is either 0 or y when y is in a
neighborhood of x. Thus Π_{L}_{θ} is surely strongly semismooth at x in this setting. In the remaining setting
of x_{1}= 0 we have x = 0. Note that the projection operator Π_{L}_{θ}is obviously positively homogeneous, i.e.,
Π_{L}_{θ}(tz) = tΠ_{L}_{θ}(z) for z ∈ R^{n}and t ≥ 0. This implies that Π^{0}_{L}

θ(h; h) = Π_{L}_{θ}(h) and Π^{0}_{L}

θ(0; h) = Π_{L}_{θ}(h).

Hence ∇Π_{L}_{θ}(h^{0})(h^{0}) = Π_{L}_{θ}(h^{0}) as h^{0} ∈ DΠ_{Lθ}. Since DΠ_{Lθ} is a dense subset of R^{n}, for any fixed h 6= 0
and V ∈ ∂BΠL_{θ}(h), there exists h^{0} ∈ DΠ_{Lθ} such that kh^{0}− hk ≤ khk^{2} and kV − ∇ΠL_{θ}(h^{0})k ≤ khk.

Hence for h sufficiently close to 0 we have

kV h − Π^{0}_{L}_{θ}(0; h)k = kV h − ∇Π_{L}_{θ}(h^{0})(h^{0}) + Π_{L}_{θ}(h^{0}) − Π^{0}_{L}_{θ}(0; h)k

= kV h − ∇Π_{L}_{θ}(h^{0})(h) + ∇Π_{L}_{θ}(h^{0})(h) − ∇Π_{L}_{θ}(h^{0})(h^{0}) + Π_{L}_{θ}(h^{0}) − Π_{L}_{θ}(h)k

≤ kV − ∇ΠL_{θ}(h^{0})kkhk + k∇ΠL_{θ}(h^{0})kkh − h^{0}k + kh − h^{0}k

≤ (r + 2)khk^{2},

where r is a bounded from above of k∂_{B}Π_{L}_{θ}(·)k near 0 since Π_{L}_{θ} is Lipschitz. Thus
L := lim sup

h→0

kV h − Π^{0}_{L}_{θ}(0; h)k

khk^{2} < ∞, i.e., (3.11)

V h − Π^{0}_{L}

θ(0; h) = O(khk^{2}) for all V ∈ ∂_{B}Π_{L}_{θ}(h).

Now let us show that V h − Π^{0}_{L}

θ(0; h) = O(khk^{2}) for any V ∈ co ∂BΠL_{θ}(h), i.e., for any hk → 0 and
Vk ∈ co ∂BΠL_{θ}(hk) we have Vkhk− Π^{0}_{L}_{θ}(0; hk) = O(khkk^{2}). Since Vk ∈ co ∂BΠL_{θ}(hk), it follows from
the Carath´eodory theorem that there are V_{k}^{i}∈ ∂BΠ_{L}_{θ}(h_{k}) and λ^{i}_{k} ≥ 0 for i = 1, . . . , n + 1 such that

Vk=

n+1

X

i=1

λ^{i}_{k}V_{k}^{i} and

n+1

X

i=1

λ^{i}_{k} = 1.

Since V_{k}^{i}∈ ∂_{B}Π_{L}_{θ}(h_{k}), it follows from (3.11) that
lim sup

k→0

kV_{k}^{i}hk− Π^{0}_{L}

θ(0; hk)k
khkk^{2} ≤ L.

Due to the boundedness of {λ^{i}_{k}}, we can assume without loss of generality that {λ^{i}_{k}} converge to some
λ¯i for i = 1, . . . , n + 1. Hence

lim sup

k→0

kV_{k}h_{k}− Π^{0}_{L}

θ(0; h_{k})k

khkk^{2} = lim sup

k→0

n+1

P

i=1

λ^{i}_{k}V_{k}^{i}h_{k}− Π^{0}_{L}_{θ}(0; h_{k})
khkk^{2}

≤ lim sup

k→0 n+1

X

i=1

λ^{i}_{k}kV_{k}^{i}h_{k}− Π^{0}_{L}

θ(0; h_{k})k
khkk^{2}

≤

n+1

X

i=1

λ¯^{i}L = L.

Thus V h − Π^{0}_{L}

θ(0; h) = O(khk^{2}) for any V ∈ co ∂BΠL_{θ}(h), i.e., ΠL_{θ} is strongly semismooth at 0. 2
The next result, which easily follows from Lemma 3.2, provides the calculation of the graphical
derivative (2.5) for the projection operator onto the circular cone.

Proposition 3.4 (calculating the graphical derivative of the projection operator). For any
x ∈ R^{n} with decomposition (3.1), the graphical derivative of ΠL_{θ}(x) is calculated by

DΠ_{L}_{θ}(x)(w) =Π^{0}_{L}_{θ}(x; w)

for any w ∈ IR^{n}. (3.12)

Proof. It follows from [31, formula 8(14)] that the graphical derivative of any closed-graph operator,
and hence of Π_{L}_{θ} in particular, can be equivalently represented as

DΠ_{L}_{θ}(x)(w) = Lim sup

τ &0 w0 →w

Π_{L}_{θ}(x + τ w^{0}) − Π_{L}_{θ}(x)

τ . (3.13)

By Lemma 3.2 the Lipschitzian mapping Π_{L}_{θ} is directionally differentiable at x. Thus the right-hand
side of (3.13) reduces to Π^{0}_{L}_{θ}(x; w), which justifies (3.12). 2

Based on the calculations provided in Lemma 3.1 and Lemma 3.2, we are now ready to establish
precise formulas for computing the regular and limiting coderivatives of the projection operator ΠL_{θ} onto
the general circular cone (1.1). We proceed similarly to the proofs of the main results of the paper [27]

by Outrata and Sun while using our calculations given above as well as in the proofs of the theorems.

Taking into account relationships (1.6) between the circular and second-order cones, it is appealing to reduce deriving coderivative formulas for the projection onto the circular cone to those obtained for the second-order one. However, it does not seem to be possible; see more discussions in Remark 4.7.

Theorem 3.5 (calculating the regular coderivative of the projection operator). For any x ∈ R^{n}
with decomposition (3.1) and any y^{∗}∈ R^{n}, the regular coderivative (2.8) of the projection operator Π_{L}_{θ}(x)
onto the circular cone (1.1) is calculated as follows:

(a) If λ1(x)λ2(x) 6= 0, then bD^{∗}Π_{L}_{θ}(x)(y^{∗}) =∇ΠLθ(x)y^{∗} .
(b) If λ1(x) = 0 and λ2(x) > 0, then

Db^{∗}Π_{L}_{θ}(x)(y^{∗}) =x^{∗}∈ R^{n}

y^{∗}− x^{∗}∈ IR+u^{1}_{x}, hx^{∗}, u^{1}_{x}i ≥ 0 .
(c) If λ1(x) < 0 and λ2(x) = 0, then

Db^{∗}Π_{L}_{θ}(x)(y^{∗}) =x^{∗}∈ R^{n}

x^{∗}∈ IR_{+}u^{2}_{x}, hy^{∗}− x^{∗}, u^{2}_{x}i ≥ 0 .
(d) If λ_{1}(x) = λ_{2}(x) = 0, then

Db^{∗}ΠL_{θ}(x)(y^{∗}) =x^{∗}∈ R^{n}

x^{∗}∈ Lθ, y^{∗}− x^{∗}∈ L^{π}

2−θ .

Proof. Due to the well-known duality between the regular coderivative and the graphical derivative of a mapping (see [31]) and by the established directional differentiability of the projection operator onto the circular cone, we have the equivalence

x^{∗}∈ bD^{∗}ΠL_{θ}(x)(y^{∗}) ⇐⇒ hx^{∗}, hi ≤y^{∗}, Π^{0}_{L}_{θ}(x; h)

for all h ∈ IR^{n}. (3.14)

Employing (3.14) and the calculation of the directional derivative of Π_{L}_{θ} in Lemma 3.2 allows us to
derive the claimed formulas for the regular coderivative of Π_{L}_{θ} in all the cases (a)–(d) of the theorem.

In case (a), pick any x^{∗}∈ bD^{∗}Π_{L}_{θ}(x)(y^{∗}) and get by using Lemma 3.2(a) and duality (3.14) that
hx^{∗}, hi ≤ hy^{∗}, Π^{0}_{L}

θ(x; h)i ⇐⇒ hx^{∗}, hi ≤ hy^{∗}, ∇Π_{L}_{θ}(x)hi

⇐⇒ hx^{∗}− ∇Π_{L}_{θ}(x)y^{∗}, hi ≤ 0,

where the last step comes from the fact that the operator ∇Π_{L}_{θ} is self-adjoint by (3.6). Hence we have
x^{∗}= ∇Π_{L}_{θ}(x)y^{∗}, i.e., bD^{∗}Π_{L}_{θ}(x)(y^{∗}) = {∇Π_{L}_{θ}(x)y^{∗}}.

In case (b) we employ Lemma 3.2(b), which gives us together with (3.14) that
x^{∗}∈ bD^{∗}Π_{L}_{θ}(x)(y^{∗}) ⇐⇒ hx^{∗}, hi ≤

y^{∗}, h − (1 + ctan^{2}θ)

(u^{1}_{x})^{T}h

−u^{1}_{x}

⇐⇒ hx^{∗}− y^{∗}, hi + (1 + ctan^{2}θ)

y^{∗},

(u^{1}_{x})^{T}h

−u^{1}_{x}

≤ 0

⇐⇒

hx^{∗}− y^{∗}, hi ≤ 0, (u^{1}_{x})^{T}h ≥ 0

hx^{∗}− y^{∗}, hi + (1 + ctan^{2}θ)(u^{1}_{x})^{T}h(y^{∗})^{T}u^{1}_{x}≤ 0, (u^{1}_{x})^{T}h ≤ 0

⇐⇒ ∃ α ≥ 0 and β ≥ 0 such that y^{∗}− x^{∗}= αu^{1}_{x}
and x^{∗}− y^{∗}+ (1 + ctan^{2}θ) (y^{∗})^{T}u^{1}_{x}u^{1}_{x}= βu^{1}_{x}

⇐⇒ ∃ α ≥ 0 and β ≥ 0 such that y^{∗}− x^{∗}= αu^{1}_{x}
and (1 + ctan^{2}θ) (y^{∗})^{T}u^{1}_{x}u^{1}_{x}= (α + β)u^{1}_{x}

⇐⇒ ∃ α ≥ 0 such that y^{∗}− x^{∗}= αu^{1}_{x}

and (1 + ctan^{2}θ)hy^{∗}, u^{1}_{x}i ≥ α (3.15)

⇐⇒ ∃ α ≥ 0 such that y^{∗}− x^{∗}= αu^{1}_{x} and hx^{∗}, u^{1}_{x}i ≥ 0. (3.16)
The last equivalence above comes from the following arguments: if (3.16) holds, then

(1 + ctan^{2}θ)hy^{∗}, u^{1}_{x}i = (1 + ctan^{2}θ)hx^{∗}+ αu^{1}_{x}, u^{1}_{x}i ≥ α(1 + ctan^{2}θ)ku^{1}_{x}k^{2}= α;

conversely, the validity of (3.15) implies that

hx^{∗}, u^{1}_{x}i = hy^{∗}, u^{1}_{x}i − αhu^{1}_{x}, u^{1}_{x}i ≥ 1

1 + ctan^{2}θα − 1

1 + ctan^{2}θα = 0.

In case (c) we have the equivalencies by using Lemma 3.2(c) and duality (3.14):

x^{∗}∈ bD^{∗}Π_{L}_{θ}(x)(y^{∗}) ⇐⇒ hx^{∗}, hi ≤

y^{∗}, (1 + tan^{2}θ)

(u^{2}_{x})^{T}h

+

u^{2}_{x}

⇐⇒

( hx^{∗}, hi ≤ 0, (u^{2}_{x})^{T}h ≤ 0,

D

x^{∗}− (1 + tan^{2}θ)

(y^{∗})^{T}u^{2}_{x}
u^{2}_{x}, hE

≤ 0, (u^{2}_{x})^{T}h ≥ 0

⇐⇒ ∃ α ≥ 0 such that x^{∗}= αu^{2}_{x} and (1 + tan^{2}θ)(y^{∗})^{T}u^{2}_{x}≥ α

⇐⇒ ∃ α ≥ 0 such that x^{∗}= αu^{2}_{x} and hy^{∗}− x^{∗}, u^{2}_{x}i ≥ 0,
which readily justify the claimed result in this case.

In case (d) we have x = 0 and then proceed by using Lemma 3.2(d) together with (3.14). This yields
x^{∗}∈ bD^{∗}ΠL_{θ}(0)(y^{∗}) ⇐⇒ x^{∗}, h ≤ y^{∗}, ΠL_{θ}(h)

for all h ∈ IR^{n}

⇐⇒ x^{∗}, Π_{L}_{θ}(h) + Π_{L}_{θ}^{◦}(h) ≤ y^{∗}, Π_{L}_{θ}(h)

for all h ∈ IR^{n}

⇐⇒ x^{∗}− y^{∗}, Π_{L}_{θ}(h) + x^{∗}, Π_{L}_{θ}^{◦}(h) ≤ 0 for all h ∈ IR^{n} (3.17)

⇐⇒ x^{∗}∈ Lθ and y^{∗}− x^{∗}∈ L^{π}

2−θ, (3.18)

where the last equivalence is justified as follows. Relationship (3.18) =⇒ (3.17) is implied by the inclusion
x^{∗}− y^{∗} ∈ −L^{π}_{2}−θ = (L_{θ})^{◦}. For the converse implication, observe that the validity of (3.17) gives us
hx^{∗}− y^{∗}, hi ≤ 0 for all h ∈ L_{θ} and hx^{∗}, hi ≤ 0 for all h ∈ (L_{θ})^{◦}, which yields in turn the fulfillment of
x^{∗}− y^{∗}∈ (Lθ)^{◦}= −L^{π}

2−θ and x^{∗}∈ ((Lθ)^{◦})^{◦}= Lθ since Lθ is a closed and convex cone. 2

To calculate next the coderivative (2.6) of the projection operator ΠL_{θ}, for any x, y^{∗}∈ R^{n} we define
A(x, y^{∗}) :=x^{∗}∈ R^{n}

y^{∗}− x^{∗}∈ IR+u^{1}_{x}, hx^{∗}, u^{1}_{x}i ≥ 0 , (3.19)
B(x, y^{∗}) :=x^{∗}∈ R^{n}

x^{∗}∈ IR+u^{2}_{x}, hy^{∗}− x^{∗}, u^{2}_{x}i ≥ 0 . (3.20)
Theorem 3.6 (calculating the coderivative of the projection operator). For any x ∈ R^{n} with
decomposition (3.1) and any y^{∗} ∈ R^{n}, the coderivative (2.8) of the projection operator Π_{L}_{θ}(x) onto the
circular cone (1.1) is calculated as follows:

(a) If λ1(x)λ2(x) 6= 0, then D^{∗}Π_{L}_{θ}(x)(y^{∗}) =∇ΠLθ(x)y^{∗} .
(b) If λ_{1}(x) = 0 and λ_{2}(x) > 0, then

D^{∗}Π_{L}_{θ}(x)(y^{∗}) =h

∂_{B}(Π_{L}_{θ})(x)y^{∗}i [ n

x^{∗} ∈ R^{n}

y^{∗}− x^{∗}∈ IR_{+}u^{1}_{x}, hx^{∗}, u^{1}_{x}i ≥ 0o
.

(c) If λ1(x) < 0 and λ2(x) = 0, then
D^{∗}Π_{L}_{θ}(x)(y^{∗}) =h

∂_{B}(Π_{L}_{θ})(x)y^{∗}i [ n

x^{∗} ∈ R^{n}

x^{∗}∈ IR_{+}u^{2}_{x}, hy^{∗}− x^{∗}, u^{2}_{x}i ≥ 0o
.

(d) If λ1(x) = λ2(x) = 0, then
D^{∗}Π_{L}_{θ}(x)(y^{∗}) =h

∂B(Π_{L}_{θ})(x)y^{∗}i [ h [

ξ∈bd(Lπ 2−θ)/{0}

n

x^{∗} ∈ IR^{n}

y^{∗}− x^{∗}∈ IR+ξ, hx^{∗}, ξi ≥ 0oi

[ h [

η∈bd(L_{θ})/{0}

n
x^{∗}

x^{∗}∈ IR+η, hy^{∗}− x^{∗}, ηi ≥ 0oi [ hn
x^{∗}

x^{∗}∈ Lθ, y^{∗}− x^{∗}∈ L^{π}

2−θ

oi ,

where the B-subdifferential of ΠL_{θ} at x is calculated in Lemma 3.1.

Proof. Using the well-known representation of the coderivative (2.6) via the outer limit (1.7) of the
regular one (see [16, 31]) as well as the continuity of Π_{L}_{θ}, we get

D^{∗}Π_{L}_{θ}(x)(y^{∗}) = Lim sup

v→x v∗ →y∗

Db^{∗}Π_{L}_{θ}(v)(v^{∗}). (3.21)
This allows us to calculate D^{∗}Π_{L}_{θ} by passing to the limit in the relationships of Theorem 3.5.

In case (a) we easily get from (3.21) and Theorem 3.5(a) that
D^{∗}Π_{L}_{θ}(x)(y^{∗}) = Lim sup

v→x v∗ →y∗

Db^{∗}Π_{L}_{θ}(v)(v^{∗}) (3.22)

= Lim sup

v→x v∗ →y∗

∇ΠLθ(v)v^{∗} = ∇ΠLθ(x)y^{∗} .