**Variational analysis of circular cone programs**

Jinchuan Zhou^{a}, Jein-Shah Chen^{b}and Boris S. Mordukhovich^{cd}^{∗}

*a**Department of Mathematics, School of Sciences, Shandong University of Technology, Zibo, P.R.*

*China;*^{b}*Department of Mathematics, National Taiwan Normal University, Taipei, Taiwan;*

*c**Department of Mathematics, Wayne State University, Detroit, MI, USA;*^{d}*King Fahd University of*
*Petroleum and Minerals, Dhahran, Saudi Arabia*

*(Received 18 January 2014; accepted 12 July 2014)*

This paper conducts variational analysis of circular programs, which form a new class of optimization problems in nonsymmetric conic programming, important for optimization theory and its applications. First, we derive explicit formulas in terms of the initial problem data to calculate various generalized derivatives/co- derivatives of the projection operator associated with the circular cone. Then we apply generalized differentiation and other tools of variational analysis to establish complete characterizations of full and tilt stability of locally optimal solutions to parameterized circular programs.

**Keywords: variational analysis; optimization; generalized differentiation; conic**
programming; circular cone; second-order cone; projection operator; full and tilt
stability

**AMS Subject Classifications: 90C30; 90C31; 49J52**

**1. Introduction**

*The circular cone [1,2] is a pointed, closed, convex cone having hyperspherical sections*
orthogonal to its axis of revolution about which the cone is invariant to rotation. Let its
half-aperture angle be*θ ∈ (0,*^{π}_{2}*). Then the n-dimensional circular cone denoted by L** _{θ}*can
be expressed as follows (see Figure1):

*L** _{θ}* : =

*x= (x*1*, x*2*) ∈ IR × IR*^{n}^{−1}*x cos θ ≤ x*1

=

*x= (x*1*, x*2*) ∈ IR × IR*^{n}^{−1}*x*_{2}* ≤ x*1tan*θ*

*.* (1.1)

When*θ = 45*^{◦}*, the circular cone reduces to the well-known second-order cone (SOC*
for short, also known as the Lorentz cone and the ice-cream cone) given by

*K** ^{n}*: =

*x= (x*1*, x*2*) ∈ IR × IR*^{n}^{−1}*x*_{2}* ≤ x*1

=

*x= (x*1*, x*2*) ∈ IR × IR*^{n}^{−1}*x* cos 45^{◦}*≤ x*1

*.* (1.2)

∗Corresponding author. Email: boris@math.wayne.edu

© 2014 Taylor & Francis

Figure 1. The graphs of circular cones.

*Concerning SOC, for any vector x* *= (x*1*, x*2*) ∈ IR × IR*^{n}^{−1}we can decompose it as
*x= λ*1*(x)u*^{(1)}*x* *+ λ*2*(x)u*^{(2)}*x* *,* (1.3)
where*λ*1*(x), λ*2*(x) and u*^{(1)}*x* *, u*^{(2)}*x* *are the spectral values and the associated spectral vectors*
*of x relative toK** ^{n}*defined by, respectively,

*λ**i**(x) : = x*1*+ (−1)*^{i}*x*2*,*

*u*^{(i)}* _{x}* : =

⎧⎨

⎩

1 2

1, (−1)^{i}*x*_{2}

*x*2

*if x*2*= 0,*

1 2

1, (−1)^{i}*w*

*if x*2*= 0, i = 1, 2,*

with*w being any unit vector in IR*^{n}^{−1}*. If x*2= 0, decomposition (1.3) is unique. Using this
*decomposition, for any f* : IR → IR we consider [3,4] the vector function associated with
*K*^{n}*, n*≥ 1 by

*f*^{soc}*(x) := f (λ*1*(x))u*^{(1)}*x* *+ f (λ*2*(x))u*^{(2)}*x* *, x = (x*1*, x*2*) ∈ IR × IR*^{n}^{−1}*.* (1.4)
*If f is defined only on some subset of IR, then f*^{soc}is defined on the corresponding subset
of IR* ^{n}*. Definition (1.4) is unambiguous whether x2

*= 0 or x*2= 0.

Note that circular cone systems described by (1.1) with*θ = 45*^{◦} naturally arises in
many real-life engineering problems. In particular, we refer the reader to the recent paper
[5] and the bibliographies therein to the important class of optimal grasping manipulation
*problems for multi-fingered robots in which the grasping force of the i th finger is subject*
to a contact friction constraint given by

*u**i 2**, u**i 3**)* *u**i 1**,* (1.5)

*where u**i 1**is the normal force of the i th finger, u**i 2**and u**i 3**are the friction forces of the i th*
finger,* · is the 2-norm and μ is the friction coefficient; see Figure*2.

It is easy to see that (1.5) is a circular cone constraint corresponding to the description
*u**i* *= (u**i 1**, u**i 2**, u**i 3**) ∈ L** _{θ}*in (1.1) with the angle

*θ = tan*

^{−1}

*μ < 45*

^{◦}.

Observe that a possible way to deal with circular cone constraints is to scale*L** _{θ}*as SOC
by

*L**θ* *= A*^{−1}*K** ^{n}* and

*K*

^{n}*= AL*

*θ*with

*A*=

tan*θ 0*

0 *I*

*,* (1.6)

Figure 2. The grasping force forms a circular cone where*α = tan*^{−1}*μ < 45*^{◦}.

which is justified in [2, Theorem 2.1]. However, this approach may not be acceptable from both theoretical and numerical viewpoints. Indeed, the ‘scaling’ step can cause undesirable numerical performance due to round-off errors in computers, which has been confirmed by experiments. Furthermore, we will see in what follows that applying (1.6) does not help to obtain some major results of the paper while being useful in deriving the other ones.

Optimization problems with both SOC and circular cone constraints belong to a broad
*and important class in modern optimization theory known as conic or cone-constrained pro-*
*gramming; see, e.g. [6–8] and the references therein. However, the main difference between*
circular cone constraints and those given by SOC and most of the other constraint systems
in conic programming is that the circular cone*L*_{θ}*is non-self-dual, i.e. nonsymmetric, which*
makes its study more challenging and rather limited.

In contrast to symmetric conic programming, we are not familiar with a variety of publications devoted to their nonsymmetric counterparts. Referring the reader to [9–12]

and the bibliographies therein, observe that there is no unified way to handle nonsymmetric
cone constraints, and each study uses certain specific features of the nonsymmetric cones
under consideration. The previous papers [2,13] concerning the circular cone show that some
properties holding in the SOC framework can be extended to the circular cone setting. At
the same time, some other SOC properties fail to be satisfied for the general nonsymmetric
circular cone, where the angle*θ = 45*^{◦}plays a crucial role; see [14].

This paper is mainly devoted to two major interrelated issues of variational analysis
and optimization for problems involving circular cone constraints. Our first goal is to
calculate, entirely in terms of the initial circular cone data, some generalized differential
constructions of variational analysis that have been proven to be important for various
aspects of optimization. Namely, we derive explicit formulas to calculate generalized
*differential constructions for the (metric) projection operator associated with the general*
*circular cone that are known as the B-subdifferential, directional derivative, graphical*
*derivative, regular derivative, regular coderivative, and (limiting) coderivative. Except*
*the B-subdifferential and the (regular and limiting) coderivatives, the results obtained are*
new even for the symmetric SOC case. The obtained calculations allow us, in particular, to
*prove the strong semismoothness of the projection operator onto the circular cone, which is*
important for many applications including those to numerical optimization. Furthermore,
we establish new relationships between these generalized differential constructions for the
projection operator onto the circular cone and the metric projection onto the orthogonal
spaces to the spectral vectors in the circular cone representation.

*The second major goal of this paper is to completely characterize the notions of tilt*
*stability and full stability of mathematical programs with circular cone constraints. These*
fundamental stability concepts were introduced in optimization theory by Rockafellar and
his collaborators [15,16] and then have been intensively studied by many researchers,
especially in the recent years, for various classes of optimization problems; see, e.g.

[7,8,16–27] and the references therein. The construction of the second-order subdifferen-
*tial/generalized Hessian in the sense of Mordukhovich [28] (i.e. the coderivative of the first-*
order subgradient mapping) plays a crucial role in the characterization of tilt and full stability
obtained in the literature. In this paper we establish, by using the obtained second-order
calculations and the recent results of [25], complete characterizations of full and tilt stability
for locally optimal solutions to mathematical programs with circular cone constraints
expressed entirely in terms of the initial program data via certain second-order growth
and strong sufficient optimality conditions under appropriate constraint qualifications.

The rest of the paper is organized as follows. In Section2we recall and briefly discuss the generalized differential constructions of variational analysis employed in deriving the main results of this paper. Section3is devoted to calculating the generalized derivatives listed above for the projection operator onto the circular cone. In Section4we represent these generalized differential constructions for the aforementioned projection operator via the orthogonal projections generated by the spectral vectors of the circular cone. Finally, Section5applies the second-order subdifferential of the indicator function associated with the circular cone and related to the above coderivative calculations to establish complete characterizations of full and tilt stability of mathematical programs with circular cone constraints.

Throughout the paper we use the standard notation and terminology of variational
analysis; see, e.g. [29,30]. Given a set-valued mapping/multifunction F: R* ^{n}*→→ R

*, recall that the constructions*

^{m}Lim sup

*x**→ ¯x* *F(x) :=*

*y*∈ R^{m}* ∃ sequences x**k* *→ ¯x, y**k* *→ y such that*
*y**k* *∈ F(x**k**) for all k ∈ IN := {1, 2, . . .}*

*,* (1.7)

Lim inf

*x**→ ¯x* *F(x) :=*

*y*∈ R^{m}* for any x**k* *→ ¯x, ∃ y**k**→ y such that*

*y*_{k}*∈ F(x**k**) for all k ∈ IN* (1.8)
*are known as the (Painlevé–Kuratowski) outer limit and inner limit of F as x* *→ ¯x,*
respectively. For a set* ⊂ R*^{n}*, the symbol x→ ¯x signifies that x → with x ∈ .*^{}

**2. Tools of variational analysis**

In this section we briefly review those tools of generalized differentiation in variational analysis, which are widely used in the subsequent sections. We start with geometric notions.

Given a set * ⊂ IR*^{n}*locally closed around x* *∈ , the (Bouligand–Severi) tan-*
*gent/contingent cone to at ¯x ∈ is defined by*

*T*_{}*( ¯x) := Lim sup*

*t*↓0

* − ¯x*

*t* =

*d* ∈ IR^{n}* ∃t**k**↓ 0, d**k* *→ d with ¯x + t**k**d*_{k}*∈ *
(2.1)

via the outer limit (1.7), while the (Clarke) regular tangent cone to* at ¯x ∈ is given by*
*T*
_{}*( ¯x) := Lim inf*

*x**→ ¯x*^{}

*T*_{}*(x)* (2.2)

via the inner limit (1.8). The (Fréchet) regular normal cone to* at ¯x ∈ is*
*N*
_{}*( ¯x) :=*

*z*∈ IR^{n}*
z, x − ¯x ≤ o*

*x − ¯x*

*for all x∈ *

*,* (2.3)

*and the (Mordukhovich, limiting) normal cone to at ¯x ∈ can be equivalently defined*
by

*N*_{}*( ¯x) := Lim sup*

*x**→ ¯x*^{}

*N*_{}*(x) = Lim sup*

*x**→ ¯x*

cone

*x− **(x)*

*,* (2.4)

where_{}*denotes the (Euclidean) projection operator onto, and where ‘cone’ stands for*
the conic (may not be convex) hull of the set in question.

*Consider next a set-valued mapping H* : IR* ^{n}*⇒ IR

^{m}*with its graph and domain given*by

*gph H* :=

*(x, y) ∈ IR** ^{n}*× IR

^{m}*y∈ H(x)*

and *dom H* :=

*x*∈ IR^{n}*H(x) = ∅*
*,*
*respectively. The graphical derivative of H at( ¯x, ¯y) ∈ gph H is defined by*

*D H( ¯x, ¯y)(w) :=*

*z*∈ IR^{m}* (w,z) ∈ T**gph H**( ¯x, ¯y)*

*, w ∈ R*^{n}*,* (2.5)
via the tangent cone (2.1), while the (limiting) coderivative is defined via the normal cone
(2.4) by

*D*^{∗}*H( ¯x, ¯y)(y*^{∗}*) :=*

*x*^{∗}∈ IR^{n}* (x*^{∗}*, −y*^{∗}*) ∈ N**gph H**( ¯x, ¯y)*

*, y*^{∗}∈ R^{m}*,* (2.6)
where we drop *¯y in the derivative/coderivative notion if H is single-valued at ¯x. Simi-*
*larly, the regular derivative and the regular coderivative of H at( ¯x, ¯y) are defined via,*
respectively, (2.2) and (2.3) by

*D H*
*( ¯x, ¯y)(w) :=*

*z*∈ IR^{m}* (w,z) ∈
T*_{gph H}*( ¯x, ¯y)*

*, w ∈ R*^{n}*,* (2.7)

*D*
^{∗}*H( ¯x, ¯y)(y*^{∗}*) :=*

*x*^{∗}∈ IR^{n}* (x*^{∗}*, −y*^{∗}*) ∈
N*_{gph H}*( ¯x, ¯y)*

*, y*^{∗}∈ R^{m}*.* (2.8)
*Now let f* : IR^{n}*→ R := (−∞, ∞] be an extended real-valued function finite at*

*¯x ∈ R** ^{n}*. To define the second-order subdifferential construction needed in what follows,
we proceed in the way of [28,29] and begin with the first-order (limiting) subdifferential of

*f at* *¯x given by*

*∂ f ( ¯x) :=*

*v ∈ R*^{n}* (v,−*1*) ∈ N**epi f*

*¯x, f ( ¯x)*

(2.9)
via the normal cone (2.4) of the epigraph*{(x, μ) ∈ R*^{n}*× R| μ ≥ f (x)} of f . Observe*
*the representation N*_{}*( ¯x) = ∂δ*_{}*( ¯x) the normal cone (2.4) via the subdifferential (2.9) of*
*the indicator functionδ**(x) of equal to 0 if x ∈ and ∞ otherwise. The second-order*
*subdifferential (or generalized Hessian) of f at* *¯x relative to ¯y ∈ ∂ f ( ¯x) is defined as the*
coderivative (2.6) of the first-order subdifferential (2.9) by

*∂*^{2}*f( ¯x, ¯y)(u) := (D*^{∗}*∂ f )( ¯x, ¯y)(u), u ∈ IR*^{n}*.* (2.10)

*Finally in this section, consider a single-valued mapping F*: IR* ^{n}* → IR

*locally*

^{m}*Lipschitzian around*

*¯x and recall that F is almost everywhere differentiable in a neigh-*bourhood of

*¯x with the derivative ∇ F(x) by the classical Rademacher theorem; see [30].*

*Then the B-subdifferential of F at* *¯x is defined by*

*∂**B**F( ¯x) :=*

*x*lim*k**→x**∇ F(x**k**)** F is differentiable at x**k*

*.* (2.11)

*Recall also that F is directionally differentiable at¯x if the limit*
*F*^{}*(x; h) := lim*

*t*→0^{+}

*F(x + th) − F(x)*

*t* *exists for all h*∈ IR^{n}*.* (2.12)
*Having this, F is said to be semismooth at* *¯x if F is locally Lipschitzian around ¯x, direc-*
tionally differentiable at this point, and satisfies the relationship

*V h− F*^{}*(x; h) = o*

*h*

*for any V* *∈ co ∂**B**F(x + h) as h → 0.* (2.13)
*Furthermore, F isρ-order semismooth at x with 0 < ρ < ∞ if (*2.13) is replaced above by

*V h− F*^{}*(x; h) = O*

*h*^{1}^{+ρ}

*for any V* *∈ co ∂**B**F(x + h) as h → 0.* (2.14)
The case of*ρ = 1 in (2.13) corresponds to strongly semismooth mappings.*

**3. Generalized differentiation of the projection operator onto the circular cone**
In this section we derive precise formulas for calculating the above generalized derivatives
of the projection operator onto the circular cone (1.1). First we recall the following spectral
*decomposition from [2, Theorem 3.1] of any vector x* *= (x*1*, x*2*) ∈ IR × IR*^{n}^{−1}relative to
the circular cone*L** _{θ}*:

*x= λ*1*(x)u*^{1}_{x}*+ λ*2*(x)u*^{2}_{x}*,* (3.1)
*where the spectral valuesλ*1*(x) and λ*2*(x) are defined by*

*λ*1*(x) := x*1*− x*2*ctanθ, λ*2*(x) := x*1*+ x*2* tan θ,* (3.2)
*and where the spectral vectors u*^{1}_{x}*and u*^{2}* _{x}*are written as

*u*^{1}* _{x}* := 1
1+ ctan

^{2}

*θ*

1 0 0 ctanθ

1

*− ¯x*2

*, u*^{2}*x* := 1
1+ tan^{2}*θ*

1 0
0 tan*θ*

1

*¯x*2

(3.3)
with*¯x*2*:= x*2*/x*2* if x*2*= 0 and ¯x*2equal to any unit vector*w ∈ IR*^{n}^{−1}otherwise. Given
*any f*: R → R we construct the vector function

*f*^{L}^{θ}*(x) := f*
*λ*1*(x)*

*u*^{1}_{x}*+ f*
*λ*2*(x)*

*u*^{2}_{x}*.* (3.4)

associated with circular cone. It follows from [2] that the projection_{L}_{θ}*(x) of x onto*
*L*_{θ}*, which is a single-valued and Lipschitzian operator, corresponds to f(t) := (t)*_{+} =
max{t, 0} in (3.4), i.e. we have

_{L}_{θ}*(x) =*

*x*_{1}*− x*2*ctanθ*

+*u*^{1}* _{x}*+

*x*_{1}*+ x*2* tan θ*

+*u*^{2}_{x}*.* (3.5)

*Our first result in this section provides a complete calculation of the B-subdifferential*
(2.11) of the projection operator (3.5) entirely in terms of the initial data of the general
circular cone (1.1). This result is widely used in what follows.

Le m m a *3.1 (calculating the B-subdifferential of the projection operator)* *For any x*∈
IR^{n}*with the spectral decomposition (3.1), the B-subdifferential of the projection operator*

_{L}_{θ}*is calculated as follows:*

(a) *Ifλ*1*(x)λ*2*(x) = 0, then *_{L}_{θ}*is differentiable at x and∂**B**(*_{L}_{θ}*)(x) = {∇*_{L}_{θ}*(x)}.*

(b) *Ifλ*1*(x) = 0 and λ*2*(x) > 0, then*

*∂**B**(*_{L}_{θ}*)(x) =*

*I, I +* 1
tan*θ + ctanθ*

*− tan θ* *¯x*2

*¯x*2 *−ctanθ ¯x*2*¯x*_{2}^{T}

*.*

(c) *Ifλ*1*(x) < 0 and λ*2*(x) = 0, then*

*∂**B**(**L**θ**)(x) =*

0*,* 1

tan*θ + ctanθ*

ctan*θ* *¯x*_{2}^{T}

*¯x*2 tan*θ ¯x*2*¯x*_{2}^{T}

*.*

(d) *Ifλ*1*(x) = λ*2*(x) = 0, then*

*∂**B**(*_{L}_{θ}*)(x) =*

1

tan*θ + ctanθ*

×

ctanθ *w*^{T}

*w*

tan*θ + ctanθ*
*a I*+

tan*θ − a(ctanθ + tan θ)*
*ww*^{T}

×

*a∈ [0, 1]*

*w = 1*

0, I

*.*

*Proof* *In case (a) the function f(t) = (t)*_{+}is differentiable at*λ**i**(x) for i = 1, 2. Hence,*
it follows from [13, Theorem 2.3] that_{L}_{θ}*is also differentiable at x. Furthermore, in this*
case we have by (3.5) that

*L**θ**(x) =*

⎧⎨

⎩

*x* if *λ*1*(x) > 0 and λ*2*(x) > 0,*
0 if *λ*1*(x) < 0 and λ*2*(x) < 0,*
*(x*1*+ x*2* tan θ)u*^{2}*x* if *λ*1*(x) < 0 and λ*2*(x) > 0.*

In particular,*x*2* = 0 when λ*1*(x) < 0 and λ*2*(x) > 0, and thus ∇x*2* = ¯x*2. This gives
us

*∂**B*_{L}_{θ}*(x) =*

*∇*_{L}_{θ}*(x)*
*,*
where the derivative of_{L}_{θ}*at x is calculated by*

*∇*_{L}_{θ}*(x)*

=

⎧⎪

⎪⎪

⎪⎨

⎪⎪

⎪⎪

⎩

*I* if *λ*1*(x) > 0 and λ*2*(x) > 0,*

0 if *λ*1*(x) < 0 and λ*2*(x) < 0,*

1
tan*θ + ctanθ*

⎡

⎣ctanθ *¯x*_{2}^{T}

*¯x*2

*x*_{1}*+ x*2* tan θ*

*x*2 *I*− *x*_{1}

*x*2*¯x*2*¯x*_{2}^{T}

⎤

*⎦ if λ*1*(x) < 0 and λ*2*(x) > 0.*

(3.6)

In case (b) we have*x*2 = 0, and so it follows from [13, Theorem 3.1] that

*∂**B*

*f*^{L}^{θ}*(x) =*

⎧⎪

⎪⎪

⎪⎨

⎪⎪

⎪⎪

⎩

*ξ* * ¯x*_{2}^{T}* ¯x*2 *a I* *+ (η − a) ¯x*2*¯x*_{2}^{T}

*a*= *f*
*λ*2*(x)*

*− f*
*λ*1*(x)*
*λ*2*(x) − λ*1*(x)*
*ξ − ctanθ ∈ ∂**B**f(λ*1*(x))*
*ξ + tan θ ∈ ∂**B**f(λ*2*(x))*
*η = ξ − (ctanθ − tan θ)*

⎫⎪

⎪⎪

⎪⎬

⎪⎪

⎪⎪

⎭
*.*

This implies by the obvious calculation

*∂**B**(t)*_{+}=

⎧⎨

⎩

1 *for t> 0,*
*{0, 1} for t = 0,*
0 *for t< 0*

*that the B-subdifferential of the projection operator is represented as*

*∂**B**(**L**θ**)(x) =*

⎧⎪

⎪⎨

⎪⎪

⎩

*ξ* * ¯x*_{2}^{T}* ¯x*2 *a I+ (η − a) ¯x*2*¯x*_{2}^{T}

*a*= 1

*ξ − ctanθ ∈ {0, 1}*

*ξ + tan θ = 1*

*η = ξ − (ctanθ − tan θ)*

⎫⎪

⎪⎬

⎪⎪

⎭*.* (3.7)

Analysing (3.7) in the case of*ξ −ctanθ = 1 and ξ + tan θ = 1 shows that ξ = 1, = 0,*
and*η = 1. Hence (*3.7) reduces in this case to I . For*ξ − ctanθ = 0 we know that*

*ξ =* ctanθ

tan*θ + ctanθ, =* 1

tan*θ + ctanθ, and η =* tan*θ*
tan*θ + ctanθ,*
and so Equation (3.7) in this case takes the form of

⎡

⎢⎣

ctan*θ*
tan*θ + ctanθ*

1

tan*θ + ctanθ¯x*_{2}* ^{T}*
1

tan*θ + ctanθ¯x*2 *I*+

$ tan*θ*

tan*θ + ctanθ* − 1

%

*¯x*2*¯x*_{2}^{T}

⎤

⎥⎦

*= I +* 1

tan*θ + ctanθ*

*− tan θ* *¯x*_{2}^{T}

*¯x*2 *−ctanθ ¯x*2*¯x*_{2}^{T}

*,*

*which gives us the B-subdifferential representation*

*∂**B*

*L**θ*

*(x) =*

*I, I +* 1
tan*θ + ctanθ*

*− tan θ* *¯x*2

*¯x*2 *−ctanθ ¯x*2*¯x*_{2}^{T}

*.*

*In case (c) we also have x*2= 0. Similarly to case (b), it is not hard to verify that

*∂**B*

_{L}_{θ}*(x) =*

'

0, 1

tan*θ + ctanθ*

ctan*θ* *¯x*_{2}^{T}

*¯x*2 tan*θ ¯x*2*¯x*_{2}^{T}

(

*.*

*It remains to consider case (d) when x* = 0. Then the result of [13, Theorem 3.4] tells
us that

*∂**B*

_{L}_{θ}*(x)*

=

⎧⎪

⎪⎪

⎪⎪

⎪⎨

⎪⎪

⎪⎪

⎪⎪

⎩

*ξ* *w*^{T}

*w aI + (η − a)ww*^{T}

*either a= ξ ∈ {0, 1}, = 0*
*or a∈ [0, 1]*

*ξ − ctanθ = 0*
*ξ + tan θ = 1*

*η = ξ − (ctanθ − tan θ)*

*w = 1*

⎫⎪

⎪⎪

⎪⎪

⎪⎬

⎪⎪

⎪⎪

⎪⎪

⎭

=

1

tan*θ + ctanθ*

ctanθ *w*^{T}

*w*

tan*θ + ctanθ*
*a I* +

tan*θ − a(ctanθ + tan θ)*
*ww*^{T}

×

*a* *∈ [0, 1]*

*w = 1*

0*, I*

*,*

which thus completes the proof of the lemma.

*Our next goal is to verify the directional differentiability of the projection operator*
(3.5) and derive formulas for calculating its directional derivative (2.12). Observe to this
end that the result of [13, Theorem 2.2] tells us that the vector function f^{L}* ^{θ}* from (3.4) is

*directionally differentiable at x provided that f is directionally differentiable atλ*

*i*

*(x) for*

*i= 1, 2. Moreover, for x*2= 0 we have

*f*^{L}^{θ}

_{}

*(x; h) =* 1

1+ ctan^{2}*θ* *f*^{}

*x*_{1}*; h*1*− h*2*ctanθ* 1 0
0 ctanθ

1

*− ¯h*2

+ 1

1+ tan^{2}*θ* *f*^{}

*x*_{1}*; h*1*+ h*2* tan θ* 1 0
0 tan*θ*

1

*¯h*_{2}

*= f*^{}

*x*_{1}*; h*1*− h*2*ctanθ*

*u*^{1}_{h}*+ f*^{}

*x*_{1}*; h*1*+ h*2* tan θ*

*u*^{2}_{h}*.* (3.8)
*On the other hand, for x*2= 0 we denote

*M*_{x}_{2} :=

⎡

⎣0 0

*0 I*− *x*2*x*_{2}^{T}

*x*2^{2}

⎤

⎦

and arrive at the following relationships:

*f*^{L}^{θ}

_{}
*(x; h)*

= 1

1+ ctan^{2}*θ* *f*^{}
)

*λ*1*(x); h*1−*x*_{2}^{T}*h*_{2}

*x*2ctan*θ*

* 1 0 0 ctanθ

1

*− ¯x*2

− ctanθ
1+ ctan^{2}*θ*

*f(λ*1*(x))*

*x*2 *M*_{x}_{2}*h*

+ 1

1+ tan^{2}*θ* *f*^{}
)

*λ*2*(x); h*1+*x*_{2}^{T}*h*_{2}

*x*2 tan*θ*

* 1 0
0 tan*θ*

1

*¯x*2

+ tan*θ*
1+ tan^{2}*θ*

*f(λ*2*(x))*

*x*2 *M*_{x}_{2}*h*

*= f*^{}
)

*λ*1*(x); h*1− *x*_{2}^{T}*h*2

*x*2ctanθ

*
*u*^{1}_{x}*+ f*^{}

)

*λ*2*(x); h*1+*x*_{2}^{T}*h*2

*x*2 tan*θ*

*
*u*^{2}* _{x}*
+

*f(λ*2

*(x)) − f (λ*1

*(x))*

*λ*2*(x) − λ*1*(x)* *M**x*_{2}*h.* (3.9)

*This leads us to calculate the directional derivative (2.12) of the projection operator*
(3.5).

Le m m a 3.2 (calculating the directional derivative of the projection operator) *The pro-*
*jector operator (3.5) is directionally differentiable at any point x* ∈ IR^{n}*with the spectral*
*decomposition (3.1), and its directional derivative at x in any direction h*∈ R^{n}*is calculated*
*as follows:*

(a) *Ifλ*1*(x)λ*2*(x) = 0, then *^{}_{L}_{θ}*(x; h) = ∇*_{L}_{θ}*(x)h.*

(b) *Ifλ*1*(x) = 0 and λ*2*(x) > 0, then *^{}_{L}_{θ}*(x; h) = h − (1 + ctan*^{2}*θ)*
*(u*^{1}_{x}*)*^{T}*h*

−*u*^{1}_{x}*.*
(c) *Ifλ*1*(x) < 0 and λ*2*(x) = 0, then *^{}_{L}_{θ}*(x; h) = (1 + tan*^{2}*θ)*

*(u*^{2}*x**)*^{T}*h*

+*u*^{2}_{x}*.*
(d) *Ifλ*1*(x) = λ*2*(x) = 0, then *^{}_{L}_{θ}*(x; h) = **L**θ**(h).*

*Proof* The directional differentiability of (3.5) at x follows from the discussions above.

*Moreover, in case (a), corresponding to f(t) = (t)*_{+}in (3.4), we get the differentiability of

_{L}* _{θ}* at this point, and hence

^{}

_{L}

_{θ}*(x; h) = ∇*

_{L}

_{θ}*(x)h for all h ∈ R*

*.*

^{n}*In case (b) we have x*2= 0. It follows from (3.9) that

^{}_{L}_{θ}*(x; h) =*
)

*h*_{1}−*x*_{2}^{T}*h*_{2}

*x*2ctan*θ*

*

+

*u*^{1}* _{x}*+
)

*h*_{1}+ *x*_{2}^{T}*h*_{2}

*x*2 tan*θ*

*

*u*^{2}_{x}*+ M**x*_{2}*h*

*= (1 + ctan*^{2}*θ)*
*(u*^{1}*x**)*^{T}*h*

+*u*^{1}_{x}*+ h*

+

⎛

⎜⎜

⎜⎜

⎝

tan^{2}*θ*
1+ tan^{2}*θ*

)

*−h*1+*x*_{2}^{T}*h*_{2}

*x*2ctanθ

*

tan*θ*
1+ tan^{2}*θ*

)

*h*_{1}+*x*_{2}^{T}*h*_{2}

*x*2 tan*θ*

*

−*x*_{2}^{T}*h*_{2}

*x*2

*¯x*2

⎞

⎟⎟

⎟⎟

⎠

*= (1 + ctan*^{2}*θ)*
*(u*^{1}_{x}*)*^{T}*h*

+*u*^{1}_{x}*+ h*

−

⎛

⎜⎜

⎜⎝

tan^{2}*θ*
1+ tan^{2}*θ*

)

*h*_{1}− *x*_{2}^{T}*h*_{2}

*x*2ctan*θ*

*

tan^{2}*θ*
1+ tan^{2}*θ*

*h*1*− ¯x*2^{T}*h*2ctanθ

*− ctanθ ¯x*2

⎞

⎟⎟

⎟⎠

*= (1 + ctan*^{2}*θ)*
*(u*^{1}*x**)*^{T}*h*

+*u*^{1}_{x}*+ h*

− tan^{2}*θ*

1+ tan^{2}*θ(1 + ctan*^{2}*θ)*

*(u*^{1}_{x}*)*^{T}*h* 1 0
0 ctanθ

1

*− ¯x*2

*= (1 + ctan*^{2}*θ)*
*(u*^{1}*x**)*^{T}*h*

+*u*^{1}_{x}*+ h − (1 + ctan*^{2}*θ)*
*(u*^{1}*x**)*^{T}*h*

*u*^{1}_{x}

*= h − (1 + ctan*^{2}*θ)*
*(u*^{1}*x**)*^{T}*h*

−*u*^{1}_{x}*,*

*where the representations t* *= (t)*_{+}*+ (t)*_{−}*for all t*∈ IR are used together with
tan*θ*

1+ tan^{2}*θ*
)

*h*1+*x*_{2}^{T}*h*_{2}

*x*2 tan*θ*

*

−*x*_{2}^{T}*h*_{2}

*x*2

= tan*θ*
1+ tan^{2}*θ*

)

*h*_{1}+*x*_{2}^{T}*h*2

*x*2 tan*θ −*1+ tan^{2}*θ*
tan*θ*

*x*_{2}^{T}*h*2

*x*2

*

= tan*θ*
1+ tan^{2}*θ*

)

*h*_{1}−*x*_{2}^{T}*h*_{2}

*x*2ctanθ

*
*, and*

*(u*^{1}*x**)*^{T}*h*

+−
*(u*^{1}*x**)*^{T}*h*

= −
*(u*^{1}*x**)*^{T}*h*

−*.*

In case (c) we employ (3.9) again to get the conclusion claimed. The final case (d) yields
*x*= 0, and hence representation (3.8) gives us the equalities

^{}_{L}_{θ}*(x; h) = (h*1*− h*2*ctanθ)*_{+}*u*^{1}_{h}*+ (h*1*+ h*2* tan θ)*_{+}*u*^{2}_{h}*= *_{L}_{θ}*(h),*

which therefore complete the proof of the lemma.

*The following theorem uses the previous considerations to establish the strongly semis-*
*moothness property of the projection operator*_{L}* _{θ}*. It has been well recognized the impor-
tance of this property of Lipschitzian mappings in many aspects of variational analysis and

*optimization; in particular, to establish the quadratic rate of convergence of the so-called*

*semismooth Newton method; see [31,32].*

Th e o r e m 3.3 (strong semismoothness of the projection operator) *The projection oper-*
*ator*_{L}_{θ}*in (3.5) is strongly semismooth over*R^{n}*.*

*Proof* The proof is inspired by [33, Proposition 4.5]. Note first that the directional differ-
entiability of the Lipschitz continuous projection operator*L**θ* from Lemma3.2, and thus
it remains to show that representation (2.14) holds for it with*ρ = 1.*

To verify our claim, deduce from the proof of Lemma3.1that

_{L}_{θ}*(x) =*

⎧⎪

⎪⎨

⎪⎪

⎩

*x* if *λ*1*(x) ≥ 0 and λ*2*(x) ≥ 0,*

1

1+ tan^{2}*θ(x*1*+ x*2* tan θ)*
1 0

0 tan*θ*
1

*¯x*2

if *λ*1*(x) < 0 and λ*2*(x) > 0,*

0 if *λ*1*(x) ≤ 0 and λ*2*(x) ≤ 0.*

(3.10)
*Then we split the subsequent proof into two cases: x*2*= 0 and x*2= 0.

*Case 1* *When x*2 = 0, we can easily observe that in all the formulas from (3.10)
corresponding to this case, the projection operator_{L}* _{θ}* is a piecewise

*C*

^{2}-smooth mapping whose strong semismoothness is well known in optimization.[34] It verifies the claim in this case.

*Case 2* *For x*2*= 0, suppose first that x*1*= 0. Then λ**i**(x) = x*1*= 0, i = 1, 2. Since*
*λ**i**(y) is Lipschitz continuous by [13, Lemma 2.1], we get from (3.10) that*_{L}_{θ}*(y) is either*
*0 or y when y is in a neighbourhood of x. Thus*_{L}_{θ}*is surely strongly semismooth at x*
*in this setting. In the remaining setting of x*1*= 0 we have x = 0. Note that the projection*
operator*L**θ* is obviously positively homogeneous, i.e.*L**θ**(tz) = t**L**θ**(z) for z ∈ R*^{n}

*and t* *≥ 0. This implies that *^{}_{L}_{θ}*(h; h) = *_{L}_{θ}*(h) and *^{}_{L}_{θ}*(0; h) = *_{L}_{θ}*(h). Hence*

*∇*_{L}_{θ}*(h*^{}*)(h*^{}*) = *_{L}_{θ}*(h*^{}*) as h*^{} *∈ D*_{}_{Lθ}*. Since D*_{}* _{Lθ}* is a dense subset ofR

*, for any*

^{n}*fixed h= 0 and V ∈ ∂*

*B*

_{L}

_{θ}*(h), there exists h*

^{}

*∈ D*

_{}*such that*

_{Lθ}*h*

^{}

*− h ≤ h*

^{2}and

*V − ∇**L**θ**(h*^{}*) ≤ h. Hence for h sufficiently close to 0 we have*

*V h − *^{}_{L}_{θ}*(0; h) = V h − ∇*_{L}_{θ}*(h*^{}*)(h*^{}*) + *_{L}_{θ}*(h*^{}*) − *^{}_{L}_{θ}*(0; h)*

*= V h − ∇**L**θ**(h*^{}*)(h) + ∇**L**θ**(h*^{}*)(h) − ∇**L**θ**(h*^{}*)(h*^{}*)*
*+*_{L}_{θ}*(h*^{}*) − *_{L}_{θ}*(h)*

*≤ V − ∇**L**θ**(h*^{}*)h + ∇**L**θ**(h*^{}*)h − h*^{}* + h − h*^{}

*≤ (r + 2)h*^{2}*,*

*where r is a bounded from above of∂**B*_{L}_{θ}*(·) near 0 since *_{L}* _{θ}* is Lipschitz. Thus

*L*:= lim sup

*h*→0

*V h − *^{}_{L}_{θ}*(0; h)*

*h*^{2} *< ∞, i.e.* (3.11)

*V h− *^{}_{L}_{θ}*(0; h) = O(h*^{2}*) for all V ∈ ∂**B*_{L}_{θ}*(h).*

*Now let us show that V h− *^{}_{L}_{θ}*(0; h) = O(h*^{2}*) for any V ∈ co ∂**B**L**θ**(h), i.e. for any*
*h*_{k}*→ 0 and V**k* *∈ co ∂**B**L**θ**(h**k**) we have V**k**h*_{k}*− *^{}_{L}_{θ}*(0; h**k**) = O(h**k*^{2}*). Since V**k* ∈
co*∂**B**L**θ**(h**k**), it follows from the Carathéodory theorem that there are V*_{k}^{i}*∈ ∂**B**L**θ**(h**k**)*
and*λ*^{i}_{k}*≥ 0 for i = 1, . . . , n + 1 such that*

*V** _{k}* =

*n*+1

1

*i*=1

*λ*^{i}*k**V*_{k}* ^{i}* and

*n*+1

1

*i*=1

*λ*^{i}*k* *= 1.*

*Since V*_{k}^{i}*∈ ∂**B**L**θ**(h**k**), it follows from (*3.11) that

lim sup

*k*→0

*V*_{k}^{i}*h*_{k}*− *^{}_{L}_{θ}*(0; h**k**)*

*h**k*^{2} *≤ L.*

Due to the boundedness of*{λ*^{i}_{k}*}, we can assume without loss of generality that {λ*^{i}* _{k}*} converge
to some ¯

*λ*

*i*

*for i*

*= 1, . . . , n + 1. Hence*

lim sup

*k*→0

*V**k**h**k**− *^{}_{L}_{θ}*(0; h**k**)*

*h**k*^{2} = lim sup

*k*→0

*n*2+1

*i*=1*λ*^{i}_{k}*V*_{k}^{i}*h*_{k}*− *^{}_{L}_{θ}*(0; h**k**)*

*h**k*^{2}

≤ lim sup

*k*→0
*n*+1

1

*i*=1

*λ*^{i}_{k}*V*_{k}^{i}*h**k**− *^{}_{L}_{θ}*(0; h**k**)*

*h**k*^{2}

≤

*n*+1

1

*i*=1

*¯λ*^{i}*L* *= L.*

*Thus V h* *− *^{}_{L}_{θ}*(0; h) = O(h*^{2}*) for any V ∈ co ∂**B**L**θ**(h), i.e. **L**θ* is strongly

semismooth at 0.

The next result, which easily follows from Lemma3.2, provides the calculation of the
*graphical derivative (2.5) for the projection operator onto the circular cone.*

Pr o p o s it io n 3.4 (calculating the graphical derivative of the projection operator) *For*
*any x*∈ R^{n}*with decomposition (3.1), the graphical derivative of**L**θ**(x) is calculated by*

*D**L**θ**(x)(w) =*

^{}_{L}_{θ}*(x; w)*

*for any* *w ∈ IR*^{n}*.* (3.12)
*Proof* It follows from [30, formula 8(14)] that the graphical derivative of any closed graph
operator, and hence of*L**θ* in particular, can be equivalently represented as

*D**L**θ**(x)(w) = Lim sup*

*w→w**τ0*

_{L}_{θ}*(x + τw*^{}*) − *_{L}_{θ}*(x)*

*τ* *.* (3.13)

By Lemma3.2the Lipschitzian mapping_{L}_{θ}*is directionally differentiable at x. Thus the*
right-hand side of (3.13) reduces to^{}_{L}_{θ}*(x; w), which justifies (3.12).*
Based on the calculations provided in Lemmas3.1and3.2, we are now ready to establish
*precise formulas for computing the regular and limiting coderivatives of the projection*
operator_{L}* _{θ}*onto the general circular cone (1.1). We proceed similarly to the proofs of the
main results of the paper [35] by Outrata and Sun while using our calculations given above
as well as in the proofs of the theorems. Taking into account relationships (1.6) between the
circular and second-order cones, it is appealing to reduce deriving coderivative formulas for
the projection onto the circular cone to those obtained for the second-order one. However,
it does not seem to be possible; see more discussions in Remark4.7.

Th e o r e m 3.5 (calculating the regular coderivative of the projection operator) *For any*
*x* ∈ R^{n}*with decomposition (3.1) and any y*^{∗} ∈ R^{n}*, the regular coderivative (2.8) of the*
*projection operator**L**θ**(x) onto the circular cone (*1.1) is calculated as follows:

(a) *Ifλ*1*(x)λ*2*(x) = 0, then
D*^{∗}_{L}_{θ}*(x)(y*^{∗}*) =*

*∇*_{L}_{θ}*(x)y*^{∗}
*.*
(b) *Ifλ*1*(x) = 0 and λ*2*(x) > 0, then*

*D*
^{∗}_{L}_{θ}*(x)(y*^{∗}*) =*

*x*^{∗}∈ R^{n}*y*^{∗}*− x*^{∗}∈ IR_{+}*u*^{1}_{x}*,
x*^{∗}*, u*^{1}* _{x}* ≥ 0

*.*(c)

*Ifλ*1

*(x) < 0 and λ*2

*(x) = 0, then*

*D*
^{∗}*L**θ**(x)(y*^{∗}*) =*

*x*^{∗}∈ R^{n}*x*^{∗}∈ IR+*u*^{2}_{x}*,
y*^{∗}*− x*^{∗}*, u*^{2}*x* ≥ 0
*.*
(d) *Ifλ*1*(x) = λ*2*(x) = 0, then*

*D*^{∗}*L**θ**(x)(y*^{∗}*) =*

*x*^{∗}∈ R^{n}*x*^{∗}*∈ L**θ**, y*^{∗}*− x*^{∗}*∈ L*^{π}_{2}_{−θ}*.*

*Proof* Due to the well-known duality between the regular coderivative and the graphical
derivative of a mapping (see [30]) and by the established directional differentiability of the
projection operator onto the circular cone, we have the equivalence

*x*^{∗}∈
*D*^{∗}*L**θ**(x)(y*^{∗}*) ⇐⇒
x*^{∗}*, h ≤*3

*y*^{∗}*, *^{}_{L}_{θ}*(x; h)*4

*for all h* ∈ IR^{n}*.* (3.14)

Employing (3.14) and the calculation of the directional derivative of_{L}* _{θ}* in Lemma3.2
allows us to derive the claimed formulas for the regular coderivative of

*L*

*θ*in all the cases (a)–(d) of the theorem.

*In case (a), pick any x*^{∗} ∈
*D*^{∗}*L**θ**(x)(y*^{∗}*) and get by using Lemma*3.2(a) and duality
(3.14) that

*
x*^{∗}*, h ≤
y*^{∗}*, *^{}_{L}_{θ}*(x; h) ⇐⇒
x*^{∗}*, h ≤
y*^{∗}*, ∇**L**θ**(x)h*

*⇐⇒
x*^{∗}*− ∇*_{L}_{θ}*(x)y*^{∗}*, h ≤ 0,*

where the last step comes from the fact that the operator*∇*_{L}* _{θ}* is self-adjoint by (3.6).

*Hence, we have x*^{∗}*= ∇*_{L}_{θ}*(x)y*^{∗}, i.e.
*D*^{∗}_{L}_{θ}*(x)(y*^{∗}*) = {∇*_{L}_{θ}*(x)y*^{∗}}.

In case (b) we employ Lemma3.2(b), which gives us together with (3.14) that

*x*^{∗}∈
*D*^{∗}_{L}_{θ}*(x)(y*^{∗}*) ⇐⇒
x*^{∗}*, h ≤*
5

*y*^{∗}*, h − (1 + ctan*^{2}*θ)*
*(u*^{1}*x**)*^{T}*h*

−*u*^{1}* _{x}*
6

*⇐⇒
x*^{∗}*− y*^{∗}*, h + (1 + ctan*^{2}*θ)*
5

*y*^{∗}*,*
*(u*^{1}*x**)*^{T}*h*

−*u*^{1}* _{x}*
6

≤ 0

⇐⇒

*
x*^{∗}*− y*^{∗}*, h ≤ 0,* *(u*^{1}*x**)*^{T}*h*≥ 0

*
x*^{∗}*− y*^{∗}*, h + (1 + ctan*^{2}*θ)(u*^{1}_{x}*)*^{T}*h(y*^{∗}*)*^{T}*u*^{1}_{x}*≤ 0, (u*^{1}_{x}*)*^{T}*h*≤ 0

*⇐⇒ ∃ α ≥ 0 and β ≥ 0 such that y*^{∗}*− x*^{∗}*= αu*^{1}*x*
*and x*^{∗}*− y*^{∗}*+ (1 + ctan*^{2}*θ)*

*(y*^{∗}*)*^{T}*u*^{1}_{x}

*u*^{1}_{x}*= βu*^{1}_{x}

*⇐⇒ ∃ α ≥ 0 and β ≥ 0 such that y*^{∗}*− x*^{∗}*= αu*^{1}*x*
and *(1 + ctan*^{2}*θ)*

*(y*^{∗}*)*^{T}*u*^{1}_{x}

*u*^{1}_{x}*= (α + β)u*^{1}_{x}

*⇐⇒ ∃ α ≥ 0 such that y*^{∗}*− x*^{∗}*= αu*^{1}*x*

and *(1 + ctan*^{2}*θ)
y*^{∗}*, u*^{1}_{x}* ≥ α* (3.15)

*⇐⇒ ∃ α ≥ 0 such that y*^{∗}*− x*^{∗}*= αu*^{1}*x* and *
x*^{∗}*, u*^{1}*x** ≥ 0.* (3.16)
The last equivalence above comes from the following arguments: if (3.16) holds, then

*(1 + ctan*^{2}*θ)
y*^{∗}*, u*^{1}*x** = (1 + ctan*^{2}*θ)
x*^{∗}*+ αu*^{1}*x**, u*^{1}*x** ≥ α(1 + ctan*^{2}*θ)u*^{1}*x*^{2}*= α;*

conversely, the validity of (3.15) implies that
*
x*^{∗}*, u*^{1}*x** =
y*^{∗}*, u*^{1}*x** − α
u*^{1}*x**, u*^{1}*x* ≥ 1

1+ ctan^{2}*θα −* 1

1+ ctan^{2}*θα = 0.*

In case (c) we have the equivalencies by using Lemma3.2(c) and duality (3.14):

*x*^{∗}∈
*D*^{∗}*L**θ**(x)(y*^{∗}*) ⇐⇒
x*^{∗}*, h ≤*
5

*y*^{∗}*, (1 + tan*^{2}*θ)*
*(u*^{2}*x**)*^{T}*h*

+*u*^{2}* _{x}*
6

⇐⇒

'*
x*7 ^{∗}*, h ≤ 0,* *(u*^{2}*x**)*^{T}*h≤ 0,*
*x*^{∗}*− (1 + tan*^{2}*θ)*

*(y*^{∗}*)*^{T}*u*^{2}_{x}

*u*^{2}_{x}*, h*8

*≤ 0, (u*^{2}*x**)*^{T}*h*≥ 0

*⇐⇒ ∃ α ≥ 0 such that x*^{∗}*= αu*^{2}*x* and *(1 + tan*^{2}*θ)(y*^{∗}*)*^{T}*u*^{2}_{x}*≥ α*

*⇐⇒ ∃ α ≥ 0 such that x*^{∗}*= αu*^{2}*x* and *
y*^{∗}*− x*^{∗}*, u*^{2}*x** ≥ 0,*
which readily justify the claimed result in this case.