3 Second-Order Directional Derivative

to appear in Journal of Optimization Theory and Applications, 2016

Parabolic second-order directional differentiability in the Hadamard sense of the vector-valued functions associated with

circular cones

Jinchuan Zhou ¹ Department of Mathematics

School of Science

Shandong University of Technology Zibo 255049, Shandong, P.R.China

E-mail: jinchuanzhou@163.com

Jingyong Tang²

College of Mathematics and Information Science Xinyang Normal University

Xinyang 464000, Henan, P.R.China E-mail: jingyongtang@163.com

Jein-Shan Chen ³ Department of Mathematics National Taiwan Normal University

Taipei 11677, Taiwan E-mail: jschen@math.ntnu.edu.tw

June 28, 2015

(1st revised on March 9, 2016) (2nd revised on April 8, 2016)

Abstract. In this paper, we study the parabolic second-order directional derivative in the Hadamard sense of a vector-valued function associated with circular cone. The

1The author’s work is supported by National Natural Science Foundation of China (11101248, 11271233), Shandong Province Natural Science Foundation (ZR2012AM016), and Young Teacher Sup- port Program of Shandong University of Technology.

2The author’s work is supported by Science Technology Research Projects of Education Department of Henan Province (13A110767).

3Corresponding author. The author’s work is supported by Ministry of Science and Technology, Taiwan.

vector-valued function comes from applying a given real-valued function to the spec- tral decomposition associated with circular cone. In particular, we present the exact formula of second-order tangent set of circular cone by using the parabolic second-order directional derivative of projection operator. In addition, we also deal with the relation- ship of second-order differentiability between the vector-valued function and the given real-valued function. The results in this paper build fundamental bricks to the character- izations of second-order necessary and sufficient conditions for circular cone optimization problems.

Keywords Parabolic second-order derivative, circular cone, second-order tangent set

1 Introduction

The parabolic second-order derivatives were originally introduced by Ben-Tal and Zowe in [1, 2], please refer to [3] for more details about properties of parabolic second-order derivatives. Usually the parabolic second-order derivatives can be employed to char- acterize the optimality conditions for various optimization problems, see [1, 4, 5, 6, 7]

and references therein. The so-called generalized parabolic second-order derivatives are studied in [4, 5, 8], whereas the parabolic second-order derivatives for certain types of functions are investigated in [5, 8, 9, 10]. In this paper, we mainly focus on the parabolic second-order directional derivative in the Hadamard sense for the vector-valued func- tions associated with circular cones. This vector-valued function, called circular cone function, comes from applying a given real-valued function to the spectral decomposition associated with circular cone.

For the circular cone function, by using the basic tools of nonsmooth analysis, various properties such as directional derivative, differentiability, B-subdifferentiability, semis- moothness, and positive homogeneity, have been studied in [11, 12]. The aforementioned results can be regarded as the first-order type of differentiability analysis. Here, we further discuss the second-order type of differentiability analysis for the circular cone function. As mentioned above, the concept of parabolic second-order directional differ- entiability plays an important role in second-order necessary and sufficient conditions.

Recently, there was an investigation on the parabolic second-order directional derivative of singular values of matrices and symmetric matrix-valued functions in [10]. Inspired by this work, we study the parabolic second-order directional derivative for the vector-valued circular cone function. The relationship of parabolic second-order directional derivative between the vector-valued circular cone function and the given real-valued function is established, in which we do not require that the real-valued function is second-order dif- ferentiable. This allows us to apply our result to more general nonsmooth functions. For example, we obtain the exact formula of second-order tangent set by using the parabolic second-order directional differentiability of projection operator associated with circular

cone, which is corresponding to the nonsmooth max-type function. In addition, we study the relationship of second-order differentiability between circular cone function and the given real-valued function. It is surprising that, not like the first-order differentiability, the relationship in the second-order differentiability case really depends on the angle.

This further shows the essential role played by the angle in the circular cone setting.

2 Preliminaries

The n-dimensional circular cone is defined as

L_θ := {x = (x₁, x₂)^T ∈ R × Rⁿ⁻¹ : cos θkxk ≤ x₁},

which is a non-symmetrical cone in the standard inner product. In our previous works [12, 13, 14, 15], we have explored some important features about circular cone, such as characterizing its tangent cone, normal cone, and second-order regularity, etc.. In particular, the spectral decomposition associated with Lθ was discovered, i.e., for any x = (x₁, x₂) ∈ R × Rⁿ⁻¹, one has

x = λ₁(x)u⁽¹⁾_x + λ₂(x)u⁽²⁾_x , (1) where

λ₁(x) := x₁− kx₂k cot θ, λ₂(x) := x₁+ kx₂k tan θ and

u⁽¹⁾_x := 1 1 + cot²θ

1 0

0 cot θ · I

  1

−¯x₂



, u⁽²⁾_x := 1 1 + tan²θ

1 0

0 tan θ · I

  1

¯ x₂



with ¯x₂ := x₂/kx₂k if x₂ 6= 0, and ¯x₂ being any vector w ∈ Rⁿ⁻¹ satisfying kwk = 1 if x2 = 0. With this spectral decomposition (1), we can define a vector-valued function associated with circular cone as below. More specifically, for a given real-valued function f : R → R, the circular cone function f^Lθ : Rⁿ→ Rⁿ is defined as

f^Lθ(x) := f (λ₁(x)) u⁽¹⁾_x + f (λ₂(x)) u⁽²⁾_x .

Let X, Y be normed vector spaces and consider x, d, w ∈ X. Assume that ψ : X → Y is directionally differentiable. The function ψ is said to be parabolical second-order directionally differentiable in the Hadamard sense at x, if ψ is directionally differentiable at x and for any d, w ∈ X the following limit exists:

ψ⁰⁰(x; d, w) := lim

t↓0 w0→w

ψ(x + td +¹₂t²w⁰) − ψ(x) − tψ⁰(x; d)

1

2t² . (2)

To the contrast, the function ψ is said to be parabolical second-order directionally differ- entiable at x, if w⁰ is fixed to be w in (2). Generally speaking, the concept of parabolical

second-order directional differentiability in the Hadamard sense is stronger than that of parabolical second-order directional differentiability. However, when ψ is locally Lipschitz at x, then these two concepts coincide. It is known that if ψ is parabolical second-order directional differentiability in the Hadamard sense at x along d, w, then

ψ



x + td + 1

2t²w + o(t²)



= ψ(x) + tψ⁰(x; d) + 1

2t²ψ⁰⁰(x; d, w) + o(t²). (3) At the first glance on (3), the concept of parabolical second-order directional differentia- bility in the Hadamard sense is likely to say that ψ has a second-order Taylor expansion along some directions. In fact, for the expression (3), the main difference lies on the appearance of w. Why do we need such expansion (3)? We say a few words about it.

For standard nonlinear programming, corresponding to the nonnegative orthant, a poly- hedral is targeted. Hence, considering the way x + td, a radial line, is enough. However, for optimization problems involved the circular cones, second-order cones or semidefinite matrices cones, they are all non-polyhedral cones. Thus, we need to describe the curves thereon. To this end, the curved approach x + td +¹₂t²w is needed, which, to some extent, reflects the non-polyhedral properties of non-polyhedral cones. This point can be seen in Section 3, where the parabolic second-order directional derivative is used to study the second-order tangent sets of circular cones. The exact expression of second-order tangent set is important for describing the second-order necessary and sufficient conditions for conic programming, since its support function is appeared in the second-order necessary and sufficient conditions for conic programming; see [16] for more information.

3 Second-Order Directional Derivative

For subsequent analysis, we will frequently use the second-order derivative of ¯x := _kxk^x at x 6= 0. To this end, we present the second-order derivative of ¯x in below theorem. For convenience of notation, we also denote Φ(x) := ¯x for x 6= 0, which does not cause any confusion from the context.

Theorem 3.1. Let a function Φ : Rⁿ → Rⁿ be given as Φ(x) := _kxk^x for x 6= 0. Then, the function Φ is second-order continuous differentiable at x 6= 0 with

J Φ(x) = I − ¯x¯x^T kxk and

J²Φ(x)(w, w) = −2 ¯x^Tw kxk²



w + w^T  3¯x¯x^T − I kxk³



wx, ∀w ∈ Rⁿ.

Proof. It is clear that Φ is second-order continuous differentiable because of x 6= 0. The Jacobian of Φ at x 6= 0 is obtained from direct calculation. To obtain the second-order

derivative, for any given a ∈ Rⁿ, we define ψ : Rⁿ→ R as ψ(x) := Φ(x)^Ta = x^Ta

kxk.

We also denote h(x) := a^Tx and g(x) := 1/kxk so that ψ(x) = h(x)g(x). Since x 6= 0, it is clear that g and h are twice continuously differentiable at x with J h(x) = a, J²h(x) = O, and

J g(x) = − x¯

kxk², J²g(x) = −(I − ¯x¯x^T) − 2¯x¯x^T

kxk³ = 3¯x¯x^T − I kxk³ . Hence, from the chain rule, we have J ψ(x) = g(x)J h(x) + h(x)J g(x) and

J²ψ(x) = J g(x)^TJ h(x) + h(x)J²g(x) + g(x)J²h(x) + J h(x)^TJ g(x), which implies

J²ψ(x)(w, w) = 2J g(x)(w)J h(x)(w) + h(x)J²g(x)(w, w) + g(x)J²h(x)(w, w)

= 2J g(x)(w)J h(x)(w) + h(x)J²g(x)(w, w)

= a^T



−2x¯^Tw

kxk²w + w^T  3¯x¯x^T − I kxk³

 wx



. (4)

On the other hand, we see that J²ψ(x)(w, w) = a^TJ²Φ(x)(w, w). Since a ∈ Rⁿ is arbitrary, this together with (4) yields

J²Φ(x)(w, w) = −2x¯^Tw

kxk²w + w^T  3¯x¯x^T − I kxk³

 wx, which is the desired result. 2

Next, we characterize the parabolic second-order directional derivative of the spectral values λi(x) for i = 1, 2.

Theorem 3.2. Let x ∈ Rⁿ with spectral decomposition x = λ₁(x)u⁽¹⁾x + λ₂(x)u⁽²⁾x given as in (1). Then, the parabolic second-order directional differentiability in the Hadamard sense of λ_i(x) for i = 1, 2 reduces to the parabolic second-order directional differentiability.

Moreover, given d, w ∈ Rⁿ, we have

λ⁰⁰₁(x; d, w) =







w₁−

¯

x^T₂w₂+^kd2k2_kx^{−(¯xT2d2)2}

2k



cot θ, if x₂ 6= 0,

w₁− ¯d^T₂w₂cot θ, if x₂ = 0, d₂ 6= 0, w₁− kw₂k cot θ, if x₂ = 0, d₂ = 0, and

λ⁰⁰₂(x; d, w) =







w₁+

¯

x^T₂w₂+^kd2k2_kx^{−(¯xT2d2)2}

2k



tan θ, if x₂ 6= 0,

w₁+ ¯d^T₂w₂tan θ, if x₂ = 0, d₂ 6= 0, w1+ kw2k tan θ, if x2 = 0, d2 = 0.

Proof. Note that λ_i(x) for i = 1, 2 is Lipschitz continuous [12], hence the parabolic second-order directional differentiability in the Hadamard sense of λ_i(x) for i = 1, 2 reduces to the parabolic second-order directional differentiability.

To compute the parabolic second-order directional derivative, we consider the follow- ing three cases.

(i) If x₂ 6= 0, then x + td + ¹₂t²w = (x₁ + td₁ + ¹₂t²w₁, x₂ + td₂ + ¹₂t²w₂). Note that λ⁰₁(x; d) = d₁− ¯x^T₂d₂cot θ and

kx₂+ td₂+ 1

2t²w₂k = kx₂k + t¯x^T₂d₂+ 1 2t²



¯

x^T₂w₂+ kd₂k²− (¯x^T₂d₂)² kx₂k



+ o(t²).

Thus, we obtain

λ₁(x + td + ¹₂t²w) − λ₁(x) − tλ⁰₁(x; d)

1

2t² → w₁−



¯

x^T₂w₂+ kd₂k²− (¯x^T₂d₂)² kx₂k

 cot θ.

(ii) If x₂ = 0 and d₂ 6= 0, then x + td + ¹₂t²w = (x₁ + td₁ + ¹₂t²w₁, td₂ + ¹₂t²w₂) and λ⁰₁(x; d) = d₁− kd₂k cot θ. Hence

λ1(x + td +¹₂t²w) − λ1(x) − tλ⁰₁(x; d)

1

2t² → w₁− ¯d^T₂w₂cot θ.

(iii) If x2 = 0 and d2 = 0, then x + td + ¹₂t²w = (x1 + td1 + ¹₂t²w1,¹₂t²w2). Thus, λ⁰₁(x; d) = d₁ and

λ₁(x + td + ¹₂t²w) − λ₁(x) − tλ⁰₁(x; d)

1

2t² → w₁− kw₂k cot θ.

From all the above, the formula of λ⁰⁰₁(x; d, w) is proved. Similar arguments can be applied to obtain the formula of λ⁰⁰₂(x; d, w). 2

The relationship of parabolic second-order directional differentiability in the Hadamard sense between f^Lθ and f is given below.

Theorem 3.3. Suppose that f : R → R. Then, f^Lθ is parabolic second-order direction- ally differentiable at x in the Hadamard sense if and only if f is parabolic second-order directionally differentiable at λ_i(x) in the Hadamard sense for i = 1, 2. Moreover, (a) if x₂ = 0 and d₂ = 0, then

(f^Lθ)⁰⁰(x; d, w) = f⁰⁰(x₁; d₁, w₁− kw₂k cot θ) u⁽¹⁾_w + f⁰⁰(x₁; d₁, w₁+ kw₂k tan θ) u⁽²⁾_w ;

(b) if x₂ = 0 and d₂ 6= 0, then

(f^Lθ)⁰⁰(x; d, w) = f⁰⁰ x₁; d₁− kd₂k cot θ, w₁− ¯d^T₂w₂cot θ
u⁽¹⁾_d +f⁰⁰ x₁; d₁+ kd₂k tan θ, w₁+ ¯d^T₂w₂tan θ
u⁽²⁾_d

+ 1

tan θ + cot θ



f⁰(x1; d1+ kd2k tan θ) − f⁰(x1; d1− kd2k cot θ)



J eΦ(d)w;

(c) if x₂ 6= 0, then

(f^Lθ)⁰⁰(x; d, w) = f⁰⁰ x₁− kx₂k cot θ; d₁− ¯x^T₂d₂cot θ, w₁− ¯x^T₂w₂ + d^T₂J Φ(x₂)d₂ cot θ
u⁽¹⁾_x +f⁰⁰ x₁+ kx₂k tan θ; d₁+ ¯x^T₂d₂tan θ, w₁+ ¯x^T₂w₂+ d^T₂J Φ(x₂)d₂ tan θ
u⁽²⁾_x

+ 2

cot θ + tan θΓ₁J eΦ(x)d + 1

cot θ + tan θΓ₂



J eΦ(x)w + J²Φ(x)(d, d)e

 ,

where

Γ₁ := f⁰(x₁+ kx₂k tan θ; d₁+ ¯x^T₂d₂tan θ) − f⁰(x₁− kx₂k cot θ; d₁− ¯x^T₂d₂cot θ) Γ2 := f (x1+ kx2k tan θ) − f (x1− kx2k cot θ)

and eΦ(x) := (1, Φ(x₂))^T for all x ∈ Rⁿ with x₂ 6= 0.

Proof. “⇐” Suppose that f is parabolic second-order directionally differentiable at λ_i(x) for i = 1, 2 in the Hadamard sense. Given d, w ∈ Rⁿ and w⁰ → w, we consider the following four cases. First we denote z := x + td + ¹₂t²w⁰.

Case 1: For x₂ = 0, d₂ = 0, and w₂ = 0, we have f^Lθ(x) = f (x₁), 0
= f (x₁)u⁽¹⁾z + f (x₁)u⁽²⁾z and

(f^Lθ)⁰(x; d) = (f⁰(x₁; d₁), 0) = f⁰(x₁; d₁)u⁽¹⁾_z + f⁰(x₁; d₁)u⁽²⁾_z .

Note that u⁽ⁱ⁾z → u⁽ⁱ⁾_ξ as i = 1, 2 for some ξ ∈ {(1, w) : kwk = 1}. Thus, we conclude that f^Lθ(x + td +¹₂t²w⁰) − f^Lθ(x) − t(f^Lθ)⁰(x; d)

1

2t² → f⁰⁰(x₁; d₁, w₁)u⁽¹⁾_ξ + f⁰⁰(x₁; d₁, w₁)u⁽²⁾_ξ

= (f⁰⁰(x₁; d₁, w₁), 0).

Case 2: For x2 = 0, d2 = 0, and w2 6= 0, since f is parabolic second-order directionally differentiable, we have

f (λ1(z)) − f (x1) − tf⁰(x1; d1)

1

2t² → f⁰⁰(x1; d1, w1− kw2k cot θ) and f (λ₂(z)) − f (x₁) − tf⁰(x₁; d₁)

1

2t² → f⁰⁰(x₁; d₁, w₁+ kw₂k tan θ).

Note that u⁽ⁱ⁾z → u⁽ⁱ⁾w for i = 1, 2. Therefore, we also conclude that f^Lθ(x + td +¹₂t²w⁰) − f^Lθ(x) − t(f^Lθ)⁰(x; d)

1 2t²

→ f⁰⁰(x₁; d₁, w₁− kw₂k cot θ)u⁽¹⁾_w + f⁰⁰(x₁; d₁, w₁+ kw₂k tan θ)u⁽²⁾_w . In summary, from Cases 1 and 2, we see that under x₂ = 0 and d₂ = 0

(f^Lθ)⁰⁰(x; d, w) = f⁰⁰(x₁; d₁, w₁− kw₂k cot θ)u⁽¹⁾_w + f⁰⁰(x₁; d₁, w₁+ kw₂k tan θ)u⁽²⁾_w . Case 3: For x₂ = 0, d₂ 6= 0, we have

(f^Lθ)⁰(x; d) = f⁰(x₁; d₁− kd₂k cot θ)u⁽¹⁾_d + f⁰(x₁; d₁+ kd₂k tan θ)u⁽²⁾_d . Note that

f (x1+ td1+ 1

2t²w₁⁰ − tkd2+1

2tw⁰₂k cot θ) (5)

= f



x₁+ td₁ +1

2t²w⁰₁− th

kd₂k cot θ +1

2t ¯d^T₂w⁰₂cot θ + o(t)i

= f



x₁+ td₁ +1

2t²w₁− th

kd₂k cot θ +1

2t ¯d^T₂w₂cot θi

+ o(t²)



= f (x1) + tf⁰(x1; d1 − kd2k cot θ) + 1

2t²f⁰⁰(x1; d1 − kd2k cot θ, w1− ¯d^T₂w2cot θ) + o(t²), where we use the facts that w⁰ → w and f is parabolic second-order directionally differ- entiable at λ₁(x) in the Hadamard sense. Similarly, we obtain

f (x₁+ td₁+ 1

2t²w₁⁰ + tkd₂+ 1

2tw₂⁰k tan θ) (6)

= f (x₁) + tf⁰(x₁; d₁ + kd₂k tan θ) + 1

2t²f⁰⁰(x₁; d₁+ kd₂k tan θ, w₁ + ¯d^T₂w₂tan θ) + o(t²).

Thus, the first component of ^fLθ(x+td+

1

2t²w⁰)−f^Lθ(x)−t(f^Lθ)⁰(x;d)

1

2t² converges to

1

1 + cot²θf⁰⁰(x1; d1− kd2k cot θ, w1− ¯d^T₂w2cot θ) + 1

1 + tan²θf⁰⁰(x1; d1+ kd2k tan θ, w1 + ¯d^T₂w2tan θ).

In addition, according to Theorem 3.1, we know d₂+¹₂tw⁰₂

kd₂+¹₂tw⁰₂k = Φ(d2+1 2tw⁰₂)

= Φ(d₂) + 1

2tJ Φ(d₂)w₂⁰ + 1

8t²J²Φ(d₂)(w₂⁰, w⁰₂) + o(t²)

= Φ(d₂) + 1

2tJ Φ(d₂)w₂⁰ + 1

8t²J²Φ(d₂)(w₂, w₂) + o(t²). (7)

Hence, it follows from (5), (6), and (7) that

−f (λ₁(z))Φ(d₂ +1

2tw₂⁰) + f (x₁)Φ(d₂) + tf⁰(x₁; d₁ − kd₂k cot θ)Φ(d₂)

= −1

2tf (x₁)J Φ(d₂)w₂⁰ − 1 2t²



f⁰⁰(x₁; d₁− kd₂k cot θ, w₁− ¯d^T₂w₂cot θ)Φ(d₂) +f⁰(x₁; d₁− kd₂k cot θ)J Φ(d₂)w⁰₂+1

4f (x₁)J²Φ(d₂)(w₂, w₂)



+ o(t²) and

f (λ₂(z))Φ(d₂+1

2tw⁰₂) − f (x₁)Φ(d₂) − tf⁰(x₁; d₁+ kd₂k tan θ)Φ(d₂)

= 1

2tf (x₁)J Φ(d₂)w⁰₂+1 2t²



f⁰⁰(x₁; d₁+ kd₂k tan θ, w₁+ ¯d^T₂w₂tan θ)Φ(d₂) +f⁰(x₁; d₁ + kd₂k tan θ)J Φ(d₂)w⁰₂+1

4f (x₁)J²Φ(d₂)(w₂, w₂)



+ o(t²).

Thus, the second component of ^fLθ(x+td+

1

2t²w⁰)−f^Lθ(x)−t(f^Lθ)⁰(x;d)

1

2t² converges to

1 tan θ + ctanθ



κ₁J Φ(d₂)w₂+ κ₂Φ(d₂)

 , where

κ₁ := f⁰(x₁; d₁+ kd₂k tan θ) − f⁰(x₁; d₁− kd₂k cot θ)

κ₂ := f⁰⁰(x₁; d₁+ kd₂k tan θ, w₁ + ¯d^T₂w₂tan θ) − f⁰⁰(x₁; d₁− kd₂k cot θ, w₁− ¯d^T₂w₂cot θ).

To sum up, we can conclude that (f^Lθ)⁰⁰(x; d, w)

= f⁰⁰(x₁; d₁− kd₂k, w₁− ¯d^T₂w₂cot θ)u⁽¹⁾_d + f⁰⁰(x₁; d₁+ kd₂k tan θ, w₁+ ¯d^T₂w₂tan θ)u⁽²⁾_d

+ 1

tan θ + cot θ



f⁰(x₁; d₁+ kd₂k tan θ) − f⁰(x₁; d₁− kd₂k cot θ)



J eΦ(d)w.

Case 4: For x₂ 6= 0, under this case, we know

(f^Lθ)⁰(x; d) = f⁰ λ₁(x); d₁− ¯x^T₂d₂cot θ
u⁽¹⁾_x + f⁰ λ₂(x); d₁+ ¯x^T₂d₂tan θ
u⁽²⁾_x +f (λ₂(x)) − f (λ₁(x))

λ₂(x) − λ₁(x)

0 0

0 I − ¯x₂x¯^T₂

 d.

Note that

kx₂+ td₂+ 1

2t²w₂⁰k = kx₂k + t¯x^T₂d₂+ 1 2t²h

¯

x^T₂w₂⁰ + d^T₂J Φ(x₂)d₂i

+ o(t²)

= kx₂k + t¯x^T₂d₂+ 1 2t²h

¯

x^T₂w₂ + d^T₂J Φ(x₂)d₂i

+ o(t²).

Since f is parabolic second-order directionally differentiable at λ₁(x) in the Hadamard sense, we have

f (x₁+ td₁+ 1

2t²w₁⁰ − kx₂+ td₂+ 1

2t²w₂⁰k cot θ)

= f (x₁− kx₂k cot θ) + tf⁰(x₁− kx₂k cot θ; d₁ − ¯x^T₂d₂cot θ) +1

2t²f⁰⁰

x₁− kx₂k cot θ; d₁− ¯x^T₂d₂cot θ, w₁−h

¯

x^T₂w₂+ d^T₂J Φ(x₂)d₂i cot θ

+ o(t²).

Besides, we know that Φ(x₂ + td₂+1

2t²w⁰₂) = Φ(x₂) + tJ Φ(x₂)d₂+1 2t²



J Φ(x₂)w⁰₂+ J²Φ(x₂)(d₂, d₂)



+ o(t²)

= Φ(x₂) + tJ Φ(x₂)d₂+1 2t²



J Φ(x₂)w₂+ J²Φ(x₂)(d₂, d₂)



+ o(t²).

Thus, the first component of ^fLθ(x+td+

1

2t²w⁰)−f^Lθ(x)−t(f^Lθ)⁰(x;d)

1

2t² converges to

1

1 + cot²θf⁰⁰

x₁− kx₂k cot θ; d₁− ¯x^T₂d₂cot θ, w₁ −h

¯

x^T₂w₂+ d^T₂J Φ(x₂)d₂i cot θ

+ 1

1 + tan²θf⁰⁰

x₁+ kx₂k tan θ; d₁ + ¯x^T₂d₂tan θ, w₁+h

¯

x^T₂w₂+ d^T₂J Φ(x₂)d₂i

tan θ .

Moreover, the second component of ^fLθ(x+td+

1

2t²w⁰)−f^Lθ(x)−t(f^Lθ)⁰(x;d)

1

2t² converges to

− cot θ 1 + cot²θ



f (x₁− kx₂k cot θ)J Φ(x₂)w₂+ J²Φ(x₂)(d₂, d₂) + 2f⁰(x1− kx2k cot θ; d1 − ¯x^T₂d2cot θ)J Φ(x2)d2

+ f⁰⁰ x₁− kx₂k cot θ; d₁− ¯x^T₂d₂cot θ, w₁− ¯x^T₂w₂ + d^T₂J Φ(x₂)d₂ cot θ
Φ(x₂)



+ tan θ

1 + tan²θ



f (x₁+ kx₂k tan θ)J Φ(x₂)w₂+ J²Φ(x₂)(d₂, d₂) + 2f⁰(x1+ kx2k tan θ; d1+ ¯x^T₂d2tan θ)J Φ(x2)d2

+ f⁰⁰(x₁ + kx₂k tan θ; d₁+ ¯x^T₂d₂tan θ, w₁ + ¯x^T₂w₂+ d^T₂J Φ(x₂)d₂ tan θ)Φ(x₂)

 . To sum up, we can conclude that

(f^Lθ)⁰⁰(x; d, w)

= f⁰⁰



x₁− kx₂k cot θ; d₁− ¯x^T₂d₂cot θ, w₁−



¯

x^T₂w₂+ d^T₂J Φ(x₂)d₂

 cot θ

 u¹_x

+f⁰⁰



x₁+ kx₂k tan θ; d₁+ ¯x^T₂d₂tan θ, w₁+



¯

x^T₂w₂+ d^T₂J Φ(x₂)d₂

 tan θ

 u²_x

+ 2

cot θ + tan θΓ₁J eΦ(x)d + 1

cot θ + tan θΓ₂



J eΦ(x)w + J²Φ(x)(d, d)e

 ,

where we use the facts that J eΦ(x)w = (0, J Φ(x₂)w₂) and J²Φ(x)(d, d) = (0, Je ²Φ(x₂)(d₂, d₂)).

“⇒” Suppose that f^Lθ is parabolic second-order directionally differentiable at x in the Hadamard sense. Given ˜d, ˜w ∈ R and ˜w⁰ → ˜w. To proceed, we also discuss the following two cases.

Case 1: For x₂ = 0, let d = ˜de, w⁰ = ˜w⁰e, and w = ˜we. Denote z := x + td +¹₂t²w⁰. Then f (x₁+ t ˜d +¹₂t²w˜⁰) − f (x₁) − tf⁰(x₁, ˜d)

1

2t² = f^Lθ(z) − f^Lθ(x) − t(f^Lθ)⁰(x; d)

1

2t² , e

 . Thus, we obtain f⁰⁰(x1; ˜d, ˜w) = h(f^Lθ)⁰⁰(x; d, w), ei.

Case 2: For x₂ 6= 0, let d = ˜du⁽¹⁾x , w⁰ = ˜w⁰u⁽¹⁾x , and w = ˜wu⁽¹⁾x . Then, we have x + td + 1

2t²w⁰ = (λ₁(x) + t ˜d + 1

2t²w˜⁰)u⁽¹⁾_x + λ₂(x)u⁽²⁾_x with t > 0 satisfying t ˜d + ¹₂t²w˜⁰ < λ₂(x) − λ₁(x). This implies

f^Lθ



x + td +1 2t²w⁰



= f (λ1(x) + t ˜d +1

2t²w˜⁰)u⁽¹⁾_x + f (λ2(x))u⁽²⁾_x and (f^Lθ)⁰(x; d) = f⁰(λ1(x); ˜d)u⁽¹⁾x . Thus,

f (λ₁(x) + t ˜d +¹₂t²w˜⁰) − f (λ₁(x)) − tf⁰(λ₁(x); ˜d)

1 2t²

= (1 + cot²θ) f^Lθ(x + td + ¹₂t²w⁰) − f^Lθ(x) − t(f^Lθ)⁰(x; d)

1

2t² , u¹_x

 , which says

f⁰⁰(λ₁(x); ˜d, ˜w) = (1 + cot²θ)D

(f^Lθ)⁰⁰(x; d, w), u⁽¹⁾_x E .

The similar arguments can be used for f at λ₂(x). From all the above, the proof is complete. 2

4 Second-order Tangent Sets

In this section, we turn our attention to f being the special function f (t) = max{t, 0}. In this case, the corresponding f^Lθ is just the projection operator associated with circular cone. For x ∈ L_θ, from [16], we know the tangent cone is given by

T_Lθ(x) := {d : dist(x + td, L_θ) = o(t), t ≥ 0}

= {d : Π_Lθ(x + td) − (x + td) = o(t), t ≥ 0}

= {d : Π⁰_Lθ(x; d) = d}, (8)

which, together with the formula of Π⁰_L

θ, yields

TL_θ(x) =







Rⁿ, if x ∈ intL_θ,

L_θ, if x = 0,

{d : d^T₂x2− d1x1tan²θ ≤ 0}, if x ∈ bdLθ/{0}.

Definition 4.1. [16, Definition 3.28] The set limits T_S^i,2(x, d) :=



w ∈ Rⁿ : dist



x + td +1 2t²w, S



= o(t²), t ≥ 0



and

T_S²(x, d) :=



w ∈ Rⁿ : ∃ t_n↓ 0 such that dist



x + t_nd +1 2t²_nw, S



= o(t²_n)



are called the inner and outer second-order tangent sets, respectively, to the set S at x in the direction d.

In [13], we have shown that the circular cone is second-order regular, which means T_L^i,2

θ(x; d) is equal to T_L²

θ(x; d) for all d ∈ TL_θ(x). Since the inner and outer second-order tangent sets are equal, we simply say that T_L²

θ(x; d) is the second-order tangent set.

Next, we provide two different approaches to establish the exact formula of second-order tangent set of circular cone. One is following from the parabolic second-order directional derivative of the spectral value λ₁(x), and the other is using the parabolic second-order directional derivative of projection operator Π_Lθ.

Theorem 4.1. Given x ∈ L_θ and d ∈ T_Lθ(x), then

T_L²_θ(x, d) =







Rⁿ, if d ∈ intTL_θ(x), TL_θ(d), if x = 0,

{w : w^T₂x₂cot θ − w₁x₁tan θ ≤ d²₁tan θ − kd₂k²cot θ}, otherwise.

Proof. First, we note that L_θ = {x : −λ₁(x) ≤ 0}. With this, we have w ∈ T_L²

θ(x; d) ⇐⇒ −λ₁(x + td + 1

2t²w + o(t²)) ≤ 0

⇐⇒ −λ₁(x) − tλ⁰₁(x; d) −1

2t²λ⁰⁰₁(x; d, w) + o(t²) ≤ 0. (9) The case of x ∈ intL_θ (corresponding to −λ₁(x) < 0) or x ∈ bdL_θ and d ∈ intTL_θ(x) (corresponding to λ₁(x) = 0 and −λ⁰₁(x; d) < 0) ensures that (9) holds for all w ∈ Rⁿ.

For the case x = 0 and d = 0, it follows from Theorem 3.2 and (9) that w ∈ T_L²

θ(x; d) =⇒ −w₁+ kw₂k cot θ ≤ 0 ⇐⇒ w ∈ L_θ.

Conversely, if w ∈ Lθ, then dist(¹₂t²w, Lθ) = 0 due to Lθ is a cone, which implies w ∈ T_L²

θ(x; d). Hence, T_L²

θ(x; d) = T_Lθ(x).

For the case x = 0 and d ∈ bdTL_θ(x)\{0} = bdL_θ\{0}, it follows from Theorem 3.2 and (9) that

w ∈ T_L²_θ(x; d) =⇒ −w1d1tan²θ + d^T₂w2 ≤ 0 ⇐⇒ w ∈ TL_θ(d).

Conversely, if w ∈ T_Lθ(d), then dist(d + tw, L_θ) = o(t) and hence dist(d + ¹₂tw, L_θ) = o(¹₂t) = o(t). Thus, we obtain dist(x + td +¹₂t²w, L_θ) = dist(td +¹₂t²w, L_θ) = o(t²), which means w ∈ T_L²

θ(x; d).

The case remained is x ∈ bdL_θ/{0} and d ∈ bdTL_θ(x), i.e., x₁ = kx₂k cot θ and d^T₂x₂ = d₁x₁tan²θ. Since x₂ 6= 0, −λ₁ is second-order differentiable at x. Hence it follows from Theorem 3.2 that

T_L²

θ(x; d) = {w : −λ⁰⁰₁(x; d, w) ≤ 0} =w : −x₁w₁tan θ + x^T₂w₂cot θ + kd₂k²cot θ − d²₁tan θ ≤ 0 , where the last step is due to ¯x^T₂d₂ = d₁tan θ. 2

As below, we provide the second approach to establish the formula of second-order tangent set by using the parabolic second-order directional derivative of projection oper- ator associated with circular cone. To this end, we need a technical lemma.

Lemma 4.1. For x ∈ Lθ and d ∈ TL_θ(x), we have

T_L²_θ(x, d) = {w : Π⁰⁰_Lθ(x; d, w) = w}.

Proof. The desired result follows from T_L²

θ(x, d) = {w : dist(x + td + 1

2t²w, L_θ) = o(t²), t ≥ 0}

= {w : ΠL_θ(x + td +1

2t²w) − (x + td + 1

2t²w) = o(t²), t ≥ 0}

= {w : ΠL_θ(x + td +1

2t²w) − ΠL_θ(x) − tΠ⁰_L

θ(x; d) −1

2t²w = o(t²), t ≥ 0}

= {w : Π⁰⁰_Lθ(x; d, w) = w},

where the third step uses the fact that d = Π⁰_L

θ(x; d) since d ∈ TL_θ(x) by (8). 2 Recall first from [15] that ΠL_θ, the projection operator, is the vector-valued function corresponding to f (t) = max{t, 0}. To present the second approach, we will also use the parabolic second-order directional derivative of the f (t) = max{t, 0}, which can be found in [10]. Now the second approach to prove Theorem 4.1 is given below.

Proof. Notice first that as x₁ > kx₂k cot θ or x₁ = kx₂k cot θ 6= 0 and d₁ ≥ ¯x^T₂d₂cot θ, then

2

tan θ + cot θΓ₁J eΦ(x)d + 1

tan θ + cot θΓ₂ J eΦ(x)w + J²Φ(x)(d, d)e

=



0, w₂−



¯

x^T₂w₂− (¯x^T₂d₂)²

kx₂k +kd₂k² kx₂k

 x₂ kx₂k

T

. (10)

As x₁ ≥ 0 and d₁ ≥ kd₂k cot θ, we know that 1

tan θ + cot θ



f⁰(x₁; d₁+kd₂k tan θ)−f⁰(x₁; d₁−kd₂k cot θ)



J eΦ(d)w =



0, w₂− ¯d^T₂w₂d¯₂

T

. (11) We point it out that, in the above formulas (10) and (11), we have applied the parabolic second-order directional derivative of the max-type function f (t) = max{t, 0}. To pro- ceed, we discuss the following three cases.

Case 1: For d ∈ intT_Lθ(x), we keep going to discuss three subcases.

Subcase (1): x = 0. Under this subcase, we see d ∈ intL_θ, i.e., d₁ > kd₂k cot θ. If d₂ = 0, then d₁ > 0 which yields

f⁰⁰(x₁; d₁, w₁− kw₂k cot θ)u¹_w+ f⁰⁰(x₁; d₁, w₁+ kw₂k tan θ)u²_w = w, ∀w ∈ Rⁿ. If d₂ 6= 0, it then follows from (11) that

(f^Lθ)⁰⁰(x; d, w) = (w₁− ¯d^T₂w₂cot θ)u⁽¹⁾_d + (w₁+ ¯d^T₂w₂tan θ)u⁽²⁾_d +



0, w₂ − ¯d^T₂w₂d¯₂

T

= w.

Subcase (2): x ∈ intL_θ. Under this subcase, it is clear that T_Lθ(x) = Rⁿ. If x₂ = 0, it follows from Theorem 3.3 that (f^Lθ)⁰⁰(x; d, w) = w whenever d₂ = 0 or d₂ 6= 0 due to x₁ > 0 in this case. If x₂ 6= 0, from (10), we know that

(f^Lθ)⁰⁰(x; d, w) =





w1



¯

x^T₂w₂+ d^T₂J Φ(x₂)d₂



x2

kx2k



+





0 w₂−



¯

x^T₂w₂+ d^T₂J Φ(x₂)d₂



x2

kx2k



= w.

Subcase (3): x ∈ bdL_θ/{0}. Then d ∈ intTL_θ(x) means d^T₂x₂ < d₁x₁tan²θ = d₁kx₂k tan θ, i.e., ¯x^T₂d₂cot θ < d₁. Thus, (f^Lθ)⁰⁰(x; d, w) = w for all w ∈ Rⁿ by the similar argument as above.

In summary, we have T_L²

θ(x, d) = Rⁿ in this case.

Case 2: For x = 0, since d ∈ TL_θ(x) = L_θ, we see that d₁ ≥ kd₂k cot θ. It only remains to show the case of d₁ = kd₂k cot θ. If d₂ = 0, then d₁ = 0, and hence

(f^Lθ)⁰⁰(x; d, w) = f⁰⁰(x₁; d₁, w₁− kw₂k cot θ)u¹_w+ f⁰⁰(x₁; d₁, w₁+ kw₂k tan θ)u²_w