Asia-Pacific Journal of Operational Research, vol.24, pp. 401-420, 2007

### On some NCP-functions based on the generalized Fischer-Burmeister function

Jein-Shan Chen ^{1}
Department of Mathematics
National Taiwan Normal University

Taipei, Taiwan 11677

June 2, 2005

(revised November 18, 2005) (second revised January 11, 2006)

Abstract In this paper, we study several NCP-functions for the nonlinear comple-
mentarity problem (NCP) which are indeed based on the generalized Fischer-Burmeister
*function, φ*_{p}*(a, b) = k(a, b)k*_{p}*−(a+b). It is well known that the NCP can be reformulated*
as an equivalent unconstrained minimization by means of merit functions involving NCP-
functions. Thus, we aim to investigate some important properties of these NCP-functions
that will be used in solving and analyzing the reformulation of the NCP.

Key words. NCP-function, complementarity, merit function, bounded level sets, sta- tionary point.

## 1 Introduction

*The nonlinear complementarity problem (NCP) [13, 20] is to find a point x ∈ IR** ^{n}* such
that

*x ≥ 0,* *F (x) ≥ 0,* *hx, F (x)i = 0,* (1)

*where h·, ·i is the Euclidean inner product and F = (F*_{1}*, F*_{2}*, · · · , F** _{n}*)

*maps from IR*

^{T}*to IR*

^{n}

^{n}*. We assume that F is continuously differentiable throughout this paper. The NCP*has attracted much attention due to its various applications in operations research, eco- nomics, and engineering [9, 13, 20].

There have been many methods proposed for solving the NCP [13, 20]. Among which, one of the most popular and powerful approaches that has been studied inten- sively recently is to reformulate the NCP as a system of nonlinear equations [17] or as an

1E-mail: jschen@math.ntnu.edu.tw, TEL: 886-2-29325417, FAX: 886-2-29332342.

unconstrained minimization problem [8, 10, 14]. Such a function that can constitute an
*equivalent unconstrained minimization problem for the NCP is called a merit function.*

In other words, a merit function is a function whose global minima are coincident with the solutions of the original NCP. For constructing a merit function, the class of func- tions, so-called NCP-functions and defined as below, serves an important role.

*A function φ : IR*^{2} *→ IR is called an NCP-function if it satisfies*

*φ(a, b) = 0* *⇐⇒* *a ≥ 0, b ≥ 0, ab = 0.* (2)

Many NCP-functions and merit functions have been explored during the past two decades [6, 16, 25, 26]. Among which, a popular NCP-function intensively studied recently is the well-known Fischer-Burmeister NCP-function [10, 11] defined as

*φ*_{FB}*(a, b) =√*

*a*^{2}*+ b*^{2}*− (a + b).* (3)

*With the above characterization of φ*_{FB}, the NCP is equivalent to a system of nonsmooth
equations:

Φ_{FB}*(x) =*

*φ*_{FB}*(x*_{1} *, F*_{1}*(x))*

*·*

*·*

*·*

*φ*_{FB}*(x*_{n}*, F*_{n}*(x))*

*= 0.* (4)

For each NCP-function, there is a natural merit function, Ψ_{FB} : IR^{n}*→ IR*_{+} given by
Ψ_{FB}*(x) :=* 1

2*kΦ*_{FB}*(x)k*^{2} = 1
2

X*n*

*i=1*

*φ*_{FB}*(x*_{i}*, F*_{i}*(x))*^{2}*,* (5)
from which the NCP can be recast as an unconstrained minimization:

*x∈IR*min* ^{n}*Ψ

_{FB}

*(x).*(6)

In this paper, we are particularly interested in the generalized Fischer-Burmeister
*function, i.e., φ** _{p}* : IR

^{2}

*→ IR given by*

*φ*_{p}*(a, b) := k(a, b)k*_{p}*− (a + b),* (7)
*where p is a positive integer greater than one and k(a, b)k** _{p}* =

^{q}

^{p}*|a|*

^{p}*+ |b|*

^{p}*means the p-*

*norm of (a, b). Notice that φ*

_{p}*reduces to the well known Fischer-Burmeister function φ*

_{FB}

*when p = 2 and its related properties were recently presented in [3, 4]. Corresponding*

*to φ*

*p*

*, we define ψ*

*p*: IR

^{2}

*→ IR*+ by

*ψ**p**(a, b) :=* 1

2*|φ**p**(a, b)|*^{2}*.* (8)

*Then both φ*_{p}*and ψ** _{p}* are NCP-functions and yield a merit function
Ψ

_{p}*(x) :=*

X*n*

*i=1*

*ψ*_{p}*(x*_{i}*, F*_{i}*(x)) =* 1
2

X*n*

*i=1*

*φ*_{p}*(x*_{i}*, F*_{i}*(x))*^{2}*,* (9)
from which the NCP can be reformulated as an unconstrained minimization:

*x∈IR*min* ^{n}*Ψ

_{p}*(x).*(10)

However, there has some limitations for the (generalized) Fischer-Burmeister functions
and some of its variants when dealing with monotone complementarity problem. In
particular, its natural merit function Ψ* _{p}* does not guarantee bounded level sets for this
class of problem which is an important class (see page 4 of [1]). Some modifications to the
Fischer-Burmeister have been proposed to conquer the above problem, see [16, 25]. In
this paper, we extend these modifications to the generalized Fischer-Burmeister function

*φ*

*. More specifically, we study the following NCP-functions:*

_{p}*φ*1*(a, b) := φ**p**(a, b) − αa*+*b*+*,* *α > 0,*
*φ*_{2}*(a, b) := φ*_{p}*(a, b) − α(ab)*_{+}*,* *α > 0,*
*φ*_{3}*(a, b) :=* ^{q}*[φ*_{p}*(a, b)]*^{2}*+ α(a*_{+}*b*_{+})^{2}*, α > 0,*
*φ*4*(a, b) :=* ^{q}*[φ**p**(a, b)]*^{2}*+ α[(ab)*+]^{2}*, α > 0,*

(11)

*The function φ*1 *is called penalized Fischer-Burmeister function when p = 2 and was*
*studied in [1]. The functions φ*_{2}*, φ*_{3}*, φ*_{4} *generalize the merit functions of p = 2, which*
*were discussed in [25, 27]. Note that for i = 1, 2, 3, 4, we have*

*φ*_{i}*(a, b) ≡ φ*_{p}*(a, b)* (12)

*for all (a, b) ∈ N**−* (this notation is used in [25]) where

*N*_{−}*:= {(a, b)| ab ≤ 0}.* (13)

*Thus, φ*_{i}*where i = 1, 2, 3, 4 are only different in the first or third quadrant.*

*Similarly, for each φ**i* *there is an associated ψ**i* : IR^{2} *→ IR*+ given by
*ψ*_{i}*(a, b) :=* 1

2*|φ*_{i}*(a, b)|*^{2} *i = 1, 2, 3, 4,* (14)
*which is also an NCP-function for every i. Moreover, for φ ∈ {φ*_{1}*, φ*_{2}*, φ*_{3}*, φ*_{4}*}, we can*
define

*Φ(x) =*

*φ(x*_{1} *, F*_{1}*(x))*

*·*

*·*

*·*
*φ(x*_{n}*, F*_{n}*(x))*

*,* (15)

from which the NCP is equivalent to the unconstrained minimization:

*x∈IR*min^{n}*Ψ(x)* (16)

where

*Ψ(x) :=* 1

2*kΦ(x)k*^{2} = 1
2

X*n*

*i=1*

*φ(x*_{i}*, F*_{i}*(x))*^{2} (17)

*is the natural merit function corresponding to φ ∈ {φ*1*, φ*2*, φ*3*, φ*4*}.*

The paper is organized as follows. In Sec. 2, we review some background definitions
*including monotonicity, P*_{0}-function, semismoothness, e.t.c. and known results about Ψ_{p}*and its related properties. In Sec. 3, we show that all (φ** _{i}*)

^{2}

*, i ∈ {1, 2, 3, 4} are continu-*

*ously differentiable and investigate properties of the merit function Ψ constructed via φ*

*i*

*with i ∈ {1, 2, 3, 4}. In particular, it provides bounded level sets for a monotone NCP*
with a strictly feasible point. In addition, we give conditions under which a stationary
point of Ψ is a solution of the NCP. In general, the analytic techniques used in this paper
are similar to those in [1, 8, 25] since the work is somewhat considered the extensions of
NCP-functions studied in those literatures.

Throughout this paper, IR^{n}*denotes the space of n-dimensional real column vectors*
and ^{T}*denotes transpose. For any differentiable function f : IR*^{n}*→ IR, ∇f (x) denotes*
*the gradient of f at x. For any differentiable mapping F = (F*1*, ..., F**m*)* ^{T}* : IR

^{n}*→ IR*

*,*

^{m}*∇F (x) = [∇F*_{1}*(x) · · · ∇F*_{m}*(x)] denotes the transpose Jacobian of F at x. We denote*
*by kxk*_{p}*the p-norm of x and by kxk the Euclidean norm of x. In this whole paper, we*
*assume p is a positive integer greater than one.*

## 2 Preliminaries

In this section, we recall some background concepts and materials which will play an
*important role in the subsequent analysis. Let F : IR*^{n}*→ IR*^{n}*. Then, (1) F is mono-*
*tone if hx − y, F (x) − F (y)i ≥ 0, for all x, y ∈ IR*^{n}*. (2) F is strictly monotone if*
*hx − y, F (x) − F (y)i > 0, for all x, y ∈ IR*^{n}*and x 6= y. (3) F is strongly monotone*
*with modulus µ > 0 if hx − y, F (x) − F (y)i ≥ µkx − yk*^{2}*, for all x, y ∈ IR*^{n}*. (4) F is a*
*P*_{0}-function if max

*1≤i≤n*
*xi6=yi*

*(x*_{i}*− y*_{i}*)(F*_{i}*(x) − F*_{i}*(y)) ≥ 0, for all x, y ∈ IR*^{n}*and x 6= y. (5) F is a*
*P -function if max*

*1≤i≤n**(x*_{i}*− y*_{i}*)(F*_{i}*(x) − F*_{i}*(y)) > 0, for all x, y ∈ IR*^{n}*and x 6= y. (6) F is a*
*uniform P -function with modulus µ > 0 if max*

*1≤i≤n**(x*_{i}*− y*_{i}*)(F*_{i}*(x) − F*_{i}*(y)) ≥ µkx − yk*^{2}, for
*all x, y ∈ IR*^{n}*. (7) F is a R*_{0}*-function if for every sequence {x*^{k}*} satisfying {kx*^{k}*k} → ∞,*
lim inf

*k→∞*

min_{i}*x*^{k}_{i}

*kxk*^{k}*≥ 0, and lim inf*

*k→∞*

min_{i}*F*_{i}*(x** ^{k}*)

*kxk*^{k}*≥ 0, there exists an index j such that*

*{x*^{k}_{j}*} → ∞ and {F*_{j}*(x*^{k}*)} → ∞.*

It is clear that strongly monotone functions are strictly monotone, and strictly mono-
*tone functions are monotone. Moreover, F is a P*_{0}*-function if F is monotone and F*
*is a uniform P -function with modulus µ > 0 if F is strongly monotone with modulus*
*µ > 0. In addition, when F is continuously differentiable, we have the following: (i) F is*
*monotone if and only if ∇F (x) is positive semi-definite for all x ∈ IR*^{n}*. (ii) F is strictly*
*monotone if ∇F (x) is positive definite for all x ∈ IR*^{n}*. (iii) F is strongly monotone if*
*and only if ∇F (x) is uniformly positive definite. An R*_{0}-function can be viewed as a
*generalization of a uniform P -function since every uniform P -function is an R*0-function
(see [2, Prop. 3.11]).

*A matrix M ∈ IR*^{n×n}*is a P*_{0}-matrix if every of its principal minors is nonnegative,
*and it is a P -matrix if every of its principal minors is positive. In addition, it is said to*
*be a R*_{0}-matrix if the following system has only zero solution:

*x ≥ 0,*

*M*_{i}*x = 0 if x*_{i}*> 0,*
*M*_{i}*x ≥ 0 if x*_{i}*= 0,*

*It is obvious that every P -matrix is also a P*_{0}-matrix and it is known that the Jacobian of
*every continuously differentiable P*_{0}*-function is a P*_{0}-matrix. For more properties about
*P -matrix and P*_{0}*-matrix, please refer to [7]. It is also known that F is an R*_{0}-function if
*and only if M is an R*0*-matrix when F is an affine function, see [2, Prop. 3.10].*

*Next, we recall the definition of semismoothness. First, we introduce that F is strictly*
*continuous (also called ‘locally Lipschitz continuous’) at x ∈ IR** ^{n}* [24, Chap. 9] if there

*exist scalars κ > 0 and δ > 0 such that*

*kF (y) − F (z)k ≤ κky − zk ∀y, z ∈ IR*^{n}*with ky − xk ≤ δ, kz − xk ≤ δ;*

*and F is strictly continuous if F is strictly continuous at every x ∈ IR*^{n}*. If δ can be*
*taken to be ∞, then F is Lipschitz continuous with Lipschitz constant κ. We say F is*
*directionally differentiable at x ∈ IR** ^{n}* if

*F*^{0}*(x; h) := lim*

*t→0*^{+}

*F (x + th) − F (x)*

*t* exists *∀h ∈ IR** ^{n}*;

*and F is directionally differentiable if F is directionally differentiable at every x ∈ IR** ^{n}*.

*Assume F : IR*

^{n}*→ IR*

^{m}*is strictly continuous. We say F is semismooth at x if F is*

*directionally differentiable at x and, for any V ∈ ∂F (x + h) (the generalized Jacobian),*we have

*F (x + h) − F (x) − V h = o(khk).*

*We say F is ρ-order semismooth at x (0 < ρ < ∞) if F is semismooth at x and, for any*
*V ∈ ∂F (x + h), we have*

*F (x + h) − F (x) − V h = O(khk*^{1+ρ}*).*

*We say F is semismooth (respectively, ρ-order semismooth) if F is semismooth (respec-*
*tively, ρ-order semismooth) at every x ∈ IR*^{k}*. We say F is strongly semismooth if it is*
1-order semismooth. Convex functions and piecewise continuously differentiable func-
tions are examples of semismooth functions. Examples of strongly semismooth functions
*include piecewise linear functions and LC*^{1} *functions meaning smooth functions with*
gradients being locally Lipschitz continuous (strictly continuous) [7, 22]. The composi-
*tion of two (respectively, ρ-order) semismooth functions is also a (respectively, ρ-order)*
semismooth function. The property of semismoothness plays an important role in nons-
mooth Newton methods [21, 23] as well as in some smoothing methods mentioned in the
previous section. For extensive discussions of semismooth functions, see [11, 18, 23].

*To end this section, we collect some useful properties of φ*_{p}*, ψ** _{p}* defined as in (7) and
(8), respectively, that will be used in the subsequent analysis. All the proofs can be found
in [3].

*Property 2.1 ([3, Prop. 3.1, Lem. 3.1]) Let φ** _{p}* : IR

^{2}

*→ IR be defined as (7). Then*

*(a) φ*

*p*

*is an NCP-function, i.e., it satisfies (2).*

*(b) φ*_{p}*is sub-additive, i.e., φ*_{p}*(w + w*^{0}*) ≤ φ*_{p}*(w) + φ(w*^{0}*) for all w, w*^{0}*∈ IR*^{2}*.*
*(c) φ*_{p}*is positive homogeneous, i.e., φ*_{p}*(αw) = αφ*_{p}*(w) for all w ∈ IR*^{2} *and α ≥ 0.*

*(d) φ**p* *is convex, i.e., φ**p**(αw + (1 − α)w*^{0}*) ≤ αφ**p**(w) + (1 − α)φ**p**(w*^{0}*) for all w, w*^{0}*∈ IR*^{2}
*and α ≥ 0.*

*(e) φ*_{p}*is Lipschitz continuous with L*_{1} = 1 +*√*

*2, i.e., |φ*_{p}*(w) − φ*_{p}*(w*^{0}*)| ≤ L*_{1}*kw − w*^{0}*k;*

*or with L*2 = 1 + 2^{(1−1/p)}*, i.e., |φ**p**(w) − φ**p**(w*^{0}*)| ≤ L*2*kw − w*^{0}*k**p* *for all w, w*^{0}*∈ IR*^{2}*.*
*(f) φ*_{p}*is semismooth.*

*(g) If {(a*^{k}*, b*^{k}*)} ⊆ IR*^{2} *with (a*^{k}*→ −∞) or (b*^{k}*→ −∞) or (a*^{k}*→ ∞ and b*^{k}*→ ∞), then*
*we have |φ*_{p}*(a*^{k}*, b*^{k}*)| → ∞ for k → ∞.*

*Property 2.2 ([3, Prop. 3.2]) Let φ*_{p}*, ψ*_{p}*be defined as (7) and (8), respectively. Then*
*(a) ψ*_{p}*is an NCP-function, i.e., it satisfies (2).*

*(b) ψ*_{p}*(a, b) ≥ 0 for all (a, b) ∈ IR*^{2}*.*

*(c) ψ*_{p}*is continuously differentiable everywhere.*

*(d) ∇*_{a}*ψ*_{p}*(a, b) · ∇*_{b}*ψ*_{p}*(a, b) ≥ 0 for all (a, b) ∈ IR*^{2}*. The equality holds ⇐⇒ φ*_{p}*(a, b) = 0.*

*(e) ∇**a**ψ**p**(a, b) = 0 ⇐⇒ ∇**b**ψ**p**(a, b) = 0 ⇐⇒ φ**p**(a, b) = 0.*

From these properties, it was proved in [3] that Ψ*p**(x) ≥ 0 for all x ∈ IR** ^{n}* and
Ψ

_{p}*(x) = 0 if and only if x solves the NCP (1), where Ψ*

*: IR*

_{p}

^{n}*→ IR is defined as (9).*

*Moreover, suppose that the NCP has at least one solution. Then x is a global minimizer*
of Ψ_{p}*if and only if x solves the NCP. In addition, it was also shown in [3] that if F is*
*either monotone or P*0-function, then every stationary point of Ψ*p* is a global minima of
(10); and therefore solves the original NCP. We will investigate the analogous results for
*the merit function Ψ which is based on φ** _{i}* studied in this paper. On the other hand, as
mentioned the natural merit function induced from the generalized Fischer-Burmeister
(which behaves like the Fischer-Burmeister function) does not guarantee bounded level

*sets under the assumption of F being monotone. Instead, there needs that F is strongly*

*monotone or uniform P -function to ensure that the property is held. Another main*purpose of this work is to obtain same results for the merit function Ψ studied in this

*paper under the weaker assumption that F is monotone only (see Sec. 4).*

## 3 *Properties of φ and ψ*

*In this section, we investigate properties of φ ∈ {φ*_{1}*, φ*_{2}*, φ*_{3}*, φ*_{4}*} and ψ ∈ {ψ*_{1}*, ψ*_{2}*, ψ*_{3}*, ψ*_{4}*}*
*defined as in (11) and (14), respectively. These include strong semismoothness of φ and*
*continuous differentiability of ψ. First, we denote*

*N*_{φ}*:= {(a, b)| a ≥ 0, b ≥ 0, ab = 0}.* (18)
*This notation is adopted from [1] and it is easy to see that (a, b) ∈ N*_{φ}*if and only if (a, b)*
*satisfies (2). Now we are ready to show the favorable properties of φ and ψ.*

*Proposition 3.1 Let φ ∈ {φ*_{1}*, φ*_{2}*, φ*_{3}*, φ*_{4}*} be defined as in (11). Then*
*(a) φ(a, b) = 0 ⇐⇒ (a, b) ∈ N**φ* *⇐⇒ (a, b) satisfies (2).*

*(b) φ is strongly semismooth.*

*(c) Let {a*^{k}*}, {b*^{k}*} ⊆ IR be any two sequences such that either a*^{k}_{+}*b*^{k}_{+} *→ ∞ or a*^{k}*→ −∞*

*or b*^{k}*→ −∞. Then |φ(a*^{k}*, b*^{k}*)| → ∞ for k → ∞.*

*Proof. (a) It is enough to prove the first equivalence. Suppose φ(a, b) = 0, for i = 2, 3, 4,*
*φ*_{i}*(a, b) = 0 yields φ*_{p}*(a, b) = 0 which says (a, b) ∈ N*_{φ}*by Property 2.1(a). For i = 1,*
*φ*_{1}*(a, b) = 0 implies φ*_{p}*(a, b) = αa*_{+}*b*_{+}*. Since α could be any arbitrary positive number,*

*the above leads to φ*_{p}*(a, b) = a*_{+}*b*_{+} *= 0 which which says (a, b) ∈ N** _{φ}* by Property 2.1(a)
again.

*On the other hand, suppose (a, b) ∈ N**φ* *then φ**p**(a, b) = 0 by by Property 2.1(a). Since*
*a ≥ 0, b ≥ 0, we obtain a*_{+}*b*_{+} *= ab = 0. Hence we see that all φ*_{i}*(a, b) = 0, i = 1, 2, 3, 4.*

*(b) The verification of strong semismoothness of φ is a routine work which can be done*
as in [25, Lemma 1]. We omit it.

*(c) This follows from Property 2.1(g) and definition of (·)*_{+}. *2*

*Proposition 3.2 Let Φ be defined as in (15) with φ ∈ {φ*_{1}*, φ*_{2}*, φ*_{3}*, φ*_{4}*}. Then*
*(a) Φ is semismooth.*

*(b) Φ is strongly semismooth if every F*_{i}*is LC*^{1} *function.*

*Proof. By using Prop. 3.1(b) and the fact that every LC*^{1} function is strongly semis-
mooth, the results follow. *2*

The following is a technical lemma which describes the generalized gradients of all
*φ*_{i}*, i = 1, 2, 3, 4 defined as in (11). It will be used for proving Prop. 3.3.*

*Lemma 3.1 Let φ*1*, φ*2*, φ*3*, φ*4 *be defined as (11).*

*(a) The generalized gradient ∂φ*_{1}*(a, b) of φ*_{1} *at a point (a, b) is equal to the set of all*
*(v**a**, v**b**) such that*

*(v*_{a}*, v** _{b}*) =

µ

*a*^{p−1}

*k(a,b)k*^{p−1}*p* *− 1,* _{k(a,b)k}^{b}^{p−1}*p−1*

*p* *− 1*

¶

*− α*

µ

*b*_{+}*∂a*_{+}*, a*_{+}*∂b*_{+}

¶

*,*
*if (a, b) 6= (0, 0) and p is even,*

µ

*sgn(a)·a*^{p−1}

*k(a,b)k*^{p−1}*p* *− 1,* ^{sgn(b)·b}_{k(a,b)k}*p−1*^{p−1}

*p* *− 1*

¶

*− α*

µ

*b*_{+}*∂a*_{+}*, a*_{+}*∂b*_{+}

¶

*,*
*if (a, b) 6= (0, 0) and p is odd,*
*(ξ − 1, ζ − 1),* *if (a, b) = (0, 0),*

(19)

*where (ξ, ζ) is any vector satisfying |ξ|*^{p−1}^{p}*+ |ζ|*^{p−1}^{p}*≤ 1 and*

*∂z*_{+} =

*1,* *if z > 0,*
*[0, 1], if z = 0,*
*0,* *if z < 0.*

*(b) The generalized gradient ∂φ*_{2}*(a, b) of φ*_{2} *at a point (a, b) is equal to the set of all*
*(v**a**, v**b**) such that*

*(v*_{a}*, v** _{b}*) =

µ *a*^{p−1}*k(a, b)k*^{p−1}*p*

*− 1,* *b*^{p−1}*k(a, b)k*^{p−1}*p*

*− 1*

¶

*− α(b, a),*
*if (a, b) 6= (0, 0), ab > 0 and p is even,*
*(v**a**, v**b*) =

µ *a*^{p−1}*k(a, b)k*^{p−1}*p*

*− 1,* *b*^{p−1}*k(a, b)k*^{p−1}*p*

*− 1*

¶

*− α(b, a) · [0, 1],*
*if (a, b) 6= (0, 0), ab = 0 and p is even,*

*(v*_{a}*, v** _{b}*) =

µ *a*^{p−1}*k(a, b)k*^{p−1}*p*

*− 1,* *b*^{p−1}*k(a, b)k*^{p−1}*p*

*− 1*

¶

*,*
*if (a, b) 6= (0, 0), ab < 0 and p is even,*
*(v**a**, v**b*) =

µ*sgn(a) · a*^{p−1}*k(a, b)k*^{p−1}*p*

*− 1,* *sgn(b) · b*^{p−1}*k(a, b)k*^{p−1}*p*

*− 1*

¶

*− α(b, a),*

*if (a, b) 6= (0, 0), ab > 0 and p is odd,* (20)
*(v*_{a}*, v** _{b}*) =

µ*sgn(a) · a*^{p−1}*k(a, b)k*^{p−1}*p*

*− 1,* *sgn(b) · b*^{p−1}*k(a, b)k*^{p−1}*p*

*− 1*

¶

*− α(b, a) · [0, 1],*
*if (a, b) 6= (0, 0), ab = 0 and p is odd,*

*(v**a**, v**b*) =

µ*sgn(a) · a*^{p−1}*k(a, b)k*^{p−1}*p*

*− 1,* *sgn(b) · b*^{p−1}*k(a, b)k*^{p−1}*p*

*− 1*

¶

*,*
*if (a, b) 6= (0, 0), ab < 0 and p is odd,*
*(v**a**, v**b**) = (ξ − 1, ζ − 1) − α(b, a) · [0, 1],*

*if (a, b) = (0, 0),*

*where (ξ, ζ) is any vector satisfying |ξ|*^{p−1}^{p}*+ |ζ|*^{p−1}^{p}*≤ 1.*

*(c) φ*_{3} *is continuously differentiable everywhere except at (0, 0) with*

*∇*_{a}*φ*_{3}*(a, b) =*

*φ**p**(a,b)·*

·

*ap−1*
*k(a,b)k**p−1*

*p*

*−1*

¸

*+α(a*+*)(b*+)^{2}

*φ*3*(a,b)* *,*

*if (a, b) 6= (0, 0), and p is even,*

*φ**p**(a,b)·*

·

*sgn(a)·ap−1*
*k(a,b)k**p−1*

*p*

*−1*

¸

*+α(a*+*)(b*+)^{2}

*φ*3*(a,b)* *,*

*if (a, b) 6= (0, 0), and p is odd,*

(21)

*∇*_{b}*φ*_{3}*(a, b) =*

*φ**p**(a,b)·*

·

*bp−1*
*k(a,b)k**p−1*

*p*

*−1*

¸

*+α(a*+)^{2}*(b*+)

*φ*3*(a,b)* *,*

*if (a, b) 6= (0, 0), and p is even,*

*φ**p**(a,b)·*

·

*sgn(b)·bp−1*
*k(a,b)k**p−1*

*p*

*−1*

¸

*+α(a*+)^{2}*(b*+)

*φ*3*(a,b)* *,*

*if (a, b) 6= (0, 0), and p is odd,*

(22)

*and ∂φ*_{3}*(0, 0) = (v*_{a}*, v*_{b}*) where (v*_{a}*, v*_{b}*) ∈ (−∞, ∞).*

*(d) φ*_{4} *is continuously differentiable everywhere except at (0, 0) with*

*∇*_{a}*φ*_{4}*(a, b) =*

*φ**p**(a,b)·*

·

*ap−1*
*k(a,b)k**p−1*

*p*

*−1*

¸

*+α(ab)*+*·b*

*φ*4*(a,b)* *,*

*if (a, b) 6= (0, 0), and p is even,*

*φ**p**(a,b)·*

·

*sgn(a)·ap−1*
*k(a,b)k**p−1*

*p*

*−1*

¸

*+α(ab)*+*·b*

*φ*4*(a,b)* *,*

*if (a, b) 6= (0, 0), and p is odd,*

(23)

*∇*_{b}*φ*_{4}*(a, b) =*

*φ**p**(a,b)·*

·

*bp−1*
*k(a,b)k**p−1*

*p*

*−1*

¸

*+α(ab)*+*·a*

*φ*4*(a,b)* *,*

*if (a, b) 6= (0, 0), and p is even,*

*φ**p**(a,b)·*

·

*sgn(b)·bp−1*
*k(a,b)k**p−1*

*p*

*−1*

¸

*+α(ab)*+*·a*

*φ*4*(a,b)* *,*

*if (a, b) 6= (0, 0), and p is odd,*

(24)

*and ∂φ*_{4}*(0, 0) = (v*_{a}*, v*_{b}*) where (v*_{a}*, v*_{b}*) ∈ (−∞, ∞).*

*Proof. (a) First, we note that φ**p* is continuously differentiable everywhere except at
*(0, 0) (see [3]). Hence, by the Corollary to Prop. 2.2.1 in [5], φ** _{p}* is strictly differentiable

*everywhere except at the origin. Let φ*

_{+}

*(a, b) := a*

_{+}

*b*

_{+}

*. Then φ*

_{+}is strictly differentiable

*at the origin as proved in [1, Prop. 2.1]. Both φ*

_{1}

*and φ*

_{+}are strongly semismooth functions, we know that they are locally Lipschitz (strictly continuous) functions. Thus, the Corollary 2 to Prop. 2.3.3 in [5] yields

*∂φ*_{1}*(a, b) = ∂φ*_{p}*(a, b) − α · ∂φ*_{+}*(a, b).*

*On the other hand, the generalized gradient of φ** _{p}* can be verified as below (see [3]):

*∂φ*_{p}*(a, b) =*

µ *a*^{p−1}*k(a, b)k*^{p−1}*p*

*− 1,* *b*^{p−1}*k(a, b)k*^{p−1}*p*

*− 1*

¶

*, if (a, b) 6= (0, 0) and p is even,*

*∂φ*_{p}*(a, b) =*

µ*sgn(a) · a*^{p−1}*k(a, b)k*^{p−1}*p*

*− 1,* *sgn(b) · b*^{p−1}*k(a, b)k*^{p−1}*p*

*− 1*

¶

*, if (a, b) 6= (0, 0) and p is odd,*

*∂φ**p**(a, b) = (ξ − 1, ζ − 1), if (a, b) = (0, 0),* (25)

*where (ξ, ζ) is any vector satisfying |ξ|*^{p−1}^{p}*+ |ζ|*^{p−1}^{p}*≤ 1. In addition, it was already shown*
in [1, Prop. 2.1] that

*∂φ*_{+}*(a, b) = (b*_{+}*∂a*_{+}*, a*_{+}*∂b*_{+}*).*

Thus, the desired results follow.

(b) Following the same arguments as in part(a) and using the fact that

*∂(ab)*+ =

*(b, a),* *if ab > 0,*
*(0, 0),* *if ab < 0,*
*(b, a) · [0, 1], if ab = 0,*
the desired results hold.

*(c) It is known that (φ** _{p}*)

^{2}

*and (a*

_{+}

*b*

_{+})

^{2}are continuously differentiable. Then the desired results follow by direct computations using the chain rule and the fact that

*∂(√*
*z) =*

1
2*√*

*z,* *if z > 0,*
*(−∞, ∞), if z = 0.*

(d) Same arguments as part(c). *2*

*Proposition 3.3 Let ψ ∈ {ψ*_{1}*, ψ*_{2}*, ψ*_{3}*, ψ*_{4}*} be defined as in (14). Then*
*(a) ψ(a, b) = 0 ⇐⇒ (a, b) ∈ N*_{φ}*⇐⇒ (a, b) satisfies (2).*

*(b) ψ is continuously differentiable on IR*^{2}*.*
*(c) ∇*_{a}*ψ(a, b) · ∇*_{b}*ψ(a, b) ≥ 0 for all (a, b) ∈ IR*^{2}*.*

*(d) ψ(a, b) = 0 ⇐⇒ ∇ψ(a, b) = 0 ⇐⇒ ∇**a**ψ(a, b) = 0 ⇐⇒ ∇**b**ψ(a, b) = 0.*

Proof. (a) The proof is straightforward by the same arguments as in Prop. 3.1(a).

(b) The ideas for the proof are indeed borrowed from [8, Prop. 3.4].

*For i = 1 and p is even, ψ*_{1}*(a, b) =* ^{1}_{2}*(φ*_{1}*(a, b))*^{2}. By the chain rule (see [5, Theorem
*2.2.4]), we obtain ∂ψ*_{1}*(a, b) = ∂φ*_{1}*(a, b)*^{T}*φ*_{1}*(a, b). We will show that ∂φ*_{1}*(a, b)*^{T}*φ*_{1}*(a, b) is*
*single-valued for all (a, b) ∈ IR*^{2} *because the zero of φ*1 cancels the multi-valued portion
*of ∂φ*_{1}*(a, b)** ^{T}*. To see this, we discuss several cases as below.

*(i) If a > 0, b > 0, then (b*_{+}*∂a*_{+}*, a*_{+}*∂b*_{+}*) = (b, a) which is single-valued. Hence, by (19),*
*it is easy to see that ∂φ*_{1}*(a, b)*^{T}*φ*_{1}*(a, b) is single-valued.*

*(ii) If a > 0, b < 0, then (b*+*∂a*+*, a*+*∂b*+*) = (0, a) which is single-valued. Hence, by (19),*

*∂φ*_{1}*(a, b)*^{T}*φ*_{1}*(a, b) is single-valued.*

*(iii) If a > 0, b = 0, then (b*_{+}*∂a*_{+}*, a*_{+}*∂b*_{+}*) = (0, a · [0, 1]) which is multi-valued. However,*
*under this case, we observe that φ*_{1}*(a, b) = k(a, b)k*_{p}*− (a + b) − αa*_{+}*b*_{+} = 0. Hence,

*∂φ*1*(a, b)*^{T}*φ*1*(a, b) is still single-valued.*

*(iv) If a < 0, b > 0 or a < 0, b < 0, or a < 0, b = 0, then (b*_{+}*∂a*_{+}*, a*_{+}*∂b*_{+}*) all equals (0, 0)*
*which is single-valued. Hence, by (19), ∂φ*_{1}*(a, b)*^{T}*φ*_{1}*(a, b) is single-valued.*

*(v) If a = 0, b > 0, then (b*_{+}*∂a*_{+}*, a*_{+}*∂b*_{+}*) = (b · [0, 1], 0) which is multi-valued. However,*

*under this case, we observe that φ*_{1}*(a, b) = k(a, b)k*_{p}*− (a + b) − αa*_{+}*b*_{+} = 0. Hence,

*∂φ*1*(a, b)*^{T}*φ*1*(a, b) is still single-valued.*

*(vi) If a = 0, b < 0, then then (b*_{+}*∂a*_{+}*, a*_{+}*∂b*_{+}*) = (0, 0) which is single-valued. Hence, by*
*(19), ∂φ*_{1}*(a, b)*^{T}*φ*_{1}*(a, b) is single-valued.*

*(vii) If a = 0, b = 0 then φ*_{1}*(a, b) = 0. Hence, ∂φ*_{1}*(a, b)*^{T}*φ*_{1}*(a, b) is single-valued.*

*Thus, by applying the Corollary to Theorem 2.2.4 in [5], the above facts yield that ψ*1

*is continuously differentiable everywhere. For p is odd, going over the same cases, the*
proof follows.

*For i = 2, ψ*_{2}*(a, b) =* ^{1}_{2}*(φ*_{2}*(a, b))*^{2}*. We discuss the following cases : (i) (a, b) 6= (0, 0) and*
*ab > 0, (ii) (a, b) 6= (0, 0) and ab = 0, (iii) (a, b) 6= (0, 0) and ab < 0, (iv) (a, b) = (0, 0).*

*From (20), we know that ∂φ*_{2}*(a, b) becomes multi-valued when ab = 0 or (a, b) = (0, 0).*

*However, φ*_{2}*(a, b) = 0 under these two cases which implies that ∂φ*_{2}*(a, b)*^{T}*φ*_{2}*(a, b) is still*
*single-valued. Hence, ψ*_{2} is continuously differentiable everywhere by the Corollary to
Theorem 2.2.4 in [5] again.

*For i = 3, 4, from (21)-(24), it is trivial that ∂φ*_{3}*(a, b), ∂φ*_{4}*(a, b) are single-valued when*
*(a, b) 6= (0, 0). When (a, b) = (0, 0), we observe that φ*_{3}*(a, b) = φ*_{4}*(a, b) = 0. Hence,*

*∂φ*3*(a, b)*^{T}*φ*3*(a, b) and ∂φ*4*(a, b)*^{T}*φ*4*(a, b) are still single-valued, which yield that ψ*3*, ψ*4

are continuously differentiable everywhere by the same reason as above.

*(c) For i = 1, ψ*_{1} = ^{1}_{2}*(φ*_{1})^{2}, we employ and go over the cases discussed as in part (b).

*(i) If a > 0, b > 0, then (b*_{+}*∂a*_{+}*, a*_{+}*∂b*_{+}*) = (b, a). Hence, from (19), we obtain that*

*∇**a**ψ*1*(a, b) =*

Ã *a*^{p−1}*k(a, b)k*^{p−1}*p*

*− 1 − αb*

!

*φ*1*(a, b), ∇**b**ψ*1*(a, b) =*

Ã *b*^{p−1}*k(a, b)k*^{p−1}*p*

*− 1 − αa*

!

*φ*1*(a, b),*
*for both p are even and odd. Then, ∇*_{a}*ψ*_{1}*(a, b) · ∇*_{b}*ψ*_{1}*(a, b) equals*

µ *a*^{p−1}*k(a, b)k*^{p−1}*p*

*− 1 − αb*

¶µ *b*^{p−1}*k(a, b)k*^{p−1}*p*

*− 1 − αa*

¶

*φ*^{2}_{1}*(a, b).*

Since,

¯¯

¯¯ ^{a}

*p−1*

*k(a,b)k*^{p−1}*p*

¯¯

¯¯*≤ 1,*

¯¯

¯¯ ^{b}

*p−1*

*k(a,b)k*^{p−1}*p*

¯¯

¯¯*≤ 1, and αa > 0, αb > 0, we know*

µ *a*^{p−1}*k(a, b)k*^{p−1}*p*

*− 1 − αb*

¶

*< 0 and*

µ *b*^{p−1}*k(a, b)k*^{p−1}*p*

*− 1 − αa*

¶

*< 0,*
*which implies that ∇**a**ψ*1*(a, b) · ∇**b**ψ*1*(a, b) ≥ 0.*

*(ii) If a > 0, b < 0, then (b*_{+}*∂a*_{+}*, a*_{+}*∂b*_{+}*) = (0, a). Hence, from (19), we have*

*∇*_{a}*ψ*_{1}*(a, b) =*

Ã *a*^{p−1}*k(a, b)k*^{p−1}*p*

*− 1*

!

*φ*_{1}*(a, b), ∇*_{b}*ψ*_{1}*(a, b) =*

Ã *b*^{p−1}*k(a, b)k*^{p−1}*p*

*− 1 − αa*

!

*φ*_{1}*(a, b),*
*for p is even; and*

*∇**a**ψ*1*(a, b) =*

Ã *a*^{p−1}*k(a, b)k*^{p−1}*p*

*− 1*

!

*φ*1*(a, b), ∇**b**ψ*1*(a, b) =*

Ã *−b*^{p−1}*k(a, b)k*^{p−1}*p*

*− 1 − αa*

!

*φ*1*(a, b),*

*for p is odd. Again, since*

¯¯

¯¯ ^{a}

*p−1*

*k(a,b)k*^{p−1}*p*

¯¯

¯¯ *≤ 1,*

¯¯

¯¯ ^{b}

*p−1*

*k(a,b)k*^{p−1}*p*

¯¯

¯¯ *≤ 1, and αa > 0, it can be easily*
*verified that ∇**a**ψ*1*(a, b) · ∇**b**ψ*1*(a, b) ≥ 0.*

*(iii) If a > 0, b = 0, then φ*_{1}*(a, b) = 0 which says ∇*_{a}*ψ*_{1}*(a, b) = 0 = ∇*_{b}*ψ*_{1}*(a, b). Hence,*

*∇*_{a}*ψ*_{1}*(a, b) · ∇*_{b}*ψ*_{1}*(a, b) = 0.*

*(iv) If a < 0, b > 0 or a < 0, b < 0, or a < 0, b = 0, then (b*_{+}*∂a*_{+}*, a*_{+}*∂b*_{+}*) = (0, 0). Hence,*
from (19), we have

*∇*_{a}*ψ*_{1}*(a, b) =*

Ã *a*^{p−1}*k(a, b)k*^{p−1}*p*

*− 1*

!

*φ*_{1}*(a, b), ∇*_{b}*ψ*_{1}*(a, b) =*

Ã *b*^{p−1}*k(a, b)k*^{p−1}*p*

*− 1*

!

*φ*_{1}*(a, b),*
*for p is even; and*

*∇*_{a}*ψ*_{1}*(a, b) =*

Ã*sgn(a) · a*^{p−1}*k(a, b)k*^{p−1}*p*

*− 1*

!

*φ*_{1}*(a, b), ∇*_{b}*ψ*_{1}*(a, b) =*

Ã*sgn(b) · b*^{p−1}*k(a, b)k*^{p−1}*p*

*− 1*

!

*φ*_{1}*(a, b),*

*for p is odd. Again, by*

¯¯

¯¯ ^{a}

*p−1*

*k(a,b)k*^{p−1}*p*

¯¯

¯¯*≤ 1, and*

¯¯

¯¯ ^{b}

*p−1*

*k(a,b)k*^{p−1}*p*

¯¯

¯¯*≤ 1, the desired inequality holds.*

*(v) If a = 0, b > 0, then φ*_{1}*(a, b) = 0 which says ∇*_{a}*ψ*_{1}*(a, b) = 0 = ∇*_{b}*ψ*_{1}*(a, b). Hence,*

*∇*_{a}*ψ*_{1}*(a, b) · ∇*_{b}*ψ*_{1}*(a, b) = 0.*

*(vi) If a = 0, b < 0, then then (b*_{+}*∂a*_{+}*, a*_{+}*∂b*_{+}*) = (0, 0). Hence, from (19), we have*

*∇*_{a}*ψ*_{1}*(a, b) = −φ*_{1}*(a, b), ∇*_{b}*ψ*_{1}*(a, b) =*

Ã *b*^{p−1}*k(a, b)k*^{p−1}*p*

*− 1*

!

*φ*_{1}*(a, b),*
*for p is even; and*

*∇*_{a}*ψ*_{1}*(a, b) = −φ*_{1}*(a, b), ∇*_{b}*ψ*_{1}*(a, b) =*

Ã *−b*^{p−1}*k(a, b)k*^{p−1}*p*

*− 1*

!

*φ*_{1}*(a, b),*

*for p is odd. By the same reasons as in previous discussions, we obtain that ∇*_{a}*ψ*_{1}*(a, b) ·*

*∇**b**ψ*1*(a, b) ≥ 0.*

*(vii) If a = 0, b = 0, then φ*_{1}*(a, b) = 0. Hence, ∇*_{a}*ψ*_{1}*(a, b) = 0 = ∇*_{b}*ψ*_{1}*(a, b) and*

*∇*_{a}*ψ*_{1}*(a, b) · ∇*_{b}*ψ*_{1}*(a, b) = 0.*

*For i = 2, ψ*_{2} = ^{1}_{2}*(φ*_{2})^{2}, we discuss discuss four cases as in part (b).

*(i) If (a, b) 6= (0, 0) and ab > 0, from (20), we have*

*∇*_{a}*ψ*_{2}*(a, b) =*

Ã *a*^{p−1}*k(a, b)k*^{p−1}*p*

*− 1 − αb*

!

*φ*_{2}*(a, b), ∇*_{b}*ψ*_{2}*(a, b) =*

Ã *b*^{p−1}*k(a, b)k*^{p−1}*p*

*− 1 − αa*

!

*φ*_{2}*(a, b),*
*for p is even; and*

*∇*_{a}*ψ*_{2}*(a, b) =*

Ã*sgn(a) · a*^{p−1}*k(a, b)k*^{p−1}*p*

*− 1 − αb*

!

*φ*_{2}*(a, b), ∇*_{b}*ψ*_{2}*(a, b) =*

Ã*sgn(b) · b*^{p−1}*k(a, b)k*^{p−1}*p*

*− 1 − αa*

!

*φ*_{2}*(a, b),*
*for p is odd. By the same reasons as in previous discussions, it can be easily verified that*

*∇*_{a}*ψ*_{1}*(a, b) · ∇*_{b}*ψ*_{1}*(a, b) ≥ 0.*

*(ii) If (a, b) 6= (0, 0) and ab = 0, then φ*_{2}*(a, b) = 0. Hence, ∇*_{a}*ψ*_{2}*(a, b) = 0 = ∇*_{b}*ψ*_{2}*(a, b)*
*and ∇**a**ψ*2*(a, b) · ∇**b**ψ*2*(a, b) = 0.*

*(iii) If (a, b) 6= (0, 0) and ab < 0, the arguments are the same as case (iv) for i = 1 except*
*that φ*_{1} *is replaced by φ*_{2}.

*(iv) If (a, b) = (0, 0), then φ*_{2}*(a, b) = 0. Hence, ∇*_{a}*ψ*_{2}*(a, b) = 0 = ∇*_{b}*ψ*_{2}*(a, b) and*

*∇**a**ψ*2*(a, b) · ∇**b**ψ*2*(a, b) = 0.*

*For i = 3, ψ*_{3} = ^{1}_{2}*(φ*_{3})^{2}, we have two cases as below.

*(i) If (a, b) 6= (0, 0), from (21)-(22), we have*

*∇*_{a}*ψ*_{3}*(a, b) = φ*_{p}*(a, b)*

µ *a*^{p−1}*k(a, b)k*^{p−1}*p*

*− 1*

¶

*+ α(a*_{+}*)(b*_{+})^{2}*,*

*∇*_{b}*ψ*_{3}*(a, b) = φ*_{p}*(a, b)*

µ *b*^{p−1}*k(a, b)k*^{p−1}*p*

*− 1*

¶

*+ α(a*_{+})^{2}*(b*_{+}*),*
*for p is even; and*

*∇*_{a}*ψ*_{3}*(a, b) = φ*_{p}*(a, b)*

µ*sgn(a) · a*^{p−1}*k(a, b)k*^{p−1}*p*

*− 1*

¶

*+ α(a*_{+}*)(b*_{+})^{2}*,*

*∇**b**ψ*3*(a, b) = φ**p**(a, b)*

µ*sgn(b) · b*^{p−1}*k(a, b)k*^{p−1}*p*

*− 1*

¶

*+ α(a*+)^{2}*(b*+*),*

*for p is odd. Thus, ∇*_{a}*ψ*_{3}*(a, b) · ∇*_{b}*ψ*_{3}*(a, b) equals*
*φ*^{2}_{p}*(a, b)*

µ

*a*^{p−1}*k(a,b)k*^{p−1}*p* *− 1*

¶µ

*b*^{p−1}*k(a,b)k*^{p−1}*p* *− 1*

¶

*+ α*^{2}*(a*_{+})^{3}*(b*_{+})^{3}
*+ φ*_{p}*(a, b)*

µ

*a*^{p−1}*k(a,b)k*^{p−1}*p* *− 1*

¶

*α(a*_{+})^{2}*(b*_{+}*) + φ*_{p}*(a, b)*

µ

*b*^{p−1}*k(a,b)k*^{p−1}*p* *− 1*

¶

*α(a*_{+}*)(b*_{+})^{2}
or

*φ*^{2}_{p}*(a, b)*

µ

*sgn(a)·a*^{p−1}*k(a,b)k*^{p−1}*p* *− 1*

¶µ

*sgn(b)·b*^{p−1}*k(a,b)k*^{p−1}*p* *− 1*

¶

*+ α*^{2}*(a*+)^{3}*(b*+)^{3}
*+ φ*_{p}*(a, b)*

µ

*sgn(a)·a*^{p−1}*k(a,b)k*^{p−1}*p* *− 1*

¶

*α(a*_{+})^{2}*(b*_{+}*) + φ*_{p}*(a, b)*

µ

*sgn(b)·b*^{p−1}*k(a,b)k*^{p−1}*p* *− 1*

¶

*α(a*_{+}*)(b*_{+})^{2}*.*

Note that in the above expressions, it is trivial that the first and second terms are nonnegative. We also notice that

*(a*_{+}*)(b*_{+}) =

( *ab, if a > 0, b > 0*
*0,* *else.*

*Therefore, we only need to consider the subcase of a > 0, b > 0 for the third and fourth*
terms. In fact, summing up the third and fourth term under this subcase gives

*αab · φ**p**(a, b)*

"µ *a*^{p−1}*k(a, b)k*^{p−1}*p*

*− 1*

¶

*a +*

µ *b*^{p−1}*k(a, b)k*^{p−1}*p*

*− 1*

¶

*b*

#