It is shown, for instance, that the Clarke generalized Jacobian is an approximate Jacobian for a locally Lipschitz map

APPROXIMATE JACOBIAN MATRICES FOR NONSMOOTH CONTINUOUS MAPS AND C¹-OPTIMIZATION^∗

V. JEYAKUMAR^† AND D. T. LUC^‡

Abstract. The notion of approximate Jacobian matrices is introduced for a continuous vector- valued map. It is shown, for instance, that the Clarke generalized Jacobian is an approximate Jacobian for a locally Lipschitz map. The approach is based on the idea of convexificators of real- valued functions. Mean value conditions for continuous vector-valued maps and Taylor’s expansions for continuously Gˆateaux differentiable functions (i.e.,C¹-functions) are presented in terms of ap- proximate Jacobians and approximate Hessians, respectively. Second-order necessary and sufficient conditions for optimality and convexity ofC¹-functions are also given.

Key words. generalized Jacobians, nonsmooth analysis, mean value conditions, optimality conditions

AMS subject classifications. 49A52, 90C30, 26A24 PII.S0363012996311745

1. Introduction. Over the past two decades, a great deal of research has focused on the study of first- and second-order analysis of real-valued nonsmooth functions [2, 3, 4, 5, 11, 12, 14, 15, 21, 23, 24, 20, 25, 27, 28, 29, 30, 34, 35]. The results of nonsmooth analysis of real-valued functions now provide basic tools of modern analysis in many branches of mathematics, such as mathematical programming, control, and mechanics. Indeed, the range of applications of nonsmooth calculus demonstrates its basic nature of nonsmooth phenomena in the mathematical and engineering sciences.

On the other hand, research in the area of nonsmooth analysis of vector-valued maps has been of substantial interest in recent years [2, 6, 7, 8, 9, 10, 18, 21, 22, 23, 24, 29, 31]. In particular, it is known that the development and analysis of generalized Jacobian matrices for nonsmooth vector-valued maps are crucial from the viewpoint of control problems and numerical methods of optimization. For instance, the Clarke generalized Jacobian matrices [2] of a locally Lipschitz map play an important role in the Newton-based numerical methods for solving nonsmooth equations and optimiza- tion problems (see [26] and other references therein, and see also [17, 18, 19] for other applications). Warga [32, 33] examined derivative (unbounded derivative) containers in the context of local and global inverse function theorems as set-valued derivatives for locally Lipschitz (continuous) vector-valued maps. Mordukhovich [21, 22] devel- oped generalized differential calculus for general nonsmooth vector-valued maps using the set-valued derivatives, called coderivatives [9, 21].

Our aim in this paper is to introduce a new concept of approximate Jacobian matrices for continuous vector-valued maps that are not necessarily locally Lipschitz, develop certain calculus rules for approximate Jacobians, and apply the concept to optimization problems involving continuously Gˆateaux differentiable functions. This

∗Received by the editors November 8, 1996; accepted for publication (in revised form) October 2, 1997; published electronically July 9, 1998. This research was partially supported by a grant from the Australian Research Council.

http://www.siam.org/journals/sicon/36-5/31174.html

†Department of Applied Mathematics, University of New South Wales, Sydney 2052, Australia (jeya@maths.unsw.edu.au). Some of the work of this author was carried out while visiting the Centre for Experimental and Constructive Mathematics at the Simon Fraser University, Canada.

‡Institute for Mathematics, Hanoi, Vietnam (dtluc@thevinh.ac.vn). Some of the work of this author was done while visiting the University of New South Wales.

1815

concept is a generalization of the idea of convexificators of real-valued functions, studied recently in [4, 5, 13], to vector-valued maps. Convexificators provide two- sided convex approximations [30] for real-valued functions. Unlike the set-valued generalized derivatives [9, 21, 22, 32, 33], mentioned above for vector-valued maps, the approximate Jacobian is defined as a closed subset of the space of (n×m) matrices for a vector-valued map fromRⁿ intoR^m.

Approximate Jacobians not only extend the nonsmooth analysis of locally Lip- schitz maps to continuous maps but also unify and strengthen various results of nonsmooth analysis. They also enjoy useful calculus, such as the generalized mean value property and chain rules. Moreover, approximate Jacobians allow us to present second-order optimality conditions in easily verifiable forms in terms of approximate Hessian matrices for C¹-optimization problems, extending the corresponding results forC^1,1-problems [7].

The outline of the paper is as follows. In section 2, approximate Jacobian ma- trices are introduced, and it is shown that for a locally Lipschitz map the Clarke generalized Jacobian is an approximate Jacobian. Various examples of approximate Jacobians are also given. Section 3 establishes mean value conditions for continuous vector-valued maps and provides necessary and sufficient conditions in terms of ap- proximate Jacobians for a continuous map to be locally Lipschitz. Various calculus rules for approximate Jacobians are given in section 4. Approximate Hessian matri- ces are introduced in section 5, and their connections toC^1,1-functions are discussed.

Section 6 presents generalizations of Taylor’s expansions forC¹-functions. In section 7, second-order necessary and sufficient conditions for optimality and convexity of C¹-functions are given.

2. Approximate Jacobians for continuous maps. This section contains no- tation, definitions, and preliminaries that will be used throughout the paper. Let F :Rⁿ→R^mbe a continuous function which has components (f1, . . . , fm). For each v∈R^m, the composite function, (vF) :Rⁿ →R, is defined by

(vF)(x) =hv, F(x)i= Xm i=1

vifi(x).

The lower Dini directional derivative and the upper Dini directional derivative ofvF atxin the directionu∈Rⁿ are defined by

(vF)⁻(x, u) := lim inf

t↓0

(vF)(x+tu)−(vF)(x)

t ,

(vF)⁺(x, u) := lim sup

t↓0

(vF)(x+tu)−(vF)(x)

t .

We denote byL(Rⁿ,R^m) the space of all (n×m) matrices. Theconvex hull and the closed convex hull of a set Ain a topological vector space are denoted by co(A) and co(A), respectively.

Definition 2.1. The map F : Rⁿ → R^m admits an approximate Jacobian

∂^∗F(x)atx∈Rⁿ if∂^∗F(x)⊆L(Rⁿ,R^m)is closed, and for each v∈R^m,

(2.1) (vF)⁻(x, u)≤ sup

M∈∂^∗F(x)hMv, ui ∀u∈Rⁿ.

A matrixM of∂^∗F(x) is called an approximate Jacobian matrix ofF atx. Note that condition (2.1) is equivalent to the condition

(2.2) (vF)⁺(x, u)≥ inf

M∈∂^∗F(x)hMv, ui ∀u∈Rⁿ.

It is worth noting that the inequality (2.1) means that the set ∂^∗F(x)v is an upper convexificator [13, 16] of the function vF atx. Similarly, the inequality (2.2) states that ∂^∗F(x)v is a lower convexificator of vF at x. In the casem= 1, the inequality (2.1) (or (2.2)) is equivalent to the condition

(2.3) F⁻(x, u)≤ sup

x^∗∈∂^∗F(x)hx^∗, ui and F⁺(x, u)≥ inf

x^∗∈∂^∗F(x)hx^∗, ui;

thus, the set∂^∗F(x) is a convexificator of F atx. Also note that in the casem= 1, condition (2.3) is also equivalent to the condition that for eachα∈R,

(2.4) (αF)⁻(x, u)≤ sup

x^∗∈∂^∗F(x)hαx^∗, ui ∀u∈Rⁿ.

Similarly, the condition (2.3) is also equivalent to the condition that for eachα∈R,

(2.5) (αF)⁺(x, u)≥ inf

x^∗∈∂^∗F(x)hαx^∗, ui ∀u∈Rⁿ.

For applications of convexificators, see [5, 13, 16]. To clarify the definition, let us consider some examples.

Example2.2. IfF :Rⁿ→R^mis continuously differentiable atx, then any closed subset Φ(x) ofL(Rⁿ,R^m) containing the Jacobian∇F(x) is an approximate Jacobian ofF atx. In this case, for eachv∈R^m,

(vF)⁻(x, u) =h∇F(x)v, ui ≤ sup

M∈Φ(x)hMv, ui ∀u∈Rⁿ.

Observe from the definition of the approximate Jacobian that for any mapF :Rⁿ → R^m, the whole spaceL(Rⁿ,R^m) serves as a trivial approximate Jacobian forF at any point inRⁿ. Let us now examine approximate Jacobians for locally Lipschitz maps.

Example 2.3. Suppose that F : Rⁿ → R^m is locally Lipschitz at x. Then the Clarke generalized Jacobian ∂_CF(x) is an approximate Jacobian of F at x. Indeed, for eachv∈R^m,

(2.6) ∂^◦(vF)(x) =∂_CF(x)v.

Consequently, for eachu∈Rⁿ,

(vF)^◦(x, u) = max

ξ∈∂^◦(vF)(x)hξ, ui= max

M∈∂CF(x)hMv, ui, where

∂_CF(x) =co{ lim

n→∞∇F(x_n)^T :x_n ∈Ω, x_n→x},

Ω is the set of points in Rⁿ where F is differentiable, and the Clarke directional derivative ofvF is given by

(vF)^◦(x, u) = lim sup

x0→xt↓0

hv, F(x⁰+tu)−F(x⁰)i

t .

Since for eachu∈Rⁿ,

(vF)⁻(x, u)≤(vF)^◦(x, u) ∀u∈Rⁿ, the set∂_CF(x) is an approximate Jacobian ofF atx.

For the locally Lipschitz mapF :Rⁿ→R^m, the set

∂BF(x) :={ lim

n→∞∇F(xn)^T :xn∈Ω, xn →x}

is also an approximate Jacobian of F at x. The set ∂_BF(x) is known as the B- subdifferential of F at x, which plays a significant role in the development of nons- mooth Newton methods (see [26]). In passing, note that for eachv∈R^m,

∂^◦(vF)(x) =co(∂M(vF)(x)) =co(D^∗F(x)(v)),

where the set-valued mappingD^∗F(x) fromR^m into Rⁿ is the coderivative ofF at x and ∂_M(vF)(x) is the first-order subdifferential of vF at x in the sense of Mor- dukhovich [22]. However, for locally Lipschitz maps, the coderivative does not appear to have a representation of the form (2.6), which allowed us above to compare ap- proximate Jacobians with the Clarke generalized Jacobian. The reader is referred to [9, 21, 22, 29] for a more general definition and associated properties of coderivatives.

A second-order analogue of the coderivative for vector-valued maps is given recently in [10].

Let us look at a numerical example of a locally Lipschitz map where the Clarke generalized Jacobian strictly contains an approximate Jacobian.

Example 2.4. Consider the functionF :R²→R² F(x, y) = (|x|,|y|).

Then

∂^∗F(0) =

1 0 0 1

,

1 0 0 −1

,

−1 0

0 1

,

−1 0

0 −1

is an approximate Jacobian of F at 0. On the other hand, the Clarke generalized Jacobian

∂CF(0) =

α 0 0 β

: α, β∈[−1, 1]

, which is also an approximate Jacobian ofF at 0 and contains∂^∗F(0).

Observe in this example that∂_CF(0) is the convex hull of∂^∗F(0). However, this is not always the case. The following example illustrates that even for the case where m= 1, the convex hull of an approximate Jacobian of a locally Lipschitz map may be strictly contained in the Clarke generalized Jacobian.

Example 2.5. Define F:R²→Rby

F(x, y) =|x| − |y|.

Then it can easily be verified that

∂₁^∗F(0) ={(1,1),(−1,−1)} and ∂₂^∗F(0) ={(1,−1),(−1,1)}

are approximate Jacobians ofF at 0, whereas

∂_BF(0) ={(1,1),(−1,1),(1,−1),(−1,−1)}

and

∂CF(0) =co({(1,1),(−1,1),(1,−1),(−1,−1)}).

It is also worth noting that

co(∂₁^∗F(0))⊂co(∂MF(0)) =∂CF(0).

Clearly, this example shows that certain results, such as mean value conditions and necessary optimality conditions that are expressed in terms of∂^∗F(x), may provide sharp conditions even for locally Lipschitz maps (see section 3).

Let us now present an example of a continuous map where the Clarke generalized Jacobian does not exist, whereas approximate Jacobians are quite easy to calculate.

Example 2.6. Define F:R²→R² by F(x, y) = (p

|x|sgn(x) +|y|, p

|y|sgn(y) +|y|),

where sgn(x) = 1 for x >0, 0 forx= 0, and −1 for x <0. ThenF is not locally Lipschitz at (0,0), and so the Clarke generalized Jacobian does not exist. However, for eachc∈R, the set

∂^∗F(0,0) =

α 1 0 β

,

α −1

0 β

: α, β≥c

is an approximate Jacobian ofF at (0,0).

3. Generalized mean value theorems. In this section we derive mean value theorems for continuous maps in terms of approximate Jacobians and show how locally Lipschitz vector-valued maps can be characterized using approximate Jacobians.

Theorem 3.1. Let a, b∈Rⁿ and F :Rⁿ→R^m be continuous. Assume that for each x∈[a, b],∂^∗F(x)is an approximate Jacobian of F atx. Then

F(b)−F(a)∈co(∂^∗F([a, b])(b−a)).

Proof. Let us first note that the right-hand side above is the closed convex hull of all points of the formM(b−a), whereM ∈∂^∗F(ζ) for someζ∈[a, b]. Letv∈R^mbe arbitrary and fixed. Consider the real-valued functiong: [0,1]→IR

g(t) =hv, F(a+t(b−a))−F(a) +t(F(a)−F(b))i.

Thengis continuous on [0,1] withg(0) =g(1). Sogattains a minimum or a maximum at some t₀ ∈ (0,1). Suppose that t₀ is a minimum point. Then, for each α ∈ R, g⁻(t₀, α)≥0. It now follows from direct calculations that

g⁻(t₀, α) = (vF)⁻(a+t₀(b−a), α(b−a)) +αhv, F(a)−F(b)i.

Hence, for eachα∈R,

(vF)⁻(a+t₀(b−a), α(b−a))≥αhv, F(b)−F(a)i.

Now, by takingα= 1 andα=−1, we obtain that

−(vF)⁻(a+t0(b−a), a−b)≤ hv, F(b)−F(a)i ≤(vF)⁻(a+t0(b−a), b−a)i.

By (2.1), we get

M∈∂^∗F(a+tinf0(b−a))hMv, b−ai ≤ hv, F(b)−F(a)i ≤ sup

M∈∂^∗F(a+t0(b−a))hMv, b−ai.

Consequently,

hv, F(b)−F(a)i ∈co(∂^∗F(a+t0(b−a))v)(b−a), and so

(3.1) hv, F(b)−F(a)i ∈co(∂^∗F([a, b])v)(b−a).

Since this inclusion holds for eachv∈R^m, we claim that F(b)−F(a)∈co(∂^∗F([a, b])(b−a)).

If this is not so, then it follows from the separation theorem hp, F(b)−F(a)i − > sup

u∈co(∂^∗F([a,b])(b−a))hp, ui

for some p∈ R^m since co(∂^∗F([a, b])(b−a)) is a closed convex subset of R^m. This implies

hp, F(b)−F(a)i>sup{α:α∈co(∂^∗F([a, b])p)(b−a)}, which contradicts (3.1).

Similarly, ift₀is a maximum point, theng⁺(t₀, α)≤0 for eachα∈R. Using the same line of arguments as above, we arrive at the same conclusion, and so the proof is complete.

Corollary 3.2. Let a, b∈Rⁿ and F :Rⁿ →R^m be continuous. Assume that

∂^∗F(x)is a bounded approximate Jacobian ofF atxfor eachx∈[a, b]. Then (3.2) F(b)−F(a)∈co(∂^∗F([a, b])(b−a)).

Proof. Since for eachx∈[a, b],∂^∗F(x) is compact, the set co(∂^∗F([a, b])(b−a) =co{∂^∗F([a, b])(b−a)}

is closed, and so the conclusion follows from Theorem 3.1.

In the following corollary we deduce the mean value theorem for locally Lipschitz maps (see [1, 6]) as a special case of Theorem 3.1.

Corollary 3.3. Let a, b ∈ Rⁿ and F : Rⁿ → R^m be locally Lipschitz on Rⁿ. Then

(3.3) F(b)−F(a)∈co(∂CF([a, b])(b−a)).

Proof. In this case the Clarke generalized Jacobian∂_CF(x) is a convex and com- pact approximate Jacobian ofF atx. Hence, the conclusion follows from Corollary 3.2.Note that even for the case whereF is locally Lipschitz, Corollary 3.2 provides a stronger mean value condition than condition (3.3) of Corollary 3.3. To see this, let n= 2, m= 1,F(x, y) =|x| − |y|,a= (−1,−1), andb= (1,1). Then condition (3.2) of Corollary 3.2 is verified by

∂^∗F(0) ={(1,−1),(−1,1)}.

However, condition (3.3) holds for∂_CF(0), where

∂CF(0) =co({(1,1),(−1,−1),(1,−1),(−1,1)})⊃∂^∗F(0).

As a special case of the above theorem, we see that ifF is real-valued, then an asymptotic mean value equality is obtained. This was shown in [13].

Corollary 3.4. Let a, b ∈ X and F : Rⁿ → R be continuous. Assume that, for each x∈[a, b],∂^∗F(x)is a convexificator of F. Then there existc∈(a, b) and a sequence {x^∗_k} ⊂co(∂^∗F(c))such that

F(b)−F(a) = lim

k→∞hx^∗_k, b−ai.

Proof. The conclusion follows from the proof of Theorem 3.1 by noting that a convexificator∂^∗F(x) is an approximate Jacobian ofF at x.

We now see how locally Lipschitz functions can be characterized using the above mean value theorem. We say that a set-valued mappingG:Rⁿ→L(Rⁿ,R^m) islocally boundedatxif there exist a neighborhoodU ofxand a positiveαsuch that||A|| ≤α for each A ∈ G(U). Recall that the map G is said to be upper semicontinuous at xif for each open set V containingG(x) there is a neighborhood U of xsuch that G(U)⊂V. Clearly, ifGis upper semicontinuous atxand ifG(x) isbounded, thenG is locally bounded atx.

Theorem 3.5. Let F :Rⁿ →R^m be continuous. ThenF has a locally bounded approximate Jacobian map ∂^∗F atxif and only if F is locally Lipschitz atx.

Proof. Assume that ∂^∗F(y) is the approximate Jacobian of F for each y in a neighborhoodU ofxand that∂^∗Fis locally bounded onU. Without loss of generality, we may assume that U is convex. Then there exists α > 0 such that ||A|| ≤α for eachA∈∂^∗F(U). Letx, y∈U. Then [x, y]⊂U, and by the mean value theorem,

F(x)−F(y)∈co(∂^∗F([x, y])(x−y))⊂co(∂^∗F(U)(x−y)).

Hence,

kF(x)−F(y)k ≤ kx−ykmax{kAk:A∈∂^∗F(U)}.

This gives us that

kF(x)−F(y)k ≤αkx−yk, and soF is locally Lipschitz atx.

Conversely, if F is locally Lipschitz at x, then the Clarke generalized Jacobian can be chosen as an approximate Jacobian forF, which is locally bounded atx.

4. Calculus rules for approximate Jacobians. In this section, we present some basic calculus rules for approximate Jacobians. We begin by introducing the notion of regular approximate Jacobians which are useful in some applications.

Definition 4.1. The mapF :Rⁿ→R^madmits a regular approximate Jacobian,

∂^∗F(x)atx∈Rⁿ if∂^∗F(x)⊆L(Rⁿ,R^m)is closed, and for each v∈R^m,

(4.1) (vF)⁺(x, u) = sup

M∈∂^∗F(x)hMv, ui ∀u∈Rⁿ, or equivalently,

(4.2) (vF)⁻(x, u) = inf

M∈∂^∗F(x)hMv, ui ∀u∈Rⁿ.

Note that in the casem= 1, this definition collapses to the notion of the regular convexificator studied in [13]. Thus, a closed set∂^∗h(x)⊂Rⁿ is a regular convexifi- cator of the real-valued functionhatxif for eachu∈Rⁿ,

h⁻(x, u) = inf

ξ∈∂^∗h(x)hξ, ui and h⁺(x, u) = sup

ξ∈∂^∗h(x)hξ, ui.

It is evident that these equalities follow from (4.1) by takingF =handv=−1 and v= 1, respectively.

It is immediate from the definition that ifF is differentiable atx, then{∇f(x)}

is a regular approximate Jacobian ofF atx. However, ifF is locally Lipschitz atx, then the Clarke generalized Jacobian∂CF(x) is not necessarily a regular approximate Jacobian ofF atx. It is also worth noting that if∂^∗₁F(x) and∂₂^∗F(x) are two regular approximate Jacobians ofF at x, thenco(∂₁^∗F(x)) =co(∂^∗₂F(x)).

In passing, we note that ifF is locally Lipschitz on a neighborhoodU ofx, then there exists a dense set K⊂U such thatF admits a regular approximate Jacobian at each point ofK. By Rademacher’s theorem, the dense subset can be chosen as the set whereF is differentiable.

Theorem 4.2 (Rule 1). Let F and H be continuous maps from Rⁿ to R^m. Assume that ∂^∗F(x)is an approximate Jacobian of F atx and∂^∗H(x)is a regular approximate Jacobian of H at x. Then the set ∂^∗F(x) +∂^∗H(x) is an approximate Jacobian of F+H atx.

Proof. Letv∈R^m, u∈Rⁿ be arbitrary. By definition, hv, F+Hi⁻(x, u) = lim inf

t↓0

hv, F(x+tu)−F(x) +H(x+tu)−H(x)i

t .

Let{tn}be a sequence of positive numbers converging to 0 such that hv, F+Hi⁻(x, u) = lim

n→∞

hv, F(x+tnu)−F(x) +H(x+tnu)−H(x)i

t_n .

Further, let{sn}be another sequence of positive numbers converging to 0 such that hv, Fi⁻(x, u) = lim inf

t↓0

hv, F(x+tu)−F(x)i

t = lim

n→∞

hv, F(x+snu)−F(x)i

s_n .

Then we have

n→∞lim

hv, F(x+snu)−F(x)i

s_n ≤ sup

M∈∂^∗F(x)hMv, ui

and

lim sup

n→∞

hv, H(x+snu)−H(x)i

sn ≤ hv, Hi⁺(x, u) = sup

M∈∂^∗H(x)hMv, ui.

Consequently,

hv, F+Hi⁻(x, u)≤ lim

n→∞

hv, F(x+s_nu)−F(x)i

sn +hv, H(x+s_nu)−H(x)i sn

≤ sup

M∈∂^∗F(x)hMv, ui+ sup

N∈∂^∗H(x)hNv, ui

= sup

P∈∂^∗F(x)+∂^∗H(x)hP v, ui.

Since u and v are arbitrary, we conclude that ∂^∗F(x) +∂^∗H(x) is an approximate Jacobian ofF+H atx.

Note that as in the case of convexificators of real-valued functions [18], the set

∂^∗F(x) +∂^∗H(x) is not necessarily regular atx.

Theorem 4.3 (Rule 2). Let F : Rⁿ → R^m and H : IR^m → R^l be continuous maps. Assume that∂^∗F(x)is a bounded approximate Jacobian ofF atxand∂^∗H(x) is a bounded approximate Jacobian ofH atF(x). If the maps∂^∗F and∂^∗H are upper semicontinuous atxandF(x), respectively, then∂^∗H(F(x))∂^∗F(x)is an approximate Jacobian of H◦F atx.

Proof. Letw∈R^l andu∈R^mbe arbitrary. Consider the lower Dini directional derivative ofhw, H◦Fiat x:

hw, H◦Fi⁻(x, u) = lim inf

t↓0

hw, H(F(x+tu))−H(F(x))i

t .

By applying the mean value theorem (see Theorem 3.1) toH andF, we obtain F(x+tu)−F(x)∈tco(∂^∗F([x, x+tu])u),

H(F(x+tu))−H(F(x))∈co(∂^∗H([F(x,), F(x+tu)])(F(x+tu)−F(x))) It now follows from the upper semicontinuity of ∂^∗F and ∂^∗H that for an arbitrary small positivewe can find t₀>0 such that fort∈(0, t₀) we have

∂^∗F([x, x+tu])⊆∂^∗F(x) +B1,

∂^∗H([F(x), F(x+tu)])⊆∂^∗H(F(x)) +B₂,

whereB1 andB2 are the unit balls inL(Rⁿ,R^m) andL(R^m,R^l), respectively. Using these inclusions, we obtain

hw, H(F(x+tu))−H(F(x))i

t ∈ hw, Ai,

where

A:=co((∂^∗H(F(x))∂^∗F(x) +(∂^∗H(F(x))B₁+B₂∂^∗F(x)) +²B₂B₁)u).

Since∂^∗H(F(x)) and∂^∗F(x) are bounded, we can findα >0 such thatkMk ≤αfor allM ∈∂^∗H(F(x)) orM ∈∂^∗F(x).Consequently,

hw, H◦Fi⁻(x, u)≤ sup

M∈∂^∗H(F(x))∂^∗F(x)hMw, ui+ 2kuk+²ku|.

Asis arbitrary, we conclude that∂^∗H(F(x))∂^∗F(x) is an approximate Jacobian of H◦F at x.

5. Approximate Hessian matrices. In this section, unless stated otherwise, we assume that f : Rⁿ → R is a C¹- function, that is, a continuously Gˆateaux differentiable function, and introduce the notion of approximate Hessian for such functions. Note that the derivative off, which is denoted by ∇f, is a map from Rⁿ toRⁿ.

Definition 5.1. The function f admits anapproximate Hessian ∂_∗²f(x)atxif this set is an approximate Jacobian to∇f atx.

Note that ∂_∗²f(x) = ∂^∗∇f(x) and the matrix M ∈ ∂²_∗f(x) is an approximate Hessian matrix ofF at x. Clearly, iff is twice differentiable atx, then∇²f(x) is a symmetric approximate Hessian matrix off atx.

Let us now examine the relationships between the approximate Hessians and the generalized Hessians, studied for C^1,1-functions, that is, Gˆateaux differentiable functions with locally Lipschitz derivatives. Recall that if f :Rⁿ → Ris C^1,1, then the generalized Hessian in the sense of Hiriart-Urruty, Strodiot, and Hien Nguyen [7]

is given by

∂_H²f(x) =co{M : M = lim

n→∞∇²f(xn), xn∈∆, xn→x},

where ∆ is the set of points inRⁿ wheref is twice differentiable. Clearly,∂_H²f(x) is a nonempty convex compact set of symmetric matrices. The second-order directional derivative off at xin the directions (u, v)∈Rⁿ×Rⁿ is defined by

f^◦◦(x;u, v) = lim sup

y→xs→0

h∇f(y+su), vi − h∇f(y), vi

s .

Since (v∇f)⁻(x, u)≤f^◦◦(x;u, v), for each (u, v)∈Rⁿ and f^◦◦(x;u, v) = max

M∈∂_H²f(x)hMu, vi= max

M∈∂_H²f(x)hMv, ui,

∂_H²f(x) is an approximate Hessian off atx.

The generalized Hessian off atxas a set-valued map,∂^◦◦f(x) :Rⁿ→Rⁿ,which was given in Cominetti and Correa [3], is defined by

∂^◦◦f(x)(u) ={x^∗∈Rⁿ:f^◦◦(x;u, v)≥ hx^∗, vi∀v∈Rⁿ}.

It is known that the mapping (u, v) −→f^◦◦(x;u, v) is finite and sublinear and that ∂^◦◦f(x)(u) is a nonempty, convex, and compact subset of Rⁿ, and for each x, u, v∈Rⁿ,

f^◦◦(x;u, v) = max{hx^∗, vi:x^∗∈∂^◦◦f(x)(u)}.

Moreover, for eachu∈Rⁿ,

∂^◦◦f(x)(u) =∂²_Hf(x)u.

Iff is twice continuously differentiable atx, then the generalized Hessian∂^◦◦f(x)(u) is a singleton for everyu∈IRⁿ.

In [34, 35], another generalized second-order directional derivative and a general- ized Hessian set-valued map for aC^1,1 functionf at xwere given as follows:

f(x;u, v) = sup

z∈Rⁿlim sup

s↓0

h∇f(x+sz+su), vi − h∇f(x+sz), vi

s ,

∂f(x)(u) ={x^∗∈X^∗:f(x;u, v)≥ hx^∗, vi ∀v∈X}.

It was shown that the mapping (u, v) −→ f(x;u, v) is finite and sublinear;

∂f(x)(u) is a nonempty, convex, and compact subset of Rⁿ; and ∂f(x)(u) is singled-valued for each u∈IRⁿ if and only iff is twice Gˆateaux differentiable at x.

Further, for eachu∈Rⁿ,∂f(x)(u)⊂∂^◦◦f(x)(u) =∂_H²f(x)u. If for each (u, v)∈Rⁿ the functiony−→f(y;u, v) is upper semicontinuous atx, then

∂f(x)(u) =∂²_Hf(x)u.

The following proposition gives us necessary and sufficient conditions in terms of approximate Hessians for aC¹-function to beC^1,1.

Proposition 5.2. Let f : Rⁿ → R be a C¹-function. Then f has a locally bounded approximate Hessian map ∂²_∗f atxif and only if f isC^1,1 atx.

Proof. This follows from Theorem 3.5 by takingF as∇f.

We complete this section with an example showing that for a C^1,1 function the approximate Hessian may be a singleton which is contained in the generalized Hessian of Hiriart-Urruty, Strodiot, and Hien Nguyen [7].

Example 5.3. Let g be an odd, linear piecewise continuous function on R as follows. g(x) =xforx≥1 andg(0) = 0;g(x) = 2x−1 forx∈[¹₂,1];g(x) =−¹₂x+¹₄ forx∈[¹₆,¹₂];g(x) = 2x−¹₆ forx∈[₁₂¹,¹₆];g(x) =−¹₄x+₄₈¹ forx∈[₆₀¹,₁₂¹], etc. Let

G(x) = Z _|x|

0 g(t)dt, x∈R.

Define

f(x, y) =G(x) +y² 2 .

Then the functionf is a C^1,1 function, and the generalized Hessian off at (0,0) is

∂_H²f(0) =

α 0 0 1

: α∈[0, 2]

.

However, the approximate Hessian off at (0,0) is the singleton

∂_∗²f(0) =

0 0 0 1

.

6. Generalized Taylor’s expansions for C¹-functions. In this section, we see how Taylor’s expansions can be obtained for C¹- functions using approximate Hessians.

Theorem 6.1. Letf :Rⁿ →Rbe continuously Gˆateaux differentiable onRⁿ; let x, y∈Rⁿ. Suppose that for eachz∈[x, y],∂_∗²f(z)is an approximate Hessian off at z. Then there existsζ∈(x, y)such that

f(y)∈f(x) +h∇f(x), y−xi+1

2coh∂_∗²f(ζ)(y−x),(y−x)i.

Proof. Let h(t) = f(y +t(x−y)) +th∇f(y+t(x−y)), y−xi+¹₂at²−f(y), where a=−2(f(x)−f(y) +h∇f(x), y−xi). Then h(0) = 0, h(1) =f(x)−f(y) +

h∇f(x), y−xi+ ¹₂a = 0, and h is continuous. So h attains its extremum at some γ ∈(0,1). Suppose thatγ is a minimum point ofh. Now, by necessary conditions, we have for allv∈R

h⁻(γ;v)≥0.

Then 0≤h⁻(γ;v)

= lim inf

λ→0⁺

h(γ+λv)−h(γ) λ

= lim

λ→0⁺

f(y+ (γ+λv)(x−y))−f(y+γ(x−y)) λ

+1 2 lim

λ→0⁺

a(γ+λv)²−aγ² λ + lim inf

λ→0⁺

(γ+λv)h∇f(y+ (γ+λv)(x−y)), y−xi −γh∇f(y+γ(x−y)), y−xi λ

=vh∇f(y+γ(x−y)), x−yi+aγv+vh∇f(y+γ(x−y)), y−xi +γlim inf

λ→0⁺

h∇f(y+ (γ+λv)(x−y)), y−xi − h∇f(y+γ(x−y)), y−xi λ

=aγv+γlim inf

λ→0⁺

h∇f(y+ (γ+λv)(x−y)), y−xi − h∇f(y+γ(x−y)), y−xi

λ .

Letζ=y+γ(x−y). Thenζ∈(x, y), and forv= 1 we get 0≤aγ+γlim inf

λ→0⁺

h∇f(y+γ(x−y) +λ(x−y)), y−xi − h∇f(y+γ(x−y)), y−xi λ

≤a+ sup

M∈∂_∗²f(ζ)hM(y−x), x−yi.

This gives us that

a≥ inf

M∈∂_∗²f(ζ)hM(y−x), y−xi.

Similarly, forv=−1, we obtain 0≤ −aγ+γlim inf

λ→0⁺

h∇f(y+γ(x−y) +λ(y−x)), y−xi − h∇f(y+γ(x−y)), y−xi λ

≤ −a+ sup

M∈∂_∗²f(ζ)hM(y−x), y−xi;

thus,

a≤ sup

M∈∂_∗²f(ζ)hM(y−x), y−xi.

Hence, it follows that

M∈∂inf²_∗f(ζ)hM(y−x), y−xi ≤a≤ sup

M∈∂²∗f(ζ)hM(y−x), y−xi, and so

a∈coh∂_∗²f(ζ)(y−x),(y−x)i;

thus,

(6.1) f(y)−f(x)− h∇f(x), y−xi=a 2 ∈ 1

2coh∂²_∗f(ζ)(y−x),(y−x)i.

The case whereγ is a maximum point ofhalso yields the same condition (6.1). The details are left to the reader.

Corollary 6.2. Let f : Rⁿ →R be continuously Gˆateaux differentiable on Rⁿ and x, y ∈ Rⁿ. Suppose that for each z ∈ [x, y], ∂_∗²f(z) is a convex and compact approximate Hessian of f atz. Then there exist ζ ∈ (x, y) and M_ζ ∈ ∂_∗²f(ζ) such that

f(y) =f(x) +h∇f(x), y−xi+1

2hM_ζ(y−x), y−xi.

Proof. It follows from the hypothesis that for each z ∈ [x, y], ∂_∗²f(z) is convex and compact, and so thecoin the conclusion of the previous theorem is superfluous.

Thus, the inequalities

M∈∂inf²∗f(ζ)hM(y−x), y−xi ≤a≤ sup

M∈∂²_∗f(ζ)hM(y−x), y−xi give us that

a∈ h∂_∗²f(ζ)(y−x),(y−x)i.

Corollary 6.3 (see [7]). Let f :Rⁿ → R be C^1,1 and x, y∈Rⁿ. Then there exist ζ∈(x, y)andM_ζ ∈∂_H²f(ζ)such that

f(y) =f(x) +h∇f(x), y−xi+1

2hMζ(y−x), y−xi.

Proof. In this case, the conclusion follows from the above corollary by choosing the generalized Hessian∂_H²f(x) as an approximate Hessian off for eachx.

7. Second-order conditions for optimality and convexity ofC¹-functions.

In this section, we present second-order necessary and sufficient conditions for opti- mality and convexity ofC¹-functions using approximate Hessian matrices. Consider the optimization problem

(P) minimizef(x) subject tox∈Rⁿ,

where f : Rⁿ −→ R is a continuously Gˆateaux differentiable function on Rⁿ. We say that a map F :Rⁿ →R^madmits asemiregular approximate Jacobian ∂^∗F(x) at x∈Rⁿ if∂^∗F(x)⊆L(Rⁿ,R^m) is closed, and for eachv∈R^m,

(vF)⁺(x, u)≤ sup

M∈∂^∗F(x)hMv, ui ∀u∈Rⁿ.

Similarly, the C¹-function f : Rⁿ → R admits a semiregular approximate Hessian

∂_∗²f(x) atxif this set is a semiregular approximate Jacobian to∇f at x.

Of course, every semiregular approximate Hessian to f at x is an approximate Hessian atx. For aC^1,1 functionf :Rⁿ →R, the generalized Hessian,∂_H²f(x),of f atxis a bounded semiregular approximate Hessian off atxsince

(v∇f)⁺(x, u)≤f^◦◦(x;u, v) = max

M∈∂_H²f(x)hMu, vi= max

M∈∂_H²f(x)hMv, ui.

Theorem 7.1. For the problem (P), let x¯ ∈ Rⁿ. Assume that ∂_∗²f(¯x) is a semiregular approximate Hessian off atx.¯

(i) If x¯ is a local minimum of(P), then∇f(¯x) = 0, and for each u∈IRⁿ,

M∈∂sup∗²f(¯x)hMu, ui ≥0.

(ii) If x¯ is a local maximum of(P), then∇f(¯x) = 0, and for eachu∈Rⁿ,

M∈∂inf_∗²f(¯x)hMu, ui ≤0.

Proof. Let u∈Rⁿ. Since ¯x is a local minimum of (P), there existsδ > 0 such that for eachs∈[0, δ],

f(¯x+su)≥f(¯x).

Then, by the mean value theorem, for eachs∈(0, δ], there exists 0< t < ssuch that h∇f(¯x+tu), ui ≥0.

So, there exists a positive sequence{t_n} ↓0 such thath∇f(¯x+t_nu), ui ≥0. Now, as

∇f(¯x) = 0, it follows that

(u∇f)⁺(¯x;u) = lim sup

s↓0

h∇f(¯x+su), ui − h∇f(¯x), ui s

≥0.

Since∂_∗²f(x) is a semiregular approximate Hessian off atx, we have (u∇f)⁺(¯x;u)≤ sup

M∈∂_∗²f(¯x)hMu, ui, and hence,

M∈∂sup_∗²f(¯x)hMu, ui ≥0.

On the other hand, if f attains a local maximum at ¯x, then it follows by the similar arguments as above that for eachu∈Rⁿ,

M∈∂inf_∗²f(¯x)hMu, ui ≤0.

Note in this case that it is convenient to use the inequality (u∇f)⁻(¯x, u)≥ inf

M∈∂²∗f(¯x)hMu, ui.

Let us look at a numerical example to illustrate the significance of the optimality conditions obtained in the previous theorem.

Example 7.2. Define f :R²→Rby f(x, y) = 2

3|x|³²+1 2y². Thenf isC¹but is notC^1,1since the gradient

∇f(x, y) =p

|x|sgn(x), y

is not locally Lipschitz at (0,0). Evidently, (0,0) is a minimum point off,∇f(0,0) = (0,0), and

∂_∗²f(0) =

α 0 0 1

: α≥0

is a semiregular approximate Hessian off at (0,0). And for each u= (u1, u2)∈R²,

M∈∂sup_∗²f(0)hMu, ui= sup{αu²₁+u²₂ : α≥0} ≥0.

Hence, the statement (i) of Theorem 7.1 is verified. However, the generalized Hessians [7] do not apply to this function.

Corollary 7.3. For the problem (P), let x¯ ∈ Rⁿ. Suppose that ∂_∗²f(¯x) is a bounded semiregular approximate Hessian of f atx.¯

(i) If x¯ is a local minimum of (P), then ∇f(¯x) = 0, and for each u∈Rⁿ there exists a matrixM ∈∂_∗²f(¯x)such that hMu, ui ≥0.

(ii) If x¯ is a local maximum of (P), then ∇f(¯x) = 0, and for each u∈Rⁿ there exists a matrixM ∈∂_∗²f(¯x)such that hMu, ui ≤0.

Proof. Since ∂²_∗f(¯x) is closed and bounded, it follows from Theorem 7.1 that

∇f(¯x) = 0, and for eachu∈IRⁿ,

M∈∂max_∗²f(¯x)hMu, ui ≥0,

and so the first conclusion holds. The second conclusion similarly follows from Theo- rem 7.1.

We now see how optimality conditions for the problem (P) wheref isC^1,1follows from Corollary 7.3 (cf. [7]).

Corollary 7.4. For the problem (P), assume that the function f is C^1,1 and

¯ x∈Rⁿ.

(i) If x¯ is a local minimum of (P), then ∇f(¯x) = 0, and for each u∈Rⁿ there exists a matrixM ∈∂_H²f(¯x)such that hMu, ui ≥0.

(ii) If x¯ is a local maximum of (P), then ∇f(¯x) = 0, and for each u∈Rⁿ there exists a matrixM ∈∂_H²f(¯x)such that hMu, ui ≤0.

Proof. The conclusion follows from Corollary 7.3 by choosing ∂_H²f(¯x) as the semiregular bounded approximate Hessian∂_∗²f(¯x) off at ¯x.

Clearly, the conditions of Theorem 7.1 are not sufficient for a local minimum, even for aC²-functionf. The generalized Taylor’s expansion is now applied to obtain a version of second-order sufficient condition for a local minimum. For related results, see [34, 16].