A Jacobi-type Newton method for Nash equilibrium problems with descent guarantees

A Jacobi-type Newton method for Nash equilibrium problems with descent guarantees

L. F. Bueno

G. Haeser

O. Kolossoski

September 22, 2022

Abstract

A common strategy for solving an unconstrained two-player Nash equilibrium problem with continuous variables is applying Newton’s method to the system of nonlinear equations obtained by the corresponding first-order necessary optimality conditions. However, when taking into account the game dynamics, it is not clear what is the goal of each player when considering that they are taking their current decision following Newton’s iterates. In this paper we provide an interpretation for Newton’s iterate in view of the game dynamics as follows: instead of minimizing the quadratic approximation of their objective function param- eterized by the other player current decision (as a typical Jacobi-type strategy), we show that the Newton iterate follows this approach but with the objective function parameterized by a prediction of the other player action, considering that they are following the same New- tonian strategy. This interpretation allows us to present a new Newtonian algorithm where a backtracking procedure is introduced in order to guarantee that the computed Newtonian directions, for each player, are descent directions for their corresponding parameterized func- tions. Thus, besides favoring global convergence, our algorithm also favors true minimizers instead of maximizers or saddle points, differently from the standard Newton method, which does not consider the minimization structure of the problem in the non-convex case. Thus, our method is more robust in comparison with other Jacobi-type strategies or the pure Newtonian approach, which is corroborated by our illustrative numerical experiments. We also present a proof of the well-definiteness of the algorithm under some standard assumptions, together with a thorough analysis of its convergence properties taking into account the game dynamics.

Keywords: Nash equilibrium, Newtonian method, Jacobi-type methods, Non-convex games, Game dynamics.

1 Introduction

The Nash Equilibrium Problem (NEP), introduced in [Nash (1951)], models an economic game where each player aims at minimizing their own objective function, but the decision taken by each player influences the payoff of the other players. This is a fundamental problem in economics and so- cial behavior theory that has been studied extensively. See, for instance, [Arrow and Debreu (1954)];

[Myerson (1991)]; [Aubin (1993)]. For surveys on the subject, see [Facchinei and Kanzow (2007)];

[Fischer et.al. (2014)]. In this paper we consider the unconstrained two-player NEP given by

∗Institute of Science and Technology, Federal University of S˜ao Paulo, S˜ao Jos´e dos Campos, SP, Brazil.

([email protected])

†Department of Applied Mathematics, University of S˜ao Paulo, S˜ao Paulo, SP, Brazil. ([email protected])

‡Department of Applied Mathematics, University of S˜ao Paulo, S˜ao Paulo, SP, Brazil.

([email protected])

Minimize f1(x1, x2),

x1∈R^{n1 (1)}

and

Minimize f2(x1, x2),

x2∈R^{n2 (2)}

wheref₁, f₂:Rⁿ¹×Rⁿ² →Rare (possibly non-convex) twice continuously differentiable functions that describe each player’s objective to be minimized, parameterized by the other player’s decision.

A (local) solution to the NEP (1-2) is a point (x^∗₁, x^∗₂)∈Rⁿ¹×Rⁿ²such thatx^∗₁is a (local) minimizer for (1) withx₂=x^∗₂, andx^∗₂is a (local) minimizer for (2) with x₁=x^∗₁.

In order to solve the NEP (1-2), some studies deal with a reformulation of the problem, such as a

variational inequality problem or as a complementarity problem, see for instance [Majig and Fukushima (2008)];

[Facchinei and Pang (2003)]. One problem with this approach is that several solutions of the re- formulated problem may not be solutions of the associated equilibrium problem. Another possi- bility is to reformulate the NEP as an optimization problem via the Nikaido-Isoda function (see [Nikaido and Isoda (1955)]), which transforms the NEP into a minimax problem. A penalty update scheme was also proposed in the works of [Fukushima and Pang (1998)]; [Facchinei and Kanzow (2009)].

Those indirect approaches are somewhat computationally demanding, as they require solving an optimization problem at each step of the algorithm. In order to avoid such heavy computations, the method proposed in this paper aims at solving the system (1-2) without rewriting it as an indirect problem.

One may also consider partitioning methods, namely Jacobi- and Gauss-Seidel-type splitting methods (see [Facchinei and Kanzow (2007)]; [Yuan (2011)]) that deal with problem (1-2) by it- erating the idea of fixing the variables of one player and taking a step towards minimizing the problem of the other player. In particular, a standard Jacobi strategy would fix preliminary de- cisions for each player, and then iteratively update them simultaneously by solving a model for the corresponding optimization problem parameterized by the fixed decision of the other player.

Alternatively, so-called best response methods fix the decision of each player as the best possible response, see [Caruso et.al. (2018)] and [Caruso et.al. (2020)]. Those methods also require solving an optimization problem at each step for finding the best response for the other player before solving the corresponding optimization problem for one player, which may also be computationally expensive. However, the advantage of theses approaches is that they are able to treat problem (1-2) in its original form, without losing information due to reformulations.

The game structure plays a key role when the optimization problems involved are non-convex.

Several important applications result in such problems, including NEPs (see [Chui and Pang (2021)]).

Moreover, the dynamics of the algorithms, namely, the way that the iterates are computed, may capture the true dynamics of the game, reflecting the way a competitive game actually unfolds in practice. That is, we suspect that economic systems behave in practice as a dynamical system, where each player considers the current decision of the other players and a model for predicting their future behavior, before optimizing their strategy. We expect that Jacobi-type methods should capture this behavior, predicting accurately the final outcome of a game, even if no equilibrium is reached. In fact, differently from what is usual in an algorithm for solving a single optimization problem, where we expect and hope to build a sequence converging to a solution of the problem, in a NEP viewed as such dynamical system, the dynamics may converge to an equilibrium state, but other interesting phenomena may be observed such as orbiting, cycling or divergence. The reader should have this observation in mind when considering what type of theoretical property should be expected by an algorithm aiming to capture such dynamics.

In this paper we introduce a method based on a Jacobi-type approach within a Newtonian framework. The main novelty of our approach is that instead of taking a decision based on the

current decision of the opponent, our Newton model is built around a prediction of the opponent step. This should reflect the reality of equilibrium games in practice, where each player tries to make the best decision based on what is the expected response of the other player. The algorithm proposed in this paper is posed in such a way that each player aims at decreasing their objective function with respect to the predicted decision of the other player. This avoids the heavy com- putations of best response methods, while still reflecting the dynamics of equilibrium games in practice.

Additionally, we note that our method considers the true minimization structure of the system (1-2) in a Newtonian framework, instead of considering only the system of equations given by the necessary optimality conditions associated with each problem. The idea is that one should favor minimizers in place of maximizers or saddle points when considering the solutions of this system of equations, which is usually not considered in other Newtonian algorithms for solving non-convex instances of (1-2).

In a related work, a Jacobi-type approach using quadratic approximations of players’ objective functions was considered in [Yuan (2011)]. The proposed algorithm consists in a trust region method which does not directly use the Hessian information on its iterative step; however, it relies on some strong assumptions on the true Hessians that we do not rely on in our work. Additionally, for each player, the method in [Yuan (2011)] is based on the current response of the other players, rather than on a predicted one.

The paper is organized as follows: In Section 2 we introduce our Jacobi-type algorithm in a Newtonian framework. In Section 3 we establish some notations and summarize the overall assumptions to be considered throughout the paper, together with some preliminary results. In Section 4 we establish the well-definiteness of our algorithm. In Section 5 we state some theoretical properties of the algorithm, establishing convergence to stationarity under certain conditions. In section 6 we illustrate the behavior and effectiveness of the proposed method with some numerical experiments. Finally, some concluding remarks and future prospective works are given in Section 7.

Notation: Given a twice continuously differentiable function f : Rⁿ¹⁺ⁿ² → R, (x1, x2)7→f(x1, x2), fori= 1,2 we denote by∇x_if(x) its partial gradient with respect to variable xievaluated atx. The partial Hessian with respect to variablesxiandxj evaluated atxis denoted by∇²_xixjf(x), fori, j= 1,2. We use∥ · ∥to denote the 2-norm of vectors and matrices, whileu^Tv denotes the canonical inner product of vectorsuand v of the same dimensions. We denote byN the set of positive integers.

2 A Descent Newton Algorithm for the Two-Player Nash Equilibrium Problem

Recall that a solution (x1, x2) to the equilibrium problem (1-2) satisfies the following first-order necessary optimality condition:

∇x₁f1(x1, x2)

∇x2f₂(x₁, x₂)

!

= 0

0

!

. (3)

Most algorithms for solving the NEP (1-2) rely on approaching directly the nonlinear system of equations (3), disregarding the minimization structure of problem (1-2). This poses no issues whenf1andf2are convex with respect to their decision variables, however, in the presence of non- convexities, the algorithm may find local maximizers or saddle points as often as local minimizers.

Our goal in this paper is to propose an algorithm that takes into account the minimization structure

of the problem in order to find local minimizers more often, thus better reflecting the behavior of two-player games in practice.

Given a current iterate (x^k₁, x^k₂), the classical Newtonian step (d^k₁, d^k₂) for solving (3) is given by solving the following linear system:

∇²_x1x1f1(x^k₁, x^k₂) ∇²_x1x2f1(x^k₁, x^k₂)

∇²_x

1x₂f₂(x^k₁, x^k₂) ∇²_x

2x₂f₂(x^k₁, x^k₂)

! d^k₁ d^k₂

=− ∇x₁f1(x^k₁, x^k₂)

∇_x2f₂(x^k₁, x^k₂)

!

. (4)

It is well known that, for optimization problems, applying Newton’s method to the correspond- ing first-order necessary optimality conditions can be better interpreted as minimizing a quadratic approximation of the objective function. With this interpretation, we can extend Newton’s method to non-convex problems, and we can also provide elements for globalization schemes. Two well- stablished strategies for doing this are the use of trust regions and line search techniques, ensuring progress near the current point. The first one guarantees the existence of a minimizer for the quadratic approximation of the objective function by establishing a compact domain for the sub- problem. In line search approaches, one should use some positive definite approximation for the Hessian of the objective function to obtain a coercive model and descent directions. In this case, one must carefully choose a convenient step-size along the Newton direction.

In order to exploit the minimization structure of the NEP (1-2), we shall consider an idea which resembles a standard line-search globalization described above. Given an approximate solution (x^k₁, x^k₂) at some iterationk, the most natural step to be made by the first player would be associated with minimizing, with respect to variable d₁, a simplified model for the functionf₁(x^k₁+d₁, x^k₂).

Namely, the first player would select its response by computing Minimize ϕ1(x^k₁+d1, x^k₂),

d1

whereϕ1 is an approximation of f1.

Similarly, the second player would seek to minimize at the same time a model for the function f2(x^k₁, x^k₂+d2) with respect to d2, leading to

Minimize ϕ₂(x^k₁, x^k₂+d₂).

d2

This approach is related to Jacobi’s method and its theoretical investigation has been conducted in [Yuan (2011)] under a trust-region globalization. The main concern that we want to address is to base the minimization model for each player on a prediction of the other player’s action, rather than on its actual decision. Thus, the main contribution of this work is considering the case when the direction d^k₁, the decision of the first player, is obtained by a Newton step with respect to a predicted behavior of the second player. Namely, given x^k₂ +d^k₂, a predicted decision for player two, the problem we aim to solve for player one is:

Minimize ϕ1(x^k₁+d1, x^k₂+d^k₂).

d1 (5)

Inspired by a Newtonian appeal, let us consider the second-order Taylor approximation off1, which leads to the following convex quadratic function:

f1(x^k₁+d1, x^k₂+d^k₂)≈f1(x^k₁, x^k₂+d^k₂) +∇x₁f1(x^k₁, x^k₂+d^k₂)^Td1+1

2d^T₁H₁^kd1, (6) where H₁^k is a positive definite approximation for the Hessian ∇²_x1x1f1(x^k₁, x^k₂ +d^k₂). Since d^k₂ needs to be found together withd^k₁, suppose that the same approach is done simultaneously by the

other player in order to make their decisiond^k₂. Then, given (x^k₁, x^k₂), we would still have a Nash equilibrium problem in the variables (d1, d2), however, the resulting NEP is simpler to solve, since the objective functions are parameterized convex quadratics in the corresponding decision variables.

But, since the term ∇x1f₁(x^k₁, x^k₂+d^k₂)^Td₁ might still combine the variables in a non-linear way, the approximated problem might still be hard to solve. Thinking in eliminating this inconvenient with a typical idea from Newton’s method, we use linear approximations of the gradients in the following way:

∇x₁f1(x^k₁, x^k₂+d^k₂)≈ ∇x₁f1(x^k₁, x^k₂) +∇²_x1x2f1(x^k₁, x^k₂)d^k₂. (7) In this way, our approximated problem (5) is now clearly defined by combining approximations (6) and (7), which has a simple structure and its solution is readily given by solving the linear system:

H₁^kd1+∇²_x1x2f1(x^k₁, x^k₂)d^k₂+∇x₁f1(x^k₁, x^k₂) = 0.

Repeating the approach for the second player, with a positive definite H₂^k ≈ ∇²_x2x2f2(x^k₁+ d^k₁, x^k₂), our iteration is based on the solution of the following linear system:

H₁^k ∇²_x

1x₂f₁(x^k₁, x^k₂)

∇²_x1x2f2(x^k₁, x^k₂) H₂^k

! d₁ d₂

=− ∇_x1f₁(x^k₁, x^k₂)

∇x₂f2(x^k₁, x^k₂)

!

. (8)

Given the equivalence with the standard Newtonian system (4), it is natural to chooseH₁^k ≈

∇²_x1x1f1(x^k₁, x^k₂) and H₂^k ≈ ∇²_x2x2f2(x^k₁, x^k₂). Thus, we have associated the standard Newtonian system (4) with the original NEP, where every player makes its decision considering the predicted decision of the other player. It now seems natural to consider descent conditions associated with the predicted functionsf1(x1, x^k₂+d^k₂) andf2(x^k₁+d^k₁, x2). However, we cannot guarantee that the resulting directions from (8) are descent directions for these functions, as is shown in the following example.

Example 2.1 Consider the equilibrium problem where f1(x1, x2) := ¹₂x²₁+ (x²₂+ 2x2+ 1)x1 and f2(x1, x2) := ¹₂(x2+ 2)². In this case, we have

• ∇x1f1(x1, x2) =x1+x²₂+ 2x2+ 1,∇²_x1x1f1(x1, x2) = 1,∇²_x1x2f1(x1, x2) = 2x2+ 2

• ∇x₂f2(x1, x2) =x2+ 2,∇²_x2x2f2(x2, x2) = 1 and∇²_x1x2f2(x2, x2) = 0.

Taking(x^k₁, x^k₂) := (0,0), the Newton direction (d^k₁, d^k₂)obtained from (8)is the solution of:

1 2 0 1

! d₁ d₂

=− 1

2

! .

Sod^k₁= 3 andd^k₂=−2. Note that∇x1f₁(x^k₁, x^k₂) =∇x1f₁(x^k₁, x^k₂+d^k₂) = 1 and therefored^k₁ is not a descent direction forf₁(x₁, x^k₂+d^k₂)nor for f₁(x₁, x^k₂) atx^k₁.

This means that we do not expect the descent conditionf1(x^k₁+td^k₁, x^k₂+d^k₂)< f1(x^k₁, x^k₂+d^k₂) to hold even for arbitrarily small t. Therefore, an algorithm that requires an Armijo-like line search could be not well-defined. The reason for not obtaining a descent direction when minimizing the convex quadratic approximation of the function comes from the fact that we did not use its true gradient. That is, the approximation of ∇x₁f1(x^k₁, x^k₂+d^k₂) in (7) might not be accurate enough when ∥d^k₂∥ is large. To improve this approximation, we should consider also taking small steps along the direction d^k₂. In this way, the line search should be done simultaneously forf1 and f2, considering a simultaneous backtracking in d^k₁ and d^k₂. Thus, replacingd^k₂ by td^k₂ in the previous

derivation of (8), we arrive at the following system of equations for computing (d₁, d₂), given a tentative step lengtht:

H₁^k t∇²_x1x2f₁(x^k₁, x^k₂) t∇²_x1x2f2(x^k₁, x^k₂) H₂^k

! d₁ d₂

=− ∇x1f₁(x^k₁, x^k₂)

∇x₂f2(x^k₁, x^k₂)

! .

Note that for t >0 sufficiently small, we expect the solution (d₁, d₂) to be such thatd₁ is a descent direction for problem (1) while d₂ is a descent direction for problem (2) considering the predicted decision for the other player. Hence, our method favors local minimizers for solving problem (1-2). Notice, however, that the solution (d₁, d₂) depends on the choice of the step length t, thus, once a step length is rejected by the descent condition, a new direction (d₁, d₂) must be recomputed, similarly to a trust region approach. Finally, once the descent condition is met for some t, the new iterate is defined as (x^k+1₁ , x^k+1₂ ) := (x^k₁, x^k₂) +t(d₁, d₂).

Our algorithm selects the stepsizes based on a standard Armijo-type condition, namely, checking whether the decrease along a component (with the other component fixed at the predicted point) is proportional to what is predicted by the first order approximation of the function. In mathematical terms, for someα∈(0,1), we perform the check

f₁(x^k₁+td₁, x^k₂+td₂)≤f₁(x^k₁, x^k₂+td₂) +αt∇_x1f₁(x^k₁, x^k₂+td₂)^Td₁, f2(x^k₁+td₁, x^k₂+td₂)≤f2(x^k₁+td₁, x^k₂) +αt∇x₂f2(x^k₁+td₁, x^k₂)^Td₂,

decreasing tand recomputing the directions whenever one of these inequalities do not hold.

Before formally presenting the algorithm, let us discuss some other conditions needed in the line search procedure. Differently from optimization problems, since the directions are not necessarily descent directions whentis large, we must ensure a negative slope over the search direction when computing the step-size t. In doing so, we are lead to also avoiding directions that are near orthogonal to the gradient of the predicted function. Additionally, in order to avoid stagnation when far from stationary points, we also demand that the direction is not too small compared with the corresponding gradient of the predicted function.

Mathematically, forθ∈(0,1) andγ >0, our line search checks also if

∇_x1f₁(x^k₁, x^k₂+td₂)^Td₁≤ −θ∥∇_x1f₁(x^k₁, x^k₂+td₂)∥ · ∥d₁∥,

∥d₁∥ ≥γ∥∇x₁f1(x^k₁, x^k₂+td₂)∥, (9) with a similar requirement for the second player. Notice that, in optimization, similar inequalities are standard requirements on the direction for guaranteeing global convergence of a descent method using an Armijo line search.

Guided by all these ideas, we formalize below the algorithm for computing a sequence{(x^k₁, x^k₂)}k∈N

for solving the NEP (1-2), where we note that the algorithm includes a safeguarding strategy that replaces the mixed hessians with zeroes whenever stationarity for the corresponding player has been reached andtis small enough, which will be explained later.

Algorithm 1Jacobi-type descent Newton algorithm

Step 0. Given (x⁰₁, x⁰₂)∈Rⁿ¹×Rⁿ², letα∈(0,1),τ ∈(0,1],θ >0, γ >0 and setk:= 0.

Step 1. Compute g^k₁ :=∇x₁f1(x^k₁, x^k₂) and g₂^k :=∇x₂f2(x^k₁, x^k₂), choose symmetric positive defi- nite matricesH₁^k andH₂^k, and sett:= 1.

Step 2. Set

M1:=

∇²_x1x2f1(x^k₁, x^k₂), if∥g₁^k∥>0 ort > τ,

0, otherwise;

M2:=

∇²_x1x2f2(x^k₁, x^k₂), if∥g₂^k∥>0 ort > τ,

0, otherwise.

(10)

Step 3. Check if the matrix

"

H₁^k tM1

tM₂ H₂^k

#

is non-singular. If not, sett:=t/2 and repeat Step 3.

Step 4. Find (d₁, d₂) by solving the linear system H₁^k tM1

tM2 H₂^k

! d₁ d₂

=− g₁^k g₂^k

!

. (11)

Step 5. Check the inequalities

f1(x^k₁+td₁, x^k₂+td₂)≤f1(x^k₁, x^k₂+td₂) +αt∇x₁f1(x^k₁, x^k₂+td₂)^Td₁, (12)

∇x₁f1(x^k₁, x^k₂+td₂)^Td₁≤ −θ∥∇x₁f1(x^k₁, x^k₂+td₂)∥ · ∥d₁∥, (13) γ∥∇x₁f1(x^k₁, x^k₂+td₂)∥ · ∥g^k₁∥ ≤ ∥d₁∥ · ∥g^k₁∥; (14) f2(x^k₁+td₁, x^k₂+td₂)≤f2(x^k₁+td₁, x^k₂) +αt∇x₂f2(x^k₁+td₁, x^k₂)^Td₂, (15)

∇x₂f2(x^k₁+td₁, x^k₂)^Td₂≤ −θ∥∇x₂f2(x^k₁+td₁, x^k₂)∥ · ∥d₂∥, (16) γ∥∇x2f2(x^k₁+td₁, x^k₂)∥ · ∥g^k₂∥ ≤ ∥d₂∥ · ∥g^k₂∥. (17) If one of the inequalities (12)-(17) do not hold, sett:=t/2 and go to Step 2.

Step 6. Set t^k := t, d^k₁ := d₁, d^k₂ := d₂ and update x^k+1₁ :=x^k₁+t^kd^k₁, x^k+1₂ := x^k₂+t^kd^k₂, and k:=k+ 1. Go to Step 1.

Notice that for finding the direction (d₁, d₂), we use the linear system (11), which fori= 1,2 replaces the mixed Hessian matrixt∇²_x1x2fi(x^k₁, x^k₂), used in the previous discussion when deducing (8), with zeroes wheneverg^k_i = 0 andt≤τ. This is done in order to forced_ito be zero if stationarity is already reached for playeriand the step size is sufficiently small. As a consequence, a standard Newton step is set for the other player. The reason for doing this is that in these situations it may not be possible to obtain a function decrease for player i. We also replaced the aforementioned condition (9), which controls the ratio between gradient and direction, by (14). Both conditions are the same when g₁^k ̸= 0, but using (14) ensures that when stationarity is reached for the first player, inequality (14) is automatically satisfied. A similar situation occurs for the second player with respect to condition (17).

However, takingd^k_i = 0 may not be the most adequate strategy, even ifg_i^k= 0, since stationarity for player i may be lost after a step is made for the other player. Thus, this is done only as a safeguarding procedure after the stepsize is at least as small as a threshold valueτ ∈(0,1]. Notice that when τ := 1, the safeguarding procedure is always activated whenever g_i^k = 0, while when τ <1, it is activated only after rejecting the Newtonian step at least once. This is done in order to try using the Newtonian direction as much as possible due to its fast local properties.

3 General Assumptions

The assumptions stated in this section are supposed to hold throughout the paper without specific mentioning. Before stating them, we establish some notation that shall be used in the sequel. We denote

g^k :=

g₁^k g₂^k

=

∇_x1f₁(x^k₁, x^k₂)

∇_x2f₂(x^k₁, x^k₂)

, H_t,k :=

H₁^k tM₁ tM₂ H₂^k

,

d^k :=

d^k₁ d^k₂

, d:=

d₁ d₂

, and x^k :=

x^k₁ x^k₂

.

Additionally, for each player i = 1,2, we denote the index of the other player by ¬i. The general assumptions that we use in all our analysis are the following:

Assumption A There exists C_G > 0 such that max{∥∇xif_i(x)∥,∥∇x¬if_i(x)∥} ≤ C_G for all x∈Rⁿ¹⁺ⁿ² andi∈ {1,2}.

Assumption B There existsL >0 such that ∇_xif_i(·, x_¬i) areL-Lipschitz continuous for every fixedx_¬i andi∈ {1,2}. As a consequence, we have that

f_i(x_i+td_i, x_¬i)≤f_i(x) +t∇xif_i(x)^Td_i+t²L 2 ∥di∥² for every x= (xi, x_¬i)∈R^ni+n¬i, di∈Rⁿⁱ, andt >0.

Assumption C There exists CH > 0 such that ∥∇²_x1x2fi(x)∥ ≤ CH for all x ∈ Rⁿ¹⁺ⁿ² and i∈ {1,2}.

Assumption D There existsCR>0 and functionsr1 andr2such that fori∈ {1,2}and for all x= (xi, x¬i)∈R^ni+n¬i, alld¬i∈R^n¬i andt >0 it holds

∇xif_i(x_i, x_¬i+td_¬i) =∇xif_i(x) +t∇²_x1x2f_i(x)d_¬i+r_i(t, x, d_¬i), (18) and

∥r_i(t, x, d_¬i)∥ ≤C_Rt²∥d_¬i∥².

Assumption E There existλmax≥λmin>0 such that for alli∈ {1,2}andk∈Nthe eigenvalues ofH_i^k lie in [λmin, λmax]. As a consequence, for allxi∈Rⁿⁱ, we have that

∥H_i^k∥ ≤λmax, ∥(H_i^k)⁻¹∥ ≤ 1 λmin

, and λmin∥xi∥²≤x^T_iH_i^kxi≤λmax∥xi∥², i∈ {1,2}.

We highlight that all Assumptions A to D are simple assumptions on the smoothness and bound- edness of the derivatives of the functionsf1andf2, while Assumption E can be easily guaranteed when choosing the matrices H_i^k.

We end this section with a few results that arise from Assumptions A-E. The first result guarantees uniform boundedness of the matricesHt,k andH⁻¹_t,k for small enough values oft.

Lemma 3.1 For everyk∈Nandt∈(0,1], the matrix H_t,k satisfies

∥Ht,k∥ ≤µmax:=

q

λ²_max+ 4λmaxCH+C_H². (19) Moreover, if t≤ _8λ^λ2min

maxCH, thenHt,k is non-singular and∥H_t,k⁻¹∥ ≤µmin, where the constantµmin

is given by

µmin :=

√2 λmin

. (20)

Proof:

By the definition ofHt,k, for a vector z= (z1, z2)∈Rⁿ¹×Rⁿ², we have that Ht,kz=

H₁^k tM1

tM2 H₂^k

· z1

z2

=

H₁^kz1+tM1z2

tM2z1+H₂^kz2

.

Therefore, we can write

∥Ht,kz∥²= (H₁^kz1+tM1z2)^T(H₁^kz1+tM1z2) + (tM2z1+H₂^kz2)^T(tM2z1+H₂^kz2)

=∥H₁^kz₁∥²+ 2tz₁^TH₁^kM₁z₂+∥tM1z₂∥²+∥H₂^kz₂∥²+ 2tz₂^TH₂^kM₂z₁+∥tM2z₁∥². (21) Now, by the definition ofMi on (10), we see that∥Mi∥ ≤ ∥∇²_xix¬ifi(x^k)∥, i= 1,2. Hence, by the Cauchy-Schwarz inequality, we have that

∥Ht,kz∥²≤ ∥H₁^kz₁∥²+ 2t∥H₁^k∥ · ∥∇²_x1x2f₁(x^k)∥ · ∥z1∥ · ∥z2∥+t²∥∇²_x1x2f₁(x^k)∥²· ∥z2∥² +∥H₂^kz2∥²+ 2t∥H₂^k∥ · ∥∇²_x

2x₁f2(x^k)∥ · ∥z1∥ · ∥z2∥+t²∥∇²_x

2x₁f2(x^k)∥²· ∥z1∥². Using Assumptions C and E, and the fact thatt≤1, we have that

∥Ht,kz∥²≤λ²_max∥z1∥²+ 2tλmaxCH· ∥z1∥ · ∥z2∥+t²C_H² · ∥z2∥² +λ²_max∥z2∥²+ 2tλmaxCH· ∥z1∥ · ∥z2∥+t²C_H²∥z1∥²

≤λ²_max ∥z1∥²+∥z2∥²

+ 4λmaxCH· ∥z1∥ · ∥z2∥+C_H² ∥z1∥²+∥z2∥²

= λ²_max+C_H²

∥z1∥²+∥z2∥²

+ 4λ_maxC_H∥z1∥ · ∥z2∥.

Hence, observing that∥zi∥ ≤ ∥z∥ fori= 1,2, we have that

∥Ht,kz∥²≤ λ²_max+ 4λmaxCH+C_H²

∥z∥²,

and thus (19) holds. To prove the bound on ∥Ht,k∥, recalling (21), we have that

∥H_t,kz∥²=∥H₁^kz₁∥²+ 2tz^T₁H₁^kM₁z₂+∥tM₁z₂∥²+∥H₂^kz₂∥²+ 2tz₂^TH₂^kM₂z₁+∥tM₂z₁∥²

≥λ²_min∥z1∥²−2tλ_max∥M1∥ · ∥z1∥ · ∥z2∥+∥tM1z₂∥²+λ²_min∥z2∥²

−2tλmax∥M2∥ · ∥z1∥ · ∥z2∥+∥tM2z1∥².

Since∥tMiz_¬i∥²≥0,∥Mi∥ ≤C_H,∥z∥²=∥z1∥²+∥z2∥², and∥zi∥ ≤ ∥z∥, we have

∥Ht,kz∥²≥ λ²_min−4tλmaxCH

∥z²∥.

Thus, fort≤ _8λ^λ2min

maxC_H, we have∥Ht,kz∥²≥ ^λ2min₂ ∥z∥², soHt,k is non-singular and∥H⁻¹_t,k∥ ≤

√2 λ_min. 2

The next result guarantees boundedness of the computed directionsdfor sufficiently smallt.

Lemma 3.2 For everyk ∈N, if t ≤ _8λ^λ2min

maxCH then∥d∥ ≤ 2CGµmin =:Cd, where µmin is given by (20).

Proof: By the system equation (11) and using Lemma 3.1, we have that

∥d∥=∥H⁻¹_t,kg^k∥ ≤ ∥H⁻¹_t,k∥ ·(∥g^k₁∥+∥g₂^k∥)≤2CGµmin.

2 The next result shows that for each component i= 1,2, the gradientsg_i^k are comparable withd_i provided thattis bounded by a fraction of∥g^k_i∥.

Lemma 3.3 For everyk∈N,i= 1,2, ift satisfies t≤min

λ²_min

8λ_maxC_H, ∥g_i^k∥ 2C_HC_d

, (22)

whereC_d is given as in Lemma 3.2, then 1

2∥g_i^k∥ ≤ ∥H_i^kd_i∥ ≤ 3

2∥g_i^k∥. (23)

Proof: We shall prove the result for i = 1, being the result analogous for i = 2. By the first equation of system (11), the definition of M₁, and Assumption C, together with Lemma 3.2, we have that

∥H₁^kd₁∥=∥ −tM1d₂−g₁^k∥ ≤tCHCd+∥g₁^k∥.

Thus, from (22), we conclude the right-hand side inequality of (23). Similarly, to prove the left-hand side inequality in (23), considering again the first equation of system (11), we have

∥g^k₁∥=∥ −H₁^kd1−tM1d2∥ ≤ ∥H₁^kd1∥+tCHCd.Thus, the result follows from (22). 2

4 Well Definiteness of the Algorithm

This Section is devoted to proving well-definiteness of Algorithm 1. As previously mentioned, by the way Algorithm 1 is built, in cases where one of the gradients is zero, say ∥g₂^k∥ = 0, and the step-sizet has sufficiently decreased (t≤τ), the direction for the first player becomes a standard Newton direction for optimization with Armijo line search, which is known to be well-defined and satisfy conditions (12)-(14) with t ≤ ^2(1−α)λ_Lλ ^2min

max , θ = _λ^λmin

max and γ = _λ¹

max (see for instance the discussion in [Bertsekas (2016)], pages 29-36, and Lemma 2.20 in [Izmailov and Solodov (2014)]).

Thus, we focus on the case where ∥g_i^k∥ ̸= 0, i= 1,2, and we show that the inequalities (12)-(17) are satisfied for sufficiently smallt.

In order to prove that Algorithm 1 is well-defined, we are going to prove that each pair of inequalities in (12)-(17) hold for t sufficiently small, meaning that the stepsize does not decrease indefinitely. We begin with the gradient/direction ratio inequalities (14) and (17). However, we show a stronger version bounding the directions by the gradient on the points (x^k_i, x^k_¬i+td_¬i) from both above and below.

Lemma 4.1 For everyk∈Nandi= 1,2, if t satisfies t≤min

∥g_i^k∥ 8CHCd

, λ²_min 8λmaxCH

, (24)

whereC_d is given as in Lemma 3.2, then it holds γ

∇x_ifi(x^k_i, x^k_¬i+td_¬i)

≤ ∥d_i∥ ≤β

∇x_ifi(x^k_i, x^k_¬i+td_¬i)

, (25) for every γ andβ such that

γ≤ 2 3λmax

, β ≥ 2 λmin

. (26)

Proof: We prove the result only fori= 1, as the proof fori= 2 is similar. By (11), we have that

−g₁^k=H₁^kd₁+tM1d₂. (27)

Also, we have that

∇x₁f1(x^k₁, x^k₂+td₂) =g^k₁+t Z 1

0

∇²_x1x2f1(x^k₁, x^k₂+ξtd₂)dξ

d₂. (28)

Combining (27) and (28) we obtain

∇_x1f₁(x^k₁, x^k₂+td₂) =−H₁^kd₁+t

−M₁+ Z 1

0

∇²_x

1x₂f₁(x^k₁, x^k₂+ξtd₂)dξ

d₂. (29) By the triangular inequality, we have that

∥∇x₁f1(x^k₁, x^k₂+td₂)∥ ≥

∥H₁^kd₁∥ −t

−M1+ Z 1

0

∇²_x1x2f1(x^k₁, x^k₂+ξtd₂)dξ

d₂

. (30) and

∥∇x₁f1(x^k₁, x^k₂+td₂)∥ ≤ ∥H₁^kd₁∥+t

−M1+ Z 1

0

∇²_x1x2f1(x^k₁, x^k₂+ξtd₂)dξ

d₂

. (31) Next, we prove that for tsufficiently small it holds

1

2∥H₁^kd₁∥ ≤ ∥∇_x1f₁(x^k₁, x^k₂+td₂)∥ ≤ 3

2∥H₁^kd₁∥. (32)

Let us suppose that

−M1+ Z 1

0

∇²_x1x2f1(x^k₁, x^k₂+ξtd₂)dξ

d₂̸= 0, since otherwise this is trivial from (29). Hence, if we selecttsatisfying

t≤ ∥H₁^kd₁∥

2

−M1+ Z 1

0

∇²_x1x2f1(x^k₁, x^k₂+ξtd₂)dξ

d₂

, (33)

we see from (30) that the left-hand side inequality in (32) holds and, by (31), we obtain the right-hand side inequality. Now, using thatλ_min∥d₁∥ ≤ ∥H₁^kd₁∥ ≤λ_max∥d₁∥, (32) implies that

λ_min

2 ∥d₁∥ ≤ ∥∇_x1f₁(x^k₁, x^k₂+td₂)∥ ≤ 3λ_max 2 ∥d₁∥, which implies the desired bounds in (25).

Finally, let us show thatt satisfying (24) implies the bound (33). By Assumption C and by Lemma 3.2, we see that

−M1+ Z 1

0

∇²_x1x2f1(x^k₁, x^k₂+ξtd₂)dξ

d₂

≤

∥M1∥+

Z 1

0

∇²_x1x2f₁(x^k₁, x^k₂+ξtd₂)dξ

· ∥d₂∥

≤

∥M1∥+ Z 1

0

∥∇²_x1x2f1(x^k₁, x^k₂+ξtd₂)∥dξ

· ∥d₂∥ ≤2CHCd, therefore, we have that

∥H₁^kd₁∥

4C_HC_d ≤ ∥H₁^kd₁∥

2

−M1+ Z 1

0

∇²_x1x2f1(x^k₁, x^k₂+ξtd₂)dξ

d₂ ,