A NEW CLASS OF INTERIOR PROXIMAL METHODS FOR OPTIMIZATION OVER THE POSITIVE OCTANT

(1)

A NEW CLASS OF INTERIOR PROXIMAL METHODS FOR

OPTIMIZATION OVER THE POSITIVE OCTANT

Sissy da Silva Souza

Universidade Federal do Rio de Janeiro

Cidade Universitária, Centro de Tecnologia, Bloco H, Rio de Janeiro, RJ sissy@cos.ufrj.br

Paulo Roberto Oliveira

Universidade Federal do Rio de Janeiro

Cidade Universitária, Centro de Tecnologia, Bloco H, Rio de Janeiro, RJ poliveir@cos.ufrj.br

RESUMO

Neste trabalho apresentamos uma família de métodos interior-proximais com métrica variável para resolver problemas de otimização sob restrições positividade. O algoritmo está definido na forma exata e o parâmetro de regularização é escolhido convenientemente de forma a obter pontos interiores. Mostramos a boa-definição do algoritmo e estabelecemos convergência fraca ao conjunto solução do problema.

PALAVRAS CHAVE. Algoritmos de Pontos Interiores. Métodos Proximais. Problemas de Otimização. Programação Matemática.

ABSTRACT

In this work we present a family of variable metric interior proximal methods for solving optimization problems under positivity constraint. The algorithm is defined in exact form and the regularization parameter is conveniently chosen to guarantee interior points. We show the well definedness of the algorithm and we establish weak convergence to the solution set of the problem presents.

KEYWORDS. Interior point algorithms. Proximal methods. Optimization problem. Mathematical Programming.

(2)

1. Introduction

We study the optimization problem defined by (OP)

min

f

(

x

)

s.t. x≥0.

We consider the following assumptions:

(A1)

f

:

R

n

→

R

n

∪

{

+∞

}

is a coercive continuous proper convex function; (A2)

S

∗

(OP

)

, the solution set of (OP), is nonempty.

A well know method for solving problems such as (OP) is the proximal algorithm. This method seems to have been applied the first time to convex problems by Martinet (1970) and the first important results in the more general case of maximal monotone operators are due to Rockafellar (1976). Many papers have generalized the algorithm presented in Rockafellar (1976) (see for example, Burachik and Iusem (1998), Ferreira and Oliveira (2002), Lemaire (1989)) and some of them lead to interior point proximal methods (see for instance Auslender at al (1999), Kaplan and Tichatschke (2004), Oliveira and Oliveira (2002)).

For solving the problem (OP) we propose a family of interior proximal methods whose kernels are metrics generated by diagonal matrices, constructed upon the r-power of the iterates, with

r

≥

2

, and the regularization parameter is conveniently chosen at each iteration to guarantee the positivity of the iterates. We extend the technical report presented by Oliveira and Oliveira (2002), where the autors study the problem (OP) by considering the Assumptions (H1), (H2) and that the objective function is continuously differentiable with lipschtzian gradient, they assume

r

≥

1

and the regularization parameters depend of the Lipschtz constant of the gradient of

f

. We show the same results obtained by Oliveira and Oliveira (2002) under weakker

hypotheses than those assumed by them.

The structure of the work is simple. In Section 2 we define the Algorithm, we present the results obtained and two numerical examples. The conclusion is given in Section 3, and the references are in section 4.

Throughout this work, we used the following notation: •

R

₊n

=

{(

x

₁

,...,

x

_n

)

∈

R

n

:

x

_i

≥

0 ,

∀

i

=

1 ,...,

n

}

;

•

R

₊n₊

=

{(

x

₁

,...,

x

_n

)

∈

R

n

:

x

_i

>

0 ,

∀

i

=

1 ,...,

n

}

;

• .,. and

||

.

||

denotes the Euclidean inner product and the Euclidean norm, respectively;

•

(

1

,...,

)

r n r r

x

=

denotes the vector which components are the r-power of

x

∈

R

n;

• r

X

denotes the diagonal matrix of order n, which elements of the diagonal are the components of

x

r.

•

max(

x

r

)

and

min(

x

r

)

denote, respectively, the largest and smallest component of

x

r, or equivalently, the largest and smallest eigenvalue of the matrix

X

r.

2. Method VMIP

Take a real number

r

≥

2

and a sequence of positive real numbers

{

α

k

}

such that

N

k

a

≤

k

≤

∀

∈

<

1

0 α

, for some 0< a <1. Algorithm VMIP Initialization: Choose

(

x

0

,

g

0

)

∈

R

₊n₊

×

R

n,

g

0

∈

∂

f

(

x

0

)

(3)

Iterative Step: Given

(

x

k

,

g

k

)

∈

R

n

×

R

n,

g

k

∈

∂

f

(

x

k

)

, compute k r k k k

g

x

α

β

=

−1

+

))

(max(

||

(1) and find

(

x

k+1

,

g

k+1

)

∈

R

n

×

R

n,

g

k+1

∈

∂

f

(

x

k+1

)

satisfying

.

0 )

(

)

(

1 1

+

− +

−

=

+ k r k k k k

x

X

g

β

(2) If

g

k

=

0

or

x

k

=

x

k+1 stop, else Take k = k+1.

2.1 Well Definedness and Properties

By using the Assumption (A1) on f, we can show that

Proposition 1. The sequence

{

x

k+1

}

generated by the Algorithm VMIP is well defined.

Remark 1. Observe that to solve the equation (2) is equivalent to find

x

k+1 such that

x

1

arg

min

{

f

k

(

x

)}

R x k n ∈ +

₌

, where _Xk r k k k

x

f

x

f

=

+

||

−

||

₍ ₎−

2 )

(

)

(

β

.

Using the last remark , we can prove the two next results.

Proposition 2. Let

{

x

k+1

}

be the sequence generated by the Algorithm VMIP. Then the sequence

)}

(

{

f

x

k+1 is decreasing and convergent.

Lemma 1. Let

{

x

k+1

}

and

{

β

_k

}

be the sequences generated by the Algorithm VMIP. Then

+∞

<

−

− + +∞ =

∑

_Xk r k k k K

x

( ) 1 0

||

β

.

From Lemma 1, the definition of

{

β

k

}

and the lower boundedness of the sequence

}

{

α

k we obtain the following result.

Corollary 1. Let

{

x

k+1

}

and

{

β

k

}

be the sequences generated by the Algorithm VMIP. Then

+∞

<

−

− + +∞ =

∑

_Xk r k k k

x

1 ₍ ₎ 0

||

. 2.2 Convergence Analysis

We use the following definition about weak convergence of a sequence

{

x

k

}

⊂

R

n to a set

U

⊂

R

n.

Definition 1. (Iusem (1995), Definition 1) A sequence

{

x

k

}

⊂

R

nis said to be weakly convergent to a

U

⊂

R

n if: (i) The sequence

{

x

k

}

is bounded; (ii) lim( +1− )=0

+∞ →

k k

k x x ; (iii) All cluster points of

{

x

k

}

belong to U.

We prove that the sequence

{

x

k+1

}

generated by the Algorithm VMIP converges

weakly to the solution set

S

∗

(OP

)

.

(4)

Proposition 3. The sequence

{(

x

k+1

,

g

k+1

)}

generated by the Algorithm VMIP is bounded.

This last proposition implies that

Corollary 2. The sequence

{

β

_k

}

generated by the Algorithm VMIP is bounded,

and

{

x

k+1

}

be the sequence generated by the Algorithm VMIP. Then 0 ) ( lim +1− = +∞ → k k k x x .

From the characterization of

{

k+1

}

x

presented in Remark 1 and some known properties, such as monotonicity of

∂

f

and Cauchy-Schwarz inequality, we can prove that all iterates

x

k+1

obtained by the Algorithm VMIP are interior points. Moreover, by observing that

0 ,...,

1

0 ∀

=

⇔

⋅

=

⋅

g

i

n

X

g

x

r i

i , and by using previou results we obtain that all limit point

)

,

(

x

g

of

{(

x

k+1

,

g

k+1

)}

satisfies the complementarity condition. Similarly to the proofs presented in Iusem (1995) and Oliveira and Oliveira (2002), we also show that

(

x

,

g

)

∈

R

₊n

×

R

₊n.

Thus, we conclude that all limit points of

{(

x

k+1

,

g

k+1

)}

satisfy the KKT conditions of the

problem (OP), that is,

Proposition 5. All limit points of the sequence

{

x

k+1

}

generated by the Algorithm VMIP belong to

S

∗

(OP

)

.

From Propositions 3, 4 and 5 results

Teorema 1. The sequence

{

x

k+1

}

generated by the Algorithm VMIP converges weakly to the solution set

S

∗

(OP

)

.

We also determine a convergence estimative:

{(

x

k+1

,

g

k+1

)}

and

{

β

_k

}

be the sequences generated by the Algorithm VMIP. Then for any

x

∗

∈

S

∗

(OP

)

the following convergence estimate holds:

}. , { max ) ( ) ( 1 1 ,..., 0 + − = ∗ _≤ ₋ − k k n k n g x x x f x f 2.3 Examples

We study three examples that illustrat the application of the method VMIP. We implemented the Algorithm VMIP in MatlabR12.

Example 1. We consider the problem

min

x

T

Qx

+

c

T

x

+

d

s

.

t

.

x

≥

0

,where

⎟⎟

⎟

⎠

⎞

⎜⎜

⎜

⎝

⎛

−

=

1

5 .

0

5 .

0

5 .

0

5 .

0

1

5 .

0

5 .

0

5 .

0

5 .

0

1

5 .

0

5 .

0

5 .

0

5 .

0

1 Q

,

c

T

=

(

2

0

1

1 )

and d=0.

We consider in the implementation of the Algorithm VMIP,

r

≥

2

,

)

1 (

2

1

5 .

0 +

+

=

k

α

, and

x

0

=

(

5

8

6

3 )

T. The exact solution is

T

x

0

=

(

5

8

6

3 )

_{The exact}

solution is

x

∗

=

(

0

0 )

T. The algorithm stop in k =301, when the distance between successive iterates is less than the zero of MATLAB, and

(

0 .

002

0 .

03

0 .

004

0 .

004 )

T is the

(5)

point obtained.

Example 2. We consider the problem

min

2 2

.

0

1

+

e

s

t

x

≥

x

x . We consider in the implementation of the Algorithm VMIP, r =2.5,

)

1 (

2

1

5 .

0 +

+

=

k

α

, and

x

0

=

(

5

3 )

T. The exact solution is

(

0

0 )

T. The algorithm stop in k =82, when the distance between successive iterates is less than the zero of MATLAB, and

(

0 .

1

0 .

04 )

T is the point obtained.

Example 3. We consider the problem

min

(

x

₁

−

100 )

2

+

(

x

₂

−

5 )

2

s

.

t

.

x

≥

0

. We consider in the

implementation of the Algorithm VMIP, r =2.2,

)

1 (

2

1

5 .

0 +

+

=

k

α

,and

x

0

=

(

15

6 .

5 )

T. The exact solution is

(

100

5 )

TThe algorithm stop in k =20, when the distance between

successive iterates is less than the zero of MATLAB, and

(

100

5 .

02 )

T is the point obtained.

3. Conclusion

In this work we have presented a class of proximal point method with variable metric for solving optimization problem on the positive octant. We showed that the algorithm in the exact form is well defined and it converges weakly to the solution set. We present some examples that illustrate the performance of the Algorithm VMIP. An advantage of this method is due to the fact that the regularization parameters are given explicitly and they guarantee the strict viability of all iterates. We also study the inexact form of the algorithm and we guaranteed the well-definedness and the property of weak convergence to the solution set, but this class of inexact methods is not discoursed here.

Referências

Auslender, A and Teboulle, M and Ben-Tiba, S. (1999), A Logarithmic-Quadratic Proximal

Method for Variational Inequalities, Computational Optimization and Applications, 12, 31-40.

Burachik, R.S. and Iusem, A.N. (1998), A generalized proximal point algorithm for the

variational inequality problem in a Hilbert space, SIAM Journal on Optimization, 8, 197-216.

Ferreira, O.P. and Oliveira, P.R. (2002), Proximal Point Algorithm on Riemannian Manifolds, Optimization, 51, 257-270.

Iusem, A.N. (1995), An Interior Multiplicative Method for Optimization Under Positivity

Constraints, Acta Applicadae Mathematicae, 38, 163-184.

Kaplan, A. and Tichatschke, R. (2004), Interior proximal method for variational inequalities:

case of nonparamonotone operators, Set-Valued Analysis, 12, 357-382.

Lemaire, B., The proximal point, International Series of Numerical Mathematics, J.P. Penot, Ed.

Birkhauser, Basel, 87, 73-87, 1989.

Martinet, B. (1970) Règularisation d'inéquations variationelles par approximations succesives, Revue Française d'Informatique et Recherche Opérationnelle, 2, 154-159

Oliveira, G.L. and Oliveira, P.R., A New Class of Proximal Interior-Point Methods for

Optimization under Positivity Constraints, ES-570, PESC/COPPE-UFRJ, 2002

(http://www.optimization-online.org/DB_HTML/2002/02/441.html)

Rockafellar, R.T. (1976), Monotone Operators and the proximal point algorithm, SIAM J. Control Optim., 14, 877-898.