https://doi.org/10.1007/s10957-020-01787-7

**On the Linear Convergence of Forward–Backward Splitting** **Method: Part I—Convergence Analysis**

**Yunier Bello-Cruz**^{1}**·Guoyin Li**^{2}**·Tran T. A. Nghia**^{3}

Received: 19 March 2019 / Accepted: 9 November 2020

© Springer Science+Business Media, LLC, part of Springer Nature 2020

**Abstract**

In this paper, we study the complexity of the forward–backward splitting method with Beck–Teboulle’s line search for solving convex optimization problems, where the objective function can be split into the sum of a differentiable function and a nons- mooth function. We show that the method converges weakly to an optimal solution in Hilbert spaces, under mild standing assumptions without the global Lipschitz conti- nuity of the gradient of the differentiable function involved. Our standing assumptions is weaker than the corresponding conditions in the paper of Salzo (SIAM J Optim 27:2153–2181, 2017). The conventional complexity of sublinear convergence for the functional value is also obtained under the local Lipschitz continuity of the gradient of the differentiable function. Our main results are about the linear convergence of this method (in the quotient type), in terms of both the function value sequence and the itera- tive sequence, under only the quadratic growth condition. Our proof technique is direct from the quadratic growth conditions and some properties of the forward–backward splitting method without using error bounds or Kurdya-Łojasiewicz inequality as in other publications in this direction.

**Keywords** Nonsmooth and convex optimization problems·Forward–Backward
splitting method·Linear convergence·Quadratic growth condition

**Mathematics Subject Classification** 65K05·90C25·90C30

### B

^{Guoyin Li}

g.li@unsw.edu.au Yunier Bello-Cruz yunierbello@niu.edu Tran T. A. Nghia nttran@oakland.edu

1 Department of Mathematical Sciences, Northern Illinois University, DeKalb, IL 60115, USA 2 Department of Applied Mathematics, University of New South Wales, Sydney 2052, Australia 3 Department of Mathematics and Statistics, Oakland University, Rochester, MI 48309, USA

**1 Introduction**

In this paper, we mainly consider an optimization problem of minimizing the sum of two convex functions, one of which is differentiable and the other is nondifferen- tiable. Many problems in this format have appeared in different fields of science and engineering including machine learning, compressed sensing, and image processing.

A particular class of this problem is the so-called1-regularized problem that usually provides sparse solutions with many applications in signal processing and statistics [1–3].

Among many methods solving the aforementioned optimization problems, the
forward–backward splitting method (FBS in brief, known also as the proximal gradient
method) [2,4–9] is very popular due to its simplicity and efficiency. It is well-known
that this method is globally convergent to an optimal solution with the complexity
*O(k*^{−}^{1}*)*on the iterative cost values under the assumption that the gradient of the
differentiable function is globally Lipschitz continuous [6]. An important advance
was due to [10], where the sublinear convergence was improved to*o(k*^{−}^{1}*)*in Hilbert
spaces. Independently, by using some line searches motivated from the work of Tseng
[11] and under a more restricted step size rule, Bello-Cruz and Nghia [12] achieve the
same complexity with the milder assumption that the aforementioned gradient is only
*locally*Lipschitz continuous in finite-dimensional spaces. The results in [12] were fur-
ther extended in [13] to different line searches for the FBS method and more general
classes of optimization problems even in infinite-dimensional spaces. A recent work
of Bauschke, Bolte, and Teboulle [14] also tackles the absence of Lipschitz continuous
gradient by introducing the so-called NoLips algorithm close to the FBS method with
the involvement of Bregman distance. Their algorithm provides the sublinear com-
plexity *O(*^{1}_{k}*)*and guarantees the global convergence with an additional hypothesis
about the closedness of the domain of an auxiliary Legendre function defined there.

Unfortunately, the latter assumption as well as those in [12] are not satisfied for the
Poisson inverse regularized problems with Kullback–Leibler divergence [15,16], one
of the main applications in [14]. This situation was overcome in [13] by using the
FBS method and would be revisited again in our sequence [17] with further advanced
achievements on the linear convergence in the quotient type (often referred as Q-linear
convergence). Indeed, Salzo in [13, Section 4] proposed some new hypotheses to avoid
the standard assumptions [6,10,12] on the domain of cost functions. Although these
hypotheses are valid in many natural situations of optimization problems, some non-
trivial parts of them can be further relaxed. In Sect.3, by reanalyzing the theory of the
FBS method in [12,13], we relax and weaken several conditions assumed in [13] to
acquire the same global convergence and sublinear complexity in Hilbert spaces.^{1}The
proof of many results in this section modifies our corresponding ones in [12] under
the simplified standing assumptions. This section is not a major part of the paper, but
the corresponding results will be used intensively in our main Sect.4.

The central part of our paper, Sect.4devotes to the linear convergence of the FBS method without the global Lipschitz continuity of gradient mentioned above in finite

1 We are indebted to some remarks from one referee that allow us to extend the results from finite dimensions to Hilbert spaces in the current version.

dimensions. Despite the popularity of the FBS method, the linear convergence of this method has been established recently by using Kurdya-Łojasiewicz inequality [18–22]

(even for nonconvex optimization problems) or some error bound conditions [23–27]

with the base from [28]. Those conditions are somehow equivalent in convex settings;

see, e.g., [18,19,25]. Linear convergence of the FBS method was also obtained in
Hilbert spaces [5] for Lasso problems under some nontrivial conditions such as*finite*
*basis injectivity*and*strict sparsity pattern. These conditions were fully relaxed in the*
papers [19,20], which also work in infinite dimensions for more general optimization
problems. Our results can be also extended to infinite dimensions by following some
ideas of [19], but for simplicity, we restrict ourselves on finite dimensions in this sec-
tion. Our approach is close to the works of Drusvyatskiy and Lewis [25] and Garrigos,
Rosasco, and Villa [19,20] by using the so-called*quadratic growth condition*known
also as *2-conditioned property. However, our proof of linear convergence is more*
direct just from quadratic growth condition without using KŁ inequality as in [18–21]

or the error bound [28] as in [23,25] and reveals the*Q-linear convergence for the FBS*
sequence rather than the*R-one obtained in all the aforementioned works. Some of our*
linear rates are sharper than those in [19,25]. The property of quadratic growth condi-
tion is indeed automatic in many classes of optimization problems including Poisson
inverse regularized problem [14–16], least-square1regularized problems [3,5,6,19],
group Lasso [23] or under mild second-order conditions on initial data [2,30–34]; see
our second part [17] for further studies in this direction.

**2 Preliminaries**

Throughout the paper, *H* is a Hilbert space, where · and·,·denote the cor-
responding norm and inner product in *H*. We use 0*(H)* to denote the set of
proper, lower semicontinuous, and convex functions on *H*. Let *h* ∈ 0*(H), we*
write dom*h* := {x ∈ *H* :*h(x) <* +∞}. The subdifferential of*h* at*x*¯ ∈ dom*h* is
defined by

*∂h(¯x)*:= {v∈*H* : v,*x*− ¯*x ≤h(x)*−*h(x),*¯ *x*∈*H*}. (1)
We say*h* satisfies the*quadratic growth condition*at*x*¯ with modulus*κ >*0 if there
exists*ε >*0 such that

*h(x)*≥*h(x*¯*)*+*κ*

2*d*^{2}*(x*;*(∂h)*^{−}^{1}*(*0*))* for all *x*∈B*ε**(x*¯*).* (2)
Moreover, if in additionally*(∂h)*^{−}^{1}*(0)*= { ¯*x},h*is said to satisfy the*strong*quadratic
growth condition at*x*¯with modulus*κ*.

Some relationship between the quadratic growth condition and the so-called metric
subregularity of the subdifferential could be found in [18,29–31,34] even for the case
of nonconvex functions. The quadratic growth condition (2) is also called*quadratic*
*functional growth*property in [26] when*h*is continuously differentiable over a closed
convex set. In [19,20],*h*is said to be 2-conditioned onB*ε**(x*¯*)*if it satisfies the quadratic
growth condition (2).

The quadratic growth condition is slightly different in [25] as follows:

*h(x)*≥*h(x)*¯ +*c*

2*d*^{2}*(x;(∂h)*^{−}^{1}*(0))* for all *x*∈ [h *<h*^{∗}+*ν]* (3)
for some constants*c, ν >* 0, where[h *<* *h*^{∗}+*ν] := {x* ∈ *H* :*h(x) <* *h*^{∗}+*ν}*

and*h*^{∗}:=inf*h(x)*=*h(x). It is easy to check that this growth condition implies (2).*¯
Indeed, suppose that (3) is satisfied for some*c, ν >*0 Define*η*:=

2*ν*

*c* and note that

¯

*x*∈ [h *<h*^{∗}+*ν]. Take anyx*∈B_{η}*(¯x), ifx*∈ [h*<h*^{∗}+*ν], inequality (2) is trivial.*

If*x*∈ [h*/* *<h*^{∗}+*ν], it follows that*
*h(x)≥h(¯x)*+*ν*=*h(¯x)+c*

2*η*^{2}≥h(*x)*¯ +*c*

2x− ¯*x*^{2}≥*h(x)*¯ +*c*

2*d*^{2}*(x;(∂h)*^{−}^{1}*(0)),*
which clearly verifies (2). Thus, (2) is weaker than (3), but it is equivalent to the local
version of (3):

*h(x)*≥*h(x)*¯ +*c*

2*d*^{2}*(x;(∂h)*^{−}^{1}*(0))* for all *x* ∈ [h *<h*^{∗}+*ν] ∩*B_{ε}*(¯x)*
for some constants*c, ν, ε >*0. This property has been showed recently in [18, Theo-
rem 5] to be equivalent to the fact that*h*satisfies the Kurdyka-Łojasiewicz inequality
with order_{2}^{1}.

To complete this section, we recall two important notions of linear convergence in
our study. Let*μ*be a real number such that 0*< μ <*1. A sequence*(x*^{k}*)**k*∈N⊂*H* is
said to be R-linearly convergent to*x*^{∗}with rate*μ*if there exists*M* *>*0 such that

x* ^{k}*−

*x*

^{∗}≤

*Mμ*

*=*

^{k}*O(μ*

^{k}*)*for all

*k*∈N.

We say*(x*^{k}*)**k*∈Nis*Q-linearly convergent*to*x*^{∗}with rate*μ*if there exists*K* ∈Nwith
x^{k}^{+}^{1}−*x*^{∗} ≤*μx** ^{k}*−

*x*

^{∗}for all

*k*≥

*K.*

From the definitions, it can be directly verified that Q-linear convergence implies R-linear convergence with the same rate. On the other hand, in general, R-linear convergence does not imply Q-linear convergence.

**3 Global Convergence of Forward–Backward Splitting Methods in**
**Hilbert Spaces**

Throughout the paper, we consider the following optimization problem

*x*min∈H *F(x)*:= *f(x)*+*g(x),* (4)

where *f,g*∈0*(H)*and *f* is differentiable on int*(*dom*f)∩*dom*g; see our standing*
assumptions below. This section reanalyzes the theory for forward–backward splitting

(FBS) methods:

*x*^{k}^{+}^{1}=prox_{α}_{k}_{g}*(x** ^{k}*−

*α*

*k*∇

*f(x*

^{k}*))*(5) with the proximal operator defined later in (8) and the stepsize

*α*

*k*

*>*0, when the gradient∇

*f*is not globally Lipschitz continuous [12,13]. The results here will be employed in our main topic about the linear convergence of the FBS method in Sect.4.

The convergence of the FBS method without Lipschitz continuity of∇*f* was stud-
ied in [12] via some line searches with the standard assumption dom*g* ⊂ dom *f*
together with some assumptions on the continuity of∇*f*. In [13], Salzo simplified
these assumptions and significantly extended it to more general frameworks via many
line searches. It is worth noting that the standard assumption dom*g*⊂dom *f*, used in
[12], is not satisfied by several important optimization problems including the Poisson
inverse regularized problems with Kullback–Leibler divergence [15,16]. This situa-
tion was overcome in [13, Section 4 and 5] with the following assumptions on *f* and
*g:*

**H1.** *f,g* ∈0*(H)*are bounded from below with int*(dom* *f)*∩dom*g* = ∅.

**H2.** *f* is differentiable on int(dom*f)*∩dom*g,*∇*f* is uniformly continuous on any
weakly compact subset of int(dom *f)*∩dom*g, and*∇*f* is bounded on any sub-
level sets of*F.*

**H3.** For*x*∈int(dom *f)*∩dom*g,*{F ≤*F(x)} ⊂*int(dom*f)*∩dom*g, and*

*d({F*≤ *F(x)};H* \int*(*dom *f)) >*0*.* (6)
These assumptions hold under some mild conditions; see [13, Proposition 4.2] for
details. However, they are not trivial even in finite dimensions. Throughout the paper,
we focus on the FBS method with the popular Beck–Teboulle line search [6] that was
also studied in [13]. Next, following the techniques used in [12,13], we show that the
global convergence of the FBS method holds true under the below simplified standing
assumptions that relax some conditions from**H1–H3:**

**A1.** *f,g* ∈0*(H)*and int(dom *f)*∩dom*g*= ∅.

**A2.** *f* is differentiable at any point in int(dom *f)∩domg,*∇*f*is uniformly continuous
on any weakly compact subset of int(dom *f)*∩dom*g.*

**A3.** For any*x* ∈int*(*dom *f)*∩dom*g,*{*F* ≤ *F(x)} ⊂*int*(*dom*f)*∩dom*g* and for
any weakly compact subset*X* of{*F* ≤*F(x)}*we have

*d(X*;*H* \int(dom *f)) >*0. (7)

* Remark 3.1* (The standing assumptions in finite dimensions) When dim

*H*

*<*∞,

**A2**means that

*f*is continuously differentiable on int(dom

*f)*∩dom

*g. Moreover, in*

**A3**the distance gap requirement (7) is automatically true as

*X*∩

*(H*\int

*(*dom

*f))*= ∅ and

*X*is compact. On the other hand, the condition (6) is still nontrivial; see the example below.

Clearly, some boundedness assumptions in **H1–H2**are relaxed in our **A1–A2.**

Moreover, the positive distance gap requirement (6) in (H3) is slightly stronger than
(7) in**(A3). But, it should be noted that condition (6) can, in general, be easier to verify**

in infinite-dimensional spaces. Even in finite dimensions,**(A1–A3)**are strictly weaker
than (H1–H3). To see this, consider *p*∈ {1*,*2},

*C**p*:= {x=*(x*1*,x*2*)*∈R^{2}:*x*1*x*2≥ *p,x*1*>*0,*x*2*>*0},

and *f,g* :R^{2}→R∪ {+∞}be defined as *f(x)*= −log*x*1−log*x*2if*x*∈R^{2}_{++}and
+∞otherwise, and*g(x)*=*δ**C*_{1}*(x)*where*δ**C*_{1} is the indicator function of the set*C*1

defined above. It can be directly verified that the assumptions (A1–A3) are satisfied.

On the other hand,**H1**fails because *f* is unbounded on

int*(*dom *f)*∩dom*g*=R^{2}_{++}∩*C*1=*C*1= {(*x*1*,x*2*)*

∈R^{2}:*x*1*x*2≥1,*x*1*>*0,*x*2*>*0}.

Moreover,*(2,*1)∈int(dom *f)∩domg*and since*F(2,*1)=*(f*+g)(2,1)= −log 2,
we get

{*x*: *F(x)*≤ −log 2} = {(*x*1*,x*2*)*∈R^{2}:*x*1*x*2≥2*,x*1*>*0*,x*2*>*0} =*C*2*.*
So, clearly **H2** also fails because ∇*f(x*1*,x*2*)* = *(−*_{x}^{1}_{1}*,*−_{x}^{1}_{2}*)* is unbounded on
the sublevel set of *F* above. Finally, observing that *(*^{2}_{k}*,k)* ∈ *C*2 and *(0,k)* ∈
R^{2}\int*(domf)*=R^{2}\R^{2}_{++}, we see that

*d*

{F≤ −log 2};R^{2}\int*(dom* *f)*

=*d*

*C*2;R^{2}\R^{2}_{++}

≤ lim

*k*→∞

*(*2

*k,k)*−*(0,k)*
= lim

*k*→∞

2
*k* =0.

This shows that**H3**fails in this case.

Next, let us recall the proximal operator prox* _{g}*:

*H*→dom

*g*given by

prox_{g}*(z)*:=*(Id* +*∂g)*^{−}^{1}*(z)* for all *z*∈*H,* (8)
which is well-known to be a single-valued mapping with full domain. With*α >*0, it
is easy to check that

*z*−prox_{α}_{g}*(z)*

*α* ∈*∂g(*prox_{α}_{g}*(z))* for all *z*∈*H.* (9)
Let*S*^{∗} be the optimal solution set to problem (4) and*x*^{∗}be in int(dom *f)*∩dom*g*
due to our assumption**A3. Then,***x*^{∗}∈ *S*^{∗}iff

0∈*∂(f* +*g)(x*^{∗}*)*= ∇*f(x*^{∗}*)*+*∂g(x*^{∗}*).* (10)
The following lemma, a consequence of [4, Theorem 23.47] is helpful in our proof of
the finite termination of Beck–Teboulle’s line search under our standing assumptions.

**Lemma 3.1** *Let g*∈0*(H)and letα >*0. Then, for every x∈dom*g, we have*

prox_{α}_{g}*(x)*→*x as* *α*↓0*.* (11)

Under our standing assumptions, we define the*proximal forward–backward oper-*
*ator*

*J* :

int*(*dom*f)*∩dom*g*

×R++→dom*g*
by

*J(x, α)*:=prox_{α}_{g}*(x*−*α∇f(x))* for all *x* ∈int*(*dom *f)*∩dom*g, α >*0*.* (12)
The following result is essentially from [12, Lemma 2.4]. Even if the standing assump-
tions are different, the proof is quite similar. A more general variant of this result could
be found in [35, Lemma 3].

**Lemma 3.2** *For any x* ∈int*(*dom *f)*∩dom*g, we have*
*α*2

*α*1

x−*J(x, α*1*) ≥ x*−*J(x, α*2*) ≥ x*−*J(x, α*1*)* *for all* *α*2≥*α*1*>*0. (13)
* Proof* By using (9) and (12) with

*z*=

*x*−

*α∇f(x), we have*

*x*−*α∇f(x)*−*J(x, α)*

*α* ∈*∂g(J(x, α))* (14)

for all*α >*0. For any*α*2≥*α*1*>*0, it follows from the monotonicity of*∂g*and (14)
that

0≤

*x*−*α*2∇*f(x)*−*J(x, α*2*)*

*α*2 − *x*−*α*1∇*f(x)*−*J(x, α*1*)*

*α*1 *,J(x, α*2*)*−*J(x, α*1*)*

=

*x*−*J(x, α*2*)*

*α*2 −*x*−*J(x, α*1*)*

*α*1 *, (x*−*J(x, α*1*))*−*(x*−*J(x, α*2*))*

= −x−*J(x, α*2*)*^{2}

*α*2 −x−*J(x, α*1*)*^{2}
*α*1

+ 1

*α*2

+ 1
*α*1

x−*J(x, α*2*),x*−*J(x, α*1*)*

≤ −*x*−*J(x, α*2*)*^{2}

*α*2 −*x*−*J(x, α*1*)*^{2}
*α*1

+ 1

*α*2+ 1
*α*1

*x*−*J(x, α*2*) · x*−*J(x, α*1*).*

This gives us that

x−*J(x, α*2*) − x*−*J(x, α*1*)*

·

x−*J(x, α*2*) −α*2

*α*1x−*J(x, α*1*)*

≤0.

Since *α*2

*α*1 ≥1, we derive (13) and thus complete the proof of the lemma.

Next, let us present Beck–Teboulle’s backtracking line search [6], which is specifi-
cally useful for forward–backward methods when the Lipschitz constant of∇*f* is not
known or hard to estimate.

**Linesearch BT**(Beck–Teboulle’s Line Search)
Given*x*∈int(dom *f)*∩dom*g,σ >*0 and 0*< θ <*1.

**Input.**Set*α*=*σ* and*J(x, α)*=prox_{α}_{g}*(x*−*α∇f(x))*with*x*∈dom*g.*

**While** *f(J(x, α)) >* *f(x)*+ ∇*f(x),J(x, α)*−*x +* 1

2αx−*J(x, α)*^{2}*,***do**
*α*=*θα.*

**End While**
**Output.***α.*

The output*α*in this line search will be denoted by*L S(x, σ, θ). Let us show the well-*
definedness and finite termination of this line search under the standing assumptions
**A1–A3**below.

**Proposition 3.1** (Finite Termination of Beck–Teboulle’s Line Search) *Suppose that*
*assumptions***A1–A2***hold. Then, for any x* ∈int*(*dom *f)*∩dom*g, we have*

(i) *The above line search terminates after finitely many iterations with the positive*
*outputα*¯ =*L S(x, σ, θ).*

(ii) x−*u*^{2}− J*(x,α)*¯ −*u*^{2}≥2*α[F(J*¯ *(x,α))*¯ −*F(u)]for any u*∈*H.*
(iii) *F(J(x,α))*¯ −*F(x)*≤ − 1

2*α*¯*J(x,α)−x*¯ ^{2}≤0. Consequently, by**A3, J***(x,α)*¯ ∈
int*(*dom *f)*∩dom*g.*

* Proof* Take any

*x*∈int

*(*dom

*f)*∩dom

*g. Let us justify (i) first. Note that*

*J(x, α)*is well-defined for any

*α >*0 because∇

*f(x)*exists due to assumption

**A2. If**

*x*∈

*S*

^{∗}, where

*S*

^{∗}is the optimal solution set to problem (4), then

*x*=

*J(x, σ)*due to (10) and (9). Thus, the line search stops with zero step and gives us the output

*σ*and

*x*=

*J(x, σ)*∈ int(dom

*f)*∩dom

*g. Ifx*∈

*/*

*S*

^{∗}, suppose by contradiction that the line search does not terminate after finitely many steps. Hence, for all

*α*∈

*P*:=

{σ, σθ, σθ^{2}*, . . .}*it follows that

∇*f(x),J(x, α)*−*x +* 1

2αx−*J(x, α)*^{2}*<* *f(J(x, α))*− *f(x).* (15)
Since prox_{α}* _{g}*is non-expansive, we have

*J(x, α)*−*x ≤ prox*_{α}*g**(x*−*α∇f(x))*−prox_{α}_{g}*(x) + prox*_{α}*g**(x)*−*x*

≤*α∇f(x) + prox*_{α}_{g}*(x)*−*x.* (16)

Lemma3.1tells us that

*J(x, α)*−*x* →0 as *α*↓0*.* (17)

Since*x* ∈int*(*dom *f)*, there exists∈Nsuch that*J(x, α)*∈int*(*dom *f)*for all
*α*∈*P*^{}:= {σ θ^{}*, σθ*^{+}^{1}*, . . .} ⊆P.*

Thanks to the convexity of *f*, we have

*f(x)*− *f(J(x, α))*≥ ∇*f(J(x, α)),x*−*J(x, α)* for *α*∈*P*^{}*.* (18)
This inequality together with (15) implies

1

2α*J(x, α)*−*x*^{2}*<*∇*f*

*J(x, α)*

− ∇*f(x),J(x, α)*−*x*

≤∇*f*

*J(x, α)*

− ∇*f(x)*· J(x, α)−*x,*
which yields*J(x, α)*=*x*and

0*<J(x, α)*−*x*

*α* *<*2∇*f*

*J(x, α)*

− ∇*f(x)* for all *α*∈*P*^{}*.* (19)
Since*x*− *J(x, α) →* 0 as*α* → 0 by (17), *J(x, α)* ∈ int*(*dom*f)*∩dom*g* for
sufficiently small*α >*0. Due to the continuity of∇*f* at*x* ∈int*(*dom *f)*∩dom*g*by
assumption**A2, we obtain from (19) that**

*α→*lim0*,α∈P*^{}

x−*J(x, α)*

*α* =0*.* (20)

Applying (9) with*z*=*x*−α∇*f(x)*gives us that*x*−*J(x, α)*

*α* −∇*f(x)*∈*∂g(J(x, α)).*

Taking*α* → 0, we have−∇*f(x)* ∈ *∂g(x)*due to the demiclosedness of the subd-
ifferentials; see, e.g., [38, Theorem 4.7.1 and Proposition 4.2.1(i)]. It follows that
0 ∈ ∇*f(x)*+*∂g(x), which contradicts the hypothesis thatx* ∈*/* *S*^{∗}by (10). Hence,
the line search terminates after finitely many steps with the output*α*¯ .

To proceed the proof of (ii), note that

*f(J(x,α))*¯ ≤ *f(x)*+ ∇*f(x),J(x,α)*¯ −*x* + 1

2*α*¯*x*−*J(x,α)*¯ ^{2}*.* (21)
Moreover, by (9) , we have

*x*−*J(x,α)*¯

¯

*α* − ∇*f(x)*∈*∂g(J(x,α)).*¯
Pick any*u*∈*H*, we get from the latter that

*g(u)*−*g(J(x,α))*¯ ≥

*x*−*J(x,α)*¯

¯

*α* − ∇*f(x),u*−*J(x,α)*¯ *.* (22)

Note also that *f(u)*−*f(x)*≥ ∇*f(x),u*−*x*.This together with (22) and (21) gives
us that

*F**(**u**)*=*(**f*+*g**)(**u**)*≥ *f**(**x**)*+*g**(**J**(**x**,**α))*¯
+

*x*−*J**(**x**,**α)*¯

¯

*α* − ∇*f**(**x**),**u*−*J**(**x**,**α)*¯ + ∇*f**(**x**),**u*−*x*

= *f**(**x**)*+*g**(**J**(**x**,**α))*¯ +1

¯

*α**x*−*J**(**x**,**α),*¯ *u*−*J**(**x**,**α) + ∇*¯ *f**(**x**),**J**(**x**,**α)*¯ −*x*

≥ *f**(**J**(**x**,**α))*¯ +*g**(**J**(**x**,**α))+*¯ 1

¯

*α**x*−*J**(**x**,**α),*¯ *u*−*J**(**x**,**α)−*¯ 1

2*α*¯*J**(**x**,**α)*¯ −*x*^{2}*.*
It follows that

x−*J(x,α),*¯ *J(x,α)*¯ −*u ≥ ¯α[F(J(x,α))*¯ −*F(u)] −*1

2*J(x,α)*¯ −*x*^{2}*.*
Since 2*x*−*J(x,α),*¯ *J(x,α)*¯ −*u* = *x*−*u*^{2}− *J(x,α)*¯ −*x*^{2}− *J(x,α)*¯ −*u*^{2}*,*
the latter implies that

x−*u*^{2}− *J(x,α)*¯ −*u*^{2}≥2*α[F*¯ *(J(x,α))*¯ −*F(u)],*
which clearly ensures (ii).

Finally, (iii) is a direct consequence of (ii) with *u* = *x. It follows that* *J(x,α)*¯
belongs to the sublevel set{F ≤ *F(x)}. By assumption***A3,***J(x,α)*¯ ∈ int(dom *f).*

The proof is complete.

Now, we recall the forward–backward splitting method with line search proposed by [6] as following.

**Forward–Backward Splitting Method with Backtracking Line Search**(FBS
method)

**Step 0.**Take*x*^{0}∈int(dom*f)*∩dom*g,σ >*0 and 0*< θ <*1.

**Step k.**Set

*x*^{k}^{+}^{1}:=*J(x*^{k}*, α**k**)*=prox_{α}_{k}_{g}*(x** ^{k}*−

*α*

*k*∇

*f(x*

^{k}*))*(23) with

*α*−1:=

*σ*and

*α**k* :=*L S(x*^{k}*, α**k*−1*, θ).* (24)
The following result which is a direct consequence of Proposition3.1plays the central
role in our study.

**Corollary 3.1** (Well-definedness of the FBS method) *Let x*^{0} ∈int(dom*f)*∩dom*g.*

*The sequence(x*^{k}*)**k*∈N*from the FBS method is well-defined, x** ^{k}* ∈ int

*(domf)*∩

*g,*

*and f is differentiable at any x*

^{k}*for all k*∈N. Moreover, for any x∈

*H, we have*

(i) *x** ^{k}*−

*x*

^{2}−

*x*

^{k}^{+}

^{1}−

*x*

^{2}≥2

*α*

*k*

*F(x*^{k}^{+}^{1}*)*−*F(x)*
*.*

(i) *F(x*^{k}^{+}^{1}*)*−*F(x*^{k}*)*≤ − 1

2*α**k*x^{k}^{+}^{1}−*x*^{k}^{2}*.*

* Proof* Thanks to Proposition3.1(iii),

*x*

*∈int(dom*

^{k}*f)∩*dom

*g*and

*f*is differentiable at any

*x*

*by assumption*

^{k}**A2. This verifies the well-definedness of**

*(x*

^{k}*)*

*k*∈N. Moreover, both (i) and (ii) are consequence of (ii) and (iii) from Proposition3.1by replacing

*u*=

*x,x*=

*x*

*,*

^{k}*α*¯ =

*α*

*k*, and

*J(x,α)*¯ =

*J(x*

^{k}*, α*

*k*

*)*=

*x*

^{k}^{+}

^{1}. The following result recovers the global convergence of the FBS method without supposing the Lipschitz continuity on the gradient∇

*f*in [13, Theorem 3.18] and [12, Theorem 4.2] with our simplified assumptions

*(A1*−

**A3). The proof is similar to that**of [12, Theorem 4.2] with some modifications and ideas from [13].

**Theorem 3.1** (Global Convergence of the FBS Method) *Let(x*^{k}*)**k*∈N*be the sequence*
*generated from the FBS method. The following statements hold:*

(i) *If S*^{∗}= ∅, then*(x*^{k}*)**k*∈N*weakly converges to a point in S*^{∗}*. Moreover,*

*k*lim→∞*F(x*^{k}*)*= min

*x*∈H *F(x).* (25)

(ii) *If S*^{∗}= ∅, then we have

*k*lim→∞x* ^{k}* = +∞

*and*lim

*k*→∞*F(x*^{k}*)*= inf

*x*∈H *F(x).*

* Proof* Let us justify (i) by supposing that

*S*

^{∗}= ∅. By Corollary3.1(i), for any

*x*

^{∗}∈

*S*

^{∗}we have

x* ^{k}*−

*x*

^{∗}

^{2}− x

^{k}^{+}

^{1}−

*x*

^{∗}

^{2}≥2α

*k*[F

*(x*

^{k}^{+}

^{1}

*)*−

*F(x*

^{∗}

*)] ≥*0. (26) It follows that the sequence

*(x*

^{k}*)*

*k*∈Nis Fejér monotone with respect to

*S*

^{∗}. Thus, it is bounded according to [4, Proposition 5.4]. Define

*M*:=sup{x

*−*

^{k}*x*

^{∗}:

*k*∈N}

*<*

+∞for some*x*^{∗}∈ *S*^{∗}, we get from (26) that

0≤2α*k*[F*(x*^{k}^{+}^{1}*)*−*F(x*^{∗}*)] ≤ x** ^{k}*−

*x*

^{∗}

^{2}− x

^{k}^{+}

^{1}−

*x*

^{∗}

^{2}

=*(x** ^{k}*−

*x*

^{∗}+ x

^{k}^{+}

^{1}−

*x*

^{∗})·

*(x*

*−*

^{k}*x*

^{∗}− x

^{k}^{+}

^{1}−

*x*

^{∗})

≤2Mx* ^{k}*−

*x*

^{k}^{+}

^{1}. This yields that

0≤ *F(x*^{k}^{+}^{1}*)*−*F(x*^{∗}*)*≤*M* x* ^{k}*−

*x*

^{k}^{+}

^{1}

*α**k* *.* (27)

Note further from Corollary3.1(ii) that

2*σ*[*F(x*^{k}*)*−*F(x*^{k}^{+}^{1}*)] ≥*2*α**k*[*F(x*^{k}*)*−*F(x*^{k}^{+}^{1}*)] ≥ x** ^{k}*−

*x*

^{k}^{+}

^{1}

^{2}

*.*

As*(F(x*^{k}*))**k*∈Nis decreasing and bounded below by*F(x*^{∗}*)*, the latter implies

∞*>*2σ[F*(x*^{0}*)*−*F(x*^{∗}*)] ≥*
∞
*k*=0

2σ[F(x^{k}*)*−*F(x*^{k}^{+}^{1}*)] ≥*
∞
*k*=0

x* ^{k}*−

*x*

^{k}^{+}

^{1}

^{2}

*.*(28)

Thus,x* ^{k}*−x

^{k}^{+}

^{1}→0 as

*k*→ ∞. Let

*x*¯be a weak accumulation point of

*(x*

^{k}*)*

*k*∈N. Let us find a subsequence

*(x*

^{n}

^{k}*)*

*k*∈Nweakly converging to

*x. As*¯

*F*is l.s.c. and convex, it is also weakly l.s.c.. Moreover, since

*(F(x*

^{k}*))*

*k*∈Nis decreasing, we have

*F(x*

^{0}

*)*≥

*F(x).*¯ According to assumptions

**A2**and

**A3,**

*f*is differentiable at

*x*¯∈int(dom

*f)*∩dom

*g.*

As*(α**k**)**k*∈Nis decreasing by the construction of the line search in the FBS method,
*α*:= lim

*k*→∞*α**k* = lim

*k*→∞*α**n**k*exists.

**Case 1:***α >*0. Note from (9) and (23that *x*^{n}* ^{k}*−

*α*

*n*

*∇*

_{k}*f(x*

^{n}

^{k}*)*−

*x*

^{n}

^{k}^{+}

^{1}

*α**n** _{k}* ∈

*∂g(x*

^{n}

^{k}^{+}

^{1}

*),*which implies

*x*^{n}* ^{k}* −

*x*

^{n}

^{k}^{+}

^{1}

*α**n** _{k}* + ∇

*f(x*

^{n}

^{k}^{+}

^{1}

*)*− ∇

*f(x*

^{n}

^{k}*)*∈ ∇

*f(x*

^{n}

^{k}^{+}

^{1}

*)*+

*∂g(x*

^{n}

^{k}^{+}

^{1}

*).*(29)

Applying Corollary 3.1(ii), we have *x*^{n}* ^{k}* −

*x*

^{n}

^{k}^{+}

^{1}

^{2}

*α*

*n*

*k*

→ 0 as the sequence
*(F(x*^{k}*))**k*∈Nis decreasing and thus converging. Since*α**n**k* ≥*α >*0, *x*^{n}* ^{k}*−

*x*

^{n}

^{k}^{+}

^{1}

*α**n*_{k}

also converges to 0. Moreover, note from (28) that*x*^{n}* ^{k}*−

*x*

^{n}

^{k}^{+}

^{1}→0

*,*∇

*f(x*

^{n}

^{k}*)*−

∇*f(x*^{n}^{k}^{+}^{1}*) →* 0 due to the uniform continuity of∇*f* on the weakly compact set
*x, (x*¯ ^{n}^{k}*)**k*∈N*, (x*^{n}^{k}^{+}^{1}*)**k*∈N

of[F ≤*F(x*^{0}*)]*as in assumption**A2.**

Taking*k*→ ∞in (29) gives us that 0∈ ∇*f(¯x)+∂g(x)*¯ due to the demiclosedness
of the subdifferentials, which implies *x*¯ ∈ *S*^{∗}. Furthermore, since*(F(x*^{k}*))**k*∈N is
decreasing, (25) is a consequence of (27).

**Case 2:***α*=0. Define*α*ˆ*n**k* := *α**n*_{k}

*θ* *> α**n**k* *>*0 and*x*ˆ^{n}* ^{k}* :=

*J*

*x*

^{n}

^{k}*,α*ˆ

*n*

*k*

∈dom*g. It*
follows from the definition of**Linesearch BT**that

*f(x*ˆ^{n}^{k}*) >* *f(x*^{n}^{k}*)*+ ∇*f(x*^{n}^{k}*),x*ˆ^{n}* ^{k}* −

*x*

^{n}*+ 1 2*

^{k}*α*ˆ

*n*

*k*

x^{n}* ^{k}*− ˆ

*x*

^{n}

^{k}^{2}

*.*(30)

Due to Lemma3.2, we have

x^{n}* ^{k}* − ˆ

*x*

^{n}*= x*

^{k}

^{n}*−*

^{k}*J(x*

^{n}

^{k}*,α*ˆ

*n*

*k*

*) ≤*

*α*ˆ

*n*

_{k}*α**n** _{k}*x

^{n}*−*

^{k}*J(x*

^{n}

^{k}*, α*

*n*

*k*

*)*

= 1

*θ*x^{n}* ^{k}*−

*x*

^{n}

^{k}^{+}

^{1}→0,

which tells us that*(x*ˆ^{n}^{k}*)**k*∈Nweakly converges to*x*¯ ∈ int*(*dom *f)*∩dom*g. Define*
*A* := { ¯*x, (x*^{n}^{k}*)**k*∈N}. It follows that*A* ⊆ [*F* ≤ *F(x*^{0}*)]*, which is weakly compact.

By assumption**A3,***d(A,H* \int*(*dom *f)) >*0. As*x*^{n}* ^{k}*− ˆ

*x*

^{n}*→0, we find some*

^{k}*K*

*>*0 such that

*x*ˆ

^{n}*∈int*

^{k}*(*dom

*f)*and thus

*x*ˆ

^{n}*∈int*

^{k}*(*dom

*f)∩*dom

*g*for any

*k>K*. Hence,∇

*f(x*ˆ

^{n}

^{k}*)*is well-defined by assumption

**A2. It follows that**

*f(x*^{n}^{k}*)*≥ *f(x*ˆ^{n}^{k}*)*+ ∇*f(x*ˆ^{n}^{k}*),x*^{n}* ^{k}*− ˆ

*x*

^{n}*. This together with (30) gives us that*

^{k}1

2*α*ˆ*n** _{k}*x

^{n}*− ˆ*

^{k}*x*

^{n}

^{k}^{2}

*<*∇

*f(x*

^{n}

^{k}*)*− ∇

*f(ˆx*

^{n}

^{k}*),x*

^{n}*− ˆ*

^{k}*x*

^{n}

^{k}≤ ∇*f(x*^{n}^{k}*)*− ∇*f(x*ˆ^{n}^{k}*) · x*^{n}* ^{k}*− ˆ

*x*

^{n}*. Hence, we have*

^{k}∇*f(x*^{n}^{k}*)*− ∇*f(x*ˆ^{n}^{k}*) ≥* 1
2*α*ˆ*n**k*

x^{n}* ^{k}*− ˆ

*x*

^{n}*. (31)*

^{k}As

¯

*x, (x*^{n}^{k}*)**k*∈N*, (x*ˆ^{n}^{k}*)**k*∈N

is a weakly compact subset of int*(*dom *f)*∩dom*g* and
ˆ*x*^{n}* ^{k}*−

*x*

^{n}*→0, assumption*

^{k}**A2**tells us that∇

*f(x*

^{n}

^{k}*)−∇f(ˆx*

^{n}

^{k}*) →*0 as

*k*→ ∞. We derive from (31) that

*k*lim→∞

1 ˆ

*α**n** _{k}*x

^{n}*− ˆ*

^{k}*x*

^{n}*=0. (32)*

^{k}Applying (9) with*z*=*x*^{n}* ^{k}*− ˆ

*α*

*n*

*∇*

_{k}*f(x*

^{n}

^{k}*)*gives us that

*x*

^{n}*− ˆ*

^{k}*x*

^{n}

^{k}ˆ
*α**n**k*

= ∇*f(ˆx*^{n}^{k}*)*+*x*^{n}* ^{k}*− ˆ

*α*

*n*

*k*∇

*f(x*

^{n}

^{k}*)*− ˆ

*x*

^{n}*ˆ*

^{k}*α**n**k*

∈ ∇*f(x*ˆ^{n}^{k}*)*+*∂g(ˆx*^{n}^{k}*).*

By letting*k*→ ∞, it is similar to the conclusion after (29) that 0∈ ∇*f(x)*¯ +*∂g(¯x)*
due to the demiclosedness of the subdifferentials, i.e.,*x*¯ ∈ *S*^{∗}. To proceed the proof
of (25) in this case, we observe from Lemma3.2that

*x*^{n}* ^{k}* − ˆ

*x*

^{n}*=*

^{k}*x*

^{n}*−*

^{k}*J(x*

^{n}

^{k}*,α*

*n*

*k*

*θ* *)*≥ *x*^{n}* ^{k}* −

*J(x*

^{n}

^{k}*, α*

*n*

_{k}*) = x*

^{n}*−*

^{k}*x*

^{n}

^{k}^{+}

^{1}. This together with (32) yield

*x*

^{n}*−*

^{k}*x*

^{n}

^{k}^{+}

^{1}

*α**n**k*

→0 as*k* → ∞. Since*(F(x*^{k}*))**k*∈Nis
decreasing, we derive from the latter and (27) that

0= lim

*k*→∞*M* *x*^{n}* ^{k}*−

*x*

^{n}

^{k}^{+}

^{1}

*α**n** _{k}* ≥ lim

*k*→∞*F(x*^{n}^{k}^{+}^{1}*)*−*F(x*^{∗}*)*= lim

*k*→∞*F(x*^{k}*)*−*F(x*^{∗}*)*≥0,
which ensures (25).

From the above two cases, we have (25) and the fact that any weak accumulation
point of the Fejér sequence*(x*^{k}*)**k*∈Nbelongs to*S*^{∗}. The classical Fejér theorem [4,

Theorem 5.5] tells us that*(x*^{k}*)**k*∈Nweakly converges to some point of*S*^{∗}. This verifies
(i) of the theorem.

The proof for the second part (ii) follows the same lines in the proof of [12, Theo-

rem 4.2].

The following proposition shows that when *f* is locally Lipschitz continuous, the
step size*α**k* is bounded below by a positive number. The second part of this result
coincides with [6, Remark 1.2].

**Proposition 3.2** (Boundedness from Below for the Step Sizes) *Let* *(x*^{k}*)**k*∈N *and*
*(α**k**)**k*∈N*be the two sequences generated from the FBS method. Suppose that S*^{∗}= ∅
*and that the sequence(x*^{k}*)**k*∈N*is converging to some x*^{∗} ∈ *S*^{∗}*. If*∇*f is locally Lip-*
*schitz continuous around x*^{∗} *with modulus L, then there exists some K* ∈ N*such*
*that*

*α**k*≥min

*α**K**,θ*
*L*

*>*0 *for all k>K.* (33)
*Consequently, for k* *>* *K the line search L S(x*^{k}*, α**k*−1*, θ)* *needs at most* log_{θ}

min

1*,*_{α}_{K}^{θ}_{L}*steps.*

*Furthermore, if*∇*f is globally Lipschitz continuous on*int(dom*f)*∩dom*g with*
*uniform modulus L thenα**k* ≥ min{σ,^{θ}* _{L}*}

*for all k*∈ N. In this case, line search

*L S(x*

^{k}*, α*

*k*−1

*, θ)needs at most*log

_{θ}min{1,_{σ}^{θ}* _{L}*}

*steps for any k.*

* Proof* To justify, suppose that

*S*

^{∗}= ∅, the sequence

*(x*

^{k}*)*

*k*∈Nis converging to

*x*

^{∗}∈

*S*

^{∗}, and that∇

*f*is locally Lipschitz continuous around

*x*

^{∗}with constant

*L*

*>*0. We find some

*ε >*0 such that

∇*f(x)*− ∇*f(y) ≤Lx*−*y* for all *x,y*∈B*ε**(x*^{∗}*),* (34)
whereB*ε**(x*^{∗}*)*is the closed ball in*H* with center*x*^{∗}and radius*ε*. Since*(x*^{k}*)**k*∈Nis
converging to*x*^{∗}, there exists some*K* ∈Nsuch that

*x** ^{k}*−

*x*

^{∗}≤

*θε*

2+*θ* *< ε* for all *k>K* (35)
with 0*< θ <*1 defined in**Linesearch BT. We claim that**

*α**k*≥min

*α**k*−1*,θ*
*L*

for all *k>K.* (36)

Suppose by contradiction that*α**k* *<*min{α*k*−1*,*^{θ}* _{L}*}. Then,

*α*

*k*

*< α*

*k*−1, and so, the loop in

**Linesearch BT**at

*(x*

^{k}*, α*

*k*−1

*)*needs more than one iteration. Define

*α*ˆ

*k*:=

^{α}

_{θ}

^{k}*>*0 and

*x*ˆ

*:=*

^{k}*J(x*

^{k}*,α*ˆ

*k*

*)*, we have

*f(x*ˆ^{k}*) >* *f(x*^{k}*)*+ ∇*f(x*^{k}*),x*ˆ* ^{k}*−

*x*

*+ 1*

^{k}2*α*ˆ*k*x* ^{k}*− ˆ

*x*

^{k}^{2}

*.*(37)

Furthermore, it follows from Lemma3.2that
x* ^{k}*− ˆ

*x*

*= x*

^{k}*−*

^{k}*J(x*

^{k}*,α*ˆ

*k*

*) ≤*

*α*ˆ

*k*

*α**k*x* ^{k}*−

*J(x*

^{k}*, α*

*k*

*) =*1

*θ*x* ^{k}*−

*x*

^{k}^{+}

^{1}. This together with (35) yields

ˆ*x** ^{k}*−

*x*

^{∗}≤ ˆ

*x*

*−*

^{k}*x*

*+*

^{k}*x*

*−*

^{k}*x*

^{∗}

≤ 1

*θ*x* ^{k}*−

*x*

^{k}^{+}

^{1}+ x

*−*

^{k}*x*

^{∗}

≤ 1
*θ* · 2θε

2+*θ* + *θε*

2+*θ* =*ε.* (38)

Since*x*^{k}*,x*ˆ* ^{k}*∈B

_{ε}*(x*

^{∗}

*)*by (35) and (38), we get from (34) that

*f(x*ˆ

^{k}*)*−

*f(x*

^{k}*)*− ∇

*f(x*

^{k}*),x*ˆ

*−*

^{k}*x*

^{k}= 1

0

∇*f(x** ^{k}*+

*t(x*ˆ

*−*

^{k}*x*

^{k}*))*− ∇

*f(x*

^{k}*),x*ˆ

*−*

^{k}*x*

*dt*

^{k}≤ 1

0

*t L ˆx** ^{k}*−

*x*

^{k}^{2}

*dt*=

*L*

2 ˆ*x** ^{k}*−

*x*

^{k}^{2}

*.*

Combining this with (37) yields*α*ˆ*k*≥ _{L}^{1} and thus*α**k* ≥ ^{θ}* _{L}*. This is a contradiction.

If there is some *H* *>* *K* with*H* ∈ Nsuch that*α**H* *>* ^{θ}* _{L}*, we get from (36) that

*α*

*k*≥

_{L}*for all*

^{θ}*k*≥

*H. Otherwise,*

*α*

*k*

*<*

^{θ}*for any*

_{L}*k*

*>*

*K*, which implies that

*α*

*k*=

*α*

*k*−1=

*α*

*K*for all

*k>K*due to (36) and the decreasing property of

*(α*

*k*

*)*

*k*∈N. In both cases, we have (33).

Now suppose that the line search*L S(x*^{k}*, α**k*−1*, θ)*needs*d* repetitions. It follows
from (33) that

*α**K**θ** ^{d}*≥

*α*

*k*=

*α*

*k*−1

*θ*

*≥min*

^{d}*α**K**,* *θ*
*L*

*,*
which tells us that*d* ≤log_{θ}

min

1*,*_{α}_{K}^{θ}* _{L}* .

Finally, suppose that ∇*f* is globally Lipschitz continuous with modulus *L* on
int*(*dom *f)∩*dom*g*subset of int*(*dom*f)*. By using Proposition3.1(iii), we can repeat
the above proof without concerning*ε,K* and replace (36) by*α**k*≥min

*σ,*^{θ}_{L}

.

Next, we present the conventional complexity *o(k*^{−}^{1}*)* of the FBS method [10,
Theorem 3] but under our standing assumptions and that ∇*f* is *locally*Lipschitz
continuous. The complexity remains valid when replacing the local Lipschitz condition
there by the weaker one that the step size{α* ^{k}*}is bounded below by a positive number;

see Proposition3.2. This idea was initiated in [12, Theorem 4.3] in finite dimensions and extended to different kinds of line searches in Hilbert spaces in [13], e.g., [13,

Corollary 3.20]. For the completeness, we provide a short proof^{2}, which could be
obtained by following the method of proof as in [13, Theorem 3.18(iii)(d)] as well.

**Theorem 3.2** (Sublinear Convergence of the FBS Method) *Let* *(x*^{k}*)**k*∈N *be the*
*sequence generated in the FBS method. Suppose that S*^{∗} = ∅*and that*∇*f is locally*
*Lipschitz continuous around any point in S*^{∗}*. Then,*

*k*lim→∞*k*

*F(x*^{k}*)*− min

*x*∈H *F(x)*

=0. (39)

* Proof* By Proposition3.2,

*α*

*k*is bounded below by some

*α >*0. Take any

*x*

^{∗}∈

*S*

^{∗}, we obtain from Corollary3.1(i) that

x* ^{k}*−

*x*

^{∗}

^{2}− x

^{k}^{+}

^{1}−

*x*

^{∗}

^{2}≥2α[F(x

^{k}^{+}

^{1}

*)*−

*F(x*

^{∗}

*)] ≥*0. (40) As

*x*

*−*

^{k}*x*

^{∗}

^{2}is decreasing,

*(x*

*−*

^{k}*x*

^{∗}

^{2}

*)*

*k*∈Nis converging. Given

*ε >*0, there exists

*K*∈N(large enough) such that|x

^{K}^{+}

*−*

^{k}*x*

^{∗}

^{2}− x

*−*

^{K}*x*

^{∗}

^{2}| ≤

*ε, for any*

*k*∈N. It follows from (40) that

*ε*≥ x* ^{K}*−

*x*

^{∗}

^{2}− x

^{k}^{+}

*−*

^{K}*x*

^{∗}

^{2}=

*k*+*K*−1

*=**K*

*(x** ^{}*−

*x*

^{∗}

^{2}− x

^{+}^{1}−

*x*

^{∗}

^{2}

*)*

≥2α

*k*+*K*−1

*=**K*

[F(x^{+}^{1}*)*−*F(x*^{∗}*)].*

Since*(F(x*^{k}*))*is decreasing, we get that
*ε*≥2αk(F*(x*^{K}^{+}^{k}*)*−*F(x*^{∗}*))*=2α *k*

*K*+*k(K*+*k)(F(x*^{K}^{+}^{k}*)*−*F(x*^{∗}*)),*
which implies that

0≤lim inf

*k*→∞ *k(F(x*^{k}*)*−*F(x*^{∗}*))*≤lim sup

*k*→∞ *k(F(x*^{k}*)*−*F(x*^{∗}*))*

=lim sup

*k*→∞ *(K*+*k)(F(x*^{K}^{+}^{k}*)*−*F(x*^{∗}*))*≤ *ε*
2α*.*

This verifies (39) and completes the proof of the theorem.

**4 Local Linear Convergence of Forward–Backward Splitting Methods**
In this section, we obtain the local Q-linear convergence for the FBS method under the
quadratic growth condition and local Lipschitz continuity of∇*f*, which is automatic
in many problems including Lasso problem and Poisson linear inverse regularized

2 We are grateful to one of the referees whose remarks lead us to this simple proof.