Studying Lagrangian bounds for a class of many-to-many assignment problems

Studying Lagrangian bounds for a class of many-to-many assignment problems

Igor S. Litvinchev, Jania Saucedo

Department of Mechanical and Electrical Engineering, UANL, Mexico email address : [email protected], [email protected]

Socorro Rangel (Corresponding author)

Department of Computer Science and Statistics, UNESP Rua Crist´ov˜ao Colombo, 2265

S.J. Rio Preto, SP, 15054-000, Brazil

email address : [email protected], Phone: + 55 17 32212201 December, 2007

Abstract

Lagrangian relaxation in continuous and integer optimization problems is frequently used to calculate bounds for the optimal objective. It is shown how these bounds can possibly be improved by estimating the complementarity term arising in the Lagrangian function. Small exemples are given to illustrate the procedure and results of a numerical study for a class of many-to-many assignment problems are presented.

KeywordsLagrangian bounds, integer programming, many-to-many assignment problems.

Introduction

Most large scale optimization problems exhibit a structure that can be exploited to construct efficient solution techniques.

In one of the most general and common forms of the structure the constraints of the problem can be divided into “easy”

and “complicating” (Lasdon, 2002). In other words, the problem would be an “easy” problem if the complicating constraints could be removed. One typical example is a block-separable problem decomposing into a number of smaller independent subproblems if the binding constraints could be relaxed .

A well-known way to exploit this structure is to form the Lagrangian relaxation with respect to complicating constraints.

That is, the complicating constraints are relaxed and a penalty term is added to the objective function to discourage their violation. Typically, the penalty is a linear combination of the associated slacks with coefficients called Lagrange multipliers.

The optimal value of the Lagrangian problem, considered for fixed multipliers, provides an upper bound (for maximization problem) for the original optimal objective. The problem of finding the best, i.e. bound minimizing Lagrange multipliers, is called the Lagrangian dual. Lagrangian bounds are widely used as a core of many numerical techniques, e.g. in branch- and-bounds schemes for integer and combinatorial problems. The bounds are also used in convex optimization techniques to measure the progress of the main algorithm and derive stopping criteria. The literature on Lagrangian relaxation is quite extensive. We refer only to a few pioneer and/or survey papers: Everett (1963), Held and Karp (1970), Geoffrion (1974), Guignard and Kim (1987), Shapiro (1974), Fisher (1985), Beasley (1993), Lemar´echal (2001), Guignard (2003), and Frangioni (2005).

There are often many ways in which a given problem can be relaxed in a Lagrangian fashion. For example, for a specific partition of variables some constraints can be considered as binding and hence be candidates to relax. On the other hand, the same restrictions can be treated as a block for another partition of variables. Such a double decomposable structure can be found in the generalized assignment problem, the multiple knapsack problem, and the facility location problem, to mention a few.

In this paper we concentrate on tightening the Lagrangian bounds by estimating the penalty term that arises in the Lagrangian problem. It is well known that under certain convexity and regularity conditions, the penalty turns to zero for the optimal primal-dual solution (complementarity condition). However, for nonconvex problems the complementarity condition is not necessarily fulfilled for the optimal primal-dual pair.

The remainder of the paper is organized as follows. Section 1 states the basic constructions to obtain the modified Lagrangian bounds. In section 2 resemblance with Lagrangian decomposition is discussed. Constructing localization used in the modified bound is considered in Section 3. Section 4 presents some small examples. In Section 5 the modified Lagrangian bound is applied to a class of many-to-many assignment problems focusing on the algorithmic aspects. Section 6 reports some numerical indications and Section 7 concludes.

1 Basic Constructions

Consider the problem:

(P) z^∗= max{cx|Dx≤d, x∈X},

where Dx ≤d are m “complicating” constraints, and x∈ X are “nice” ones in the sense that the optimization problem formed with only these constraints is easier than the original problem. The setX⊆Rⁿ can be of general structure. Denote byx^∗ an optimal solution of (P).

The convex relaxation to (P) is defined by:

(PR) zR= max{cx|Dx≤d, x∈conv(X)},

whereconv(X) is the convex hull of the set X.Since X ⊆conv(X), thenz^∗≤zR. LetxR be an optimal solution to (PR) anduR the optimal Lagrange multipliers associated to the complicating constraints.

Letu={ui} ≥0 be anm-vector of Lagrange multipliers. The Lagrangian problem is defined as:

z(u) = max

x∈X{cx+u(d−Dx)}.

We assume for simplicity thatz(u) has an optimal solution for all u≥ 0. Since x^∗ is feasible to the Lagrangian problem, then:

cx^∗+u(d−Dx^∗)≤z(u) for anyu≥0.

Sincex^∗ is feasible to (P) andu≥0, we haveu(d−Dx^∗)≥0 which results in the well known Lagrangian bound:

(LB) z^∗≤z(u) for anyu≥0.

The best Lagrangian bound and the associated Lagrange multipliersu^∗ are defined from the Lagrangian dual problem:

(D) z(u^∗) = min

u≥0z(u)≡wD

For convex programs (P) under certain regularity conditions we haveu^∗(d−Dx^∗) = 0, but even so,u(d−Dx^∗) can be strictly positive foru6=u^∗.IfX is nonconvex, the complementarity termu(d−Dx^∗) can be strictly positive even foru=u^∗. Thus we may try to strengthen the Lagrangian bound z(u) and the valuewD of the Lagrangian dual by estimating more tightly the complementarity term.

We assume a certain information about an optimal solution to (P),x^∗: Assumption. A setW ⊆Rⁿ is known, such thatx^∗∈W.

We will refer toW as the localization of x^∗,or simply thelocalization. The set W can be defined by manipulating the constraints of the original problem, by querying a decision maker, etc. We will distinguish the case ofa priori localization, when the setW is definedbeforethe Lagrangian problem has been solved for someu≥0 and the associated boundz(u) has been calculated, anda posteriori localization, whenW is defined or correctedafter the Lagrangian problem has been solved.

Denote byθ(u) the optimal value of the following auxiliary problem used to estimate the complementarity term:

(AP) θ(u) = min

y∈Wu(d−Dy), u≥0.

Proposition 1. LetW be a localization.Then:

(MLD) z^∗≤zM(u)≡z(u)−θ(u) for anyu≥0

provides a modified Lagrangian bound, and the best bound can be obtained from the modified Lagrangian dual:

(MD) wM D = min

u≥0zM(u).

Proof. By the definition ofz(u) and sincex^∗∈X it follows thatz^∗≤z(u)−u(d−Dx^∗).Then we have:

z^∗≤min

u≥0{z(u)−u(d−Dx^∗)} ≤max

y∈Wmin

u≥0{z(u)−u(d−Dy)}

≤min

u≥0max

y∈W{z(u)−u(d−Dy)}= min

u≥0{z(u)−min

y∈Wu(d−Dy)} ≡wM D

≤z(u)−min

y∈Wu(d−Dy)≡zM(u) for anyu≥0 which completes the proof. The second inequality in the sequence follows fromx^∗∈W.

Observation 1. From the proof of Proposition 1 it follows that the value:

b

wM D ≡maxy∈Wminu≥0{z(u)−u(d−Dy)}

also provides an upper bound forz^∗ which is at least as good aswM D.We will not considerwbM D here focusing on analysis ofwM D only.

Observation 2. Note that in generalwM D6=z(u^∗)−θ(u^∗),and the dual problems (D) and (MD) may result in different optimal Lagrange multipliers.

To understand the modified Lagrangian dual problem, we suppose for simplicity that either the setsX andW contain large but finite number of points{x1, ..., xT}and{y1, ..., yL},or bothX andW are bounded polyhedrons with their vertices.

Proposition 2. wM D= max{cx|Dx≤Dy, x∈conv(X), y∈conv(W)}.

Proof. By the definition ofw_{M D}, z_M(u) andz(u) we have:

wM D= minu≥0{maxx∈X{cx+u(d−Dx)} −miny∈W{u(d−Dy)}}

= minu≥0{maxt=1,..T{cxt+u(d−Dxt)} −minl=1,...L{u(d−Dyl)}}=

minη−ξ (MP)

η≥cxt+u(d−Dxt), t= 1, .., T ξ≤u(d−Dyl), l= 1, ..., L

u≥0, η, ξ∈R¹,

where variables η, ξ have been introduced to represent max and min with respect to x∈X and y ∈W respectively. The master problem (MP) is a linear program. Taking its dual gives:

wM D = maxXT

t=1λt(cxt) (DMP)

XL

l=1µl(d−Dyl)−XT

t=1λt(d−Dxt)≤0 X_T

t=1λt= 1, X_L

l=1µl= 1, λ, µ≥0.

Now settingx=P_T

t=1λtxt, y=P_L

l=1µlyl,withP_T

t=1λt= 1, P_L

l=1µl= 1, λ, µ≥0 we get:

wM D= maxcx Dx≤Dy

x∈conv(X), y∈conv(W) as desired.

Observation 3. It is known (Geoffrion, 1974) that the Lagrangian dual (LD) is equivalent to the primal relaxation:

wD= max{cx|Dx≤d, x∈conv(X)}.

LetX ={x≥0 |Ax≤b, x∈U},where U contains integrality restrictions on some or all the components of x. Suppose the Integrality Property is fulfilled, that isconv(X) ={x≥0 |Ax ≤b}. Then Lagrangian relaxation is equivalent to the LP-relaxation:

wD=zLP ≡max{cx|Dx≤d, Ax≤b, x≥0}.

It follows from Proposition 2 that for the modified dual we may have wM D < zLP even for the problem with Integrality Property.

The modified Lagrangian dual (MD) is a nonlinear programming problem with respect to u and can be solved by subgradient or dual ascent methods (Lemar´echal, 2001). We can also apply constraint or column generation techniques to the linear master program (MP) or its dual (DMP),respectively. For more techniques suitable to solve (MD) see Guignard (2003) and the references therein.

2 Resemblance with Lagrangian decomposition

The modified Lagrangian bound considered in the previous section can also be obtained through reformulation/relaxation of the original problem. ¹ Bearing in mind Proposition 2, consider the problem

(PM) wLA = max{cx|Dx≤Dy, x∈X, y∈W}.

Since a pair (x, y) = (x^∗, x^∗) is feasible to (PM), then (PM) is a relaxation of the original problem (P) and hencez^∗≤wLA. Moreover, if localizationW is such thaty∈W impliesDy≤d(for example, constraintsDy≤dare included explicitly in the definition ofW),then an optimal solution to (PM) is feasible to (P) andz^∗≥wLA.For such a localization we havez^∗=wLA 1We would like to thank an anonymous referee for pointing out this way to introduce the technique and the resemblance between the modified bound, presented in Section 1, and Lagrangian decomposition.

and (PM) is equivalent to the original problem.Thus (PM) is closely connected with the original problem, providing at least the upper bound for the optimal objectivez^∗.

Compute now the (best) Lagrangian bound corresponding to dualization of constraints Dx ≤ Dy. It is not hard to verify that the value of this dual bound coincides withwM D. Introducing in the original problem the “copy” (x =y) (or the agregated “copy”, Dx = Dy)) constraints and dualizing them is the approach used in Lagrangian decomposition (or aggregate Lagrangian decomposition) (see, e.g., Guignard (2003)). From this point of view the modified Lagrangian bound can be considered as a “relaxed” version of Lagrangian decomposition, since instead of equivalently transforming the original problem by introducing the copy constraints, the relaxed problem (PM) is considered.

On the other hand, Lagrangian decomposition can be interpreted in terms of the auxiliary problem,θ(u), used to reduce the duality gap. Let inequalitiesl≤T x≤rbe fulfilled for any feasible solution to the original problem. Here matrixT and column vectorsl, r, xare dimensioned accordingly. For the bounded original feasible set suchl, ralways exist for any matrix T. Thus we may add these “dummy” constraints to the original problem without changing its feasible set and then calculate the modified Lagrangian bound dualizing the “dummy” constraints:

z^∗= max{cx|Dx≤d, x∈X, l≤T x≤r} ≤

max{cx+α(r−T x) +β(T x−l)|Dx≤d, x∈X} −min{α(r−T y) +β(T y−l)|y∈W}= max{[c−(α−β)T]x|Dx≤d, x∈X}+ max{(α−β)T y|y∈W}=

(DC) max{(c−λT)x|Dx≤d, x∈X}+ max{λT y|y∈W} ≡zDC(λ),

where Lagrange multipliersα, β≥0, λ=α−β has no sign restrictions (DC stands for “dummy constraints”).

Let X = {x | Ax≤ b, x ∈U}, where U contains sign restrictions on xand integrality restrictions on some or all the components ofx. Relaxing constraintsAx≤bin the first optimization problem in (DC) and definingW ={Ay≤b, y∈U} in the second, we get

z^∗≤zDC(λ)≤max{(c−λT)x|Dx≤d, x∈U}+ max{λT y|Ay≤b, y∈U}.

If T = I (identity matrix), the above expression coincides with the Lagrangian decomposition bound, while an arbitrary matrixT results in the Lagrangian aggregation bound (Guignard (2003)). For T = A we may define W ={Ay =F(y), F(y) ≤ b, y ∈ U}, where F(y) is a vector-function dimensioned accordingly. Relaxing again constraints Ax ≤ b in the first optimization problem in (DC) and estimating the second maximum in (DC) by relaxing constraitsAy =F(y) (after substitutionF(y) in the objective) we get

z^∗≤zDC(λ)≤max{(c−λA)x|Dx≤d, x∈U}+ max{λF(y)|F(y)≤b, y ∈U}.

which coincides with the Lagrangian substitution bound (see Guignard (2003) and the references therein).

We see that the standard Lagrangian decomposition/substitution can be obtained from (DC) by estimating the optimal values of the subproblems in (DC) using constraints relaxation. However, instead of estimating by simply dropping constraints we may dualize them in the Lagrangian fashion or use the modified Lagrangian bound considered in Section 1.

Consider, for example, the case T =I and dualize constraints Ax≤bin the first subproblem in (DC) with multipliers u≥0. We have:

z^∗≤zDC(λ)≤zDC(λ, u)≡max{(c−λ)x+u(b−Ax)|Dx≤d, x∈U}+ max{λy|y∈W}.

It is also possible to estimate the strength of Lagrangian dual corresponding tozDC(λ, u) (similar to Proposition 2):

(DDC) z^∗≤ min

u≥0,λzDC(λ, u) = max{cx|Ax≤b, x∈conv{W} ∩conv{Dx≤d, x∈U}}.

Ifconv{W} ⊆ {Ax≤b},then constraints Ax≤b are redundant in (DDC). For example, forW ={Ax≤b, x∈U}we have z^∗≤ min

u≥0,λzDC(λ, u) = max{cx|x∈conv{Ax≤b, x∈U} ∩conv{Dx≤d, x∈U}},

which is exactly the dual bound corresponding to the standard Lagrangian decomposition (see, e.g., Guignard (2003)). As was shown earlier, Lagrangian decomposition bound can be obtained from (DDC) forT =I andW ={Ax≤b, x∈U}by simply dropping constraintsAx≤bin the first subproblem of (DDC). Thus, Lagrangian decomposition bound can not be improved by dualizing constraintsAx≤bin (DDC) instead of simply relaxing them. However, if the conditionconv{W} ⊆ {Ax≤b}

is not fulfilled (e.g.,W is defined by a linear combination of inequalitiesAx≤b), constraintsAx≤bmay be active in (DDC) and the boundzDC(λ) can be improved by dualizing them.

3 Localizations

One of the most critical issues in using the modified Lagrangian dual (MD) is constructing a suitable localizationW.From the definition of zM(u) and wM D it follows that, in general, the smaller the set W is, the better is the modified upper bound. That is, for example, defining W by the original constraints, it is worth to retain as many constraints as possible.

Alternatively, the localization should be simple enough to guarantee that the auxiliary problem (AP) is at least as easier to solve as the original problem (P). Moreover, to strengtheningwD, θ(u) should be positive at least for someu≥0.

Consider the original problem (P) withX ={x|Ax≤b, x∈U},whereU contains sign restrictions onxand integrality restrictions on some or all the components ofx. Define a localization by the original constraints as follows:

W0={y|Dy≤d, y∈U}.

Obviously, x^∗ ∈ W0, θ(u) = miny∈W0u(d−Dy) ≥ 0 for any u ≥ 0 and then by (MLD), (MD) we have wM D ≤ wM. Thus we may possibly strengthenwD by considering the modified Lagrangian dual (MLD). A single-constraint localization fW0={y|uDy≤ud, y∈U}has similar properties.

If the localization is decomposable, that isW0=W01×...W0L, the (AP) problem is reduced toLindependent subproblems of smaller dimensions. There are many classes of problems with both X and W0 having this structure and resulting in decomposition of both the Lagrangian problem to calculatez(u) and the auxiliary problem to calculateθ(u). In plane words, this is often the case for problems with “xij” variables: generalized assignment, facility location, multiple knapsack, cutting and packing problems,among others.

Instead of calculatingθ(u) exactly by solving the auxiliary problem, we may estimateθ(u) using as the auxiliary problem its standard Lagrangian relaxation or else by calculating the modified Lagrangian bound, presented in Section 1. To describe this “nested Lagrangian relaxation”, suppose that the localization has the formW1={y |y ∈Y, P y≤p},where Y ⊆Rⁿ, the matrix P and the vector pare dimensioned correspondingly. We assume that the set Y has a favorable structure (for example, decomposable) and will handle constraintsy∈Y explicitly, while restrictionsP y≤pwill be dualized. We have:

θ(u) = min

y∈Wu(d−Dy)≥min

y∈Y{u(d−Dy) +v(P y−p)} for anyv≥0, and hence:

θ(u)≥max

v≥0 min

y∈Y{u(d−Dy) +v(P y−p)}.

Then it follows that:

(MD(W1)) wM D(W1) = min

u≥0{z(u)−θ(u)} ≤min

u≥0{z(u)−max

v≥0min

y∈Y{u(d−Dy) +v(P y−p)}} ≤

u,v≥0min{z(u)−min

y∈Y{u(d−Dy) +v(P y−p)}} ≡weM D(W1) withz^∗≤weM D(W1).

It is easy to verify that similar to Proposition 2 we have:

e

wM D(W1) = max{cx|Dx≤Dy, P y≤p, x∈conv(X), y∈conv(Y)}.

In particular, forX ={x|Ax≤b, x∈U} we may defineWf1 using all original constraints, Y ={Dy ≤d, y∈U} and {P y≤p}={Ay≤b}. And to estimate the auxiliary problem,θ(u), we may either relax constraintsAy≤bor dualize them.

If we simply dropp these constraints, the strength of the corresponding modified boundwM D(Y) is given by:

wM D(Y) = max{cx|Dx≤Dy, x∈conv(Ax≤b, x∈U), y∈conv(Dy≤d, y∈U)}.

Dualizing constraintsAy≤bwe get:

e

wM D(Wf1) = max{cx|Dx≤Dy, Ay≤b, x∈conv(Ax≤b, x∈U), y∈conv(Dy≤d, y∈U)}.

If in the latter problem constraintsAy≤b are active, then we may haveweM D(Wf1)< wM D(Y) (see Example 1 in Section 4). It is not hard to verify thatwM D(Y) coincides with the aggregate Lagrangian decomposition boundwLDA:

wLDA= min

u [max{(c−uD)x|Ax≤b, x∈U}+ max{uDy|Dy≤d, y∈U}]

and thus it may be possible to improve this bound.

By the definition ofθ(u) in (AP) and since x^∗ ∈W we haveθ(u)≤u(d−Dx^∗) for anyu≥0, and hence:

(LFC) uDx≤ud−θ(u)

defines an u-family of cuts preserving the original optimal solution. We will refer to these as Lagrangian feasible cuts.

Consider the separation problem for the family (LFC). Explicitly, we are given a point bx and wish to know whether it satisfies all (LFC).

For a givenxbthe Lagrangian dual separation problem is stated as follows:

(LDS) wLDS(bx) = min

u≥0{u(d−Dbx)−θ(u)}

and bxis eliminated by the corresponding (LFC) iffw_LDS(bx) <0. By the definition ofθ(u) we have θ(u) ≤u(d−Dx) for anyx∈W andu≥0,and thusx∈W will never be cut off by (LFC). Note that by complementarityuR(d−DxR) = 0 and hence ifθ(u_R)>0,thenx_R is cut off by (LFC) associated tou=u_R.

In the separation problem (LDS) we are interested mainly in the sign ofwLDS(bx).Sinceθ(u) is homogenous with respect tou,so iswLDS(bx),and hence without loss of generality we can normalizeu.For example, by adding the restrictioneu= 1, where e is the m-dimensional unit vector. Similar to the proof of the Proposition 2, it can be demonstrated that for the normalized problem (LDS) we have:

wLDS(bx) = max{t|te≤D(x−bx), x∈conv(W), t∈R¹}.

4 Examples

Example 1. Consider the following binary problem, Freville and Hanafi (2005):

x1+ 2x2+ x3+ 2x4 → max 8x1+ 16x2+ 3x3+ 6x4 ≤ 18 5x1+ 10x2+ 8x3+ 16x4 ≤ 19

x∈ {0,1}.

Its optimal solution isz^∗= 2, x^∗={(0,1,0,0),(0,0,0,1),(1,0,1,0}.The linear programming relaxation giveszLP = 3.04 with the associated optimal primal and dual solutionx_LP = (0,0.89,0,0.63), u_R= (0.061,0.102).

DefiningX={x∈ {0,1} |5x1+ 10x2+ 8x3+ 16x4≤19}and dualizing the first restriction we get:

z(u) = 18u+ max

x∈X{(1−8u)x1+ (2−16u)x2+ (1−3u)x3+ (2−6u)x4} such that:

z(u) =













18u, u≥ ¹₃ 2 + 12u, ₁₃¹ ≤u < ¹₃

3−u, 0≤u < ₁₃¹













and:

wD= min

u≥0z(u) =z(1

13) = 212

13 ≈2.923.

Consider now the modified Lagrangian dual problem (MD) forW0={x∈ {0,1} |8x1+ 16x2+ 3x3+ 6x4≤18}:

wM D(W0) = min

u≥0{z(u)−θ(u)}, θ(u) = min

y∈W0

{u(18−8y1−16y2−3y3−6y4)}.

We haveθ(u) =u,such that:

wM D(W0) = min

u≥0{z(u)−u}=z(1 13)− 1

13 = 211

13 ≈2.846< wD.

It is not hard to verify that for the bi-knapsack problem wM D(W0) coincides with aggregate Lagrangian decomposition boundwLDA (see also Example 7.4 in Guignard (2003)).

Suppose now that the localizationW1 is defined by the two original constraints, such that the first restriction is used to define the conditiony∈Y and is handled explicitly, while the second is dualized in the estimation ofθ(u). Then we have:

θ(u) = min

y∈W1

{u(18−8y1−16y2−3y3−6y4)}=u(18−max

y∈W1

{8y1+ 16y2+ 3y3+ 6y4}).

Using standard Lagrange technique to estimate the maximum in the latter expression, we get:

z= max

y∈W1

{8y₁+ 16y₂+ 3y₃+ 6y₄} ≤min

v≥0max

y∈Y{8y₁+ 16y₂+ 3y₃+ 6y₄−v(5y₁+ 10y₂+ 8y₃+ 16y₄−19)}= minv≥0{19v+ max{(8−5v)y1+ (16−10v)y2+ (3−8v)y3+ (6−16v)y4|y∈ {0,1},8y1+ 16y2+ 3y3+ 6y4≤18}.

Denoting the inner maximum byϕ(v) we obtain:

ϕ(v) =













19v, u≥ ⁸₅ 16 + 9v, ₁₉¹ ≤u < ⁸₅ 17−10v, 0≤u < ₁₉¹













with minv≥0ϕ(v) =ϕ(₁₉¹) = 16₁₉⁹ ≈16.47.Respectively, we haveθ(u)≥u(18−minv≥0ϕ(v)) = 1¹⁰₁₉u≈1.53uand:

z^∗≤min

u≥0{z(u)−θ(u)} ≤weM D(W1) = min

u≥0{z(u)−110

19u}= 2199

247 ≈2.806< wM D, where the minimum is attained foru= 1/13.

Now estimateθ(u) using the modified Lagrangian bound. As before, we need to estimatez.We havez≤minv≥0{z(v)− θ(v)},where

z(v) = 19v+ max{(8−5v)y1+ 2(8−5v)y2+ (3−8v)y3+ 2(3−8v)y4|y∈ {0,1},8y1+ 16y2+ 3y3+ 6y4≤18}

and

θ(v) = min{v(19−5y1−10y2−8y3−16y4|5y1+ 10y2+ 8y3+ 16y4≤19, y∈ {0,1}}=v.

We have

z(v)−θ(v) =













18v, u≥ ⁸₅ 16 + 8v, ₁₉¹ ≤u < ⁸₅ 17−11v, 0≤u < ₁₉¹













with the minimal value 16₁₉⁸ attained forv= 1/19.Thenθ(u)≥u(18−16₁₉⁸) and z^∗≤min

u≥0{z(u)−θ(u)} ≤min

u≥0{z(u)−111

19u}= 2198

247 ≈2.802<weM D(W1).

Example 2. Consider the following nonlinear problem:

z^∗= max{f(x)|Ax=b, x≥0}

with concavef(x). DefineX ={x|Ax=b}and W ={y|Aye =eb, y≥0}, where matrix [A,e eb] contains a subset of rows of the extended matrix [A, b].

Dualizing nonnegativity restrictionsx≥0, the Lagrangian bound calculated for a fixedu≥0 is:

z(u) = max{f(x)−ux|Ax=b}, while the modified Lagrangian boundzM(u) is defined by:

zM(u) =z(u)−θ(u), θ(u) = min{yu|Aye =eb, y≥0}.

In particular, if uis strictly positive and the matrix [A,eeb] contains at least one row with positive elements, then θ(u)>0 andz_M(u)< z(u).Note that strictly positive multipliers are frequently used in interior point techniques (see, e.g., Terlaky, 1996 and the references therein).

5 Many-to-many assignment problems

In this section we consider a class of generalized assignment problems (GAP) which can be characterized as a many-to-many assignment and has a double-decomposable structure. We specify basic steps of the proposed scheme to tightening Lagrangian bound focusing in the algorithmic aspects.

Assignment problems (AP) involve optimally matching the elements of two or more sets, where the dimension of the problem refers to the number of sets of elements to be matched. When there are only two sets, they may be referred as

“tasks” and “agents”. Thus, for example, “tasks” may be jobs to be done and “agents” the people or machines that can do them. In its original version, the AP involves assigning each task to a different agent, with each agent being assigned to at most one task (a one-to-one assignment). In the generalized assignment problem (GAP) each task is assigned to one agent, as in the classic AP, but it allows for the possibility that an agent may be assigned more than one task, while recognizing that a task may use only part of an agent’s capacity rather than all of it. Thus GAP is an one-to-many assignment problem that recognizes capacity limits (see Martelo and Toth (1990), Pentico ( 2007) and the references therein). We consider a further generalization of AP which is a many-to-many assignment recognizing capacity limits of both tasks and agents. Such a situation arises, for example, in a medical center, where doctors (agents) have to attend their patients (tasks) in a limited time period, while patients can’t also spend a lot time in the center. This leads to the following model:

(MMAP) zip = max

Xm

i=1

Xn

j=1

cijxij

s.t.

Xn

j=1

aijxij ≤bi i= 1,2..., m, Xm

i=1

dijxij ≤dj j= 1,2..., n, xij ∈ {0,1},

where xij = 1 if agent i is assigned to taskj, 0 otherwise, cij = is the profit (utility) of assigning agent i to task j, aij is the amount of agenti’s capacity used to execute taskj,andbi is the available capacity of agenti.We also assume that each task has its own capacity (time) limit, such thatdij is the amount of taskj’s capacity used when executed by agenti, and dj is the available capacity of taskj.

Note that (MMAP) has a double-decomposable structure: if we dualize the firstmrestrictions, then the relaxed problem decomposes intonindependent subproblems, each having a single knapsack-type restrictionP_m

i=1dijxij ≤dj,while relaxing the second group of restrictions we getmsingle knapsack constrained subproblems. Let the second group of inequalities be treated as complicating. Define:

X ={xij ∈ {0,1} | Xn

j=1

aijxij≤bi i= 1,2..., m}= Ym

i=1

Xi

Xi ={xij ∈ {0,1} | Xn

j=1

aijxij ≤bi} W1={yij∈ {0,1} |y∈Y, P y≤p}

{P y≤p} ≡ { Xn

j=1

aijyij≤bi i= 1,2..., m}

Y ={yij ∈ {0,1} | Xm

i=1

dijyij ≤dj j = 1,2..., n}= Yn

j=1

Yj.

Yj={yij ∈ {0,1} | Xm

i=1

dijyij ≤dj}

Original restrictions included inX will be considered as “easy”, while those inY be treated as “complicating”. Localization W1 will be used in the modified Lagrangian dual (MD(W1)) to calculate the value weM D(W1). We will handle constraints y∈Y explicitly, while restrictions P y≤pwill be dualized in the auxiliary problem. Letu={uj, j = 1,2...n} andv={vi, i= 1,2...m}be the Lagrange multipliers. Similar to section 3 we obtainz^∗≤weM D(W1) with:

(MD(W1)) weM D(W1) = min

u,v≥0{z(u)−θ(u, v)}, where for the (MMAP):

z(u) =X

j

ujdj+ max

x∈X{X

i

X

j

(cij−ujdij)xij} θ(u, v) =X

j

ujdj−X

i

vibi+ min

y∈Y{X

j

X

i

(viaij−ujdij)yij}

The modified Lagrangian dual (MD(W1)) is a nonlinear programming problem with respect to variablesu, v≥0 and can be solved, for example, by subgradient or dual ascent techniques. In Section 6 we consider two approaches for solving the dual problem: Benders and subgradient methods. Constraint generation scheme (Benders method) transforms (MD(W1)) to a large scale linear programming problem. Benders method was used because it generates upper and lower bounds forweM D(W1) at each iteration thus producing near-optimal solutions with guaranteed quality. The subgradient method is typically much faster than constraint generation scheme and it is interesting to compare the bounds obtained within a limited computational time.

Consider first the constraint generation scheme. Let{x^t_ij, t= 1,2...T} and{y^l_ij, l= 1,2...L} be all feasible points ofX andY. ThenweM D(W1) can be stated as:

(MP1) we_{M D}(W₁) = minη−ξ

(1) η≥X

j

ujdj+X

i

X

j

(cij−ujdij)x^t_ij, t= 1,2...T,

(2) ξ≤X

j

ujdj−X

i

vibi+X

j

X

i

(viaij−ujdij)y^l_ij, l= 1,2...L,

u, v ≥0, ξj, ηi∈R¹

The latter is a LP problem having (2 +m+n) variables and a large number of constraints - one for each feasible point of X, Y. To solve the master problem (MP1) we use constraint generation scheme in the form of Benders algorithm. We omit here the complete description of this well known iterative method (see Lasdon, 2002, and Martin, 1999 for details) and focus only on the constraint generation scheme.

Consider that on thekth iteration we have a restricted master problem (RMP1k), having fewer constraints (1) and (2) when compared to (MP1). Denote its optimal solution by (u, v, ξ, η)^k.To check the feasibility of this solution to all constraints (1) we need to verify if:

η^k−X

j

u^k_jdj≥X

i

X

j

(cij−u^k_jdij)x^t_ij for allt= 1,2...T or equivalently:

(3) η^k−X

j

u^k_jdj ≥max

x∈X

X

i

X

j

(cij−u^k_jdij)xij ≡X

i x∈Xmaxi

X

j

(cij−u^k_jdij)xij

where maximization overX is reduced to independent maximizations overXi due to decomposable structure ofX.That is to verify (3) we need to solvemindependent subproblems each one having a single knapsack constraint andnbinary variables.

Denote byx^k_ij their optimal solution.

Similarly, to check feasibility with respect to constraints (2) we need to verify:

(4) ξ^k−X

j

u^k_jdj+X

i

v^k_ibi≤min

y∈Y

X

j

X

i

(v_i^kaij−u^k_jdij)yij ≡X

j y∈Yminj

X

i

(v^k_iaij−u^k_jdij)yij,

which results in solvingn independent subproblems with a single knapsack constraint and m binary variables. Let y_ij^k be their optimal solution.

If (3) and (4) are fulfilled, stop with (u, v, ξ, η)^k optimal to (MP1). If (3) fails, add:

η≥X

j

ujdj+X

i

X

j

(cij−ujdij)x^k_ij

to the restricted master problem. If (4) fails, add:

ξ≤X

j

ujdj−X

i

vibi+X

j

X

i

(viaij−ujdij)y_ij^k

to the restricted master problem. So in each iteration we add at most two constraints to (RMP1k) to get the next restricted master problem (RMP1k+ 1).

Note that on thekth iteration of Benders technique we have a lower and an upper bound for the optimal valueweM D(W1) of (MP1):

(5) η^k−ξ^k ≤weM D(W1)≤ min

s=1,2..k{z(u^s)−θ(u^s, v^s)},

where the minimum is taken over all previous iterations and for iterationswe have:

z(u^s) =X

j

u^s_jdj+X

i

X

j

(cij−u^s_jdij)x^s_ij,

θ(u^s, v^s) =X

j

u^s_jdj−X

i

v_i^sbi+X

j

X

i

(v^s_iaij−u^s_jdij)y^s_ij. So we can stop if the difference between the bounds in (5) is smaller than a certain threshold.

Note also that if η^k−ξ^k > η^k−1−ξ^k−1, we can eliminate those constraints (1), (2) which are not active in the optimal solution (u, v, ξ, η)^k (see Lasdon, 2002 for details).

Assume that the objective coefficients in (MMAP) are nonnegative and hence zip ≥ 0. Then for optimal solution to (MP1), (u, v, ξ, η)^∗,we have:

0≤zip ≤weM D(W1) =η^∗−ξ^∗, η^∗=z(u^∗)≥zip≥0.

Thus we can add to the master problem (MP1) trivial restrictionsη−ξ≥0, η≥0 from the very beginning to avoid unlimited decrease of the objective on the early iterations of Benders algorithm.

Observation 4. We can transform the dual problem (MD(W1)) to a slightly different master problem if it is noticed that:

z(u) =X

j

ujdj+X

i x∈Xmaxi

{X

j

(cij−ujdij)xij} and

θ(u, v) =X

j

ujdj−X

i

vibi+X

j y∈Yminj

{X

i

(viaij−ujdij)yij}

due to decomposable structure ofX andY.Let{x^t_ij, t= 1,2...Ti}and{y_ij^l , l= 1,2...Lj}be all feasible points ofXi andYj. Then (MD(W1)) can be transformed equivalently to the following master problem:

e

wM D(W1) = minX

i

vibi+X

i

ηi−X

j

ξj

ηi≥X

j

(cij−ujdij)x^t_ij, t= 1,2...Ti, i= 1,2...m, ξj ≤X

i

(viaij−ujdij)y^l_ij, l= 1,2...Lj, j= 1,2...n,

The latter is a LP problem having 2(m+n) variables and one restriction for each feasible point of Xi, Yj. It also can be solved by Benders technique. It is not hard to verify that checking overall feasibility of the restricted master problem solution results in the same subproblems as before. However, in contrast to the previous form of the master problem, here we can generate more restrictions in each iteration, at mostm+n. We don’t consider further this form of the master problem leaving this interesting topic for a future research.

Benders technique provides (near) optimal solution with guaranteed quality. However, its convergence is rather slow.

Another popular approach to solve the dual problem is subgradient optimization, first used in the Lagrangian context by Held and Karp (1970). Here we present only the basic steps of the subgradient method applied to calculateweM D(W1) for

(MMAP), the standard Lagrangian bound can be calculated in a similar way. A more detailed discussion of subgradient optimization can be found for example in Guignard (2003) and Martin (1999).

After some algebraic transformations we getweM D(W1) = minu,v≥0ϕ(u, v),where:

ϕ(u, v) =X

ivibi+X

imax

x∈Xi

{X

j(cij−ujdij)xij}+X

jmax

y∈Yj

{X

i(ujdij−viaij)yij}.

Let u^k, v^k be the values of the multipliers obtained in the kth iteration, ϕ^k = ϕ(u^k, v^k), and x^k_ij, y_ij^k be the associated subproblems solutions:

x^k_ij= arg max

x∈Xi

{X

j(cij−u^k_jdij)xij}, y^k_ij= arg max

y∈Yj

{X

i(u^k_jd_ij−v_i^ka_ij)y_ij. A sudgradient is directly identified after solving the associated subproblems as:

α^k_i = [∂ϕ/∂vi]^k=bi−X

jaijy_ij^k, β_j^k = [∂ϕ/∂uj]^k =−X

idijx^k_ij+X

idijy_ij^k. Denote bys^k a vector composed of all{α^k_i, β_j^k} and letλ^k ={v^k, u^k}, and let:

¯λ^k+1=λ^k−εk(ϕ^k−ϕlb) s^k ks^kk²,

where εk ∈ (0,2], ϕlb is a lower bound on ϕ^∗ = weM D(W1). Since zip ≤weM D(W1), we may setϕlb equal to the objective function value of (MMAP) associated to a given feasible solution.

The multipliers for the next iteration are defined as the projection of ¯λ^k+1 onto the nonnegative orthant (λ must be nonnegative):

λ^k+1= max{0; ¯λ^k+1}where max is taken componentwise.

The method is not monotone. That is, it is not necessary that ϕ^k ≥ϕ^k+1.In practice, the parameter εk is modified in the interval (0,2] as follows. Begin withε_k = 2. If afterK consequtive iterations with a fixed value forε_k the functionϕis not improved “sufficiently”, then a smaller value ofεk is used,say,εk =εk/2.

6 Numerical indications

The objective of the numerical study is to compare the relative quality of the bounds, their proximity to the optimal objective as well as constraint violations for the Lagrangian solutions generated by different approaches. We numerically compare the Lagrangian bounds for two sets of instances of (MMAP): small instances with sizesm×nform∈ {5,8,10}andn= 50, and large instances withm∈ {5,10,20}andn= 100.The data were random integers generated as follows:

cij∈U[10,50], aij∈U[5,25], dij ∈U[3,20]

with:

bi=α(X

j

aij−1), dj=α(X

j

dij−1), 0< α≤1

and divided in three classes (a,b, andc) with respect to values ofα: a(α= 1),b(α= 0.9), c(α= 0.8) (Martelo and Toth, 1990).

Both Lagrangian bounds, classical and modified, were calculated by the the two approaches presented in Section 5. Note that to compute the classical bound by the Benders method we may set ξ = 0, and ignore restrictions (2) in the master problem (MP1). The algorithm terminates if the difference between bounds (5) is smaller than 1. In the subgradient method we usedK= 5 and if (ϕ^k−ϕ^k+1)/ϕmax< δ(= 0.005) for 5 consecutive iterations with fixedεk, this parameter was modified toεk =εk/2.Hereϕmax is the best current objective value. The optimization models and the algorithms were coded in the sintaxe of the modeling language AMPL (Fourer et. al., 1993). All the optimization subproblems associated to both Benders and subgradient algorithms were solved by the system CPLEX 10.0 ( ILOG, 2001). The runs were executed on a Pentium 4, 3.2GHz, 2GB RAM.

For all problem instances we have calculated:

zip - optimal objective of the original integer problem, zlp - optimal objective of the LP relaxation,

zlag - classical Lagrangian boundwD(W1), zmd - modified Lagrangian boundweM D(W1) thus obtaining three upper bounds forzip.

The relative quality of the bounds was measured by:

rel1 = zmd−zip

z_lag−z_ip100%, rel2 = zmd−zip

z_lp−z_ip 100%, andrel3 = zlag−zip

z_lp−z_ip 100%,

where rel1 represents improvement of the modified bound over classical, rel2 shows the strength of the modified bound over LP relaxation, andrel3 compares the quality of classical bound with LP-bound. The proximity to the optimal integer solution was represented by:

gap1 = zmd−zip

zip 100%, gap2 = zlag−zip

zip 100% andgap3 = zlp−zip

zip 100%

The Benders results for 9 small instances are reported in Table 1, while Table 2 presents Benders results for the large instances. For each problem instance it is shown the values ofm,n, its class (a,b, or c), the relative quality of the bounds (rel1, rel2, rel3), the proximity to the optimal integer solution (gap1, gap2, gap3), and the iterations number to compute the classical and the modified bounds (iter.(zlag) and iter.( zmd) respectively).

For all problem instances the Lagrangian modified bound is stronger than the classical one (rel2 < rel3), and for some clusters of data - significantly stronger (instances classa). The improvement for problems in classc is smaller than for the other classes. In the cases at whichzlp is closer to zip (small values for gap3) there is little scope for bound improvement.

That is the case for the instance 20×100b which has gap3 = 0.78 and gap1 = 99.98 (See last line of Table 2). We have generated also instance 20×100c. For that problem we were not able to find the optimal solution. The best integer solution found after examining 316,560 nodes in the branch and cut tree hadgap3 = 0,61%.