Optimal Control of a SEIR Model with Mixed
Constraints and L
1Cost
Maria do Ros´ario de Pinho1, Igor Kornienko1, and Helmut Maurer2
1
University of Porto, Faculdade de Engenharia, ISR, Porto, Portugal, mrpinho,[email protected]
2 Institut f¨ur Numerische und Angewandte Mathematik, Universit¨at M¨unster,
M¨unster, Germany, [email protected]
Abstract. Optimal control can help to determine vaccination policies for infectious diseases. For diseases transmitted horizontally, SEIR com-partment models have been used. Most of the literature on SEIR models deals with cost functions that are quadratic with respect to the control variable, the rate of vaccination. Here, we propose the introduction of a cost of L1 type which is linear with respect to the control variable.
Our starting point is the recent work [1], where the number of vaccines at each time is assumed to be limited. This yields an optimal control problem with a mixed control-state constraint. We discuss the necessary optimality conditions of the Maximum Principle and present numerical solutions that precisely satisfy the necessary conditions.
Keywords: optimal control, epidemiology, mixed constraints, numeri-cal solutions, bang-bang control
1
Introduction
SEIR models are widely used to model the spreading of an infectious diseases in a certain population. This models divide the individuals in four compartments relevant to the epidemic. An individual can be in the S compartment if vulner-able (or susceptible) to the disease. Those infected but not vulner-able to transmit it are in the E compartment, exposed. Infected individuals capable of spreading the disease are in the I compartment and those who are immune are in the R compartment. In SEIR models everyone is assumed to be susceptible to the disease by birth and the disease is transmitted to the individual by horizontal incidence, i.e., a susceptible individual becomes infected when in contact with infectious individuals.
Let S(t), E(t), I(t), and R(t) denote the number of individuals in the sus-ceptible, exposed, infectious and recovered compartments at time t respectively. The total population is N (t) = S(t)+E(t)+I(t)+R(t). The disease transmission in a certain population is described by the parameters, e, the rate at which the exposed individuals become infectious, g the rate at which infectious individuals recover and a denotes the death rate due to the disease. Also b is the natural
birth rate and d denotes the natural death rate. Let c be the incidence coeffi-cient of horizontal transmission. Then the rate of transmission of the disease is cS(t)I(t). For simplicity the parameters are assumed constants (although they may vary in reality if the time horizon is big). For more information about such model we refer the reader to [2], [9], [13] and references within.
Based on such models, different vaccines policies have been studied based on optimal control. Such scientific exercise are of interest to compare different vaccination scenarios. In [13] and [1] different policies are confronted. However, in both such works, the cost of the optimal control problems considered is defined by an integrand quadratic with respect to control. A special feature of [1] is that a somewhat realist scenario of limited supplies of vaccine at each instant is considered. From the point of view of optimal control, this is translated by the introduction of a mixed constraint (also known as state dependent control constraint). Here, we assume that the vaccine is effective so that all vaccinated susceptible individuals become immune. Let u(t) represent the percentage of susceptible individuals being vaccinated per unit of time. Taking all the above considerations into account we are led to the following dynamical system:
˙
S(t) = bN (t) − dS(t) − cS(t)I(t) − u(t)S(t) (1) ˙
E(t) = cS(t)I(t) − (e + d)E(t) (2)
˙
I(t) = eE(t) − (g + a + d)I(t) (3)
˙
R(t) = gI(t) − dR(t) + u(t)S(t) (4)
˙
N (t) = (b − d)N (t) − aI(t) (5)
with the initial conditions
S(0) = S0, E(0) = E0, I(0) = I0, R(0) = R0, N (0) = N0. (6)
To keep track of the number of vaccinated individuals we introduce an extra variable W that satsifies the equation
˙
W (t) = u(t)S(t), W (0) = 0. (7)
Like other models in epidemiology, SEIR models represent only a rough approx-imation of reality. However, they can be of use to provide new insights into the spreading of diseases and, when optimal control is applied, to propose better vaccination policies. In this paper, we are not treating a particular disease in a specific population. Rather, we want to illustrate how different optimal control formulations can be used to propose new vaccination policies, when different scenarios and cost functionals are considered.
Here, our stepping stone is the paper by Biswas, Paiva, de Pinho [1] who study situations, where the number of vaccines is limited or the capability of vaccinating is limited at each time. In [1], the overall limit of vaccines W (T ) ≤ WM considered in [13] is replaced by a mixed control-state constraint of the
form:
where V0is an upper bound on vaccines available at each instant t. The inequality
(8) is also known in the literature as state dependent control constraint. The constraint W (T ) ≤ WM in [13] should be satisfied only at the end terminal time
T , while the mixed constraint (8) is to be satisfied at almost every instant of time during the whole vaccination program.
Both papers [1] and [13] consider a control quadratic cost functional of L2–
type. It has been argued that a control quadratic cost is not appropriate for problems with a biological or biomedical background. Therefore, in our work we consider a L1 cost functional that is linear with respect to the control variable u. Since the control variable appears linearly in the dynamics and constraints as well, the (augmented) Hamiltonian function is linear with respect to the control variable. The evaluation of the necessary optimality condition of the Maximum Principle shows that any optimal control must be a concatenation of bang-bang and singular arcs. Here, the term ”bang-bang” or ”singular” refers to the mixed constraint itself which will be made clear in section 3. In this paper, we do not further discuss singular controls, since our numerical solutions furnish only bang-bang controls. We obtain a numerical solution of our problem by applying direct optimization method, i.e., we discretize the control problem and use nonlinear programming methods. Then we employ the Imperial College London Optimal Control Software – ICLOCS – version 0.1b ([6]). This is an optimal control inter-face, implemented in Matlab, that calls the IPOPT – Interior Point OPTimizer – an open-source software package for large-scale nonlinear optimization. For a study of different optimal control solvers see [14]. As for information on IPOPT we refer the reader to [16]. Alternatively, we use the Applied Modeling Program-ming Language AMPL [7] which can be interfaced to IPOPT and a number of other optimization solvers.
Although we do not show that the numerical solution is indeed a (local) optimum, we do however validate our findings. Using the Lagrange multipliers provided by the optimization solver, we can validate our numerical solution by showing that it satisfies precisely the necessary condition of optimality.
2
The Optimal Control Problem with Mixed Constraints
We consider the following optimal control problem with a mixed control-state constraint: (P1) Minimize J1(x, u) = Z T 0 (AI(t) + Bu(t)) dt subject to ˙ S(t) = bN (t) − dS(t) − cS(t)I(t) − u(t)S(t), ˙
E(t) = cS(t)I(t) − (e + d)E(t), ˙
I(t) = eE(t) − (g + a + d)I(t), ˙
N (t) = (b − d)N (t) − aI(t), u(t)S(t) ≤ V0,
u(t) ∈ [0, 1] for a.e. t ∈ [0, T ],
We have removed the equation (4) for R, since the number of recovered individ-uals is given by R(t) = N (t) − S(t) + E(t) + I(t), and the equation (7) for W which is redundant for the control problem. Thus the state variable is
x = (S, E, I, N ) ∈ R4. Observe that our cost is now the L1 cost
J1(x, u) =
Z T
0
(AI(t) + Bu(t)) dt
in contrast to [13] and [1], where the control quadratic cost
J2(x, u) =
Z T
0
A2I(t) + B2u2(t) dt
is considered (here A, B, A1and B1are given constants). The convexity of J2(u)
with respect to u is advantageous for the numerical approach, since it allows to express the control variable by the state and adjoint variable. In both cases, the cost functional is a weighted sum of the overall cost of caring for the infected individuals and the cost of vaccination. Observe, however, that in the case of J2(x, u) the cost of vaccination will depend on u2, a small quantity compared to
u which takes values less than 1. In this respect, J1(x, u) is a more realistic cost
functional.
To simplify the analysis of the necessary optimality conditions of the control problem (P1), it is convenient to rewrite it in the form of a general optimal
control problem with a mixed control-state constraint:
(Pmixed) Minimize Z T 0 L(x(t), u(t)) dt subject to ˙ x(t) = f (x(t)) + g(x(t))u(t) a.e. t, m(x(t), u(t)) ≤ 0 a.e. t, u(t) ∈ [0, 1] a.e. t, x(0) = x0, x(T ) ∈ Rn, where x = (S, E, I, N ), L(x, u) = AI + Bu = L1(x) + L2(u), f (x) = f1(x) + A1x, f1(x) = c(−SI, SI, 0, 0)T, g(x) = (−S, 0, 0, 0)T, m(x, u) = uS − V 0, and A1= −d 0 0 b 0 −(e + d) 0 0 0 e −(g + a + d) 0 0 0 −a b − d .
The initial condition x0and parameters will be specified in Table 1. The
differen-tial equation ˙x(t) = f (x(t)) + g(x(t))u(t) is affine in the control and is nonlinear in the state x due to the term f1(x). Note that the mixed control-state constraint
satisfies the standard regularity condition
mx(x(t), u(t)) = S(t) 6= 0 ∀ t ∈ [0, T ] with u(t)S(t) = V0. (9)
Here and in the following, partial derivatives are denoted by subscripts. More-over, appealing to [4], for example, it is an easy task to prove the existence of a solution (x∗, u∗) to problem (Pmixed)).
3
Necessary Optimality Conditions for (P
1)
Let (x∗, u∗) be a minimizer for our problem (P1) (or (Pmixed)). In the following,
we shall evaluate the necessary optimality condition of the Maximum Principle. Since we are maximizing −J1(x, u), the standard Hamiltonian function is given
by
H(x, p, u) = −λL(x, u) + hp, f (x) + g(x)ui, λ ∈ R,
where p = (pS, pE, pI, pN) ∈ R4 denotes the adjoint variable. In the augmented
Hamiltonian, the mixed constraint m(x, u) ≤ 0 is adjoined by a multiplier q ∈ R to the Hamiltonian:
H(x, p, q, u) = H(x, p, u) − q m(x, u).
Here, the minus sign is due to the fact that the Maximum Principle assumes that the control-state constraint is written in the form m(x, u) ≥ 0. In view of the regularity condition (9), Theorem 7.1 in [5] (cf. also [8, 11]) asserts the existence of a scalar λ ≥ 0, an absolutely continuous function p : [0, T ] → R4 and an integrable function q : [0, T ] → R such that the following conditions are satisfied almost everywhere:
(i) max{|p(t)| : t ∈ [0, T ]} + λ > 0,
(ii) (adjoint equation and transversality condition)
− ˙p(t) = Hx[t] = −λLx[t] + hp(t), fx[t] + gx[t]u∗(t)i − hq(t), mx[t]i,
−p(T ) = (0, 0, 0, 0),
(iii) (maximum condition for Hamiltonian H)
H(x∗(t), p(t), u∗(t)) = max
u { H(x∗(t), p(t), u) | 0 ≤ u ≤ 1, m(x∗(t), u) ≤ 0 },
(iv) (local maximum condition for augmented Hamiltonian H)
(v) (complementarity condition)
q(t) m(x∗(t), u∗(t)) = q(t) (u∗(t)S∗(t) − V0) = 0 and q(t) ≥ 0.
In (iv), N[0,1](u∗(t)) stands for the normal cone from convex analysis to [0, 1] at
the optimal control u∗(t) (see e.g. [3]) and it reduces to {0} when u∗(t) ∈]0, 1[.
Since the terminal state x(T ) is free, it is easy to prove that the above necessary conditions hold with λ = 1; for a complete discussion see [1]. Hence, our problem is normal. We can further prove the existence of a constant Kq1 such that
|q(t)| ≤ K1
q|p(t)| (10)
for almost every t ∈ [0, T ] (see [5]).
Now we want to extract information from the conclusions (i)–(v) with λ = 1 that later will be used to validate our numerical solution. The adjoint equations in (ii) for the adjoint variable p = (pS, pE, pI, pN) are explicitly given by
− ˙pS(t) = −(d + cI∗(t) + u∗(t))pS(t) + cI∗(t)pE(t) − u∗(t)q(t), (11)
− ˙pE(t) = −(e + d)pE(t) + epI(t), (12)
− ˙pI(t) = −cS∗(t)pS(t) + cS∗(t)pE(t) − (g + a + d)pI(t) − apN(t) − A, (13)
− ˙pN(t) = bpS(t) + (b − d)pN(t). (14)
Next, we evaluate the maximum condition (iii) for the Hamiltonian H. We define the switching function φ by
φ(x, p) = Hu(x, u, p) = −B − pSS, φ(t) = φ(x(t), p(t)) (15)
and see that the condition (iii) is equivalent to the maximum condition
φ(t)u∗(t) = max
u { φ(t)u | 0 ≤ u ≤ 1, u S∗(t) ≤ V0}. (16)
This yields the control law
u∗(t) = min 1, V0 S∗(t) , if φ(t) > 0 0 , if φ(t) < 0. (17)
Any isolated zero of the switching function φ(t) yields a switch of the control from min{1, V0/S∗(t)} to 0 or vice versa. If, however, φ(t) = 0 holds on an
interval [t1, t2] ⊂ [0, T ], then we have a singular control. We do not enter here
into a detailed discussion of singular controls, since they never appeared in our computations. Moreover, our computations show that 0 < u∗(t) < 1 holds along
a boundary arc of the mixed constraint uS ≤ V0, i.e., whenever u∗(t) = V0/S∗(t).
Hence, the control is determined by
u∗(t) =
(
V0/S∗(t) , if φ(t) > 0
Due to 0 < u∗(t) < 1 the multiplier µ(t) in (iv) vanishes which yields the relation
0 = µ(t) = Hu[t] = −B − pS(t)S∗(t) − q(t)S∗(t).
This allows us to compute the multiplier q(t) for which we get in view of the complementarity condition (v) q(t) = − B S∗(t) − ps(t) = φ(t)/S∗(t), if u∗(t) = V0/S∗(t), 0, if u∗(t) < V0/S∗(t) . (19)
4
Numerical Results
In Table 1 we present the values of the initial conditions, parameters and con-stants which have been used in our computations. Apart from the weights A and B they coincide with those in [13]. As stated before in the Introduction, we use
Table 1. Parameters with their clinically approved values and constants as in [13]. Parameter Description Value
b natural birth rate 0.525 d natural death rate 0.5 c incidence coefficient 0.001 e exposed to infectious rate 0.5 g recovery rate 0.1 a disease induced death rate 0.2 A weight parameter 5 B weight parameter 10 T number of years 20 S0 initial susceptible population 1000
E0 initial exposed population 100
I0 initial infected population 50
R0 initial recovered population 15
N0 initial population 1165
W0 initial vaccinated population 0
the software package ICLOCS [6], resp., the programming environment AMPL and IPOPT [7, 16] to solve problem (Pmixed). As in [1] we consider the final
time T = 20 (days). We choose a rather fine grid with N = 10000 nodes with stepsize ∆t = T /N = 0.002 and use the implicit Euler scheme as integration method. The acceptable convergence tolerance is set to εrel = 10−9. For the
mixed constraint u(t)S(t) ≤ V0= 125 we find the optimal control
u∗(t) =
(
125/S∗(t) for 0 ≤ t ≤ t1,
0 for t1< t ≤ T = 20.
This shows that the constraint itself when expressed as the new control variable v = uS is a bang-bang control with only one switch at t1; cf. [10]. We obtain the
numerical results
J1(x, u) = 1692.2, t1= 17.89,
S(T ) = 1723.8, E(T ) = 7.7030, I(T ) = 4.7038, N (T ) = 1824.2.
The total amount of vaccines is W (T ) = 2235.8 The optimal trajectories and optimal control are presented in Figure 1.
900 1000 1100 1200 1300 1400 1500 1600 1700 1800 0 5 10 15 20 time t (years) Susceptible Population S 0 10 20 30 40 50 60 70 80 90 100 0 5 10 15 20 time t (years) Exposed Population E 0 5 10 15 20 25 30 35 40 45 50 55 0 5 10 15 20 time t (years) Infectious Population I 1100 1200 1300 1400 1500 1600 1700 1800 1900 0 5 10 15 20 time t (years) Total Population N 0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0 2 4 6 8 10 12 14 16 18 20 time t (years) Optimal Vaccination u 0 500 1000 1500 2000 2500 0 5 10 15 20 time t (years) Vaccinated Population W
Fig. 1. Optimal trajectories and control (vaccination) for mixed constraint uS ≤ 125. Top row: (left) susceptible population S, (right) exposed population E. Middle row: (left) infectious population I, (right) total population N . Bottom row: (left) vaccination (control) u, (right) vaccinated population W .
The adjoint variables, the switching function φ(t) and multiplier q(t) are displayed in Figure 2. It can be seen in Figure 2, bottom row, that the switching function φ(t) satisfy exactly the control law (17) while the multiplier q(t) obeys the multiplier rule (19).
-0.7 -0.6 -0.5 -0.4 -0.3 -0.2 -0.1 0 0 2 4 6 8 10 12 14 16 18 20 time t (years) adjoint variable pS -10 -8 -6 -4 -2 0 0 2 4 6 8 10 12 14 16 18 20 time t (years) adjoint variable pE -20 -15 -10 -5 0 0 2 4 6 8 10 12 14 16 18 20 time t (years) adjoint variable pI -2.5 -2 -1.5 -1 -0.5 0 0 2 4 6 8 10 12 14 16 18 20 time t (years) adjoint variable pN 0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0 2 4 6 8 10 12 14 16 18 20 time t (years)
control u and scaled switching function φ u φ 0 20 40 60 80 100 120 140 160 0 2 4 6 8 10 12 14 16 18 20 time t (years)
constraint uS <= 125 and scaled multiplier q u*S
q
Fig. 2. Adjoint variables, multiplier and switching function for mixed constraint u(t)S(t) ≤ 125. Top row: (left) adjoint variable pS, (right) adjoint variable pE. Middle
row: (left) adjoint variable pI, (right) adjoint variable pN. Bottom row: (left) control
u and (scaled) switching function φ(t) satisfying (17), (right) constrained function uS and multiplier q satisfying (19).
5
Conclusion
We considered an optimal control problem with mixed constraints and L1 cost
for a SEIR epidemic model of human infectious diseases. In this optimal control problem the control appears linearly. We discussed the necessary conditions of the Maximum Principle and obtained explicit formulas for the switching function and the multiplier associated with the mixed constraint in terms of state and adjoint variables. The optimal control problem is solved by discretization and nonlinear programming methods. The numerical solution shows that the con-straint itself, when regarded as the new control variable v = uS, is a bang-bang control. We never encountered singular controls.
Since the numerical approach furnishes as well the adjoint variables, we could verify that the computed solution satisfies the necessary optimality conditions precisely. The study of sufficient conditions is beyond the scope of this paper. In a future work, we shall investigate the numerical verification of second-order sufficient conditions using the methods in [10, 12]. This also allows us to study sensitivity analysis and compute parametric sensitivity derivatives.
6
Acknowledgement
The financial support of FCT projects PTDC/EEA-ELC/122203/2010, PTDC/EEI-AUT/1450/2012—FCOMP-01-0124-FEDER-028894, and European Union FP7 (FP7-PEOPLE-2010-ITN, Grant Agreement no. 264735-SADCO) are gratefully acknowledged.
References
1. M.H.A. Biswas, L.T. Paiva and M.d.R. de Pinho, A SEIR model for control of infectious diseases with constraints, Mathematical Biosciences and Engineering, to appear, (2014).
2. F. Brauer and C. Castillo-Chavez, “Mathematical Models in Population Biology and Epidemiology,” Springer-Verlag. New York, 2001.
3. F. Clarke, “Optimization and Nonsmooth Analysis,” John Wiley, New York, 1983. 4. F. Clarke, “Functional Analysis, Calculus of Variations and Optimal Control,”
Springer-Verlag, London, 2013.
5. F. Clarke and MdR de Pinho, Optimal control problems with mixed constraints, SIAM J. Control Optim. 48, (2010), 4500–4524.
6. P. Falugi, E. Kerrigan and E. van Wyk, ”Imperial College London Optimal Control Software User Guide (ICLOCS),” Department of Electrical and Electronic Engi-neering, Imperial College London, London, England, UK, 2010.
7. R. Fourer, D.M. Gay and B.W. Kernighan, ”AMPL: A Modeling Language for Mathematical Programming,” Duxbury Press, Brooks–Cole Publishing Company, 1993.
8. M.R. Hestenes, ”Calculus of Variations and Optimal Control Theory” 2nd Edition (405 pages), John Wiley, New York, 1980.
9. H.W. Hethcote, The basic epidemiology models: models, expressions for R0,
pa-rameter estimation, and applications. In: Mathematical Understanding of Infectious Disease Dynamics (S. Ma and Y. Xia, Eds.), Vol. 16. Chap. 1, pp. 1–61, World Scientific Publishing Co. Pte. Ltd., Singapore, 2008.
10. H. Maurer and N.P. Osmolovskii, Second-order conditions for optimal control problems with mixed control-state constraints and control appearing linearly, Pro-ceedings of the 52nd IEEE Conference on Control and Design (CDC 2013), Firenze, (2013), 514–519.
11. H. Maurer and S. Pickenhain, Second order sufficient conditions for optimal control problems with mixed control-state constraints, J. Optimization Theory and Applications, 86 (1995), 649-667.
12. N. P. Osmolovskii and H. Maurer, ”Applications to Regular and Bang-Bang Control: Second-Order Necessary and Sufficient Optimality Conditions in Calculus of Variations and Optimal Control,” SIAM Advances in Design and Control, 24, SIAM Publications, Philadelphia, 2013.
13. R.M. Neilan and S. Lenhart, ”An Introduction to Optimal Control with an Appli-cation in Disease Modeling” DIMACS Series in Discrete Mathematics, 75 (2010), 67–81.
14. L.T. Paiva, Optimal Control in Constrained and Hybrid Nonlinear Systems, Project Report (2013), http://paginas.fe.up.pt/∼faf/ProjectFCT2009/report.pdf.
15. Heinz Schaettler and Urszula Ledzewicz, “Geometric Optimal Control. Theory, Methods and Examples”, Springer, New York, 2012.
16. A. W¨achter and L. T. Biegler, On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming, Mathematical Pro-gramming, 106 (2006), 25–57.
