Bruno Martins Lima
ECONOMIC MODEL PREDICTIVE CONTROL AND OPTIMAL ESTIMATION FOR NONLINEAR SYSTEMS
Disserta¸cão submetida ao Programa de Pós-gradua¸cão em Engenharia de Automa¸cão e Sistemas da Uni-versidade Federal de Santa Catarina para a obten¸cão do Grau de Mestre em Engenharia de Automa¸cão e Sis-temas.
Orientador: Prof. Dr. Julio Elias Normey Rico Coorientador: Prof. Dr. Alejandro H. González Coorientador: Prof. Dr. Daniel Martins Lima
Florianópolis 2018
Lima, Bruno Martins
Economic model predictive control and optimal estimation for nonlinear systems / Bruno Martins Lima ; orientador, Julio Elias Normey Rico, coorientador, Alejandro Hernán González, coorientador, Daniel Martins Lima, 2018. 127 p.
Dissertação (mestrado) - Universidade Federal de Santa Catarina, Centro Tecnológico, Programa de Pós Graduação em Engenharia de Automação e Sistemas, Florianópolis, 2018.
Inclui referências.
1. Engenharia de Automação e Sistemas. 2. Controle preditivo. 3. Estimação de estados. 4. Sistemas não lineares. 5. Filtro de Kalman. I. Rico, Julio Elias Normey. II. González, Alejandro Hernán . III. Lima, Daniel Martins IV.
Universidade Federal de Santa Catarina. Programa de Pós-Graduação em Engenharia de Automação e Sistemas. V. Título.
Bruno Martins Lima
ECONOMIC MODEL PREDICTIVE CONTROL AND OPTIMAL ESTIMATION FOR NONLINEAR SYSTEMS
Esta Disserta¸cão foi julgada adequada para a obten¸cão do Título de “Mestre em Engenharia de Automa¸cão e Sistemas”, e aprovada em sua forma final pelo Programa de Pós-gradua¸cão em Engenharia de Automa¸cão e Sistemas.
Florianópolis, 06 de Julho de 2018.
Prof. Dr. Werner Kraus Júnior Coordenador do Curso
Universidade Federal de Santa Catarina
Banca Examinadora:
Prof. Dr. Julio Elias Normey Rico Orientador
Universidade Federal de Santa Catarina
Prof. Dr. Aguinaldo Silveira e Silva Universidade Federal de Santa Catarina
Dr. Mario Cesar Massa de Campos
Centro de Pesquisa da PETROBRAS (CENPES) (Videoconferência)
Dr. Paulo Renato da Costa Mendes Universidade Federal de Santa Catarina
Este trabalho é dedicado aos meus colegas da área, aos meus queridos pais e irmãos.
AGRADECIMENTOS
Obrigado aos meus pais, ao meu irmão Hugo e familiares pelo amor incondicional e por me apoiar sempre, mesmo de longe. Muito obrigado ao Daniel por servir de inspira¸cão, exemplo e companhia nas nerdices durante todo o período de faculdade. Agrade¸co especialmente por sempre estar atento na orienta¸cão e corre¸cões do trabalho, e pelas discussões produtivas sobre controle preditivo.
Agrade¸co ao meus colegas de turma e agregados pela companhia e paciência. Obrigado especialmente ao Henrique pelo "tough love" necessário em muitos momentos. Obrigado, Jeane, pelo apoio e com-panheirismo e por me ajudar na minha recupera¸cão.
Obrigado aos meus colegas de mestrado pela companhia no dia à dia e por me aguentar meus trejeitos estranhos. Agrade¸co especial-mente à Marina por me ajudar a sair da cama de manhã e cuidar de mim, e à Mariana pelo seu peculiar senso de humor.
Agrade¸co ao professor Julio pela eterna paciência e compreen-são, especialmente nos meus momentos mais difíceis. Obrigado pela oportunidade de trabalhar com controle preditivo e pela liberdade dada para explorar temas novos na área.
Gracias à Alejandro director y Antonio por presentarme una visión totalmente diferente y interesante de control predictivo. Muchas gracias Mr. Anderson, Agu y Marcelo por la amistad y calurosa acogida en Santa Fe.
Agrade¸co ao Marcelo e Mario por ter me dado a oportunidade de compartilhar meus conhecimentos com o grupo de pesquisa da Petro-bras e obrigado à empresa pelo financiamento deste trabalho.
I believe in no God, no invisible man in the sky. But there is something more powerful than each of us, a combination of our efforts, a Great Chain of industry that unites us. But it is only when we struggle in our own inter-est that the chain pulls society in the right direction. The chain is too powerful and too mysterious for any government to guide. Any man who tells you different either has his hand in your pocket, or a pistol to your neck. (Andrew Ryan)
RESUMO
Processos não lineares frequentemente aparecem na indústria e representam um desafio para estima¸cão e controle. Para lidar com eles, usualmente é necessário usar técnicas não lineares que levam a teorias mais avan¸cadas, implementa¸cão complexa e alto custo compu-tacional. Para estima¸cão, as versões lineares e não lineares do filtro de Kalman e Moving Horizon Estimation (MHE) foram implementadas em um processo benchmark e comparadas em diferentes cenários. As técnicas lineares se mostraram inadequadas para o processo não li-near. O MHE não linear tem a melhor performance já que usa um horizonte de estima¸cão e restri¸cões mas necessita mais poder computa-cional. Também foi mostrado como projetar um observador com atraso com um foco em robustez. Uma compara¸cão foi feita entre uma estru-tura observador-preditor e um preditor de Smith filtrado. O primeiro mostrou bons resultados e é esperado que facilite o projeto de predito-res para sistemas multi-variáveis. Para controle, estratégias ótimas com um foco em objetivos econômicos foram explorados. Quatro tipos de controle preditivo baseado em modelo (MPC) foram implementados: MPC não linear de seguimento, MPC econômico puro (EMPC), EMPC re-gularizado (reg-EMPC) e EMPC com restri¸cão estabilizante (EMPC-sc). As condi¸cões que levam a estabilidade foram apresentadas. O EMPC puro tem a melhor performance econômica mas tem um comporta-mento periódico. O reg-EMPC garante estabilidade mas é conservativo. O EMPC-sc tem uma boa performance econômica, é estável e tem um projeto relativamente simples.
Palavras-chave: Controle ótimo. Controle de processos. Controle preditivo. Estima¸cão de estados. Observadores. Filtro de Kalman. Estima¸cão de parâmetros. Otimiza¸cão. Sistemas dinâmicos não li-neares. Reatores químicos.
RESUMO EXPANDIDO
Introdu¸cão
Controle Preditivo baseado em Modelo (MPC) é um termo geral usado para descrever algoritmos avan¸cados de controle baseado em um mo-delo explícito, horizonte de predi¸cão e fun¸cão objetivo. MPC possui décadas de sucesso na indústria, especialmente em processos petroquí-micos. O MPC, para processos grandes, geralmente é implementado em três camadas: otimiza¸cão estática (RTO), MPC e parte regulatória. O RTO é utilizado para otimiza¸cão estática baseada em objetivos econô-micos enquanto o MPC possui mais foco em seguimento de referência. Mais recentemente, houve um interesse de incluir objetivos econômicos na formula¸cão do MPC para obter comportamento ótimo também na parte dinâmica e a essa ideia geral se dá o nome de MPC Econômico (EMPC). Além do controle, outra parte importante é a estima¸cão de estados e perturba¸cões já que nem todas as variáveis são mensuráveis e o modelo possui incerteza. Uma classe muito utilizada é a de es-timadores ótimos baseados em mínimos quadrados como o filtro de Kalman (KF). Uma extensão do KF é o estimador de horizonte des-lizante(MHE) que é baseado em otimiza¸cão e possui restri¸cões e um horizonte de estima¸cão. Os estimadores também podem ser usados para diminuir os efeitos de atraso no sistema através de uma estrutura chamada observador-preditor (O\&P ).
Objetivos
O principal objetivo é investigar se técnicas avan¸cadas não lineares de controle e estima¸cão, quando comparadas com alternativas mais simples, apresentam uma melhor performance enquanto possuem esta-bilidade, robustez e facilidade de implementa¸cão.
Metodologia
O objetivo é dividido em duas partes: estima¸cão e controle. Na primeira parte, técnicas lineares e não lineares de estima¸cão são implementadas em um sistema benchmark. As técnicas implementadas são o KF de regime permanente, KF linear e estendido, e MHE linear e não linear. Suas performances e rejei¸cão de perturba¸cão são avaliados segundo índices como o somatório dos erros de estima¸cão ao quadrado norma-lizado. Na segunda parte, os MPC para tracking são comparados com técnicas de EMPC utilizando critérios econômicos.
Resultados e Discussão
um custo computacional maior. O seu uso não é justificável para o pro-cesso apresentado mas espera-se que ele tenha vantagens significativas quando as restri¸cões do sistema estão ativas. Resultados preliminares
de umO\&P baseado em KF foram apresentados. Foi mostrado que
o O\&P e o popular Preditor de Smith Filtrado (FSP) são equivalen-tes. Assim, as ferramentas no domínio da frequência do FSP podem ser utilizados noO\&P . Foi mostrado através de simula¸cões como a sintonia do filtro afetam a resposta na frequência e, portanto, a robus-tez. Espera-se que oO\&P tenha vantagens no projeto de preditores multi-variáveis quando comparado ao FSP e isso será propriamente analisado em trabalhos futuros. Sobre MPC, duas técnicas de EMPC que garantem a estabilidade foram apresentadas: o EMPC regularizado (reg-EMPC) e o EMPC com restri¸cão estabilizante (EMPC-sc). Eles fo-ram comparados com técnicas de EMPC puro e MPC para seguimento de referência. O reg-EMPC alcan¸ca estabilidade adicionando pesos de seguimento de referência e um algoritmo simples para encontrar-los automaticamente a cada período de amostragem foi apresentado. A performance econômica do reg-EMPC se mostrou conservadora demais. O EMPC-sc mostrou melhores resultados por ter parâmetros de sintonia mais flexíveis mas possui um custo computacional um pouco maior.
Considera¸cões Finais
Sistemas multi-variáveis não lineares representam um desafio em con-trole de processos. Quando objetivos econômicos são levados em conta, o problema fica ainda maior já que nesse caso o sistema opera próximo de restri¸cões e de forma pouco estável. Por isso, é interessante a ideia de utilizar técnicas mais avan¸cadas para esses casos. Porém, o projeto e implementa¸cão se tornam mais complicados e devem efetuados com cuidado. Neste trabalho, foram comparadas técnicas avan¸cadas de con-trole e estima¸cão baseados em otimiza¸cão e o resultado foi positivo.
Palavras-chave: Controle ótimo. Controle de processos. Controle preditivo. Estima¸cão de estados. Observadores. Filtro de Kalman. Estima¸cão de parâmetros. Otimiza¸cão. Sistemas dinâmicos não li-neares. Reatores químicos.
ABSTRACT
Nonlinear processes frequently appear in the industry and rep-resent a challenge for estimation and control. To deal with them, it is often necessary to use nonlinear techniques which leads to more ad-vanced theory, complex implementation and high computational cost. For estimation, the linear and nonlinear versions of Kalman filter and Moving Horizon Estimation (MHE) were implemented in a benchmark process and compared in different scenarios. The linear techniques are shown to be inadequate for the nonlinear process. The nonlinear MHE has the best performance since it uses an estimation horizon and con-straints but it requires more computation power. It was also shown how to design an observer for systems with delay with a focus in robust-ness. A comparison was made between an observer-predictor structure and a filtered Smith Predictor. The former showed good results and is expected to facilitate the design of predictors for multivariable sys-tems. For control, optimal strategies with a focus in economic objectives are explored. Four types of Model Predictive Control (MPC) are imple-mented: pure tracking nonlinear MPC, pure Economic MPC (EMPC), regularized EMPC (reg-EMPC) and EMPC with stabilizing constraint (EMPC-sc). The conditions that leads to stability are presented. The pure EMPC has the best economic performance but has a periodic be-havior. The reg-EMPC guarantees stability but is conservative. The EMPC-sc has a good economic performance, is stable and is fairly easy to design.
Keywords: Optimal control. Process control. Predictive contro. State estimation. Observers. Kalman filter. Parameter estimation. Optimization. Nonlinear dynamic systems. Chemical reactors.
LIST OF FIGURES
1.1 The traditional paradigm used in the industry. Source: [1] 30 1.2 Basic elements of MPC. Source: <commons.wikimedia.
org/wiki/File:MPC_scheme_basic.svg> . . . 32
2.1 Ellipse representing a 2D normal distribution . . . 45
2.2 Schematic representation of the CSTR. Source: [2] . . . 51
2.3 Results for initial estimation error. Linear MHE gives the same results as nonlinear MHE and KF the same as EKF . 56 2.4 Results for the disturbances. LMHE gives the same re-sults as NMHE and KF the same as EKF . . . 57
2.5 Estimation for the linear techniques with disturbance inT0 58 2.6 Estimation for the nonlinear techniques with disturbance inT0 . . . 59
2.7 Histogram for computation time for the NMHE . . . 59
2.8 Case 1 from 0s to 1000s. The KF and LMHE give very similar results. The same for EKF and NMHE . . . 61
2.9 Case 1 from1000s to 1500s for cA0 andT0. LMHE have the same result as NMHE . . . 62
2.10 Case 1 from1000s to 1500s for cA . . . 62
2.11 Noise attenuation for EKF . . . 63
2.12 Estimation ofTcfor different tuning . . . 64
3.1 Filtered Smith Predictor structure . . . 68
3.2 Observer-predictor structure . . . 68
3.3 Frequency domain magnitude plot of filterFKP(z) for differentR . . . 74
3.4 Frequency domain magnitude plot of filterFKP(z) for differentQd . . . 75
3.5 Frequency domain magnitude plot of filterFKP(z) for differentQs . . . 76
3.6 Nominal closed-loop responses of all cases . . . 77
3.7 Frequency domain magnitude plot of filterFKP(z) for all cases. . . 78
3.8 Closed loop-responses with model uncertainty in all cases. 79 4.1 Basic block diagram for control . . . 81
4.2 Depiction of the bounds inV (x) . . . 84
4.3 Prediction (dashed line) and closed-loop trajectory (bold line).N = 5 . . . 89
4.4 Depiction of dissipation . . . 92
(–) for bounds . . . 102 4.8 Predictions of NMPC with N = 30. Jtr = 0.0361 and
Jeco= 0.2172. (-) for real value, (–) for set-point and (.-) for prediction . . . 103 4.9 Response of an EMPC withN = 100. Jeco = 0.2222. (-)
for real value, (.-) for set-point and (–) for bounds . . . . 104 4.10 Response of a reg-EMPC withN = 30. Jtr= 0.2710 and
Jeco= 0.2160. (-) for real value, (.-) for set-point and (–) for bounds . . . 105 4.11 Weights calculated for reg-EMPC each sampling time . . 106 4.12 EMPC-sc withN = 30 and \delta = 0.5. Jeco= 0.2178. (-) for
real value, (.-) for set-point and (–) for bounds . . . 108 4.13 Response ofcB withN = 30 and varying \delta . . . 109 4.14 Predictions of EMPC-sc withN = 30 and \delta = 0.5. (-) for
real value, (–) for set-point and (.-) for prediction . . . . 110 4.15 Response ofcB with\delta = 0.9 and varying N . . . 111 4.16 Response ofcB for EMPC-sc with\delta = 0.9 and N = 30.
The prediction is shown every 10 steps . . . 112 4.17 Response ofcB for LQR with a EKF, first2000 s . . . 113 4.18 Comparison of two tuning for the estimation of the
dis-turbances . . . 114 4.19 Response of the concentrations for an EMPC-sc withN =
30 and \delta = 0.9. Disturbance in cA0att = 0s and in T0at t = 3000s . . . 115 4.20 Response of the concentrations for an reg-EMPC and
LIST OF TABLES
2.1 Parameters for the CSTR . . . 53
2.2 Normalization and tuning matrices . . . 55
2.3 JN M SEfor all variables and algorithms. Total in absolute and its relative to the EKF . . . 57
2.4 Two extreme cases for parameter uncertainty . . . 60
2.5 TotalJN M SEper algorithm for all cases . . . 60
2.6 TotalJN M SEwith and without noise per algorithm . . . 64
4.1 Normalization, tuning weights and values used . . . 100
4.2 Computational effort andJecofor each horizon . . . 106
4.3 Jeco,Jtrand computational effort for different\delta . . . 107
4.4 Normalization and tuning weights for EKF . . . 111
4.5 JecoandJtr for different controller.N = 30 and no esti-mator . . . 116
ACRONYMS CV Controlled Variable
DMC Dynamic Matrix Control DP Dynamic Programming DTC Dead-time Compensator EKF Extended Kalman Filter
EMPC Economic Model Predictive Control EMPC-sc EMPC with stabilizing constraint FSP Filtered Smith Predictor
GPC Generalized Predictive Control KF Kalman Filter
KP Kalman Predictor
LQG Linear-quadratic-Gaussian Control LQR Linear-quadratic Regulator
MHE Moving Horizon Estimation
MIMO Multiple-input and Multiple-output MPC Model Predictive Control
MV Manipulated Variable
NMHE Nonlinear Moving Horizon Estimation NMPC Nonlinear Model Predictive Control OCP Optimal Control Problem
PID Proportional Integral-Derivative Controller
reg-EMPC Regularized Economic Model Predictive Control RTO Real Time Optimization
SP Smith Predictor
ssKF Steady-state Kalman Filter UKF Unscented Kalman Filter
SYMBOLS
w, v state and output noise, respectively \in is an element of
\forall for all
\bigtriangledown gradient or Jacobian of a function \bigtriangledown 2 hessian of a function
ak, a(k) variablea at time step k
a sequence of variable a as in a = \{ a0, a1, ...\} \^ a estimate of variablea aSP set-point of variablea | a| norm of vectora \| a\| 2
Q squared Euclidean norm of the form a\prime Qa
min minimum
A\prime transpose of a matrix A - 1 inverse of a matrixA
d integrating disturbance
f (\cdot ) system function in discrete time, x+= f (x, u, d)
fc(\cdot ) system function in continuous time, \.x = fc(x, u, d)
h(\cdot ) output function,y = h(x, d) \ell (\cdot ) stage cost
\ell e(\cdot ) economic stage cost \ell tr(\cdot ) tracking stage cost
k time step
\kappa (\cdot ) generic control law
\kappa i(\cdot ) generic control law at stagei
\kappa f(\cdot ) generic control law in terminal region \BbbX f
m input dimension
n state dimension
N prediction horizon length
q disturbance dimension
p output dimension
\BbbR real numbers
\BbbR n n-dimensional vectors \BbbR m\times n real-valuedm\times n matrices
t time
V (\cdot ) value function V
x state vector
x+ value of x at next sample time (dis-crete time system)
\.x time derivative ofx (continuous time system)
\BbbX state constraint set \BbbX f terminal region
y output vector, usually measurable z - 1 delay operator
CONTENTS
1 Introduction 29
1.1 Problem Statement . . . 31 1.1.1 Model Predictive Control . . . 32 1.1.2 Optimal Estimation . . . 34 1.2 Objectives . . . 35 1.3 Methodology . . . 36 1.4 Final Remarks . . . 37 2 Optimal Estimation 39 2.1 Offset-free Control . . . 40 2.2 Kalman Filter . . . 44 2.2.1 Steady-state Kalman Filter . . . 46 2.2.2 Extended Kalman Filter . . . 47 2.3 Moving Horizon Estimation . . . 48 2.4 Simulation study and comparison . . . 50 2.4.1 Disturbance estimation . . . 54 2.4.2 Parameter uncertainties . . . 59 2.4.3 Noise attenuation . . . 61 2.4.4 Discussion . . . 64 2.5 Conclusion . . . 66
3 Estimators for time-delay systems 67
3.1 Observer-predictor . . . 69 3.2 Frequency Domain Analysis . . . 72 3.2.1 Tuning guidelines of the KP . . . 73 3.3 Simulation study . . . 76 3.4 Conclusion . . . 79
4 Economic Model Predictive Control 81
4.1 Lyapunov Stability . . . 83 4.1.1 Linear Unconstrained Case - LQR . . . 85 4.1.2 Stability of MPC . . . 87 4.1.2.1 Terminal "ingredients" . . . 88 4.1.3 Stability of EMPC . . . 91 4.2 Regularized EMPC . . . 93 4.3 EMPC - Stabilizing Constraint . . . 97 4.4 Simulation study . . . 99 4.4.1 reg-EMPC . . . 104 4.4.2 EMPC-sc . . . 107 4.4.3 Closed-loop with estimator . . . 109
5 Conclusion 119
1 INTRODUCTION
Model Predictive Control (MPC) is a broad term used to de-scribe advanced control algorithms based on an explicit model, pre-diction horizon and objective function [3]. It has a receding strat-egy meaning that only the first control action is implemented at each instant and the horizon is displaced towards the future. MPC is decades old with Dynamic Matrix Control (DMC) [4] being one of the first algorithms and has been successful in the industry, specially on oil and chemical processes.
In the mid-70s, the first algorithms were heuristically created and, because of their success, they quickly spread through the in-dustry [5]. Many companies created their own version with differ-ent acronyms such as DMC, MAC, ID-COM, DMCplus, among oth-ers. Independently, Generalized Predictive Control (GPC) was devel-oped with better theoretical characteristics such as transfer function model and stochastic aspects. At first, it was awkward to apply to multivariable problems and lacked constraints so it was less used in practice [5]. Nevertheless, these problems were solved later and GPC continue to be popular among researchers to this day.
In the following years, while the commercial MPC algorithms continued to be improved and used on different processes, the the-oretical aspects were being developed. Researchers realized that MPC was not so different than earlier optimal algorithms such as the Linear Quadratic Regulator (LQR). The difference is that, for MPC, an explicit solution for the control law is difficult to find since it has a finite-horizon and constraints. Despite that, the main stabil-ity results1 were developed based on Lyapunov functions, terminal cost and terminal constraint. Most of those theoretical results do apply to nonlinear systems but several practical difficulties emerge. Because of that, MPC for nonlinear systems continues to be a chal-lenge.
In the 90s, different approaches to MPC appeared with re-searches on hybrid systems (binary/integer variables), robust MPC (e.g. min-max) and explicit MPC (similiar to lookup tables). Also, customized optimization algorithms that exploit the particularity and sparsity of the MPC problem started to appear and continue to be developed to this day2. This allows MPC to be used on faster processes and to embed on dedicated hardware which in turn
ex-1For a review of MPC stability, refer to [6].
2For a review in Fast NMPC and EMPC refer to [7]. 29
pands the use cases3.
On Figure 1.1 it is shown the architecture that is commonly used in large industrial processes. The upper layer is the Real-Time Optimization (RTO) which is usually used to find the optimal steady-state operation of the process based on economic objectives. Set-points are calculated and passed to the supervisory control. This part is implemented with a MIMO technique such as MPC which has faster sampling time, a tracking objective and maybe simpler models. Then, the regulatory control receives the set-points from the upper layer and acts on the process using PIDs.
Figure 1.1: The traditional paradigm used in the industry. Source: [1].
More recently, there has been increased interest in applying an economic objective on MPC to achieve an optimal dynamic be-havior. Although this is not a new concept, only recently the re-searchers have developed the conditions for closed-loop stability and performance under Economic MPC (EMPC). This technique is hard to implement since it often has a nonconvex objective and might even have a periodic optimal behavior, e.g. chemical reactors [9]. Periodic EMPC and average constraints were also studied.
Besides control, another important part of the solution is the estimator which attempts to solve many problems. Firstly, often some states of the system are not directly measured but are neces-sary for control and monitoring. It might be cost effective to reduce
1.1. Problem Statement 31
the number of sensors employed. Besides that, even if all the states are measured, the sensors usually are corrupted by noise and the signal must be filtered. Another important task of the estimator is to detect disturbances and uncertainties in the model. This informa-tion is used to update the process model and is sent to the controller to achieve offset-free behavior.
One important class of optimal estimators is based on a least squares objective such as the Kalman Filter (KF) [10]. KF has a stochastic formulation, can be either continuous or discrete, can es-timate states and parameters, allows sensor fusion, is recursive, not particularly hard to implement and have extensions for nonlinear processes [11]. Because of these advantages, it is widely used in the industry, specially on electronics [12].
The idea behind the KF has been used on an optimization-based estimator which is called Moving Horizon Estimation (MHE). It is analogous to MPC in the sense that it allows constraints on the model and have a receding horizon. This represents an advan-tage compared to KF and can show better performance on some problems [13]. The downside is that MHE needs to solve an opti-mization problem each sampling time which is time consuming but, due to their similarities, MHE and MPC can use the same numerical software.
On the next section, the control and estimation problems will be formally described and some notation used throughout this docu-ment will be introduced. After that, the objectives and methodology of this thesis will be presented.
1.1 PROBLEM STATEMENT
We start by defining the continous dynamic time-invariant model used in this work,
\.x = fc(x, u, d)
y = h(x, d) (1.1)
which has dimensionsx\in \BbbR nfor the states,u\in \BbbR m for the control action, d \in \BbbR q for the disturbance and y \in \BbbR p for the measured output.fc(\cdot ) and h(\cdot ) being nonlinear functions. This model can be transformed to discrete time by any method such as Runge-Kutta
and the result is given by:
x+= f (x, u, d)
y = h(x, d) (1.2)
withx+denoting the state at the next time step. Often this model is augmented with stochastic components on the state and output equation like
x+= f (x, u, d) + w
y = h(x, d) + v (1.3)
withw \in \BbbR n and v \in \BbbR p being random variables which are com-monly modeled by zero-mean Gaussian noise. For variables at a specific timek, two equivalent notations will be used: x(k) and xk. 1.1.1 Model Predictive Control
The MPC is often visualized as in the Figure 1.2. Using the current state of the plant, the algorithm predicts the future output based on a model of the process. With this information, the con-troller calculates a control sequence that minimize the distance be-tween the prediction and the reference. To limit the computational cost, the optimization is only performed in a finite prediction hori-zon.
Figure 1.2: Basic elements of MPC. Source: <commons.wikimedia. org/wiki/File:MPC_scheme_basic.svg>.
1.1. Problem Statement 33
Many control algorithms can be classified as MPC since they have the same basic elements, even though they seem very differ-ent from each other. Furthermore, differdiffer-ent authors sometimes use very different notation which makes the comparison between tech-niques harder. In particular, the algorithms differ on the model used, e.g. step-response (DMC), transfer function (GPC), neural networks [14] among others. They all solve a complex4optimization problem at each sampling time. A sufficiently general description of MPC is given by VN(\=x0, d) = \mathrm{m}\mathrm{i}\mathrm{n}u Vf(xN) + N - 1 \sum i=0 \ell (xi, ui) \mathrm{s}.\mathrm{t}. xi+1= f (xi, ui, d), yi= h(xi, d), 0\leq g(xi, ui),
u\in \BbbU , x \in \BbbX , \forall i \in [0, N - 1], x0= \=x0, xN \in \BbbX f
(1.4)
withx\=0as initial condition for the states (known or estimated),\ell (\cdot ) is a general stage cost,Vf(xN) is a terminal cost and g(\cdot ) are gen-eral nonlinear path constraints. A variable in bold represents its sequence in the horizon, e.g.u =\{ u0, u1, ..., uN - 1\} . Since it is usu-ally assumed an integral model for the disturbance, its prediction is constant on the horizon and the notationi for d is dropped.
For tracking MPC, a quadratic formulation is used for the stage cost and the notation is given by5
\ell tr(xi, ui) =\| xi - xSP\| 2Q+\| ui - uSP\| 2R, (1.5) and for the terminal cost is the same withVf(xN) =\| xN - xSP\| 2P. Noises do not appear on (1.4) because their prediction value is zero. There are many ways to describe the constraints of the sys-tem. Most of the processes have box constraints, specially in the actuators, which are written asulb\leq u \leq uub. Geometries can also be defined such as polytopes (e.g.Ainx \leq bin) and ellipsoids (e.g. x\prime P x\leq 1).
4Since most of the time the variables have only real values, "com-plex" in this work should not be confused with "complex numbers".
5Squared Euclidean norm given by\| z\| 2
To find the optimal steady-state operation of the controlled system, an optimization problem is usually employed at the RTO layer. This problem is defined by
VSP(d) = \mathrm{m}\mathrm{i}\mathrm{n}
xSP, uSP \ell e(xSP, uSP, d) \mathrm{s}.\mathrm{t}. xSP = f (xSP, uSP, d),
0\leq g(xSP, uSP), uSP \in \BbbU , xSP \in \BbbX
(1.6)
with the pair(xSP, uSP) being the optimal operation state, \ell e(\cdot ) a economic objective andg(\cdot ) general nonlinear constraints.
1.1.2 Optimal Estimation
Starting from initial estimatex\^0| 06, the goal is to estimatexk at each time step using all the past measurements, fromt = 0 up to t = k. Similarly to MPC, the estimator accomplish this using a dy-namic model of the system and its disturbances. When the estimator minimizes some kind of objective function, it can be denominated optimal.
The algorithm can be explained in an intuitive way. Starting from an initial estimate of the state, the estimator uses a model rep-resentation of the plant to predict the output in the next sampling time. When the measurement is available, the predicted output is compared to y and this information is used to update the model. The current estimate of the state is passed to the controller so that it can be used to calculate the control action. This procedure is re-peated each time step.
The estimators will all be analyzed in the same framework called Full Information Estimation (FIE)7. It has the best theoreti-cal properties in terms of stability and optimality, in a least squares sense [15]. It also represents the ideal case where all past measure-ments, from zero tok, are processed using a nonlinear model and constraints. The famous Kalman Filter can be seen as a particular
6The notation x\^
a| b means the estimate of variable x at time a using information up to timeb.
7Details on Full Information Estimation and stability can be found in [15], section 4.2.
1.2. Objectives 35
type of FIE as will be shown in Chapter 2. The optimization problem of the FIE is given by
Vk(\=x, u) = \mathrm{m}\mathrm{i}\mathrm{n} \^ x0, w \ell ac(\^x0 - \=x) + k - 1 \sum i=0 \ell (wi, vi)
\mathrm{s}.\mathrm{t}. x\^i+1= f (\^xi, ui) + wi, \forall i \in [0, k - 1], yi= h(\^xi) + vi,
0\leq g(\^xi, ui), \^
x\in \BbbX , \forall i \in [0, k].
(1.7)
with \ell ac(\cdot ) as initial penalty and \ell (\cdot ) as the stage cost which pe-nalize the model disturbance and fitting error. Both of them must be positive definite so they usually have a quadratic cost with \ell (w, v) = \| w\| 2
Q - 1 +\| v\| 2R - 1, \ell ac(\^x0 - \=x) = \| \^x0 - \=x\| 2P - 1 and in-vertibleQ, R and P . \=x is the initial estimation of the states at k = 0, i.e.x = \^\= x0| 0. The result is a estimation ofx at time k (xk| k) given y, \=
x and u.
To achieve offset-free on the estimation, it is common to model the disturbance as a random-walk process (d+ = d + w
d) and then augment the state equations with \widetilde x+ = f
a(\widetilde x, u, w) and \widetilde
x = [x\prime d\prime ]\prime . Then, a problem similar to (1.7) can be solved. This procedure will be discussed in detail in Chapter 2.
With the motivation for advanced techniques discussed and the basic notation defined, the objectives and methodology of this thesis will be presented.
1.2 OBJECTIVES
Nonlinear processes represent a challenge for the controller and, because of that, it is necessary to use advanced algorithms for control and estimation. But that comes with a price. Nonlinear tech-niques are more complicated to implement, harder to prove stability and demand more computation power. Thus, it is important to com-pare them to consolidated algorithms to justify their use. With that in mind, the main objective of this thesis is the following:
Main Objective: to inquire if advanced nonlinear techniques for control and estimation, compared to simpler alternatives, have a better performance while being stable, robust and implementable.
This objective is divided in two parts: estimation and control. In the first part, linear and nonlinear optimal estimation techniques will be implemented. Their performance, constraint satisfaction and disturbance rejection will be assessed for a benchmark system. The comparison will be made between steady-state KF, linear and Ex-tended KF, and (non)linear MHE. These algorithms were chosen because they all have the same formulation which makes it easier to make a comparison. It will also be discussed how to design esti-mators for time-delay systems.
The second part is related to the Nonlinear MPC and optimal economic performance. First, LQR and MPC will be compared and the role of constraints will be stressed. Another important aspect is stability which is challenging to guarantee for NMPC and EMPC. Two types of EMPC that guarantee stability will be discussed: a reg-ularized EMPC and an EMPC with stabilizing constraint.
1.3 METHODOLOGY
Since the objective of this thesis is to compare techniques, two aspects are important: a benchmark system and assessment cri-teria. Benchmarking is important because it allows validation, re-producibility and comparability to other articles without the need to master and implement different methods [16]. Also, benchmark problems are useful while learning new algorithms because one can implement them and then easily compare to published results. With the process and control problem clearly defined, it is easier to repro-duce the results.
It is also important to clearly define the criteria used to com-pare the performance of estimation and controller. For estimation, the normalized mean squared error (NMSE) will be used which is given by
JN M SE= \sum N
i=1(z(i) - \^z(i))2 \sum N
i=1(z(i) - \=z(i))2
(1.8)
withz, \^z and \=z being, respectively, the "true" value, the estimation, and the mean ofz and N the number of data points. z can be a state (x) or disturbance (d).
The controllers that will be presented in this work have a clear objective: to minimize the stage cost in a prediction horizon. Because of that, this function is also a good candidate for a
perfor-1.4. Final Remarks 37
mance index. The index used will be the average stage cost written as J = 1 N N \sum i=1 \ell (xi, ui). (1.9) where N is the number of sampling steps of the simulation. This index has two variations:Jtr when the tracking stage cost is used andJecowhen the economic objective is used.
1.4 FINAL REMARKS
This work will explore optimal estimation and control with a focus on nonlinear systems, stability and economic performance. Comparisons will be made between different techniques found in the literature but it is not exhaustive in the sense that it will not compare many different types of techniques. In addition, since only one benchmark will be analysed, it does not mean that all the con-clusions draw from the simulations will hold for all processes. Per-formance indexes will be used when possible for an objective com-parison but other aspects will be assessed subjectively. Most of the work developed tries to convince the reader of some propositions but they are not theoretical proofs.
This master dissertation is supported by the Research Center of Petrobras (Centro de Pesquisas Leopoldo Américo Miguez de Mello - CENPES). The work here described was developed in a project by CENPES in conjunction with the Department of Automation and Systems (Departamento de Automa¸cão e Sistemas - DAS). The name of the project is "Development of Nonlinear Model Predictive Controllers and Performance Assessment of Predictive Controllers for Oil Production Plataforms" and the professors responsible are Prof. Julio E. Normey-Rico and Prof. Leandro B. Becker. This thesis also has the financial support from Coordination of Improvement of Higher Education Personnel (Coordena¸cão de Aperfei¸coamento de Pessoal de Nível Superior - CAPES).
In the fist half of 2017, the Montevideo Group’s Association of Universities (Asociación de Universidades Grupo Montevideo - AUGM) supported an exchange program to the Universidad Nacional del Litoral (UNL) in Santa Fé, Argentina. The adviser was Prof. Alejan-dro H. González which oversaw the activities there. The goal of this experience was to exchange knowledge on MPC, MHE and to learn more about EMPC and stability.
This thesis is organized as follows. Chapters 2 and 3 focus on the estimation problem. In Chapter 2 starts by explaining how to model the disturbance to achieve zero error in steady-state. Then, three variations of Kalman Filter and two Moving Horizon Estima-tors will be introduced. A simulation study of a benchmark system was used to compare the different techniques. In Chapter 3, it will be discussed one way to design observers for systems with time-delay. The observer proposed will be compared to the popular Smith Predictor. The Chapter 4 focus in MPCs that optimize an economic objective. Two EMPC will be presented and their performance will be compared using a benchmark system. Finally, the conclusions and future works will be summarized in Chapter 5.
2 OPTIMAL ESTIMATION
Estimation1 is an important part in control theory, although it might seem neglected by the control community2. In the control literature, the states are often assumed to be known and determin-istic but usually this is not the case. In industrial processes, not all states are directly measured either because it is impossible to do so or because the sensors required are expensive. Since the states are often used in the feedback loop, it is necessary to estimate them somehow.
Beside the state, it is important in estimation to deal with uncertainties. One of them is noise which is common in sensors and must be filtered. Another uncertainty is disturbance which is an unwanted input that affects the controlled variables. Disturbance modeling is directly linked to offset-free control and will be further discussed in this chapter. Another problem that appears often in the literature is parameter estimation. Parameter can be loosely defined as a variable that is considered to be constant when compared to the time-scale of the dynamic system. In this work, since parameters and disturbances are modeled in the same way, they are equivalent.
Just like in control, it is interesting to perform the estima-tion by minimizing some sort of criteria or index. This leads to op-timal estimation which is a broad term to describe techniques that have an objective function. Full Information Estimation is a general framework to study optimal estimation which leads directly to Mov-ing Horizon Estimation. Another important class of estimator is the Kalman Filter and its extensions. They have a stochastic framework which is interesting since the estimation problem is often described by stochastic processes.
An alternative is to design the estimator by pole placement such as the Luenberger observer [18]. This is done by choosing the eigenvalues of the state equations of the observer given by
eig(A - LC) with L as the gain. Its tuning, i.e. the position of the 1In the literature, one can also find the terms "observer" and "fil-ter". The former is commonly used for state estimation and the lat-ter for noise rejection. In this work, "estimation" is prefered but the others are used as well.
2Side note: in [17], the authors argue that, while the control community studies both estimation and control, they prefer the lat-ter because it is has a more "active" role and moves the system around. Therefore, it is inherently exciting. Estimation is passive and boring in comparison.
observer poles, is done by trial and error and a loose guideline is to choose them two to five times faster than the system’s poles. An extended version exists for nonlinear system which depends on lin-earization of the model at each sampling time [19]. Despite its sim-plicity, the Luenberger observer has a straightforward interpretation in the frequency domain for the linear case.
Estimation and system identification are closely related with the first being an online version of the second. Both of them attempt to infer information about the system by observing the measured output and most of them are based on a least-squares formulation [20]. The main difference is that system identification is also con-cerned with the choice of model to be used on the problem while in estimation and control the model structure (equations and parame-ters) are usually given.
In this chapter, several optimal estimation algorithms will be presented and compared. They were chosen because of their similar-ities and their applicability for nonlinear systems. First, disturbance modeling to achieve offset-free control will be discussed. Then, with the Full Information Estimation (1.7) in mind, Kalman filter and its extensions will be detailed. Next, the Moving Horizon Estimation will be presented with its particularities3. Finally, all of the algo-rithms will be applied to a benchmark problem and compared us-ing objective criteria. Their advantages and disadvantages will be pointed out.
2.1 OFFSET-FREE CONTROL
As noted in section 1.1.2, it is common to augment the state-space model of the system with an integral model of the disturbance d: \biggl[ x(k + 1) d(k + 1) \biggr] =\biggl[ f(x(k), u(k), d(k))d(k) \biggr] y(k) = h(x(k), d(k)) (2.1)
which can be represented as: \widetilde x+ = f
a(\widetilde x, u) with x = [x d]\widetilde \prime . It is still possible to model different kinds of disturbances, e.g ramp or sine. To achieve that, it is necessary to augment the state-space equations with the desired dynamic. For example, to model a ramp, 3Basic theoretical aspects of KF, EKF and MHE were researched on the final project for the bachelor’s degree.
2.1. Offset-free Control 41
two integrators would be added. For sine, two more states would be added with their poles at the desired frequency.
With the model (2.1) in mind, an offset-free tracking problem can be defined. Given an output set-pointySP \in \BbbY , the goal is to define an output feedback law u = \kappa (y)4 or a state-feedback law u = \kappa (x) such that:
1. Constraints are satisfied at all times.
2. The closed-loop system reaches an equilibrium. 3. The following condition holds true:
\mathrm{l}\mathrm{i}\mathrm{m}
k\rightarrow \infty y(k) = ySP (2.2) To achieve these goals, it is necessary that the disturbances reach a constant value and that there is no noise. So, the following assumption is necessary.
Assumption 2.1.1. Disturbances are bounded and asymptotically constant such that:
\mathrm{l}\mathrm{i}\mathrm{m}
k\rightarrow \infty d(k) = \=d (2.3) With noise, the system as a whole is constantly excited and, because of that, the observer and controller will constantly respond to it. Nevertheless, if the noise has zero mean as stated before, it is possible to control the system to a neighbourhood of the desired operating point. With that in mind, the noise termsw and v will be dropped in this section for brevity.
Since the modes introduced to the model are unstable, it is important to check the detectability of (2.1) because this property is a necessary and sufficient condition for the existence of a stable estimator. If the system is not detectable, an asymptotic estimator that achieves zero error, i.e.e(k) = y(k) - \^y(k) \not = 0, does not exist and, because of that, it is not possible to achieve offset-free control.
For nonlinear systems, the notion of detectability is given by output-to-state stability (OSS)5 but it is hard to define and check. 4\kappa (y) is an explicit or implicit function ofy. In its simplest case, it is a gainKy.
5OSS is analogous to input-state stability which is used in control [21].
Nevertheless, this property can be assessed locally by linearizing the nonlinear system in an operating point(\=x, \=d, \=u, \=y), i.e.
\delta x(k + 1) = A \delta x(k) + B \delta u(k) + Bd\delta d(k) \delta d(k + 1) = \delta d(k)
\delta y(k) = C \delta x(k) + Cd\delta d(k)
(2.4)
with\delta x = x - \=x, \delta u = u - \=u, \delta d = d - \=d, \delta y = y - \=y and
A = df dx \bigm| \bigm| \bigm| \bigm| x,\=\=u, \=d , B = df du \bigm| \bigm| \bigm| \bigm| x,\=\=u, \=d , Bd= df dd \bigm| \bigm| \bigm| \bigm| x,\=\=u, \=d , C = dh dx \bigm| \bigm| \bigm| \bigm| \= x, \=d , Cd= dh dd \bigm| \bigm| \bigm| \bigm| \= x, \=d
Now, the following condition for the existence of a stable es-timator can be defined.
Lemma 2.1.2. The linearized augmented model (2.4) is detectable if and only if the nonaugmented linear system(A, C) is detectable and the Hautus condition holds [22]:
rank\biggl[ I - A - BC C d d
\biggr]
= n + q (2.5)
withn and q being the number of states and integrating disturbances, respectively.
It is only possible to find a pair(Bd, Cd) that makes the system detectable if the number of disturbances is less or equal than the number of output (q \leq p). But detectability does not automatically imply offset-free control though. In [22], the author shows that for q < p it is possible to design an observer-controller pair that does not achieve offset-free while being detectable. Forq = p, though, it is shown that if the Lemma 2.1.2 is true, any asymptotically stable observer will guarantee offset-free, i.e. the condition is necessary and sufficient [23].
An equivalent statement for (2.5) is that the transfer func-tion from the disturbance to the output, which is given byGd(z) = Cd+ C(zI - A) - 1Bd, does not have a transmission zero at z = 1 [24]. This zero means that the static gain(Gd(1)) loses rank for con-stant inputs. This is undesirable because, by the rank-nullity theo-rem given by
2.1. Offset-free Control 43
rank(Gd(1)) + dim(null(Gd(1))) = q,
if rank(Gd(1)) \not = q, the dimension of the kernel will be different than zero. This means that there is a nonempty subspace ofd which does not appear in the output. So, it is impossible to reconstruct the disturbance by measuringy.
This test returns either true or false, i.e. either the system is detectable or not. But it might be interesting to check if the equation (2.5) is nearly losing rank which means that the system is almost undetectable. This can be accomplished by performing a singular value decomposition (SVD) of the matrix and checking the singular values. If there are relatively small values compared to the others, this means that the matrix is almost losing a rank. Then, for this system, it will be hard to design an observer that works well. Then, it is necessary to change the disturbance model or check if there was an implementation mistake of the estimator.
With these conditions in mind, another important aspect is how to design the disturbance model, i.e. where the integral distur-banced appears in the state-space model. If the disturbance has a physical meaning, it is desirable to incorporate its equations on the state-space model to achieve a meaningful estimator performance. The idea is that when the disturbance is properly modelled, the er-ror between the nominal and uncertain model will be smaller, there-fore, the estimator will be more robust.
Another choice is to use input disturbances, e.g.ud = u + d. This ensures that d affects the system with the same dynamic of the process and that it has a physical meaning. Also it is simple to design a controller that rejects the disturbance because it is only necessary to negate its effect, i.euSP = - d.
The simplest choice is to use an output disturbance model withy = h(x) + d. This automatically ensures detectability because q = p. This is typically used with a dead-beat observer with \^d = y - \^y and is the "default" option for classical algorithms such as DMC6. The downside is that it is known to work only with open-loop sta-ble systems in this form[25], although it is possista-ble to modify it to work for integrating processes[26]. Another characteristic is that 6It is interesting to note that classical techniques provides offset-free control by design which makes it harder to commit errors. It seems that state-space formulation (observer plus controller) allows more freedom but requires caution at the design phase.
the observer with output disturbance model can be sluggish if the disturbance does not appear directly in the output because its effect have to pass through the dynamic model first.
This section described some basic conditions to achieve offset-free control which is closely related to estimators and disturbance modelling. In the following sections, a class of optimal estimators will be defined and compared.
2.2 KALMAN FILTER
Kalman filter (KF), also known as linear quadratic estimator, is a class of recursive algorithms that uses the past measurements to estimate the current state based on statistical knowledge. The filter is named after Rudolf E. Kálmán which published important results in this field [10]. This algorithm has been widely successful in the industry with applications in state and parameter estimation, sensor fusion, signal processing and much more7.
To fully understand the algorithm, it is necessary to have some background in statistics. First, consider a linear state-space model given by
x(k + 1) = A x(k) + B u(k) + w(k)
y(k) = C x(k) + v(k) (2.6)
with w and v being zero-mean Gaussian noise. Notice that, com-pared to the model (1.3), it does not have the disturbance because it can always be modeled as augmented state as described in Sec-tion 2.1. Gaussian noise are represented as a mean and a covariance matrix with the notation beingw\sim \scrN (0, Q) and v \sim \scrN (0, R). Usu-allyQ and R are modeled by covariance matrices that are diagonal, for example: Q = \left[ \delta 2 1 0 0 0 \delta 2 2 0 0 0 \delta 2 3 \right] , R =\biggl[ \delta 2 1 0 0 \delta 2 2 \biggr] (2.7)
with \delta being the standard deviation of each variable. In \BbbR p, this matrix define an ellipsoid(v\prime R - 1v = 1) with its semi-axes being \delta . This concept can be visualized in Figure 2.1 forR. If there were el-ements outside the main diagonal, it would mean that the ellipsoid is rotated. The noisev has a 68% chance to be in this region if it 7For a recent review of the industrial applications of KF, see [12].
2.2. Kalman Filter 45
is modeled as a Gaussian noise. To find a region with 95% chance, just multiplyR by 22 which is the same as computing the region with two standard deviation.
v 1 -3 -2 -1 0 1 2 3 v 2 -2 -1 0 1 2 δ 2 δ 1
Figure 2.1: Ellipse representing a 2D normal distribution.
In an optimization point of view, the Kalman filter solves the following unconstrained weighted least squares problem:
Vk= min \^ x(0),w\| \^x(0) - \=x\| 2 P - 1+ k - 1 \sum i=0 \| wi\| 2Q - 1+ k \sum i=0 \| vi\| 2R - 1 (2.8)
which minimizes the estimated disturbance sequencew and the fit-ting error(v = y - h(\^x)) according to the positive definite matrices Q, R and P given an initial estimate \=x. For unconstrained linear systems, it has a recursive solution which will be described here.
The filter is initialized by two components: the expected meanx\^0 and the expected covariance P0| 0. This matrix represents the confidence in the state estimate since it defines an ellipsoidal re-gion aroundx\^0, i.e.(\^x0 - \=x)\prime P - 1(\^x0 - \=x) = 1. P0| 0must be positive definite and symmetric. An easy choice is diagonal matrix such as P0| 0= 4Q.
After the filter is initialized, it performs two steps each sam-pling time: prediction and update. At first, an a priori state estimate is calculated using the discrete model. At the same time, the state noise is propagated through the model resulting in an a priori
co-variancePk| k - 1. The prediction equations are given by: \^
xk| k - 1= A\^xk - 1| k - 1+ Buk - 1
Pk| k - 1= APk - 1| k - 1A\prime + Q (2.9) with the notationx\^k| k - 1meaning the estimate of x\^k with informa-tion up to timek - 1. Notice that in the second equation, Pk - 1| k - 1is propagated through the model(APk - 1| k - 1A\prime ) and then it is added Q. Since Q is positive, Pk| k - 1will always be "bigger" thanPk - 1| k - 1 which means that the estimatex\^k| k - 1is more uncertain.
Next, it is the update step. First, the Kalman gainK is calcu-lated usingP , C and R. Then, using K, the estimates \^xk| k - 1 and Pk| k - 1are updated according to the following equations:
K = Pk| k - 1C\prime \bigl( CPk| k - 1C\prime + R \bigr) - 1 \^ xk| k= \^xk| k - 1+ K\bigl( yk - C \^xk| k - 1 \bigr) Pk| k= (I - KC)Pk| k - 1 (2.10)
It is interesting to notice that Pk| k is updated by subtract-ing KCPk| k - 1 which makes it "smaller". This means that the un-certainty ofx\^k| k decreases.
The Kalman filter have some interesting properties. Ifw and v are zero-mean, uncorrelated and white, then KF is the best linear solution for the optimization problem defined in (2.8) [11]. 2.2.1 Steady-state Kalman Filter
If the linear system and the noise covariance matrices are time-invariant, the filter converges to a "steady-state" gain K\infty which can be calculated using any standard software. Then, the up-date equation is given by
\^
xk| k= \^xk| k - 1+ K\infty \bigl( yk - C \^xk| k - 1 \bigr) = (I - K\infty C)\^xk| k - 1+ K\infty yk
(2.11) The underlying algorithm solves a discrete algebraic Riccati equation (DARE)8that results in a pair(K\infty , P\infty ) . The steady-state KF (ssKF) is not optimal because it does not use the optimal gain at each time step, although its performance approaches the ideal whenk\rightarrow \infty .
2.2. Kalman Filter 47
There are systems that the DARE does not converge, which means that there is no solution forK\infty . Also, there are problems where the recursive algorithm converges to differentK\infty depending on initial condition forP0| 0. Nevertheless, if the system is detectable, there is at least one solution given by(K\infty , P\infty ).
The resulting filter is very similar to the Luenberger observer since both have a constant gain and are linear. They differ, though, in the design and tuning. The Luenberger observer uses pole place-ment to design a stable filter for the linear system. It is easy to make the connection between frequency response of the filter and the chosen poles but it is not clear how each pole affects each state. Meanwhile, the ssKF has tuning parameters for each variable which might facilitate the design for large systems.
The KF has some extensions for the nonlinear case. One of the simplest adaptations is the Extended Kalman Filter, which will be presented next.
2.2.2 Extended Kalman Filter
The problem for nonlinear system is that there is no easy way to propagate the covariance matrixP through the model. The Ex-tended Kalman Filter solves this using the nonlinear model for the estimate of the mean and using a linearized model to calculateP and the Kalman gainK. The algorithm is very similar to the linear case with a few modifications.
First, it is necessary to initialize the filter in the same way as done in the linear KF. Then, it is necessary to linearize the system around x\^k - 1| k - 1 to use the linearized model in the update of P . This system dynamic matrix is defined by:
Ak - 1= dxdf \bigm| \bigm| \bigm| \bigm| x,u\^ . (2.12)
With this information, the update equations are given by: \^
xk| k - 1= f (\^xk - 1| k - 1, uk - 1) Pk| k - 1= Ak - 1Pk - 1| k - 1A\prime k - 1+ Q
(2.13) Next, the output equation is linearized usingx\^k| k - 1:
Ck= dh dx \bigm| \bigm| \bigm| \bigm| x\^ . (2.14)
Then, the update step is given by: K = Pk| k - 1Ck\prime (CkPk| k - 1Ck\prime + R) - 1 \^ xk| k = \^xk| k - 1+ K\bigl( yk - h(\^xk| k - 1) \bigr) Pk| k = (I - KCk)Pk| k - 1 (2.15)
Notice that the estimate of the state and the output are propa-gated through the nonlinear models while the gain and covariance matrix are calculated with the linearized model. Since the mean andP use different models, EKF might have poor performance for highly nonlinear systems.
EKF is a very practical solution and only recently its stability and convergence were proven. This requires small initial estimation error, small noise terms and no model mismatch and these condi-tions can be somewhat restrictive [27]. Furthermore, it does not incorporate physical state constraints which may cause an estima-tion failure.
In [13], the author shows some situations where the EKF have an inadequate performance. If the nonlinear model has mul-tiple states that satisfy the steady-state measurements, the filter might converge to the incorrect one with poor initial guess. This estimate can even be physically impossible such as negative concen-tration. The state constraints can be applied to xk| k but this can cause a very poor performance. This motivates the use of a better estimation algorithm.
2.3 MOVING HORIZON ESTIMATION
The Moving Horizon Estimation attempts to solve the Full Information Estimation described in (1.7). Since the FIE grows in-definitely, the computational complexity scales at least linearly with the horizon lengthk. As stated before, this problem only has a re-cursive solution if the system is linear, unconstrained, and the cost functions are quadratic in which it becomes the Kalman Filter. What MHE does is to optimize only in a finite horizonT approximating
2.3. Moving Horizon Estimation 49
the past information in the arrival cost. The problem is written as:
\mathrm{m}\mathrm{i}\mathrm{n} \^ xk - T,w \scrZ k - T(\^xk - T) + k - 1 \sum i=k - T \| wi\| 2Q - 1+ k \sum i=k - T \| vi\| 2R - 1
\mathrm{s}.\mathrm{t}. x\^i+1= f (\^xi, ui) + wi, \forall i \in [k - T, k - 1],
yi= h(\^xi) + vi,
0 \leq g(\^xi, ui),
\^
xi\in \BbbX , \forall i \in [k - T, k].
(2.16)
withu and y being known variables.\scrZ k - T is the arrival cost which summarizes past data. Ideally, one would solve the Full Information Estimation fromk = 0 up to k - T and use the solution as \scrZ k - T but, of course, this only transfer the original implementation problem. So, it is necessary to approximate\scrZ k - T somehow.
The simplest option is to ignore the arrival cost altogether, i.e. zero prior weighting. Analyzing the objective function of MHE one can see that if a big horizon is used,\scrZ k - T becomes small compared to the stage cost and it could be discarded. Nevertheless, zero prior weighting might require a big horizon to achieve a performance comparable to the Full Information Estimation and this can lead to a high computational cost [15].
Another option is to approximate the arrival cost by the Kalman Filter since it is a recursive solution for FIE. If no constraints are active and the model is linear, the KF is the exact solution for \scrZ k - T and if the model is not highly nonlinear, EKF might be a good approximation for it. This filtering update is implemented by ap-plying the KF equations in parallel to MHE at the beginning of the estimation horizon. This means that before MHE is solved, the fil-ter estimatex = \^\= xk - T | k - T andPk - T | k - T. Then, the arrival cost is given by
\scrZ k - T(\^xk - T) =\| \^xk - T - \=x\| 2P - 1
k - T | k - T . (2.17) Given a horizon T , the filtering update gives a better per-formance compared to zero prior weighting because it better ap-proximates the FIE. Because of that, a smaller horizon is necessary to achieve a good performance, decreasing the computational cost. Also, it is important to notice that when k \leq T , the MHE has an increasing horizon and is equivalent to FIE.
The MHE also allows the use of more advanced models. For instance, it is possible to implement Differential-Algebraic Equa-tions (DAE) of the form:
\.x = fc(x, z, u, d) 0 = g(x, z, u, d) y = h(x, z, d)
(2.18)
where z are algebraic variables and g(\cdot ) are equations that define them. This model can be used in the MHE with the proper use of a integration method such as Runge-Kutta or a DAE solver.
It might be useful on some cases to write the process with DAE. Algebraic equations of this form usually arise from conserva-tion laws such as mass, energy and momentum balances [28]. This makes the modelling of the process simpler and more accurate. It is also possible to implement path constraints with algebraic equa-tions. When compared to KF, the possible use of DAE in MHE might represent a significant advantage.
With the estimation horizon, it is also possible to more ac-curately model parameters (p) of the process. With the KF, dis-turbances and parameter are modelled in the same manner: as a random-walk process. In MHE, though, it is possible to introduce a optimization variablep that is constant in the whole horizon as opposed tod which changes each sampling time. This type of mod-elling might be useful in some cases.
The use of DAE and parameters in MHE will not be anal-ysed in details in this work but they are interesting topics for future works.
2.4 SIMULATION STUDY AND COMPARISON
In this section, the estimators described will be implemented and tested in a benchmark problem relevant in the industry. The process chosen is the Continuous Stirred Tank Reactor (CSTR) that is common in chemical plants and represents a control challenge. Firstly, the steady state gain changes sign at the operating point so linear controllers with integral action do not work well. Secondly, the zero dynamics changes from minimum phase to non-minimum phase at the operating point which makes the system highly non-linear. A complete description of this system, including economic objectives and uncertainties, can be found in [2].
2.4. Simulation study and comparison 51
A schematic of the process can be seen in Figure 2.2. The CSTR has a tank with volumeVR that receives a flow \.V of a solvent with substance A with concentrationcA0and temperatureT0. Then, a chemical reaction transforms A in B which is the desired product.
Figure 2.2: Schematic representation of the CSTR. Source: [2].
Two unwanted reactions also occur which is substance B to C and A to D. The reactions are summarized by:
A k1 - \rightarrow B k2
- \rightarrow C 2A k3
- \rightarrow D
Outside the tank, there is a cooling jacket that is used to con-trol the temperature of the reaction. The heat removal \.QK is one of the control actions. The nonlinear differential equations that
de-scribe the system are given by: \.cA= \. V VR (cA0 - cA) - k1(T )cA - k3(T )c2A \.cB= - \. V VR cB+ k1(T )cA - k2(T )cB \. T = V\. VR (T0 - T ) + kwAR \rho CpVR (Tc - T ) - 1 \rho Cp \Bigl( k1(T )cA\Delta HAB + k2(T )cB\Delta HBC+ k3(T )c2A\Delta HAD \Bigr) \. Tc= 1 mKCP K \Bigl( \. QK+ kwAR(T - Tc) \Bigr) (2.19)
withcA andcB being the concentration of substances A and B, T andTcare the temperature in the reactor and in the cooling jacket. The reaction velocitieski are given by:
ki(T ) = ki0exp \biggl( Ei T + 273.15 \biggr) , i = 1, 2, 3. (2.20)
Only the first two states have constraints, cA, cB \geq 0. The inputs are constrained as follows:
3h - 1\leq V\. VR \leq 35h - 1 - 9000kJ h \leq \.QK \leq 0 kJ h 100\circ \mathrm{C}\leq T0 \leq 115\circ \mathrm{C} 4.5mol
l \leq cA0\leq 5.7 mol
l
(2.21)
with u = [ \.V /VR Q\.K]\prime as the manipulated variables and d = [cA0 T0]\prime as the unmeasured disturbances. The sampling time is Ts= 20s. The measured variables are the concentration cB and the temperatureT . The values for the physical parameters are given in Table 2.1.
The goal is to maximize the yield \Phi of product B which is defined as the ratio betweencB and the reactant cA [2]. The
opti-2.4. Simulation study and comparison 53
Table 2.1: Parameters for the CSTR. Name of the parameter Symbol Value
Collision factor for reactionk1 k10 1.287\cdot 1012h - 1 Collision factor for reactionk2 k20 1.287\cdot 1012h - 1 Collision factor for reactionk3 k30 9.043\cdot 109molAh1 Activation energy for reactionk1 E1 - 9758.3 K Activation energy for reactionk2 E2 - 9758.3 K Activation energy for reactionk3 E3 - 8560 K Enthalpies of reactionk1 \Delta HAB 4.2 molBkJ Enthalpies of reactionk2 \Delta HBC - 11 molBkJ Enthalpies of reactionk3 \Delta HAD - 41.85 molBkJ
Density \rho 934.2 mkg3
Heat capacity Cp 3.01 kgKkJ
Heat transfer coefficient kw 4032 hKmkJ2 Surface of cooling jacket AR 0.215 m2
Reactor volume VR 0.01 m3
Coolant mass mK 5 kg
Heat capacity of coolant CP K 2 kgKkJ
mal operating point is slightly different from the one given by the article: \= x = \left[ 2.16 1.08 104.9 104.9 \right] , u =\= \biggl[ 7.93 - 5 \biggr] , d =\= \biggl[ 5.1 104.9 \biggr] (2.22)
Since the system is open-loop unstable, it is necessary to have a controller to test the estimators at the operating point. The con-troller chosen is an LQR that uses a linearized model at the operat-ing point (2.22) and it has access to the real values of the states and disturbances. It was tuned to have a settling time aroundt = 500 s. The controller has access to the real values of the states and distur-bances so that it does not interfere in the analysis of the estimators. Since there will be changes in the operating point for different disturbances, it is necessary to have an algorithm to find the steady-state set-points ofu and x for a given d. It was designed an RTO with an objective that minimizes the squared Euclidean norm of the distance between cB andc\=B, i.e. \| cB - 1.08\| 2. The result are set-pointsxSP anduSP that are used by the controller.
All the estimators compared here have the same design pa-rameters and initial estimate. One thing that helps when designing estimators and controllers is to normalize the tuning parameters ac-cording to some criteria because it is easier to make comparisons between the variables. The criteria used here is the range, which is a region that the variable is expected to operate. For instance, if the variable has bounds, one could define the range as the upper minus the lower bound. If this is not the case, simulations can be used to find an operating range or the designer could use practical knowledge about the process.
For the CSTR, the concentration states are normalized by 1 mol/l. The temperature states are assumed to operate in the same range ofT0, i.e.15\circ \mathrm{C}. The range of the disturbances are defined by (2.21). The control inputs were normalized using their bounds. The normalization of the inputs and states were used in the design of the controller. The values used are summarized in Table 2.2.
For the estimators, the tuning parameters are the covariance matrices for the noises andP0| 0. Since the noise variance has the same unit as the variable, it is interesting to tune them as a per-centage of the range. So, Q and R are calculated by choosing a percentage, multiplying by the range and raising to the power of two to find the variance. For instance, variance for the disturbance cA0is given by\delta 52= (0.20\cdot 0.6)2.
The output’s noise is assumed to be higher than the state noise soR is bigger (in percentage) than Q. Also, the disturbances are tuned to have a higher variance so that the estimator converges faster for them. Using this method, the normalization and tuning pa-rameters are shown in Table 2.2. For the initial covariance estimate, it was chosen to be ten times bigger thanQ, i.e. P0| 0= 10Q.
The linear and nonlinear version of KF and the MHE will be implemented, including the steady-state KF. The MHE algorithms use a filtering update for the arrival cost, as described in Subsec-tion 2.3, and the horizon is equal to 10 sampling times. There will be three scenarios to be analyzed: one for disturbance estimation, one for parameter uncertainties and another for noise. Then, there will be a general discussion about the different estimators and their results.
2.4.1 Disturbance estimation
In the first scenario, the estimators must reject the initial es-timation error and estimate the disturbances. The system starts at
