The of Control: A Brief Overview

Robin H. Pearce

Abstract Control form a vital part of engineering, where anything that needs to be regulated or optimised can be done so with a technique that is encompassed by . Since the 16th century, problems have been modelled as dynamical systems so that a mathematical control strategy could be devised and an equivalent physical strategy invented. Methods for controlling mechanical, analogue and digital systems have been invented and some are widely used. I will briefly introduce and explain many of the relevant topics considered to be part of control theory.

“Control Theory” is somewhat an umbrella term, are both linear and time-invariant. These systems used to describe a wide range of analytic techniques are known as LTI systems and are extremely useful and styles which are applied to the control of a math- in control and signal processing. ematical . Much of this theory is linked to sys- When considering the simplest, stateless LTI SISO tems theory, since many of the problems that can be system, we have that solved can also be modelled as a . The methods involved range from classical open or y(t) = (u ? h)(t) closed loop control, up to more modern disturbance rejection and noise filtering, as well as stabilisation where h(t) is the of the system. By and fine-tuning. taking the of both sides, we end up with Y (s) = U(s)H(s) Systems where Y (s), U(s) and H(s) are the laplace transforms Many of the systems can be modelled as a system of of y(t), u(t) and h(t) respectively. The function H(s) differential equations. Usually these are modelled in is known as the of the system and continuous time, however discrete time equations can can be written as also be useful. The basic system model in continuous Y (s) time is H(s) = U(s)

x˙(t) = f(x(t), u(t), t) Closed loop system y(t) = h(x(t), u(t), t) One of the simplest and most useful control systems where x(t) is the state, u(t) is the control input and is a closed-loop controller, similar to the one y(t) is the system output at time t. The easiest sys- seen in Figure 1. In this system, the output is mea- tem to consider is one where x, y and u are all . sured and sent back into the system to help regulate This is known as a SISO system, which stands for sin- the response of the system. Using the same notation gle input single output. Generally, these systems look before, we can consider two controllers, G1(S) and at signal processing, particularly for systems which G2(s), so that we can find the closed-loop transfer

1 The Theory of Control Robin H. Pearce

function H˜ (s), that is the function which satisfies where N(s) and D(s) are polynomials, then the sta- ˜ bility can be determined by the location of all s which Y (s) = R(s)H(s) satisfy D(s) = 0. By the Routh-Hurwitz criterion, where R(s) is the laplace transform of the reference the system will be stable if the real parts of all such signal r(t) and U(s) is the laplace transform of the s are negative. If we are given a particular system signal u(t) before it enters the plant H(s). where H(s) is unstable, then we can design a feed- back controller like the one in Figure 1, such that H˜ (s) or Hc(s) is stable by choosing G1(s) and G2(s). If

NH (s) N1(s) N2(s) H(s) = ,G1(s) = ,G2(s) = , DH (s) D1(s) D2(s) then we can rewrite the closed loop transfer function as NH N1D2 Hc(s) = . DH D1D2 + NH N1N2 Figure 1: Closed-loop feedback controller This system will now be stable if the solutions to DH D1D2 + NH N1N2 = 0 are all in the left-hand By starting with Y (s) = U(s)H(s) and substitut- plane, that is the real part of all the solutions are negative. The simplest set of controls for the closed ing back for U(s), G1(s), E(s), R(s) and G2(s), we end up with the expression loop system are to take G1(s) = K and G2(s) = 1, so now the system is stable if the zeros of 1 + KH(s) G (s)H(s) Y (s) = R(s) 1 are in the left-hand plane. 1 + G2(s)G1(s)H(s) so we have an expression for H˜ (s), where Y (s) = States R(s)H˜ (s). Quite often these systems are being con- trolled because some property of stability is desired. So far, the control methods considered are for sys- tems where the state doesn’t matter. In many real life examples this is not the case, so we need to con- Stability sider methods which control the state of a system. Most often, systems will be either linear or easily ap- There are two main types of stability in control sys- proximated by a . These linear systems tems: BIBO stability and internal stability. BIBO take the form stands for bounded inupt bounded output, and con- cerns the systems ability to return bounded outputs when given bounded inputs. The second type of sta- x˙(t) = Ax(t) + Bu(t) bility is often the most important, and can be deter- mined using the Routh-Hurwitz criterion. y(t) = Cx(t) + Du(t) Internal stability refers to the ability of the system where x(t) ∈ n, u(t) ∈ m and y(t) ∈ p. The to return to a state of equilibrium after an arbitrary R R R matrices are all constant and the system is commonly displacement from equilibrium. This is commonly as- referred to as an (A, B, C, D) system. The discrete sociated with the location of the poles of the transfer time equivalent is just the same, except t is replaced function, H(s). That is, if by n andx ˙(t) becomes x(n + 1). N(s) For a linear autonomous system,x ˙(t) = Ax(t) + H(s) = A(t−t0) D(s) g(t), solutions take the form Φ(t, t0) = e ,

2 The Theory of Control Robin H. Pearce

where eA(t−t0) is the exponential. The laplace we get an augmented system which only depends on transform for eAt is (sI −A)−1 which is known as the x(t) and r(t). Also, if (A, B) is controllable, then resolvant. For the full (A, B, C, D) system, the same it is possible to choose the eigenvalues of the state computations lead to transition matrix to be what we want by making the right choice of F . H(s) = C(sI − A)−1B + D Of course, eAt can often be difficult to calculate for Observability non-diagonal systems, so we can consider a similarity transform P x˜ = x, where det(P ) 6= 0. The system is A system is observable if we can determine the initial −1 −1 now a (P AP, P B, CP, D) system, where P AP condition for the state given the inputs and outputs At is diagonal and e is in Jordan normal form. over a finite time interval. The observability of a sys- tem can be found from the matrices A and C using a Controllability similar method to controllability, where the observ- ability matrix is given by For any (A, B, C, D) system, we can find all states obs(A, C) = [CT ,AT CT , (A2)T CT , ..., (An−1)T CT ]T that we can reach through controlling the system. A state xd is considered reachable or controllable-from- The condition for observability of a system is sim- the-origin if there exists a control input u that takes ilar to that for controllability; that is, the observ- the state x(t) from the zero state to xd in a finite ability matrix has full if and only if the system amount of time. is observable. One interesting result is that there is The converse definition is that a state xs is called a duality between controllability and observability. controllable if there exists a control input u that takes When given a standard (A, B, C, D) system, we can the state from xs to the zero state in a finite amount define the dual system as of time. Any set of states can also be called reachable or controllable if every state in the set is reachable or controllable. If all states are reachable or control- x˙(t) = AT x(t) + CT u(t) lable, then the system is a reachable or controllable T T system. y(t) = B x(t) + D u(t) To determine if a system is controllable, we can The result is that the dual system is controllable calculate the controllability matrix from the matrices if and only if the original system is observable and A and B, defined to be the original system is controllable if and only if the con(A, B) = [B, AB, A2B, ..., An−1B] dual system is observable. In real life, not all systems are observable, so it is useful to design an observer to where the system is controllable if and only if the con- estimate the state of the system and make corrections trollability matrix has full rank. If the matrix does based on this estimation. not have full rank, we can still work out the control- lable subspace. If Φ(t) is nonsingular, then reachabil- ity implies controllability and vice versa. This con- Observers cept is important if we are trying to ensure stability of a system by controlling the state to the origin. When a system is not observable, it is possible to The linear state feedback law is one way of en- create a state estimator and use that to control the suring stability in a system when the pair (A, B) is system instead. The only problem remaining is to controllable. If we set make sure that the estimate starts and remains accu- rate. The Luenberger Observer is given by the system u(t) = F x(t) + r(t) on the next page.

3 The Theory of Control Robin H. Pearce

at a particular state and R is the cost of implement- ing a control. If either of these matrices were nega- xˆ˙(t) = Axˆ(t) + Bu(t) + K(y − yˆ) tive it would represent being paid for implementing yˆ(t) = Cxˆ(t) + Du(t) a control or being at a state. This representation of LQR is looking for a control This augmented system is now a (A − KC, [B − u(t) which will regulate the system at the origin such KD,K],C, [D, 0]) system whose input is now that the cost is minimal. Here, x can be anything [uT , yT ], that is the input and output of the origi- we want to control to be zero, for instance, it could nal system. All that remains is to ensure that the be the error between the state and a state estimator distance between the estimate and the actual state is such as the one presented in the last section. This is stable. If we define the error between the state and often used for tracking where we need to the estimate to be control the system to another non-zero point, where we can choose to make the cost proportional to the e(t) = x(t) − xˆ(t) distance between the estimate and the state. The for this system is a linear state then it turns out that the estimation error actually feedback control law, similar to the ones presented behaves like an autonomous system, where previously. In this case, e˙(t) = (A − KC)e(t) u(t) = (−R−1BT P (t))x(t) Now if we choose K such that the real parts of the where P (t) solves the riccati differential equation eigenvalues of (A − KC) are all strictly negative, the estimate will be asymptotically stable, meaning the −P˙ (t) = AT P (t) + P (t)A − P (t)BR−1BT P (t) + Q estimation error will disappear as t → ∞. We can also combine this with a linear state feedback similar This equation is specified “backward in time”, that to the one on the last page to control the estimated is we start with a final condition and work backwards system to the desired state while the observer controls to the initial condition. This leads us into the topic the estimate to the actual state. of .

Linear Quadratic Regulator Model Predictive Control

For the Linear Quadratic Regulator (LQR), we will Also called “receding horizon control”, Model Pre- consider the system dictive Control, or MPC for short, works by solv- ing a problem over a short planning horizon, taking x˙(t) = Ax(t) + Bu(t) the first step and then recalculating the optimal con- where A and B are constant and the system is con- trol over the new planning horizon. This method trollable. Now, we will consider a cost for controlling is not optimal, however it is usually less computa- the system, given by a cost functional J. The in- tionally taxing and also allows for small unexpected tegrand of J is quadratic in x and u and is of the disturbances without throwing out the whole control form scheme. MPC is not guaranteed to be stable, however there Z T T T  T are different methods for applying the MPC so that J(u) = x Qx + u Ru dt + x(T ) Qf x(T ) the resulting system will be stable. One of these 0 methods is to enforce an end-point for the system to T T T where Q = Q ≥ 0, Qf = Qf > 0 and R = R > 0. drive towards. This usually makes the system stable This is necessary since Q represents the cost of being and uses the ideas found in .

4 The Theory of Control Robin H. Pearce

Dynamic Programming The Kalman Filtering works similarly to the Luenberger Observer that was seen on the There are a number of methods that fall into the previous page, however this time the matrix K de- category of dynamic programming, such as Dijkstra’s pends on time. The estimator for a simple case where shortest path algorithm, which finds the shortest or B = D = 0 this time is cheapest path through a network. The main result from dynamic programming that applies to Control xˆ(n + 1) = Axˆ(n) + K(n)(y(n + 1) − CAxˆ(n)) Theory is the Principle of Optimality, which states This is the Linear Minimum Mean Squared Error that if the optimal path between points a and b passes (LMMSE) estimator for the state x(n). If the noise through a third point c, then the optimal path from terms are gaussian, then the LMMSE is also the op- c to b is the same as the section of the optimal path timal MSE estimator. This means that the Kalman from a to b starting from c. Filtering Algorithm will give the most accurate esti- This principle is used when the minimal mates of the state when the measurements contain cost of a functional similar to that of the LQR on the gaussian noise. previous page. By applying the ideas of dynamic pro- gramming to a continuous system, we can derive the Hamilton-Jacobi- for finding opti- Non-linear Control mal solutions to LQR. The HJB equation is All of the systems considered so far have been lin- ∗ ∗ Jt ((x, t) + minu[g(x, u, t) + Jx a(x, u, t)] = 0 ear, which makes their analysis much easier, however many real world systems are non-linear and some- ∗ where J is the minimal cost functional. All of times a linear approximation is not sufficient. For the techniques presented thus far are for determin- non-linear systems, many of the concepts such as sta- istic systems with no noise, however stochasticity is bility, controllability and observability still mean the present in almost all physical systems, so it is impor- same, however the equivalent results are more depen- tant to consider techniques for dealing with uncer- dent on the particular model. tainty. The biggest difference between a linear and non- linear system is the location and number of critical points. Linear systems only have one critical point Linear Quadratic Gaussian and at the origin, making stability an important topic, Kalman Filtering however non-linear systems can have multiple critical points which make the dynamics even more difficult Many real world systems are imperfect and have in- to control. It is also possible that non-linear systems herent uncertainty in the measurements. We can sim- will exhibit properties such as periodic orbits, bifur- ulate this by considering a modification to the stan- cations and chaos, as well as finite escape time. dard linear system where we add some noise terms to the state and the output. The system now becomes Conclusion

x˙(t) = Ax(t) + Bu(t) + ξx(t) There are many topics and methods that fall into the category of Control Theory, but what they all have y(t) = Cx(t) + Du(t) + ξy(t) in common is that they are designed to solve real where ξ is a random process, that is a sequence of world problems. Many of these are particularly use- random variables. In the continuous time case, this ful for engineers designing just about anything that is similar to Brownian motion and other methods for moves. The topics in Control Theory are important solving stochastic differential equations are required, for many reasons, most of all because they keep our making the discrete time case much simpler. world stable.

5