Control Adaptive Systems Natural and artificial nervous systems are often Inman Harvey thought of as control systems or controllers. Informatics We will start by taking a quick look at Control Engineering because this field has many obvious Lecture 17: parallels, influences, applications and useful Control Engineering concepts with respect to autonomous robotics.
WARNING: I expect to run out of time before end of this Lecture – and will complete it next week
Spring 2010
Control Overview
Control engineering grew out of applied mechanics Linear systems and its development in the mid 20th century was Block diagrams tightly bound up with the Cybernetics movement. Feedforward control (Process regulation, target-seeking devices, etc.) Feedback control Nonlinear control Control engineering is the application of Adaptive control mathematical techniques to the design of Fuzzy control algorithms and devices to control processes or Control theory in neuroscience pieces of machinery. It often uses some kind of model of the thing to be controlled.
Spring 2010 Spring 2010
Conant & Ashby (1970) Conant & Ashby (1970)
“In this paper a theorem is presented which shows, under very broad conditions, that any regulator that is maximally both successful and simple must be isomorphic with the system being regulated”.
“The theorem has the interesting corollary that the living brain, so far as it is to be successful and efficient as a regulator for survival, must proceed, in learning, by the formation of a model (or models) of its environment” Download from: http://www.scribd.com/doc/2947203/Every-Good- (Why not read the paper and see if you agree?) Regulator-of-a-System-Must-Be-a-Model-of-that- System-Conant-Ashby Spring 2010 Spring 2010 Let’s start with linear systems Example
If a system can be modelled by a set of linear differential equations there are well understood Forced spring techniques for getting exact analytical solutions, and so for designing controllers so that the output of the system is the required one.
Spring 2010 Spring 2010
Real systems … Linearization
... are non-linear in almost every practical application. The most common trick: Approximate the nonlinear system by a linear system in the region of interest. Example: a pendulum Use Taylor series expansion and throw away all non-linear terms.
For the pendulum, if θ ≈ 0 then sin(θ) ≈ θ and so: Much of control engineering theory and practice is concerned with linear systems. Control of non- linear systems is a very complex area.
Spring 2010 Spring 2010
Block diagrams Block diagrams
They have their own algebra; Block diagrams are often used for system analysis and design of controllers as they facilitate the study of input/output relationships: transfer G1(t) G2(t) = G2(t)G1(t) functions.
u(t)G(t) y(t) The transfer operator could be anything (a function, a derivative, an integration of the input u(t) = input signal, etc.) It is often convenient to use complex y(t) = output variables applying the Laplace transform which allows a simplified mathematical treatment: G(t) = transfer function or operator: differentiation becomes multiplication, and y(t) = G(t)[u(t)] integration becomes division. This is also known as “working in the frequency domain”, as opposed to “working in the time domain”.
Spring 2010 Spring 2010 Open loop (feedforward) control Open loop (feedforward) control
reference disturbance disturbance reference disturbance disturbance
++ ++ Controller + Plant + output Controller + Plant + output
A reference signal (sometimes called the setpoint) The plant is the control engineer’s name for the gives information about the desired output. This physical system that is being controlled – for could be you setting the knob on your toaster to 4, example, a jet engine. The output is what we want to or 7; it could also be an outside temperature sensor control – it might be thrust, or turbine speed, or in a central heating controller. The controller fuel consumption, or brownness of toast, and it’s generates the control input, which is calculated at often called the “controlled variable”. A disturbance the time or in advance (i.e. at design time) to is any external or internal factor that causes the produce the desired output. output to deviate from what is desired. Spring 2010 Spring 2010
Open loop (feedforward) control Feedforward control of toasters
Here’s how the cheapest toasters work. When you reference disturbance disturbance switch the toaster on (usually pushing a lever that also lowers the toast into the toaster) the heating ++ coils are powered for a fixed time, which has been Controller + Plant + output calculated by the manufacturer to give good enough The important thing about the reference signal in a results in most cases. pure feedforward controller is that it doesn’t contain any information about the current output. Technically, this means that the control system cannot compensate for disturbances that occur after the control stage. The controller contains an implicit (or sometimes explicit) model of the plant that will produce the “expected” output. Spring 2010 Spring 2010
Feedforward control of toasters Biological feedforward control
In more expensive toasters (now more or less Saccadic eye movements use standard) you can turn a knob to vary the time feedforward control. They according to how brown you want your toast. show highly stereotyped velocity profiles. In both types, the controller uses no direct information about the current state of the toast. The fastest movement would use ‘bang-bang’ control (either The output is almost always either “on” or “off”. maximum speed or zero), but This is known as “bang-bang” control. It sounds since noise in muscles is crude, but it can be very efficient, and crops up proportional to the control even in very sophisticated controllers. signal, errors would be large Actual profiles are well predicted by optimising for end-point accuracy Spring 2010 Spring 2010 Feedback – negative and positive Feedback – negative and positive
Consider a system – physical, biological, economic – If the net effect of the feedback is to oppose, and that is in equilibrium. What will happen if you therefore reduce, the original disturbance, it is introduce a small disturbance somewhere in the called negative feedback. A system with negative system? feedback is therefore in principle capable of maintaining its equilibrium, and so negative feedback The effects of the disturbance will propagate is associated with stability. through the system. If the parts of the system are But if the net effect of the feedback is to sufficiently well interconnected, some of these reinforce, and thereby increase, the original effects will find their way back to the site of the disturbance, it is called positive feedback. If disturbance – this is called feedback. sustained, this can lead to a runaway increase in the size of the disturbance, taking the system far from equilibrium, and so positive feedback is associated with instability.
Spring 2010 Spring 2010
Feedback in biology Feedback in biology
Both kinds of feedback are seen in biological Negative feedback, exploited for its stabilising systems. For example, positive feedback features in effects, is found at all levels in biology, from some examples of ant behaviour. When a colony of physiology, where its involvement in homeostasis is Pheidole dentata is attacked by large numbers of perhaps best known, to the control of behaviour. fire ants outside the nest, some Pheidole workers It was the convergence between ideas of negative retreat to the nest and begin to run through the feedback in the control of behaviour and of galleries, laying odour trails and stimulating other machines that influenced Norbert Wiener in his workers do to the same. This behaviour spreads original formulation of cybernetics. rapidly through the nest, and when some threshold Classic paper: Arturo Rosenblueth, Norbert Wiener is reached, the workers seize the eggs, larvae, and and Julian Bigelow in: Philosophy of Science, pupae, and flee the nest, returning some time later 10(1943), S. 18–24. when the fire ants have departed.
Spring 2010 Spring 2010
Feedback in engineering Feedback in toasters
If you have a system that you would like to control If you have a system that you would like to control automatically, then adding a controller with some automatically, then adding a controller with some form of feedback from the output may be a good form of feedback from the output may be a good option. option.
For example, some expensive toasters now have sensors that measure the colour of the toast, and stop the toaster when the desired colour is reached. Some others have radioactive sensors that detect the particles released by toasting, and stop the toaster when they reach a certain density. These are feedback controlled toasters…
Spring 2010 Spring 2010 Feedback control (often called PID: >90% of real world control closed-loop control Reference/ Proportional control: input is proportional to error. setpoint disturbance disturbance For high gains or large errors, it will tend to overshoot and oscillate. There is always a steady- state error that cannot be corrected. + + +++ Controller Plant output - Integral control: can get rid of steady-state error by integrating it over time
The output is sensed and compared with the Derivative control: can reduce settling time by reference. The resulting error signal is fed back giving a better dynamical response. Reduces to the controller. The controller generates a overshoot by damping. control input calculated to bring the output closer to the reference – i.e. to reduce the error.
Spring 2010 Spring 2010
PID: >90% of real world control PID control
Spring 2010 Spring 2010
PID: >90% of real world control PID: >90% of real world control
Q: So how do you find the right parameters? Q: So how do you find the right parameters?
A: (i) It depends what you mean by ‘right’ A: (i) It depends what you mean by ‘right’ (ii) They may not exist (ii) They may not exist
Ideally, you want your system to be “critically damped” – with zero overshoot and minimum settling time.
Spring 2010 Spring 2010 PID: >90% of real world control Problems with feedback
There are lots of problems with practical implementations of feedback controllers. One is that sensing the output, preparing the sensed data for input to the controller (signal conditioning), processing the signal in the controller to generate the control input, and actually causing the control element (e.g. a valve) to change, all takes time.
Spring 2010 Spring 2010
Problems with feedback Feedback plus feedforward
There are lots of problems with practical There are many advantages in combining feedback implementations of feedback controllers. One is that and feedforward control. sensing the output, preparing the sensed data for input to the controller (signal conditioning), processing the signal in the controller to generate the control input, and actually causing the control element (e.g. a valve) to change, all takes time. During that time, the sign of the error could change due to the intrinsic dynamics of the plant (e.g. a spring-mass somewhere). The negative feedback will now be positive feedback, and the system will be unstable. Standard methods can usually handle this, but time delays can still be a problem for PIDs.
Spring 2010 Spring 2010
Feedback plus feedforward Feedback plus feedforward
There are many advantages in combining There are many advantages in combining feedback and feedforward control. feedback and feedforward control.
1) Feedforward control can deal with the main 1) Feedforward control can deal with the main requirements of the reference signal, so the requirements of the reference signal, so the feedback just has to handle the error between feedback just has to handle the error between the feedforward controller’s ‘expected’ output the feedforward controller’s ‘expected’ output and the actual output. This reduced demand can and the actual output. This reduced demand can improve the feedback controller’s performance improve the feedback controller’s performance and stability. and stability. 2) Feedforward control is fast because it doesn’t have to wait for the feedback signal. This can improve both responsiveness and stability.
Spring 2010 Spring 2010 Controlling non-linear systems Typical non-linearities
Linear controllers, like PID, are generally valid only On top of intrinsic system non-linearities some over small operational ranges in non-linear systems. typical non-linearities can be found in practically all However, there are techniques (e.g. gain systems. scheduling) for enabling them to cope with some non-linearity. Saturation
Non-linear systems (which include almost all Deadzone biological systems) often have multiple equilibrium points, plus periodic, or chaotic attractors. Small disturbances (even noise) can induce radically different behaviours. Hysteresis Non-linear control is a specialised and very difficult (maths-heavy) area. Spring 2010 Spring 2010
Adaptive control Adaptive control
As well as suffering from short-term There are dozens of different methods of disturbances, many systems undergo longer term adaptive control: direct/indirect model reference or permanent changes. Animals grow. Missiles burn adaptive controllers (MRAC), self-tuning fuel, and get lighter, and change their moments of controllers, model identification adaptive inertia. Joints become stiff. In these controllers, etc. etc. circumstances, it makes sense for the control system to adapt to the changes.
Spring 2010 Spring 2010
Indirect Model Reference Adaptive control Adaptive Control reference disturbance disturbance A feedforward reference model of the plant is dynamically refined as the system runs. May be + ε + +++ able to cope with various levels of uncertainty and Controller Plant adapt to different working regimes in non-linear systems. -
The model’s output is used for comparison with Reference - actual output. The error is used to estimate Estimated Model + modification in parameters following an adaptation parameter εm law; new parameters are fed back both to the values Adaptation controller and the model. law
Spring 2010 Spring 2010 Actor-critic architecture Neural network control
Revisited … Explicit memory structure devoted to representing the Different kinds of neural networks can be used as policy feedforward reference models, as estimators, as Makes it easier to select actions (don’t have to search predictors, or (less often) as controllers (both through each action-value pair) open- and closed-loop). More appropriate for control (as compared to prediction)
“Linear mindset” persists, and there are still very few practical applications using neural networks as non-linear controllers. (There may be legal reasons for this).
There is some interesting work on neuroevolution of control systems - Google “Gomez & Miikkulainen”
Spring 2010 Spring 2010
Fuzzy control Fuzzy control
Fuzzy controllers: Rule based controllers. Do not Inverted pendulum need to handle complex computations. Follows (rule set) scheme: Fuzzification, Inference, Defuzzification. NS=Negative Small Often quite robust. PL=Positive Large Z=Zero Degrees of memberships to fuzzy classes determine the strength of the rules that apply.
Spring 2010 Spring 2010
Robotics Robotics: controlled trajectories
A traditional method to control Amazing feats of engineering – e.g. Honda’s robot motion is to solve inverse dynamics of the robot, that is, to Asimo calculate the required torque to generate the desired trajectories.
Very computationally intensive as the number of degrees of freedom increases (sometimes even impossible). Also very brittle as many parameters (such as load, friction, etc.) vary with time.
But that's often how it's done!
Spring 2010 Spring 2010 Questions… Usual view Sensory inputs and internal drives provide setpoints for the brain which controls the body. How relevant is control engineering for understanding (Kawato on forward models in the cerebellum) natural adaptive behaviour? Depends on how far you take the metaphor of the brain as the controller of the body.
Spring 2010 Spring 2010
Usual view Emulation theory A basic schematic: Rick Grush:“.. in addition to simply engaging with the environment, the brain constructs neural circuits that act as models of the body and environment”.
“[m]odels can also be run off-line in order to produce imagery, estimate outcomes of different actions, and evaluate and develop motor plans”
Combines forward modelling with Kalman filters, …
Grush, R. (2004). The emulation theory of representation: motor control, imagery, and perception. Behavioral and Brain Sciences 27(4):377-442 Spring 2010 Spring 2010
Perceptual control theory Questions…
Isn't there a contradiction between thinking of an autonomous system as controlled? Is not autonomy PCT (perceptual control theory, W.T. Powers). the equivalent of self-law? Seth (2007) formalizes Behaviour controls perception by minimizing the autonomy as ‘self-causality’, a system that controls error between some desired perceptual setpoint itself. and actual perception. Setpoints are the outcome of other controlled processes that cascade back What about mutual feedback between nervous into the brain. system, rest of the organism, and environment? Can that be described using the language of control The reference variable for each negative engineering? feedback loop is set from within the system (the organism) rather than by a separate controller. Seth, A.K. (2007). Measuring autonomy by multivariate autoregressive modelling. In Proc. 9th European Conference on Artificial Life (ECAL2007), pp.475-485.
Spring 2010 Spring 2010