CONTROL THEORY
K<0, a>0 AND
ANALYSIS
LECTURES PRESENTED AT AN INTERNATIONAL SEMINAR COURSE K<0, a<0 TRIESTE, 11 SEPTEMBER - 29 NOVEMBER 1974 co-*0-f ORGANIZED BY THE INTERNATIONAL CENTRE \ FOR THEORETICAL PHYSICS i TRIESTE
VWÍ INTERNATIONAL ATOMIC ENERGY AGENCY, VIENNA, 1 976
CONTROL THEORY AND TOPICS IN FUNCTIONAL ANALYSIS Vol. I
INTERNATIONAL CENTRE FOR THEORETICAL PHYSICS, TRIESTE
CONTROL THEORY AND TOPICS IN FUNCTIONAL ANALYSIS
LECTURES PRESENTED AT AN INTERNATIONAL SEMINAR COURSE AT TRIESTE FROM 11 SEPTEMBER TO 29 NOVEMBER 1974 ORGANIZED BY THE INTERNATIONAL CENTRE FOR THEORETICAL PHYSICS, TRIESTE
In three volumes
VOL. I
INTERNATIONAL ATOMIC ENERGY AGENCY VIENNA, 1976 THE INTERNATIONAL CENTRE FOR THEORETICAL PHYSICS (ICTP) in Trieste was established by the International Atomic Energy Agency (IAEA) in 1964 under an agreement with the Italian Government, and with the assistance of the City and University of Trieste.
The IAEA and the United Nations Educational, Scientific and Cultural Organization (UNESCO) subsequently agreed to operate the Centre jointly from 1 January 1970.
Member States of both organizations participate in the work of the Centre, the main purpose of which is to foster, through training and research, the advancement of theoretical physics, with special regard to the needs of developing countries.
CONTROL THEORY AND TOPICS IN FUNCTIONAL ANALYSIS IAEA, VIENNA, 1976 STI/PUB/415 ISBN 92-0 —130076-X
Printed by the IAEA in Austria March 1976 FOREWORD
The International Centre for Theoretical Physics has maintained an interdisciplinary character in its research and training programmes in different branches of theoretical physics and related applied mathematics. In pursuance of this objective, the Centre has - since 1964 - organized extended research courses in various disciplines; most of the Proceedings of these courses have been published by the International Atomic Energy Agency. The present three volumes record the Proceedings of the 1974 Autumn Course on Control Theory and Topics in Functional Analysis held from 11 September to 29 November 1974. The first volume consists of fundamental courses on differential systems, functional analysis and optimization in theory and applications; the second contains lectures on control theory and optimal control of ordinary differential systems; the third volume deals with infinite dimensional (hereditary, stochastic and partial differential) systems. The programme of lectures was organized by Professors R. Conti (Florence, Italy), L. Markus (Warwick, United Kingdom) and C. Olech (Warsaw, Poland).
Abdus Salam
CONTENTS OF VOL. I
Basic concepts of control theory (IAEA-SMR-17/6)...... 1 L. Markus Concepts of stability and control (IAEA-SMR-17/1)...... 53 P.C. Parks Foundations of functional analysis theory (IAEA-SMR-17/2)...... 101 Ruth F. Curtain Control theory and applications (IAEA-SMR-17/3) ...... 179 A.J. Pritchard Finite-dimensional optimization (IAEA-SMR-17 /4 )...... 223 D. Q. Mayne Reachability of sets and time- and norm-minimal controls (IAEA-SMR-17/37)...... 261 A. Marzollo Existence theory in optimal control (IAEA-SMR-17/61) ...... 291 C. Olech Asymptotic control (IAEA-SMR-17/48) ...... 329 R. Conti Controllability of non-linear control dynamical systems (IAEA-SMR-17/71) ...... 361 C. Lobry Introduction to convex analysis (IAEA-SMR-17/63)...... 385 J.P. Cecconi An introduction to probability theory (IAEA-SMR-17/29)...... 419 J. Zabczyk Secretariat of Seminar...... 463
IAEA-SMR-17/6
BASIC CONCEPTS OF CONTROL THEORY
L. MARKUS School of Mathematics, University of Minnesota, Minneapolis, Minnesota, United States of America
Abstract
BASIC CONCEPTS OF CONTROL THEORY. After a philosophical introduction on control theory and its position among various branches of science, mathematical control theory and its connection with functional analysis are discussed. A chapter on system theory concepts follows. After a summary of results and notations in the general theory of ordinary differential equations, a qualitative theory of control dynamical systems and chapters on the topological dynamics, and the controllability of linear systems are presented. As examples of autonomous linear systems, the switching locus for the synthesis of optimal controllers and linear dynamics with quadratic cost optimization are considered.
I. WHAT IS CONTROL THEORY - AND WHY?
Just what is control theory? Who or what is to be controlled and by whom or by what, and why is it to be controlled? In a nutshell, control theory, sometimes called automation, cybernetics or systems theory, is a branch of applied mathematics that deals with the design of machinery and other engineering systems so that these systems work, arid work better than before. As an example, consider the problem of controlling the temperature in a cold lecture hall. This is a standard engineering problem fam iliar to us all. The thermal system consists of the furnace as the heating source, and the room thermometer as the record of the temperature of the hall. The external environment we assume fixed and not belonging to the thermo dynamic system under analysis. The basic heating source is the furnace, but the control of the furnace is through a thermostat. The thermostat device usually contains a thermometer to measure the current room temperature and a dial on which we set the desired room temperature. The control aspect of the thermostat is that it compares the actual and the desired temperatures at each moment and then it sends an electric signal or control command to the furnace to turn the fire intensity up or down. In this case, the job of the control engineer is to invent or design an effective thermostat. Let us next look at a control problem from biology. Parts of the world are being overrun by an increasing population of rats. Here the system consists of the living population of rats and the environmental parameters that affect that population. The natural growth of the rat population is to be controlled towards some desired number, say, zero. Here the job of the control engineer is to build a better mouse-trap. From this viewpoint control theory does not appear too sinister. On the other hand, it does not seem too profound. So let me elaborate on the
1 2 MARK US
structure of control theory to indicate the reasons why UNESCO and the ICTP believe this subject is important. To organize these ideas, I shall discuss control theory from four viewpoints:
(i) as an intellectual discipline within science and the philosophy of science; (ii) as a part of engineering, with industrial applications; (iii) as a part of the educational curriculum at university; (iv) as a force in the world related to technological, economic and social problems of the present and the future.
First consider the philosophical position of the discipline of control theory. Within the framework of metaphysics, control theory is a teleological science. That is, the concepts of control theory involve ideas such as purpose, goal-seeking and ideal or desirable norms. These are terms of nineteenth century biology and psychology, terms of volition, will and motiva tion such as were introduced by Aristotle to explain the foundations of physics, but then carefully exorcized by Newton when he constructed an unhuman geometric mechanics. So control theory represents a synthesis of the philosophies of Aristotle and Newton showing that inanimate deterministic mechanisms can function as purposeful self-regulating organisms. Recall how the inanimate thermostat regulates the room temperature towards the agreed ideal. Another philosophical aspect of control theory is that it avoids the concepts of energy but, instead, deals with the phenomenon of information in physical systems. If we compare the furnace with the thermostat we note a great disparity of size and weight. The powerful furnace supplies quantities of energy: a concept of classical physics. However, the tiny but ingenious thermostat deals with information - an aspect of modern statistical physics and mathematics. Thus control theory rests on a new category of physical reality, namely information, which is distinct from energy or matter. Possibly, this affords a new approach to the conundrum of mind versus matter, concerning which the philosophical journal Punch once r e m a r k e d , "What is matter? — Never mind. What is mind? - No matter. " But what are the problems, methods and results of control theory as they are interpreted in modern mathematical physics or engineering? In this sense control theory deals with the inverse problem of dynamical systems. That is, suppose we have a dynamical system, for example many vibrating masses interconnected by elastic springs. Such a dynamical system is described mathematically by an array of ordinary differential equations that predict the evolution of the vibrations according to Newton's laws of motion. The customary or direct problem of dynamics is the analysis of the physical system (to obtain the array of mathematical differential equations) and then the analysis of the differential equations to compute the solutions describing the vibrations. For example, we might try to locate all equilibrium states of the dynamical system and to compute which of these are stable. The inverse problem, that is, control theory applied to the vibrating system, concerns the question of synthesis as the inverse of analysis. Here we specify a goal and seek to modify the physical and mathematical systems to incorporate this goal. For instance, we might pick one of the known equilibrium states and insist that this must be stable so that, whatever the IAEA-SMR-17/6 3 current state of the vibrating system, it must tend towards the desired equilibrium . In terms of the physics this means that we seek to synthesize new forces (for instance, frictional damping) into the system to achieve the goal. In terms of mathematics we seek to add or synthesize new coefficients (of some specified type) into the differential equations to produce the required type of stability. In the simplest case the vibrating system consists of a single mass on a linear spring. If the displacement from equilibrium is x at time t, then Newton's law asserts that the acceleration d2 x / d t2 is equal to the spring force -x (using units in which the mass is unity) and we obtain the differential equation of motion
Can we introduce a control force u, depending on x and dx/dt, so that every solution of
dt^ = -x + u returns to the rest equilibrium state x = 0, dx/ dt = 0, after a finite duration? This is the problem of controllability. Can we find a control law u* (x, dx/ dt) such that the time of return to rest is of m inim al duration, as compared with the corresponding duration for all control forces u with magnitudes lim ited by a prescribed bound? This is the problem of optimal control. Thus the mathematical problem of control theory concerns the modifica tion of differential equations, within prescribed lim itations, so that the solutions behave in some desired specified manner. Moreover, control dynamics can refer to ordinary differential equations as in the case of the finite set of vibrating masses, or partial differential equations as in the case of temperature variations throughout a lecture hall. In fact, more complicated types of functional equations, linear or non-linear, deterministic or stochastic, finite- or infinite-dimensional can arise in control theory. The common feature of all these problems within control theory is that we prescribe the desired behaviour of the solutions and then we seek to modify the coefficients of the dynamical equations so as to induce this behaviour. Hence control theory takes its format from mathematics but also it gives to mathematics new types of stimulating questions and challenging problems. In classical mathematical physics, we know all about the physical laws guiding the development of the phenomenon we may be studying; we know all the rules of the game and we want to predict the outcome. That is physics. In control theory we do not know the rules, in fact we can change them within certain lim itations as we proceed, but we do know exactly how we want the game to end. Thus the mathematical problems of control theory are inverse to the usual problems of mathematical physics. We assume only a limited knowledge of our. differential equations, which are partially subject to our control, but we specify definitively the development of the system from the present state to the future goal. Now let us turn to the engineering side of control theory. Control theory is concerned with doing whereas classical physics is prim arily 4 MARKUS interested in understanding. Naturally, the best procedure would involve comprehension first and action second, but we cannot always hope for the b e s t. Often we must act to guide some process when we do not understand it thoroughly. For instance, we do not have a complete knowledge of the love- life of rats, yet we can hope to control the rat population effectively by a cleverly chosen system of traps and poisons. If one method is not effective, we can always try another until we meet success. You know the old saying, "If at first you don't succeed, try, try again - and then give up. There's no use in making a fool of yourself". But mathematical control theoreticians do not mind being fools provided their successive trials gradually converge to an effective functional control synthesis. The process of measuring just how far short of perfection each method falls, and then correcting the next method accordingly, is called the principle of feedback, a repulsive word for an important concept. Scientists did not invent the feedback control process; it is a phenomenon of nature. Almost all biological processes involve self-regulation through some biochemical feedback controls. In physiology this concept is called homeostasis. Using feedback control we can solve problems that we do not understand. This circumstance is what makes control theory such a powerful tool for technological problems in engineering and for economic and social problems in urban planning. Suppose that an engineer wishes to supervise an industrial plant making glass, or paper or steel. In practice, this is the main application of control theory, in controlling the quality of the product of a manufacturing process. Consider the process of making paper. Scientists know very little about the basic chemistry of wood pulp or fibre sludge, yet we demand that the final paper sheeting should roll out with a very precisely controlled thickness. So we monitor the thickness as the paper rolls out and, if there is an error, we can make successive corrections to the temperature of the pulp tank and the speed of the rollers, until the correct thickness is achieved. Since very rapid adjustments of the operating conditions are required, automatic measurements and computer regulation of the plant must replace inaccurate and awkward human beings. Thus automatic factories arise involving no labourers - but, instead, hordes of computer r e p a ir m e n . How can universities teach this science of trial-and-error, of input- output comparison and of control action upon the pragmatic principle of feedback rather than upon the rational methods of analyses? In the first place, we usually have some general or qualitative ideas concerning the dynamics of the plant we plan to control. Thus the basic engineering and mathematical techniques that must be taught are found within the qualitative theory of dynamical systems. Dynamical-system theory, or the study of changing physical quantities such as the positions of a system of vibrating masses, has been a central part of mathematical science since Newton. From the purely mathematical viewpoint, control theory falls under the domain of dynamical-system theory, with heavy overtones of statistics and probability and with some aspect of optimization, linear programming and the calculus of variations. Naturally, there is also a close relation to computer science since IAEA-SMR-17/6 5 engineering control processes often involve computer machines as part of the feedback loop through which information and control commands flow. Finally, I shall discuss some applications of control methods to socio economic problems. Let us take first an example from economics where control methods are well accepted and are used as an effective practical procedure, namely in the Keynesian macro-economics of a nation state. In the Keynesian fiscal theory all data, significant for the economic description of a nation, are aggregated into a few categories. These economic levels are such things as gross national product, unemployment, investment of heavy industry, production of consumer goods, and their numerical measures define the current economic state of the nation. The government seeks to control the economic state of the nation by regulating the level of taxation and the prim e interest rate on money. If the economy sags, then this information is fed back to the government who then adjusts the controls of taxation and interest to revive and improve the economic state. From the economic viewpoint the Keynesian control theory seems to work, but there are political and moral decisions that must be resolved. What is a desirable economic state? If there is an economic squeeze who should suffer during the re-adjustment? Should the burden be placed on poor unemployed young people, or on old-age pensioners crushed by inflation? This political or social decision overrides the economic programme. Nevertheless, the engineering control theory approach to macro-economics, as outlined by Keynes, gives us a method of attacking national economic p r o b le m s . In the micro-economic theory of a business company or firm we encounter other control methods and concepts involving personnel organiza tion, inventory, distribution and production. In fact, the interaction between manufacturing firm s and the national government will become more and more important as we require pollution control to become a part of the production process. Keynesian fiscal theory and the organizational theory of the firm are relative success stories for control methods within economic systems. But now we come to a tragic failure. Lake Erie is one of the Great Lakes of the United States. It is also one of the great sinks of industrial and human wastes. A major govern mental plan to clean up Lake Erie was initiated several years ago, but the plan failed. The technological difficulties were overcome with new types of industrial and sanitary purification plants, the economic problems were met with adequate m illion dollar budgets from national and state agencies, but the political problems were unsolvable. Lake Erie is surrounded by a variety of countries, districts, towns and cities each with a different political structure. Co-operation and compromise were necessary for the salvation of Lake Erie, but co-operation proved politically difficult. What is needed to clean up Lake Erie is a new commission with appropriate political connections and authority. Just how this commission should be constituted is not clear. This is a problem for control theoreticians of political science. New towns are now being planned in England. For each town the technological structure (road, transport, sewage etc. ) is carefully planned. The economic viability (industrial plants, railroads etc. ) is considered. But is the political organization of the town and regional government analysed as an integral segment of the over-all system design problem? Such 6 MARKUS political design questions sometimes are faced, bat in a rather unsystematic w a y . In summary, the influence of control theory on significant social problems proceeds through several levels indicated as follows:
POLITICS Feedback Synthesis with Logical Decisions ECONOMICS Optimal control with cost criteria ENGINEERING Control input-output design BASIC SCIENCE Dynamical Systems
On the most fundamental or bottom level of the four layers encountered in the control problem, we require basic science. This level tells us what is possible. For instance, we know the physical or biological or economic laws describing the dynamical systems we seek to control. The more we under stand about the world the better we can control our environment. The next level involves engineering theory and practice as a method of improving technological processes by controlling production. This level tells us what is feasible. In the third level of activity the economics of production must be analysed to find the best or optimal control, or at least a satisfactory control. This level tells us what is efficient. But we must still decide on the goal of our control system. The optimal must be defined in terms of a cost criterion that rests on human judgements and political and m oral principles. But more than this, the implementation of the economic-engineering-scientific control design depends on the decision-making capability of the commission (its logical powers of analysis) and on the political strength of its authority. Each person has his own limitations and his own interests. No one can range over the totality of the topics in the four levels of problem solving described above. Each has his own narrow expertise. Yet each one can appreciate how his efforts fit into a wider pattern, a continuum of knowledge. This interdisciplinary approach may offer a means of effectively attaching the behavioural sciences to the natural sciences with the glue of control theory. We can only try and wait and hope.
II. MATHEMATICAL CONTROL THEORY AND FUNCTIONAL ANALYSIS
1. CONTROL OF ORDINARY DIFFERENTIAL SYSTEMS
The mathematical formulation of control theory, within the framework of dynamical systems, involves an inverse problem. In the standard theory of differential equations, we are given the coefficients and we seek to compute the solutions; in control theory the solutions are prescribed (or some aspect of their behaviour is prescribed) and we seek the corresponding restrictions on the coefficients. From this general viewpoint, control theory resembles a boundary value problem rather than an initial value p r o b le m . The physical foundation for this inverse nature of control theory is that the descriptions of science according to Newton, Maxwell, and others, are IAEA-SMR-17/6 7 posed in terms of differential equations. The state of the physical system (mechanical displacements and velocities, or electrodynamical field strengths, etc. ) are computed upon integrating differential equations whose coefficients contain physical quantities such as force and charge. In other words, we must invert the differential operators to find the state produced by the controlling forces or electric charges. A different approach is often encountered in engineering, biological, economic, and social systems where the plant dynamics are unknown — no equations of Newton or Maxwell are available. In this case, we take a phenomenological approach and seek to investigate and control the system after preliminary experimentation or calibration. That is, we put in various control signals or forces, and then we measure or observe the state or some function of the state that is put out. By comparing the input with the output, we try to build up a model of the internal plant dynamics, and to determine the appropriate control. This is an approach of identifica tion theory and adaptive control. When we further seek the best control for some purpose, we encounter the theory of optimal control. It will be a principle goal of this course to compare these two approaches to control th e o ry : dynamical systems and optimal control input-output analysis and adaptive control The physical or engineering problem of control design is delimited only by experience, tradition, practicability, and ingenuity. However, in mathematical control theory we must clarify the class of admissible controllers a priori. We can seek controllers only within certain selected classes such as linear, or piecewise linear, which are often suggested by engineering examples and motivations. As remarked above, control theory, as interpreted within the framework of dynamical systems or differential equations, leads to problems that are the inverses of the classical mathematical investigations. The classical theory of differential equations deals with analysis whereas control theory deals with synthesis. In the classical approach to dynamical systems we are given the differential equations of motion, and then we try to analyse the behaviour of the resulting motions or solutions. In control theory we prescribe the desired behaviour of the solutions, and then we try to synthesize the differential equations to yield these motions. Of course, the procedure of synthesis means, in mathematical terms, that the basic form of the underlying differential equations can be modified by adjustment of certain control parameters or functional coefficients which are selected from certain admissible classes; whereas the synthesis means, in engineering terms, that the prim ary machine or plant can be modified by the adjustment of gains in feedback loops or the insertion of auxiliary devices of certain practical types. Hence, to each part of classical theory of differential equations, say stability or oscillation theory, there corresponds a field of control theory with inverse problems. For instance, consider the classical stability analysis of the damped linear oscillator
x + 2bx + k2x = 0 8 MARKUS with constant coefficients. This oscillator is asymptotically stable (in the sense that all solutions approach x = x = 0 as t -» + oo) if and only if b > 0 and k 2 > 0. As an inverse problem, assume к > 0 fixed and try to choose b > 0 so that the solutions are damped at the maximal rate. That is, define the cost or efficiency of the control parameter b to be
C (b) = m a x {R e Х г , R e X2} where X is any eigenvalue satisfying X2 + 2bX + k 2= 0. Then we seek to select b to m inim ize C(b). An easy calculation shows that the optimal control b* m inim izing C(b) is b* = k. It is interesting to note that this is the standard value for critical damping, and hence we see that this fam iliar physical adjustment is explained as an elementary result in control theory. As another illustration consider the forced oscillator
x + 2 bx + k 2x = sin ut for constants b > 0, к > 0, to > 0. Classical analysis shows that there is a unique periodic solution
x = A s in (ut +
A = [ (k2 -w2 )2 + 4b2io2 ]'*
For the inverse problem, fix к > \f2 and b > 0, and try to choose the frequency u > 0 of the control input sin ut so as to maximize the amplitude A(u) of the response output. An easy calculation shows that the optimal c o n tr o l u * m a x im iz in g А (и) = [ (k 2 - u 2 )2 + 4b2u 2 ]"* is u* = (k 2- 2b 2)^, w h ic h is assumed positive. Again we find this value fam iliar since u* is the resonating frequency, and hence this basic engineering tuning is explained as an elementary result in control theory. These control-theoretic results are interesting in that they illuminate well-known physical and engineering practice. Yet, they are not typical of the modern theory of control. In the next section, we comment on a standard formulation of modern control theory, as interpreted within the framework of ordinary differential equations. Later we examine other approaches through partial differential equations and other functional e q u a tio n s . In control theory we consider a process or plant or dynamical system described by a differential system,
x = f(x, u)
where x is the real-state n-vector at time t, and the coefficient f is an n-vector function of the present state x and the control m-vector u. For sim plicity we assume the process is autonomous (time-independent) and that f is continuous with continuous first derivatives for all x eR n and u GRm, that is IAEA-SMR-17/6 9
f: Rn x R m -> R n is in class C1, so that there exists a unique response x(t) from an arbitrary initial state xo for each controller u(t) on 0 S t S T. We might seek to control x(t) between given initial and final states in ■ some fixed duration 0 S t S T,
x(0) = x 0, x(T) = x 1 by choosing a control function u(t) from some admissible function class (say ue L^JO, T], that is, u(t) is a bounded measurable function on 0 á t ê T ). Hence x(t) is a solution of the two-point boundary value problem, with separated end conditions,
x = f(x, u(t)), x(0) = x0, x(T) = x x
This constitutes the basic problem of controllability in control theory. Among all solutions x(t) to this boundary value problem, that is for all admissible control functions, we might try to select and describe the. optimal solution x*(t) for the optimal controller u*(t) which minim izes some given cost or performance functional C(u), This leads to the central problem of optimal control theory, for which there is a vast literature. Next let us examine these general concepts for more specific problems and particular examples.
Example 1. Consider a linear vibrator, say, an airplane wing whose vertical vibrations are to be controlled by aerodynamical wing-tabs (Fig. 1). We select suitable units (m = к = 1) so the free-dynamics system is x + x = 0 and the controlled system is x + x = u. This is equivalent to the differential system
x = У
f = x + u
where x is the displacement, у is the velocity, and u is the control force.
Here the state or ГХ^ is a vector in the real number plane R2, and
FIG .l. Airplane wings as linear vibrator. 10 MARKUS the control u is a real scalar in R 1 = R at each time t e R. In vector notation, we write x = x1, y = x2,
x 1 = x 2 = f 1 (x 1, x 2, u)
x 2 = -x1 + u = f 2 (x 1, x 2, u) and the matrix notation for this linear system is
For each initial state ( X° ) in R 2 at time t = 0, and each controller ЧУо / u(t) on t § 0, there is a unique solution or response x(t, x0, Уо)> y(t, x 0,y o ) . The choice of u as a function of t, i. e. a pre-programmed control strategy, corresponds to open-loop control. A more important physical approach, although more difficult mathemati cally, is to use closed-loop control where u (x, y) depends only on the state at each moment. In this case, the control depends on the feedback loop carrying the current state back to influence the future evolution of the
differential system, and the physical system is self-correcting in case unforeseen disturbances arise from time to time.
CONTROL PLANT RESPONSE CONTROL RESPONSE INPUT OR PROCESS OUTPUT INPUT OUTPUT
u(x,x) u (t). x+x = u 1 * 1 t
OPEN-100P CONTROL — 4 ------1 [■<- - и 1----- ’ x CLOSED-LOOP CONTROL
FIG.2, Closed-loop and open-loop control.
Let us examine the simple regulator problem of controlling an initial state (x0, y0) e R 2 to the origin (0, 0), Using a closed-loop feedback control, say u = - y, we can make the origin asymptotically stable for
x + x = -x
o r
x = у
ÿ = -x - y
That is, for each solution x(t), y(t) = x(t) the curve x(t), y(t) approaches the origin in R 2 as t -» -l-оо. Yet the origin is not attained in any finite time. Moreover, using pure feedback control u(x, y), say u(0, 0) = 0 and u(x, y) e C1, 1AEA-SMR-17/6 11
F IG .3 . Phase-plane system with u = 0, u = + 1 and u = - 1 .
it is im possible to steer (xo, yo) to the origin in a finite time (since the origin is then an equilibrium or critical point for the autonomous differential system x + x = u(x, x)). A more difficult mathematical analysis provides a discontinuous feedback controller that regulates all solutions to the origin in finite duration, but we defer this discussion until later in the course (Fig. 2 ). Now, try to regulate the linear oscillator by an open-loop controller u(t), which can be discontinuous if necessary. If we sketch the phase-plane system with u = 0, we find a family of periodic orbits encircling the origin. For u = +1 or u = -1 we find sim ilar portraits but merely displaced one unit toward the right or left, respectively (Fig. 3). W ith a simple scheme of switching between the values u = +1 and u = - 1 each initial point can be steered to the origin in finite time. A more involved study shows that each (x 0,y o ) in R 2 can be steered to any required final state (хц, yi) in R2. In this sense the system is controllable in the plane.
Example 2. Consider the double integrator describing a free accelerator according to the linear control system
x = uorx = yo r/x \_ /o i\/x \ /o\ У = u \ y ) " \ 0 0 ) \ y ) \ l ) u
If u = 0, then the velocity y is constant and the displacement x increases linearly with time t. It is easy to verify that this system is also controllable in the (x, y)-plane and that each initial state (x0, y0) can be regulated to the origin in finite time by some open-loop controller. Now, consider the control u applied directly to the velocity rather than the acceleration according to the system (Fig. 4):
x = y + u
ÿ = 0
This system is not too easily interpreted in physical terms but it sometimes is used to describe the motion of a particle along a moving platform, where у is the velocity relative to the platform and u is the velocity of the platform along the ground. 12 MARKUS
У
FIG.4. Control applied to velocity.
It is easy to see that this system is not controllable since у = y 0 r e g a r d less of the control u. However, suppose that only the displacement x is to be controlled, and the velocity у ignored. That is, there are observations z
according to
Then the observed output z can be controlled by u. Once again let us look at these control problems from a more general viewpoint. Consider the control system
or, in vector language, x = f(t, x, u). We often concentrate on the autonomous case where f does not depend explicitly on t so
x = f( x ,u ) where the smooth (highly differentiable) coefficients f(x, u) define the basic dynamics of a physical process or plant. The state of the process is the real n-vector x, and the control is the real m-vector u, at each time t. Consider the regulator problem of choosing a control u(x, t) so that every solution of
x = f(x, u(x, t)) tends to the origin, x(t) -> 0 as t -* +oo. If u = u(t), the control is called open-loop and corresponds to a pre-programmed strategy, which usually depends on knowledge of the initial state. If u = u(x) this is closed-loop feedback control and is sometimes called a synthesis for a stabilizing c o n tr o l. In case the control system is linear, we write
x = A x + Bu for x e Rn, u e R m, and the real coefficient matrices A(t) and B(t) are known. Let us discuss here the autonomous linear control problem where A and В IAEA-SMR-17/6 13 are real constant matrices. The feedback regulator problem consists in finding some real constant matrix К (often called the feedback gain) such that u = Kx stabilizes the linear system, i. e.
x = Ax + BK x = (A + BK) x has all solutions x(t) -> 0 in Rn as t -* +00. It is well-known that a necessary and sufficient condition for such stability is that all eigenvalues X of (A + BK) have negative real parts (lie in left-half complex number plane). Later we shall find conditions on the m atrix pair (A, B) that guarantee the existence of some stabilizing gain matrix K. Suppose the linear autonomous control system
x = A x + Bu is accompanied by a given observation relation
z = C x for the observed output z specified by the given real constant matrix C. Let us examine this total system from the input-output analysis of experi mental investigations. For each input u(t), in some appropriate set of control functions, we generate a state response x(t) and then an observed output z(t) = С x(t). The input-output relation or transfer function is described by
u(t) -*• z(t)
To be more explicit, let us restrict u(t) to periodic functions, say complex exponentials, with amplitude U(io) for frequency и
u = U e ian and take the corresponding steady-state periodic response
x = X e iuJt, z = Z eiwt
Here the differential system becomes
(iw) X = A X + BU or the complex response amplitude, for U = 1, is
X = (i wl - A )-1 B, Z = С (iul - A )’ 1 В .
This is the amplitude-frequency response relation. If we allow the frequency to be any complex number p, then
X = (pi - A ) '1 B, Z = С (pi - A )’ 1 В which is the transfer function (or matrix). Note that
T(p) = С (pi - A )’ 1 В 14 MARK US is a rational function with poles at the eigenvalues of A. The basic problem of system identification concerns the computation of the matrices А, В, С from the transfer function T(p). In particular, if for all complex p
Ci (pi - A j )"1 B j = С (pi - A ) '1 В when can we conclude that A = А г, В = B1( 0 = 0!?
2. FUNCTIONAL ANALYSIS APPLIED TO CONTROL THEORY
Functional analysis is a method of treating analytical problems in differential and integral equations by means of geometric methods. While the same calculations might be performed without using the language of functional analysis, the geometric concepts prove very useful in suggesting various possible approaches to the problems. For example, consider the collection of all real-valued continuous functions of a real variable on the unit interval, say f(t) on 0 s t 6 1. T h a t is, we consider the space С [0,1] of all continuous maps of [0, 1 ] in to R . By classical analysis, С [0, l] is a real vector space. If we define the distance between two members f and g of the space С [ 0, 1 ] by
II f - g II = m ax |f(t) - g(t) I o s t s i then С [ 0, 1 ] becomes a complete m etric space, in fact, a Banach space with the norm ||f|| = ||f - o||. The Weierstrass approximation theorem asserts that the collection P of real polynomials on 0 á t s 1 is dense in С [0, 1]. Thus the language of functional analysis permits us to interpret this classical uniform approximation result as the assertion that P is a dense linear subspace of С [0, 1]. Note that this geometric approach is convenient but m ust be followed with care — since in Rn a dense linear subspace must coincide with all Rn . Functional analysis is particularly useful in control theory since we must often deal with a space ^ o f control functions u(t) and a space ^"of response functions x(t). The transfer function is then some map
5Г: u(. ) - x(.) which can only be studied precisely when the detailed properties of the spaces and ^ a r e specified. For instance, since discontinuous controllers are often employed the space ‘й'might be L j [ 0, 1 ] (integrable functions on 0 s t s 1 ) or perhaps some other class. Let us briefly consider the famous linear-quadratic problem of optimal control theory in order to appreciate the connections with functional analysis. Consider the linear autonomous control system
i = A x + B u
for the state x e Rn and control u e Rm at each time t. We seek an open loop controller u(t) on 0 s t s 1 to steer an initial state x(0) = x 0 to the origin x(l) = 0. Moreover, we judge the performance of each such controller, steering xq to 0, by a cost functional IAEA-SMR-17/6 15
i
C (u) = f I u(t) I2 dt 0 I 12 I i| 2 which depends on a quadratic integrand |u| = > |uJ| . i=1 The optimal controller u*(t) is required to m inim ize this cost among all competing controllers. Here, u(t) ranges over some subset S of the linear space L 2 [0, 1] of l all m-vector functions for which f |u(t) | 2 dt < o o. For each u(t) in S we и о have the real-valued cost C(u), and it seems reasonable to expect the optimal controller u*(t) to satisfy the necessary condition
= 0 at u = u*(t) du
But how is this functional derivative to be defined or computed? We-shall answer these questions later in the paper. The response x(t) to any controller in S is
x(t) = eAt x 0 + eAt / e"AsBu(s) ds
0 = eA x 0 + e A J e~As Bu(s) ds о
If we define the linear operator L carrying L 2 [ 0,1 ] to R n,
l L u = J e"AsBu(s) ds о ‘ then our problem is to solve the linear system
L u = x0
1 1/2 while m inim izing the norm ||u||llu II = / |u(t)|2 dt
Since the set of all possible solutions u e L 2 [ 0, 1 ] of the linear system L u = x 0 is some linear sub-manifold S (if not empty), we must find the point u* e S nearest to the origin of L 2 [0, 1]. That is, u* is at the point of S which is tangent to the sphere of radius ||u* || about the origin. This geometric approach will prove useful in solving this important problem later in the course. 16 MARKUS
III. SYSTEM THEORY CONCEPTS
The basic stages in the construction and application of any theory of control systems are:
(i) Modelling (ii) Identification (iii) Controllability and Stability Analysis (iv) Optimization for Optimal Control (v) Implementation.
In the first stage (i)we model the physical process by a mathematical dynamical or control system and in the last stage (v) we reinterpret our mathematical results in terms of engineering hardware. Thus these opening and closing stages are not strictly within the domain of mathematics and we concentrate on the middle three stages, identification, control and stability analysis and optimization, in this course. (i) Naturally the physically significant variables and relations affect the choice of the mathematical model and all the further control analysis. For instance, should a certain process be described by a linear autonomous control system in R n
x = A x + Bu 0 S t < oo or, perhaps, a discrete-time difference scheme
x t+1 = Axt + B ut t = 0, 1, 2, . . . would be more appropriate? In the modelling process the structure of the particular type of dynamical system is selected, say, ordinary or partial differential system, linear or non-linear, deterministic or stochastic. For example, the vibrations of a string can be studied by the wave partial differential equation
92w _ Э2 w 3 t 2 Эх2 where w(x, t) is the displacement of the string at position 0 S x S 1 a t tim e t i 0, Suppose one end is fixed w(0, t) = 0 and the initial state is given w (x, 0) = w 0 (x), (9w/ 9t) (x, 0) = v 0 (x) on 0 s x í 1 . Then we can apply the control u(t) at the endpoint x = 1 by, say,
w (l, t) = u(t) and seek to bring the vibrating string to the rest state w(x, T) = 0, (9w/9t) (x, T) = 0 at a later time T > 0. This is a form of boundary control, as distinct from distributed control where we modify the partial differential equation according to IAEA-SMR-17/6 17
Another approach would be to consider the string as a chain with a finite number of rigid links, and to treat the dynamics as a high-order system of ordinary differential equations. The choice of the mathematical model depends on the physical system, and on some decision as to what are the significant features and the negligible aspects. In another type of problem, say, the population dynamics of a community we might decide that the population P(t) grows according to the law
P(t) = a P(t), P(0) = P 0 where a > 0 is the net birth-rate. A control might be introduced by an emigration rate u(t) so
P = a P - u(t)
However, let us suppose that the birth-rate is not constant, but depends on the size of the previous generation a = a -f3 P(t-l), for constants a > 0, j3 > 0. Then we have a differential-delay equation
P (t) = [a-/3 P (t- l) ] P (t) - u(t)
A still more complicated assumption on the birth-rate leads to other differential-functional equations such as
o 0
-1 -1
Let us note that partial differential equations, such as the wave equation or heat equation, and differential-delay or differential-functional equations can often be analysed as dynamical systems in appropriate infinite-dimensional spaces. For example, the initial state W 0 = (w 0(x), v 0(x)) o f
becomes after time t > 0 the state Wt = (w(x, t), (3w/3t) (x, t)), a pair of real valued functions on 0 S x S 1. Hence, the map Tt of the state space is defined by
T t : W 0 - W t and the basic flow condition is satisfied
Tt2 ° T , Tt2 + ti for t j i 0, t2 È 0
For the differential-delay equation in R
x(t) = f(x(t), x(t-l)) the initial state is a given continuous function x o(0) on -1 s 0 s 0, Then the future solution x(t) is defined for t S 0 and the state x at time t > 0 is the 18 MARKUS past unit-time segment of x(t) that is, xt(0) = x(t + 0) on - 1 â 0 s 0. Hence the solution x(t) in R can be defined by a flow Tt from xo(0) to xt(0), say elements of the function space С [-1,0]. That is, consider
Tt : x0(.)-*• xt(. ) again satisfying the basic flow condition for future times t s 0, Of course, the precise specification of the function space of states and the appropriate metric or topology is important in any serious study of these flo w s . (ii) From input-output experiments (perhaps analysed statistically) we ' identify the control system, within some accepted class of systems. For instance, from a rational transfer function matrix T(p) can we compute the matrices (А, В, C) such that
T(p) = С (pi - A )"1 В
Note that two systems for T(p) = — = -% P
and
Yet, the first is a minim al model, for a state space x e R, and seems preferable. It turns out that the m inim al model, for a given transfer function, is characterized by the properties of being controllable and observable. Roughly speaking, the system is controllable in case the origin can be controlled to every point of the state space. Also it is observable in case no two different state trajectories xx (t) = x 2 (t), for fixed control, can yield the same observation z(t). (iii) and (iv). We omit any further discussion of these topics here since they have been illustrated by some earlier examples, and they will form the central part of the development of the course.
IV. GENERAL THEORY OF ORDINARY DIFFERENTIAL EQUATIONS, SUMMARY OF RESULTS AND NOTATIONS
We shall prim arily be interested in first-order ordinary differential s y s te m s
j T i — = f^t.x^x2, ..., xn) i = 1 , . . . , n or, in vector notation,
x = f (t, x)
Here, x is a real n-vector at each time t and we seek a solution x(t) = = f(t, This is the Cauchy or initial-value problem, which has a unique solution x(t) under very general conditions on the coefficient n-vector f(t, x), as stated later. If the system is autonomous, so the coefficient vector f = f(x) is independent of the time t, then we usually denote the solution initiating at xo w h e n t 0 = 0, by x = x = f(x) satisfying the initial data x0when t = t0. In the special case when f(t, x) is linear in x we can use m atrix notation x 1 = aJ(t)xJ + b'(t) (sum on repeated index j) o r x = A(t) x + b(t) When b(t) = 0 we have a homogeneous linear system. If A and b are constant matrices, the linear system is autonomous, and the solution is 0 Let us note that a scalar higher-order ordinary differential equation can be reduced to the study of a first-order vector system by a suitable change of notation. That is, the scaler equation, (with the real-valued fu n c tio n can be w iitten in terms of the variables x = x 1 and (or x = x2, x = x 2 = x 3, e tc .) 20 MARKUS Then the n-vector has a time-derivative denoted by the n-vector which we can write as / f ^ t , x)\ f(t, x) = f 2 (t, x) \f“ ( t ,x ) / (and we usually omit the underlining of the vectors). Note that a real-valued function x(t) whose n-th derivative exists and satisfies the scalar equation always determines a n-vector x(t) whose first- derivative exists satisfying the vector system, and vice versa. Also note that the degree of differentiability of the coefficients and the properties of being autonomous or linear are transferred straightforwardly from the n-th-order scalar ordinary differential equation to the vector system. Because of the greater symmetry and generality of the vector system, we make this format the principal mathematical object for study. In any careful analysis of ordinary differential equations we must declare the hypotheses of differentiability on the coefficient vector f(t, x), and the domain in which the solutions are defined. For instance, consider the non-linear differential equation in R 1 x = (x)2 with initial condition x = x 0 > 0 at t = 0. Note that the coefficient is a quadratic polynomial, hence differentiable to all orders everywhere in R1, Yet the solution is given by 1 . 1 . 1 , _ 1 -- + — = t so x = —----- fo r t < -- X x 0 2 _ _ t x 0 Xo dx (upon separating the variables, as in ^ = dt). Hence the solution x(t) exists only on a time duration r. < t < t + w h e re t_ = -oo a n d t + = l / x 0 . This elementary example shows that we can only expect local solutions, even when the coefficients are very smooth. In control theory, where the coefficients may be discontinuous in t, other complications arise. We first state a "smooth" existence theorem, and then indicate the modifications that lAEA-SMR-17/6 21 must be made when incorporating control dynamics. The proofs of these basic theorems are found in advanced texts. We first consider a vector autonomous differential system x = f(x) with f(x) in class C k in an open set 0 с Rn, for some differentiability order к = 1 , 2 , . . . , oo. Here a vector (or a matrix) is in a class Ck in case each of its components is continuous in 0 , and there has continuous partial derivatives of all orders s к (or all orders if we take к = o o ). Of course, a real analytic function (written fe C ") specified in each locality in © by a n absolutely convergent power series, is necessarily in class C“. Theorem. Consider the vector ordinary differential equation in an open set 0 c R " , 5 П x = f (x) with f(x) in C k {0 ) for some fixed к = 1, 2, 3,. .. , o o , u . L e t x 0 be a given initial p o in t in 0 . Then there exists a solution x = (p(t,xo) o f^in 0 , initiating at ¿(t) = ü ( Remarks on global solutions. Let x = cp(t, x0) on т. < t < т+ be a solution of SP in 0 с R n. I f t + < + oo, then x(t) must eventually leave and stay out of any prescribed compact set К с 0 as t -» r+. In particular, when 0 = Rn and x(t) remains bounded for t > 0, then we can conclude that t + = + o o , and a sim ilar condition enforces r. = - oo. T h u s if |f(x) I is bounded (or even grows linearly with |x|) then every solution x(t) is defined for all times t e R 1 in 0 = R n . Remarks on dependence on parameters Next consider the differential system in Rn x = f(t, x, X) for a constant parameter X in Rm. We reduce this non-autonomous system to an autonomous differential system in a space of 1 + n + m dimensions. That is, let (t, x , X) be the co-ordinates of a vector in R 1+n+m and consider "¡7 = f(t - t 0,x ,X ) T"dT = 1 dT 22 MARKUS for f(s, x, X) e Ck in an open set, say R 1+n+m. Here we prescribe the initial data t = t 0, x = x0, X = X 0 at т = 0, and denote the unique solution by x = Ф(т, t 0, x 0, X 0), t = t + t 0, X = X 0 T h e n x = ф {t - t0, t0, x 0,X 0) is the required solution of x = f (t, x, X0 ) passing through x 0 when t = t0. Note that this vector function of (t, t0,X 0 ,X0) is in class Ck in an open subset of R1+1+n+m. It is this theorem which justifies differentiation of the solution with respect to initial data and parameter components. However, for even the simplest control systems we encounter more technical problems. Consider in Rn x = f(x, u) where the n-vector f(x, u) is in class Ck for the state n-vector x and the control m-vector u. Now if u(t) is a controller in Ь„[0, T] (bounded measurable on the interval 0 s t s T) then f(x, u(t)) is not necessarily continuous in t. However f(x, u(t)) and (9f/ Эх) (x, u(t)) lie in L„[ 0, T] for each fixed x e R n (since |f(x, u) | and |(3f/Эх) (x, u) | are bounded when |x| and |u| are bounded) and f(x, u(t)) is in Ck for each fixed t. These conditions are enough to ensure the existence of an absolutely continuous solution x(t), whose derivative exists everywhere on 0 s t s r+ s T excepting a set of times of measure zero, and which then satisfies the differential system. We emphasize that x(t) is a continuous curve in Rn and the possibility of corners, or discontinuities of x(t), causes no special difficulties. We state the fundamental existence and uniqueness theorem needed in control theory, and refinements can be found in the text of Lee-Markus. Theorem. Consider the n-vector differential system 50 x‘ = f‘(t, x 2, x 2, . . . , x n) i = 1 , . . . , n where f(t, x) is defined for t in an open interval/ с R 1 and x in an open set О C R n , A s s u m e (a) For each fixed t e /’the functions f 1(t, x) a re in C x{@), (b) For each fixed x € © the functions fl(t, x) are measurable in t in f , (c) Given any compact sets and К © there exists an integrable function m(t) on such that ! f(t, x) I g m(t) and lf“ (t»x)| s m (t) for all (t, x) e x К (note J m (t)d t < o o ) . ■>*c Then for each initial point (t0, x0) in /х й there exists a unique solution of Sf-. lAEA-SMR-17/6 23 x = T-(tо» x 0) < t < r+(t 0, x 0) Moreover;the n-vector function = f(t, t for all т. < t < t + . For each fixed (t, to) the function d ( Э tpl \ V a f 1 ,, ... .. ( dip11 dt =Л ЭР *о'хо)) 4=1 Example. Control of non-linear vibrator with one degree of freedom Consider a particle with displacement x along a straight track at time t. Newton's equations of motion for the particle of mass m assert m i = F We assume that the total force F is the superposition of an elastic spring force Fs = - g(x), a frictional force Ff = -f(x, x), and an external control force Fc = u(t). Then, taking units where m = 1 for convenience, the control dynamical system becomes x + f(x, x) + g(x) = u(t) or introducing the velocity y, & = y у = -f (x, y) - g(x) + u(t) We impose the following physically motivated assumptions: (i) f(x, y) and g(x) are in C 1 in the (x, y)-phase plane R 2 and u(t) is bounded and measurable on 0 s t < » , 24 MARKUS (ii) xg(x) > 0 for all x f 0 (restoring-force condition) and lim G(x) = oo where X N - * “ G (x ) = f g (s )d s . о (iii) yf(x, y) È 0 for all (x, y) (frictional-damping condition.) By assumption i) there exists a unique solution x(t, x 0,y 0), y(t, x 0, y 0) initiating at (xo,y0) in R 2 , and defined for some maximal future time t + > 0. We shall next show that conditions ii) and iii) guarantee the global existence of all such future trajectories, that is, t + = + o o . For this proof we pick a finite time T > 0 and show that the trajectory must remain within a certain compact subset КС R 2 on the duration 0 S t s m in(r+, T). From this it follows that т+ > T, according to the general theory summarized above. In the (x, y) phase-plane define the (energy or Liapunov) function V (x, y) = ^ + G(x) Then each level curve V(x, у) = с (where с is a positive constant) is a simple closed curve encircling the origin in R 2 . We compute the time derivative of V(x, y) along a solution curve x(t), y(t) by ~dV i7x+i7y 9V . 8V . = g(x)y , . + yí-f-g , , + u(t)] H e n ce V = — yf (x, у ) + yu(t) Note that the inequality V S 0, which holds for the free motion where u(t) = 0, already shows that each free trajectory lies within the level curve V(x, y) = V(x 0,y 0) and so is defined for all future times. But we must allow controlled motion with [ u(t) | â u 0 (the essential least upper bound on 0 â t < oo). For any controlled trajectory V s ! У I uo g ( ' I - + u 0 S (V + 1) |u0 I The along the controlled trajectory ^ (V + 1) s (V + 1) u 0 so by the usual inequalities we find, for all times in 0 S t S T, V + 1 S [V(x 0, y 0) + 1] eu°T = Cl H e n ce V(x(t), y (t) ) S [V(x 0, y Q) + 1] eu°T = Cl IAEA-SMR-17/6 25 and so the future trajectory lies within the simple closed level curve V(x, y) = c-l and must be defined on the full interval 0 S t i T . T hu s t + > T for each T > 0 and so t + = + oo. V. QUALITATIVE THEORY OF CONTROL DYNAMICAL SYSTEMS Let us return to the study of autonomous differential systems in Rn, S f) x = f(x) For sim plicity of exposition we assume throughout this section that f(x) e C 1 in all Rn, and that each solution x(t) = Фс : R n -> R n : x 0- x t = By the uniqueness and regularity theorems of the previous section we know ' th a t Ф с is a bijective map of Rn onto Rn, and both Ф с and its inverse map $ .t are differentiable of class C1. Thus Ф( is a C1-diffeomorphism of Rn onto Rn for each given real number t (assuming that each trajectory is defined for all times). Moreover, because the system J^s autonomous we can conclude that x = f(t, x) with f(t + P, x) = f(t, x) for some period P > 0, the map 26 MARKUS x 0 - к -» Фкр = (Ф р)к fo r к = 0, ±1 , ±2 , . . . is a homomorphism of the integer group Z into Diff (Rn). By means of the geometry of the flow #t (or the discreteflow (ФР)к for periodic systems) we can analyse the qualitative properties of the solutions of the autonomous system ^in Rn. For example, let us consider two kinds of stability for5f Liapunov and structural stability, as discussed next. Let us study the behaviour of the solutions of Sf) x = f(x) • (f(x) in C 1 in R n ) near a critical (or equilibrium) point x 0 where f(x0) = 0. For simplicity tak e x 0 to be the origin so ^ v = f( ♦1 - Ш ( ° Ы i = l , . . . , n or in matrix notation Of v = A v w h e re A = (0) It seems plausible that if all linearized solutions v(t) -» 0, then the non-linear system w ill be asymptotically stable about the origin. This conjecture is true (we return to the details later) and hence we conclude thatá^is asymp totically stable at the origin provided all eigenvalues Aj of the matrix A lie in the left-half complex plane, that is Re Xj < 0. Next let us contrast Liapunov stability with structural stability of a system5*in Rnv The first concept deals with stability of some equilibrium point (or an invariant set) under perturbations of the initial data, and the second refers to stability of the fam ily of all solutions of Sunder perturbations of the coefficients of the differential system. Let us compare the two differential systems in the (x, y) plane IAEA-SMR-17/6 27 The first is the conservative harmonic oscillator x + x = 0, and the second is a damped oscillator x + x + x = 0. Both are stable about the critical point at the origin, in fact the damped oscillator is globally asymptotically stable. Yet the stability of can be destroyed by an arbitrarily small perturbation, say x = y + e x 3 e > 0 y = -x + e y3 where the polar radius r = (x2 + y2)1^2 in c r e a s e s , ^ ( ^ / ¿ ) = xx + yÿ = xy + ex4 - xy + ey4 = e(x4 + y4) > 0 But consider a sm all C 1 -perturbation of 5^, say, X = У + Cj (x, y) y = -x - y + e2 (x, y) 2 w h e re ^ | e j (x, у) | + | Эе j /Эх | + | д е ^/д у | < e(x, y) for a suitable continuous bound о e(x,y) > 0 in R . It can be shown that when e(x, y) is suitably small, every such perturbation of still has the same qualitative form for its solution curve family in R 2 ; namely, a single critical point towards which all solution curves tend as t increases. To summarize the above comparisons, we define5^to be structurally stable in Rn if its solution family is topologically unchanged whenever ^ is modified by a suitably sm all C 1-perturbation. Then 5^ is structurally stable in R 2 , b u t J f is not structurally stable. Finally, let us consider a control system in Rn x = f(x, u) for state x e Rn and control u e Rm at each time t. We assume f(x, u) in C 1 in all R n+m and use controllers u(t) in L„[0, T] for some finite duration 0 S t á T. We shall define the control system to be (completely) controllable on [ 0, T ] in case: for each pair of points x0and xx in Rn there exists a controller u(t) for which the response initiating at x 0 terminates at X} . T h a t is, the solution x(t) = x(0) = x0, x(T) = x: so u(t) steers or controls x(t) from x 0 to X j, We shall show later that a linear autonomous control system in Rn x = Ax + Bu 28 MARKUS is controllable if and only if r a n k [ В, A B , А 2 В , A 3 B , . . . , A n_1 B] = n For general non-linear systems in Rn no such useful controllability criterion is kn o w n . Example. Consider the system in the (x, y) plane R 2 x = y ÿ = x (corresponding to x - x = 0) The origin is the unique critical point in R2. An elementary sketch of the solutions shows that the origin is a saddle point and hence it is not Liapunov s ta b le . It seems plausible that this linear system has its qualitative appearance unchanged when the coefficients are slightly perturbed. This assertion is correct (although a difficult theorem) and so the linear saddle system is structurally stable in R 2 . Next consider the control system x - x = u o r H e re A = [ ° M , В = ( ° ), and rank [В, AB] = rank ^ = 2 о Thus the linear system is controllable in R . VI. TOPOLOGICAL DYNAMICS. FLOWS IN METRIC SPACES To construct a general theory of qualitative dynamics applying to ordinary, delay, and partial differential equations we allow the state space M to be an arbitrary metric space on which the dynamics evolves continuously with the time t e R. Thus, there is a distance metric in M satisfying the usual axioms. Definition. A topological flow Ф in a metric space M is a map (continuous in two arguments): Ф : R x M -* M : (t, x0) -> xt = q>(\, x 0) 1AEA-SMR-17/6 29 such that: for each t 6 R we note the restriction to M Ф£ : x 0 - xt is a topological map of M onto M and further Фо = id e n tity , Ф-t = (Ф ^ ’ 1 and more generally (group property) Rem ark, t -> Фг is a homomorphism of R into the group Top(M) of all topological maps of M onto itself. (a) If this homomorphism is defined only on the semi-group R,.(tsO) into the group Cont(M), then this defines a (future) semi-flow in M. (b) If Ф is defined only on a neighbourhood of the slice t = 0 in the product space R x M, then Ф is a local flow. Note that a local flow Ф is defined for |t| < e uniformly for all x 0 in a prescribed compact subset K c M . Of course, the concept of a future local flow is immediate. E x a m p le s . 1) Consider the autonomous ordinary differential system in Rn i = f(x) with f(x) in C 1 in Rn. Then the solutions : x o - x t = x o) define a local flow. If |f(x) | < k|x| in all R n, then the flow is globally defined for all t € R (and this also happens in important cases which do not satisfy a linear growth hypothesis). 2) The non-autonomous ordinary differential system in Rn x = f(t, x) w ith f(t, x) in C 1 in R1+n defines an autonomous system and flow in R1+n. Namely, write g = f ( t , x ) and then the solutions yield a local flow in R1+n: Ф т : (t0, x 0) ~ (t + t 0, <р(т + t 0, t 0, x 0)) 30 MARKUS If f (t + P, x) s f(t, x) is periodic in t, then the flqw can be defined in the product of the circle S 1 w ith R n . 3) A differentiable n-manifold M is a metric space which is covered by a fam ily of local coordinate charts, each of which maps an open set of M topologically onto an open set in Rn, with the additional requirement that the coordinate transformations defined between overlapping charts should be in class С . Thus differential systems can be defined globally on M by defining them in each coordinate chart (x1, .. ., xn) as x1 = f‘(x), i = 1 , . . . , n with the usual (contravariant or tangent) vector transformation rule in over lapping (x1, .. ,, xn) to yield xl = f‘(x) i = 1 , . . . , n w ith We can always picture an n-manifold as an n-surface differentiably embedded in some Euclidean space (actually R2n), and then an ordinary differential system on M is merely a C1-vector field f everywhere tangent to M. Then the solution trajectories of f define a local flow on M. If M is a compact manifold (say a sphere Sn or a torus Tn), then each vector field f defines a global flow on M. 4) Consider the ordinary differential-delay system in Rn x(t) = f(x(t), x(t-l)) with the n-vector f(x, y) in C 1 in R2n. For each initial state xo(0) € CJ-1,0] (the space of continuous n-vectors on - 1 S в £ 0) let x(t) be the corresponding solution in Rn for some maximal duration 0 S t < t+ . Define the future states by x t (e) = x(t + e) in cn[ - l, o] Take the Banach space C n[-1, 0] as the state space M and then the solutions define a future local flow in Cn[-1, 0], Ф : R x Cn[-1,0] - Cn[-1,0] : (t,x0) - xt If |f(x, у) I s k( |x I + |y |) is a linear growth condition in R2n, then Ф is a global semi-flow — for instance in the case of a linear differential-delay s y s te m &(t) = Ax(t) + Bx(t-l) A more general hereditary process in Rn is defined by a differential- functional system IAEA-SMR-17/6 31 x(t) = f(xt) for t § 0 w h e re f : C n [-1, 0] - R n satisfies a Lipschitz condition. Again there is a future local flow (global if f satisfies a linear growth condition) in the state space Cn[-1, 0]. 5) Consider the linear parabolic partial differential equation + bl(x) + c(x)w + f(x) Э х ^ х -* with real-valued solutions w(x, t), for x in a bounded open domain D с Rn (with smooth boundary 9D) and t è 0. Assume the coefficients а^(х), Ь*(х), c(x), f(x) are С in D and that a1J (x) = aJ1 (x) is strictly positive definite in D. Then a continuous solution w(x, t) in D x R+ w ill be determined by suitable Cauchy data: w(x, 0) = wo(x) initial function in D vanishing on 9D w(x, t) = 0 for x € 9 D and all t s 0 In more detail, if w 0 (x) E C 0(D) (continuous on D but vanishing on 3D), then w(x, t) G C” (D x (t > 0)) is the unique classical solution. In particular, there is a future semi-flow in the state space Co(D) (Banach space with max n o r m ) $t : wo -» wt (where wt = w(x, t )) fo r t § 0 Tajee the same parabolic partial differential equation but use the state space H^D), the Hilbert space arising as the completion of C“ (D), C°° functions on D vanishing in a neighbourhood of 9D, under the Sobolev norm unique weak solution wt 6 H 1 (D) and a corresponding semi-flow Фг : wq wt in H 1(D) for t ï 0 Let us return to the general theory of a flow Ф in a metric space M Ф : R x M -» M, Ф£ : M M Define the future trajectory or orbit of a point Рц e M to be the map R + -» M : P 0 -* Pt or the point set U^Pt. The past trajectory is sim ilar, and the full trajectory of P 0 consists of both past and future trajectories of Po. Theorem. The orbit of Po G M is one of the three types i) a single point P0, that is a critical or stationary or equilibrium point w h e re ii) a simple closed curve (topological circle), that is cp{t + r , P 0) = P r o o f Consider the set T с R of all times for which cp(t, P 0) = P 0. T h e n T is a closed subgroup of R and so either i) T = R ii) T = {кт} for к € Z and some positive т iii) T = 0 In case i) Definition. A set S с M is invariant under the flow Ф г in case: P 0 G S im plies the orbit P, E S for all t. That is, an invariant set S is the union of complete orbits. Usually we study closed invariant sets, since S is invariant. We next describe the various types of (Liapunov, future) stability that might hold for an invariant set. Definition. Let S с M be an invariant set for a flow 3>t. Then S is stable in case: for each neighbourhood U of S there exists a sm aller neighbourhood V of S so S с V с U, such that P, EU for all t г 0 provided P 0 e V . If, further, lim dist(Pt,S) = 0, then S is asymptotically stable. If, still further, Qt -* S regardless of the initial point Q 0 G M, then S is globally asymptotically s ta b le . Example. In R 2 the systems x = y x = y s p ir a l ÿ = -x (centre), and ÿ = -x - y (focus) are stable about the critical point S = (0, 0), but * = У (s ad d le ) ÿ = x is not stable about S. In polar co-ordinates in R2, consider the system r = (1 - r ), 0 = - 1 Then the periodic orbit r(t) = 1, 0(t) = -t is a stable invariant set. IAEA-SMR-17/6 33 Definition. Let Фс define a flow in M. The future (or positive or u) lim it set of a point Po (or the orbit Pt) is the set w(P0) = n ü P^ tx > О C - and the past (or negative or a) lim it get is «(P o ) = П и P t tL < 0 t It is easy to show that w(Po) can be characterized as the set of all points Q € M for which some sequence of points on the future orbit Pt approach Q. That is, Q 6 u ( P 0) ju s t in case: 3 t k -> + oo for which lim Pt = Q tk~ ~ From the definitions it is easy to verify that io(P0) and «(P q) are connected, closed, invariant sets. In case the future orbit of P q lies in some compact subset of M (say M itself is compact), thenu(P0) is a nonempty compact set. Definition. Pj e M is future-recurrent (or Poisson stable) in case P 0 E u ( P 0), T h a t is , P 0 is future-recurrent in case the future orbit Pt recurs back to an arbitrarily sm all neighbourhood of P 0 for some arbitrarily late times. Past recurrence is sim ilar, and recurrence means both past and future recurrence. The set of all periodic points of $t is an invariant set, and is included in the invariant set of all recurrent points. But these invariant sets may not be closed in M. We next define a larger invariant set that is closed in M . Definition. Let Фг define a flow in M. The non-wandering (or regionally recurrent) set Í) с M consists of all points Pu G M such that each neighbourhood N of Po in M recurs to meet itself at arbitrarily large times. That is, P 0 £ Г2 just in case: for each neighbourhood N of P 0 there exist arbitrarily large tim e s tk-» +00 when Ф1к N meets N. It is easy to prove that Q is a closed invariant subset of M which contains all recurrent points. While we require only future regional recurrence for P 0 £ Г2, this automatically im plies past regional recurrence. VII. TOPOLOGICAL DYNAMICS. MINIMAL SETS AND STABILITY THEORY In the study of a flow on a metric space M we often search for an invariant set S с M. Then Фг defines a flow on S which might be rather sim pler and which helps to understand the total flow on M. Example. In the space R2n with coordinates (x1, x2, xn,y 1 , y 2, ..., y n ) we take any real C 2 function H(x, y), and then determine a dynamical system of Hamiltonian form 34 MARKUS ЭН , . y i = “ э ? ( x ' y ) Then, for each constant C, the level hypersurface H(x, у) = С is an invariant subset of R2n. This is just the assertion of classical mechanics that a Hamiltonian dynamical system preserves the energy H. Namely, dH _ ЭН . ЭН . TT TT d t “ Эх Эу У x У - Ну H x = 0 Definition. Let Ф£ be a flow on a metric space M. A non-empty compact invariant set E of #t is called m inim al in case E contains no proper compact invariant subset. Theorem. Let S be a non-empty compact invariant set for a flow $t in a metric space M. Then S contains some m inim al set E for Ф£. P r o o f Partially order the compact invariant subsets of $t by the usual relation of inclusion. Use Zorn's lemma to select a maximal linearly ordered chain of such sets, and then E is the intersection of the members of this chain. Thus this result is proved by Zorn's lem m a or by use of the axiom of choice. Q . E . D . Theorem. Let E be a minim al set for a flow Фг in a metric space M. Then the flow in E is such that: 1) e a c h p o in t P 0 £ E is topological-transitive in E, that is, e(Pb) = u(P0) = E. 2) each point P q £ E is recurrent in E, and with bounded time gaps between recurrences to any prescribed neighbourhood. P r o o f T a k e P j€ E and then u(P0) is a nonempty compact invariant set in E. Since E is assumed minim al, u (P q) = E. A sim ilar argument proves e(P0) = S. Next take any neighbourhood N of Po in E and suppose Pt lies outside N for a sequence of future time intervals L^ with lengths tending toward infinity. Let tk be the mid-time of L^ and consider the sequence Ptl( in the compact set E. Select a subsequence t ki such that Ptk -> Q in E. But cj(Q) = E so the orbit through Q meets N after some finite duration L. By continuity when Ptk. is sufficiently near Q, the orbit of Pti^ must also meet N after duration L. But this contradicts the supposition that Lk¡ -* °o. Q. E .D . IAEA-SMR-17/6 35 Now we return to the study of ordinary differential systems in Rn in order to make some applications of our abstract theory of topological d y n a m ic s . Theorem. Consider a differential system in Rn S?) x = f (x) j f (x) e C 1 in R n with a critical point, say at the origin, f(0) = 0. Assume there exists a real function V(x) £ C 1 (Rn ), called Liapunov function, satisfying i) V(x) > 0 in Rn except V(0) = 0 ii) !^r f’(x> á 0 in Rn . Then the origin is (future, Liapunov) stable. Furthermore assume that the set Z . { * : S r f ' W = o } in Rn is such that; in some neighbourhood N 0 of the origin, Z n No contains no m inim al set other than the origin. Then the origin is asymptotically s ta b le . Moreover,if we can take N 0 = R n and also assum e V(x) -* « as |x| -*■ oo, then the origin is globally asymptotically stable. P r o o f Take a neighbourhood N of the origin, say with compact closure N, and d e fin e e = min V(x) > 0 xe9N Take a subneighbourhood U wherein V(x) < e/2. Then for x 0 G U the s o lu tio n curve of^never meets 3N, since along x(t, xo) we compute d v a v t , s . dT = эГ f(x) S V (x, (t, x 0)) < e /2 fo r t г о Thus the origin is stable. Now assume Z n N 0 contains no minim al set other than the origin. Take a subneighbourhood U 0 с N0 so x 0 £ U 0 has a future trajectory in N0, and suppose u(x0) contains a point Q / 0. Note V(x) = V(Q) > 0 must be constant on u(x0), since the montonic function V(x(t, x0)) -» V(Q). So to(x0) does not' contain the origin. Hence w(x0) contains some m inim al set K, not lying in Z. Then we choose a point у e К - Z so V(y(t)) < V(y(0)) = V(Q), for sm all t > 0. But 36 MARKUS this contradicts the constancy of V(x) onw(x0). Thus we must conclude that u ( x 0) = {0}, and so5^is asymptotically stable. Finally, assume Z contains no m inim al set other than the origin and also V(x) -» oo as |x| -* oo. Then each sim ple closed curve V(x) = constant encloses an invariant region which can play the role of N 0 above. Hence x 0 6 Nq has a future trajectory that remains forever in the compact set No, and u(x0) = {0}. Q. E .D . Example. Consider an autonomous system of non-linear mechanics with 1 -degree of freedom x + f(x)x + g(x) = 0 with f(x) and g(x) in C 1 for x in R, and we assume f(x) > 0, and xg(x) > 0 fo r x f 0 Then the system in the (x, y) phase plane Rz is x = У, j = - g(x) - yf(x) which we show to be asymptotically stable about the origin. If also x G (x) = J' g(s)d s -» oo a s |x| -> oo, then the system can be seen to be globally о asymptotically stable. To verify these assertions, consider the Liapunov function V (x, y) = ÿ + G (x) Note V > 0 except at the origin. Also V = g(x)y + y [ - g(x) - yf(x)] = - y2f (x) S 0 Thus the origin is stable. Next consider the set in R 2 Z = {(x, y) : -y 2f(x) = 0} so Z is just the x-axis where y = 0. On the set Z we observe the dynamics y = -g(x), so the only invariant subset of Z is the origin. By the above theorem the origin is then asymptotically stable. If, further, G(x) -» oo as |x| -* oo then the level curves V(x, y) = const, are closed curves encircling the origin. The theorem then asserts that the system is globally asymptotically stable about the origin in R 2. Note that the asymptotic stability is a consequence of the positive damping f(x) > 0. If instead we have a conservative system with f(x) = 0, then every solution lies on some level curve of V(x, y) and so the origin is stable. If V -» oo a s |x| + |y| -* oo, then each level curve of V is a closed curve encircling the origin, so each solution is periodic. IAEA-SMR-17/6 37 VIII. CONTROLLABILITY OF LINEAR SYSTEMS For an autonomous linear differential equation in Rn, x = A x the origin is an asymptotically stable critical point if and only if all the eigenvalues A j of A lie in the left-hand complex plane, that is, Re X j < 0 . To prove this make a linear co-ordinate transformation in Rn, y = Px, and compute the linear system у = P x = P A x = ( P A P -1) у = A y Choose P so that PAP "1 = A is in suitable Jordan canonical form, and for sim plicity of exposition we assume A = diagiX-L, X2, .. ., An} with real Xj < 0 Then the solution initiating at y 0 is y(t) = e ^ ' y i + ... e x"ty n Thus x(t) = P_:Ly(t) -> 0 exponentially as t -> + oo, so the origin is globally asymptotically stable. Even for the non-linear differential equation in Rn, x = f(x) = Ax + h(x) where h(x) £ C 1 is of higher order near x = 0, the condition Re Xj < 0 im plies (local) asymptotic stability. The proof again uses the linear co-ordinates у = Px and the Liapunov function V(y) = -y'Ay >0, for y f 0 T he n -y'A [Ay + h(P_1 y)] and the negative-definite quadratic form -y'A2y dominates the higher-order terms near у = 0. However, note that the solution x(t) = P_1y(t) merely tends asymptotically towards the origin as t -*• + oo, and we might wish to steer an initial state x 0 to x = 0 in a finite time duration by applying a suitable controller u(t). Definition. The autonomous linear control system 5?) x = Ax + Bu, x G Rn, u £ Rm is (completely) controllable in Rn on the duration 0 s t g'T in case: for each pair of states x 0 and x, in Rn there exists a piecewise continuous controller u(t) on 0 s t s T steering the response x(t) from x(0) = x 0 to x(T) = x*. 38 MARKUS If the duration T < oo is allowed to vary with the choice of points x0, х г, then áfis controllable on finite durations. T h e o r e m ¡£ ) x = Ax + Bu, x £ Rn, u G Rm is controllable on a given finite duration 0 s t s T if and only if r a n k [ B, A B , A 2 B, . . ., A" '1 B] = n Proof F ix x 0 and choose a controller u(t) to get a response 0 Then x(T) = eAT x 0 +{L}T, for a linear subspace {L}T С R n. Assume that rank [B, AB, A 2 B, .. ., An"1 B] = n so this n x nm matrix has n independent rows. Then a row n-vector rj annihilates [B , A B , A 2B , . . . , A n"1 B] if and only if 17 = 0. That is, 17B = 0, 17A B = 0, r]A2B = 0, . . ., т}Ап_1В = 0 hold if and only if 17 = 0. Hence 2 i7e 'As В = 17 [I - A s -Ц-р A2 - ...] В = 0 on OsssT im p lie s 17 = 0 . Suppose dim {L}T < n so there exists 17 f 0 for which rj{L}T = 0 or T rjeAT/ e"As Bu(s) ds = 0, for all u(s). Then (rjeAT) e”As В = 0 so 17 e AT = 0. 0 This contradicts the supposition 17 f 0 so we conclude {L}T = Rn and is controllable on 0 s t s T, Conversely, assume ¡ â is controllable - or only the weaker assertion that the origin can be steered to all points of Rn in various finite durations. Let {L},, = k^ 0{L}k = Rn. Since the linear spaces {L}k are nested increasing fo r к > 0 (insert a zero control for duration a before a controller on 0 S t S к to construct a controller on 0 s t s к + o), the reachable set {L}„ is a linear space with the same dimension as{L}N, for some large N. That is, dim {L}n = n and so only the zero vector is orthogonal to {L}N. Thus, for any row n-vector 17 , N im p lie s 17 = 0 0 Hence, writing rjj = rjeAN, N implies 171= 0 IAEA-SMR-17/6 39 Now suppose rank [B, AB, A2 B , . . . , A n_1 B] < n so there exists n 1 f 0 r ^ B = 0, r)1 AB = 0, r)1 A 2B = 0...... r i j A ^ B = 0 By the Cayley Hamilton Theorem An is a real linear combination of (I, A, A2, . . ., A ""1 ) so r)1 AnB = 0. Continuing we have r)1 e ‘ AsB = 0 w ith r¡1 f 0. This contradicts the assertion that {L}N = Rn, so we conclude rank [ B , A B , A 2B , ..., А П_1 В] = n, as required. Q. E.D. Remarks. is controllable on [ 0, T] if and only if is controllable the unit duration [0, 1]. Also the proof of the theorem shows that {L}„ = Rn implies that-S'is controllable on [0, Т]. Hence controllability on variable finite durations is the same requirement as controllability on a fixed duration for autonomous linear systems This last rem ark is not valid for time-dependent linear systems. However, the same methods as above (Hermes-La Salle) prove that a linear system with С coefficients x = A(t)x + B(t)u is controllable in Rn on any fixed duration 0 S t S T provided rank [В, ГВ, Г 2В , ..., r k_1B ]t=0 = n H e re k S 1 is any integer and the differential operator Г is given by TB(t) = -A(t) B(t) + B(t) Rem ark. Note that the class of controllers u(t) on 0 S t S T is not particularly significant in the controllability of £ f. In fact, we could take u(t) belonging to L} [ 0, T], or any dense linear subspace thereof. Corollary. For scalar controllers, so m = 1 and В = b is a column vector, £ £ i s controllable in Rn if and only if det I b, Ab, A2b, ..., A n' 1 b| f 0 Theorem. An autonomous linear scalar process x(n> + a 1 x (n"1) + . . . + a nx = u or the corresponding control system in Rn x n = - ajjX 1 - а Пш1х 2 -... - a - ^ " + u is controllable in Rn with u G R 1 . 40 M ARK US Moreover, every autonomous linear process in Rn i f ) x = Ax + bu, u e R 1 which is controllable, is linearly equivalent to some scalar control processif. P r o o f For the matrices the controllability criterion is easily verified by computation. Next consider the controllable process ¡ £ in Rn and consider the non singular n x n matrix P = [А П_1 Ь, A n‘ 2 b , ____ A2b, A b , b] Introduce new linear co-ordinates in Rn by x = P_1x so к = P 1A P x + P _1b u By direct m atrix multiplication verify Also AP = PN or P '1AP = N where we define Here the real constants a , a 2 j . . . , a n are uniquely specified by the characteristic equation for A, A n - A n 1 4- 0*2 A n 2 + . . . + IAEA-SMR-17/6 41 Thus âfis linearly equivalent to a system in Rn x = N x + b a u But if we begin with a scalar process with matrix Aj, and corresponding characteristic equation A” = " ai - a 2 A j - . . . - a n then is linearly equivalent to provided we take - a l = a \> - a 2 = a 2> ■ • • > ‘ a n = fín Since linear equivalence is a transitive relation, £ £ is linearly equivalent to the scalar system ¿2?, as required. Q. E.D. Theorem. Consider a controllable linear autonomous process Ç?) x = Ax + bu, x G R n, u G R 1 Then there exists a linear feedback u = kx, for constant row vector k, such th a t & = (A + b k) x is asymptotically stable towards the origin. P r o o f It is easy to see that the property of possessing a linear feedback which yields asymptotic stability, is invariant under the relation of linear equivalence or change of linear co-ordinates x = P_1x in Rn. B u ta is linearly equivalent to a scalar process^ x u = ( a n- 1 ) x + (a n.j- n) x + . . . + (a j- n) x^11'1) to o b ta in + n x *"'11 + . . . + rix + x = 0 But this has the characteristic equation Xn + n X " '1 + . . . + n X + 1 = (X + 1)D with all eigenvalues X = - 1 < 0. Q. E.D . 42 MARKUS IX. LINEAR SYSTEMS WITH CONTROL RESTRAINTS Consider an autonomous linear control process in R n âf) x = Ax + Bu where the controllers are in Ъ ъ [ 0 , T] with value u(t) restrained to some s e t Q с Rm. We shall always assume that Q is a convex compact set which contains the origin of Rm in its interior. For instance Q could be the cube I u-* I s l fo r j = 1 , 2 , . , , , m , Definition. Consider an autonomous linear process in Rn i f ) x = Ax + Bu with u(t) с £2, a convex compact neighbourhood of the origin in R m. The set of attainability from x 0 a t tim e T > 0 is T K x0(T) = "jx(T) = eAT x 0 + e'As Bu(s) ds | all admissible u(s) с Г2 о Theorem. KXo(T) is a compact convex subset of Rn which varies continuously with T > 0. A lso K Xo(T) has a nonempty interior if and only if is controllable. P r o o f The subset of L 2 [0, T] of admissible controllers is a convex set which is weakly compact (note: a closed ball in L 2 is weakly compact - also the weak lim it of a non-negative function is non-negative and this can be used to prove that the weak lim it of admissible controllers is itself admissible). Since integration is a linear compact operator from L 2 to Rn, we find that K Xo(T) is convex and compact. The continuity (Hausdorff metric) is evident from the boundedness of x(t). The controllability of yields an interior of K X|)(T) just as is proved in earlier theorems where there is no restraint on u. For details see Lee- M a r k u s . Definition. The set of null-controllability for SC) x = Ax + Bu, u(t) c i 2 c R m is the subset of Rn consisting of all states that can be steered to the origin in finite times. Theorem. Consider as above, 5?) x = Ax + Bu, x e Rn, u c ñ c R m Then the set of null controllability is all Rn provided both i) All eigenvalues of A have negative real parts ii) r a n k [ B, A B , A2 B, . .., A n' 1B] = n . IAEA-SMR-17/6 43 Example. Consider the damped linear oscillator with control x + 2/3& + k2x = u(t), 0 > 0, к > 0, |u(t) | ё 1 Then every initial state (x 0, y 0 = x0) lies in Ф = R 2 . X. TIME-OPTIMAL CONTROL OF AUTONOMOUS LINEAR SYSTEMS Again we consider an autonomous linear control process £ ? ) x = Ax + Bu, x 6 R " , u £ f i where Q is a convex compact neighbourhood of the origin in Rm. Let an initial state xo lie in the null-controllability set so xo can be steered to the origin in finite time by some admissible controller u(t) с Q on 0 s t ё t i . Definition. Among all admissible controllers u(t) ей on various [0, tj] s te e r in g x 0 to X j = 0, a time-optimal controller u*(t) on [ 0, t*] is d e fin e d by the requirement t* = inf tj. Theorem. Consider the autonomous linear process ¿f) & = Ax + Bu, x € Rn, u G Г2 with initial state x 0 e Then there exists a time-optimal controller u*(t) on 0 S t S t* steering x 0 to the origin in m inim al time t*. P r o o f T ak e xo f 0 and consider the moving set of attainability K Xo(t) for t > 0. Since K X|)(t) is always a compact set in Rn, and it moves continuously with t, we find that K X|)(t) meets xx = 0 for a first time t* > 0. The corresponding controller u*(t) which yields this point х ге K x (t*) is the required optimal controller. Q.E.D. If fi is a segment |u| s i, and-Sfis controllable, then u*(t) is the unique (almost everywhere) controller steering x 0 to X j = 0 in minimal time t*. This will follow from the characterization of u*(t) by means of the maximal principle of Pontryagin, as proved below. Theorem. (Max. Principle). Consider the autonomous linear control system in R n £f) x = Ax + Bu with controllers u(t) с Г2, a convex compact neighbourhood of the origin in R m. L e t x 0 6 so th a t x 0 can be steered to the origin Xj = 0 in finite time, and so there exists an optimal controller u*(t) with response x*(t) for m inim al tim e 0 S t g t* . Then there exists a nonzero solution ^'(t) of the adjoint linear system f¡ = - r ) A 44 MARKUS f o r w h i c h r)*(t) В u*(t) = max rj*(t) В u alm ost everyw here on 0 S t S t * U ë Q Thus if we define the Ham iltonian function H(rj, x, u) = r)[Ax + Bu] then rj*(t), x*(t) satisfy- ЭН* . ЭН* , x = ~эГ' " = -1 Г (,7'хЛ) w h e r e H*(r), x, t) = H(t 7, x, u*(t)) a n d H(r)=:= (t), x, u*(t)) = m ax H(r)*(t), x, u) U € Q P r o o f We need only show that >l*(t)B u*(t) = max ri*(t)B u U 6 Q almost everywhere, for a non-trivial adjoint row vector since the other assertions concerning the Hamiltonian are merely re-statements. This m axim al principle w ill be an analytical statement of the geometric condition that x-j = 0 necessarily lies on the boundary ЭК of the convex compact set of attainability K Xo(t*). Since K Xo(t*) is convex, choose an outward unit norm al vector (normal to a supporting hyperplane) at xj e ЭК. Then Xj is the point of K Xo(t*) farthest in the direction of r¡ 1г th a t is , i l j X j й r¡1 x f o r a l l X € K Xo(t*) t* tv r¡1 e M"' + e"As Bu*(s) ds й гцем'' + тпге м ^ J ' e‘As Bu(s)ds о о for all admissible controllers u(s) on 0 s s s t*. L e t n*(t) = (r7x eAl'' ) e "Al, so if;'(t*) = n1 IAEA-SMR-17/6 45 be a non-trivial solution of the adjoint differential system ^ = -17A , fo r a row n-vector r). Then we obtain the maximal principle: t* t* J ' r)*(s) Bu*(s) ds i J r j (s) B u (s) ds 0 0 But this yields the pointwise result: r¡*(s) Bu*(s) = max rj*(s)Bu u s a Since if such an equality failed on some positive duration S we could define a new controller max rf:‘(s)Bu for s e S U € u*(s) otherwise Then û(s) (which is an admissible controller) would contradict the above integral inequality for the maximal principle. Q. E.D. Corollary. For the case of a scalar controller where m = 1 so Г2 is a segment, say |u| s i, the maximal principle asserts u,:'(t) = sgn rf(t) b Hence u*(t) is a bang-bang controller with a finite number of switches, and u*(t) is uniquely specified by the maximal principle, provided d e t [ b, A b , A2b, . . . , A ^ b ] f 0 P r o o f For a time-optimal controller u*(t) on 0 S t S t* we must have tj* (t) b u* (t) = max r]*(t)bu= |ii*(t)b| u e £2 T h u s u*(t) = sgn rr (t) b Now assume that 5fis controllable so d e t [ b , A b , A? b, . . . , An 1 b ] f 0 Let u-^t) be any admissible controller steering x 0 to x x = 0 on Os t S t* . Then u^t) also satisfies the maximal principle so rj°:= (t) b [u*(t) - u х(t)] = 0 almost everywhere 46 MARKUS I f u *(t) ф Uj(t) on some positive duration £, then rj*(t) b = 0 fo r t 6 E S in c e r)*(t) = (rj1 eAt"') e~At is a real analytic vector, and since E contains an infinite set of points, r¡*e‘ Atb = 0 f o r a ll 0 s t s t* where we set r¡* = r¡1eÁl'1' as a nonzero row vector. Then at t = 0 we have r)*b = 0. Differentiate and set t = 0 to get rj*Ab = 0 and continue to find П* [b, A b , A2 b, . . ., A "'1 b] = 0 But this contradicts the controllability assumption for r/* f 0. T hu s UjJt) = u*(t) almost everywhere In other words, there is a unique optimal controller for the m inim al time t*. Moreover any controller uj (t) on 0 s t s t* which satisfies the maximal principle must be this optimal u*(t). Q. E.D. More general bang-bang theorems are available for the restraint set Г2 as a polyhedron in R ro - see the text of Lee-Markus. XI. SWITCHING LOCUS FOR THE SYNTHESIS OF OPTIMAL CONTROLLERS Consider an autonomous linear system in Rn ¿£ ) x = Ax + Bu for а с П, a convex compact neighbourhood of the origin in Rm. For any initial state x 0 in the set of null controllability in Rn, there exists an optimal controller u*(t) steering x*(t) from x 0 to X j = 0 in minim al time t*. Also u*(t) satisfies the maximal principle rj*(t) Bu*(t) = max Bu almost everywhere on 0 s t i t*, U ê O for some nontrivial adjoint vector 17* (t) s o lv in g rj = - r ) A In the case when В = b is a vector, so m = 1 for scalar controllers |u(t)| s 1 , and when SS is controllable, the optimal controller is characterized by the m aximal principle as a bang-bang controller with switches only at the finite set of zeros of ri*(t)b. That is, u* (t) = sgn 17* (t) b We illustrate this theory by constructing the switching locus, and synthesizing the optimal controller as a feedback control, for the case of IAEA- SMR-17/6 47 the linear oscillator x + x = u or the system in R 2 x = y S f ) ÿ = -x + u w ith Г2 : I u I S i H e re !) = A(;)+bu' A=(-?í> b=(? Thus the adjoint equation for 17 = (rj1, 172) is П = -r}A o r {r} j , r¡2 ) = - (17 з_, r)2 ) q ^ T h u s = Ч 2 * ^ 2 = _ r h s o ^ 2 + ^ 2 = 0 H e n ce 17! (t) is a sinusoid with a constant duration of v between successive z e r o s . The maximal principle now asserts that u*(t) = sgn (i]*(t), 17*(t)) = sgn Г)*(t) Hence the optimal controller u*(t) switches between the extreme values +1 and -1 every -n units of time. The main difficulty in finding u*(t), steering a given point (^y°^) ( 0 ) ^ time t*, is that we do not have the initial (or terminal) data for rjg(t). But we do know that each extremal controller (satisfying the maximal principle) acts as the unique optimal controller for some initial point in Rn. Thus, we proceed to construct all possible extremal controllers, and observe the corresponding extremal responses, to find the correct optimal controller for ( X° ). У х Л \ У о / The optimal response from ( u ) to the origin must follow arcs of \У о / the solutions of the extremal differential systems x = у x = у 5Í) and 5J) у = -x - 1 y = - x + 1 Since the extremal differential systems Sf. and 5^ are autonomous, we can construct extremal responses that terminate at the origin by a process of 48 MARKUS backing out of the origin as -t increases. That is, we start the extremal response at t = 0 at the origin and follow the appropriate solution curves of 5? and backwards in time (switching every n units of time) to reach the point (^y0^) at some negative value of t = -t*. Then reverse the time sense and start from (^y°^) at t = 0 to arrive at the origin at t = t*, thus obtaining x * ( "t ) the optimal response j, the optimal time t*, and the optimal controller u *(t). If we start at the origin at t = 0 with г)г =1, rj2= 0, then rj2 (t) = -sint and, o n the in t e r v a l -it < t < 0 w e h ave sgn r¡2(t) = +1. The corresponding extremal response traces out the solution curve of through the origin, that is, X = - COS t +1 ОП -7Г < t < 0 Г : + у = s in t o r (х - l )2 + y 2 = 1, y < О On the other hand if we start with rj1 = -1 , rj2 = 0, th e n n 2 (t) = sin t and, on -7Г < t < 0, we have sgn rj2 (t) = -1. The corresponding extremal response c u rv e of SC is p _ X = COS t -1 ОП -7Г < t < 0 у = - sin t o r (x + 1)2 + y2 = 1 , у > 0 However, for any other choice of (r^, rj2) at t = 0 with r)2 > 0, the extremal response traces back from the origin along Г+ until rj2 (t) = 0. At this point the extremal response switches to a solution of SC_ w h ic h it follows for a duration of length ir before switching again to a solution of A sim ilar process occurs for initial data with r]2 {0) < 0 but here the extremal response starts back from the origin along Г. . The switching locus W., which consists of all the points where all the above extremal responses switch between the fam ilies SC_ a n d SC^ is n o t difficult to describe in this example. In fact, 'W is made up of the arcs and П and their successive transportations along the appropriate solution families of SC an d SC for durations of length ir. Note that such a transporta tion is just a rigid rotation of the phase plane through ir radians about the corresponding centre (1, 0) or (-1, 0). This process of successive switching leads to a final construction of 'Ж'as a continuous curve over the entire x-axis, composed of semi-circles of radius one, as pictured in Fig. 5. Define the synthesis for real (x, y) f (0, 0) by Г -1 if (x, y) lies above W o r on Г . Ф(х<у) = j 0 if (x, y) lie s on 0Г otherwise [ +1 if (x, y) lies below 9Г or on Г+ IAEA-SMR-17/6 49 + = -1 У \ Ц/=+1 FIG.5. Switching locusOS^for x + x = u. Then the optimal control is given by the feedback u = Ф(х, у) and each optimal response is a solution of x + x = Ф (х , y) XII. LINEAR DYNAMICS WITH QUADRATIC COST OPTIMIZATION Consider an autonomous linear control system in Rn à?) -к = A x + B u with admissible controllers u(t) as m-vectors which are square-integrable on a given finite duration 0 s t S T. That is, the only restraint on u(t) is that T w h e re || u ||2 = u'u in Rm о ■ Then for a given initial state x 0 G Rn there exists a response x(t) on 0 S t S T and we define the cost of the controller u(t) to be T о where W = W ' > 0 and U = U 1 > 0 are positive-definite constant matrices used II и 2 i) 112 to define the norm s ||x||w = x 'ffx , Ilu II и = u'Uu. We seek an optimal controller u*(t) m inim izing the cost or performance functional C(u) so C(u*) = inf C(u) 50 MARKUS By techniques sim ilar to those of the m inim al time-optimal problem it can be shown (see Lee-Markus) that there exists a unique optimal controller u*(t) and that it is characterized by the maximal principle: u *(t) = U 1 B'rj*(t)' almost everywhere on 0 S t S T where r)*(t) is a row n-vector solution of = x'W -r)A, rj(T) = 0 w ith x = Ax + BU ^B'i)', x(0) = x 0 We can express this optimal control through a feedback gain matrix E*(t), independent of x0, so u*(t) = E*(t) x*(t) Here (see Lee-Markus for details) it can be established that E*(t) = U _:LB'E (t) where E(t) is the solution of the matrix Riccati differential equation Ê = W - A'E - E 'A - EBU _1BE with the term inal data E(T) = 0. Using this gain matrix E*(t) = \J"1 B 'E ( t) , which can be computed in advance of the control programme, we obtain the optimal response x*(t) as a solution of x = Ax + В [U '1B'E(t)x], x(0) = x 0 In the lim iting case where T = + oo the Riccati differential equation is replaced by the quadratic equation A'E + E'A + EBU _1 B 'E = W which has a unique symmetric negative definite solution E. Then a unique optimal controller u*(t) exists (here we must assume thatbifis controllable in Rn) and can be determined by a feedback synthesis u*(t) = U 1 B 'E x *(t) so x*(t) satisfies x = (A + BU _1B 'E ) x, x(0) = x о The m inim al cost for 0 IAEA-SMR-17/6 51 is ju s t C(u*) = -x¿Ex 0 which is a positive definite quadratic form in x0. It is interesting to derive this result by the intuitive methods of dynamic programming. Let us call the minimal cost from x 0 on [t0, T] to be T v ( x 0,t0) = min f llxW l C + d t u e L 2[t0, T] J 4 Then each controller u(t) on [t0, T] with corresponding response xu(t) initiating at xq at t = t 0 h a s co st to + à X f (llxJw + lluO dt+/ to+ 5 f (Ilxullw+ IMIu) dt + ¥(хи(*0 + *0+6) to T h u s t0 + 6 V ( x 0,t 0)= m in \ j ( | | x j ^ + Iu ||ц ) dt + V (x u (t0+ ó), t 0+ 6)| u(t)€L2[t0tT] , to This equation illustrates the idea of dynamic programming in that we think of the optimal-control program decomposed into the sum of two programs, on [ t0, t 0 + 6] and on [t0+ 6, T] with the dynamics indicated by the possibility of varying 6. We ignore problems of differentiability and convergence and proceed formally. Using a Taylor series expansion in terms of the small number 6 > 0 we o b ta in V(x„, t0) = min i i I I ! , + l|u(to)|2 ] + V (x 0,t0) u(t) 1 | 7 (x o-t0) ^ ( t 0) + ^(x 0,t0) 6 + 0 (ô) W r ite d x u (t0) = A x 0 + Bu(t0) dt 52 MARKUS and let б -* 0 to o b ta in where we have written the generic point (x0, t0) as (x, t). Thus we must compute the m inim um of the real function h ( u ) = H I’ + |j [Ax + B u ] for each fixed (x, t). But so take ЭУ u (Эх ■) Hence V(x, t) must be the solution of the non-linear partial differential e q u a tio n with term inal data V(x, T) = 0 In the case T = + oo we can verify that V(x, t) = -x'Ex is a solution of this Bellman dynamic programming partial differential equation just in case E satisfies W = E B U ' 1 B'E + A'E + E'A as specified earlier. IAEA-SMR-17/1 CONCEPTS OF STABILITY AND CONTROL P.C. PARKS Control Theory Centre, University of Warwick, Coventry, United Kingdom Abstract CONCEPTS OF STABILITY AND CONTROL. The first part o f the paper deals with the role transfer functions play in control problems (closed loop, Nyquist stability criterion, sampled data systems and z-transforms; the "hog cycle"; spring oscillations; Lyapunov functions; the Zubov method; positive-real functions and the Popov criterion; the circle criterion; linear time-delay systems; equations with periodic coefficients; stability of repeated processes). In the second part the author considers the control of systems which are described by partial differential equations (heat-conduction equation; wave equation; control of the heat and the wave equations; parasitic oscillations; noise in linear systems; discrete noise processes). Many examples are given and briefly discussed. 1. CONCEPTS OF STABILITY AND CONTROL; TRANSFER FUNCTIONS The mathematical theory of control has always been prompted by practical considerations and much of the development of theory has been carried out by electrical and mechanical engineers, who have introduced a number of concepts and words which are not always fam iliar to the mathe matician brought up on traditional courses in pure and applied mathematics. The mathematician has made important contributions at critical stages in the development of control theory — for example Maxwell (stability, 186 8), Hurwitz (stability, 1895), Wiener (random processes, 1941), Pontryagin (optimal control, 1950) — but these developments have depended on a good liaison between the engineer and the mathem atician — for example between the turbine engineer Stodola and the mathematician Hurwitz. It is important therefore for mathematicians interested in control theory to fam iliarize themselves with these control engineering concepts. One such concept which appears in any book or paper on control engineering is that of the transfer function. Consider the tower crane shown in Fig. 1.1. The load В is suspended freely by a cable of length i from a horizontally moving trolley A. Considering movements in the plane of the tower and jib we wish to set up a mathematical relationship between the "input" to the system which is the distance CA = x of the trolley from the datum point C, and the horizontal movement of the load В measured as DB = у as shown. Considering small motions of the load (so that y - x « i) we obtain the equation of motion m + i ^ x } m d t2 + m C d t 1 SL where m is the mass of the load, T = mg is the tension in the cable and me is a damping constant representing aerodynamic damping of the motion (probably rather small). 53 54 PARKS К x lt) a D a+cD+K у tt ) FIG. 1.2. Transfer function (1.2). FIG. 1. 3. Frequency response of (1.1). W e o b ta in d2y _ + c Ú L + £ , r - £ ( 1 . 1 ) d t2 + c dt + У = 7 X as the second order differential equation relating the "output" y(t) to the " in p u t '1 x(t). Given x(t) and initial conditions on y(t) and dy(t)/dt, Eq. (1.1) may be solved by classical methods. Now the control engineer has developed the useful concept of the transfer function to represent this input/output relationship. W riting D for d/dt we can say that К ( 1 . 2 ) a D 2 + cD + K w h e re К = g/£, a = 1, and the expression K/(arf +cD+K) is the transfer function between у and x. It is also usual to draw a "block diagram" of this relationship as shown in Fig. 1.2. It is only necessary to cross-multiply Eq.(1.2) to recover the differential equation (1.1). The transfer function plays two other useful roles — if D is replaced by iu we obtain the system frequency response which relates the output to input when the input is a sine-wave at frequency w (rad/s). This is shown plotted on the Argand diagram in Fig. 1. 3. For a given u the radius OP gives the ratio of output amplitude to input amplitude and the angle AOP represents the phase lag angle of the output sinusoid relative to the input s in u s o id . IAEA-SMR-n/1 55 ERROR,. ylt) 0,ID) SIGNAL ERROR ytt) x(t) GID) SIGNAL FIG. 1 .4. Feedback system. FIG. 1. 5. Unit feedback. If D is replaced by s as used in Laplace transform notation, the transfer function then represents the Laplace transform of the unit impulse response of the system, that is the Laplace transform of y(t) when x(t) is a unit impulse a t t = 0+, the system starting with zero initial conditions on y(t) and dy/dt. The unit impulse response h(t) (t > 0) is a useful concept since it may be used to write down the solution y(t) for a very general input x(t) in the form of a "convolution integral" y(t) h (r ) x (t- r) dT r= 0 It often happens that we build up a feedback system involving a number of transfer functions Gi(D), G 2(D), etc., where each G¡(D) is a ratio of polynomials in D as in Fig. 1.4. Here we may operate by the ordinary rules of algebra to obtain the "closed-loop transfer function" relating y(t) to x(t) which is, in this case, V G2(D) Gi(D) x (1 + G 2 (D) 0,(0) G 3 (D)) (The circle® represents a subtraction to form the "error signal" which is x(t) - G 3(D)y(t) hence y(t) = G 2 (D) G x (D) [ x(t) - G 3 (D)y(t) ] . ) A special case is shown in Fig. 1. 5 where we call G(D) the open-loop transfer function and G(D)/(1 + G(D)) is the closed loop transfer function. Returning to the differential equation (1.1) and thinking about the classical method of solution where y(t) = particular integral + complementary function, we observe the complementary functions, or transient solutions in the language of control engineers, determined by dt + fy = 0 This is satisfied by у = Ae where X2 + cX+f = 0 or, looking at the transfer function^ 0 g/l___ D + cD + gj SL 56 PARKS by the denominator of the transfer function replacing D by X and equating to zero. Replacing D by s and regarding s as a complex number the roots in X are poles of the transfer function g / l s2 + cs + gj i If the transient solutions are to die away as time increases, then the roots in X must be real and negative, or complex with negative real parts. Alternatively "the poles must lie in the left-hand half of the complex plane". This is a general requirement for stability of the transfer function, and especially when it is the transfer function of a closed-loop system. The behaviour of the system also depends on the transient solutions, for example, these may be very oscillatory even though eventually damped out. A useful test, theoretically and practically, is the unit-step response of the system. The unit-step responses of the second-order system (D 2 + 2ÇunD + u2)y = u2x (a standard form for second-order systems) for various values of Ç is shown in F ig.1.6. — 4 — s V ir 2ir 3 ît U-n шп‘ Step responses o f the second-order system (D2 + 2£w n D i ùj2r A classical stability problem solved by Hermite (1854), Routh (1877) and Hurwitz (1895) was to find necessary and sufficient conditions on the coefficients of the linear differential equation dny + a + a ^ + a + a v = 0 n-l dt iW d t" ' " d t " '1 + 32 d t"’ 2 for damped or stable transients. H e r m it e 1 s solution, which deserves to be better known,is that the n X n m a t r ix n “ n -l 0 a n-3a n 0 0 H = l a 4+ a 2 a 3 0 a3 0 a l a 2 _ a 3 0 0 a j a 3 be positive definite (that is the m atrix of a positive-definite quadratic form). IAEA-SMR-17/1 57 Im Im Re 0 Re FIG. 1.7. Contour C, FIG. 1.8. Indented contour. In the 1930s another technique was developed to investigate stability of closed-loop systems from a knowledge of the open-loop frequency response. This is the celebrated Nyquist stability criterion. If G(D) is the open-loop transfer function then the closed-loop transfer function is G(D)/(1 +G(D)) and we are thus investigating roots in s of the equation 1+G(s) = 0. Now G(s) = q(s)/p(s), where p(s) and q(s) are polynomials, and so we are interested in the roots of p(s) + q(s) = 0. We now consider the contour С on the Argand diagram shown in Fig. 1. 7. Applying "the principle of the argument" to the function 1 +G(s) of the complex variable s a r g [ l + G(s)] = 2 7 r ( N - P ) where N = number of zeros of 1 +G(s) and P = number of poles of 1 +G(s) within C. Here we have assumed no zeros or poles actually lie on С itself. If the degree of p(s) is greater than that of q(s), then the contribution to [arg(l +G(s))] on the arc of radius R is zero as R -» 00 and we have to consider the change in argument of 1+ G(iu) as и varies from + 00 th ro u g h 0 to -oo, that is 1 +G(s) for s = iu on the im aginary axis. If there are no poles of 1 +G(s), that is zeros of p(s) in the right-hand half-plane so that P = 0, and furthermore we require N = 0 for stability (so that no roots of 1 + G (s) = 0 have positive real parts), then 1 +G(iu) must not encircle the origin as u varies form +°o through 0 to -°o. Alternatively, G(iu) must not encircle the point (-1, 0). This is the Nyquist stability criterion in its simplest form. A modification that is quite often necessary occurs when p(s) has a zero at the origin so that G(s) has a pole there. It is then necessary to "indent” the contour С by a small sem icircle as shown in Fig. 1.8. The behaviour of G(s) at infinity is determined by what happens as s describes the sm all semi-circle surrounding the origin. An example is shown in Fig. 1.9. 2. TRANSFER FUNCTIONS AND STABILITY 2.1. Sampled data systems and z-transforms Discussion hitherto has concerned ordinary differential equations and continuous functions of time t. An important class of control systems uses discrete time inputs and outputs: these are known as sampled data control systems and a theory analogous to the Laplace transform for continuous systems has been built up based on the so-called z-transform. This theory may be applied also to difference equations which occur directly in some applications, for example,economic models. 58 PARKS s(s + q) Closed-loop system stable,P = 0, N = 0 within indented contour. Closed-loop system unstable,P = 0, N = 1 within indented contour. K>0, Q < 0 .x'lUo-O* / Л A-1.0) i <*>-► + 00 CJ-»-00 Closed-loop system unstable,P = 1, N = 2 within ^ j i indented contour. \ !/ K<0,i a<0 о 3 r I \ w-»-оъУ \ ---- 1------^ -----1------Closed-loop system unstable,P = 1, N = 1 within M ,0) indented contour. FIG. 1 .9. Nyquist criterion with an open-loop pole at s = 0. Difference equations describe the behaviour between input and output sequences, for example У + У = x ; n=0, ^n+l 2¿ J n n* 1,2,3, ' » •... • • where the initial condition is yo = 1 a n d xnis the sequence ( 1 , 1 , 1 , 1 , ...) By a step-by-step process we obtain the solution as l 3 5 11 У1 = 2 , У2 = 4, Уз = 3 , У4 = IAEA-SMR-17/1 59 The z-transform of a sequence {ar}= {a0, a1( a 2, . . .} is by definition th e s e r ie s ^ ar z'r = Â(z) r = 0 The z-transform of a shifted sequence iar+1} = i3 !, a2> a3> • • •} is CO a r + 1 z"r = (Â(z) - a„) X z r=0 The difference equation may be solved by "taking sequences" and then "taking z-transforms" to obtain (Y(z) - y„) z + i Y(z) = l+z- 1 +z-2+ .. . = - Ц - = -f- 1 -z z _ i H e n ce - y0Z z ” Z +1 + (z - 1) (z +|) 111 Z + 5 These two terms correspond to the "complementary function" and "particular integral" terms that we encounter in both differential and difference e q u a tio n s . Now the z-transform of the geometric sequence {1 , a2, a3 , . .. } is 1 + az_1 + a2 z ‘ 2 + . . . = --- — - = —-— 1 - az г - a so we may write the inverse transform of Y(z) to give the solution {yn}= (yo -!){1. i. - j , ..■} +1 {l, l. 1. 1. ...} o r w hen Уо = 1 {Уп}={1, h I, f, ...} (Note that lim yn = |). 2.2. Sampled data systems Consider the system shown in Fig. 2.1. A continuous input x(t) is sampled by the switch Sx and the samples modelled as impulses are fed to the first-order system with the transfer function K/(s+a). The switch Si closes for a short time Д at equally spaced times h apart. An impulse of magnitude x(nh)A is thus fed to the first-order system. The sampling may be organized so that Д = 1. The response of the first-order system to this impulse is x(nh)A k(t-nh) for t > nh, where k(t) for t > 0 is the unit impulse 60 PARKS nh ln+l)h t —» ♦* -** FIG. 2 .1 . System with sampled input. a >0 FIG. 2 .2 . Second-order continuous linear closed-loop system (a) stable for all К > 0, (b) stable for 0 < К < 2a coth (ah/2). response of the first-order system. If we consider the output sampled by a second synchronized switch S2 then we find that y(nh) = x(nh)A k(0) + x((n-l)h)Ak(h) + x ((n - 2 )h )A k (2 h) + . . . + x (0)A k(nh) Considering now the z-transform of {y(nh)} we find that z-transform of sampled output = (z-transform of sampled input) X (z-transform of the sampled unit impulse response) X Д Another arrangement is the "sample-and-hold" device which has Д = 1 followed by the transfer function (1 -e"sh)/s which m ultiplies any subsequent transfer function such as K/(s+a) a b o v e . We note that sampling can change the character of the system, for example the second-order continuous linear closed-loop system shown in Fig. 2. 2a is stable for all К > 0, whereas 2.2b, with Д = 1 and sampling internal h, is stable for 0 < К < 2a coth(ah/2) only. This requires some analysis as follows: F o r F ig . 2. 2b: (z-transform of output) = z transform of unit impulse response of К /s (s+a) X (z-transform of input — z-transform of output) IAEA-SMR-17/1 61 and we are interested in the "closed-loop z-transfer function" H(z)/[1 + H(z)] whose H(z) is the z-transform of the unit impulse response. Now the (continuous) unit impulse response of K/s(s+a) is the inverse Laplace transform of this function of s which is (К/а) (1-e'at). The function H(z) is g iv e n by K f z z_ a 4 2 - 1 z_e Stability of the closed loop depends on the zeros in z of 1 +H(z) = 0 which must lie inside the unit circle. W e h ave z 2 - z (l+ e"ah) + e’ ah + — (1 -e_ah) z = 0 Q2 1 2 a" ■2 -2 FIG. 2.3 . Unit triangle. Now the roots of z 2 + a xz + a2 = 0 lie inside the unit circle if (а1( a2) lies inside the triangle shown in Fig. 2.3. W e h av e a 2 = e'ah and 3l = J (1 -e"ah) - (l+ e 'ah) and so - 1 -e"ah < j (1 -e"ah) - ( 1 +e’ ah) < l+ e"ah o r 0 The general stability criterion for roots of polynomial equations in z to lie within the unit circle on the complex plane is associated with the names of Schur, Cohn and Jury. One version is given below (Jury): Let the polynomial be F(z) = an +a.z +a„z2 + . . . +a zn, a > 0 ' ' 0 1 2 n ’ n We construct the following table of coefficients and 2 X 2 determinants formed by cross-multiplication: 62 PARKS ... a n an-l an-2 • ■ a2 a i a o a 0 a l a2 a n-l a „ A l = b 0 b i b 2 . . . b j cr V l ^n-2 bn-3 о Д 2 = c o C 1 C2 •• Cn-2 Cn-3 Cn-4 • -. . . c 0 <а с i = t 0 *1 w h e re _ a n a k b 0 ^n-k-l c 0 cn-k-2 » Ck = * = , e tc . a 0 a n-k' W - l c n-2 Ck then the conditions for all the roots of F(z) = 0 to lie within the unit circle a r e F(l) > 0, (-1)" F(-l) > 0, А г > 0, Д 2 > 0, ... > 0. 2.3. Example of dynamical systems described by difference equations — the h o g c y c le Difference equation models occur in various fields of applications. A famous model appearing in many textbooks of mathematical economics is the "hog cycle" ("hog" (USA) = "pig" (UK)). We suppose there are two curves describing "demand" D and "production" P versus price p as in Fig. 2.4 . We suppose that farmers base their production on the previous year's price and that the price adjusts itself so that supply equals demand. We have: g(Pn) = D (n ) = p (n ) = f(Pn-l) If we approximate the curves of f and g by straight lines so that P = a p + b D = -cp + d; а, с > 0 th e n -cpn + d = ap,,.! + b o r the hog price cycle will be oscillatory as -а/с < 0, and stable or unstable depending on whether a/c < 1 o r > 1 respectively. If the cycle is stable it tends to the equilibrium point where the production and demand curves intersect. The diagram in Fig. 2.4 which traces the movement of prices is known as a "cob-web diagram ". IAEA-SMR-n/1 63 Q. О o' Z D,P p. PRICE {/) о Û_ U_ о P d Z D,P FIG. 2.4. The hog cycle. -J—lPo—- 1} P , PRICE 2.4. Stability In the m aterial considered so far stability has involved the decay of the complementary functions as time tends to infinity. This has involved roots in the left-hand half of the complex plane or roots inside the unit circle for the continuous time and discrete time systems, respectively. More precise definitions of stability can be given when the systems are expressed in phase space or in state space form. Consider the unforced second-order differential equation (a) By introducing the phase-plane co-ordinates xj = y and x 2 = dy/dt we may replace the single second-order equation by two first-order equations: d x , 2 dt a2Xl - aiX2 or in matrix form О I—» ii X 1 _ ¿ 2 . - - a 2 ~a l - L x 2J Instead of plotting the solutions of (a), i.e . y(t) or dy/dt with respect to tim e, we consider plotting the phase-plane diagram of (b) in which we plot x 2 (= dy/dt) against Xi (= y) where, for varying t, the point (xj, x2) describes a curve or trajectory on the phase-plane. Figure 2.5 shows some of the possibilities for various values of ajand a2, where the arrows indicate 64 PARKS FIG.2.5. Phase-plane diagrams: (a) a1>2 : 0, a| < 4a (b) al — 0, ^ (c) ay > 0, a^ >4ag. increasing time. The origin of the phase-plane is itself a solution and trajectory and is a singular point or equilibrium point. The stability of an equilibrium point may be defined by considering the behaviour of trajectories in its neighbourhood. If from all starting points in a small spherical neighbourhood of the equilibrium point the trajectories subsequently stay close to the equilibrium point, then the equilibrium point is stable. If, in addition, the trajectories tend to the origin as time tends to infinity then we say the equilibrium point is asymptotically stable. Thus the origin is asymptotically stable in Figs 2.5a and 2.5c but stable only in Fig. 2. 5b. A more form al mathematical definition of the local stability properties of an equilibrium point represented by the vector x 0 is as follows: If given any e > 0 there exists a 6 = 6(e) > 0 such that for all l|x(t„) - x0ll < 6< " x (t) - x 0 11 < e f°r all * - t0, then the equilibrium point x.Q is stable. If, in addition, lim | [ x(t) - I = 0 th e n x 0 is asymptotically stable. In this definition || • || represents the Euclidean norm. A number of more sophisticated definitions such as "uniform asymptotic stability" may be developed from examination of the lim iting process 11 x (t) - x 0 11 -*■ 0 as t -*■ °o. So far we have considered as an example a linear system only (Eqs (a), (b) and Fig. 2.5). A non-linear system may have a number of equilibrium points as shown in Fig. 2.6 for soft-spring oscillations (Fig. 2. 6a): mx + cx + k,x - kgX3 = 0 k i K3 3 с x 9 ----- x-, + — x , -----x„ m m 1 m ¿ m , c, k 1( k3 > 0 IAEA-SMR-17/1 65 FIG.2. 6. Equilibrium points of simple non-linear systems; (a) soft spring, (b) asymmetric spring. and an asymmetric spring (La Salle) (Fig. 2. 6b): x + | x + 2 x + x 2 =0 •2 . 1 ¿ 2 = ' 2xl ' x i 3. STABILITY. I. 3.1. Liapunov functions An important idea for investigating the stability of equilibrium points of linear and non-linear systems is the concept of the Liapunov function (Liapunov, 1892). This is illustrated in Fig. 3.1 where the origin of the system shown is surrounded by a nest of closed curves given by V (xx, x2) = bnxi + 2b 12x 1x2 + b 22x 2 = constant > 0 w h e re x i = x 2 "I x 2 = a2x 1 " alx 2 ' are the system equations discussed in Section 2.4. with a, >0, a„ > 0. 1 ¿ 66 PARKS We exam ine the rate of change of V follow ing the trajectories of the system equations. This quantity, som etim es called the Eulerian derivative of V, is given by dV _ _9V_ ЭУ dx2 dt Эхх dt Эх2 dt = (2b11x 1 + 2 b 1 2 X 2 ) x 2 + (2 Ь 12Х 2 + 2 b 22X 2 ) ( - a 2 X j - a 1 x 2 ) on differentiation by parts and substitution for Xj and x2 from the system equations. We now try to choose bn, b12 and b22 so that (i) the contours of V form closed curves surrounding the origin, (ii) dV/dt g 0, but ^ 0 except at x = 0. If this is possible we can deduce that the origin is an asym ptotically stable equilibrium point, for V w ill decrease continually along any trajectory, which m ust be m oving inw ards through the contours of V and thus tending to the origin as t ^ °°. In the case of a linear system such as that given in Fig . 3. 1 we can make dV/dt i 0 by equating it identically to a negative-definite quadratic form (e.g. -2xj - 2x|) w hich w ill in general give rise to 3 equations for the 3 unknowns Ьц, b 1 2 , b22. We m ay then examine V for positive definiteness for which necessary and sufficient conditions are Ьц > 0, Ьц b22> bi2. These conditions w ill also be necessary and sufficient for asym ptotic stability of the origin. We note that V -» 0 as t -*■ and since V m ajorizes the norm II x I) = s/ xf + x|, this proves that ||x || -» 0 which is what we m ean by asym ptotic stability of the point x = 0_. This idea m ay be generalized into general situations, for exam ple to discuss the stability of system s governed by partial differential equations. W ith non-linear system s we m ay still be able to find Liapunov functions and deduce stability properties, even though we cannot find any explicit solutions of the non-linear differential equations them selves. This m akes the technique particularly attractive. Certain system s, e .g. m odel-reference adaptive control system s, m ay be synthesized as stable system s by choosing V positive definite and m aking dV/dt negative by suitable feedback arrangem ents. 3 .2. .Construction of Liapunov functions Several methods for constructing Liapunov functions have been devised, som e of w hich are briefly listed below: IAEA-SMR-17/1 67 The Liapunov m atrix equation for linear system s Given the linear constant system s x = Ax and the quadratic form V = _xT P x then dV/dt = xT (PA + A TP)x and, by equating dV/dt to -xTQx, we obtain the Liapunov m atrix equation P A + ATP = - g The variable-gradient method Given the system of equations x =_f(x), we consider the gradient vector j _ ( 9V 9V 9V Y sradV = W ' ¿v •■••srj and try to construct its elem ents so that ( i ) ( g r a d V)T- f ^ g s 0 (ii) grad V is in fact the gradient of a scalar. If the vector to be found is g_ = (gj, . . ., gn) then (i) £T • f s 0 ( i i ) c u r l g_ = 0 where the n X n m atrix cu rl£ is defined as a m atrix with its (i, j)th element gij given by _ 9 g j 9 g ¡ 9 x ¿ 9 x j The curl m atrix is antisym m etric by definition, and so there are ^ conditions on the g¡ to be satisfied, nam ely 9 g j 9 g ¡ a í = a i ; ^ = 1 < 2 ...... n , A certain arbitrariness is present in this technique. The Zubov method This is of particular interest when the f¡[ (elem ents of J) are algebraic form s with linear, quadratic, cubic, ...., parts. The Liapunov function V is taken as a sum of V2, quadratic, V3, cubic, V4 quartic, . . .. form s. F irst the linear part of_f is used to construct V2 with a negative quadratic form as derivative, using the technique for linear system s. Then V3 is found using V2> the linear and quadratic parts of f to keep dV/dt unchanged. V 4 , .... are found sim ilarly in succession in a logical but increasingly com plex process. There m ay be problem s of convergence and sign- definiteness of the resulting function V. 68 PARKS Integration by parts This is an im portant idea which has been exploited by Brockett to provide an im portant connection between Liapunov functions and frequency response methods. The following is a basic theorem: Theorem (Brockett, 1964) S u p p o s e p(D) = Dn + ajD "'1 + a2Dn'2 + . .. + an where D = d/dt and the a¡ are real. Let q(D) be another polynom ial in D of degree less than or equal to that of p(D) and such that q(z)/p(z) is a positive real function. If q(D), p(D) and [E v . q(D)p(-D)]" do not have a common factor then V ( x ) = J q(D)yp(D)y - { [ E v . q(D)p(-D)]‘ y}2 dt is a positive definite Liapunov function for p(D)y = 0 with a non-positive derivative, where x is the phase-space vector X = [ y , D y , ...... D ^ y F The theorem requires some explanatory notes: (i) A positive real function ф(z) is such that ф(z) is real if z is real and R e ф(z) Ш 0 f o r R e z ё 0. In particular ф(iu) § 0 for real и which enables the im portant connection with frequency response to be m ade. (ii) The notation (Ev. (• ) m eans the "even part of", e.g. Ev. (z3 + z2 + z - l ) = z 2 -l. (iii) The notation [ • ]" m eans that part of an even function w hich has zeros in the left-hand half of the z-plane, e.g. [ z2 - 1 ]" = [ (z + 1 ) (z - 1 ) ]' = z + 1. (iv) V(x) is in fact a quadratic form in the phase-space variables x i = У» x 2 = ° У - • • • - x n = D ^ y . An exam ple of this technique is given below: p(D) = (D + 2) (D + 3) = Dz + 5D + 6, q(D) = (D + 1) We note that is positive real (see Fig. 3.2). P(Z ) Ev. q(D) p(-D) = Ev. (D + 1) (DZ - 5 D + 6 ) = E v . (D3 - 4D 2 + D + 6) = - 4D2 + 6 . [E v . q(D)p( -D ) ]’ = [ -4 D 2 + 6 ]" = I (2D +■/6} ( - 2 D + JW )]' = ( 2 D + JÏÏ) Figure 3.2с n shows ь. a plot t , of л q(icj) 6 + 4u2 + i(u - (j3) n p ( iu ) ( 6 - u ¿)¿ + 2 5 u z IAEA-SMR-17/1 69 Im w =0 J FIG. 3.2. Plot of q(iw)/p(iu). = J (ÿ ÿ+ ÿ y+ 5 ÿ 2 + 5yj+6yÿ-4y2-4 yÿ dt = l ÿ2+ÿy + ( ll -4 ч/б") ^ = [yÿ] which is positive-definite. — = - (2 y + s/"6 y )2 which is negative-definite, dt 4. STABILITY. II 4.1. Positive-real functions and the Popov criterion The positive real-function idea may be extended to the non-linear feed back arrangement shown in Fig.4.1 a to give the following result: Theorem (Brockett): "The zero solution of Fig.4. 1 a is asymptotically stable for all admissible f(y) if there exists an a > 0 such that (1 +ois)G(s) is a positive real function.” By an admissible non-linearity we mean (i) f(y) defined, continuous and single-valued for all y, (ii) f(0 ) = 0 , yf(y) > 0 , y f 0 , The proof of this result uses the Liapunov function formed by the integration- by-parts technique explained in Section 3. We consider the variable x in Fig.4. lb which satisfies the equation P(D)x = -f {q(D)x} We multiply by (1 +aD) q(D)x and form the new equation (1 + iíD) q(D)x p(D)x + ffDq(D)x f{q(D)x} = -q(D)x f{q(D)x} 70 PARKS pis) (a) fly) u=0 1 q(s) pTs) (b) fly) FIG. 4 .1 . Non-linear feedback system. FIG. 4 .2 . Modified polar plot. FIG.4.3. Shifted Popov line in modified polar plot. FIG.4.4 Time-varying feedback. We take V(x) as the time integral of the left-hand side having first subtracted {[Ev. (1 +aD)q(D)p(-D) ]"x }2 from both sides. The left-hand side integrates into a positive-definite quadratic form in the phase-space variables by the result for the linear system p(D)y = 0, given in Section 3, plus q(D)x a I f(cr)dcr 0 which is also non-negative for a > 0. The derivative |=-q(D )x f{q(D)x} -'{[Ev. ...]'x} 2 s 0 The condition of the theorem that (1 +as)G (s) is positive-real has an important graphical interpretation if we consider s = iw, for then the plot of u Im G(iu) against Re G(iu) must lie to the right of the straight line through the origin with slope i/o (see Fig.4. 2). The diagram is called a "modified polar plot" and the straight line the "Popov line". Of course the multi plication of Im G(iu) by u considerably changes the diagram from the more familiar frequency response diagram where Im G(iu) is plotted against Re G(iu). IAEA-SMR-17/1 71 FIG. 4. 5. Circle theorem disc. A useful extension to this result in which further restrictions on the non-linearity are traded in for relaxations on the transfer function is embodied in the following theorem: "The system of Fig.4.1 a has an asymptotically stable null solution for all admissable f(y) such that 0 § y f(y) s ky2 if there exists an a > 0 such that (1 + «s) G (s) + 1/k is positive real. " This yields a new Popov line as shown in F ig .4 .3 , the modified polar plot of G(iu) having to lie to the right of this line. 4.2. The circle criterion This technique can be extended to the time varying system shown in F ig .4 .4 when the " circle theorem " is obtained: that is the loop of F ig .4.4 is asymptotically stable provided the Nyquist locus G(iu) = q(iu)/p(iuj) does not encircle or enter the circular disc with diameter ( - 1/ 0 , 0 ), {-1 / or, 0 ) as shown in F ig .4. 5, where 0 s (3 < f(t) < a < °о (Notice that here we are using the usual Nyquist diagram, not the modified polar plot of the previous discussion. ) The proof is indirect, using first the following theorem: "if is positive real, the loop of Fig.4.4 is asymptotically stable for 0 < f(t) < k." Adding and subtracting /3q(D)x to the basic equation p(D)x + f(t) q(D)x = 0 to obtain P(D)x + |3q(D)x + (f(t) - 3) q(D)x = 0 'we can deduce that the loop is asymptotically stable for 0 < f(t) - /3 < a - (3 if »q(s) + p(s)/(3q(s) + p(s) is positive-real. The bilinear mapping W = (az + 1 )/ ()3z + 1), which maps the disc of Fig.4.5 into the left-hand half plane and q(s)/p(s) into [aq(s) + p(s)]/[j3q(s) + p(s)],completes the proof. 4.3. Linear time-delay systems A number of control processes involve pure time delays in the forward loop or in the feedback loop. An example is given in F ig .4. 6 . (The transfer-function of a time-delay T is e 'sT.) 72 PARKS u=0 FIG.4 .6 . Tim e delay system for К > 0, a > 0. The stability of the loop is conveniently handled by the Nyquist criterion, bearing in mind the frequency response function of the time-delay T is e"itjT, a unit circle in the complex plane described in a clockwise direction as to increases from 0 to + °o. The open-loop frequency response is sketched in Fig.4. 7. The closed loop will be unstable for sufficiently large K/a, since the ( - 1 , 0 ) point will be encircled. Conditions on the characteristic equation (which, for the example of F ig .4. 6 , is s + a + Ke”sT) for stability and instability have been given by Pontryagin (1942). 4.4. Equations with periodic coefficients These occur naturally in a number of problems, for example the whirling of unsymmetrical shafts in unsymmetrical bearings or in the analysis of helicopter blade dynamics, but more especially when examining the stability of non-linear oscillations since the linearized equation for small disturbances occurring on top of the "steady-state" oscillation will be a linear differential equation with periodic coefficients. The calculation of stability boundaries of these equations is usually difficult and the boundaries are com plicated. The Mathieu equation x + (6 + e cos t)x = 0 is a famous example with a stability diagram in terms of 6 and e shown in F ig .4 . 8 . Use of Liapunov functions or the circle theorem can sometimes give useful sufficient (but not necessary) stability criteria. IAEA-SMR-17/1 73 FIG. 4. 9. Growing disturbance in repeated process (ploughing). -»-| Gls)|* -j G(s)|« -| G(s) |~> FIG. 4.10. Mathematical representation of process of Fig. 4. 9. 4.5. Stability of repeated processes A topic of recent interest is the stability of repeated processes, for example coal-cutting, agricultural ploughing, machining of metals, and control' of strings of urban transport vehicles by automatic means. If a slight irregularity occurs on one run of the tractor ploughing a field will the disturbance be magnified subsequently (see Fig.4. 9), which depicts a series of plough furrows following a step disturbance in the first furrow. Mathematically the process is that of F ig .4.10 where an initial wave form is fed through the transfer function G(s) many times. The stability criterion is (i) that G(s) itself is stable (that is with all its poles in the left-hand half plane), (ii) that I G(iu) I s 1 for all real и . _ The Fourier transform of the output after n stages will be (G(iu))n U(iu) where TT(iu) is the Fourier transform of the input disturbance, and so the Fourier transform tends to zero as n-> « for all и if | G(iu) j s 1. 5. CONTROL OF SYSTEMS DESCRIBED BY PARTIAL DIFFERENTIAL EQUATIONS In recent years there has been a growth in the theory of control of "distributed parameter systems", that is systems governed by partial differential equations, as opposed to "lumped parameter systems" governed by ordinary differential equations. Before discussing control of distributed parameter systems it is necessary to discuss first two particularly important examples of the governing equations — the heat conduction equation and the wave equation. 5.1. The heat conduction equation Consider the three-dimensional Cartesian frame and a small cubic element of a conducting solid as shown in Fig. 5.1. The heat flow across the face ADEF is -k (3T/3x)X óyóz into the element where к is the thermal 74 PARKS FIG. 5 .1 . Cube for which heat conduction is considered. conductivity and T = T(x, y, z, t) is the temperature. The flow out across BCHG is k — óx ) óy 6 z Эх Эх Эх Considering the other two pairs of faces the total heat input into the element is ôxôy éz [ — Гк â îl + ^- Гк 9Т1 "к . Эх . Эу . Зу . + à 3z _ or, if к is constant, Э2Т Э2Т Фт \ Эу This heat flow has increased the temperature so that, if p is the density and с the specific heat of the material, Э2Т Э T ЭТ = PC Эх2 эу" 3z2 9t on cancelling out 6 x 6 y 6 z on both sides. This is the highly important heat conduction equation. In the steady state, when ЭТ/9t = 0, we obtain Laplace's equation Э2Т ^ Э2Т ^ Э2Т . + --T + „ 9= 0 Эх2 Эу2 9z2 which has also other important applications, for example in potential theory and fluid-flow problem s. It is usual to introduce a new constant К = k/pc called the "thermal diffusivity" so that the heat conduction equation can be written neatly as К V2T = 9t IAEA-SMR-17/1 75 V2 being known as the Laplace operator ax2 Some classical solutions of the heat equation may be found by separation of the variables: for example, a thin rod lying on the x-axis gives rise to the one-dimensional heat equation 32T _ ЭТ Эх2 9t If we assume T(x, t) = X*(x) T*(t), where X* is a function of x only and T* a function of t only, then giving solutions in X := and T* so that T(x, t) = (A cos px + В sin px) e "KP 1 p f 0 = A + Bx p = 0 The boundary and initial conditions determine admissible values for p and the constants A and В for each p - the original equation being linear we may add different solutions together. For example: "The rod is of length Í and at time t = 0 is at a uniform temperature T0. One end x = 0 is reduced to zero temperature, the other end x = $ is held at T0. Find the general temperature T(x,t)". We look at solutions T(x, t) above. When x = 0 then T(0,t) = 0 for all t so A = 0. When x = SL, T(i,t) = T0 and so we may have as a possible solution r = l where the possible values of p are гт/Í (r = l, 2, 3, . . . ) . When t = 0 we have г - 1 FIG. 5 .2. Diagram of solution to heat-conduction equation. 76 PARKS and so a Fourier analysis yields 2 B, i 0I and г — 1 The solution is sketched in Fig. 5. 2. An alternative boundary condition to a given temperature on the boundary is a given heat input. This corresponds to specifying ЭТ/Эх rather than T itself in the one-dimensional case. In particular, if one end of the rod is insulated ЭТ/Эх = 0 there. The heat equation may be expressed in other co-ordinates depending on the problem in hand, for example cylindrical polar co-ordinates and spherical polar co-ordinates. Besides describing heat flow the equation represents the "consolidation equation" in soil mechanics, the diffusion of chemicals ("Fick's law"), the "skin-effect equation" in electrical field theory, the behaviour of an electrical cable with capacitance and resistance but with negligible inductance, and incompressible viscous flow. 5.2. The wave equation Waves occur all around us — in the air as sound waves, on the surface of water, in space as radio and light waves and in elastic bodies as mecha nical vibrations. The simplest mathematical model is the one-dimensional wave equation which may be used to describe planar vibrations of a string under tension - (Fig. 5 .3). If the tension is T and the mass per unit length is m, we obtain by resolving perpendicularly to the x-axis where y(x,t) is the displacement of the string and Эу/Эх is assumed to be small. We obtain the one-dimensional wave equation found to be the velocity of waves travelling along the string). The most general solution of the equation is y (x,t) = f(x -c t) + g(x + ct) IA E A -S M R - И /1 77 FIG. 5.3. Equation of motion of a string. FIG. 5.4. Waves travelling through a string. FIG. 5.5. Motion in standing wave. These are arbitrary continuous functions. As t increases, the wave form f moves to the right at a velocity с (Fig. 5.4) (and sim ilarly the wave form g moves to the left with velocity c). Of particular importance are progressive harmonic waves y(x, t) = A cos 2?r^ = д cos 2 tt(Ï - ft) where с = fX. Here A is the amplitude, X the wavelength, f the frequency in Hz, t is the time (s) and с the wave velocity. The period of the wave is l / f and the wave number 2т / A. The harmonic progressive waves travelling in opposite directions may be combined to form a standing or stationary wave: 2 7г 2?r y(x, t) = A cos — (x-ct) + A cos — (x+ct) Л À = 2An л cos 27ГХ—— cos —2 7Г ct A A = 2A cos cos 2 tr ft A Figure 5.5 illustrates the motion. The points x = X/4 ± rX/2 (r = 0, 1, 2, ...) are all called nodes and the points x = rX/2 (r = 0, 1, 2, ...) antinodes. Example: A string of length A is fixed at each end. It is plucked at its mid point which is displaced through a distant h and released. Determine the subsequent motion. 78 PARKS (хг с(',-‘о» 1 v ^ W - V X FIG. 5 .7 . Domains resulting from characteristics FIG. 5 .6 . Motion o f string with amplitude h, in the one-dimensional case. The solution may be expressed as a sum of harmonic standing waves Г 7TCt Д. sin —— cos Í r = l for which Эу/9t (x, 0) = 0. The Br must be found by Fourier analysis of the initial wave form to give = 0 , r even The motion is illustrated in Fig. 5.6. A useful property of the wave equation is the existence of characteristics. In the case of the one-dimensional wave equation with solution y(x,t) we can set up a plane with co-ordinates x and t. In this plane с (Эу/Эх) + 9y/9t is constant along straight lines with equation et + x = constant, and с (Эу/Эх) - 3y/3t is constant along lines et - x = constant. These lines are called "characteristics" and the constancy of the two quantities с (Эу/Эх) + 9y/3t and c(9y/9x) - 9y/9t on these lines is useful in constructing general solutions, especially when boundary controls are acting. Consideration of the characteristics gives rise to the concepts of domains of determinancy and dependence illustrated in Fig. 5.7. The motion at the point (xx, t^ can be found from a knowledge of y(x,to) and Эу/9t (x, to) for Xj - с (tx - t0) < x < xi * с (ti - to) on the line t = t 0 which is a "domain of dependence". On the other hand, knowledge of y(x, t0) and 9y/9t(x, t0) on this interval determines y(x, t) for all points x, t inside the shaded triangle which is the "domain of determinancy" of the interval Xj - С(^ - tQ) < X < Xj+ c(tx - t0) on t = t0. 6 . CONTROL CONCEPTS FOR PARTIAL DIFFERENTIAL EQUATIONS. I 6 .1 . Control of the heat equation Figure 6.1 shows a feedback control arrangement for heating a metal bar. The behaviour of temperature T(x, t) in the bar may be described by the heat equation IAEA-SMR-17/1 79 /////////////////////// / : hit) Ï777it T (a.t) Gis) FIG. 6 .1 . Feedback control for heating a m etallic bar. and the heat input h(t) is given by the boundary condition 9T -k Эх x = 0 If the bar is insulated as shown, there is a second boundary condition at x = £ : ЭТ = 0 Эх x = с The temperature is measured by a sensor at x = a. We observe that the control is a "boundary control" and that we have one pointwise sensor. (In general^we might have a distributed heat input, and in theory we might be able to measure the temperature at many points simultaneously. ) Taking the Laplace transform of the heat equation we obtain dfT К (x, s) = sT (x, s) - T(x, 0) (6 . 1 ) dx2 where T (x, s) is the Laplace transform of T(x, t), given by T(x, s) = J T(x,t)e‘stdt t = o We have also the transformed boundary conditions dT (j?, s) = 0 dx ■ k S (0’ s) = H(s) = G(s) (Ti(s) ■ T(a>s» where H(s) is the Laplace transform of h(t), and T¡(s) that of T¡(t). The second-order ordinary differential equation (6.1) for T(x, s) may be solved by the "method of variation of parameters" in which we assume T(x, s) = A(x) coshs/ s/K x + B(x) sinh JsjK x so that dT jX' = n/ s/K (A(x) sinh si s/K x + B(x) cosh si s/K x) 80 PARKS provided ^"dx^ cos^ ^ S/K x + sinh -J s/K x = 0 (6.2) Now d2T(x, s) _ _s_ (A(x) cosh n/ s/K x + B(x) sinh \/s/K x) dx K + v/ s/K sinh \Ts/ k x + cosh J s/ К x^ or, substituting into (6 . 1 ), J sK ^ ^ si-1* 'fs/K x + cosh n/s/K x^) = - T(x, 0) (6 . 3) Equations (6.2) and (6.3) enable us to get solutions for dA(x)/dxand dB(x)/dx giving К x cosh ^ S/ K x dx v s K 7 At x = 0 -k sTs/K B(0) = H(s) and at x = I A(i) sinh n/s/K Í + B(i) cosh J s/K £ = 0 so that BM -ÿ ÿ k -J ‘nhi1 »»>■ Л 7к s « о x A(x) = Tj | ^ sinh siiTK Ç dÇ + С о where e J ^ s K ^ s:'-n^ ^ s /^ f dÇ + C sinh sI s f К i о + cosh n/ s/K j? = 0 _ k n/ s/K ‘ J n/ sK ÏAEA-SMR-n/l 81 C sinh \l s/K i = ¿ к cosh «J s/K l + J' cosh -J s/K (£ -f ) dÇ o giving, finally, X T(x, s) = cosh si s / K x j sinh n/s/K £ df 0 4 . A - f2bJ>loosh ^ 7 K (J .E|dç , « a s â ® - sin s/ s/K í J JsK ’ k*7 s / К s in h vs / К i 0 x _ - sinh sii/Kx{ J cosh ^/K f d? +кТ^к} 0 We notice that this is made .up of two parts ~ one depending on the initial temperature distribution T(x, 0) and one depending on H(s), the heat input. The transfer function relating T(a,t) to h(t) or T(a, s) to H(s) is therefore cosh ‘Js/KSl cosh s /s /K a sinh -J s/K a _ cosh -J s/K (i -x) k n / s / K sinh n/s/K j? ks/s/K k ч/ s/K sinh si s/K & If a = 0 this reduces to cosh ч/s/K i . ~ " T / k .... <6- 5> and if a = 1 we have the transfer function k s/s/К sinh s/ s/K jP (6 - 6 ) We may analyse the stability of the closed loop using the Nyquist criterion in which case we are interested in the frequency response of Eqs (6.4) - (6 . 6 ). The frequency responses of (6.5) and (6 . 6 ) putting s = iu are shown in Fig. 6 . 2a and b. We notice that if H(s) is a constant gain К then the open loop frequency response will encircle the (-1,0) point for sufficiently large К unless a = 0. (From a practical point of view this problem is somewhat artificial as it is unlikely that h(t) can go negative, unless heating and cooling arrangements are available at x = 0 . ) FIG. 6 .2 . Frequency responses o f Eq. (6. 5) (a) and (6.6) (b). 82 PARKS Another way of relating T(x, t) to h(t) is by assuming a Fourier series for T (x,t) in the form r = 0 which satisfies the boundary condition at x = i . However, to satisfy the boundary condition at x = 0 we actually arrange for it to be satisfied at x = 0 + by considering the modified differential equation К Э2Т ^ К ЭТ + к 6<х>ь^ = ST which contains a Dirac delta function at x = 0. On substitution of the Fourier series for T(x, t), multiplication by cos and integration from x = 0 to x = Í we obtain • , K r V д 2K , , „ „ r Jp Ar ■ kjg r 1,2,3,... Á 0 = i f h« : r = 0 If h(t) is a unit impulse we obtain .. 2K V -r27tzKt/cz . К {X' > = kJ L 6 co s(r 7Tx/i) + — r = l (notice that this is not convergent for t = 0 ). Taking the Laplace transform cos (гтгх/H ) К T(X,S) kJ? X ( s + r 2TT2K/£2) + kis r = l It is not immediately obvious that this is identical to Eq.(6.4) obtained earlier which was (putting a = x) tv ^ - cosh -J s/K (f -x) (X,S) k nTs/K sinh \Ts/K Í The first expression may be regarded as a partial fraction expansion of the second, using the relationship Sinhz = z П ( 1 + ^ 2 We notice that extension of feedback control ideas to partial differential equations involve a number of difficulties: more complex solutions, more elaborate Laplace transforms, convergence of series, and use of Dirac delta functions. A comprehensive account of control of distributed parameter systems still has to be written. IAEA-SMR-n/1 83 7. CONTROL CONCEPTS FOR PARTIAL DIFFERENTIAL EQUATIONS. II 7 .1. Control of the wave equation Figure 7.1 illustrates an angular position control of a uniform flexible shaft. The torsion of the shaft gives rise to the one-dimensional wave equation _320 _ ^ ТЭ20 Э2© n Эü2c.длв z 0 , o GJ = — y where c^ = — ■ W = G J ^ OT 9Ï d x ¿ I FIG. 7.1 . Angular position control o f flexible shaft. I is the moment of inertia per unit length in the x-direction and GJ is the torsional stiffness of the shaft (in conventional notation). The boundary conditions are where C* is the torque produced by the motor. While we could develop a transfer function approach which would be quite similar to the procedure in Section 6 , we shall consider here a different problem, that of "controllability". In particular, we shall consider what torque C* should be applied to bring the shaft to rest from any given initial condition which involves specification of 0 (x, 0 ) and 3 0 /3t (x, 0 ) for each x, 0 § x § H. Let us suppose we require 30/3x(x, T) = 0 and 30/3t (x, T) = 0 for all 0 s x s S and T to be as small as possible. In this problem the characteristics in the(x, t)-plane are very useful. We know from Section 5 that 90 90 rr + c — is constant on lines x + ct = constant dt о x and — - c — is constant on lines x - ct = constant. 3t Эх Figure 7.2 shows the characteristics diagram when T = 2i/c. Let us suppose that the objectives have been achieved at t = T = 2 i/c. Then at P, both 3 0 / 3t and 3 0 / Эх are zero, and so 3 0 / 3t + с Э0/ Эх is zero along PA and 3 0 / 9t - с Э 0 /Э х is zero along PC. 84 PARKS FIG.7.2. Characteristics diagram for T = 20/c. At С the torque C* (t) is given by 2 1 -x - G j f = C * Эх and so 96_ _ 8в_ С * / 2¡t -> at C Эх ; GJ V с A lo n g C D эе , эе . . -2cC * f 2£ -x —— Y с — = constant = at Эх GJ A t D эе „ эе 2 с с* f 2i -х э 7 эГ = and along DP 1 a n d at P 1 эе эе . . 2сС* Í 2$-х —- - с — = constant ------77- I —-— (7 .1 ) at Эх GJ \ с Similarly, following the path P А В Р ', we obtain that at P' эе ^ эе с С* / x — + с — 2 ( 7 .2 ) at эх GJ Ve Thus the torque С* (t) is uniquely determined from (7.1) for £ / c S t S 2i/c and from (7.2) for 0 S t s f/c given the values of Э в /d t a n d Эе/Эх a t t = 0 an d fo r 0 s xll. The angle turned through at x = 0 may be found by taking the integral of 90/ St (0 , t) which is equal to -cC* (x/c) - c C * ((2 l -x)/c) and GJ GJ at В and C, respectively. Hence, from (7.1) and (7.2), fl(0, T) = 6(0, 0) + I ff (x, 0 )^ IAEA-SMR-17/1 85 A more general problem is how to move the shaft from an undeformed position with S(x, 0 ) = 0 , 30/3t(x, 0 ) = 0 , 0 S x s £, to a new position with 6(x, T) = e,ae/3t(x, т) = о, о § x s s. We may apply a pulse torque of magnitude C* constant over the time interval (-h, 0 ) giving initial conditions Э0 cC 36 C* . . ^ ^ . — = - r r r , -— = - — for 0 S x S c h at t = 0 3t GJ Эх GJ To bring this to rest we require a torque of magnitude -С''1' for [2£/ c) - h < t < 2H/ с . The total angle turned by the shaft is 2C* h/Ic, the total time(2i/c)+ h. By increasing C* and decreasing h we can achieve a given angle in time 2i/c , by use of two impulses at t = 0 and t = 21/c. We have concentrated on the T = 2 i/c case as an examination of the characteristics diagram corresponding to Fig. 7. 2, which reveals that in general the problem of bringing the shaft to rest in a time less than 2i/c is im possible, no matter what size the torque C* (t) is permitted. This is in marked contrast to controllability of lumped systems which may be brought to rest instantaneously if large enough controls are applied (using, of course, Dirac delta functions and their derivatives if need be). 7.2. Parasitic oscillations Many flexible devices having stiffness and inertia and obeying the wave equation or similar equations (the beam equation, for example) exhibit unstable parasitic oscillations when they are included in a feedback loop, that is when they form a link between an actuator and a feedback instrument. These parasitic instabilities may be analysed by use of the Nyquist diagram and may sometimes be avoided by careful siting of the feedback instruments. In particular, if it is possible to site the feedback instrument close to the actuator, the time delay involved in waves travelling from one to the other is avoided. It is really this time delay which causes the instability. 8 . NOISE IN LINEAR SYSTEMS An important part of linear system theory is concerned with the treatment of random inputs and outputs. Given the "power spectral density" of the input signal, then the power spectral density of the output signal is easily calculated, and other quantities such as the mean square output may then be found. Examples of random inputs to systems include the motion of a m otor car on its suspension when running over a rough road, and glint noise entering an automatic tracking radar scanner or a homing missile guidance system. Given a collection or ensemble of records of a random signal plotted as functions of time (Fig. 8 . 1), we distinguish two kinds of average of the properties of the records: (i) a time average taken along a particular record, (ii) an ensemble average taken at a particular time across the collection of records. If the ensemble averages of various properties are constant with time the records are said to be "stationary". 86 PARKS TIME AVERAGE FIG. 8.1. Ensemble and time averages. -tWO . yl«A „ i J l T L »' ply) •0.4 ply) ^ 3 1 o A 0.1 1 1 , I 1 1 - г t - 3 - 2 - 1 0 1 2 3 1 V 0 A y ^ - à - o '-'n°2 0 p(y) = l-Sly+A)+f Лу-А) ( a ) (b) FIG. 8.2. Amplitude probability distributions; (a) Gaussian noise, (b) random square wave, (c) sine wave. If;in addition,the time average equals the ensemble average, then the records satisfy the "ergodic hypothesis". We thus have the set-theoretical formulation: non-stationary signals Э stationary signals D ergodic signals The "ergodic hypothesis" is often made in engineering problems in the absence of evidence to the contrary. The most important properties of a random signal are (i) its amplitude probability distribution (ii) its frequency content or power spectral density. The amplitude probability distribution p(y) may be measured as an ensemble average, or on a particular record by considering the proportion of time p(y)6 y spent by the signal in a small interval (у, у + 6y) as 6 y -* 0 . Some examples are given in Fig. 8.2. The frequency content or power spectral density S,;:(u) may be regarded as the power or mean square output S*(u) 6u from an ideal filter with centre frequency и and small bandwidth 6u as 6u ~ 0. Mathematically it is usual to distribute the power spectral density equally over positive and negative ш, taking S(u) = |S*(u). While there are spectrum analysers which work in this way using analogue or recorded signals, another approach to spectral analysis is via the autocorrelation function of the signal y(t). The autocorrelation function R(t) is the time average y(t) y(t-r). Note that R(0) is the mean square of the signal and that R(r) = R(-t ) s R(0). The IAEA-SMR-n/l 87 autocorrelation function R(t ) and the power spectral density function S(u) are Fourier transforms one of the other: - Since S(oj) and R( t ) are both even functions we may rewrite these relationships as r =0 0 0 If we put GJ = 27rf, where f is in cycles per second (Hz) (u being in radians per second), the integrals become as shown, where S*(f) = 27rS*(u) is measured in power units per cycle per second R ( r ) c o s 2 i t î t d r R(t ) = S*(f) cos 277-fr df f = 0 known as the Wiener-Khintchine relations. This relationship may be demonstrated by taking a finite length 2T of the random signal, carrying out a Fourier analysis (regarding it as periodic with period 2T), calculating the power from the Fourier coefficients and an estimate of the power spectral density by distributing this power over the frequency interval 2ît/ 2T. The resulting expression may be written as a double integral. This integral may be evaluated in a different way giving an expression involving y(t)"y(tTr) with -T < т < T. By letting T the first of the Fourier transform pair is obtained. Some examples of the Fourier-transform relationship are given in F ig .8 .3. The reciprocal relationship between R(t) and S(u) should be noted: a "narrow" R( t ) corresponds to a "broad" S(u), and vice versa. Extreme cases of this are shown in Fig.8 .4 where we have y(t) as "direct current" and "white noise" signals. While "direct current" is a realistic signal, "white noise" is in fact a useful mathematical concept which cannot, how ever, exist in reality, since its mean square or power is infinite. We now have to consider the response of a stable linear system with transfer function H(s) and impulse response h(t) to a random input of known power spectral density Gl:1(w), say. There are two ways of thinking about 88 PARKS A sin FIG. 8.3. Autocorrelation functions and power spectral density functions; (a) sine wave, (b) wide-band noise, (c) narrow-band noise. y(t) = A (CONSTANT) y(t)="WHITE NOISE R(r) \* RM * I 2itG 0 SM SM 0 to FIG. 8.4. "Direct current" and "white noise". this — first considering the effect of the transfer function on sine waves, we can deduce heuristically that the power spectral density of the output G00(u) is given by - |H(iu)| Сгц(и) A more sophisticated approach is via the cross-correlation function R oi(t) = y0(t) yj(t - t ) and its Fourier transform, the cross-pow er spectrum G 01(u). Now Уо (*) = f h(r) yx (t - t ) dT t=0 from which it follows, on multiplying through by y1(t -Tj) and taking time averages, that R 0i(Tl) = J Ь(т) R jjf T j-t ) dT T = о where R u (t ) is the autocorrelation function of yx(t). Taking Fourier transforms yields G01(u) = H(iu) G n (u) IAEA-SMR-17 /1 89 Multiplying by yQ(t -Tj) and time-averaging yields R oo(Ti )=f h (T) R io (Ti "T) dr t = 0 Hence G „о И = H(iu) G 10(u) = H(iu) G01(-w) = H (iu)H (-iu) Gu (u) = I H(iu) 12 Gn (to) which is the result obtained above. The mean square output, cr2, may then be calculated as a2 = J' G 00(u) du = J' |Н(ш)|2 Gn (u) du = J H(iw) H(-iu) Gn (u) du This integral may be calculated by the residue theorems of the complex integral calculus, and standard forms for polynomial fractions in u are available up to degree 9. For example: if Gu = G constant ("white noise" input) and H(iu) = ^ .K,. K, a, b > 0 ' ' ( i u +a) (iu +b) then we can consider the integral Г K2G dz J (z+ a) (z+ b) ( - z + a) ( -z +b) с where С is a large semicircle in the left-hand half-plane and its diameter (the imaginary axis). The integrand has simple poles at z = -a and z = -b inside С and so by the residue theorem Г K2G dz P K2G i du . о (z+a) (z+b) (-z+a) (-z+b) J (iu+a) (iu+b) (-iu+a) (-iu+b) = 2iri (sum of residues at poles within C). „ . , lim K2G , lim K2G = 27Г1 ' ------• • ...... • - a (z +b) (-z +a) (-z +b) “ b (z +a) (-z +a) (-z + b) _ Oiri K2r I i i - - . - . _ i __ . . \ (b - a) 2a (a +b) (a -b ) (b +a) 2 b = 27ri K2G , * 0 ■ (a +b) 2 ab 90 PARKS Hence it follows that 2 - ttK2G (a +b) ab An analogous theory exists for discrete time systems described by difference equations. The output sequence {y 0(t)} is related to the input sequence {y 1(t)} (where t takes discrete values ...-3 , -2, - 1 , 0, 1, 2, 3, by the relationship n = 0 where the sequence {h(n)} corresponds to the unit impulse response h(t) in the continuous case described above. With a stationary random input with the autocorrelation function R 11(t) = yi(t)yj(t-т), defined for integer values of t , we obtain R oo(t)='X X h(k)h(-f)R n (T+i - k) k = 0 i = 0 The spectral density function is defined as S(U) = 2 7 X R(n)einw П = - OO where 7T R(n) = J S(io)e‘ inaJdu -ÏÏ (The R(n) defined for integer n are the Fourier coefficients of the periodic function S(gj) defined on the basic interval (-ж, it). ) Now the pulse transfer function using z-transform notation is H(z) = ^ h(n)z"n n - 0 and so taking the transform of Rqo(t) we obtain Soo(w) = ¿ Ê Z Z eik- h(k)e-iPwh(i) e+i(n + {- k)w Rn (n + i -k) k = 0 í = 0 = Ще-ПЩе^)Вп (и) To find the mean square output R qo(0) we need to evaluate 7Г J H(e'iw) H(eiw) Sn (u) du -ïï Such integrals can often be evaluated by considering a contour integral around the unit circle (where z = ei(J, dz = iz du) and evaluating this by use of the residue theorem. If the discrete system is stable, which is a necessary requirement, then H(z) will have all its poles inside the unit circle. IAEA-SMR-17/1 91 9. DISCRETE NOISE PROCESSES To simulate noisy systems on a digital or analogue computer it is useful to be able to make up random signals with prescribed amplitude probability distributions and with prescribed spectral density properties. Such signals are also useful for system identification using cross-correlation techniques. We shall discuss a number of examples. 9.1. The random square wave Consider time divided up into a sequence of intervals each of length h. The random square wave y(t) takes the constant value +A or - A for nh S t < (n+1) h for n = {.. ., -3, -2, -1, 0, 1, 2, 3, •••} so that (i) there is an equal probability of +A or of -A in each interval and (ii) there is no correlation between the values +A and - A in different intervals (Fig. 9. 1). The amplitude probability distribution function, p(a), is clearly two Dirac delta functions at ±A(Fig.9.2) each of magnitude j. P (a) -A 0 A a FIG. 9 .2 . Amplitude probability distribution o f Fig. 9.1. The spectral density of the random square wave is obtained from its autocorrelation function. This may be calculated by considering pairs of points, a distance т apart, with the left-hand point placed at random on the t-axis. When т = 0 we shall obtain the mean square which is clearly A2. When t > h, the mean value of y(t)y(t+r) calculated from lots of pairs of points is zero, because of the zero correlation in different intervals. When 0 < t < h the two points of one pair sometimes lie in the same interval and sometimes in adjacent intervals in proportion 1 - т/h to т/h. From this we deduce the autocorrelation function R (t) shown in Fig. 9.3. With this the power spectral density function S(u) is obtained from the Fourier transform of R ( t ) as S(tJ) = 17 J к (т )е’ 1ШТс1т _ A^h sin2 (uh/2 ) 2ir (uh/2 )2 sketched in Fig. 9.4. A modification of this random square wave is to replace the values of ±A by a sequence of random numbers with a zero mean and given variance 92 PARKS FIG. 9 .3 . Autocorrelation function of the FIG. 9 .4 . Power spectral density of the random square wave. random square wave. FIG. 9. 5, Random telegraph signal. cr2. This will produce a random stepping signal with a prescribed amplitude probability distribution and (by similar arguments) a power spectral density c^h sin2 (tüh/ 2 ) 2ж (uh/2 )2 9.2. The random telegraph signal Random telegraph signal is the name given to a signal taking the two values +A and - A with equal probability where the switching points form a Poisson process in time, with an average frequency of v switches per unit time (Fig. 9. 5). Its amplitude probability distribution is clearly the same as that shown in Fig. 9. 2, but its autocorrelation function and power spectral density are quite different from the random square wave. If we consider many pairs of points, the two points of each pair being a distance т apart, and then suppose т to be increased to т + 6т then, by considering the number of switching points which will be included in the many 6 t intervals, we deduce that R ( t + 6 t ) = - R ( t ) v 6 t + R ( t ) (1 - v 6 t ) giving the differential equation for R ( t ) ^ = -а-'Щт) Now R(0) = A2 so R(t) = A2 e’ 2^Tl. By Fourier transformation с, \ - 2 yA2 Ь(Ы) " 7г(4у2 + и2) See Figs 9. 6 and 9.7. IAEA-SMR-17/1 93 SM A2 2 i >tt 1 1 i i , -Uv -2v 0 2v Uv ы FIG. 9.6, Autocorrelation function for the FIG. 9 .7 . Power spectral density function random telegraph signal. for the random telegraph signal. FIG. 9. 8. Autocorrelation function for the differenced random square wave. FIG. 9 .9 . Power spectral density o f Fig. 9. 8, 9.3. "Differentiated" random sequences One may "tailor" power spectral densities by differencing given random sequences. For example, if we consider the random square wave or the random stepping wave formed from the random sequence {yn} say, with mean 0 and variance ct2, then we can form a new sequence {zn} given by 7-n = Уп■- Уп-i and a random stepping wave z(t) by putting y(t) = zn for nh S t < (n + 1 )h. The new wave form has the autocorrelation function shown in Fig. 9. 8 . From the Fourier transform, after some calculation, we find ол.ч - 2g2h (sin toh/2)4 b(U) 7Г (uh/2)2 See F ig.9.9. The fact that S(0) = 0 could be deduced directly from the form of R( t ) s in c e S(0) = 2 7 f R(T)dT = ° from Fig. 9.8. 9.4. An approximation to "white noise" If we examine the random stepping wave with an autocorrelation function R ( t ) = ct2 (1 - I t I /h) for O s I t I s h R ( r ) = 0, |t |> h 94 PARKS and a power spectral density олл _ sin2 (uh/2 ) b(U) 2 7Г (uh/ 2 )2 We may consider letting cr2 -* » and h -» 0 in such a way that cr2h = constant = 2tG, say. This means that the autocorrelation function becomes a delta function at t = 0 of magnitude a2h = 2irG, and the power spectral density becomes a constant (G) at all frequencies. In practice we would make h small compared with system time constants (or 2 îr/h large compared with the system pass-band), and ct2 large so that a2h/27T = G (given). 9.5. Pseudo-random binary sequences These signals, which are in fact periodic, are particularly useful for system identification using cross-correlation of output with input. There are two important classes — quadratic residue sequences and shift register or M-sequences. Quadratic residue sequences are based on work in the theory of numbers due to Gauss. We take a prime number N = 4k - 1 where к is an integer, and calculate the numbers l2, 22, 32, . . ., {4k-2/2 } 2 modulo N. We form a signal y(t) which is + A in intervals corresponding to these N - 1 numbers and is - A in the other intervals making up N intervals in all. The signal is then repeated to form a periodic signal of period Nh where h is the length of the basic interval. Ж 7h f t 2Л h FIG. 9.12. Line spectrum o f Fig. 9.10. IAEA-SMR-17/1 95 FIG. 9.13. Quadratic residue binary sequence N = 1019 96 PARKS For example: When N = 7 (k = 2) we calculate l2, 22, 32 (modulo 7) = 1, 4, 2 and thus form the signal shown in Fig. 9. 10. This signal has the autocorrelation function shown in Fig. 9.11. This gives a line spectrum (Fig.9.12) with a frequency separation of lines equal to 2 7 r /N h ra d /s. The envelope is of the form (S*n ^ )? uh/ 2 A longer quadratic residue sequence with N = 1019 and A = 1 is given in Fig. 9.13. FIG. 9.14. Tw o-level shift register with binary adder. Another form of pseudo-random binary sequence is the "M-sequence" which can be generated with a 2 -level shift register having appropriate feed back arrangements and a clock with period h. Consider the system shown in Fig. 9. 14, where a, b and с are registers and d is a binary adder. Let us initially set the binary numbers 1 in a, 0 in b and 0 in c. At each clock pulse the digits are shifted one to the right, whilst the vacant place in a is filled with the m odulo-2 sum of the digits in b and с before the shift, this sum being calculated by d. The sequence of operations is as follows: Register a b с 0 1 0 0 h 0 1 0 2 h 1 0 1 3h 1 1 0 4h 1 1 1 5h 0 1 1 6 h 0 0 1 7h 1 0 0 This pattern is then repeated. If we tap the output of the register a and convert this into a square wave where 0 becomes +A and 1 becomes - A we obtain the signal shown in Fig. 9. 15. (In this case this becom es equivalent to the waveform of Fig. 9.10 shifted by 2h to the left and the autocorrelation function of Fig. 9. 15 is identical to Fig. 9.11.) Any sequence of length N = 2k - 1 generated by a к-stage shift register is called a maximal length sequence or "M-sequence". In such a sequence the registers go through all possible states except the a ll-zero state and it follows from this that there is one more 1 than 0 in such a sequence. IAEA-SMR-И /1 97 An important property of such sequences is that if a sequence is added modulo 2 to a delayed version of itself, then the original sequence with a new delay is formed; for example Original sequence (register a) 101110010111 Single delay of h (register b) 010111001011 Addition modulo 2 111001011100 (The resulting delay is 5h. ) Modulo 2 addition of the sequence is equivalent to autocorrelation of the signal y(t) in Fig. 9.15 e.g. t 0+ h+ 2h+ 3h + 0 + h+ 2h+ 3h+ y(t) -A + A-A-A 1 0 1 1 y (t + 2h) -A-A - A + A 1 1 1 0 y ( t ) y (t + 2h) + A 2 -A 2 + A 2 -A2 0 1 0 1 It follows that the autocorrelation function has one more - A2 than +A2 adding over one period Nh. Hence R(t) = -A2/N for h S r s (N-l)h. This yields the now fam iliar form shown in F ig.9.16. FIG. 9.15. Square-wave signal for register pattern. FIG. 9.16. Autocorrelation function. d FIG. 9.17. Corresponding shift register as in Fig. 9.10. The shift register operation may be described by a polynomial in the delay operator D. Figure 9.17 shows the corresponding shift register diagram. x = D3x © D?x or D3x © D2x © x = 0 98 PARKS For example X = 0 Dx = 1 D2x = 0 D3x = 0 О О II X = 1 X D2x = 1 D3x = 0 X = 1 Dx = 1 D2x = 0 D3x = 1 looking at the table of contents of the registers a, b, с regarded as Dx, D2x and D3x. The polynomial involved, D3 © D2 © 1, is known as the "generating polynomial". To generate an M-sequence, this polynomial must be (a) irreducible, that is it must have no factors; for example D4 © D3 © D2 © 1 = (D © 1) (D3 © D © 1) is not irreducible; (b) primitive - that is it must not divide exactly into any polynomial of the form Dn © 1 for any n less than 2k - 1 where к is the degree of the original polynomial. For example D4 © D3 © D2 © D © 1 = and 5 < 24 © 1 so D4 © D 3 © D 2 © D © 1 is not primitive. Primitive polynomials up to к = 34 are given in W.W. Peterson "Error correcting codes" (Wiley, 1965); S.W. Golomb "Shift regular sequences" (Holden-Day, 1967) shows that D127 © D © 1 is a suitable generating poly nomial; it generates a sequence of length (2127 - 1) h - 1037 hi For example, if the clock frequency is 106 pulses a second the sequence will repeat itself after 3 X 1024 years.1 k We note that all irreducible generating polynomials of degree к divide D2 " 1 + 1 which means the period must be a factor of 2k - 1. If 2k - 1 is prime the period of the irreducible polynomial must be 2k - 1 and so it is also primitive. Such prime numbers are called Mersenne primes (Mersenne, 1644). 2k - 1 is known to be prime for к = 1, 2, 3, 5, 7, 13, 17, 19, 31, 61, 89, 107, 127 and 11213. Given a Mersenne prime p = 2k - 1, then Dk © D © 1 is a primitive polynomial. BIBLIOGRAPHY KUO, B.C. , Automatic Control Systems, Prentice-Hall (1962). LEFSCHETZ, S ., La SALLE, J.P. , Stability by Liapunov's Direct Method with Applications, Academic Press (1961). BROCKETT, R .W ., Finite-dimensional Linear Systems, W iley (1970). OGATA, K. , State Space Analysis o f Control Systems, Prèntice-Hall (1967). BARNETT, S ., STOREY, C . , Matrix Methods in Stability Theory, Nelson (1970). WILLEMS, J .L ., Stability Theory o f Dynamical Systems, Nelson (1970). La SALLE, J.P., LEFSCHETZ, S ., Stability by Liapunov's Direct Method with Applications, Academic Press (1961). IAEA-SMR-17/1 99 CESARI, L. , Asymptotic Behaviour and Stability Problems in Ordinary Differential Equations, Springer- Verlag (1959). PORTER, B. , Stability Criteria for Linear Dynamical Systems, Oliver and Boyd (1967). MINORSKI, N. , Non-linear oscillations, Van Nostrand (1962). WANG, P .K .C ., "Control o f distributed parameter systems", Advances in Control Systems (LEONDES, C .T ., Ed.). Academic Press (1964). BUTKOVSKII, A. G. , Distributed Control Systems, American Elsevier (1969). BUTKOVSKII, A .G . , Control of Systems with Distributed Parameters, Naiika, Moscow (1975). LIONS, J.L. , Optimal Control o f Systems Governed by Partial Differential Equations, Springer-Verlag (1971). WANG, P .K .C ., Theory o f stability and control for distributed parameter systems, Int. J. Control 7^ (1968) 101. HAMZA, M .H. (Ed.), Proc. IFAC Conference on Control o f Distributed Parameter Systems, Banff, Canada, June 1971, Vols 1 and 2. PARKS, P. C. , "On how to shake a piece o f string to a standstill” , Recent Mathematical Developments in Control (BELL, D.J. Ed.), Academic Press (1973) 267. JAMES, H.M . , NICHOLS, N.B. , PHILLIPS, R. S. , "Theory o f Servomechanisms", McGraw Hill (1947). ÂSTRÔM, K. , "Introduction to Stochastic Control Theory", Academic Press (1970). HOFFMANN de VISME, G. , Binary Sequences, English Universities Press (1971). PETERSON, W .W ., Error Correcting Codes, Wiley (1961). GOLOMB, S .W ., Shift Register Sequences, Holden-Day (1967). IAEA-SMR-17/2 FOUNDATIONS OF FUNCTIONAL ANALYSIS THEORY Ruth F. CURTAIN Control Theory Centre, University of Warwick, Coventry, United Kingdom Abstract FOUNDATIONS OF FUNCTIONAL ANALYSIS THEORY . This paper provides the basic analytical background for applications in optimization, stability and control theory. Proofs are generally omitted. The topics o f the sections are the following: 1. Normed linear spaces; 2. Metric spaces; 3. Measure theory and Lebesgue integration; 4. Hilbert spaces; 5. Linear functionals, weak convergence and weak compactness; 6. Linear operators; 7. Spectral theory; 8. Probability measures; 9. Calculus in Banach spaces. Some mathematical symbols 3 .there exists А с В A is contained in В e in, belongs to А э В A contains В V for аЦ E' complement of E' > such that f V g minimum of f and g IR real numbers f A g maximum of f and g С complex numbers 1 orthogonal to => implies  closure of A iff if and only if NORMED LINEAR SPACES Definition 1. Linear vector space A linear vector space is a set У= {x,y, z,...} of elements with an operation © between any two elements such that 1 . x®y = y®x commutative property 2. 3 e 3 » x ® e = x, Vxe'X' existence of the identity 3. 3 - x > x ® - x = e existence of an inverse 4. x ® (y ® z) = (x ® y) ® z associative property 101 102 CURTAIN (i.e. У is a commutative group under the operation ®) and there is an associated scalar multiplication by the real numbers IR or the complex numbers 1 . a (x ® у) = a x ® о у 1 2 . (a +p) x = a x ® |3x I where a, g are scalars 3. (o|8 )x=o(0 x) J 4. l-x = x This concept is best understood from some examples: Example 1. Take y=JR under addition and ® as ordinary multiplication. Example 2. Take У being a set of all polynomials of degree n with real coefficients and scalar multiplication by IR — this is a real vector space, but if we consider complex coefficients and multiplication by C, it is a complex vector space. Example 3. IRn = set of all n-tuples, x= {х х, x 2, ..., x n} with x + y = {x i + yi , х 2 +У2 , x n + yn} and ttx = {ttx 1, 0 x 2 ,., .,i f x n} Check that IRn is a vector space under scalar multiplication by IR. What if we take scalar multiplication by C? Example 4. У - set of all mXn matrices with real entries and scalar multiplication by R. Example 5. У - set of all scalar-valued functions f : S-IR, where S is any non-empty set. For any s G S, f(s) G IR we define (f + g)(s) = f(s) + g(s), for all s G S (fffHs) = o-fts), for all s G S, о 6 ]R. Example 6 . У= set of real-valued functions f : [0, lJ-'lR such that l о Define addition and scalar multiplication as in 5. We must verify that if f,gG У, then f+ g and o f G У, i.e. J'\J | f(s) +g(s) |2| ds 1 1 < oo where we have used Schwarz's inequality provided both sides exist. This last example is a linear subspace of example 5 with S = [0,1]. Definition 2. Linear subspace If У is a linear vector space, then a subset S of У is a linear subspace if x, y G S => ax + ¡3y £ S, i.e. S is closed under addition and scalar multiplication. Other examples of linear subspaces are Example 7. In example 3, let S be the set of n-tuples of the form x = { xj, x 2, 0 , ..., 0 } Example 8 . In example 4, let S be the set of m atrices with certain blocks zero. Example 9. In example 2, let S be the set of all rth-order polynomials, where r < n. Linear subspaces have the special property that they contain the zero element. A 'translated' subspace is given by Definition 3. Affine subset If 9^is a linear vector space, then an affine subset has the form M = {x: x = с + x0, where x0G S and с is fixed} for some c £ X and some linear subspâce S of У'. Example 7a. In example 3, let M be the set of n-tuples of the form x = {x j,x 2 , 1 , 1 }. Example 8 a. In example 4, let M be the set of m atrices with certain blocks of l's . Another very important type of subset of a vector space is a convex set. 104 CURTAIN Definition 4. A subset A of У is convex, if x,y e A implies that Xx + ( 1 - X )y 6 A for all X > 0 S X s 1. We now introduce the concept of the dimension of a vector space. Definition 5. Ifxi, ..,xnеУ and there are scalars oj, not all zero such that o 1x 1 + » 2 X 2 + • • • +onx n = 0 then we say this is a linearly dependent set. If no such scalars , ,.,o n exist, then хр x0 , ..xn is a linearly independent set. F or example, l ,x ,x 2 , ..,xP is a linearly independent set of nth-order polynomials, as is 1+x, -g- + 3x; however, 1 + x, | + 3x, 2x are linearly dependent. Definition 6 . If x 1,..,x n is a linearly independent set in У then we say that S = Sp {x x, .., xn } , the set of all linear combinations of xa, .., xn, has dimension n. If 0^=Sp{xj,.. .., xk} for some finite set of linearly independent elements, then У is of dimension k. If there exists no such set, then У is infinite-dimensional. For example, the dimension of example 1 is 1, example 2 is (n + 1), example 3 is n, example 4 is mn and examples 5 and 6 are infinite dimensional. Definition 7. If У= Sp {xi, .., xn}, the set {xx, ..., xn} is called a (Hamel) basis for У This basis is not unique, although the dimension of У is unique. Example 10. A basis for example 2 is {l,x ,...,x n} or, equivalently, the Legendre polynomials. A useful fact to remember is that all real vector spaces of dimension n are algebraically identical (or isomorphic to IRn). Definition 8 . Vector spaces У and 'Ж’аге isomorphic if there is a bijective linear map T :1)^ ^ such that T(»x + (3у) =геТх + (3Ty for all x, у e У and o,l3 scalars. Another example of isomorphic spaces are all n-dimensional vector spaces over the complex numbers, which are all isomorphic to Cn, the space of complex n-tuples. Definition 9. A hyper plane, H, of a vector space У is a maximal proper affine subset of У i.e. the complement of H has dimension 1. For example, hyperplanes in IRn have dimension (n - 1) (i.e. y=H + S, where H and S are disjoint and S has dimension 1). Hyperplanes in IRn have the dimension (n- 1). So far we have only considered algebraic properties of sets. In order to develop mathematical concepts for "nearness" or distance, we need some topology, namely metric spaces. IAEA-SMR-17/2 105 Definition 10. A metric space X is a set of elements {x,y, ...} and a distance function d(x,y) with the following properties: 1 . d (x,y)s 0 for all x,yGX 2 . d(x, y) = 0 if and only if x = y 3. d(x,y) = d(y,x) _ 4. d(x, y) s d(x, z) + d(z, y) for all x ,y ,z G X We call d (.,.) a metric on X. This is essentially a generalization of distance in the Euclidean plane. Example 11. Let X be the set of 2-tuples x = {xi, x2} or Cartesian co-ordinates and d(x,^r) = [ (xx - yx)2 + (x2 - y2 )2 ]1//2. Then d satisfies all the properties 1-4 and property 4 is the familiar triangular inequality. Example 12. Let X be as in ex. 11, but d(x, y) = |xj - y± | + | x 2 - y2 I or more generally, dp (x_, y) = ( |x}, -yj \p + | x2 - y 2 | p 1 S p<°° » and d'(x,y) = max {|xj - y j , |x2 - y 2 |}. So the same set X can generate different metric spaces. Example 13. X=^[a,b] the set of continuous functions on [a,b] and d(x, y) = max {| x(t) - y(t) I ; a s t s b } or Ь i / p dp(x,y) = (^J Ix(t)-y(t)|pdt^ ; рё 1 . a We sometimes use a pseudometric which satisfies conditions 1, 3, 4 of Definition 6 , but instead of 3, we have only d(x,x) = 0. d(x,y) = 0 does not necessarily imply that x = y. So far we have introduced the algebraic structure of a linear vector space which enables us to consider linear combinations of elements and then the topological concept of a metric space which enables us to measure nearness or distance and hence to consider the tools of analysis, such as open sets, closed sets, convergence and continuity. We now combine these two notions in a normed linear space. Definition 11. A normed linear space X is a linear vector space with a norm on each element, i.e. to each x€X corresponds a positive number ||x||, such that 1. II x ||=0 iff x = 0 2 . Il a x II = I a I II x II , for all scalars a 3. ||х + у | Ы | х | | + ||у|| If 1 is not necessarily true, we call it a seminorm. We note that if we define d(x, y) = || x - у || , then d is a metric on X . 106 CURTAIN Example 14. Consider R n again. This is already a vector space (see Ex.3). We can define several norms on ]Rn by H -C iw 'T i = l where lsp i" =/x e R n > I! x|| = max {| x¡ |}j- ^ w l< i< n J Example 15. Consider the space of infinite sequences, x= {x 1( x2,.. .} . Then this forms a vector space in a similar manner to Rn and we can again define a p-norm: Ы К Х кГ) for lsp< 00. i = 0 Note that ||xj|p is not finite for all infinite sequences, so to define a normed linear subspace we take the subset with finite p-norm , i.e. ip = {oo -tuples x = {x :, x2 ,...} , with ||x||p < oo}. This contrasts with the ip normed linear spaces where the sets of elements for each p are identical, although the norm is different. Now for || • || p the set of elements for each p are different. The actual proofs that || • ||p is a norm rely on Minkowski's inequality i =1 i = 1 i = 1 which holds for n finite or infinite. Finally =-(o o -tuples x, with ||x|| = sup { | x . |} < oo I ~ ~ l£i and so £ iC£2C Example 16. X=^[a,b], the space of real continuous functions on [a,b] with norm Il x(-) II = max I x(t) I astsb IAEA-SMR-17/2 107 This is called the uniform or sup norm Examplel7. X=d[a,b] under the p-norm b a That this satisfies the properties of a norm depends on the integral form of the Minkowski inequality: f Г ч^Р / r> nVp / p \!/Р i^J |x(t)+y(t) |pdtJ S I x(t) |P dty + |y(t)|P dtj a a a Equality holds only if x(t) = ky(t) almost everywhere on [a,b ], We note that although X = ^[a, b] is the same linear vector space in Exs 16 and 17 by defining two different norms, we obtain two distinct normed linear spaces. 2. METRIC SPACES We return to study the properties of metric spaces in more detail. There are two common ways of creating new metric spaces from known ones — subspaces and product spaces. Definition 12. Let (X, d) be a m etric space and A a subset of X , then we can consider (A,d) as a metric space in its own right. Then (A,d) is a subspace of (X, d). There are, in general, several metrics we can put on A, but (A, d i) is a subspace of (X. d) only when the m etrics di and d coincide on A. Definition 13. Let (X ,dx), (Y,dy) by two m etric spaces, then the product set of ordered pairs ÏX Y={(x,y) : xGX , y€Y^} may be defined to be a m etric space, the product space, in several ways. 1 . d(u!,u2) = dx(x1, x 2)+ d y(y i,y 2 ), where Ui = (xj,j^), i = 1 , 2 . 2 . d 2(ui,u2) = (d£(xb x2) +dy(yi,y 2 ) ) 1/2 3. dp(ui,u2) = (dg(x1 ,x 2) + dP(y1,y 2 ))ly,P; ISpCoo 4. d„(u1,u2) = max{dx(x 1(x2), dy(yi,y2)} Under any of these m etrics (X X Y ,d) is called the product space of (X, dx) and (Y, dy). (There are infinitely many choices of metrics for X x Y ). You can verify that if 3Tand ? are normed linear spaces, then (X, II • llx)X (Y, II • ||y ) becomes a normed linear space using any of the product m etrics derived from || • ||x and || * ||y. Continuity in metric spaces The introduction of the distance function d(.,.) allows us to generalize the definition of continuous functions on metric spaces. 108 CURTAIN Definition 14. Let f : X^ Y be a map from the metric space (X, dx ) to the metric space (Y, dy). f is continuous at x0 in X if given € > 0 , 3 a re a l number 6>0, such that dy(f(x), f(y))< G, whenever dx(x, x0) < 6 . f is con tinuous if it is continuous at each point in its domain. Definition 15. A map f : X -* Y is uniformly continuous if for each G > 0, 3 6 = ô (G )> 0 , such that for any x 0, dy(f(x0), f(x)) < G, whenever d„(x, x0 ) < 6 . n 2\1/2 Example 18. X = lRn under d 2 (u,v)=(^^ |u¡ - v¡ | j i= l m Y = ]Rm under d2 (w, z) = ^ ^ |w¡-z¡|2^ i = l and f : X^ Y is represented by the matrix F = (fjj ), i.e. y = Fx. Let x 0 be fixed in X and y 0 = Fx0. Let x be an arbitrary point in X and y = F x , then m n d(y,y ) 2 = Y, I Y f i i ( x j “ x o j > Î2 i = i j = i m n n sZ(X|f« |2)( ^ ,xi-x°j|2) 1 = 1 J = 1 J=1 by the Schwarz inequality s c2 d(x, x Q)2 , where c2 = | f ^ | ^ i . j So if we are given € > 0 we may choose 6 = 6 /с , provided с / 0. So the map f is uniformly continuous. Example 19. X = X = sPace of integrable functions on [0,T] with the metric y /2 d(j .„■c [x(t)-y(t)]2dtj 0 Define t — — Г f:X "* X by fx = y, where y(t) =J x(s)ds. о Then t ly(t)-y(t0)| = IJ x(s) - X0(s) ds I 0 IAEA-SMR-17/2 109 о о S \Гт d(x, х0 ) d (y,y 0 ) = ( / I y(t) - y0 (t) |2 dt^) o á T d(x,x0) so f is uniformly continuous. If, however, we consider the interval (-°°, oo), the map f :X->X is not t continuous, where fx = y is given by y(t) = J' x(s) ds. F or let x 0 € X be fixed and see к an e> 0, such that there is an x£X , such that d(x, x 0) < 6 and d(f(x), f(y)) ê E , for any choice of 5 . Let y0 = f(x0 ), y = f(x). t Then y(t) - y0 (t) = J'(x(s) - x0 (s))ds. f c, Osts 3 T 2 Choose x such that x(t) - x 0(t) = j " c > 3T2 . . d(x, x 0) = nГб Tc and í ct,0sts 3T2 y(t) - y0 (t) = j c ( 6 T 2 -t ), 3T2< t s 6 T 2 0 otherwise and d (y,y 0 ) = чП8 Т3 с. Let 6> 0 be given and choose с , T, such that \ГГ8 T 3c = 1, •J~6Tc<ô, i.e. d (x ,x 0)<6 and yet d(y,y0 ) й 1, for this particular x. So f is not continuous at x 0. A fundamental concept in analysis is of course convergence and we now define convergence in metric spaces. Definition 16. A sequence {xn} С metric space (X, d) converges to x0 in (X, d) if d(xn, x 0)^ 0 as n-*°o. Continuity and convergence are related concepts as is clearly seen from the following result: 110 CURTAIN Let f : (X, dx)-»(Y, dy) be a map between two metric spaces and x 0 a g point in X. Then the following two statements are equivalent: (a) f is continuous at x0. (b) lim f(xn) = f( lim xn), for every convergent sequence xn->xo, i.e. a n-. We now define Cauchy sequences. Definition 17. A sequence {xn} of elements in a metric space (X, d) is a Cauchy sequence if d(xn , xm) -» 0 as m, n -* oo . Metric spaces have the property that Cauchy sequences can have at most one lim it, but they need not have a limit point in the metric space, as is seen from the following example. Example 20. C on siderólo, 1] under the 2-norm and take the sequence {xn} where we define for OS t s 1/2 - l/n xn(t)= n/2)t - n/4+1/2 1/2 - l / n s t s 1/2 + l/n 1/2 + l /n S t á 1 Graphically this looks like lAEA-SMR-17/2 111 = -Í2 (difference in the area of triangles). = >/"2 I l/4 n - 1 / 4mI -*• 0 as m ,n -*co. so{xn} is a Cauchy sequence under the 2-norm. We easily see that the pointwise limit of x n is X * k 2 However, this function does not belong to d [0,1] because of its discontinuity at 1/2. It is rather awkward to use spaces which have Cauchy sequences whose limits do not belong to the space and so we define a class of metric spaces which always contain limit points of Cauchy sequences. Definition 18. A metric space (X, d) is said to be complete if each Cauchy sequence in (X, d) is a convergent sequence in (X, d). This concept of completeness is so important that it gives use to the definition of new types of spaces, as we shall see from the examples to follow. First let us state two major results about complete metric spaces: 1. If (X, d) is a complete metric space and (Y_, d) is a subspace of (X, d), (Y_, d) is complete iff T is a closed set in (X, d). Definition 19. A set A in X is closed if all convergent sequences in A have their limits also in A. 2. Every metric space has a completion which is unique up to an isometry. This last property allows us to define an important concept in normed linear spaces. Definition 20. A Banach space is a complete normed linear space (B-space). Some examples of Banach spaces are ^[a,b] under the sup norm (since uniformly convergent sequences of continuous functions have continuous limit functions), ip, ü", ip andi*,. However, from E x.20 we see that ^[a,b] is not complete under the 2-norm. This is an important norm used in analysis for defining'mean square convergence of functions': b (f n(t) - f(t))2 dt-* 0, as n-*«> a 112 CURTAIN and so we would like to identify what space we get when we complete ^[a,b] under the p-norm . At first guess, one would look at all Riemann integrable functions with finite p-norm, i.e. x(-) such that P \1/p / x(t)Pd t) <«> a Unfortunately, a convergent sequence of Riemann integrable functions does not always converge to a Riemann integrable function. However, it happens that if you define integration in a more sophisticated way — called Lebesgue integration, you do get this property. We shall pursue this idea later on, but for the present we may consider Lebesgue integration as a generalization of Riemann integration and just think of Lebesgue integrable functions as Riemann integrable functions and some "mavericks". Lp[a,b] denotes the space of Lebesgue integrable functions with finite p-norm, i.e. Vi/p ' P t(t) I P dty <00 and it is a В-space. It may also be considered as the completion of¿f[a,b] under the p-norm. The following examples of normed linear spaces are important in the theory of partial differential equations. Examples 21 and 23 are not complete, however, their completion under the || 'Hn.p norm gives rise to the important Sobolev spaces, which are used in distribution theory for studying partial differential equations. Example 21. Let<^1a,b] be the space of infinitely differentiable functions on [a,b]. Then the following are well-defined norms u a 1/p l|x|ln,p = J Y 1о1х(^1Р dt 1 g p < o o a i = 0 where D1 denotes the ith derivative. So.we can define infinitely many normed spacesX on ^"[a.b]. - - - Since ||xIl ||n+1 "U‘ A, p D g •'||x||n "11, p' _, we have that ^ “ [a,b] under the (n + l,p ) norm C «"[a ,b ] with the (n,p) norm. Example 22. Consider ) the space of real functions of n variables on Q which are continuously differentiable up to order k. Í2 is an open subset of!Rn. Let a= (oi, ...,an ) be a vector with positive integer entries and define o-j . Then for uG ^k(S2), the following derivative exists and i=l is continuous: IAEA-SMR-И /2 ИЗ ó k(S2 ) is of course a linear vector space and we may take the sup norm or, alternatively, the norm HI u HI = max {|| D“ u ||} 0 < ex — к J where ||d “ u || is the usual sup norm. ^ k(f2) is a В-space under the III- ¡I -norm. Example 23. Let ^ ”(Г2) be the space of infinitely differentiable functions on Í1, an open subset of lRn. Let » = (ffi...,on ) be as in Ex.22, and as before, define the space differential operator D“ u. Then we may define the following norms V/p lu Hk,p=:( / ^ |D“ u(x)|p dx) Q I a| £k II * \\ktp is also a norm on Although the spaces in Exs 21 and 23 are not complete they do have the important property that they are dense in some underlying space, the relevant Sobolev space. Definition 21. A linear subspace S of a metric space X is dense in X if the closure of S with respect to the metric D X . This means that any element xeX can be approximated by some element s 6 S as closely as we like, i.e. d(s,x) < G. Example 24. Consider £ 2 and let S = {x G i 2 >x = {x!,x2, . . . , X k , 0, ...0 ...}, к <°o} Then S is a dense subspace of i 2, but it is not a В-space itself, since the sequence { 1 , 0 , . . . } , { 1 , 1 / 2 , 0 , . . . } , { 1 , 1 / 2 , 1 / 2 2 0 , ...} , ... { 1 , 1 / 2 , 1 / 2 2,..., l/2 n,0,...} converges to {1, 1/2, 1/2 2 ,...} which is not in S. Example 25. L 2 [0, 1] = [0, 1] (closure under ¡ ‘ Цг norm) and S = {x(-) G L 2 [0, 1 ] with x(0) = 0 and x(-) g L î [0,1 ]}. This is not a closed subspace, since it is easy to construct sequences of functions in S whose derivatives converge to a function which is integrable, but its derivative is not. However, it can be shown that it is a dense subspace of L 2 [0, 1]. If we take a new norm P 1 *(-)lll = J ( |x(t) |2 + I x(t) |2) dt I for S, then S is a B -space with respect to this norm. We also define the concept of connectedness. 114 CURTAIN Definition 22. A metric space (X, d) is disconnected if it is the union of two open, non-empty disjoint subsets. Otherwise (X, d) is said to be connected. Example 26. X = [0, 1]U[5, 6 ] with the Euclidean metric is disconnected. Contractions in metric spaces This is an important concept which gives us the basic tool for proving existence and uniqueness of solutions of differential equations. Definition 23. Let (X, d) be a metric space and f : X~*X. Then f is a contraction (mapping) if there is a real number K, 0 S K< 1, such that d(f(x), f(y)) SKd(x.y) Vx.y EX. This implies that f is uniformly continuous. Contraction mapping theorem Let (X, d) be a complete metric space and f : X~>X a contraction mapping, then there is a unique x 0e X» such that f(x0) - x 0 • Here x 0 is called the fixed point of f. M oreover, if x is any point in X and we define the sequence {x n} by xj=f(x), X2 = f(Xj), ..., Xn = ftXn-j), then x n -* xo as n -♦ » . Corollary Let (X, d) be a complete metric space and f such that fp is a contraction for some p > 0. Then f has a fixed point. Example 27. Existence and uniqueness theorem for solutions of ordinary differential equations Consider (1) y(0) = 0 where f is real-valued, continuous on EX И. Then (1) is equivalent to the solution of the integral equation о IAEA-SMR-17/2 115 and this may be thought of as z = F(y), where. z(t) = о and F : Á, the space of real-valued continuous functions defined on [0 - А, 0 + А]. Now y(t) is a solution of (1) iff y= Fy, i.e. iff у is a fixed point of the map F. We now show that under appropriate conditions on f, F is a contraction. Assumptions on f: (a) |f(y,t)|sM for -liyS l, -lstsl. (b) Lipschitz condition | f(y, t) - f(x, t) | SK|y-x for (y,t), (x,t) in [ - 1 , 1 ] x [ - 1 , 1 ]. Let X = space of continuous real-valued functions 0(t), such that I 0(t) I S M 111 on [- T, T], OSTSI. MTSl, KT<1. X is a subspace of ^[-T,T] and is closed under the sup metric dK, i.e. QL d^) is a complete metric space. (i) F : X - X , since о о (ii) F is a contraction, since t for t 2 0 I F(x)(t) -F(y)(t) I = I J U(x(s),s) -f(y( s), s)] ds о о s Ktd«-.(x,y). for tsO, I F(x)(t) - F(y)(t) I g K|t|d„(x,y) 116 CURTAIN and so for 111 ST, djF(x),F(y))sKTdjx,y) F has a unique fixed point which is the solution of (1). Compactness Compactness is a very important concept in analysis, although it does not arise in finite-dimensional spaces as such. For finite-dimensional spaces all closed and bounded sets are compact. This is never true in infinite dimensions and we here define compactness for metric spaces. Definition 24. A set A in a metric space (X, d) is compact if every sequence in A contains a convergent subsequence with a limit point in A. We recall that in IRn every closed and bounded set contains a convergent subsequence. For more general topological spaces there are several different kinds of compactness, the definition above corresponding to sequential compactness. However, for metric spaces all types of compactness are equivalent. Still we can define a weaker form of compactness, relative compactness. Definition 2 5. A set А С X is relatively compact (or conditionally compact) if its closure A is compact. This means that every sequence in A contains a convergent sub sequence which converges to a point not necessarily in A. We also note the following properties of compact sets: 1. If (X, d) is compact, it is a complete metric space. 2. A compact set is closed and bounded. 3. If A is compact, every infinite subset in A has at least one point of accumulation — the Bolzano-Weierstrass property. (This is also an equivalent definition of compactness.) 4. If f:X -Y is continuous and ACX is compact, f(A) is compact. 5. A continuous function f: A->IR achieves its minimum if A is compact. Finally we shall state the Arzela-Ascoli theorem which will be used in the application sections. Let (X, di) be a compact metric space and (Y, d2) a complete metric space and A(X, Y) the space of continuous functions on X with range in Y . (/(X,Y),p) is a metric space, where p(f, g) = sup{d2 (f(x), g(x)), x £ X } and in fact (^(X,Y),p) is a complete metric space. Definition 2 6 . A c/(X ,Y ) is equicontinuous at xneX. If given £>0, 36 < 0, such that d2 (f(x) - f(x0 ) ) < 6 VfGA whenever dx(x - x 0) < 6 and xeX. _ A is equicontinuous on X if is is equicontinuous at all points in X . IAEA-SMR-17/2 117 Arzela-Ascoli theorem A c/(X ,T ) is relatively compact in (^(X,Y),p) iff A is equicontinuous on 'K and for each x E 'K. A(x) = { f(x) where f€A} is relatively compact. Deeper topological notions in metric spaces We have already mentioned that metric spaces are a special class of topological spaces and here we shall explore some of the fundamental topological concepts for metric spaces. First we show how the metric space may be characterized in terms of neighbourhoods. Definition 27. Let (X, d) be a metric space and x 0 an arbitrary point in (X,d). Bt(x0) = {x e X : d(x, xo) < r}, 0 < r Br [xq ]={x £ X ; d (x ,x o )S r} is the closed ball. ... Sr[x0] = {x e X ; d (x ,x Q) = r} is the sphere ... Definition 28. Let xq be an arbitrary point in (X, d), then a subset N of (X, d) is a local neighbourhood of x 0 if N = Br(x„ ) or Br[x0] for some r j- 0. Br(x0) are called open local neighbourhoods and Br[x0] closed local neighbourhoods. The local neighbourhood system of x 0 = {B r(x0), Br[x0]}, r > 0 = N(x0). It is now possible to redefine continuity in terms of local neighbour hood systems using the following theorem; Theorem A function f: (X, dx)-*(Y, dy) is continuous at x 0 in (X, dx) iff its inverse image of any local neighbourhood of f(x0) contains a local neighbourhood of x 0. Similarly we can redefine convergence in terms of local neighbourhood systems. 118 CURTAIN M, LOCAL NEIGHBOURHOOD N, ARBITRARY LOCAL Theorem A sequence {xn} in a metric space (X, d) converges to x 0 iff xneN, a local neighbourhood of x 0 , for all n ë m(N), a number depending on N. To -some extent our continuity and convergence may be defined independently of the metric, but we can even characterize continuity and convergence in terms of open sets. Topological spaces are defined in terms of open sets and this family is called "the topology" on the space. The topology or open sets of metric space are always definable in terms of the metric, but often several different metrics generate the same topology. In fact, equivalent metric spaces need not have the same local neighbourhood systems. Definition 29. L et (X .dj), (X, d2) be two metric spaces with the same underlying set. Then the metrics d^ d2 are equivalent if (a) f : (X, dj) -*■ (Y, d ), an arbitrary metric space, is continuous iff f : (X, d2 ) -> (Y, dy ) is continuous, and (b) a sequence {x n} converges to x 0 in (X.d^ iff {xn} converges to x0 in (X, d 2 ). In fact, either of (a) or (b) ensures the equivalence of dj and d 2 or even the condition: (c) I : (X, dj) -* (X, d2) and I' 1 : (X , d 2) -* (X, dj) are continuous (I is the . identity map). A more fundamental way of expressing equivalence of metric spaces is in terms of open sets. Definition 30. A set A in a metric space (X, d) is open if A contains a local neighbourhood of each one of its points. Note that 0 and X are always open sets. Definition 31. The class of all open sets in (X, d) is referred to as the topology (generated by the metric d) and is denoted by We now state the basic result on equivalence of metric spaces. IAEA-SMR-17/2 119 Theorem Let (X, di) and (X, d2) be two m etric spaces with the same underlying s e tX . Then d: and d2 are equivalent i î î = i Definition 32. A map f : (X, di), (Y, d2 ) is continuous if the inverse image of each open set in (Y, d2) is an open set in (X, d^. Definition 33. A sequence {xn} in a metric space (X, d) converges to xQ in X iff xn is in every open set containing x 0 for sufficiently large n. These are the usual definitions one uses for more general topological spaces. Previously we defined closed sets as those containing all their limit points, but this property may also be defined in terms of open sets. Definition 34. Let (X, d) be a metric space. A subset AC X is closed if its complement A' = X- A is an open set. Open sets and closed sets have the properties: 1 . ¡5 and X are closed and open. 2. If A¡ are closed, then Q A¡ is closed, but only a finite union is closed. 3. If Aj are open, then U A; is open, but only a finite intersection П Ai is open. 1 1=1 Finally, we define a separable metric space. Definition 35. A metric space (X, d) is separable if it contains a countable subset A which is dense in X . In the applications considered, most spaces will be separable Banach spaces, but in partial differential equation theory, you often need Fréchet spaces, which may be defined as complete metric spaces. Although they are not normed linear spaces, they do have similar properties and can be defined in terms of seminorms on locally converse topological spaces. Probably the best way of thinking of Fréchet spaces is as the inductive limit of normed linear spaces, because a more detailed explanation of these spaces is beyond the scope of this paper. 3. MEASURE AND INTEGRATION THEORY We recall that ¿[0, 1], the space of continuous functions, was not complete under the || • ||2 norm о 120 CURTAIN If we extend our class of functions to the Riemann integrable functions, then it is still not complete under this norm. Really, what we need is an integral which has the property that if fn is square integrable and fn f in mean square, then f is also square integrable. The integral which does have this property is the Lebesgue integral. With our background of metric spaces and normed linear spaces we could define Lebesgue square integrable functions to be elements of the completion of У [ 0 , 1] under the || • ||2 norm , and sim ilarly for general рё 1. However, we shall sketch the construction of the Lebesgue integral starting from basic measure theory. Measure spaces Definition 36. A measure space_i_s a couple (~X,38) consisting of a set X and a 6-algebra 38 of subsets of X. A subset of X is called measurable if it is in 38. (ст-algebra means closed under all countable set operations). Definition 37. A measure ц on a measurable space (X_,38) is a non-negative set function defined for all sets in 38 with the properties p(0) = 0 where E¿ are disjoint sets in 30. i — 1 (countably additive) The triple (X, 38, ц ) is then a measure space. The Lebesgue measure space Example 28. (IR, m) where m is the outer measure defined by mA = inf £ length (In) UIn3A where In is a countable collection of intervals covering the set A. The outer measure is not countably additive over all possible subsets of IR, so we define a subcollection the set of Lebesgue measurable s e ts , by E G if mA = m(A П E) + m(A П E') V A С IR Then you can show that ^ is a cr-algebra of sets and it also contains all subsets of ]R with measure zero, i.e. (IR, m) is a well-defined measure sp a c e . We remark that all intervals are in ^ ^ including (a, oo). Further, all Borel sets are in (this includes all open and closed s e ts ). {38, the set of Borel sets, is the smallest ff-algebra containing all the open sets of IR.) IAEA-SMR-17/2 121 Example 29. Another example is (10, 1], m), \yhere ^ i s the Lebesgue measurable sets in [0, 1]. Also (1R, âô, m). We say that p is a finite measure if /u(X) Example: ([0 ,1 ],Д m) with m([0, 1]) = 1; all probability measures. Definition 38. A measure space (_X,-á^,iu) is complete if 3S contains all the subsets of sets of measure 0. (The Lebesgue measure is complete.) All measure spaces can be completed. Definition 39. Measurable functions Let f : X^ { °°}, then f is measurable if {x : f(x) i—1 constants and Xcc-l is the characteristic function of E( 1 6 3S. у (x) = if x G Ej XeíW [ 0 i f x Ç E j All non-negative measurable functions f = lim gn , where gn is a mono tonie increasing sequence of simple functions. Example 30. f :A ^E u{°°} is Lebesgue measurable if A 6 ^ and for each a , {x : f(x) n J" gd(x= с{1л(ЩПВ) E i=l where E e 30. This integral is independent of the representation of g and has the usual properties of integrals: (a g +Ъф) dju = gdM + b j фdju E for a, b > 0 . 122 CURTAIN For non-negative measurable f on (X, 38, ц), define / f d*u = sup / gd/j where g ranges over all simple functions with Oâ gSf. This is a well-defined integral for non-negative measurable'functions with the properties 1. J (af + bg) àiu = a j fd/u + bJ gd/u for a, bs 0 . E E E 2. J' fdju S 0 with equality iff f = 0 'almost everywhere', i.e. except on a E set with measure zero. 3. If f g g a.e, then J' fd/к ê J gd;u . E E 4. If {fn} is a sequence of non-negative measurable functions I l '■*“ î J E n=l n=l E This last property is an important advantage over the Riemann integral. To extend the integral to arbitrary measurable functions we define: Definition 41. A non-negative measurable function is integrable on E if J' f dp An arbitrary measurable function is integrable on E if f+ and f* are both integrable and we define J' f d/д = J' f +dju -J' í~diÁ. E E E f+ and f are both non-negative functions defined by f- = i ( I f I - f) This integral has the properties 1 -4 and^in addition,the very important Lebesgue dominated convergence property. IAEA-SMR-П /2 123 5. Let {fn(x)} be a sequence of measurable functions on E such that fn(x)-*f(x) almost everywhere on E and | fn(x) | S g(x), an integrable function on E, then 1 / fd „ = lim J f "dM and 6 . IJ f dju I s J f I d ц E E We have now defined integration over an arbitrary measure space (X,á^,p) and two important examples are Example 31. Lebesgue integral on (X, 38, ц) or ((a,b),^m ) We note that if f is Riemann integrable on (a,b), then it is also Lebesgue integrable and the integrals agree. Example 32. Probability measures (X, 38,n), where ¡л (X) = 1. In this case the integral on (X, 38, n) is the expectation of a random variable. (These spaces will be discussed in more detail later on.) Example 33. Multiple integrals on (IRn m) or (Ijx ...x ln ,^ ^ n, m). If X = I1x...xIn measurable sets have the form EjX.-.хЕд, where E¡ is a Lebesgue measurable set on I¡ , an interval of E. Then f(x) d/u (x) = ■ J' f(xi, ...,x n) dxi... dxn E, x . . xEn En has all the usual properties of multiple integrals. Differentiation and Lebesgue integration Consider F(t) = F(a)+J'f(s)ds a. when the integral is the Lebesgue integral on ((а,Ь), m). We already know from Riemann integration theory that if f is continuous, dF/dt = f on [a,b] and, conversely, if F is continuously differentiable (C1) with dF/dt = f , then F satisfies (1). With the Lebesgue theory we can extend this fundamental theorem of calculus using the concept of absolute continuity, which is slightly weaker than continuous differentiability. 124 CURTAIN Definition 42. Let I be a compact (closed bounded) interval on E, then F is absolutely continuous on I if for every G> 0 there is a 6>0 such that whenever Ik = [ak, bk] are non-overlapping intervals in I with n n ^ |bk-ak|só, ^ i F(bk) - F(ak)| se k=l k=l F is absolutely continuous on an arbitrary interval if it is absolutely continuous on every compact subinterval. For example, all C' functions are absolutely continuous and all functions which satisfy a Lipschitz condition on I: I F(t) - F(s) I s К 11 - s I for s, t G I Then the extended fundamental theorem of calculus is F:I->IR is absolutely continuous iff F satisfies (1) for some integrable f on [a,b]. Then F' exists almost everywhere and F'=f almost everywhere. Two other extensions of the Riemann theory are: 1. Differentiation under the integral If g(t) is Lebesgue integrable on (c,°o), f(x,t) a measurable function of t for all x in (a,b) and 9f/9x(x,t) exists for all t in (c,«) with f, 9f/9x bounded on (a, b) x (c,oo) , then F(x) = / f(xt) g(t) dt is differentiable in (a,b) with (c,«) F '(x) = J I I (x,t) g(t) dt 2. Multiple integrals and change of order of integration We have already noted that multiple integrals fall into our general theory of integration on (X, 38, ц) and have the usual properties of integrals. The relation between these multiple integrals on И" and successive Lebesgue integrals on IR is stated in Fubini's theorem (a) If f(x,y) is a measurable non-negative function on И XE, then J' dx J' f(x,y) dy = JJ f(x,y) dxdv = J áyj'íix, y)dx E F EXF F E where E, F are Lebesgue measurable sets. (b) If f(x,y) is a measurable function on E X E and any of the integrals is absolutely convergent, then IAEA-SMR-17/2 125 fd/-( is absolutely convergent if J |f|d^ exists J. E E (Sim ilar results hold for integrals of functions of n variables.) The Lebesgue-Banach spaces Our discussion of the Lebesgue integral was motivated by the fact that ^[0, 1] was not closed under the Ц • ||2 norm. We now define spaces Lp(E) which overcome this problem. f : E-" E , where E is a finite Lebesgue measurable set £ ?P(E) ■ in E and J I f IP dt < °° E 1 § p<°o D e fin e i/p I f II p = ( I I f(t) Ipdt E Then by Minkowski's inequality ( / U+glpdt) Ps ( j U lpdt) P+ (J |g|pdt) p E E E we see that || • ||p is a sem inorm . It is not a norm , since || f ||p = || g ||p does not imply that f(t)= g(t) at all points in E, but only "alm ost everywhere" (i.e. they can differ on a set of points of measure zero). To define a normed linear space we need to consider equivalence classes as elements in5^P(E) : f = g iff f(x) = g(x) for all x, except in a set of measure z e r o . If we consider these equivalence classes as elements of a new space L p(E) then II-||p is a norm, where equality in L p(E) means equal almost everywhere. L p(E) is a Banach space (i.e. complete). We also define L„(E) which is a generalization of bounded functions. L (E) - -f * : E -*®., where f is a Lebesgue measurable function \ “ I on E, which is bounded almost everywhere. J If we again consider equivalence classes of elements, then L„(E) becomes a normed linear space under the norm Il f ||œ = ess sup I f(t) I tS E where the ess sup is the infinum of sup g(t), where g ranges over all elements in the equivalence class for f, i.e. ess sup f(t) = inf{K : m {t : f(t)>K = 0} 126 CURTAIN 1.5 -1 Then sup f(x) = 2, but ess sup f(x) = 1; [-1.Я [-1. l] L„(E) is again a Banach space. The Lebesgue spaces Lp(E) are building blocks in analysis and so we summarize some useful properties: 1 T -\ T —ч T —4 w h e re ¿ q is the space of infinitely differentiable functions with compact support, (f has compact support if it is zero outside compact sets.) 3. Riesz representation theorem The dual space of Lp(E) is L q(E); l/p+ 1/q = 1. A ll continuous linear functionals G on L p(E) have the form E where g e L q(E) and II G II = Il g H, . (This is essentially a consequence of the Holder inequality.) |fg|dxS f|Pdx / |g|4dxj ; l/p + l/q = l; lSp E E E (Note, however, that the dual of L«(E) is not L^tE).) 4. L 2(E) is a Hilbert space. IAEA-SMR- 17/2 127 Some extensions 1. We may sim ilarly define L p(E), where E is a finite Lebesgue measur able set in R n and f is a measurable real-valued function of n variables. Lp(E) is a Banach space under the norm lf llp = ( / |f(x i...x n)|p dxx...dxn) ; 1 S p < oo and f ||^ = ess£sup I f(xt , ...xn) 2. We may also consider L p(E; ]Rk), the space of vector-valued functions of n variables f : E С IR"-* IRk under the norm llfllp = (j~ I f U i ..... xn)lk dx x... dxn^ E where | • |k is the Euclidean norm onlRk. 3. A final example is the space of second-order stochastic processes on 10, T], which is a Hilbert space under the inner product 1 T = J J x(t,u) y(t,u) d/udt о n 4. HILBERT SPACES Since a normed linear space is characterized by its norm and its algebraic structure, we often find that supposedly different spaces are equivalent both algebraically and topologically. Definition 43. Two normed linear spaces X_and Y are topologically isomorphic if there exists a Te L(X,_Y) such that T ' 1 exists and is continuous. T is called a topological isomorphism and we write _X£Ÿ Definition 44. If further, || Tx ¡y = II x ||x , for all x £ X , we say that they are isometrically isomorphic. 128 CURTAIN A useful necessary and sufficient condition for the topological iso morphism between two spaces X and Y is 3m, M> 0, such that m II x II x S (I Tx ||y S M I x IIx , Vx G X Example 35. L e tX -ü ^ and Z = { z : [0,1] - IR, z(t) = a0 + axt; a g ^ e l R } under the norm Il z II = max {| a0 I , | a j} . Then Tx = X! + X2t for x = (x i, X2 ) is a linear map from X to Z and Il Tx II = max { I xx I , |x2|} = ||x|L and T is bounded. Also T - 1 : z->X is given by T-1z = {z(0), dz/dt|t = 0}. T ' 1 is linear and || T “1 z || = || z ||. So X= Z and even isometric ally isomorphic. This is an example of a very important result, namely: All real n-dimensional normed linear spaces are topologically iso morphic, i.e. all are equivalent to IRn. So we can find out all about real finite-dimensional normed linear spaces from Rn . Properties of ]Rn 1. e¡= {0, , 0, 1, 0, ...} form s a basis for ]Rn, n i-e- S = { x l ...... xn> = X Xi~ i- i — 1 2. All norms are equivalent. 3. ]Rn is complete (i.e. a В-space) and all its subspaces are closed. 4. All linear maps L:]Rn->Y are continuous for any normed linear space Y . 5. A corollary of 4 is that the algebraic and topological duals of ]Rn coincide. 6 . All linear transformations T: IRn -*■ IRm may be represented as mXn m atrices. This representation is not unique, but depends on the basis of IRn and lRm. In fact T is represented by a class of similar matrices. Example 36. Let X be the space of polynomials of degree s 3, i.e. x(t) =a1 +Q'2t + a-3 t 2 + » 4 t 3 Let Y be the space of polynomials of degree S 2; Let D be the derivative operator on X. The range of D is Y_ and D : X-* Y is linear. IAEA-SMR-17/2 129 If we take {1, t, t 2 , t 3} = {x 1 ,x 2 ,x 3 ,x 4} as a basis for X and { 1 , t, t 2} = { у х, y2, У3 } as a basis for Y. then )x2 = У1 ’ Dx3 - 2y 2 0 1 0 0 ' 0 Г " ' “ l" “ 2 02 = 0 0 2 0 if Dx = ft +ñ2t + /33 t2 «3 0 0 0 3 J 33_ «4 and relative to these bases D has the representation 1 — о О i 0 0 0 2 0 _ 0 0 0 3. If we change the basis of X to {1+t, t + t2, t 2 -t-t3, 1 + 12} then D has the new representation 1 1 0 0 0 2 2 0 0 0 3 3 When we consider infinite-dimensional spaces, however, the situation is not as simple. However, there is an interesting subclass of infinite dimensional spaces like ¡L which can be thought of as a generalization of ]Rn to n = » . Definition 45. An inner product on a linear space isf a bilinear function X X X - 1. (o x + |3y , z У = o ‘(x, z )> + fKy, z for scalars a , )3. 2 . 3. ( x , x ) î 0 and (x ,x )> = 0 iff x = 0. A linear space X with an inner product (.,./> is called an inner product space. П Example 37. ]Rn with <(x,y> = x.y. i = i where x = { X l, ... xn} , y = { yi , ...y n} b Example 38. L 2 [a,b] with <(x,y)>= J' x(t) y(t) dt 130 CURTAIN Other properties of the inner product 1. <(x,y^> = 0 V x £ X y = 0. 2 . 3. X can be made a normed linear space by defining || x || = */^x, x У 4. Parallelogram law l|x + y ||2 + | | x -y ||2 = 2 II x II2 + 2 ü у II2 where || • || is the norm induced by the inner product. 5. Schwarz inequality |^x,y)>| s l|x||.||y||. Definition 46. A Hilbert space is an inner product space which is complete as a normed linear space under the induced norm, i.e. a Hilbert space is a special case of a В-space. When is a В-space a Hilbert space? If its norm obeys the parallelogram law, then we can define an inner product Example 52. L 2 [a,b] is a Hilbert space with inner product b < x ,y > = y x(t) y(t)dt a and norm b I 19 \ 1/2 I x(t) I dtj as before. Let V = { y e L 2 [a, b]: y' e L 2 [a, b] and y(a) = 0 = y(b)}. Then V is a linear subspace of L 2 [a,b] and in fact is a dense subspace. On V we can define a new inner product z >v= < y . z > + < y ' , z ' > V is a Hilbert space under <( .Уу , but it is not closed under •(.,.)>. Example 53. Let Q be a closed subset in IE3 and X = ^ 2(Q), the space of complex-valued functions with continuous second partials in Г2 . „ „ / 3u Эи Эи N Define V и = ( -— , -— , -— ) and \ Э х: Э х 2 Эх3 / _ Эи 9v Эи 9v Эи Эу < U , V > = uv+ - — dxjdxgdxg Эх, Эх, Эх0 Эх0 Эх, Эх, where x = (xi, х 2 ,хз). IAEA-SMR-17/2 131 Then <., .> is an inner product on X and it induces the norm (/ Example 54. Let Q be an open subset of IRK and u е ^ п(Г2). As in E x.22 define the differential operator D“ u: a I = n and the inner product <(.,. on I D “u(x) D“ v (x ) dx , )>n induces the norm n,2 u ''".г = ( I X lD“u(x)|2d^1/2 The completion of ó n(í2) with respect to this norm is a Hilbert space, denoted Hn(Q), a Sobolev space, which is used in distribution theory in partial differential equations. Another important Sobolev space is Hg(f2), which is the completion of ¿ q(Q) under this (n, 2 ) norm. is the space of infinitely differentiable functions with compact support.) We note that the other (n,p) norms cannot be obtained from inner products (see Ex.22). An important finite-dimensional concept which generalizes Hilbert spaces is orthogonality: Definition 47. x is orthogonal (perpendicular) to y, x x y, iff ^x, у У = 0. If x i у, then the parallelogram law reduces to 2 2 which certainly looks familiar. Definition 48. If M is a subspace of a Hilbert space, H, then the orthogonal complement М± = { x e H: Mx is a closed linear subspace of H and H can be decomposed uniquely as a direct sum, 132 CURTAIN H = M e M1 also M Х1= M. I fy S H , then y = yM + yM-L, where yM6 M, y ^ G M ^ a n d || y ||2= || у м ±||2 + llyMx||2. M induces an orthogonal projection i on H. 7r : H - M where 7ry = ум . 7г is linear and bounded and ||7г||= 1 . Example 39. H = L2 [-a, a], M = { set of all even functions: x(t) = x(-t)} M is a linear subspace of H. Let P : H - H be у = Px , where y(t) +i(x(t) +x(-t)). Then P is a projection of H - M. Definition 49. An orthonormal set in a Hilbert space H is a non-empty subset {0n} of H > < 0 n,0 m> = 6nm i.e. {0 „} are mutually orthogonal unit vectors. Of course any mutually orthogonal set {un} may be normalized by Example 40. • The set 0k = {0,..., 0,1, 0, ...} is an orthonormal set. Similarly for £2 . Example 41. {sinîrnt} is an orthonormal set in L 2 t0 , 2 ]. Example 42. Any linearly independent set {x i, x 2, ...} in H can generate, an orthonormal set {0n} by the Gram-Schmidt orthogonalization process Let 0X = xx / II x x II SO Continue in the following way: й _ x n - < x n , 0i > 0i - < x n, 02>02 -<■■■>-< x , 0n-l > 0n-l ГП 11 " [j Orthonormal sets satisfy an important inequality: Bessel*s inequality. IAEA-SMR- 17/2 133 If 0n} is a finite orthonormal set in H and xeH then X 1< * . 0 1 > 1г « 1 4 * • i=l and x - i = 1 Proof 11 OS U - l < x ,^ 1>0i H2 i=l n n = n n = <х,х>-2^Г <х,0;><хТ07>+^Г ) i=l and i=i i=i = 0 This has a useful geometric interpretation: n £ < х . 0 ! > 0 , i = l is the projection of x onto the subspace spanned by {$x, 0 k} and n X <*.*,> 0i Bessel's inequality is also valid for n = oo. E arlier on.we said that a Hilbert space can be thought of as '11°°' in some sense. Well just as for IRn , where we can always express any element 134 CURTAIN as a linear combination of n basis elements, we can do a similar thing for Hilbert spaces, except of course we need infinitely many basis elements and we need enough of them to include all the elements, i.e. a complete set. Definition 50. An orthonormal set { j) n} in a Hilbert space H is complete or maximal if H = Sp{0 The following are equivalent conditions for {0¡} to be complete: 1. x 1 0i Vi =* x = 0 2. x = E This is called the Fourier expansion for x <(x,0j)> are called the Fourier coefficients 3. Parseval's equation holds: || x ||2 = L \ <(x, 0¡ )>|2 4. There is no vector y€H such that ,y/|| yl|} is an orthonormal set larger than { 0 ¡} . Example 43. Let H = L 2 [0, 2тг]. Then {0n(x) = e in* /-Jzi} is an orthonormal set in L z [0 , 2тг]. о are the usual complex Fourier coefficients of f. Bessel's inequality is 2tt One can show that this set is complete and equality holds and we can write where equality is of course in the L 2 -sense, not pointwise in general. Since H = Sp{0¡} , we can see why the Riesz-Fischer theorem holds: If {cn} is a com plex sequence such that ^ lc n|2<00> then there is an f e L 2[0,27r], whose Fourier coefficients are the cn's. Example 44. H = L 2 [0,2tt] over the real numbers. This is sim ilar to the previous example, except the complete orthonormal basis is {1/-/2Й-, 1 / 7Г sinnt, 1 / n/~7Гcosnt} and l i Example 45. H= L 2[-l, 1]. The Legendre polynomials are an orthogonal set, where We have l -l Also {Pn (x)} is complete in L2 [-1,1] and so for arbitrary f e L 2 t -l, 1] it has the expansion n=0 where each polynomial Pn (x) is of degree n and the partial sums converge to f in the L2 -norm, i.e. к n = 0 approximates f(x) in the mean square sense. Example 46. H = L 2 (0,oo). The Laguerre polynomials Ln(x) are often defined as the polynomial solutions of xy" + (1 - x)y' + ny = 0. Then {e "x^2 L n(x)} forms a complete orthonormal set for H. These last two examples are two of many such ways of expressing n = 0 as an expansion in terms of orthonormal functions. Often these arise as eigenfunction expansions of solutions of partial differential equations after 136 CURTAIN using separation of variables. These also give us good approximations for f as a finite sum of terms, which are invaluable for numerical solutions. The advantage of using expansions in terms of orthonormal к functions, as opposed to, say, finite polynomials, f(x)= ^ a nxI1 » is that, if n=0 you wish to increase your accuracy for f in the orthonormal expansion, you simply calculate one extra coefficient and add it on and repeat until the accuracy is sufficient. However, with approximation by polynomials, you need to recalculate all the coefficients for each new approximation. 5. LINEAR FUNCTIONALS, WEAK CONVERGENCE, WEAK COMPACTNESS Definition 51. A linear functional on a normed linear space X is a linear map f : X-> К (or (С if X is a complex vector space). Definition 52. The algebraic dual of X, denoted Xa, is the linear vector • space of all linear functionals on X. If we consider topological properties as well, we can look at the space of all continuous linear functionals (or equivalently all bounded linear functionals). Definition 53. The topological dual of X or the conjugate space X* is the normed linear space of all bounded linear functionals on X with the norm II f|| = sup{ I f(x) I : Il x H = 1} x S _X We rem ar» that X* is always a В-space, even when X is not. We illustrate this useful concept with the following examples: Example 47. Consider X = and define fa(x) = axXj+a 2 x2+ ...+a x for a fixed a G Jîjj , a / 0 . fa:X ^R and is linear. It is also bounded since fa(x)|= |a1x 1+a2 x 2+ ... + anx n 2 applying the useful Holder inequality i= 1 i= 1 i= 1 where l/p + l/ q= 1, valid for n finite or infinite and 1 ^ p < °°. We also have that IAEA-SMR- 17/2 137 sup j I f a (x) I : ]| X II2 = l j I a I! 2 Letting x = и -“ir- , f ( и an ^ Il a II? and hence we have the equality Il a II2 a V II a||2/ 2 Il fa II = Il a II g • So every a E X defines an element fa in the dual space; sim ilarly for ¿ 2 ■ Example 48. X = ^[a,b], b T hen I(x) =J x(t) dt is a linear functional and | I(x) | ^ sup| x(t) | • (b - a) a with equality when x^ constant. So I is a continuous linear functional with the norm (b - a). b Example 49. X = ¿£[a,b] and I y = J y ( t) x(t) dt; Iy is a linear functional on X a for integrable y. b b I I y(x) I S m a x I x(t) I У I y (t) I dt = I x II J |y(t)|dt 3 b a So Iy is continuous and in fact ||ly||= J |y(t)|dt. a Example 50. X= ^[a,b] and ót x = x(t0) о ôr is a linear functional on X with Lo |óto XI S I x(t0 ) I S II X II and we have equality when x is a constant. So our fam iliar "delta function" b which is often represented as J x(t) 6 (t - t0)dt is not a function, but a a continuous linear functional on ^[a, b] with norm 1 . It is all very well to define the dual of a space and to give some simple examples, but we would like to know more in the general case. The following famous Hahn-Banach theorem ensures us of the existence of lots of con tinuous linear functionals. Hahn-Banach theorem I Every continuous linear functional f : M-*]R defined on a linear sub space M of X, a normed linear space, can be extended to a continuous linear functional F on all of X with preservation of norm. If we take the particular subspace to be {axo} for x 0 / 0 and a any scalar, then a linear functional is f 0(y) =о|| x 0 ||, w h e re y = ax0 and f0 is 138 CURTAIN bounded with norm 1. The Hahn-Banach theorem says that there is an F 0 in X* with norm 1. The Hahn-Banach theorem also has the following geometric inter pretation, which is useful in proving the existence of optimal control theory problems. Hahn-Banach theorem II Let X be a normed linear space, M a manifold of X and A a non-empty convex, open subset of X not intersecting M. Then there exists a closed hyperplane in X containing M and not intersecting A. For many spaces there is a nice relationship between X and its conjugate space X*. n Example 51. Consider and define feip* by f(x) = ^ f.x.;; lSp<°o. f is clearly linear, and i = 1 where l/p+ l/q= 1 using Holder's inequality n i= l If we let q if f j / 0 » 0 if f, = 0 n i= 1 and П f(x)|= Y l fi l 4 i = 1 since l/p + 1 /q = 0 . IAEA-SMR-17/2 139 So we actually have H - Œ m f In fact all continuous linear functionals have this form and the dual of is , where l/p+l/q = 1 . Similarly £ * = £^. Example 52. Consider i n with ||x|| = max |x,| " lsisn n Let f(x) = ^ fjX. . Then f is a linear functional and i -1 I K ill - £ IfJIxJ * IU II.Ï l ' i l - 1* I . l l 'l i i=l F orX j= sgn f¡, we get equality and so || f ||= ||f Ц х. We can show that all continuous linear functionals must have this form, i.e. . Similarly £n* = £n andí^í^. However, 4*/^. Example 53. There is an analogous result for the Lebesgue spaces: LP [a, b]* = L4 [a, b], where 1/p + 1/q = 1; l i.e. linear functionals Fg on L p[a, b] have the form: b Fg (f) =J f(t) g(t) dt, where g e L q[a,b]. a It is easily verified that FgeLP[a,b]* since it is linear and Holder's inequality for integrals is I J fit) g(t) dt|s ÇJ I f(t) IP dt^) (^J |g(t)|4dt) a a a = II f I p II g llq » where l/p+ 1/q = 1 i.e. Fg is bounded and |j F g || S ||g||q. In fact II F g || = || g||q . These duality results do not hold for p= oo. We see that since l /p + l /q = 1, if we take a second dual of L p[a,b], i.e. (L p[a, b]*)* = L 4 [a, b]* = L p[a, b]. In general, for a normed linear space, one has X** DX and if X = X** we call X reflexive. 140 CURTAIN Definition 54. X is reflective if its second conjugate is itself, i.e. X**sX. This is a rather special property of spaces like Lp[a,b], , SLp and is not shared by many common spaces, for example ^[a, b] is not reflexive. However, all Hilbert spaces are reflexive and in fact we can say more — they are self-dual spaces. Suppose yGH and define fy:H- fy (x) = < x ,y > VxGH fy is linear and |fy(x)| â So f y is bounded and || fy In fact we can show that || fy || = ||y|| , and that every linear functional fEH* corresponds to some y€H in this way, i.e. there is a 1 - 1 correspondence between y EH and f y EH* and || f y || = || y || • H* can be considered as a Hilbert space too, by defining ( f x, f =<(y,x)'. So there is an isometric iso morphism between H and H* — usually we identify them as the same space. It follows that Hilbert spaces are necessarily reflexive. As convergence is a kep concept in analysis, we shall now have a careful look at what one means by x n- x . In fact, depending on the context we can mean many very different things. Example„ , 54.-, -Letx„(t) * u\ = Ji i / ’/t on [1/n, 1] n [_ 0 elsewhere i Then x n(.)G L p[0, 1], for all finite p, since J' | xn (t) |p dt < oo. Clearly xn(t )- l/\ft pointwise. Now lA/t G Li [0, 1], but l/\ft ^ L 2 [0 ,l ] , So x n converges in the space L x[0 , 1], but not in L 2 [0 , 1 ]. Example 55. If f(t) is integrable on (0,1), we can form its Fourier series a. + ) (a cos27rnt + b sin27mt) о n n i where i i l a0 = / f(t)dt, an = 2 / f(t) cos 2mt dt, bn = 2 / f(t) sin 2rr nt dt о Now,in general, the Fourier series of f does not converge to f pointwise, even if f is continuous. However, the Fourier series always converges in L 2 [0,1], i.e. 1 к a 0 + a nco s 27rnt + b n s in 27rnt - f(t) dt-* 0 as к - co о n^i Vf G L 2 [0, 1]. This type of convergence we have been discussing is often called: IAEA-SMR-17/2 141 Definition 55. Convergence in norm (or strong convergence) xn-* xin X means ||x-xn||^0 as n-»oo. Sometimes a sequence or series does not converge in this sense, but it does tend to something in a weaker sense, and we find the following concept useful. Definition 56. x n-> x weakly in X if f(x n)^ f(x) as n^oo for all f€ X * . Convergence in norm implies weak convergence, because if xn~*x, f(xn)-^ f(x) for all continuous functionals, i.e. for all f e X*. Example 57. i p . Consider a sequence {x k}, where xk = {x k ,.. .,x k}. What does weak convergence mean? A functional on íp isf¡(x)=x¡, corresponding to f¡ = (0, ..., 1, 0, ...). So f¡(xk) = xk -^Xj as k^oo i.e. each component of xk tends to each component of x. But iixk-xiiP=(X Iх?"xiip) p i=l -* 0 as k^oo, since n is finite, i.e. in this case, x k_>x in norm also. This is a particular property of finite dimensional spaces: weak convergence and strong convergence are equivalent. However, for infinite dimensional spaces, this is certainly not the case. Example 58. Consider and the sequence yk = (0, .,1 ,0 ,..) with a 1 in the к-th position. Then for f e í* =üq , f(yk) = fk 0 as k-»oo , since i k=l So yk converges weakly to the zero element. But II yk ~ 0 IIp = 1 -i—> 0 as к-*». Example 59. ^ [a,b] with the sup norm. Here convergence in norm is exactly uniform convergence: ||xn - x ||-*0 as n-»°o iff max | xn(t) - x(t) | — 0 as n^oo. a < t < b Weak convergence is equivalent to pointwise convergence and uniform boundedness in this space, i.e. 142 CURTAIN x n(t) -> x(t) for each t |xn(t) I § к uniformly in n and t. Weak convergence in a Hilbert space becomes a particularly simple idea: We recall that xn -*x weakly in H iff f(xn) -* f(x) Vf £H* i.e. iff i.e. iff <(xn -х,уУ -> 0 as n-oo VyGH Example 60. In L 2 [0,1], xn(t)->x(t) weakly means that l J' (xn(t) - x(t)) y(t) dt-* 0 as n -0 0 VyELj [0,1]. о There is yet another type of convergence in X-weak* convergence. We need to identify X as the dual of some space, say X = Y*. Definition 57. A sequence {y *} in Y* is weak* convergent to y* if У? (y)_>y*(y) vy e Y. We recall that Y c Y** and so elements of Y define linear-functionals on Y*, and so weak* convergence is like weak convergence, except you only use a subset of all possible linear functionals. Of course, if Y is reflexive, y = Y**, then weak convergence on Y* is just weak convergence on Y*. (This is true for Hilbert spaces). Just as it is often useful to use weaker types of convergence in a space, it is useful to have weak concepts of compactness. For example in optimization problems, one often seeks to maximize a linear functional over some set. A fundamental question is whether or not the given functional attains its maximum on the set and the main result is that a continuous functional on a compact set К of a normed linear space X achieves its maximum on K. For finite-dimensional spaces К is compact iff it is closed and bounded, but unfortunately this is not true for infinite dimensional spaces. In fact the unit ball { x 6 X : ||x||sl} is compact iff X is finite dimensional. However, one can prove that the unit ball is "weak* compact". We recall that strong convergence or convergence on norm refers to the strong topology or norm topology of X, i.e. X considered as a metric space under the metric induced by the norm. Similarly, weak convergence induces the weak topology on X and weak* convergence the weak* topology and one has two new concepts of compactness. Definition 58. A set A in X is weakly compact if for all sequences {xn} С A there is a weakly convergent subsequence with limit point in A. Definition 59. A set A in X = Y* is weak* compact if for all sequences {xn}CA there is a weak* convergent subsequence with limit point in A. Weak compactness and weak* compactness are equivalent for reflexive spaces. IAEA-SMR-17/2 143 6. LINEAR OPERATORS We are often concerned with transformations between spaces. A very large class is the class of those transformations between linear spaces which preserve the algebraic structure. Definition 60. A linear transformation T of a linear space X to a linear space Y is >T(ox+)3y) = oTx+^Ty for all x, yeX and for all scalars a, /3. Example 61. Consider the spring-mass system — r m r v — n m n r v If we assume that the friction between the mass and the surface is -b(dx/dt), the applied force is f(t) and the combined restoring force of the springs is - kx, then if initially x = 0 , dx/dt = 0 , we have the usual equation of motion . d2x dx , . 1Л f = m -т-го + b — +kx (l) a t ¿ dt with solution t x(t) = J h(t - s) f(s) ds (ii) о where Ы . 1 i X] Г X-2 Г \ r) = —7T-----r (e - e ) ПДА.! - X2) and Xb X2 are the roots of mX2 +ЬХ+ к = 0. Now if f(.) e ^[0,oo], i.e. if it is a real continuous function, then so is x and so (ii) may be written x = Lf, where L is a linear transformation from ¿í[0,a>] to itself. Strictly speaking, L is an operator since X = Y. Similarly, (i) may be written f = Tx, where T again is a linear operator on ^[0, Definition 61. A map FiX ^Y is invertible if there is a map G: Y~>X such that GF and FG are the identity maps. G is called the inverse of F . We know that the necessary and sufficient condition for F to be invertible is that F is 1 to 1 and FX = Y. For linear maps, this reduces to the condition Fx = 0, only for x= 0. From our example 61 we see that the solution of differential equations is one of finding inverses of certain transformations. 144 CURTAIN So far we have only considered algebraic properties of transformations, i.e. the preservation of the linear vector space structure. If X and Y are normed linear spaces, we may ask if a transformation preserves topological properties, for example Definition 62. A transformation T:X-> Y is continuous at x0gX if for every £ > 0 there exists a 6 > 0 such that II x - x 0 II < 6 implies that || Tx - T x 0 1| < e (Note that this definition applies whether T is linear or not.) An alternative form of the above definition is Definition 62a. TiX-’ Y, a transformation between two normed spaces, is continuous if {xn} being a Cauchy sequence in X implies that {Txn} is Cauchy in Y. Definition 63. T : X -* Y is said to be bounded if || Tx||y § к || x||x for some constant k> 0 and for all x£X . If T is bounded and linear then sup {|| Tx||y : 11 x ||x = 1} exists and is finite. We define this to be the norm of T, || T|| . It satisfies all the properties of a norm, and we note the useful property: H Tx II y s II T II II x ||x for all x £ X . For linear transformations you can show that the properties of continuity and boundedness are equivalent, and so we define Definition 64. Let X, Y be normed linear spaces and £f{X, Y) = {T:X~* Y where T is bounded and linear}. Under the norm defined above, £f{X, Y) is a normed linear space. For the special case, where X = Y, we write 5f(X) for the space of bounded linear operators on X. £f(X) is not only closed under addition and scalar multiplication, but it is closed under operator multiplication or com position, i.e. i*f(X) is an algebra of operators. For if Т,€^(Х), i.e. T¡ :X -» Y and is linear, it is readily seen that TjT 2 :X-*Y and is also linear. The boundedness follows since II T XT2 x(I S | | tJ ||t2 x|| s II t J II T2 II 11 x II This also yields the important result || T1T2 || 2 II Ti|| || Т2 || . Of course not all linear operators are bounded, for example in Ex.61. L is bounded, whereas T is unbounded. Example 62. Let X = L x [a, b], Y =-¿[a, b] and T : X ” Y be given by IAEA-SMR-П /2 145 where k(.,.) : [a,b]x [a,b]->IR is continuous in s ^nd t. Then T is an integral operator, clearly linear, and Tx(s) I S max | k(s, t) x 0< s,t< b II T x (.)L = sup I Tx(s) I S max I k(s, t) 'x a So TG№,Y) and T s max k(s, t) a Example 63. X = L 2 [0,1] and T :X -X is given by Tf = df/dt. T is linear, but cannot be bounded, as it is not even defined on all of X, but only on the subspace S>= {f€ X > f'(.) E X }, called the domain of T. T is actually an example of a closed operator, which we shall define in the next section. If, however, we take Y = /'[0 , 1], the space of all continuous functions with continuous first derivatives, then T is defined on Y and T : Y —X is bounded. This emphasizes the dependence of continuity and boundedness on the particular norm you are considering. An interesting property of continuous linear transformation is used implicitly in the principle of superposition, a handy tool in differential equations. This depends on the mathematical theorem: L :X —Y is a continuous linear transformation iff i-0 i=0 for every convergent series i=0 Again we emphasize that the continuity of L depends on the norms you choose for X and Y. -7T 146 CURTAIN j Œ > innt> 7 I ï ï sirsinns ds O n= 1 во t sinns ds ■I/èn n=l 0 oo -4- (cos nt - 1) ■I nz n=l Here we shall primarily be concerned with operators on Hilbert spaces as this case is similar to the theory of matrices. Definition 65. Let K ££f(H ), then the adjoint K* is defined by That К* always exists is easily seen bv considering < (K x,y= f(x); f is a linear functional on H and | f(x) | § ||k x | ||y|| s INI llxll ||y || s const ||x|| for fixed у So f GH* and so there is a y*G H , such that f(x) = <(x ,y * X i.e. К induces a map y-y* VyGH and we call this y* = K*y. The adjoint has the following easily verifiable properties: 1. I* = I; 0* = 0 2. (S + K)* = S* + K* 3. (a T )»= J T * 4. (ST)*=T*S* 5. Il T* II = Il T II We shall just prove the last property: Il T*y II2 = < T*y, T*y> = < TT*y,y> S ||тт*у|| llyll s II Т|1 II T*y II U у II II T*y II § II t || II у II i.e. I T* Il â (I Т II and sim ilarly ||t || â ||т*| IAEA-SMR-П /2 147 Example 65. Н=4§- Then Aeáf(H) is representable by a matrix, and A* is its transpose for the real scalars and the conjugate transpose for the complex case. b Example 6 6 . H = L 2 [a, b] and Tx(.) = J'k(.,s)x(s)ds a b b < T x ,y > = J' к (t, s) x(s) dsy(t) dt a a b b = J x(s) J k(t, s)y(t) dtds a b b - J x(s) J к (t, s) y(t) dt ds and so u T*y(.) = j f k (t ,.) y(t> dt space of functions u: [0 ,T]->]Rm with inner product Example 69. T Let H <(u,v>= J u'(s) v(s) ds о i.e. H = L, ([0, T]; IRm) (u(s) is a column vector and the prime denotes the row vector.) If B(t) is a positive symmetric m Xm matrix, then we define the operator В by (Bu)(t) = B(t)u(t) v teT V erify that В* = В . Note also that T < B u ,u y = J (B(t)u(t))'u(t)dt о T = J u(t) 'B(t) u(t) dt о g 0 since B(t) is a positive matrix. This leads us to two further definitions: Definition 6 6 . A €£f(H) is self-adjoint if A* = A. (A self-adjoint operator has the property that Definition 67. A positive operator Ae£¿\R) is a self-adjoint operator A such that ( A x ,x ) ï O VxG H. It is called strictly positive if < Ax, x)> = 0 only if x =0. Of course the self-adjoint operators are generalizations of real symmetric matrices or Hermitian matrices, and positive operators correspond to the positive definite matrices with the property x'A xs 0 . Self-adjoint and positive operators occur frequently in applications and they have several special properties which we shall list here as they are important in applications. 1. Il AII = sup I <(Ax,x^>| ¡1*11 = 1 or equivalently the smallest M such that |<^Ax,x)>|s M |j x Ц2 2. A finer inequality is m ||x ||2 â <(Ax,x^§ M||x||2 If A is positive then m ïO , and if A is strictly positive, then m>0. In the strictly positive case, the following also holds: ^ | | х| | 2 ё< А л х,х>;§ ^ Il x II2 V x£H i.e. A "1 exists and is strictly positive. 3. Every strictly positive operator A has a unique strictly positive square root A1/i2. 4. An orthogonal projection is self-adjoint, since Example 70. Consider the same space as E x.69 (often denoted by Lz([0, T]; ]Rm) and let W(t, r) be an rXmmatrix-valuedfunction on [0, T]X [0, Т]. Then the operator cl¥r \ t (O^uHt) =J W(t, T ) u ( r ) d T о maps H -Y = L 2([0,T ]; H r ). T t ^ ^ u .v ^ y = J (^J' W(t, t) u(t) dT^ v(t) dt о о IAEA-SMR-П /2 149 т t = J J u(t) 'W '(t, t) v(t) dT dt o o T T = ' W(t, t) ' v(t) dtJdT o t changing the order of integration T T = J u(t) ' (^J W(t, t) ' v(t) dt^ dT o t = <(u)c)5r*v)> T where We should note that although for simplicity we have limited the definitions of adjoint, self-adjointness and positivity to bounded linear operators, all these definitions can be extended to closed linear operators on a Banach space. 7. SPECTRAL THEORY Most of the problems of linear algebraic equations, ordinary differen tial equations, integral equations and partial differential equations can be formulated as linear operator problems. Example 67. The Fredholm integral equation is b A-f(t) -J k(t, s) f(s) ds = g(t) to be solved for the unknown function f(t), where k(., .) : [a, b] X [a, b] -* IR. b b J' J' |k(t,s)|2 dtds<°o. If we let X = L 2 [a, b] and define К : X - X by a a b P К f(t) = J к (t, s) f(s) ds then this is really an operator problem (XI - K)f = j a i l where I is the identity operator and f = (XI - КГ g, if (^1 - K) exists. Example 6 8 . Э2и Эи ...... Г u(0, t) = 0 = u (l, t) Эх2 9t \ u(x, 0) = u0 (x) 150 CURTAIN This is the familiar heat equation. If we let X = L 2[0 ,1], then we can define A : X - X b y A f = 0 for f e¿2>(A) = j g:g, Ц . 0 6 L 2 [0,1] A is a closed linear operator on X with domain S>(A) and the heat equation may be written Ж = Аu(t> In Eq.67, К is a linear bounded operator, but in Ex.6 6 , A is linear but unbounded. In fact most differential operators are unbounded, although fortunately they do form a class with nice properties; the class of closed operators. Definition 6 8 . A linear operator T:X->X is closed if for all sequences {x n} in the domain of T, ¿&(T) with x„-*x and Txn — у then x £ ^ (T ) and у = Tx. Another way of saying this is to consider the product space ¿^(T)X R(T) with typical element (x,Tx). Then T is closed if ^(T)XR(T) is closed in XXX. Ш Т ) is the range of T.) A familiar example of a closed operator which occurs in physical problems is the Laplacian. Example 69. Let ft be the unit disc in 1R2 , Q = {(x,y) : x2 + y2 S 1} and consider the Laplace operator . 32u 92u A u = 3ÏÏ2 + a72 on Ьг(а )’ the space of Lebesgue integrable functions of two variables with finite norm l|u| | = ( / f u (x,y)2dxdy У Let .^(Д ) = { u e L 2 (£2 ) : u is and Д u 6 L 2 (Г2 ) ; u = 0 on ЭГ2} (ЭП is the boundary of the disc) Then Д is a closed operator on L 2 (f2) with the domain ^ (Д ). The classical partial differential equation: Au = f; u = 0 on 3Í2 has the solution u(x, y) - J G(x,y; Ç, n) f(Ç,rj) dÇ dn where the Green's function is given by IAEA-SMR-17/2 151 where the CTj are the distances shown in the following figure: '_L j l V £2*t)2' ihv2J i.e. Д has an inverse and u=A'1f. By inspection, A" 1 is an integral operator and is linear and bounded on L 2(f2). In a similar manner, other differential equations can be rephrased as problems of finding the inverse of a closed operator. Note that for a true solution to the problem , we need the inverse to be bounded. For example if Au = f is to have a solution for all feL 2 (f2), we need Д' 1 linear and bounded, i.e. defined on all L 2 (f2). Much of functional analysis is concerned with finding inverses of operators, but for now we shall just state the important result concerning linear bounded operators: If X,Y are В-spaces and L e , Y) and L‘ exists in an algebraic sense, then it is bounded and linear. Now in finite dimensions the solution of linear equations depended on the eigenvalues of the m atrices and although the alternative theorem is probably very familiar to you, we shall formulate the results in a Hilbert space context, under the heading "Spectral theory in finite dimensions" Let X = lRn . Then T e 5f(]Rn) has a matrix representation, which is not unique as it depends on the basis used for ]Rn. However, each T is uniquely represented by an equivalence class of similar matrices (A and В are similar matrices if there exists a non-singular C, such that A= С_1ВС). Similar matrices have the same characteristic equations, and hence the same eigenvalues. So for each Te5f(IEn), we can define 1. The spectrum of T, cr(T) = {set of eigenvalues of T:X¡}. 2. The eigenvectors of T corresponding to Xt are x¡: (T-X¡I) x í =0. 3. The eigenspace of T corresponding to Xj is M¡ = Sp{x : Tx = Xjl}. 4. The projection operator Pj corresponding to M¡. Using these definitions, we can decompose T as the sum, m i=l where Xi are distinct eigenvalues and Pj is their projection operator. It can be shown that the P* are pairwise orthogonal, i.e. PiPj = 0 , i^ j. 152 CURTAIN (You may rem ember this as the fact that eigenvectors of distinct eigen values are perpendicular.) So if we take powers of T, we have rn T k = ^ Xk Pi i= l an easy calculation. Similarly, if f(t) is a polynomial in t, then m f(T) i = l The study of eigenvalues in matrices has particular significance for the algebraic linear equation y=(Xl-T)x on Bn The 'alternative' theorem states that it has a unique solution x = (XI - T) ' 1 y (i.e. (XI- T) " 1 exists as a matrix), i.e. provided X is not an eigenvalue or provided X$ a(T). This property that the existence of a solution of y = (A.I- T)x depends on the spectrum of T generalizes to the infinite dimensional case. Definition 69. Let T be a linear operator on a Banach space X, then 1. The resolvent of T, p(T) = { X6 С : (XI - T) ' 1 G (X)} 2. The spectrum of T, a( T) = (C - p ( T) That is, for X 6 p(T), the linear equation y = (Xl - T)x has a unique solution x = (XI - T ) ' 1 y for all y € X. Note that (XI - T)x = y may now be an integral equation or a differential equation (see Exs 65 and 6 6 ). An important subset of o(T) is the point spectrum: Pct (T) = {X : (XI - T) is not one to one} In the finite-dimensional case, cr(T) = P ct(T) and is a finite set, but in infinite dimensions, things are slightly more complicated. Certain operators, however, have similar properties to finite-dimensional operators (i.e. matrices). These are compact operators, a subclass of linear bounded operators, which have the special property that their spectrum is a point spectrum and has countably many elements, i.e. cr(T) = P ct(T) = {X¡; i = 1, 2 , . . oo} This means then that (XI - T)x = y has a unique solution except for countably many Xj, the Xj6 Pa(T). IAEA-SMR-17/2 153 Definition 70. A compact operator T is an operator G£f(X) which maps a bounded set A of X into a precompact set TA (i.e. TA has compact closure TA). An equivalent definition is alinear operator T which, for any bounded sequence {xn}, {Txn}, has a convergent subsequence. All finite-dimensional operators are compact, as are linear operators with finite-dimensional range, since in a finite-dimensional space every closed bounded set is compact. Another example of compact operators are integral operators; К of Ex. 65 is compact. Unfortunately, all linear operators are not compact and ct(T) may be very complicated. Example 70. X = A[a., b] and T : x(t) -*■ /n(t) x(t) where ц (t) is continuous. Consider (T - XI) x(t) = y(t) (й(t) - X) x(t) = y(t) X(t) = -jüjt)—X ’ Provided ju (t)-X / 0 and c(T) = {X>/u (t) - X = 0, for as ts b} a b This is an example of a continuous spectrum. Example 71. Х = ^ 1л Tx = x i X2 { 1 ’ 2 n } Consider (XI - T)x = у Now So ( XI - T) is not 1 - 1 if X = 1 / n; n = 1, 2 We can show that 0 ePff(T) also. 154 CURTAIN Example 72. Consider the bounded sequence cos x, cos 2x, cos 3x, ... t and the integral map T : Tx(t) = J x (s) ds, producing the sequence sinx , 1 0 ? sin 2x, ..., — sinnx,... n Then this derived sequence is convergent, because T is a compact operator on L,2(0 ,l ) . Since a knowledge of the spectrum tells us when we can hope for a solution of (XI- T)x = y, it is an important study in analysis. One very useful result for self-adjoint compact operators is that: ст(Т) С [ - |[ T fl , | | t| | ], i.e. for |x| g II T 11 , (XI - Т)_1Е£^(Х) or equivalently, (XI - T)x = у has the unique solution x = (XI - Tlÿ1 . The proof is easy and an instructive exercise. First we consider the X = 1 case: (I-T)x = y, ||t||<1 and let B k ■ I-' i=0 {Bk} is Cauchy since n 1 в к - в пь ^ Ц т1‘ - 0 as n, k - 0 0 . к because || T || < 1. âf(X) is complete and so B = ^ T Ieâf(X). But i=o 00 oo b(i - t) = Y t 1 - Y t í = I i=0 i= 1 : . в = (1 - т ) л е ^ ( х ) For X f 1 we just consider (XI - X)'1= X(I - (1/ X)T)_1 and || (1/ X )t|| < 1 is equi valent to II T [1 < X . For certain classes of linear operators you can obtain a spectral decomposition for T. For example, if T is a compact, self-adjoint operator on a Hilbert space H, then oo Tx = Y x k where Xkecr(T) and e^ are the corresponding eigenvectors. This looks very similar to the matrix case, however, for more general T, the best we can hope for is an integral decomposition of the type Tx = / XdE(X)x. Application to Fredholm integral equations Consider b Xf(t) = [ k(t, s)f(s) ds = g(t) IAEA-SMR-17/2 155 the so-called Fredholm integral equation. Let К, X be as in Ex.67 and write it as (XI- K)f= g It can be shown that if k(t, s) is integrable on [a,b]X[a,b] and b b a a then K eáf(X ) and is compact. Now b b b 2 к || = sup к (t, s) f(s) ds|2dt ) > f |f(s) ds = 1 f a a a } We know that if ||k ||< | X |, then the equation has the unique solution f ^ (X I-K H g i=o One can verify that b a where b a which gives an iterative method for finding the solution. As the series solution for f converges, a finite approximation will suffice. If || К ||> |x| , we know that, except for countably many X¡, the equation again has a unique solution. These are procedures for getting approximate solutions based on approximations on the kernel function k(t, s) but we shall only work out the case for k(t, s) = .k'.(s,t), i.e. К is self-adjoint. In this case, К has the decomposition 00 where Xk are its eigenvalues and xj, the corresponding eigenvectors. 156 CURTAIN We can now rewrite Xf - Kf = g, as oo Xf - X Xj X = Г for /. xf = Kf+g 00 = X X; X - X. x i + ¡ which solves the equation for f if X<£ct(K). b < g - x l y = f g(t)x.(t) dt Unfortunately,in practice many equations are represented by operators which are not compact or even bounded. These are the ordinary differential equations and partial differential equations, which in linear cases can be represented by closed operators. Although closed operators are not as well behaved as compact or bounded operators, they also have nice properties and a knowledge of their spectrum also gives useful information. For example, if A is a closed linear operator on a Hilbert space H and for some X, (XI - A)_1e£f(H) and is compact and self-adjoint, then A has a special eigenfunction decomposition: Ax for every x in the domain of A and {xj, x2 ,...} eigenvectors of A, and мп the eigenvalues of A. Example 73. The heat equation of Ex. 68 Assume that u(x, t) = U(x) V(t) IAEA-SMR-17/2 157 Since the right-hand side depends only on x and the left-hand side only ont they must both equal a constant, X, say A U = X U; ^ = XV dt We can show that A has the property that (X0I - A ) ' 1 is compact and self-adjoint for some Х0, i.e. A =y^Mn<--xn> xn, and Ахп = xn So a solution is u(x, t) = еМп xn(x) and the general solution is u(x, t) = У) d j^ n 1 x n(x) where the dn are determined by U0(x) = £ dnxn .'. u(x, t) eMn x n(x) n=l Unfortunately, we cannot linger on this very important and vast area of application of functional analysis to differential equations, but merely refer the reader to Naylor and Sell or Dunford and Schwartz (see Bibliography). 8 . CALCULUS IN B-SPACES In this section,we shall be mainly concerned with developing a differential calculus for operators. The fundamental idea involved is the local approximation of operators by linear operators. Unfortunately, in order to understand this concept it will be necessary to "unlearn" the interpretation of the derivative of a real-valued function of a real variable. First of all, recall that for f : IR - IR. f(x 0 +t) - f(x0 ) f' (x q ) = lim t-> о and that a good approximation to f(x) for x near Xq is L(x) = f(x 0) + f'(x 0) (x - x 0) If we consider f:IR 2 —IR, then the above expression for f 1 no longer makes sense because we cannot add the vector x 0 to the scalar t. However, since f(x) = f(x1( x2), we can consider the partial derivatives 3f f(xi +t,X2) - f(x !,x2) = lim Эх, t-o t 158 CURTAIN and 9f ffx^xg +t) - f(xj,x2) = lim 9x„ t-» о However, it is still not obvious how we should interpret f'(x 0) ( x - x 0) when x - x0 e ]R2 For И ', we have f'(x 0) (x - x 0 ) = f'(x 0)X (x - x 0) and since the generalization of scalar multiplication is the inner product, we could take f'(x0)(x - x0) = at 9f V f = 9xx ’ Эх2 This is usually written 9f , ^ 3f , df = - — dxi + - — dx2 9xj 1 9x2 But note that this definition depends on (1) the inner product in ]R2 (2) a natural basis in ]R2 (3) the fact that f is real-valued. Hence this approach will only generalize to functionals on K n. Let us now return to f : IR-* E and f(x0 +t) - f(x0) f'(x0)(x - x 0) = lim (x - x0) t-> о Write x - x0 = n and, replacing t by tr/, we have f(x„ + tri) - f ( x 0)' f'(x 0)r) = lim t This latter interpretation makes sense for f:X-*Y, where X need only be a vector space and the space Y possesses some topological structure (so that the limit operation makes sense). IAEA-SMR-n/2 159 Definition 71. Gateaux derivative Given x and r) in X if f(x +tn) - f(x) Df(x)r) = lim t exists, then f is called Gateaux differentiable at x in the direction of r). We say f is Gateaux differentiable if it is Gateaux differentiable in every direction, and in this case the operator Df(x) :X-* Y which assigns to each ri € X the vector Df(x)(ri) £ Y is called the Gateaux derivative at x Example 74. Consider f : IRn- ]R and the basis e¡ = (0, .., 0, 1, 0 ,..) then n i=l and Example 75. Let f: R 2-IR be given by Then exists iff П = (rjj, 0 ) or (0 , n2) This example shows that the existence of the partial derivatives is not a sufficient condition for the Gateaux derivative to exist. Example 76, Let f: K 2-*IR be given by x ^ 0 ; f(0 ) = 0 Then This example shows that the Gateaux derivative is not a linear operator. 160 CURTAIN Theorem: If the functional f :X-*K has a minimum or a maximum at x £ X and Df(x) exists, then Df(x) = 0. Proof: If rieX is such that Df(x)r)>0, then for t sufficiently small ^■(f(x + trj)-f(x))> 0. Consequently f(x+tr)) > f(x) if t> 0 and f(x + trj) < f(x) if t<0. A similar argument can be used if Df(x)r]<0. Example 77. Let ^[0,1] be the vector space of real-valued functions which are continuous on [0,1]. Consider T :-^[0, l]-> ]R defined by l T(y) [|(x+l)y(x )2 -y(x)]dx 0 Then l DT(y)(rj) =J [(x+ l)y(x) - 1] rj(x) dx 0 Hence for a minimum DT(y)(n) = 0 Vn Let rj(x) = (x + 1) y(x) - 1 ; then we see (x+l)y(x) - 1 = 0 .'. y(x) = l /x + 1 . The concept of the Gateaux derivative did not require any topology on the domain space, this can lead to "unusual" properties, for example: Consider f : IR2-* IR, where f(x) = — , x / 0 ; f(0) = 0, then Df(0)(rj) = 0 Vr) G X. x 2 Hence Df(0) exists and is a continuous linear operator. But f is not continuous at 0. In order to make sure that differentiable functions are continuous we now introduce the concept of a Fréchet derivative. Definition 72. Fréchet derivative Consider f:X —Y, where both X and Y are normed linear spaces. Given x£X ; if a linear operator df(x) exists which is continuous such that Г II f(x + h) - f(x) - df(x)(h) II 'I n . S i m i then f is said to be Fréchet differentiable at x and df(x)(h) is said to be the Fréchet differential of f at x with increment h. IAEA-SMR-17/2 161 It is easy to see that if the Fréchet differential exists, then the Gateaux differential exists and the two are equal. Moreover, if f has a Fréchet differential at x, then f is continuous at x. Example 78. Suppose f : IRn-»IRm is Fréchet differentiable at x, then fdî1 м Д Эх j Эх \ df(x)(n) = Élu 9fr i Эх, Э х п у If x is near x0, i.e. || x - x 0 || is sm all, then || f(x) - f(x0) - df(x0)(x - x 0) || is near zero. Here a good approximation to f(x) is f(x) = f{xQ) + df(xQ)(x - X0) So that the Fréchet derivative is essentially a linear approximation to f(x+Ax)-f(x) for ||Дх|| small. We shall now see how these concepts can be applied to problems of approximation and optimization. Newton's method Consider the non-linear operator P:X-»X and suppose we wish to find x £ X such that P(x) = 0. Given x jE X , let r¡ = x - x0. Then we require r¡ so that P(x 0 +n) = 0, i.e. P (x 0 +rj) - P(x0) = - P(x0). But if x0 is near x, a good linear approximation is P(x 0 +rj) - P(x0) = dP(x0)(rj) Hence dP(x 0)(ri) = -P(x0) and if dP(x„) is invertible, rj = - (dP(x0 ))'1 P(xQ) So that x = Xq - (dP(xo) ) - 1 P(x0). This is readily recognized as Newton's method, so that the method makes use of the Fréchet derivative to replace a non-linear problem by a sequence of linear ones. Euler-Lagrange equation A classical problem in the calculus of variations is that of finding a function x on the interval [a,b], minimizing an integral of the form b J=j'î{x(t), x(t),t)dt a To specify this problem we must agree on the class of functions within which to seek the extremum. We assume that f is continuous in x, t and x and has continuous partial derivatives with respect to x and x, and we seek a solution in the subspace of ¿ 4 a ,b ] for which x(a) = 0 = x(b). A necessary condition for an extremum is that for all admissible h DJ(x)(h) = 0 162 CURTAIN Now DJ(x)(h) = f(x+£eh, x+ah, t) dt a- 0 DJ(x)(h) Ц (x, x, t) h(t) dt + H (x, x, t)h(t) dt and it is easily verified that this differential is actually Fréchet. If we assume that a continuous partial derivative with respect to t exists, by integrating by parts, we can write ’ 9f d 8 f_" DJ(x)(h) : h dt + [ - h i _ Эх dt Эх . Эх _ Since h(a) = 0 =h(b), for an extremum b 9f___d_ df_ hdt = 0 V h e ^ la .b ] with h(a) = 0 = h(b) Эх dt dx It can be shown that this implies 9f_ _ _d_ 9f_ = Q Эх dt Эх The Euler-Lagrange equation Example 7 9. What is the lifetim e plan of investment and expenditure that maximizes total enjoyment for a man having a fixed quantity of savings? We assume the man has no other income except that obtained through investment. His rate of enjoyment at a given time is a certain function V of r, his rate of expenditure. Thus we assume it is desired to maximize: T J~ e 'flt V(r(t)) dt о where the e"6t term reflects the notion that future enjoyment is counted less today. If x(t) is the total capital at time t, then x(t) =ox(t) - r(t) where a is the interest rate. Thus we maximize T J e'6t Vicrxit) - x(t)) d t IAEA-SMR-17/2 163 subject to x(0) = s, x(T) = 0. For the Euler-Lagrange equation ce " 61 V'bx - x) + e"6t V'(ax - x) = 0 V 1 (r(t)) = V'(r(0)) e If V(r) = 2 -fr, then r(t) turns out to be r( 0 ) 2¡ a - 6 11 r(t) = r( 0 ) e2 and x(t) = x(0 ) - e L ¿p-a _ + 2(B -a e If o'> (3 > о /2 , from x(T) = 0, we have The total capital grows initially and then decreases to zero. High-order derivatives Just as in ordinary calculus, it is possible to define higher-order derivatives by induction. Let us first take an example. Example 80. For a particular case of f : K ^ -IR , take f(x) = X j + XjX2 + x2 Then f(x 0 + tri) - f(x0) о n . 3 ------1------= 2х01^ 1+Ч Л 2 + 2x01 x 02r<2 + 4x 02rl 2 + tr}^ + tx01r)| + 2tr]1x02r¡2 + 6x Qgrig + 0 (t2) where Now let x 0 +tr] = x, i.e. tr) 1 = x 1 - x 01, tr) 2 = x 2 - x 02. Then up to quadratic terms in (x - xQ ) f(x) -f(x Q) = 2x01 (Xj - x 01)+ (x j - x 01)x 22 + 2x01x 02(x2 - x 02) + 4 x §2 (x 2 - x 02) + (x i - x 01)2 + x 01(x 2 - x 02)2 + 2x 02(x i - x 0i )(x 2 - x 02 ) + 6x 02(x2 - x 02) 2 = - (x - x 0 )'2b+ (x - x 0)'Q(x - X 0) 164 CURTAIN where 2x 0i + x 02 2 b = - 2х 01х 02 + 4xq2. and 2 2 x 02 Q = 2x02 2xmC01 '+ 12xq2 If f : R2-* 1R is twice Fréchet differentiable, then its first Fréchet derivative Vf is a vector, Vf = 2b, say, and its second Fréchet derivative is a matrix, say Q, and if x is near x0, a good approximation to f(x) is f (x) = f(x0) + <2b, x - x0>+<(x - x 0), Q(x - x 0) > where the inner product is just the scalar product of vectors. In the more general case f:H->IR, where H is a Hilbert space, the same approximation is valid, where 2bEH and Q ei^H ), which has application in Iterative methods If a functional is twice Fréchet differentiable on a Hilbert space H and has, say, a minimum at x0 then near x0 the behaviour of f(x) must be of the form f(x) = f(x0) + <2b, x - x0>+<(x - x0 ), Q(x - x0) > So that many minimization problems can be examined by looking at the minimization of the functional f(x) = <^x,Qx> - 2 <(x,b^ where Q is a positive-definite self-adjoint operator on H. Let us examine two methods of minimizing this functional. 1. Steepest descent It is easy to see that the minimum is given by x o, where Qx0 = b. Write r = b - Qx and note that 2r is the negative gradient of f at the point x. Now consider the following iterative programme: where rn = b - Qxn and an is to be chosen to minimize f(xn+1). Now f(xn+1) = <(xn+anrn), Q(xn+«nrn)> - 2<(xn+o-nrn),b> = a 2 11 r n II2 X" +1=X" + 2. Conjugate gradient In this method we try to reformulate the problem as a minimum norm problem. Again consider f(x) = <(x, Qx У - 2<(x, b У and introduce a new inner product Since Qxo = b, the problem is equivalent to minimizing Suppose we generate a sequence of vectors that are orthogonal with respect <ч» )*q • These are usually called "conjugate directions". Let x 0 be expanded in a Fourier series with respect to this sequence. If x n is the coefficient of this expansion, then by the theories developed in section 4 Hxjj-XqIIq is minimized over the subspace spanned by the first n of the conjugate directions (say p2, p2, .... pn) • In order to compute the Fourier series of x0, we have to compute < x 0 ,Pi >Q = x n+i = x n + e-Pn e = Too see this, we have v _ v , < Pn,b - Q x 1 -Q y n > Уп+1_Уп - „ * <Рп.У-Уп>а ' yn Since yn is in the subspace spanned by px, . . . . p ^ and since the pj's are orthogonal with respect to <^ , <Рп’ Уп>0 = 0 and „ . . <Рп»У0 >Q Уп + 1 yn [|_ || 2 Pn ' Pfl "Q Thus „ V 9. PROBABILITY SPACES AND STOCHASTIC PROCESSES Probability theory and statistics can mean very different things to the pure mathematician or the statistician. To the pure mathematician probability theory is a very special case of a measure space (Í2 ,P , m) where ц(и) = 1, but to the statistician it means likelihoods of events, normal distri bution, x_distribution or ways of dealing with random phenomena. Here we shall provide the mathematical framework of probability theory as it fits in nicely with our functional analytic approach. Definition 73. A probability space is a measure space (i2,P,p), where ц(£1) = 1. Sets in P we call events, and ц a probability measure. It is useful to compare the different terminology used in probability theory and in measure theory for essentially the same things: probability space (С1,Р,ц) measure space sample point w e Í2 element in Q event A e P measurable set sure event Q whole space impossible event 0 empty set almost surely a.s. "I almost everywhere with probability one w.p.l / random variable measurable function expectation integral Let us consider some simple illustrative examples. Example 81. Let Г2 be the possible outcomes of tossing a die three times. clearly has 63 sample points of the form (2 , 1 , 6 ) and examples of meas urable sets in P or events are A = {a 6 is turned up in at least one of the tosses} В = {a 3 is turned up 3 times running} IAEA-SMR-n/2 167 Then you can see that ц (A) = 3/6 = 1/2 and /л (B) = 1 / 6 3 Example 82. Let £2 be the collection of all possible outcomes of flipping a coin 50 times. A typical sample point in £2 is и = (H,H, T, ...,H , T) a 50-tuple. Let p be the probability of getting H on a toss (allowing for 'unfair' coins). Then q = 1 - p is the probability of getting T. Then it is well known that the probability that you get n heads and 50 -n tails is 50! pnq50-n (50-n).'n! This is the binomial distribution. Example 83. Let a e for x > 0 f(x) • 0 otherwise If Г2 = 1R, then ц (A) = / f(x)dx is a probability measure on IR. This is called A the exponential distribution. We have already said that a random variable is a measurable function x : £2 -> R . In applications you usually think of it as some quantity whose value you can never predict exactly, but you can predict the probability that it will have a certain value. Example 81, for instance, in the die question, one random variable is the number turned up on the second throw; you cannot say what it will be, but you do know that it will be any of the numbers 1 - 6 with equal probability 1 / 6 . Definition 74. A convenient way of expressing this information is by a probability distribution function of x, denoted by F(t) and defined by F(t) = ju {и : x(u) S t} i.e. F(t) is a real-valued monotonically increasing function of one variable. F(-oo) = 0, F H = 1, 0 S F(t) â 1. Example 84. Let £2 be the 36 possible outcomes of rolling 2 dice. Let x be the random variable which assigns to each outcome the total points of the 2 dice. Its distribution function is shown in the following figure: 168 CURTAIN We note that /л {tj Sx(u) S t2} = F(t2) - F(t1). Definition 75. If F is sufficiently smooth, there exists a function f(s), the probability density function such that t where we may need to interpret the integration as an infinite summation in the case where x is a discrete random variable. Discrete random variable x takes on countably many distinct values, for example in the die experiment. Continuous random variable. The possible values of x are some interval in IR. where A for t> 0 otherwise Define the random variable x(u) Then eo the exponential distribution function and the exponential probability density function is oe at for t > 0 f ( t ) = j 0 otherwise IAEA-SMR-17/2 169 Joint distribution functions If x and y are two random variables on the same probability space (Г2, P ,ц), we define the joint probability distribution function F(t, s) = m {x(u) й t , y(co) S s} Similarly you can define the joint distribution function for n random variables F(t v M (x¡ (u) s tt ; i= 1, ...,n } The joint probability density function, if it exists, is given by ‘ 1 41 F(tj, ...tn) = J ... J f(tp ... tn) dt1 ...dtn Definition 7 6 . Expectation This mathematical definition is motivated by the notion of the mean or average value of a random event repeated infinitely many times. E {x }=J" x(w) d/u (w) provided of course x(.) is integrable or x(.) € Li(n, P, ц ). the equivalence class of measurable functions x : i i - ’ K > / I х(ш) I d/u(u) We then say that x has finite expectation. If E{ I x 12} Definition 77. If x £ L 2 (n ,P ,/и) we can define the variance cr2(x) = E{ | x - Ex |2} which is a measure of the dispersion or spread of the values around the average value. a(x) is called the standard deviation of x. Example 87. The normal distribution has the probability density function - t i- а)г f(t) = 1 e 2 C.2 n/ 2iro and Ei—i {x}г n = a, a 2 ,(x) , = a 2 170 CURTAIN F(t) = probability that the outcome is ê t. This space of second-order random variables L 2(i2,P,/u) is a Hilbert space with the inner product <(x, y )> = E{ xy} = / x(u)y(w) d/ufu). n Definition 78. We also define the covariance of x, yG L 2 (£2,P,n) cov(x, y) = E{ (x - Ex) (y - Ey)} and the correlation coefficient , . cov (x, y) Р(х-У)= а(х)а(у) Stochastic independence Intuitively this means that the outcomes of two events are not related. For example, if you toss a dice twice, the outcomes of the two tosses are independent. However, if you consider the event A= {the outcome of the first toss} and В = { the sum of the outcomes of the two tosses}, then clearly these are dependent events. This is formalized mathematically as follows: Definition 7 9. x and у are independent random variables if /u{ x(u) 5 a, y(u) S /3} = ju{x(u) Sff}n{y(u) S J3} Similarly,x1(x2, ...,x n are mutually independent if n /u{xj(u)) s e ¡; i = l . . . n } = П M{Xi(u) s /uj An important result is that if x, у 6 L 9(f2, P, u ) they are independent iff E{xy} = E{x( E{y] . IAEA-SMR-17/2 171 Definition 80. Stochastic processes A stochastic process is a family of random variables Xt, t being a parameter. Discrete stochastic process X( ; t = 0, ± 1, ± 2 , ... Continuous stochastic process {xt}, te an interval in R. For each t, xt has a well-defined distribution function. An important class of random variables in applications are Gaussian random variables. Definition 81. A random variable x is Gaussian if its probability density function is of the form (t-a)» 1 " 2°г ~ •J2-ÏÏ Го-1 a e i.e. it has a normal distribution. Important properties of Gaussian random variables 1. If x and у are Gaussian, so is ox + jSy. 2. The probability properties of x are completely determined by its expectation and its standard deviation (i.e. by its first two moments only). 3. A weighted average of a sequence of n independent identically distri buted random variables in Ьг(£2, P,ju) tends to behave like a Gaussian random variable as n ->00 (Limit theorem), i.e. we can approximate a large number of independent random factors by a Gaussian law. Definition 82. A Gaussian stochastic process is one for which each random variable x t has a Gaussian distribution. The most general Gaussian process is completely specified by parameter functions p(t) and r(s,t), where p(t) is the expectation function: /u (t) = E {x t} and r(s,t) is the covariance function: r(s, t) = Е {хгхз} - /и (s) ju(t). (r(s,t)= r(t, s) and the matrix [r(tj, 11)] is non-negative definite.) The joint probability density function of xt ...... xtn is J a .. j 1 / 2 , ^ ~ f(Sl ...... Sn>= ^ ‘jn/SS eXp { - 2 X âijtSj-MÎtpXS; -M(ti))j where (a¡j) = (r(t¡, tj) ) " 1 and its determinant is | a¡j | . A Gaussian stochastic process is a special case of second-order processes for which xte L 2 (£2, P,p) for each t. For second-order processes we define ¡j (t) = E {x t} and r(s, t) = E{xtxs} -p(t) p (s), which, although they do not completely specify the process, they do provide very useful information. 172 CURTAIN An important subclass are wide-sense stationary processes where r(t, s) = r(t - s), which are extensively used in engineering applications. DEFINITIONS In reading new material and in particular reading these rather concise notes it may be useful to have a list of the major definitions. Definition 1. Linear vector space: A set of elements with a binary operation under which it forms a group and an associated scalar multiplication by the real or complex numbers, which is associative. Definition 2. Linear subspace: if У is a linear vector space, then a subset S of У is a linear subspace if x,y e S =» ex + |3y e S. Definition 3. Affine subset: is the set M = { x :x = x 0 +c, where x0 e S, some linear subspace, and с is a fixed element of X}. Definition 4. Convexity: A subset A of У is convex if x, y£ A=»Xx + (1 - X) у e A, VX > OS X s 1. Definition 5. Linear dependence: if xlf x2, .., xn e X and 3ci ...aa not all zero such that o 1x 1+o'2 X2 +... + enxn = then x b ...,xn are linearly dependent. Linear independence: No such ax, exist. Definition 6 . Dimension: If xx, ...,xn are linearly independent and any vector in X can be represented as a linear combination of xx, ..., xn then X is said to be of dimension n. Definition 7. Basis: The set xi, ...,x n of Definition 6 is called a basis for X. Definition 8 . Isomorphic: Vector spaces X and 'W are isomorphic if Э a bijective linear map T :У~" W. Definition 9. A hyperplane is a maximal proper affine subset. Definition 10. Metric space: A set X of elements {x, y ,...} and a distance function d(x,y) with the properties (i) d(x,y ) § 0 (ii) d(x, y) = 0 iff x = у (iii) d(x, y) = d(y, x) (iv) d(x, y) S d(x, z) + d(z, y) d(x,y) is a metric on X. Definition 10a. Pseudo metric: Condition (ii) is replaced by d(x,x) = 0 (d(x,y) = 0 does not necessarily imply x = y). Definition 11. Normed linear space X is a linear vector space with a norm on each element, i.e. to each xGX corresponds a positive number ||x|| IAEA-SMR-17/2 173 such that (i) Il x II = 0 iff x = 0 (ii) II ахЦ = \a I II x H for all scalars (iii) II x + у II á H x H + И у К Definition lia . Seminorm || x|| = 0 need not imply x = 0. Definition 12. A subspace of a metric space (X, d) is (A,d) where ACX. Definition 13. Product metric space XXY={(x,y):xeX, yeY }, where (X,dx), (Y, dy) are metric spaces and the metric on XX Y is a suitable function of dx, dy, e.g. d lx ^ y j, (x2, yg)) = dx(x1; x 2) + d y(y !,y 2) . Definition 14. Continuity: ftX —Y a map between metric spaces (X, dx) and (Y, dv) is continuous at xp e X if given e> 0, 3 a real number 6 > 0 such that dy(f(x), f(x 0) ) < 6 , whenever dx (x,x0) < 6 . Definition 15. Uniform continuity: f is uniformly continuous if the above 6 = 6 (e) is independent of the point x0. Definition 16. Convergence: A sequence {xn} in a metric space (X,d) converges to x 0 in (X, d) if d(xn, x 0)-► 0 as n — oo. Definition 17. Cauchy sequence: {xn} is Cauchy if d(xn,x m)-* 0 as m, n—°o. Definition 18. Completeness: A metric space (X, d) is complete if each Cauchy sequence converges to a point in X. Definition 19. Closed set: A set A in a metric space (X,d) is closed if it contains all its limit points. Definition 20. Banach space: A complete normed linear space. Definition 21. Dense: A linear subspace S of a metric space X is dense in X if its closure with respect to the metric 2 X. Definition 22. Connected: A metric space is disconnected if it is the union of two open, non-empty, disjoint subsets. Otherwise it is connected. Definition 23. Contraction mapping: f:X -X , where (X,d) is a metric space, is a contraction mapping if 3 k, Oskë 1 such that d(f(x), f(y)) á kd(x,y) V x.yÊ X Definition 24. Compact: A set A in a metric space is compact if every sequence in A contains a convergent subsequence with limit point in A. Definition 25. Relatively compact: A set А С X is relatively compact if its closure is compact. 174 CURTAIN Definition 26. Equicontinuity: A set AC ¿(X , Y), the space of continuous functions from (X,dx), a compact metric space, to (Y,dy), a complete metric space, is equicontinuous at xo£X if given G> 0 36 > 0 such that I f(x) - f(x 0 ) I < € Vf € A, whenever | x - xoI< 6 and x 6 X, Definition 27. Open ball of a metric space (X, d) Br(x0) = {x eX ; d(x,x0) < r}, the open ball at x 0 of radius r. (Closed ball and sphere are similarly defined with S r and = r, respectively.) Definition 28. Local neighbourhood of a metric space (X,d). A subset N of X, such that N = Br(x0 ) for some r / 0. Definition 29. Equivalence of metrics. Metrics di and d2 on the space X are equivalent if (a) f : (X, dj) -* (Y, d3), an arbitrary metric space is continuous iff f :( X ,d 2)-> (Y, d3) is continuous. and (b) A sequence {xn} converges to x0 in (X, dj) iff {x n} converges to x 0 in (X, d2). Definition 30. Open set. A set A С (X,d) is open if it contains a local neighbourhood of each of its points. Definition 31. Topology: The class of all open sets of (X,d) is the topology of (X, d). Definition 32. Continuity: A map f : (X, dJ-'lY, d2) is continuous if the inverse image of each open set in (Y, d2) is open in (X,dx) (see Definition 14). Definition 33. Convergence: A sequence{xn} in a metric space (X, d) converges to x 0 in X iff xn is in every open set containing x0 for sufficiently large n (see Definition 16). Definition 34. Closed set: А С X is closed if its complement is open (see Definition 19). Definition 35. Separable metric space: (X,d) is separable if it contains a countable subset which is dense in X. Definition 36. Measure space is a couple (X,¿^) consisting of a set X and a cr-algebra ,5^ of subsets of X. (A subset of X is measurable if it is iná&) Definition 37. Measure ц on a measurable space [?L,â8) is a non-negative set function defined for all sets in âdwith the properties M 0) =0 IAEA-SMR-И /2 175 M ц E¡ where E¡ are disjoint sets in SB. [¥>.,SB, ц) is then i=l i= 1 a measure space. Definition 38. Complete measure space (Х,&,ц) is one which contains all subsets of sets of measure 0 . Definition 39. Measurable function f-.X-IRu-t00} is measurable if {x:f{x) n Definition 40. Simple function: g(x) = ^ c¡xE.(x), where c¡ are constants i=l and Xej is the characteristic function of E¡GágL Definition 41. Integrable function f:X -*E U {«) is integrable if J fd/u<°°- E Definition 42. Absolute continuity on I (compact interval in IR): For every G > 0, 36 > 0 such that n k = l n whenever For I, an arbitrary interval, F must be absolutely continuous on every compact subinterval. Definition 43. Topological isomorphism of two normed linear spaces X, Y Э Т б^ Х Д ) such that Тл€ 5^(Y,X); T is a topological isomorphism. Definition 44. Isometric isomorphism: Il Tx H y = Il x||x Vx g X Definition 45. Inner product is a bilinear function <( .,.) : XXX — С such that 1. <(ex+/Зу, z> = o-<^x, z)>+J3<(y, z)> for scalars a, fi. 2 . Definition 46, A Hilbert space is a complete normed linear space under the inner product norm |K . Definition 47. Orthogonal, x is orthogonal to у iff ^х,у)> = 0. 176 CURTAIN Definition 48. Orthogonal complement of M С Hilbert space H is M1 = { y 6 H: Definition 49. Orthonormal set in a Hilbert space H is a set {0n } such that РтУ= énin • Definition 50. Complete orthonormal set when H = Sp{}. Definition 51. Linear functional on a normed linear space X is a linear map f :X -»R . Definition 52. Algebraic dual of X, Xa is a linear vector space of all linear functionals on X. Definition 53. Topological dual of X, X* is the normed linear space of all bounded linear functionals on X. Definition 54. Reflexive space: X** = X. Definition 55. Convergence in norm: xn -» x in norm if Цхн-хЦ-'О as n->oo. Definition 56. Weak convergence f(xn)->f(x) as n -»«> VfGX*. Definition 57. Weak* convergence in X*:x*(x)-*x*(x) as n-*®, VxEX. Definition 58. Weak compactness: A set A in X is weakly compact if V {x n} С A, there is a weakly convergent subsequence with limit point in A. Definition 59. Weak* compactness: Replace weakly by weak* in Definition 58. Definition 60. Linear transformation T : X-* Y, where X , Y are linear vector spaces is > T(ox + |3y) =oTx +|3Ty. T is an operator if X = Y. Definition 61. Invertible transformation T : X -» Y is invertible if 3 G : Y - X > FG and GF are identity maps. G is the inverse of F. Definition 62. Continuous transformation T : X -» Y at x 0 if for every e > 0, 3 6>0 such that | | тх -тх 0||<€ , whenever || x - x0 || < 6. Definition 63. Bounded transformation T : X - Y if || Tx||y S k || x ||x , for some К > 0 and Vx G X. Definition 64. £f{X, Y) = { T : X-» Y, where T is bounded and linear} IAEA-SMR-17/2 177 Definition 65. Adjoint T* of Te5f(H), H Hilbert space, is given by Definition 6 6 . Self-adjoint operator: Ae5f(H), when A = A*. Definition 67. Positive operator: AG.Sf(H) if ( A x , x ) i 0. Strictly positive if<(Ax,x^ = 0 onlyifx = 0. Definition 6 8 . Closed operator T on a Banach space X if for all sequences { x j in the domain of T, ¿^(T), with x n -» x and Txn - y, then x e ® (T ) and y = Tx. Definition 69. Resolvent p(T), spectrum cr(T) of a linear operator T on a Banach space X p(T) = { XeC : (XI - T)'1 e a( T)= C-p(T) Definition 70. Compact operator Teâf(X) and T maps bounded sequences into sequences with convergent subsequences. Definition 71. Gateaux derivative: Given x and r) in X, if exists, then Df(x)r) is called the Gateaux derivative at x in the direction of r). If it exists in all directions r), Df(x) :X->Y is called the Gateaux derivative at x. Definition 72. Fréchet derivative: Consider f:X-»Y; X, Y normed linear spaces. If 3df(x)e.Sf(X, Y), such that Г II f(x + h) - f(x) - df(x) h II 1 lim X ------г—л------f = 0 l|h||-*0 L II h II J then f is Fréchet differentiable at x and df(x)(h) is the Fréchet differential at x with increment h. Definition 73. Probability space is a finite measure space (£2,P,p) with p№) = 1 . Definition 74. Probability distribution function F of a random variable x is F(t) = : x(u>) s t} 178 CURTAIN Definition 7 5. Probability density function f (if it exists) is given by t F(t) = J f(s)ds Definition 76. Expectation of a random variable x in Li(£î,P,p) is E {x } = J x(u) dц(ш) ü Definition 77. Variance of a random variable x e L 2(£2,P,p) is Definition 78, Covariance function of two random variables x, y e L 2(fi, P, м) is cov (x, y) = E{(x -E{x})(y - E{y})} Definition 79. Independent random variables x, у if й{х(ш) § a, y(u) s /3} =iu{x(u)s a} /j (y(co) S jS} Definition 80. Stochastic process is a family of random variables. Definition 81. A Gaussian random variable has a probability density function (t-a)2 f(t) = v k r e 2o! Definition 82. A Gaussian stochastic process {xt} is one for which xt is a Gaussian random variable. BIBLIOGRAPHY NAYLOR, A .W ., SELL, G .R ., Linear Operator Theory in Engineering and Science, Holt, Rinehart and Winston (1971). TAYLOR, A .E ., Introduction to Functional Analysis, Wiley, New York (1967). SIMMONS, G .F ., Introduction to Topology and Modern Analysis, McGraw-Hill (1963). BACHMAN, G ., NARICI, L ., Functional Analysis, Academic Press, New York (1966). YOSIDA, K ., Functional Analysis, Springer, Berlin (1966). KANTOROVICH, L .V ., AKILOV, G .P ., Functional Analysis in Normed Spaces, Moscow (1955). DUNFORD, N .. SCHWARTZ, J ., Linear Operators I and II, Interscience Publ. (1963). IAEA-SMR-П/З CONTROL THEORY AND APPLICATIONS A.J. PRITCHARD Control Theory Centre, University of Warwick, Coventry, United Kingdom Abstract CONTROL THEORY AND APPLICATIONS. Most o f the theories In control have been developed assuming a mathematical model in terms of differential equations. This part of the course will examine the existence, uniqueness, and regularity of both ordinary and partial, linear and non-linear differential equations. A. ORDINARY DIFFERENTIAL EQUATIONS 1. Linear autonomous systems The simplest systems in control theory are those described by equations of the form x = Ax, x(0) =x (1.1) where x : [ 0 , t ] -»Rn, A G i/(R n,Rn) and is represented by an nXn matrix, x is assumed to be Cx[0,t ] so that the above equation makes sense. Very many linear differential equations can be formulated in this manner. For instance, consider the damped harmonic oscillator x+kx + u2x = 0 , x (0 ) =Xj, x(0 ) =x0 Introducing x = y, we find x =y у = -k y - ь?х or X ' 0 1 ' X x(0 ) xo * _ÿ- - - к _ -У. -y(0). - * i - It is very useful to have an explicit representation of the solution of Eq.(l.l). To do this, we first introduce the normed linear space of nXn m atrices 179 18 0 PRITCHARD Definition If A is a matrix of numbers (a^) with n rows and n columns we define the norm of A, ü A I) by where || || is any norm on Rn. It is easy to show that b) d A + B D S ü A ü + |j В II c) H AB j) i H A d (I В У We have, in fact, introduced many different spaces depending on the definition of ¡I ||Rn , However, it is easy to see that all these spaces are topologically isomorphic (because of the equivalence of norms on Rn). Definition The exponential We define ^ by where I is the identity matrix and where we assume j| A || sk say. Then e* is well defined since and it is easy to show the following properties eAt = A » e At = e At. A dt c) det A = etrA where tr A is the trace of A d) However, eA+B Ф eAeB unless AB = BA Solution of Eq.(l.l) From the property (a) above it is obvious that x(t) = eAtx is the required solution. IAEA-SMR-17/3 181 Let us illustrate this by considering the following example: Example x + x = 0 , x (0 ) = x1, x(0 )= x 0 Then x 0 l" X x(0) xo _-l 0. -У- _y(o). Lxj J To calculate eA it is sometimes useful to use the characteristic equation det(XI-A) =0 In our case this becomes Л2 + 1 = 0 But we know that every matrix satisfies its own characteristic equation, so that A2 + I = 0 From this we obtain A2n = ( -1 )П I Hence eAt = 1 [ 1-t 2/ 2 ! + t 4/4 ! - •••] + A [ t - t 3/3 ! + t 5/5 ! -•••] = I c o s t + A s in t co s t sint -sin t cost Then x(t) co s t sint x o as required. -y(t). -sin t cost XU The inhomogeneous equation Now consider x = Ax + f(t), x(0) =x ( 1 . 2 ) where for the moment we will assume that f is continuous. Then the solution is t x(t) = el x + Г e* We note that this solution is well defined, and verify that it is a solution of Eq.(1.2) by the direct calculation: t x(t) = AeAtx+ eA(t't' f(t) + J A eA(t‘ s) f(s) ds о = Ax(t) + f(t) In particular, if f(t) =Bu(t) where u€R m is tobe thought of as a control, and В is an nXm matrix: t x(t) = é*x + J e A(t"s) В u(s) ds o is the solution of x = Ax + Bu, x(0) =x Of course, this requires that u(t) be continuous, and this is usually too strong a requirement for control problems. However, before we try to overcome this difficulty we really need to examine what we mean by a solution to a differential equation. This will be carried out in the next section, and then we shall return to the above problem. 2. Existence of solutions Consider the system of non-linear ordinary differential equations x=f(t,x), x(t )= x (2.1) where x is a vector-valued function defined on an interval I =[т,т + а] and f{t,x) is also vector-valued and defined on IX В =D where В = {x e R n, ¡I x - x II s b } We will assume that f £C°(D) and define a solution x(t) if a) (t, x(t)) £ D , t e l b) x(t) = f(t,x(t)), x(t ) =x If x is a solution on I then clearly x e C 1(I). Integrating Eq.(2.1), gives t x(t)= x + J f(s,x(s))ds (2 .2 ) T Obviously (because f 6 C°(D)), Eq.(2.1) will have a solution if and only if Eq.(2.2) has a solution. There are many different kinds of existence, and IAEA-SMR-П/З 183 uniqueness theorems for Eq.(2.2). We shall describe the Picard-Lindeloff theorem because of its historical importance, and also because the proof enables the solution to be constructed. Picard-Lindeloff theorem Let fGC°(D) and satisfy a Lipschitz condition ||f(t,x)- f(t,y) II SK||x - y II, t e l , x,y G В Assume ||f(t,x)|| Sm for (t,x) G D and set с =min (a,b/m). Then for t St St +c there exists a unique solution with x (t ) G В Proof Consider the sequence of successive approximations x 0(t) = x (2.3) T We will show by induction that xk exists on [r, т + c], xkGC 1 a n d ||xk+1-x|| Sm(t- r) k= 0 , 1,2 ... Obviously, x satisfies these conditions. Assume xk does the same, then f(t,x k(t)) is defined and continuous on [t, т+ c]. Hence from Eq.(2.3) xk+1 exists on [r, т+ c] and x k+1 GC1 and obviously T It now remains to show the convergence of xk. Set Ak(t) = ||xk+1(t) - x k(t) (I, t G [т,т+с] Then T Hence t t Ak(t) SK T r 184 PRITCHARD But = II3E1('t) - 3c II s m ( t - T ) So, by induction, Ak(t) § mKk (t - t )k+1/(k + 1 )! This shows that the series oo I k=0 is majorized by the series (m/K)(e?c-1 ) and hence the series is uniformly convergent on [t, т + с ] . Thus the series OO x„(t) + ^ (x k+i(t)-xk(t)) k = 0 is uniformly convergent and the partial sums n-l Xjjft) = x0(t) + (xk+1(t) " k = 0 tend uniformly on [t, t+ c ] to a continuous limit function x. We now need to show that x satisfies Eq.(2.2). Clearly, t t II J [f(s,x(s))- f(s ,x k(s))]ds|| S J"||f(s,x(s))- f(s ,x k(s)) К ds T T t S К J~ ¡I x (s ) xk(s) ü ds r Now [Ix(s ) - xk(s) I] -* 0 as k -*00 uniformly on [t, t + c ] and hence x(s) satisfies Eq.(2.2). To show the uniqueness of this solution, let us suppose there are two solutions Xj, x 2 on [t, t + c ] . Then t x 1( t )- x 2(t) = J [f(s, Xjfs)) “ f(s, x 2(s))] ds T Hence t Xj(t)- X2(t) II SK J ||xa(s )- X2(s)||ds Г IAEA-SMR-17/3 185 by the Lipschitz condition. We conclude the proof by an application of the following lemma: Gronwall's lemma Let a e L 1(t, r + c), a(t)g О, Д e L° (t , t + c ) and assume that b is absolutely continuous on [т, т + с]. If A(t)sb(t)+ /a(s)A(s)ds then A (t) S Ь(т) exp / a(s) ds + / b(s)exp( / a(f) df Ids Proof Set t H(t) = J a(s) A(s)ds T Then A(t) Sb(t) + H(t) Multiplying by the integrating factor t exp a(s) ds H(t) = a(t) A(t) almost everywhere. С С _d_ H(t) exp -J a(s)dsj â a(t)b(t) exp -y^a(s)dsj dt Integrating from т to t since H(t) = 0, we obtain H(t) exp a(s) ds exp a(?)d| ds S 'J b(3) d i = Ь(т) - b(t) exp a(s) ds + J b(s) exp - a(?) dÇ ds r Hence the lemma is proved. 186 PRITCHARD To apply the lemma we set д (t) = II x x(t) - x 2(t) b(t) = 0 a(t) = К Hence Xj(t) - x 2(t) II s 0 and so Xj(t) = x 2 (t) This concludes the proof of the theorem. 3. Extension of the idea of a solution The discussion in Section 2 required that f be continuous in the (t,x)-domain. This is very restrictive for control problems since if we consider the system x = f(t,u(t),x) where u is a control, we do not wish to consider only continuous controls. In proving the existence and uniqueness we wrote the differential equation in an equivalent integral form (3.1) r Now, Eq.(3.1) will make sense for a wider class of f than those in C°(D). In particular, we may ask whether or not an absolutely continuous function x defined on I satisfies the above integral equation. In this case, the differential equation x = f(t,x) will only be satisfied almost everywhere (i.e. except on a set of Lebesgue measure zero). Defining the solution in this way, Carathéodory proved the following theorem: Theorem Let f be measurable in t for each fixed x and continuous in x for each fixed t. If there exists a Lebesgue integrable function m on the inverval [т, т+а] such that Il f(t,x) ü =j m(t) (t,x)£D then there exists a solution x(t) on some interval [t, t + / 3 ] satisfying x(t) = x. IAEA-SMR-17/3 187 Proof Define M by M(t) = 0 t M(t) = / m(s) ds t e [t, t + a] T Then M is continuous, non-decreasing and M(t) =0. Hence there exists 3 such that (t,x)SD for some interval te [т,т +0] if ||x-x || SM(t). The following iteration scheme is now introduced: Set x (t) = x t e [т, t +j3/j] t-0/j T We now show that this scheme defines Xj(t) as a continuous function on [r, t + J3]. Clearly, Xj(t) = x is well defined. For any j 6 1, the first formula defines Xj on [r, т + 0 /j] and since (t, x) 6 D for t € [t, т + jS/j] the second formula defines Xj as a continuous function on (т + 3/j, т+2/3/j]. Furthermore, on this interval Xj(t)-X ü SM(t-j3/j) We can now define X]- on (r + 2j3/j, т+З/3/j] by the second formula, and so on. Note that for any two points tj, t2 we have Xj (tj ) - Xj(t2)|| S I M (tj- (3/j) - M(t2- УЗ/j) I This implies that the set {xj(t)> is an equicontinuous, uniformly bound set on [т, t + /3 ]. Hence, by Ascoli's theorem, 3 a subsequence xJk which converges uniformly on [t, t+ 0 ] to a continuous limit function" x as k-*°°. We now show that this limit function is a required solution. To do this, we shall apply the dominated-convergence theorem of Lebesgue. First note that f(t,Xj (t) II sM (t), t 6 [t, t + j8] and since f is continuous in x for fixed t: f(t, Xjk (t) ^f(t, x(t)) as k -°° 188 PRITCHARD Then by the dominated-convergence theorem lim J f(s, x^(s)) ds = J f(s,x(s))ds r T But r I- B/jk and we have II / f(s,x. (s)) ds II -* 0 as k -»00 'k Hence t x(t) = x + / f(s, x(s)) ds ■ / r The conditions of the theorem only guarantee the existence of a solution. To obtain unique solutions, it is necessary to impose further conditions, for example Lipschitz conditions. Another important problem is that of the continuation of solutions beyond T + /B. We shall not consider this problem but refer those interested to the bibliography. 4. Linear systems We now return to the problem x = Ax + Bu, x(0) =x where u is measurable on I and || u(t)|| Sm (t), where m is Lebesgue integrable on I. We see that f(t,x) = Ax + Bu is continuous in x for each t and measurable in t for each x. Moreover, for (t,x) £D f (t, x) (j S K A ü d x ü + ü В (I m(t) f(t,x) satisfies a Lipschitz condition since f(t,xi)-f(t,x 2)|| = II A(X;L - x2) I) s d A ü ü X j- x 2 ü Hence all the requirements of Carathéodory's theorem are satisfied and 3 a unique, absolutely continuous solution characterized by IAEA-SMR-17/3 189 Non-autonomous systems We can generalize the problem to consider equations of the form x = A(t)x, x(t) = x where we assume that A is measurable on I and Il A(t) ü S m(t), t £ I where m is Lebesgue integrable, and uniformly bounded. Then by Carathéodory there exists a unique solution. Now let us consider the problem for which x = ej i = 1,2,... n, where the e¿ are a basis for Rn. We might as well choose e¡ to be the vector with zeros everywhere except in the i-th place where there is a one. We denote the solution of this equation by cpj (t,T). For any x £ Rn there is a unique expansion n i=l and by linearity the corresponding solution will be n i= l If we construct the matrix Ф (t, t) for which each column is the vector cpj (t, t) then Ф (t, t) = A(t)5> (t, t) Moreover, since Ф; (T, t) = e¡, Ф(т, т) = I We call $(t, t) the fundamental matrix. The solution is given by n n = Ф{\, т ) X Note 1 In the case A(t) = A it is easy to verify that $(t,T) = eA(t"T) 190 PRITCHARD Note 2 In general, t Ф (t, т) Ф exp J' A(s) ds r Non-homogeneous system F or the controlled system x = A(t)x +Bu, x (t) =x where u G L 2 (I, ЕШ), B Ë i /( R m,R n) It is easy to show that the solution is t x(t) = ®(t,r) x + / $(t,s) Bu(s) ds ■I T B. PARTIAL DIFFERENTIAL EQUATIONS 1. Introduction The next section will be concerned with partial differential equations. There is a vast literature on this subject and it is obviously impossible to give anything m ore than a flavour of the subject in this paper. We shall, first of all, motivate the introduction of distributions and weak solutions, then go on to semigroups and mild solutions, and finally use monotone- operator theory to deduce results for non-linear partial differential equations. A partial differential equation for a scalar function u is a relation of the form where etc. Эх ’ ХУ ЭхЭу It may happen that this equation is supplemented by constraints on u and its partial derivatives on the boundary Г of the region Í2 throughout which the independent variables x,y... vary. These constraints are called boundary conditions, and if one of the variables is identified as time the constraint associated with that variable is called an initial condition. The order of the partial differential equation is the order of the highest derivative occurring in F. We shall use, throughout this paper, three examples of partial differen IAEA-SMR-17/3 191 tial equations which are of important physical significance. These examples are representatives of three classes of partial differential equations: para bolic, elliptic, hyperbolic, and we shall use them to illustrate the general results. The equation governing the conduction of heat: parabolic A good approximation to modelling the variation of the temperature T, in a rod of length I, is given by Tt. = kTx.x. This can be simplified by introducing х'/í = x, t1 = ( i 2/k)t when Tx t = T •‘■XX The boundary conditions may take a variety of forms, e.g. a) T (0 ,t)= T (l,t)= h 1(t) b> Txlx 'x = n=Txlo x 'x = l iBhzW ¿ where hj, h2 are given functions of time. The initial condition could be of the form T(x,0) = T0 (x) for some given function T0 (x). Laplace equation: elliptic In R2 this takes the form V 2d> = ф + ф =0 r v xx v yy The equation represents many different phenomena ranging from the potential of some electric field to the stream function of a fluid flow. A variety of boundary conditions can be imposed depending on the particular physical situation. If, for example, the boundary is a circle С we could have ■Hc = fi (C ) = i 2(C) where фп is the derivative of ф in an outward direction normal to С and fi,f 2 are two functions defined on C. 192 PMT CHARD The wave equation: hyperbolic Transverse vibrations of a taut string are governed by the wave equation where z is the displacement from the x-axis. If the ends of the string are fixed, then the boundary conditions are z(0 ,t) = z(l,t) = 0 The initial conditions could be z(x,0 ) = g 1(x) zt(x,0 ) =g2(x) where g , g2 are given functions. A classical or strict solution to a partial differential equation (p.d.e.) is defined to be a function u(x,y,...) such that all the derivatives which appear in the p.d.e. exist, are continuous, and such that the p.d.e., boundary conditions, and initial conditions are all satisfied. It may be thought that with this definition of a solution the main emphasis should be focussed on deriving methods by which a solution can be obtained. However, the story is not quite so simple. If we consider the wave equation z tt= z xx, z(0 ,t) = z(l,t) = 0 z(x,0 ) = zQ(x) zt(x,0 ) = 0 then it is easily verified that the solution is z(x,t) = |[z0(x+t) + z0 (x-t)] However, if z„(x) is given by z Q(x) = x, O é x é I / 2 z Q(x) = 1 - x , x S 1 0 2 1 Then the above solution cannot be a classical solution since z 0(x) is not differentiable at x =-|. So what do we mean by a solution to this problem ? It is obvious that we must widen the concept of a solution if we are going to allow initial conditions of the form given above. Let us consider another example. An obvious computational method for determining a solution to the heat conduction equation is to approximate the equation by T(x,t+At) - T(x, t) _ ^ ^ At IAEA-SMR-П /З 193 If we assume that T(x,0) is a function of the form z0 (x) (as given above), then the computation cannot even start because it is not possible to evaluate Txx(x,0) at x = l/2 It is obvious, therefore, that reasonable definitions of solutions must allow for these eccentricities, and must include a link between the spaces in which the initial and boundary conditions lie, and the space in which the solution is sought. If we transform the wave equation by setting y =x - t p =x +t we find z p y = 0 ° r z yp = 0 The equation zpy = 0 is satisfied by any function of y only, but the expression Zyp need not make sense for every such function. This is most peculiar and indicates the need for some generalized concept of a function. We shall now show how the unnatural results of the above equation can be resolved by formulating a different concept of a solution, and this will lead to particular generalized functions — distributions. If г 6 С2й (the space of twice continuously differentiable functions on Q) where £2 is some bounded domain in R2 and f GC°(£2 ) then z =f ( 1 . 1 ) РУ makes sense. Integrating by parts twice, we find JJ zcpypdydp = JJîtpdydp ( 1 .2 ) n « for all ф € С о ( £ 2 ) We define a weak solution to be a z such that formula (1.2) holds. If f is C°(£2) and z 6 C'¿[Q) then the weak solution is a classical solution. However, the concept of a weak solution allows us to consider a larger class of f. Note that since py Typ we have JJ z Фру dy dp = JJ Z Фур dy dp =JJ f ф dy dp Í2 Í2 Q 194 PRITCHARD so that the weak solutions of Zpy = f « and Zyp=f are the same. One way of looking at the weak solution is to think of zpy as being represented by the linear form a zcpyp ^y dp Then the study of differential operators leads to the study of continuous linear functionals on C0 (S7). 2. Distribution theory There are many different ways in which distributions can be defined. We shall choose to define them as elements of the dual of a certain space D(f2), i.e. D'(f2). So we shall first of all define and give some properties of the space. The space Р(П) Let К be any compact subset of f2CRm and let DK(Q) be the set of all functions фе С " (Q) such that the supports of the ф'a are in K. We define a family of seminorms on DK(f2) by PK n(«í>) = sup |D“0 I where the sup is taken over all | or | §n<°° and all x € K. and э « 1 + а2 + . . . + a n D“ Inductive limit We say that a set is open in D(i2) if and only if for every convex, balanced, and absorbing set V e D(Q) the intersection V П Dk (Г2) is an open set of DK(f2) IAEA-SMR-17/3 195 containing the zero vector of D¡<(Í2) for all K. The topology defined in this way is the "inductive limit" of the DK(Q). A better grasp of these ideas can be obtained by understanding what is meant by convergence in D(Œ). It can be shown that lim фг =ф means that both of the following conditions hold: a) There exists a compact К С Г2 such that the support of r, r = 1,2 ... is in К b) For all D“, О01фг(х)->О“ф(х) as r->°° uniformly on K. Before we look at distributions in more detail, there are a number of useful technical devices which although we shall not use them are worth mentioning because of their importance in a thorough treatment of the subject. First, we define cp(x) = C f( II x||2- 1) where and с is a constant chosen so that J ф dx = 1 Rn Then it is easy to show that cp £ C ¡ (Rn). Regularization of z:z 6 If z is an arbitrary integrable function, we define its regularization z£ by Substituting ey =y' we find 196 PRITCHARD Theorem Let z be integrable and vanish outside a compact subset К of £2. Then ze 6 Cq(£2) if e is sm aller than the distance 6 from К to £2' (compliment of £2). As e-*0, ze-*z in Lp,lSp<°o and ze-z uniformly if zg cjp ). The proof of this theorem is immediate from the definition of ze and the representation A direct result of this theorem is the so-called partition of unity. Partition of unity Let {K¡ > i = 1,2... be open sets such that {K¡> covers an open set £2CRn. Then there exist functions a¡ such that a. 0 a)' Л È , * i=l b) a¡€C°° and its support lies in some K¡. c) Every compact set in £2 intersects only a finite number of the Support Of ttj. Definition: Distributions A distribution on £2 is an element T of D' (£2), and will usually be denoted by T(cp), cpGD(£2). Let us give some examples: Example 1 Let f(x) be locally integrable, i.e. J |f(x)|dx<°° for any compact KCRn К Then T(cp) = / f(x)cp(x)dx, cp€D(£2) defines a distribution on £2 which we usually denote by Tf . Note Two distributions Tfj, T(■ are equal if and only if fj=f 2 almost everywhere. Example 2. The Dirac delta function Consider the distribution T(cp) = cp(0), ф G D(£2) IAEA-SMR-17/3 197 We shall call this the Dirac delta function concentrated at the origin and write 6 (ф). More loosely,this is written as 6 (x). If instead we consider the distribution T(cp)=cp(a), ?£D(Í2) we write ба(ф) or 6 (x-a) Example 3 Both of the above examples are special cases of the following distributions. Let d m be a measure on Í2 , then a is a distribution. If for example d|u=fdx, f € L jC (Q) then we obtain the distribution constructed in Example 1. Example 4 If Т(ф) is a distribution, so is T(fç) for f G С (П). Example 5 OL If Т(ф) is a distribution, then so is T(D ф). A useful characterization of a distribution is: a linear functional T defined on C“ (Г2) is a distribution if and only if for every compact subset К of there corresponds a positive constant с and integer k, so that I Т(ф) I icPRk whenever ф £ DK(f2) We now make use of Example 5 to define the differentiation of a distribution. Differentiation of a distribution We define (Da T) (ф) by (D“ T ) M = ( - l ) |a|T(D % ), ? € D H Example 1 Consider the Heaviside function 198 PRITCHARD Then '(x)dx = cp (0 ) since ф has compact support. So symbolically H'(x) = ó (x) Example 2 We can define the product of a function f €C°° and a distribution T as in Example 4. fT(cp) = T(fq>), tp€D(n) Then the differentiation (DfT) (ф) is defined by (DfT) (ф) = -ГГ(Оф) = -T(fDqp) also DfTfa) = TfaDf) and fDT(cp) = DT(fq>) = -T(D[fcp]) Now D[fç] = fDq> +фDf Therefore (DfT) (cp) = DfTfo)+fDT(cp) Note It is not possible to define the multiplication of two arbitrary distributions. 3. Sobolev spaces Definition Hm(f2), where m is a positive integer, is the space of distributions Т(ф) such that DaT 6 L 2 (f2) for all a, |a|sm provided with the norm |a|==m LAEA-SMR-П /З 199 with the inner product <ТГ Т2>ш = 1 < D“T ’ D“T2>L^ ) |a| Hm(ii) is a Hilbert space Remark I If M> m HM(ft) CHmfflC L 2M def_ H°(fi) Remark II The delta function does not belong to any of the spaces Hm(ii), m è 0. To see this we note H ■*;■> { ти р * } ■ T {|ИГ“} Lz(i!) l|ŸV(fi) It is always possible to choose a cp€ D(f2) such that ср(0)ё к || Ф [|L2(^ for any given k. Hence 6 H°(Г2) and so by Remark I cannot belong to Hm(£2) for ms 0. This remark indicates the need to consider more general spaces of distributions. Tempered distributions Let us consider the special case of f2 = Rn. We define the Fourier Transform of ф 6 L 2(Rn) by Ф(у) ^ ф = ^ 1n/2 / exp [-i < x,y/] ф(х) dx ( 2 n V n R where < x ,y > = ^ y 1 +x2y2 + ... + x nyn Then ф -$ is an isomorphism of L2 (Rn) onto L 2(Rn) and Ф = ^2 ^n /2 J exp dx Rn Definition: Tempered distributions Let S ={ф: x“ D% 6 L 2 (Rn) for all a,0} 200 PRITCHARD Then with the seminorms S is a Frechet space. We define the tempered distribution as elements of S' = dual of S with the strong topology Note For all cpG S and for all a F(D“ cp) = (iyfFcp and we can define the Fourier transform of u G S' by û =Fu where (Fu,(p) = <(u,Fq))> V t(£S where <^, У denotes the duality between S' and S. Theorem If П =Rn, Hm(Rn) can be defined by Hm(Rn) = Ilu : u G S' and (1+ ||у||2Г /2и G L 2(Rn) with m L2(Rn) P roof From Plancherel’s theorem L2(Rn) 'L2(Rn) Hence Rn |oi|£ m But there exist constants Cj, c 2 such that |a|£m which proves the theorem. We note that the above definition of Hm(Rn) does not require m to be a positive integer, and use the above to define Hm(Rn) for all n positive and IAEA-SMR-П /З 201 negative, m integer or not. It is then obvious that H_S (Rn) Э H°(Rn) 3 Hm(Rn), -s < 0< m We can also show that a) If H°(Rn) is identified with its dual {H s(Rn)}' =h ' S(r") b) D(Rn) is dense in Hs(Rn) for all s. The definition of the space Hs(ft) is a far more complicated matter and does not fit into the above pattern in the sense that a) and b) are not true for general s. Although we will have cause to use results for these spaces the development is outside the scope of these lectures, and the interested reader is referred to the literature (e.g. the book by Lions and Magenes, see bibliography). There is a particularly simple representation of the spaces HS(F) where Г is the boundary of ft and is assumed to be a С "-manifold. If Аф. + Х.ф. =0 ч Г ) where oo U = ^Ujl//j j=l Then for all s 6 R co Hs(r) = ju: ueD '(r), Y 4 S luj |2<00} j=i Regularity It is very important to know whether a weak solution to a partial differential equation is a classical solution. This quite often involves comparing Hs(ft) spaces with L2 (ft) and Ck(ft) spaces. The .important results in this area are contained in the following Sobolev embedding theorem We shall write X Q Y to denote the continuous embedding of X in Y, i.e. u for all u Il II I skIIuIL 1 1Л ex 202 PRITCHARD If s > n/2 + k, S1CR" Then hs (ST2) q. d'(n) We shallnot prove this theorem but by a simple application of the Schwarz inequality indicate its validity. F or any ф G D(f2), f2=(0,l) Ф2(х) = Hence x e l‘ l2(«) So by continuity Sup I 12 * Il II2 for all u in the closure of D(f2) in the H1 norm. We shall denote this space by H¿(f2) and we see H'0 (Г2) Q. C°(f2) , Í2CR1 An alternative definition of Н*(Г2) is the following Definition Нр(П) Let Г2 be a bounded open set С Rn with a suitably smooth boundary Г. Then u G Нд(Г2) if and only if a) uGH1)!)) Q j b) = 0 on Г, 0 si (i) D(f2) is not dense in Hs(f2), s> 1 /2 (ii) Hg(f2) = closure of D(f2) in Hs(f2) (iii) (Hj(n)}' =H-S(fi). IAEA-SMR-И /З 203 4. Application to partial differential equations We shall begin by considering a simple class of elliptic equations П DJ (aijD 1u) = - f in Г2 I (4.1) *• u = 0 on Г where DJ'= ^ — , a.. 6 L°(iî) 9 x j ч and we assume n n o2 r i,j=l i=l We define а(и,Ф) = £ Д . D*u iÿ çd x , ^f,9 >= Jîydx. i, j =1 Q 0 for cp ED(f2). A weak solution of Eq.(4.1) is defined to be a solution of а(и,ф) = Because of the imposed conditions we obtain а(ф,ф) 2 K ü cp ü ц1(0) The proof of the existence now follows as a direct result of the Lax-Milgram theorem which states: If a(u,v) is a bounded bilinear form on a real Hilbert space X such that a(v,v) È К II v ||2 , v € X For each f SX' there exists a unique uE X so that a(u,v) = Of course,this reduces to the Riesz representation theorem when a(u,v)= a(v,u) and so is an inner product on X. We see therefore that for f e H _1(f2), since D(H) is dense in H¿(f2), there is a unique solution u of Eq.(4.2): u 6 H¿(f2). If f EHS(Q) it is possible to show that there exists uE Hs+2(f2) П H'0(f2) such that V2u = f In particular, if Í2 CR2, s = l+ e,e> 0 , u€ H 3+е(Г2) П Hj(f2) and so by the Sobolev embedding theorem u G Сг(Г2) and is therefore a classical solution. This indicates how it is possible to obtain classical solutions via weak solutions. 204 PRITCHARD 5. Evolution equations We shall again consider a space XC H where H is a real Hilbert space, then identifying H with its dual H ', we have X CH CX' where X 1 is the dual of X. We shall seek a solution of f = A(t)x + f x(0) = xQ, x QGH The precise spaces in which f and the solutions lie will be explained later. We define the bilinear form a (t ; Ф, Ф) = - a) a (t ; ф,1/') is measurable on ( 0 , t ) and I a(t; Ф, 0) i S с II ф||х |||//||х ф ,0 € Х , where |[ ||x denotes the norm on X. b) a(t; ф,ф) ёК||ф||2, K> 0 for all ф GX, t G(0,t ) so that A(t) G ^ ( L 2(0,t; X); L 2(0 ,t;X ')) i.e. if f GL2 (0,t; X), A(t)f is the function t-A (t)f (t) GX 1 We know how to define df/dt for f G L2 (0,t; X). To do this we first define f G D1 (0,т; X) by D'(0 ,t; X) =í/(D(0,t); X) the space of distribution on (0, t ) with values in X. Then if f G D '(0,t; X), f( f (ф) = Jî(t)(fi(t)dt о and identify f GL2 (0,t; X) with f (ф ) GD'(0,t; X). The derivative df/dt is then defined by df , , J ¿ф\ IAEA-SMR-17/3 205 Hence ¿ED'IO. t ; X , We now introduce the space W (0,r) W(0,r) = £L 2 (0, t ; X), G L 2(0, t ; X')} endowed with the norm TT 0 0 W (0,t ) is a Hilbert space. For more details the reader is referred to Lions and Magenes (see bibliography). We seek a solution x GW(0, t ) to the evolution equation ^ = A(t) x + f dt (5.1) x(0) = x 0GH, f G L 2(0,t; X') Uniqueness Consider E q.(5.1 ) with f = 0, x 0 = 0. Then a(t, x(t), x(t)) + < ^ (t), x(t)> = 0 But on integrating by parts T 0 Hence T 0 and so x =0 206 PRITCHARD Existence We shall assume that X is separable so that there exists a countable basis e1; e2 en, ... such that e^ ... e„ are linearly independent for all n and finite combinations are dense in X. Set n x (t) = ) q. (t) e. n / , Ш 1 i=l and choose c*in so that <£xn(t). e. > + a(t,xn(t), ep = 1 Si Sn with ata(0 ) =Qin0 and n q. . e. ->x„ in H as n -»00 mo i о Zi=l The differential equations (5.2) are of the form Bn % ■ + Ап^Уп = fn’ Уп(°) = iQino> where Bn is the matrix with elements ^e¡, ejX An(t) is the matrix with elements a(t, e¡, ej ), y„ is the vector with elements ain(t) and fn(t) is the vector with elements <(f(t), е^)>. Since the e^ are linearly independent det Bn ф 0 and so the above equations admit a unique solution. We shall show that xn-*x (a solution of the original equation) as n-»°°. Multiplying Eq.(5.2) by orjn(t) and summing over j,an easy computation yields \ "ST Hxn(t)l|2 +a(t: xn(t)’ xn(t)) = xn(t)> Then using the lower bound on a(t; xn(t), xn(t)) and integrating r r l k (T)^ + 2K / Hxn(t)llx dt S llxn(°H|2 + 2/| < f (t), xn(t)> |dt 0 0 IAEA-SMR-17/3 207 T 11хп(°}|1н+2 f II f M Hx- ü Xn(t) llX dt by the Schwarz inequality. But 2 1 2 / llf(t)llX' llXn(t)llx dtS & f llXn(t) llx + — 0 0 and ||xn( 0 )||^ S a||x0 ||* for some a, /3> 0. Hence ■ .II’ * j I I « « I I ” . Л for some c. Thus xn ranges over a bounded set in L? (0,т; X) and we may extract a subsequence so that xp->z weakly in L2 (0,t; X) We shall now show that z is the desired solution. Let i be fixed but arbitrary and let p>j, then multiplying Eq.(5.2) by cp(t),where n = p and cp(t) £ С 1(0,т) ср(т) = 0, integrating on (0,t) and setting 9 ¡(t) = T T J - Since xp-*z weakly, we have - Since the above is true for ф SC^O, т), ф(т) = 0 we may take ф€ D(0,t). Then in the sense of distributions in D'(0,t) we have But the e^ are dense in X so that f = A(t)z+f Hence dz. 9 - ^ e L (0 ,T ;X ') and so z £W (0,t ) 208 PRITCHARD It is easy to verify that z satisfies the initial condition, so we have obtained the desired solution. Example 1 Let Q= Œx (0,t) where Í2 is an open bounded set in Rn with a smooth boundary Г. Consider n where и = 0 on Г, u(x,0) = uQ(x) n n i.J =1 i=l We take X = ВЦП), so that X' = H_1(«) H = Н°(П) = L2 (Q) Then for cp, ф e Hj(i2) n X, j “1 & so that all the conditions for the uniqueness and existence theorem are satisfied. Hence, for f EL 2(0,t, H'^fyj.Uge L2 (Г2) there exists a unique solution и 6 L 2 (0,t; H'0 (£2)). Example 2 Consider the above example where f is to be thought of as a control. Let us associate with the problem a performance index C(f) and try to find the control which minimizes C(f). Before attempting to find criteria for optimality, it is necessary to consider whether or not the problem is well posed, and the existence results will play an important role. For example, if T then we know the problem is well posed since u € L2 (0,t; Hj (&)). However, if IAEA-SMR-17/3 209 the last two term s are well defined but it is not the case that u(r) lies in Hj (Œ) and the problem is not well posed. This kind of analysis becomes even m ore important when the control forces lie on the boundary of the system; for example, see Lions and Magenes (see bibliography). C. STRONGLY CONTINUOUS SEMI-GROUPS 1. Introduction We have seen (Section A) that if A e S^(X), where X is a finite-dimensional Euclidean space, the solution of x=Ax + f, x(0) =x0 (1.1) is t о for any integrable vector-valued function f. Here, It is natural to attempt to define eAt for A unbounded on X, and then derive a similar expression for the solution of the evolution equation (1.1). The above definition is difficult to use for A unbounded since D(An) D D(An+1) and the domain becomes smaller as we consider higher powers of A. More over, there is the problem of convergence! Instead, we choose a different representation of the exponential function, motivated by the scalar formula Of course, it is necessary to make sure that the limit exists, and for this we shall assume a) A is a closed operator such that D(A) is dense in X. b) (Al-A ) " 1 exists for X> 0 and || (XI - A)_1|| S l/X , A > 0. Note that (b) implies II (I - pA)- 1 1) S I, p iO so that (I - t/n А)" 1 is bounded and may be iterated. Now set 210 PRITCHARD Then II Vn(t) II S I and Vn(t) is holomorphic in t> 0 since we know the resolvent (XI- A )'1 is holomorphic for X> 0. Hence ÉXmÍíI = a ( i - - а Т (П+1) dt Vn(t) is not necessarily holomorphic at t = 0, but it is strongly continuous at t =0. To see this, note that - l H V jM x -x ll =t Л (1 - tA) Ax D 6 1К Ax II Hence VjttJx-x, as tiO for all xED(A). V1(t) is uniformly bounded, and D(A) is dense in X, and hence (t) x-*x for all x e X. It is easy to show that the same must be true for the iterates Vn(t), so that Vn(t)x - Vn(0)x=x, ti0 Existence of the strong limit lim Vn(t) ------n_*. oo W rite t-€ vn (t) x - vm (t) x = lim J ± Vm(t-s) V„ (s) ds x 1 1 1 1 CO 1 1 1 = lim -A H-* > Vm(t-s) Vn(s )x _ m + A (l--A j Vm(t-s)Vn(s)x ds Hence I (Vn(t) - Vm(t))x H S I A2x I /( f +^гУ = ^ + ¿ ) ||A2 x| Hence Vn(t)x converges uniformly in t on any finite interval for all x e D(A2). Since D(A2) is dense in X, and Vn(t) is uniformly bounded^lim Vn(t)x exists for all x 6 X, and we shall denote this limit by Tt x. n IAEA-SMR-17/3 211 Properties of Tt It is now necessary to show that Tt has the properties of the exponential function. It is obvious that Tt is strongly continuous, and Il T И S I, T0=I (1.2) Now dVn(t) _ д/т . 1 a'T \T i+\ = V l+\ A i l - Í. dt Vn(t)=VB(t)A(l-rA = A V (t)(l - - A J (1.3) But n x = (l - ^ A^ Ax -"Ax as n-*-“° for all x£D(A). Therefore Vn(t)A(^I-^Ay x -T tAx, x€D(A) and since A is closed AVn( t ) ( l - - A ) x -*■ ATt x so that A commutes with Tt or AT, Э TtA. Integrating Eq.(1.3), t Vn(t)x- Vn(0)x=Vn(t)x-x = f (l - — а Т (П+1' Ax dt, x E D(A) But uniformly for t in each finite interval. Hence t Ttx-x=J" TsAxds, xED(A) 0 212 PRITCHARD so that ^ (T x) = TtAx=ATx, x 6 D(A) Thus x(t) =Ttx Q is the solution of x = Ax, x(0) = xQ if x0 £ D(A) Utilizing this fact,it is not difficult to show that (1.4) Such an operator Tt with the properties (1.2, 1.4) is called a strongly continuous contraction semi-group. It is possible to obtain other semi-groups; for example, if we replace (b) by (i) ü (AI - A)"k II X>0, k = l, 2 ... Then it is possible to show that t J| s m (ii) Il (XI - A)"k К Ш (Х -р )"к Х>Э, к = 1,2... then Il Tt II s M eBt In these cases we shall refer to strongly continuous semi-groups. 2. Solution of the inhomogeneous equation We have seen that the solution of the homogeneous equation x = Ax, x(0) =xQ is x (t)= T tx 0 for x0SD(A) Consider now the equation x=Ax + f, x(0)=x 0 (2.1) where f is assumed to be strongly continuous with values in X, and A generates a strongly continuous semi-group Tt. IAEA-SMR-17/3 213 Suppose that x(t) is a solution, then Therefore Integrating on (0,t) we obtain x(t) = ^ x 0+ / Tt.sf(s)ds 0 Now one would hope that this solution is always the solution of Eqs (2.1) but this is not in general true. However, we can prove the following theorem: Theorem If A generates a strongly continuous semi-group Tt, and a) f(t) is continuously differentiable for të 0 b) x0eD(A) then t о is continuously differentiable and satisfies Eq.(2.1). Proof We need only show that t о satisfies the differential equation and has initial value 0 . Let t о 214 PRITCHARD Then V(t) = J Tt. : f (0 ) + J f'(r ) dr о 0 t t t d s f ( r ) d r ( 2 . 2 ) 0 r Now for x GD(A) -f- [T x ] = AT x ds s s Hence Tt x - Tr x = / ATS x ds Since A is closed and D(A) is dense in X, we have A / T, ds = Tt -T r A Tt-s ds = Tt-r -1 Using this result in the expression (2.2) ensures that V(t) €D(A) and AV(t) = (Tt -I)f(O ) +jT (Tt. r -I) f' (r) dr = -f(t) + Ttf(0) + J Tt.rf'(r)dr 0 Now V(t) = j TS f(t-s) ds о Hence dV(t) = Ttf(0) +J Tsf’ (t-s)ds dt о IAEA-SMR-17/3 215 Therefore = AV(t) + f(t) at Other results of a different form more useful for control applications have appeared in the literature (see Balakrishnan). 3. Mild solution If the conditions on f and x0 are not satisfied, but sufficient conditions are imposed so that t (3.1) о is well defined and strongly continuous, then the solution is said to be a "mild" solution (e.g. f GL 2(0,T; X)). This concept of a solution is particu larly important in control application since if f is regarded as a controller we may require it should only be piecewise continuous. There is a theorem which gives a necessary and sufficient condition for an operator A to generate a strongly continuous semi-group: Theorem If X is a Banach space such that a) D(A) is dense in X b) The resolvent (I - n'1A )"1 exists such that I) (I -n _1A)'m ü S С n = 1,2... m = 1,2... then A generates a strongly continuous semi-group Tt. Unfortunately, these conditions are not easily verified. However, there are a variety of conditions which give sufficient conditions (Hille and Phillips, and Yosida, see bibliography). D. NON-LINEAR SYSTEMS There is a growing literature on these systems (Browder, Kato, see bibliography), and it is not intended to attempt to survey the main ideas in this paper. Instead we shall take a simple example which will illustrate the kind of approach which is frequently used. We shall follow closely the kind of analysis which led to the solution of the linear elliptic problem. First, we state an abstract theorem: Theorem Let T be a mapping (possibly non-linear) of the reflexive Banach space X into its dual X ', which satisfies 216 PRITCHARD a) T is continuous from lines in X to the weak topology in X '. b) There exists с on R 1 with lim c(r) =+°o such that for all x G X (Т х ,х ) й с ( IIx У ) ü x where с) T is monotone, ie. for all x,y G X < T x -Ту, x-y> г 0 Then T maps X onto X '. We shall not prove this theorem but show how a concrete example can be formulated in such a way so that the theorem can be used. Example Consider ! ocj where A a is a non-linear partial differential operator of order m on an open bounded set Г2 , Assume A„:S)XRm^ R1 is continuous on Rm for fixed x G Г2 and measurable in x for fixed f G Rm. We assume Aa (x,?) S c [l+ ||| HRm] and consider the non-linear equation Au = f on Í2 D6u = 0 on Г I /31 g m - 1 Set a(u,i//) = X < (-l)aAa(x,u,Du, ...Dmu), D“i//> and consider a(u,i//) = with u 6 H ” M = X IAEA-SMR-17/3 217 From the assumptions we have I a(u, ф) I S c( I) u (j ) ||0 ||x , Ф e D(f2) This inequality will also hold for ф GH™(f2) since D(£2) is ^ense in HQ (Г2), so that we may restate the problem a(u,v) = <\f,vX v e x u e x We also note that a(u,v) is linear in v on X. Hence a(u,v) is a bounded linear functional on X, and so there exists a unique element Tu E X ', such that (Tu,v) = a(u,v)vex Similarly, we can impose conditions on f so that there exists a unique u E X 1 such that <(u,v)> = (f,v ^ v e x Then the problem becomes <(Tu, v^ = у У v e x i.e. Tu = u> Now we may use the theorem to determine a solution. F or m ore details see Browder (bibliography). D. FUNCTIONAL DIFFERENTIAL EQUATIONS 1. General theory In the previous sections, we have considered ordinary and partial differential equations which are examples of systems whose future behaviour depends on the present state and not on the past. However, there are many applications in control, mathematical biology, economics, etc., where the past does influence the future significantly. One class of such systems is described by differential delay equations or, more generally, by functional differential equations. For example, if we model a population by saying that the growth (or decay) is proportional to the number of people in the population between the ages of 15 and 45, then one model could be Ñ(t) = k[N(t- 15) - N(t-45)] where N(t) is the number in the population at time t, and is a constant. We see here that the current rate of change of N(t) depends on the values of N(t), 15 and 45 units of time earlier. 218 PRITCHARD A predator-prey model studied by Volterra is о Ni(t) = er Tiw 2(t) - / F1(-B)N2(t + e)àe -г о N2 (t) = " e 2 + TiNj(t) +-I / F2 {-9)N1(t + e)de -Г where Na(t) is the number of prey in the population at time t, and N2(t) the number of predators at time t, F1# F2 are given functions, and ei,e2, are constants. Here, we see that the current rates of change depend on the past history through the integral terms. Several more examples may be found in the book of Hale. The theory of existence and uniqueness of solutions of functional differen tial equations is very similar to that for ordinary differential equations except that it is necessary to consider solution segments x(t +0) -b s в й О over a time interval rather than just x(t). The following results are developed in much more detail in Hale's book (see bibliography). Consider the functional differential equation of retarded type x(t) = f(t, xt) (1) where f:RXC([-b,0] ; R n) - Rn, and C([-b,0]; Rn) = all continuous maps cp:[-b,0] -*■ Rn with norm Ik II = k w L [-b .O ] K If a GR, aê 0, and x( • )GC([-b,0); Rn) then for any t G [a,a + a], we let xtGC([-b,0] ; Rn) be defined by xt(0) =x(t + 0), -b s 0 s 0. So xt(0) is the segm ent of the curve x(t), as indicated below A function x(-) is said to be a solution of Eq.(l) if there exists aGR, a> 0 such that xGC([a-b,CT + a]; Rn) and x(t) satisfies E q.(1 ) on [cr-b, a + a). F or a given cr G R and h G C([-b,0]; Rn) we say x =x(a,h) is a solution of Eq.(l) with initial value h at a if there is an a> 0 such that x(cr,h) is a solution of Eq.(l) on [ст-b , a +a) and xo(a,h) = h. Note For the initial condition it is necessary to prescribe a function on [-b .O ]. IAEA-SMR-17/3 219 One can prove the following existence and uniqueness theorem using methods similar to those used in Section A: Theorem a) Suppose D is an open set in R X C ([-b,0], Rn) and f:D->-Rn is continuous. Then if (cr,h) £ D there exists a solution of Eq.(l) through (cr,h). b) If f is also Lipschitzian in h on all compact subsets of D then the solution is unique. Now let us see the implication of this theorem for the linear system о x(t) = Ax(t)+Bx(t + 0j) + J' С(в) x(t + в) d0 (2) -b x 0 =h Clearly, all the conditions of the theorem are satisfied and so there is a unique solution on [0 , °°). If we let x t(h) be the solution of Eq.(2) considered as a differential equation on C([-b,0]; Rn) and write xt(h) =T(t) h then Tt , ts 0 is the strongly continuous sem i-group on C ([-b,0]; Rn) with infinitesimal generator jd given by о ’ AcpiO + Bcpiej) +Jc(9) ^ ( 9 ) = \ d - b S(®) e *° with domain D(a) ={ф, ф' e C([-b,0]; Rn) j. Hence the equation may be written as an abstract evolution equation on C ([-b, 0]; Rn). x t = j ^ x t x0 =h If we consider the inhomogeneous equation x £ = j^ x , +f(t) x 0 = h where f is defined by '«■"•>-{S o efo°í 220 PRITCHARD then by analogy with Sections A and C, we may express the solution t xt = T(t) h + ^T (t - s) f(s) ds о at least for f(t) continuous. 2. Affine hereditary differential equations in M spaces It turns out that the abstract evolution equation formulation for linear delay systems is not the most appropriate for many control applications since C([-b,0]; Rn) is not a Hilbert space. However, it is possible to reformulate the system on a Hilbert space M2[-b,0; Rn] using the construction of Delfour and Mitter. Let us consider the system dx , > _ A , . xit + Qj) Г x(t + 6 ) t + 0 ë 0 — (t) - A x (t)+ B h(t + e j+J c ( e ) h(t + e) t + 6 < 0 (3) 1 -b where OátsT, A,В are nXn matrices and С £ L”([-b,0], (R°)). n Definition M space Consider the space of functions cp: [-b,0]->Rn with seminorm о |ф||м2 = 0 ^ 1 2+/ -b and is the Rn norm. We now define M ([-b,0]; Rn) to be the quotient space of L (-b,0; Rn) enerated by the equivalence classes under T 2< i.e. we say Ф1 = Ф2 in M if -Ф, ■ 0 . Thus M is a function space of a point at 0 = 0 plus a curve -b The point ф(0) is uniquely specified, but the rest of the curve is only specified almost everywhere. M2 ([-b,0]; Rn ) is a Hilbert space with inner product 0 <Ф,(//> = cp'(0 )i//(0 ) +J cp’(s)iMs)ds -b IAEA-SMR-И /З 221 Note. In Hale's formulation, cp was continuous on [-b, 0] and so was uniquely specified at all points. Another way of looking at M 2([-b,0], Rn) is to say that it is isom etrically isom orphic to RnXL 2 (-b,0; Rn). We now define another space AC2(t0,t; Rn) to be the space of absolutely continuous maps [t0,t] -*Rn with derivative in L 2(t0,t-[; Rn) and norm k M 2+ / 6 i 2ds AC tn Then we can write the homogeneous equation Ф (t) =j4 Ф(0 ) = h л/ is a closed linear operator on M2 with domain AC2 and is defined by A h(0)+B h(6. ) + / C(0)h(0)d0 0 = 0 -b j ^ h ) ( 0 ) = (5) ф о jrf is the infinitesimal generator of a strongly continuous semi-group $(t) on M2 and Eq.(3) has the unique solution (Ht)=$(t)h for h £ M2 Consider now the inhomogeneous equation on Ф (t) = jS'cMt) +f(t) (6) (0 ) = h where A is defined by expression (5), and fE L 2 (0,°°; Rn). ThenE q.(6 ) has the unique solution in t ij>(t) = $(t)h + jT $(t-s)f(s) ds о Thus we are able to allow for discontinuous inputs which are particularly important in control applications. Moreover, since M2 is a Hilbert space we are able to obtain simple optimization results for the linear quadratic problem. 222 PRITCHARD BIBLIOGRAPHY YOSIDA, K ., Functional Analysis, Springer Verlag (1968). LIONS, J.L ., MAGENES, E., Non-Homogeneous Boundary Value Problems, Springer Verlag ( 1972). КАТО, T ., Perturbation Theory for Linear Operators, Springer Verlag (1966). CARROLL, R .W ., Abstract Methods in Partial Differential Equations, Harper and Row (1969). CODD1NGTON, E .A ., LEVINSON, N .. Theory of Ordinary Differential Equations, McGraw-Hill (1955). LIONS, J.L ., Optimal Control Systems Governed by Partial Differential Equations, Springer Verlag (1972), HILLE, E., PHILLIPS, R., Functional Analysis and Semigroups, Am. Math. Soc. Colloq. 31 (1957). BROWDER, F ., Existence and Uniqueness Theorems for Solutions of Non-Linear Boundary Value Problems, Proc. Symp. Appl. Maths. A .M .S . (1965). КАТО, T ., Non-Linear Evolution Equations in Banach Space, ibid. HALE, J .K ., Functional Differential Equations, Springer Verlag (1971). DELFOUR,' M .C ., MITTER, S .K ., Hereditary DifferentialSystems with Constant Delays I. General Case, J. Diff.Eqns 12 2(1972) 213-235. IAEA-SMR-17/4 FINITE-DIMENSIONAL OPTIMIZATION D.Q. MAYNE Department of Computing and Control, Imperial College of Science and Technology, London, United Kingdom Abstract FINITE-DIMENSIONAL OPTIMIZATION. Finite-dimensional optimization problems o f the following types are considered in the paper: unconstrained optimization problem, inequality-constrained optimization problem, equally-constrained optimization problem, non-linear programming problem and optimization problems with special structures. 0. PRELIMINARIES Optimization Problem The finite-dimensional optimization problems discussed in these lectures have three ingredients: an objective function f °: Rn -> R, an inequality constraint set Í2 which is a subset of Rn, and an equality constraint set {x£R n | r(x) = 0}, where r maps Rn into Rk . The problems considered are: PI. (Unconstrained optimization problem) min{ f°(x) I x G Rn) , which is shorthand for — find an x in the set Rn such that for all x G Rn, f°(£) S f°(x). P2. (Inequality-constrained optimization problem) m in {f °(x) I xG Ш P3. (Equality-constrained optimization problem) m in{f °(x) I x G Rn , r(x) = 0} P4. (Non-linear programming problem) min{f°(x)|хеП, r(x) = 0} Г2 will generally be defined in terms of a function f:Rn^Rm, that is: 1. Г2_Д {x e R n I f(x) s 0} (the notation f(x) iz , f:Rn-Rm; zGRm, denotes f1 (x) § z1, i=l, 2, ,,.m, where f1, z 1 denote the i-th component of î, z, respectively). 223 224 MAYNE Optimization problems with special structure (for example, f°, f, r, linear or quadratic), and, therefore, simpler to solve, will also be considered. Conventions The following conventions and notation will be used: Rn denotes the Euclidean space of ordered n-tuples of real numbers (R denotes R1). x S R n has components x 1, x2, ...xn e R (x = (x1,x2, ...,x n)). x is treated as a column vector (nXl matrix) in matrix operations. denotes the norm in Rn, defined by || x|| = Symbols: V for all 3 there exists, ^ there does not exist => implies, => does not imply <=* if and only if, is equivalent to Д is defined by I~ such that {x| P} set of points having property P A + x {x|x = a + x, a eA }. n union, intersection AC В A is contained in В z 6 A, (z ^A) z belongs (does not belong) to A 0 the empty set Z= {0 ,1 , 2, 3...} the set of non-negative integers Z+ = {1,2, 3...} the set of positive integers Jm= {1, 2, ...m}, J^= {0, 1,2, ...m} Topology of Rn Let Be(x) 4 {x I II x - x II Sequences { x j G R n} denotes a sequence xi, x 2, x 3 ... in R n. x is an accumulation point of {xi} if VE> 0, VnGI, Эп'ёпэх11' £ B 6(x). x° is a limit of the IA EA -SMR-17/4 225 sequence {x j (or the sequence {x j converges to a limit x°, written x i -> x °) if V£ > 0, 3n° Эхп G Bf(x°), Vn S n°. If x is an accumulative point of {x¡}, Э a subsequence of {Xj} which converges to x, that is, 3KCZsuch that x¡ -* x for i e K. If x is a closure point of A С Rn , 3 a sequence { x, G A} having x as an accumulation point. A С R" is bounded if 3o 6 R э|| x II s a, Vx G A. A С Rn is compact if it is closed and bounded. 2. Theorem A С Rn is compact <=>every sequence {xj £A} has an accumulation point in A. 3. Theorem (Cauchy convergence criterion) {x¡ G Rn} is convergent <=>{xj} is a Cauchy sequence, that is, Ve > 0, 3 n ° e l Э ||xm- x n||< e ,v m, n Sn°. Continuity f : Rn ->Rm is continuous at x 0 if V6>0, 3e>0 3xG B e (x0)-=> f(x) G B6 (f(x0)) (note Bc is a ball in R.n, B¿ a ball in R m). f is continuous on A CRn if it is continuous at all x G A. f : R n->R is lower semi-continuous at x 0 if V6>0, Эе > 0 Э x e Be (xo) f(x) > f(x0 ) - 6 . The definitions for lower semi continuity on A, and upper semi-continuity (at x0 or on A) are sim ilar. Infimum and minimum f : A-*R is bounded from below if 3a £R (ct is a lower bound for f on A) 3f(x) è a,\f x G A. A lower bound â (for f on A) is called the infimum for f on A (a = inf {f(x) I x G A}) if Ve > 0, З х€ А Э f(x) < a + e . Upper bound and supremum are similarly defined. If there exists an x e А Эf(x) g f(x) Vx€ A, then f(x) is called the minimum of f on A, and x the minimizer of f on A. Maximum is similarly defined. Existence of minima 4. Theorem (existence) If f : A->R is lower semi-continuous and A С Rn is com pact, then 3 a solution x e A to the problem min{f(x) | xG A}. 5. (i) f not continuous, f : R-> R, f(0) = 1, f(x) = x, Vx > 0. inf {f(x) | x G [0, 1]} = 0 but there does not exist a x e [0 , 1 ] such that f(x)= 0 . (ii) A not closed, f : R-"R, x -x ; A = (0,1] VxGA, 3 x'G А Э^х')<^х) (for example, x' = x/2). Again inf {f(x) | xG A} = 0, but x satisfying f(x) = 0 does not lie in A. 226 MAYNE (iii) A not bounded. ^ R -’ R, x"x; A = R. VxeA, 3x' GA 3f(x') < f(x). Differentiability f : Rn -* R is differentiable at ЗГ, if 3 a linear function f:R n->R3 V 6 > 0, 3 e > 0 Э : 6 . II f(x°) - f (к) II < 6 II x - x II , Vx £ Bf (x) (equivalently || f(x) - f(x) || = o( || x - x|| ), where o{a)/a-"0 as a - 0). Because f is linear it may be expressed as: fix) = f(x) + < Vf(x), x - x У The function Vf: Rn-*Rn is called the gradient of f, and is regarded as a column vector in matrix operations. 7. Theorem f : Rn -* R is differentiable at x, if all the partial derivatives 3f/9xi, i= l,...n , of f exist and are continuous at x, in which case Vf = f j , f^ a f/Эх!, ...9f/3xn). f : Rn -* R is twice differentiable at x if 3 a quadratic function f : Rn -*R, x " f(x) + < V f(x), x - x > + § Il f(x) - f(x) Il < 6 II x - x||2, V x e B £(x) f is twice differentiable if the partial derivatives 92f/9x¡9xj, i, j= 1, 2, ... n exist and are continuous at x, in which case V f = f J and V2f = f xx , where fxx (x) is a nXn matrix, the Hessian of f at x, whose ij-th element is [fxx(x)]ij = 32f(x)/9xi9xj , i, j = 1, 2 ... n. If A € R lxm, II А К Д max {|| Ax|| | || x|| â l}. f : Rn -* Rm is differ entiable at x if 3f:R n-*-Rm satisfying ( 6 ). If the partial derivatives df^/dx’, i= 1, 2, ...m , j = 1, 2, ...n exist and are continuous at x, then f : Rn - R m satisfying ( 6 ) exists and is defined by: f(x) = f(x) + f x(x)(x - x); (the ij-th component of the Jacobian fx : Rn-»Rmxnis Sfi/Sxi). Mean-value theorems 7. Theorem Let f : Rn -»R be continuously differentiable (Vf exists and is continuous at all x € R n). Then Vx, h £ R n, VXGR, 3 | e [x ,x + Xh], the line segment joining x and x + Xh Э : f (x + Xh) = f(x) + X <(y f(f ), h У 8 . Theorem Iff:R n->Rm is continuously differentiable, then V x.heR ” , VXeR": l f(x + Xh) = f(x) + fx(x + t.Xh)dt Xh о IAEA-SMR-17/4 227 Convex Sets The set of points joining Xj, x2 6 Rn is denoted [x1( x2] = { x £ R n | x = x-l + A(x2-xi), Xe[0,ll}, and is called a line segment. (x 1; x 2) A { x 6 Rn | x = x i+ M x 2 -xx), X e ( 0 ,l)}. A set A С Rn is convex if, \/х1; x 2 e A, the line segment [x1( x 2 ] € A . If x 6 A, A convex, and 3 no distinct хь x2 € А Эх£ (xx, x 2), then x is an extreme point of A. The intersection of a finite or infinite family of convex sets in Rn is convex. If x 1; x2 ,...x mGR n, then the convex hull of xx, x2, ...x m, denoted co (xj x 2 . ..x m) is the set ix € Rn | x X, g 0 for i= 1 , 2 , ...m , m I i = l ХЧ-i= 1 If x 0 , x x, ...x n 6 Rn are such that (x ! - x0), (x2 - x 0), ...(x n - x 0) are linearly independent, then co (x0,xj , ..xn) is a simplex in Rn. A set С is a cone with vertex xq if x 6 С => x 0 + X(x - х0)б С, VXaO. The dimension of a convex set С С Rn is the dimension of the smallest subspace M 3C C x + M | L f o r s o m e x e R . The relative interior of such a set, is rint C ^ {x 6 l| 3e> 0 Э B£(x) П С С L}. For example, if a, b £ R n, int[a,b]=0, rint [a, b]= (a, b). Two sets are disjoint if they have no points in common. Two sets А, В С Rn are separable (strictly separable) by a hyperplane Hc(a)A {xGRn| x £ A = > ( c, x ) so ( ( с ,х ) < й ) x£B=>(c,x)io Kc ,хУ>а) НСЫ is said to separate A and B. Two sets may be separable but not disjoint, or disjoint but not separable. However: 9. Separation theorem Let A, B C R 1 be convex and let A have dimension n. Then: A and В are separable <=* int А П rint В = ф. Note, if A, B e Rn are convex and disjoint, then A, В are separable. The separation theorem may be used to prove a result, Farkas Lemma, much used in the sequel. 10. Farkas lemma Let A € R mXn, b £ Rn, and let the columns of AT be ax, a2, .. ame R n. Then the following two statements are equivalent: (i) {b} and SA {x 6 R n I x = A у .у й 0} are not disjoint ( 3y 6 R k, у й 0 ЭАту'= = £ ) a¡y¡а;У;=Ь =1 i=l (ii) { b} and S are not separable ( Convex functions Let X С Rn be convex. f:X -R is convex if x 1; x2 6 X, Хе[0,1)Ф f(x 1 + X(x2- xx)) § f(xj) + X(f(x2) - f(xj)). f is concave if - f is convex. x is a (strict) local minimizer of f : Rn -*R on XG Rn if Эе > 0 3f(x) g f(x) (f(x)> f(x)), Vx G Be (x) П X ( Vx G B£ (x) П (X\{ x }). (X \ Y Д {x G X | x $ Y}) 11. Theorem If f :X -»R is a convex function on a convex set X C R n, then each local minimum of f on X is also the (global) minimum of f on X. 1. CONDITIONS OF OPTIMALITY Assumption In the sequel, f°:R n^R, f:R n^Rm and r : Rn -> Rk will be assumed to be continuously differentiable. 1.1. Unconstrained optimization 1. Theorem If f°(x) is a minimum (or local minimum) of f 0 on X , where X is an open subset of Rn , then: V f°(x) = 0 Proof Эе > О Э Ne (x) С X. Suppose V f°(x) / 0 and let h A Vf°(x)/||vf°(x) || . By local optimality 3ei G [0,е] Э (x-eh) G X and f °(x - oh) ê f°(x) Vo'G[0,e1]. But: f°(x - ah) = f°(x) - a = f°(x) -a [c + o(o)/ff], c > 0 so that 3c G (0, e j 3 f°(x -¡yh) s f°(x) - a c/2, a contradiction. Hence Vf°(x) = 0. 1.2. Inequality constraints Consider P2 : min { f°(x) I xGS2С Rn} where Q A {xG Rn | f(x) s 0} , where f:R n->-Rm. Vx G Rn let Io(x) A { i G{1, 2, ...m } I f^x) = 0}. The following condition of optimality is due to Zoutendijk. IAEA-SMR-17/4 229 1. Theorem If xG£2 C R n is a solution to P2, then: 6 (x) A m in m ax{< V f°(x),h ^ ; If0(x) = - 6 Hence 3 o > 0 such that: f°(x +oh) s f °(x) -ab 12 f‘ (x+ah) S -a ô /2 , V iG I 0(x) fHx+ohisO, Vi ei'(x) where Iq(x) denotes the complement of I0(x) in JmA { l,2 ,...m } , contradicting the optimality of x. To proceed further, we have to introduce the concept of a 'linear approximation' to Г2 at x. A convex cone is a linearization of {U - x) Л {h 6 Rn I h = x - x, xGf2} if, V sets { h 1, h2 , ...hj} of linearly independent vectors in С, Эе > О Э co{0 , eh1, eh2 , ...eh ]} С (Q - x). If h €C , 3e>0 3c'he(!2-x), Ve'G[0,e]. 2. Theorem If either A(i) {VfMx), iGI(x)} is a set of linearly independent vectors^or A(ii) 3h G Rn Э 3. Example f\x) = (x1- l)3 +x2, f 2(x) = -x2, 5= ( 1 , 0 ), I 0 (x) = { 1 , 2 }, Vf'Hx) = (0, 1), V f 2(x) = (0, -1). С is not a linearization. 230 MAYNE 4. Theorem Suppose icefi is a solution to P2, and С is a linearization of (U-x). Then: Proof For, if not, 3heC, 6>0 Now, 3ei > 0 Эх + eheŒ, Ve e [O.ej], and 3e2 e [0, e J Э f°(x + eh) < f°(x ), V f 6 [0 ,£ 2 l. Thus V e € [ 0 ,e 2 ], x+eh6f2 and f°(x+ eh) < f°(x), contradicting the optimality of x. 4a. Corollary. <[vf0(x), h)âû, VheC. Proof This follows from Theorem 4 and the continuity of the function 5. Kuhn-Tucker Theorem Suppose x 6 fi is a solution to P2, and A(ii) of (1.2.2) holds. Then э { X' й 0, i€ I0 (x)} Э : V f°(x)+^ Х‘у^(х) = 0 Proof Since A(ii) holds we can replace С in (4) and (4a) by C, so that: - By Farkas' Lemma (0.10), Э{X*_> 01 i€ l 0 (x)} Э. : . ie l o ( x ) 1.3. Equality and inequality constraints Consider P3 : min{f°(x) I f(x) S 0, r(x) = 0 }. Let g : Rn -*■ Rk + 1 be defined by: g(x)=(f°(x), r(x)) IAEA-SMR-17/4 231 W e considerg is continuously differentiable, its Jacobian being gx We considerg first the relatively simple case when r is affine. 1. Theorem If x is an optimal solution of P3, С is a linearization of (Г2 - x) and r is affine, then 3X = (Xo , X1, ...Xk ) G R k+ \ Xo S О, Х /0 , Э: Proof Let Czâ gx(x)C SA {zG R k+1|z = -(3(1,0, ...0), /3 > 0} Cz and S are convex cones. We first prove that Cz and S are linearly separable. For, if they are not, Cz has dimension k+ 1, and all points of S are in the interior of Cz. Let z* e S; since z* G int С z, 3h* e С Э z* = g x(x)h*, that is, (fx(x)h*, rx(x)h*b -|S*(1 , 0 , ... 0 ) for some (3*> 0 , so that: f X(x)h* = - )3* < 0, rx (x)h* = 0 Since f°(x) h* = - 0* and С is a linearization, 3e > 0 Э f°(x + eh*) < f °(x), (x + eh*)Gf2, and rx(x)eh* = 0. Since r(x) = 0, and r is affine, the latter condition implies r(x + eh*) = 0. Hence x is not optimal, a contradiction, so that Cz and S are linearly separated. Hence 3XGRk+1, X/ 0, Э : < X ,gx(x)h>á 0, VhGC - Xo a 0, that is Xo s 0 Since 2. Theorem If x is an optimal solution of P3, A(ii) of (1.1.2) holds and r is affine, then 3XGRk+1, X/0, X°s0, scalars ц1 s 0, iG I0 (x )9 : к X°Vf°(x) + y X*V r‘ (x) + ^ AiIV fi(x) = 0 i=i iil 000 232 MAYNE Proof From Theorem 1.3.1 replacing С by С к =» Applying Farkas Lemma, 3 { n S 0 1 i e l 0(x)}3 к X°Vf°(x)+ У XVr^x) nVfHx) i= 1 i e l 0(x) The major task in the sequel is to remove the restriction 'r affine' in Theorems (1) and (2). This complicates the proof considerably, and use has to be made of a fixed-point theorem. 3. Theorem 1 If for som ee>0, x£R n л is a continuous map from Be(x) into Be(x), then у has a fixed point, i.e. Эх e Ве(х)Э 7 (x) = x 4. Theorem If x is an optimal solution of P3, С is a linearization of (Í2 -x), then 3X = (Xo, X1, ...Xk) e R k+1, X°g 0, Х /0 Э : Proof Firstly we show that C z and S are linearly separable. For if Cz and S are not linearly separable, then S lies in the interior of C2=> 3z°SS (z° = -j30 (l,û,...O),(3 0 > O )3 z ° 6 intCz , and 3 simplex £ = co{ 0 , z 1, z 2, ...z k+1} Э z° £ int £ and £ С Cz. Since Cz has dimension k+1, gx(x) has maximal rank. 3{hIec|i= 1,2,...к+1}Э 5. z‘ =gx(x)hi, i = 1, 2, ...k+1 Also because С is a linearization of {Q -x), z¡ and, hence h‘ , can be chosen so that 1 In the following, B£ (x) = {x | | x - x| s e } denotes a closed ball. IA EA -SMR-17/4 233 6 . co {0 ,h \ ...hk+1}CQ Now, 3r > 0 3 B r(z°) 6 Z , also В*, (tfz^ e E, Va e [0,1]. Consider the map y a : Bor (oz0) -> Rk + 1 defined by: 7. 7 a (z) = z +Q-Z0 - [g(x+H Z "1 z) - g(x)] where the ith column of H £R ,k+1)x(ktl* is h1, and the ith column of Z£R*kll*x(k + 1* is z1. (Z is invertible since Z is a simplex and Z = gx(x)H). Ifh = HZ_1 z, and z G Bar (a z °) С Z С Cz , then h £ (iî - x), and gx(x)h = z. Because gis continuously differentiable and ||z||sc, for some c<°o, Vz G Bar(oz°), it follows that 3a 0 G (0,1] Э II [g(x + H Z ' 1 z) - g(x)] - gx (x)HZ' 1 z II = II [g(x + HZ-1 z) - g(x)]- z|| S o r /2 , V z e B oit(a0 z). Applying this to (7) yields: II7° ( z ) -o° z° II Sa° r/2, Vx eBaor (a°z0), so that 7 maps Baor(oez 0) into Baor(o° z°). Since 7 “° is continuous, by Brouwer's fixed-point theorem, 3"z G Ba«r (a°z° ) Э : that is g(x + y) = g(x) + a° z° where ÿ = HZ_1 Ï 6 (i2 -5) Hence x Дх +ÿG U and f°(x) = f°(x) -a°¡3° , a°,0°> 0 r(x) = 0 contradicting the optimality of x. Hence Cz and S are linearly separated. The rest of the proof is now the same as in the proof of Theorem 1.3.1. 234 MAYNE 5. Theorem If x is the optimal solution of P3, and A(ii) of (1.1.2) holds, then 3 X € R k+1, Xo SO, X / 0, scalars ц1 SO, iG I 0 (x )9 : к i = l iel0(x) Proof The proof is the same as that for Theorem (1.3.2), using the strengthened result Theorem 1.3.4 in place of theorem 1.3.1. 6 . Corollary IfVrMx), i=l,2,._..k, Vf^x), ie i 0(x), are linearly independent, then X°< 0 . Proof Assume Xo = 0. Since Vr*(x), i= l,2 ,...k , Vf*(x), ieio(x) are linearly independent, 3h£Rn, h^ 0, Э <(угЧх),Ь>= 0, i= l,...k, к i=i i e l 0(x) => /и1 = 0, Vi e I 0(x) But 3 h 'e R n, h '/O , э<С Vrl (x) ,h ' У <0, i = 1, ...k, and this implies in turn thatXi= 0, i = l ,...k , that is, X= (Xo , X1, ...Xk) = 0, a contradiction. 2. ALGORITHM MODELS - CONVERGENCE Polak [1] gives three reasons for using simple algorithm models: i) classification of computational methods; ii) elucidation of essential features of algorithms guaranteeing convergence; iii) providing a simple procedure for obtaining 'implementable' algorithms from 'conceptual' prototypes. Each iteration of a conceptual algorithm may require an arbitrary number of arithmetical operations and function evaluations, while an implementable algorithm must require only a finite number. For example, many algorithms choose, at each iteration, a 'search direction' and then search along this direction; an algorithm which minimizes a function along this dir'ection would be conceptual. We consider algorithms for deter IAEA-SMR-17/4 235 mining points in fiCRn which are desirable. Conceptually, a point is desirable if it solves the optimization problem; more realistically, a point is desirable if it satisfies some necessary condition of optimality. The simplest algorithm for finding desirable points in a closed subset Cl of R n employ two functions, â : Í2 -* Г2 for generating new points and c :f 2 -»R for testing the desirability of a point. 1. Algorithm Model a:Q-»f2, c:fi-*R Step 0. Compute a x0e П. Set i = 0. Step 1. Set x i+1 =a(xj). Step 2. If c(x i+1) s c(xj) stop. Else set i = i + 1 and go to step 1 Under what conditions will this algorithm determine desirable points? The following theorem, due to Polak [1], shows that the essential condition is a kind of semi-continuity property of a. 2. Theorem Suppose that: (i) с is either continuous at all non-desirable points in Q, or else с is bounded from below iniî(3Ô6R эс(х) íc , V xeíí) (ii) Vxef2 which is not desirable, 3e>0, 6>0 (possibly depending on x) Э c(a(x'))-c(x')s- 6 < 0 , Vx'eB£(x)A{x'ef 2 Then, either the sequence constructed by algorithm (1) is finite, and its next to last point is desirable, or it is infinite, and every accumulation point of {x¡] is desirable. Comment There is an implicit test for desirability in algorithm (1). If c(a (x)) 2 c(x) then x is desirable. Proof If the sequence is finite, then, by step 4, Зк э с ( х к+1 ) 2 c(xk), so that xk is desirable. If the sequence is infinite and has an accumulation point x*, it has a subsequence converging to x* in U (that is 3K C Z э х (-* х * for ieK). Assume that x* is not desirable. Hence 3e>0, 6>0 and акеКэ : 3. x¡ e B£(x*), c(xi+1) - c(x¡) s - б , Visk, ieK. Hence, for any two consecutive points x ¡ , x i + j of the subsequence K, i i k, we have: 236 MAYNE С ( X j + j ) - c(Xj ) =[c(x¡+j) - c(x i+j.! )] + ... + [c(xi+1) - с (x¡)] S [c(xi+i) - c(x¡)] S - 6 Henee, {c(x¡)|ieK)} is not a Cauchy sequence, and cannot converge. But, since с is either continuous, or bounded from below and {c(x¡)|ieK} is monotonically decreasing for ieK, the sequence {c(x¡)|ieK} does converge, a contradiction. Hence x* is desirable. Comment Note that the theorem does not state that accumulation points exist, only that if they do exist they are desirable. However, if Q is compact, or if the set ¿t ^{xefi| c(x) s c(x0)} is compact (if x0 e<¿ so does x¡, ViêO) then accumulation points in Í2 do exist. An algorithm can not always be expressed in terms of a function a For example, once a search direction is chosen, the algorithm may select any one of many points satisfying some criterion. In such cases the function a should be replaced by a set-valued mapping А: Г2 -»2 n , where 2a is the set of all subsets of Q. 3. Algorithm Model A : Í2-» Ia , c:i2-»R Step 0. Compute a x0e Г2. S eti = 0. Step 1. Compute a point yeA(x¡). Setxi + 1 = y. Step 2. If c(xi + 1) S c(x¡) stop. Else set i= i+ l and go to step 1. 4. Theorem Suppose that: (i) с is either continuous or bounded from below in Г2. (ii) Vxef2, x not desirable, 3e>0, 6>0 Э: c(x") - c(x') S - 6 , Vx'eBe(x), Vx"eA(x') Then the next to last point of any finite sequence, and every accumulation point of any infinite sequence generated by the algorithm is desirable. These models, and this is one of their great virtues, can relatively easily be modified to cope with the approximations needed to make conceptual algorithms implementable. One method of modelling algorithms with approximations is to employ a set valued mapping A : R+ X Q - 2a, (e',x)-> yeA(e',x), where e 1 indicates the degree of approximation; as e'->0, A(e',-) tends, in some sense, to a 'conceptual' A satisfying Theorem 3. These comments are spelt out more precisely in Algorithm (5) and Theorem ( 6 ). IAEA-SMR-17/4 237 5. Algorithm Model A : R+ X ft - 2a , с : ft - R, e0 > 0 Step 0. Compute x0sQ. S eti= 0 . Step 1. Set € = € 0 . Step 2. Compute a y e A(e ,x¡). Step 3. If c(y) - c(x¡) s - e , set xi+1 = y, set i = i + 1 and go to step 1. Else set e = e / 2 and go to step 2 . Comment For simplicity the algorithm statement does not include a stop state ment; for a version which does, see Polak [1], 1.3.26. Hence the algo rithm either produces an infinite sequence {x¡}, or 'jams up' at some XjEft, cycling between steps 2 and 3, halving e at each cycle. 6 . Theorem Suppose that: (i) с is either continuous or bounded from below in ft. (ii) Vxeft, x non-desirable, 3e, 6 , 7 > 0 о 7. c(x") - c(x') S - 6 , Vx'eB£(x), Vx" e A(e', x'), V e'e[ 0 , 7 ] Then either the sequence {x 1} jams up at xs, where Xj is desirable (x is desirable if c(x') -c(x)iO , Vx'eA(O.x)), or the sequence is infinite, and all its accumulation points are desirable. Proof (i) First we show that the sequence cannotjamup at a non-desirable xs. For if so, the algorithm generates a sequence yj e A(e0/2 i, xs ) (from step 3), c(yj) - c(xs) > - e0/2 К However, 3 integer Ja e 0 /2 j s á and e0 / 2^ S 7 (where 6 = 6 (xs) > 0 and e = e (xs) >0 exist by assumption (ii)) Vj s J. Hence, from assumption (ii), c(yj) - c(xs) & - 6 s - eo/2 J, VjaJ, which contradicts the statement above that c ( y j)- c ( x s) > - e 0 / 2 J, VjêO. (ii) We next show that all accumulation points of an infinite sequence {x;} generated by the algorithm are desirable. For if x* is a non-desirable accumulation point of {Xj}, then 3KC{0,1, 2,...} э x x*, for isK , and 3 e, 6 , 7 > 0 satisfying (7) with x = x*. Hence 3Ji Э x¡eB f(x*), VieK, is Jx. A lso, 3 j 9 e 0 / 2 JS 6 , ande0 / 2 JS7 . From assumption (ii) c(x i+1) - c(x4) S - 6 s - e 0 / 2 J so that the test c(x i+1 ) - c(x¡) s - e in step 3 of the algorithm is satisfied with - e s - e 0 /2 J. Hence, for any two consecutive points Xj, x i+j , is in the convergent subsequence {Xj [ i eK} satisfy 238 MAYNE c(x i+j) - c(x¡) = [ c(xi+j - c (x i+j_j )] + ... tc(xi+1) - с(х4)] s-e0/2J < Hence, the sequence {c(x¡) | ie K } is not Cauchy, but from assumption (i), since {x¡ I i eK} converges, so does {с(х^ | i eK}, a contradiction. 3. UNCONSTRAINED OPTIMIZATION In this section we shall describe several algorithms for PI, and examine their convergence properties using the algorithm models of Section 2. Since many algorithms choose at each iteration a search direction s¡, and then minimize, or approximately minimize, f° along s ' (that is, minimize the function 0 :R +-*R defined by в(У) = f°(x¡ + Xs¡)) we commence by describing several procedures for approximately minimizing a function в : R+ -* R. 3.1. Algorithms for approximate minimization of в : R* ~*R 1. Golden Section Search Let в : R* -*R be convex. The golden-section algorithm consists of two parts. The first part determines an interval [а0 ,Ь 0] which brackets the minimum. The second part reduces the size of the interval bracketing the minimum to a pre-assigned value e. Both parts rely on the convexity of 6 . The first part follows. For a given p> 0, a sequence of points is calculated according to: x 0 = 0 xi+ i = x i + P until the first point x¡ is reached satisfying: e(xi+1) g e(xj) The minimum of в will lie in the interval I0 = ta0 ,b 0]A txj.j , x i+1] The second (golden-section part) is defined below: (VjgO, £j4(bj-aj), Ij Д [a j, bj]) 2. Golden Section Algorithm: [a0 ,b 0], e > 0 , Fj = (3 - »/5)/\Í2, F2 = 1 - Fx Step 0. Set i = 0. Step 1. IfijS e, set x = (a¿+bi)/2 and stop. Else proceed. Step 2. Set v¡ = a¿ tF j^ i, Wi = ai + F 2 £i. IAEA-SMR-17/4 239 Step 3. If 0(v1)<0(w1), set Ii+1â [ai+1 ,b i+1]= [aj.Wj]. If 0(v¡) S 0(wj), set Ii+iA [ai+1,b i+1] = [v¡ , b¡]. Set i = i + 1. Note that íj = F2 Í 0 • It can be shown thatif Ii+1 Д [ai+1, b i+1 ] = [a¡, w¡], then wi+1 = Vj and if II+i = [v¡,b¡], then v i+1= Wj, reducing the computation required. If x* denotes the solution to min (0(x)|xeR+), then |x-x*|se/2. Recognizing the use of the algorithms presented in this section in the sequel, we now define 0 : Rn X R+ -» R by: 3. 0(x, X) Д f°(x + Xh(x)) - f °(x) and a linear approximation 0 : Rn XR'^-'R to 0 by: 4. 0 (x, X) Д X <(vf °(x), h(x) У = Xv^0(x, 0) It is assumed that 5. Vx0(x,O) = 6 . Armijo's algorithm: ge(0,l), p>0 Step 0. S e ti= 0 , X0 = p. Step 1. If 0(x, X¡) i j6(x, X¡), stop. Else set Xi+i = (3X¡ and go to step 1. 7. Comment Figure 1 illustrates the situation. 8 . Theorem Let V f 0 and h be continuous, and 0(x, X) s j в(х, X) P X — ► 6(xrp)f 2 F IG .l. Illustration o f algorithm o f approximate minimization. ÉMx.p) 240 MAYNE Then, 3 e > 0, 6>0 (6 =7 / 2 ) э ; 0 (x ', X(x')) â -6 , V x 'e B ( (x) The proof is left to the reader as an exercise. 3.2. Algorithms for unconstrained minimization Let us now return to the problem PI : m in{f°(x) | x e Rn}. We assume that f° is continuously differentiable. The simplest algorithm is: 1. (Conceptual) Steepest Descent Algorithm Step 0. Select a x 0 6 Rn . S e ti= 0 . Step 1. Set h(xt) = - V f°(xi). If h(x¡) = 0 stop. Step 2. Compute smallest scalar X(x¡) which solves f °(x¡ + Xtxjíhíxj)) = min {f °(xj + Xh(x;) | X ë 0} Step 3. Set Xi+i= Xi + X(xj) h(xi), set i = i + 1 and go to step 1. 2. Theorem Let C(x0) Д{х| f°(x) S f°(x0)} be bounded. (i) If {xj} is finite, terminating at x k, then x k is desirable (Vf°(xk) = 0). (ii) If{x¡}is infinite, every accumulation point x* is desirable. Proof (iii) Let a:R n-*R be defined by step 3 of (1); let cAf °. с is continuous. Let в , в be defined by (4.1.3), (4.1.4). x* not desirable =»V^0(x*, 0) = <(Vf °(x*), h(x*) У = - Il V f °(x*) II 2 = - 7 < 0. Hence 3 X j> 0 э 0(x*, X) < - X7/2, VXe [0, X J so that X(x*) (see step 2 of (1)) satisfies 0(x*, X(x*)) a - X ^ /2 . Let ф : Rn -> R be defined by: Clearly, ф is continuous, and satisfies: ф(х*) = 0 (x*, X(x*))= -Xjy/2 Hence Эе > 0 э ф(х') s - Xj^/4, V x 'e B £(x*) But, from the definition of ф and X(.): е(х',Х (х 1))§ф(х')ё-Xív/4, Vx'eB£(x*) IAEA-SMR-17/4 241 But 0(x',X(x')) = f°(a(x')) - f°(x') Hence f° satisfies assumption (ii) of Theorem (2.2) (with 6 = X17 / 4 ), thus proving the result. Algorithm 1 is conceptual since each iteration involves an optimization problem, (step 2) requiring an infinite number of steps. One method of making use of the golden section search of (3.1.1), (3.1.2) to replace.step 2, together with a loop to reduce e. 3. (Golden section) Steepest descent: e 0 >O, P > 0 Step 0 . Select x0 e Rn . Set i = 0. Step 1 . Set e = e 0 . Step 2 . Let h(x¡) =-Vf 0(x¡). Ifh(Xj) = 0 stop. Step 3. Compute X (x¡ ) э | X(x¡) - X(x¡ )|s e/2, X(.) defined in step 2 of ( 1 ), using (4.1.1), (’4.1.2) on the function 6 (xi, •)• Step 4. If 6(xi,J.{xi)) â - e , set xi+1 =x¡ + X(xj)h(x¡), set i = i+land go to step 2 . Else set e = e/2 and go to step 3. Polak [1] shows that if f° is convex, C(x0) bounded, then any infinite sequence{x¡l generated by (3) satisfies f°(x{) -*■ min{f 0 (x) | x e Rn} . 4. (Armijo) Steepest descent: j3e(0,l), p>0. Step 0. Select xq e Rn, set i = 0. Step 1. Set h(x¿) = -V f°(x j). If h(x¿) = 0, stop. Step 2. Compute X(x¡) using A rm ijo's algorithm (4.1.6). Step 3. Set x i+1 = Xj + X(xi )h(xi). S e ti= i+ 1 , and go to step 1. That any accumulation point x* of an infinite sequence generated by (5) is desirable (h(x*) = 0) can be easily established, if C(x0) is bounded, using Theorem (3.1.8), identifying с with f° and defining a :R n->Rn by step 3. The convergence properties of algorithms (1), (3) and (4) are unaffected if h is defined by h(x) = -D (x)V f°(x) where D :R n->RnXm is positive definite and continuous and C(x0) is bounded. If D(x) = [f°x (x)]-1 242 MAYNE then we have Newton-Raphson-type algorithms, for which p should be set equal to 1 in (3). If f° is quadratic, the Newton-Raphson version of algorithm ( 1 ) converges in 1 iteration. 3.3. Conjugate direction and conjugate gradient algorithms To motivate the sequel we note that f° behaves like a quadratic function in the neighbourhood of any local (unconstrained) minimum x*, in the sense that if f 0 is twice differentiable: f °(x* + 6 x) = f °(x*) + -|<^6 x, f (x*) бх^ + о(||бх||2) Hence, if an algorithm has poor behaviour when applied to a quadratic f°, it will behave poorly when used for a general f°. In this connection we note that the steepest descent algorithm, when f° is quadratic, and positive definite, that is: 1. f°(x) = a+ <(b,xУ + -|^x,Cx)>where С is symmetric and positive definite, has the following property: 2 . f°(xj) - f°(x*) So' [f(x0 ) - f(x*)] where 3. a A l - (Xmin| Xmax) where Xm¡n and Xmax are the minimum and maximum eigenvalues of C lo'-'l as the 'condition number' (Xmin/Xmax)-> 0). Note, in passing, that if f° satisfies 1, f x(x)=C x + b, f xx (x) = С and x* = -С ЛЬ in the unique local minimum (if С is p.d.). Also note that f °(x + 6 x) = f°(x) + f x(x) ôx + f < 6 x , f xx(x)ôx> The Newton-Raphson algorithm satisfies xi+i =xi _ Х(х;Ш°х(х;)]л Vf°(x) = Xj - X(x1)C‘ 1 (Cxi + b) = - C '1b = x*, if X(x¡) = 1 and so behaves excellently when f° is quadratic. However, the computation of fxx is expensive. The algorithm described here tries to preserve the desirable 'second order' features of the Newton-Raphson algorithm at less expense. We assume, for sim plicity, in the sequel that f° is quadratic and p.d. 4. Conjugate direction algorithm This is of the form of the steepest descent algorithm (3.2.1) with the restriction X§ 0 removed from step 2 and step 1 replaced by: Step 1'. Set h(Xji = h¡ where {h-¡, i= 0,1, 2, ...n - 1} are С-conjugate, that is: 5. <Ь 4,С^> = 0 Vi¡Éj, i, j e0,l,2,...n - 1 which implies that h g.h j,...^ x are l.i. (linearly independent). Let, Vj = l,2 ,..n IAEA-SMR-17/4 243 j- 1 6. M j^{x|x = ^ «"¡hj, o¡ eR } j = o and 7. LjAMj+x0 8 . Fact, x solves min{f°(x) | x e Lj 1 <* V f °(x)eJ'Mj ( Proof Left as an exercise to the reader. 9. Theorem If {xi I i = l,2...n} is generated by the C.D. algorithm (and f° is quadratic and p.d.), then x¡ minimizes f° on L ¡ (and hence x n minimizes f° on Rn). Proof. xj minimizes f° on Li. Assume Xj minimizes f° on Lj, that is <(Vf0 (Xj), hj У = 0 for i= 0 , 1, ...j - 1. Now: V f°(x j+1 ) = Vf°(xj) + C(xj+1 - Xj) = V f °(xj ) + X jChj By virtue of the С -conjugate property: = 0 , for i = 0 , 1 , ...j - 1 Also since Xj +1 minimizes f 0 on { x j + Xh j | Xe R} < ^ ° (х ;+1 ),Ь ; > = 0 that is, Xj+i minimizes f 0 on L j+1, thus completing the proof by induction. The following algorithm automatically generates С-conjugate directions (giAAf°(Xj), hjAh(xj), VieZ): 10. Conjugate gradient algorithm Step 0. Select x0 eRn, seti=0, set h0 = g 0. Step 1. If gj = 0, stop. 244 MAYNE Step 2. Compute smallest scalar X¡ which solves f°(xj + X.hj) = m in{f°(x j +Xhj) | Xi 0 } Step 3. Set xi+1 = x t + X1hi . Set /3i+1 = Set h i+i= - gi+i + 01+1^ Set i = i + 1 and go to step 1. 11. Lemma Let f° be p.d. and quadratic, and assume that h0, hi , ... hj_x are С-conjugate. Then: (i) Proof (i) (h0 , hj, ...hj.j) are С-conjugate =>(Theorem 9) Xj minimizes f° on Lj=> < g j.hi> = i= •••]-!• But for i= 0 , 1 ... j - 1: < gj.h i> = < g j, - gi > = -< g j,g i> (ii) = < -g j ,C hi >, i = 0, 1, ... j - 2 But g¡ f 0 f 0 (prove) => Ch¡ = [gi+1 - g j /X ¡ =» (using i) as can be shown using (i), the definition of Xj, and the fact that Xj.j satisfies: 0 = =» Xj.i= 12. Theorem If f° is p.d. and quadratic, then algorithm (10) minimizes f° in at most n iterations. IAEA-SMR-17/4 245 Proof ho and hi are С-conjugate (prove). By induction, from Lemma 11, h0 , h i,...h n_i are С-conjugate. The result follows from Theorem 9. Polak-Ribiere have established [2] that a modified version of the algorithm (/3í+i = 3.4. Pseudo-Newton-Raphson algorithms There exists a class of algorithms, variously called pseudo- or quasi-Newton-Raphson, Variable Metric, Secant, which have the same or similar properties as the conjugate gradient algorithm when applied to quadratic functions, and in addition produce an estimate of the Hessian V 2f° or its inverse. The essential property of these methods is; that at iteration j, the estimate Hj of (V2f 0) ' 1 satisfies: 1 . Ax i = HjAg;, i= 0 , ...j - 1 where Дх;Дх;+1, Agj A gj+1- g¡, Vi. The search directions { hj} are generated according to: 2. hj = "Hj gj Since, if f° is quadratic and p.d., Ax¡ = С_1Д gi, i = 0, ...j-1, it can be seen that the restriction of Hj to CMj — the linear subspace spanned by {A go, •••Agj.i} or { CAx0, ...СДХ}_]} — is equal to the restriction of C" 1 to CMj (we shall say Hj = C ' 1 on CMj). Assume Hj is non-singular. Then, if hj e Mj from (2) and the non-singularity of H j, gj e CMj, so that H jg j= C '1 gj. Since x* = Xj -C ' 1g j, x* = xj + hj when X= 1 , Consequently, if ho , h ¡,...hn_i are l.i., Hn =C'1, and x n+1 = xn + Ahn = x* for A = 1 , where x* minimizes f° in Rn . Finally, it can be shown that if f 0 is quadratic and p.d., and Hi symmetric Vi, the algorithm generates С-conjugate search directions, and hence minimizes f° in Rn in at most n iterations. The most used algorithm for calculating {H¡} is the Davidson-Fletcher- Powell (DFP) formula [3] H 0 is chosen to be symmetric, p.d. (for example, H0 =1) and {H¡} calculated according to: 4 H. TT , A xiA xJT (Hj AgjKHjA gj )T J+1 J It can be shown that if f° is minimized exactly in each search direction, then, Vj ё 0 ( 1 ) HjAxj = 0 , i = 0 , 1 , 2 , . . . j - 1 (2) Hj+1 is p.d. symmetric. 246 MAYNE This formula seems to be robust. Convergence of the resultant algorithm when applied to f°, strictly convex and twice continuously differentiable, has been established by Powell (see Polak [1]). Another sim ilar formula is the rank 1 formula due to Sargent (see, e.g. Luenberger [3]): TT , (Axj-HjAgjKAxj-HjAgj)T 4. Jtl;+ + —T------T------J+1 ] < A Xj - H j A g j , A g j > if Axj/HjAgji otherwise Hj+1=Hj. Unlike the D.F.P. formula, this formula does not require exact minimizations of f° along search directions, and, therefore, lends itself to 'implementable' versions using, for example, A rm ijo's method. If we assume that Hj satisfies (1), that is, HjAg¡ = Ax¡, i= 0 , 1 ... j -1, then Hj+1 Agj = Ax¡, i = 0, 1, ... j. If the algorithm does minimize f° along each search direction, and f° is quadratic and p.d., convergence in at most n iterations can be established. 4. RATE OF CONVERGENCE Suppose {xi,] is a sequence converging to x*, and, for all non-negative integers k, let r)k denote || xk + 1- x* 11 / || xk- x*||p . {xkl is said to converge linearly if 3(3 £ (0, 1) э : 1 . l i m n ^ P к-* '°o i.e. if 3ki э ü xk+1 - x* Il s (3 || x k - x* || V kêkj. If {xk} converges linearly 3 с e R э l|xk II Sc (3k , Vk i klf i.e. {xk} converges at least as fast as a geometric progression. |3 is called the convergence ratio. If (3 = 0 the convergence is said to be superlinear. The order of convergence is the supremum of p e R э lim sup rjP <°°. ( lim supnE = lim s,, where k K k-»«K sk = sup{r)?l j2 k}). Convergence of any order greater than 1 is superlinear; however convergence of order 1 may be superlinear (if ¡3 = 0). As an example consider {xk = ak}, a e (0 ,l). r¡k = a, V keZ, and in fact { xk} converges linearly with convergence ratio ¡3= a. If xk = a2k, Vk e Z, a s (0,1 ), then rjk = a 2**, V k e Z , and convergence is superlinear ((3 = 0). rj2= 1, V k e Z, and thus { x k} has order of Convergence 2. These concepts enable us to compare the rate of convergence algorithms. Until now we have been m erely concerned whether they converge or not. To illustrate the type of calculations required we consider two simple conceptual algorithms, steepest descent and Newton- Raphson, i.e. we shall consider algorithms of the form: 2 . a(x) = x + A.(x) h(x) where A(x) is the minimizer of f°(x + Ah(x)), X eR +. As in section 3, 6 (x, A) denotes f°(x + Ah(x)) - f°(x). IAEA-SMR-17/4 247 The steepest descent algorithm For this algorithm h(x) = -V f°(x). We assume that f 0 is twice con tinuously differentiable, and strictly convex in the neighbourhood of x* which is assumed to be the limit point of a convergent sequence { x¡}. Hence 3e>0, m> 0, M>0, mSM Э: 3. V h e R , Vx e B £ (x*), and Эк1 эх( e B£ (x*), Viê Ц. As a consequence of (1) we have: 0(x, X) s 0 Vx, Х э х д + ХЬ(х) 6 Bf(x*). HenceViêk 4. etxj.Xixj))^ - II Vf°(xi )||2 /2M . But we also have, Vi È к: 5. f 0 (xj) -f°(x*) e [mllxj -x*||2 /2 , м|| X¡ - x* ||2 /2] and 6 . I Vf°(x¡) Il S m II Xj - x* Il so that f°(x i+i) - f°(x i) = б (x¡, X(x¡)) S -m 2 II x; - x* ||2 /2М S - (m /M )2 [f°(xi) - f°(x*)] Hence 7. f°(xi+1) - f°(x*) s [1 - (m/M)2 ][f 0(xj) - f°(x*)], which establishes the linear convergence of {f0() } with convergence ratio |3 = [1 - (m/M)2]. Convergence is possibly very slow as /3~* 1 ((m/Mj-’ O), since f 0 (xj) - f°(x*) S /31 [f0 (xkj ) - f°(x*)]/j3ki. Also Vi^k^ II x¡ - x *||2 s (2 /m) [f°(xj) - f°(x*)] S (2/m) [f°(xki ) - f°(x*)] jS1 / 3kl so that xj-^x* linearly with convergence ratio \[&. The above discussion can be generalized to deal with any algorithm of the type in (2 ), where h(x) satisfies, for some p e ( 0 ,l ] : 248 MAYNE 8 . V x e R n. (8 ) is clearly satisfied for the steepest descent algorithm (-h(x)=Vf°(x)). The Newton-Raphson algorithm As before, we assume that {x¡} converges to x*. Let H = V2 f 0 be the Hessian of f°. H and H' 1 are assumed to exist and be continuous. Hence 3 e , m , M e R+ э ; 9. I H(x) - H(y) (I ë M, ||нл (х)||ё1 / т Vx,y Be(x*). 3kj 3 x¡ e B6 (x*) Vis kj. For the Newton-Raphson algorithm, Vx e Rn : 10. a(x) = x - H_1 (x) g(x) and: 1 1 . A(x) = 1 where g:R n-,-Rn denotes the gradient Vf°. Hence, Vis 0, {x¡} satisfies: 12. g(x¡) +H(xj) [xi+1- x¡] = 0 Hence: H(xt)[xi+i- xt] = - g(xi)+H(x 1 _1)[xi - x i _1]+g(xi 4 ) so that: ‘ l + i - * ! = -НЛЦ) J [H(xul) - H(xul + s(x¡ - xul )]ds [xt - Xj.j] Hence, Vis kj. 13. II xi+i — x ¡ II á (M /2 m) II x¡ - Xi.! f ^ a ||x¡- x ul It follows from (13 ) that: lim II Xj + j - x* И / II x ; -■ x*x * I 2 = a so that the order of convergence is two (and superlinear). Superlinear convergence can also be established for the implementable version of this algorithm (employing Armijo), as well as for the conjugate gradient and quasi-Newton algorithms. IAEA-SMR-17/4 249 5. FEASIBLE DIRECTION ALGORITHMS FOR CONSTRAINED OPTIMIZATION PROBLEMS We consider here the problem: 1 . min{f °(x) I f 4 x) s 0 , i = l , 2 ,...m ) The feasible set Q is defined by 2 . £2 ={x| f 4 x) S 0 , i = l , 2 ,...m ] f i:R n->R, i = 0,l,...m , are assumed to be continuously differentiable. An algorithm is said to be a method of feasible directions, if at each non desirable point x 6 Q , it generates a new point x 'e { x + Xh(x) | X e R+} by choosing X = X(x) э x' = x + X(x) h(x) e Q and f°(x') < f°(x). Hence such a method can only be employed if £2 has an interior (or relative interior if Í2 is a subset of a linear manifold). 3. Definition. Vx0 ,x e £ 2 : <^(x0)^ {x e Œ I f°(x) s f°(x 0)} B’ (x)^ {х'еП|||х’ -x||áe} One of the simplest feasible direction algorithms is that due to Topkis and Veinott. Let S “= { h eR n| jh11 s i, i= 1, 2 ... n} , Jm^{ 1, 2, ...m }. J^= { 0 , 1 , 2 , ...m }. 4. Feasible directions algorithm 1. Step 0. Compute x 0e Í2, s e ti= 0 . Step 1. Set x = x¡ ; compute h(x) which solves: 6(x) = min m a x {(v f° (x ),h ); f i(x) + < Vf1 (x), h X i e Jm} h e s if e(x) = о stop. Step 2. Compute X(x), the smallest X which solves: f °(x + X(x) h(x)) = m in{f°(x + Xh(x)) | X eR + , x + Xh(x) e Q}. Step 3. Set x i + 1= x + X(x) h(x). S eti = i + 1. Go to Step 1. 5. Comment. h(x) can be determined in Step 1 by solving the following linear program: 250 MAYNE 6 . 0(x) = m in{a ! - CT+ i = l , 2 ...m ; I h 1 1 â 1 , i = l , 2 ,...n } 7. Theorem. Suppose that ¿(xo) is compact and has an interior. Let {xj} be generated by algorithm (4). Then either {x¡} is finite, ending at x k+i= x k and 0 (xk) = 0 , or it is infinite, with every accumulation point x* of {x¡} satisfying 0 (x*) = 0 . Proof. The finite case is trivially true. Since h(x) is not necessarily unique suppose algorithm (4) defines a map A : Rn -*■ 2 Rn (x' e A(x) if x' = x + X(x) h(x), for some h(x) solving ( 6 ). We note that S is compact, 0 and f 1, i= 0, 1, ...m are continuous and^(x0) is compact. Assume x is not desirable, i.e. 0(x) = - v< 0. The following results can then be established: 8 . 3ej > 0 э 0(x') § - v/2, Vx'e B£j(x) 9. X<(vf°(x'), h(x') У & - Xv/2 and f‘(x) + Also e (O.e^ andaX1>09 10. I < V f‘ (x' + X h ),h > - Vx'eB£(x), VXelO.Xj], VheS. Hence, using the mean-value theorem: 11. f ° ( x '+ Xh(x')) - f°(x') S - Xv/4, and f*(x'+ Xh(x')) S - Xv/4 VieJ^, Vx' e B£ (x), VX e [0, XJ Since x' e B6 (x) =» f1 ( x ' ) SO, V ie Jm , it follows from (10) that X(x') (defined in Step 2 of (4)) satisfies: 1 2 . f°(x") - f°(x ' ) § - 6 13. f ¡(x")sO , Vi e Jm Vx' e B£(x), 6= - Xjv/4, Vx" e A(x'). Д n If we define с = f u, we see that с is continuous, and that c, A satisfy the hypotheses of Theorem (2.2). Hence, any accumulation point x* of an infinite sequence generated by (4) is desirable (0(x*) =0). Note, from Section l , 6 (x*) = 0 is a necessary condition of optimality of x*. Apart from the conceptual nature of Step 2 — which can be replaced by, e.g., Armijo's algorithm — the above algorithm suffers from the disadvantage of taking into account all the constraints all the time, thus increasing the dimensionality of the linear program ( 6 ). One might IAEA-SMR-17/4 251 consider performing Step 1 (i.e. solving ( 6 )) considering only the active constraints — those for which P{x¡) = 0. However, this strategy can lead to jamming, as shown by the example given by Wolfe [6 ]. Essentially, removal of ccinstraint f] from the linear problem because f-Kx;) / 0 , may result in «(vfkxj), h(xj)^ being positive, and, if |^(х()| is small, the resultant step size A(x¡) may become arbitrarily small, leading to jamming — convergence of {x j to x* э 0(x*) / 0. Zoutendijk overcame this phenomenon by taking into account e-active constraints in choosing the search direction h(x). 14. Definition. The e-active constraint set at x is: I£(x) = { i e l J f ^ È -e } and I°(x) = {0} U I (x) e e 15. Ve e R+, 0£ : Rn -» R and h£ : Rn -*Rn are defined by: 0£ (x) = min max {< Vf‘ (x), h>} = max {VfMx), hf (x)} heSisI£°(x) iel°(x) Clearly, 0 as defined in Step 1 of (4), satisfies 0 = в0 , and h 0 = h. The minimization problem in (15) can be easily written in the form of the linear program ( 6 ). The following results can be established fairly simply: 16. 0£ (x) S 0, Vx e i l , Ve e R+ 17. I£(x)CI£, (x), Vxeft, V e,e'e R+ э e < e' 18. 0£( x ) s 0 £, (x), V x e ft, Ve , e ' e R+ э e < e 1 19. Ve e R+, V x s fi, 3ex >0 э I£(x') С I£(x), Vx'eBJlx) 20. Ve e R+, V xef2, 3e1 > 0 э l£,(x) = Ie.(x), V e'etO .ej] The following algorithm is a modification by Polak [1] to an algorithm due to Zoutendijk. 21. Feasible directions algorithm 2: e 0 > 0 , p > 0 Step 0. Compute ax0e £2. Set i = 0. Step 1. Set e = e о . Step 2. Compute a h£(xj) which solves: 0e(xi) = min max { Step 3. If (x) > - e , set e = e /2 and go to Step 2. 252 MAYNE Step 4. (Armijo) Compute the smallest integer к э f°(x ¡ +ph£(x)/2k) - f°(x.) S [0£ (Xj)p/2k]/2 fl(xj + phe(Xj)/2k) á 0 , V ie Jm Step 5. Set A.(x¡) = (p/2 k) Set xi+ j =x¡ + A(xj) h£ (x¡) Set i = i+ 1 Go to Step 1. 22. Comment. For the purpose of analysis, no stop statement has been included. Hence, the algorithm either jams up at x s, cycling between Steps 2 and 3, or produces an infinite sequence {x¡}. 23. Lemma. V non-desirable x e Q , (0O (x) / 0) 3 j(x ), e (x) = e 0 /2 j(x^ >0, э ct£ (x ) s -e (x ). Proof. Эех э I£(x) = I£l(x) V e e [O.ejJ. Hence, i f 6 0 (x)<0, then 0£ (x) = e0 (x) < 0, Ve e [0 ,e 1] Then j(x) = smallest integer эе(х ) = e0 /2j(x) satisfies e (x )S e x and e ( x ) á - 0 o(x) so that 0£ (x) = 0O (x) s - e(x), VeelO.ej] Comment. Hence the map A : Q -* 2a , { x e Г21 0o(x) = 0} is well defined by A(x) = {x'|x' = x + X(x)h£(x) (x)}, x^ D A(x) = x, if x s D. 24. Theorem Let{x¡} denote a sequence produced by (21). (a) If { Xj} jams up at x s, then 0O (xs) = 0. (b) If {Xj} is infinite then every accumulate point x* satisfies 6 0 (x*) = 0. IAEA-SMR-17/4 253 Outline of Proof (a) is established in (2 3). (b) Suppose that x e i î is non-desirable (0o (x) 0£ (x) (x) S - e(x) < 0. From (19) Эе1 > 0 э: 25. Ie(x) (x ')C I £(x) (x), W e B £i (x) Let Ф : Rn -* R be defined by: 26. ф(х') = min max Vf'(x'), h h es iel£»(x)U) Clearly ф is continuous, iMx) = 0c(x) (x) and 0£ (x) (x ')é ф(х'). Hence 3 e2 e [0 .e 1] э : 27. 0£ (x )/ 2 (x ') s et (X) (x1) s Ф (x') s - e(x)/2 , Vx' g B£ (x) (27) implies: 28. e£(x') (X') = 0 £(x) (x>/2, V x 'e B £¡(x) The rest of the proof proceeds as in the proof of 7, except for a minor modification due to use of the Arm ijo rule in Step 4. For simplicity, we have restricted ourselves to the more direct feasible direction algorithms. For a discussion of a dual method of feasible directions, due to Pironneau and Polak, see Ref.[l], 6 . PENALTY AND BARRIER FUNCTION METHODS The basic idea behind penalty function methods is to convert constrained optimization problems into unconstrained problems, by discarding the constraints, and adding to f° a penalty function which penalizes constraint deviation. More precisely, the problem: P l. min {f °(x) I x e Г2} where Q С Rn is closed (and may even have no interior if £7 represents equality constraints) is replaced by the following sequence of problems: P2. m in{f °(x) + Pj(x) I x e Rn} , i = 0, 1, 2, 3 ,... whose solutions x¡ -* x, the solution of (1), where Vie Z , p¡(x) = 0 if and only if x e Q. Note that x¡ is not necessarily feasible, Vi e z. 254 MAYNE In contrast, the Barrier Function method also replaces PI by P2 but now, Vi eZ , Pi is definedoonly on the interior of Œ (assumed to be non empty) and p.(x)sO , V xef2 and p¡(x) -»oo as x->6(f2), the boundary of Í2. 6.1. (Exterior) penalty function methods 1. Example If Í2 ={x| f'(x) SO, i e Jm} , then a useful sequence of penalty functions is given by: m 2. j=i Note, for each x, pj(x)>p¡(x) if j > i. In fact the defining properties of a sequence { p¡} of penalty functions are: 3. For i = 0, 1 , 2 , . . . : р ; :11п-*11 continuous p;(x) = 0, V x e f! Pj(x) > 0 , Vx$ Í2 Pi+i(x)> Pi(x) V xffi p¡(x) -» oo as i-oo, Vx $ £ 2 The sequence { Pi) defined in (2) satisfies (3) i f { f J|je Jm} are continuous. Let 4. Lemma ф0 (x0) S (^(Xj) S ... f °(x*) Proof Since Pi(x) s pi+1(x), Vx s Rn фi(xi)s (í>i(xi+1) S c/)i+i(xi+1)s ф1+1 (х*), Vi Also since x* e П, pt(x*) = 0, and 5. Theorem Any accumulation point of{xj} is a solution of PI. Proof (for the case when{p¡} satisfy (2 )). IAEA-SMR-17/4 255 Let{x¡] now denote a subsequence, converging to x. Then f °(xi) -* f °(x). Let f°(x*) denote the minimum value of f° on Я (x* is a minimizer of f° on Г2. By Lemma (4) { f O ^ W 0© Ф^Х^ -* Ф* S f°(x) ф ^ ) = f °(x¡) + i P(x¡) Hence, iP (x ¡)- ф* - f°(x) S 0 Since i-*oo, and P ( x ) й 0, Vx e Rn, this implies that P(xi)->0 =*-P(x) = 0 => x e f i , that is, x is feasible. Also: f°(x) = lim f °(xj) S f°(x*) Hence, x is optimal for PI. 6.2. (Interior) barrier function methods With the barrier function method, U must have an interior — problems with equality constraints cannot be handled. More precisely, Í2 must be robust, that is, that Í2 = Q (the closure of the interior of Cl is equal to Г2). This rules out equality constraints, 'whiskers', isolated points, etc. Barrier function methods establish a barrier which prevents points generated by the algorithm leaving Q: 1. Example If £2 = {x| f ‘ (x) i 0, ie J m}, then a useful sequence of penalty functions is given by: 2. p¡(x) = P(x)/i= ^ (- l/fJ(x)) / i j = i The defining properties of a sequence {pj} of barrier functions are: o 3. 0 О p¡(x)-*0 asi-^oo, Vxefi, VisZ. If x j -* x* e 6 (Í2), p.(Xj)->-oo as Vi As before, VieZ, let ф ! ^ ° + P i , let iq denote a minimizer of ф4 on Rn, and x* a minimizer of f° oij ii (because of the nature of p ¡ and ф ^ X j is also the minimizer of ф1 onfi.) It can be shown that: 4. ♦0 ( î 0 )H 1(x1)ê ... ф.(х.) s ф. + 1(х.+1) ... ëf°(x*) 256 MAYNE 5. Theorem Any accumulation point x of{x ¡} is a solution of PI. There also exist mixed penalty-barrier function methods. Penalty methods may be started at non-feasible x0, and may be used for problems with equality and inequality constraints. The sequence! x¡} is, on the other hand, not necessarily feasible. Barrier methods, which require that xq e Í2, cannot be used for problems with equality constraints but produce feasible sequences. Both are conceptual, requiring minimiza tion, Vie Z, of Ф1. However, implementable versions have been developed (see Polak 11 ]). Both methods have the disadvantage that ¡ becomes 'nasty' as i^a> in the sense that its Hessian becom es extremely ill- conditioned. For an interesting elaboration of this theme see Luenberger [3]. 7. DUALITY Consider the primal problem: P : min { f°(x) ] x e Q ] where Q Д{ x e Rn 1 f(x) § 0}, which we can imbed in the family of problems (V y e R mT: P(y): min {f°(x) ¡ x e £2(y)} where Q{y) Д{ x e Rn |f(x)sy} Let ГД {y e R m |n(y) ф 0} Г С Rm is convex provided f:R n->Rm is convex. F or, if yi, уг e Г, then fi(yi) and fi(y2)are not empty. If X]S fi(yi) and х 2 e ^ (уг) and 9 e [0 , 1 ], then: ffSxj + U - 0 )x2) = 0 f(x]) + (1 - 0 ) f(x2) S 0 yi + (1 - 0 )y2 “►0 XJ + (1 - 0 )x2 e fi( 0 yj + ( l - 0 )y 2 ) / 0 =*0yi+(l - 0)y2 e Г We define и : Г -» R by: 1. Def. u(y) Д inf{f°(x)|xe Г2(у)} and the subset A, В of Rm+1 by: lAEA-SMR-17/4 257 2. Def. А Д {(y0 ,y) e RXRm) | 3x e Rn э y0 ë f °(x), y S f(x)} = region above the graph of П 3. Def. y° A u(0) = inf{ f °(x) | x e Г2} 4. Def. В Д{(у°,у) eRm+1 |y°g y°, ySO} These definitions are illustrated in Fig.2; note, if x is a solution of P, f°(x) = y°. X 0 у FIG.2. Definitions in the duality problem. 5. Lemma (a) и is non-increasing (yj S y2 =*■ u(y£) ë u(y2)) (b) If f°, f are convex functions, и is a convex function and A is a convex set. The Lagrangian L : R n X R m -*■ R is defined by: 6 . Def. L(x, X) A f °(x) + ^A, f(x) )> Associated with L is the unconstrained problem: Px : min{ L(x, A) | x e R n} The dual function ф : R m - * R is defined by: 7. Def. (А) Д inf{ L(x, A) j x e Rn} and the dual problem is: DP. max{ ф(А) I A ê 0} 8 . Def. ф A sup{ ф(A) | Хё 0} Lemma 9 and 10 are true, whether or not , f0, f are convex, 258 MAYNE 9. Lemma ф : Rm+ R U { - 0 0 } is a concave function (R m+ Д {X e R m | Хё 0}) 10. Lemma (weak duality) If x s Í2, Xs 0, then f°(x)ë u(0)ё $ ё ф(Х) Proof f(x) s 0 , Xs 0 => <^X, f(x) У s 0 , so f °(x) ë L(x, X), V x e Q , Хё 0 Hence f°(x) ё inf (f°(x) I x e Çi] = u(0 ) й inf {L(x, X) I x s Q} ê inf{L(x, X) I x s Rn} = ф(Х) that is f°(x ) 2 и(0)гф(Х), Vx e Í2, X ё 0 Taking infimum w.r.t. Хё 0 yields the desired result. The basic problem in duality theory is to determine conditions under which the solution to the primal and dual problems are equal, that is to(0 ) =ф. 11. Theorem (sufficiency) Let (x, Я) satisfy the following optimality conditions: ( 1 ) x is a solution of P^ (2) x e ÍÍ, Хё 0 (3) У = 0 if fHx)<0 (i.e. Then: (i) x is a solution of P (to(0) = f°(x)) (ii) X is a solution of DP ($ = ф(£)) (iii) u(0 ) = ф (f°(x) = ф(Я)). Proof If хе£2=П(0), then •(X, f(x) У § 0, and: f°(x) ё f°(x) + й inf{L(x, X) I x e Rn} = L(x, X) (by (1)) = f°(x) (by (3)) i.e. x is a solution of P, and f°(x) = u(0). Also ф(Х) = L(x,X) = f°(x) (by (3)) = u(0) =>(by weak duality) u(0 ) = ф = ф(Я) i.e. X is a solution of DP. 12. Theorem (strong duality) О Let f°, f be differentiable convex functions, convex, ОеГ (i.e. 3 x e Rn э f(x) < 0), u(0)> -oo), then 3 an optimal solution Я to DP and и(0 ) = ф(Х) (If x is an optimal solution to PI, u(0) =f°(x).) Also, if X is a solution for DP, then x is a solution for P <=* (x, X) satisfy the optimality conditions of Theorem 11. Hence, the solution x to PI, can be determined by finding x: Rm+^Rn x(X) э Rn minimizes L(x,X), and then finding X* which maximizes ф(Х) = L(x(X), X) in Rm+. x = x(X*). However, this algorithm requires the (global) convexity of £2, f° and f. It is possible, though, to obtain similar results of a local nature (local duality). Under suitable conditions (including f°, f twice continuously differentiable) if PI has_a local minimum at x, with associated Lagrange multiplier X, and if V 2L(x,X) is positive definite (local convexity) then Я is local maximum of the unconstrained dual problem DP, and f °(x) = ф(£). The above discussion holds, with minor modifications, for the case when Í2 = {x e Rn I r(x) = 0}. For this case, observe that the two problems: PI. min{f°(x) I r(x) = 0} P2. min{f°(x) + с II r(x) ||2 I r(x) = 0} are equivalent, for any с > 0. A lso, that for с large enough: V2 [L(x,X ) +c II r(x) ||2] is positive definite. It can be shown that for every compact subset of Rn , there exists a c< « such that if x is a local minimum of Фс (x) = L (x , X(x)) + с II r(x) ||2 260 MAYNE where Mx) = (gx(x)gj(x ) ) - 1 gx(x)Vf°(x), then it is a local minimum for PI, so that algorithms can be constructed with a rule for automatically increasing c, producing sequences whose accumulation points satisfy necessary conditions of optimality of the original problem. ACKNOWLEDGEMENTS These notes rely, in the main, on Polak [1], particularly in the exposition of the simple but powerful algorithm models of Chapter 2, and their application for establishing convergence of algorithms for unconstrained optimization (Chapter 3) and of algorithms of the feasible-directions type for constrained optimization. The discussion of conjugate-gradient and secant-type algorithms draws on the lecture notes of a colleague, Dr. J.C. Allwright. The discussion of conditions of optimality is based on Ref.[4], and the regrettably short treatment of duality is based on the treatment by Varaiya in R ef.[5]. REFERENCES [ 1] POLAK, E., Computational Methods in Optimisation, Academic Press (1971). [2] POLAK, E., RIB1ERE, G., Note sur la convergence de méthodes de directions conjugues, Rev. Fr. Inform. Rech. Operation <16-R1 ) (1969) 35-43. [3] LUENBERGER, D.G., introduction to Linear and Nonlinear Programming, Addison-Wesley (1973). [4] CANNON, M.D., CULLUM, C.D., POLAK, E., Theory of Optimal Control and Mathematical Programming, McGraw Hill (1970). [5] VARAIYA, P.P., Notes on Optimization, Van Nostrand; REINHOLD: Notes on System Sciences (1972). [ 6] WOLFE, P., "On the Convergence of Gradient Methods under Constraints", IBM Research Report, R.C. 1752, Yorktown Heights, New York (1967), IAEA-SMR-17/37 REACHABILITY OF SETS AND TIME- AND NORM-MINIMAL CONTROLS A. MARZOLLO Electrical Engineering Department, University of Trieste and International Centre for Mechanical Sciences, Udine Italy Abstract REACHABILITY OF SETS AND TIME- AND NORM- MINIMAL CONTROLS. Standard continuous linear control systems as well as discrete time systems with "controller'' and "anti-controller” representing disturbances are treated. INTRODUCTION This paper is divided into three parts; the first two parts concern standard continuous linear control systems, whereas the third part deals with discrete time systems whose state evolution is influenced both by a "controller" and an "anti-controller", which may represent disturbances (worst-case approach). In Part I, functional-analysis methods are used to give necessary and sufficient conditions for the reachability of a given convex compact set in state space with norm-bounded controls, for different types of norms. In Part II, which again uses functional-analysis methods, these conditions are applied to obtaining both time-minimal and norm-minimal open loop controls, and a special case of Pontryagin's maximum principle is derived. These results are illustrated by an exercise at the end of Part II. Part III deals with the problem of finding the set in state space starting from which an initial state can be steered into a given target set, in the presence of an "anticontroller"; both the cases of bounded and unbounded controls and anticontrols are considered, and interesting relations are shown among possible situations depending on different information structures available to the controller. Part I and II of these notes are extensions of ideas developed in Ref.Il]. PART I. CONTROLLABILITY WITH RESPECT TO GIVEN SETS, •TIME INTERVAL AND BOUNDS ON THE NORM OF THE CONTROL Let us consider a linear control system described by a linear differential equation of the type x = A(t) x(t) + B(t)u(t) 1(1) 261 262 MARZOLLO where the state vector x(t) is in Rn, the control vector u(t) is in R m, and A(t), B(t) are matrices of appropriate dimensions." Under the usual hypo theses on the functions in A(t), B(t), and on u(t), if x(t 0) = x 0 , the system 1 (1 ) has a unique solution 4 v u(t) = V (t)x 0 + J w ( t ï , r)u(r) dr, t ê t 0 *0 where V(t) is the solution of the matrix equation V(t) = A(t)V(t). V(t 0) = I and W ^ , r) = VUJV'MrJBtT) Let us equip u with a norm over the interval K = [ t o .t j , which, for the moment, will be an L 2 2 norm, for the sake of simplicity: Çf IIu(т) II dr^) , I|u(t)|| = ^^|u¡(T)|2^) (later on, we shall extend the considerations to more general Lr>p norms, 1 s r s » , 1 < p § °°). Let Up be the class of admissible controls: Up = { u : || u || § p, p ê 0} The problem we shall now consider is the following: Problem 1,1 Given a non-empty convex compact set A in Rn ; does an admissible control exist which "steers" x 0 at time t 0 into A at time tj? In other words, does an u £ Up exist such that Vu(t1) G A, or is the system 1(1) "controllable with respect to x 0, A, Up, K? To solve Problem 1,1 let us define the linear operator A from Up into Rn: ti Au = J W(t1( T)u(r)dr Eo and the "reachable set" R at time t x R = { x :x = V(t1 )x 0 +Au, u G Up} 1(2) Problem 1,1 clearly has a solution if and only if R П A f 0 1(3) IAEA "SM R-17/37 263 The most natural way of verifying whether 1(3) holds or not is to apply the separation theorem for convex compact sets, or, more precisely, its following corollary Corollary 1,1 Two convex compact sets A and R of Rn have a non-empty intersection if and only if min From the convexity of Up, the convexity of AUp, hence of R, follows trivially. Its compactness may be obtained from classical theorem of functional analysis (see, e.g. R ef.[2]) by using the (weak) compactness of Up and the (weak) continuity of Л. Using Corollary 1,1 and recalling 1(2), we see that Problem 1,1 has a solution if and only if min ( <^x, VftjjXo У - max ■(x.Au) 1 S max(x,a), Vx 6 Rn ||u||sp a e -A Defining A* to be the adjoint operator of A, which takes Rn into U*, we have (x,V(t1)xoy - p Цл*хЦ § тах^х.аУ, Vx G Rn 1(4) a e A If A is the Euclidean sphere around x, of radius e: A = {x : x = xx +y, ¡f y U s e } 1(4) becomes < x,V (t 1)x0 - x : > s p||A*x|| + e||x||, V xG R " which is clearly equivalent to <(x, Vft^Xo У S p ||л*х ü + €, V x G R n, If x D = 1 1(5) Representing ||л*х|| and noticing that, because of the arbitrariness of x, the first term of 1(5) may be substituted by its absolute value, we have the following 264 MARZOLLO Theorem I, 1 The system 1(1) is "controllable with respect to x 0, A = {x, x = x x +y. Il y II S e} , Up, К " if and only if 1 /2 I <х » V (t Jxq - Xj, > I S p Ç j ||w*(t1> t )x ||2 dT^) +€, Vx, Il x II = 1 1(6) £o If e = 0, i.e. if A = xx, condition 1(6) may be easily verified by using standard techniques of linear algebra. Indeed, defining the vector Z of components zj as follows: z = V (t!)x 0 - xx and taking the square of both terms of 1(6 ), this inequality is equivalent to (х,г)(г,х) S p 2 W(t1( t) W *(t1( r)drx^>, Vx 6 Rn 1(7) to and defining the nX n matrix Z = z) <(z with elements z¡, z j , the last inequality is equivalent to the following one: < x,Zx> S p2 Y(t1 )x, V x 6 R" where kl Y(tx) = J W(tx, r) W *(t¿, t) d i 1(8 ) is the usual "controllability matrix" of system 1(1). Defining the nXn matrix С(^): C(tx ) = Y(t x) - - ^ - Z from 1(7) we have the following Corollary I, 2 System 1(1) is "controllable with respect to x0, x1( Up, K" if and only if the matrix C(t J is positive sem i-definite. The previous condition depends naturally on the system (through Y(ti)), on x 0, Xj (through Z), on p and on К (through Y itj)). It is easy to verify that letting x 0, x 1( p, К be arbitrary, we have the usual controllability condition: Condition 1,1 The state of the system 1(1) may be carried from any x 0 at time to into any x-l at time tx by controls u of appropriately large norm if and only if the matrix Y(ti) defined in 1(8) is positive definite. IAEA-SMR-17/37 265 Since the sufficiency is obvious, let us verify the necessity. Let С(^) be positive semi-definite, i.e. let 1(7) hold with arbitrary x0, x£ and appropriately large p. Since 'I ( x , Y(tx )x)> = У^< W*{t1 , t)x, W * ^ , r)x^> d to ti = J' IW*(t 1( r)x||2 dr ê 0, Vx € R Y(ti) is, at least, positive semi-definite. If it were singular, i.e. for some x' f 0, Y(t)x' = 0, from 1(7) we would have (<(x', z > ) 2 S 0, that is <^x', z У - 0 which is impossible for arbitrary x0, x£, hence for arbitrary z. Until now, we considered Up equipped with an L 22 _norm. The previous results, appropriately adapted, would hold for more general norms, i.e. for u G 4 p : IIu llr. p = (f l|u(T)||rP d x ) 1 < p s 00 where i/i i ( t ) Hr = (X 1м т> г) The admissible class of controls would be U?, p = {u : Il u II r> p § p} 1(9) Instead of Theorem 1,1 we would then have Theorem 1,11 The system 1(1) is "controllable with respect to x0, A, UfiP , Кц if and only if / p \!/q | < x ,x !-Vít^Xg)! S pl^J ||w*(t1( t)x||s d tJ + e, Vx £ Rn 1(10) where F or r / 2 , pf2, the verification of condition 1(9) may not be reduced to the previous simple algebraic condition of Corollary 1,2 but the following inequality should be verified 266 MARZOLLO 1/q max which implies a maximization over the unit sphere ||x|| = 1 , (or over the unit convex ball x S 1 ). PART II. TIME-AND NORM-MINIMAL CONTROLS In this part, we shall heavily rely on the results of Part I concerning the reachability of a given point Xj at time tj from x 0 at time t 0 under controls u belonging to an admissible class Up p defined in 1(9), or (x0, x ¡, Uj p , К) controllability, i.e. we shall use 1(10) which we rewrite for con venience in the case e = 0 : / p \i/q K x.xj-Vitjjx,,)!* p ( j ||w*(tx , r)x||, drj , V x € R n 11(1) We shall obtain results concerning the existence and the form of time optimal controls as well as of norm optimal controls, and shall derive a special case of Pontryagin's maximum principle. Throughout this chapter, we shall suppose x x / У (^ )х 0, that is the function u = 0 is not supposed to transfer x0 into Xj. Let us first consider time optimal controls. Theorem II, 1 If the system 1(1) is controllable relative to points x 0 and x 1( time interval K= [t0, t j and the class uf,p of admissible controls, then there exists a least interval К = [t0, t], (t0< t a tj) in which it is controllable. In other words, if a system is (x0, xx, Uf* , K) controllable, then there exists a time optimal control. Proof Let us consider the set H = {t, t> t0} over which the system is (x0, Xj, uf p , K) controllable. This set is clearly non-empty. Let us define "t = inf H We need to prove that the system 1(1) is controllable relative to x 0, xx, Up p and К = [t0 ,t], i.e. the interval К is the minimal interval over which the system 1(1 ) is controllable, which is equivalent to saying that t belongs to H. Indeed, suppose that t does not belong to H. Then, by theorem 1,1' there would exist a vector x in Rn such that IAEA-SMR-17/37 267 By the definition of t, there exists a sequence {tn} with lim tn - t and such n - * “ that for each interval Kn = [t0> tn] the system is controllable; and, therefore, 'n i/q | y(tn) s p6 (tn) with obvious meaning of 7 (tn) and 5(tn). Since y(t) and 6 (t) are continuous functions, we have lim 7 (tn ) = -y(t) = 7 = |<х,хх - V (t)x0> | П -*■ °c _ t ^ limpó(tn) = p6 (T) = p6 = p^J' ¡W*(t, t)x ||s4 dr^) t. From inequality 11(3), lim 7 (tn) S p lim 6 (tn) n-*«° n-*-» that is y S p 6 which would contradict 11(2 ). Theorem II, 2 If К = [t0 ,t] is the minimal time interval over which the system 1(1) can be transferred from x0 to xx with an admissible control belonging to Uf p , then there exists a vector x' j 0, x' G R n such that | Let the sequence { tn} converge to t, with tn < t . F or each tn, there exists a vector x n such that , pn \ i/q | The inequality 11(5) clearly holds if we replace xn by Xx}, = xn, where ||x'n || = 1. In this way, all x'n belong to the compact unit sphere of Rn, and, therefore, from the sequence {x'n} it is possible to extract a* convergent subsequence {x'nk} such that lim x},k = x ', with ||x' || =1. Since both terms of inequality 11(5) (with x^ instead of xn ) depend continuously on tn and on x'n, we have lim | 4 " x’ and tn 1/q ' 1Ím ( У HW*(tV T)X'nk ^ dT) II W*^* nl< 2 ‘о 4 " 1 Recalling inequality 11(5), we have / г \1/q I Since the system is (x0, x lf Ufp, K) controllable, hence also 11(1) holds for x = x ', the theorem is proved. Let us now turn to the problem of the existence of the so called "minimal norm" control: given the system 1(1), which is supposed to be (x0, x x, Uj p, K) controllable, does a control u exist such that ||u||r p = m in ||u||r and which transfers x 0 to x x, within K? ueUr, p ' The answer to this question is "y e s", and the control u is called "minimal-norm" control. We are going to prove this fact in the following Theorem II, 3 _ Let the system 1(1) be (x0, x : , u f p , К = [t-0, t x] ) controllable and let U С u/¡p be the set of functions u which effect the transfer from x 0 to x £ at time tj , i.e. such that ti J W (t1 ,v) u(r) dT = Xj -V (t 1 ) x 0 со Then there exists a function û such that l|û||r,p = in i l|u||r,p = p 11(6) U€U Proof Let us define p to be the inf_ ||u||r . There exists, therefore, a u e Ü ,P sequence {pn}, with pn > p and such that lim pn = p. For each pn the system n —»«. IAEA-SMR-17/3 7 269 T Pn is (x0 , Xj, Ur>p, K) controllable; therefore , 1//ч | This inequality is conserved as n tends to infinity, so we have 11 1/q | and, by Theorem 1,1', there exists a control function û 6 U with ||û||r p S p. Since û € U, and inf_||u[|r p = p , ||u(|r must be equal to p, and the infimum u e и in 11(1) is assumed. We now want to give an expression for p as a function of the parameters of the system 1(1), and of x 0, xi , t0 and tj. This expression is given in the following Theorem II,4 Using the definition of Theorem 11(3) for u, we have u||r = sup < x ,x 1 - У (Ц )х0> ( ) p = in£ IIй Hr,p 11 8 UEO I W* (tj, t)x Us dr = 1 Proof Since 11(7) is satisfied, we have p ê sup , Xj - У (^ )х 0 1/q x e r1 1 ||W*(tj, t)x|[s d r = s u p < x ,x 1 - y ( t 1 )x0> = p' 11(9) 1/q W*(t, , т)х II dr Jг Therefore, we only need to show that p cannot be larger than p '. Let us suppose the contrary. Then there exists a positive constant a such that p' < a < p. Since a< p, by 11(6), the system is not (x0, x t , K, ) controllable, and there exists a vector x' € Rn: 4 1 /q |< x ' , X j - У (^ )х 0> I > o Çj II W *(tj, t)x' Ii; dT^) 11(1 0 ) 270 MARZOLLO Since a > 0 , l^x' , x 1 - V ( t 1 )x0^>| is positive and, therefore, since û e U, hence p > 0, by 11(7) also (^J I]W*(t1 , t)x' l^dr^) is positive, and we may therefore consider the quantity x 1 (/l|w *(tj,T)x' И,4 d r ) and by 11(1 0 ) we have | p = sup | We may raise the following question: with reference to the same system, the same points, to p and hence ufiP , can p be less than p? We shall give a partial answer to this question in the following Corollary 11,3 If the system 1(1) is controllable relative to any pair of points of Rn, to class UfiP of admissible controls and to the time interval K= [t0, t], where К is the minimal time interval over which the system is controllable relative to given x 0, x x and to U£p , with p given, then p = p, where p is the minimal value of the norm of controls which effect the given transfer from xo to x j , in K. Proof Since К = [t0, t] is a time optimal interval, by Theorem II, 2 there exists a vector x' such that 11(4) holds. Since the system is controllable relative to any xo. x l. to u £ p and to К = [t0 ,"fl, by the already quoted Con dition I,Is Y(t0, t) = / W(t, T)W*(t, T)dT is positive definite and, therefore, IAEA-SMR-17/37 271 L j the integral yj ||w(t, t)x' ||s d r ) is positive. We may, therefore, write p = | < X ',X ! - V ( t jx 0)| Î, _ vi/q ||w*(t, r)x' Us Ат j and recalling 11(11), p è p. Recalling that obviously p S p, the corollary is proved. In what follow s, we shall refer to time intervals К = [t 0, t] which are minimal with respect to x0, and to classes p of admissible controls, p being also the minimal norm of controls transferring x0 into х х, in K. Beside 11(8), we shall also use < X , X ! - V (t!)x 0 > p = mm ||u||r>p max(x, xx -V U ^x^ ue U .i/q ||w*(t, t)x||s dTj (j^||w*Ct, r)x||s4d r) =1 11(8 ') We shall now see that the derivation of optimal controls (optimal in the sense just specified of being both time-minimal and norm-minimal) follows from the previous results and from a repeated use of Holder's inequality. Let û be optimal with respect to x0, xj, p, K= [t0, t] . Since û transfers x 0 into x : in the interval K= [t0, t], we have x i~ v (t)x 0 = j W(t, r ) û ( r ) d T 11( 1 2 ) *0 and since К is the minimal time interval, 11(4) holds for some x. Putting together 11(4) and 11(12) and taking the adjoint W*(t,t) of W (t,t), we have T T . 1/q Р ^ y "| w * (t, t)x||s4 d-rj = J Г 1/ ~ p (^ f IIW*(t, t ) x ||s4 d r^ = J' s ( f iir>dT^) ( J t>*ic d'’ h to = p ^ y^||w*(t, т)х||^ dT^ = P II W *X [j Sj q 11( 13) Since in 11(13) the first and last terms are equal, we must have equality at each step. Recalling the conditions for Holder's inequality to hold as an equality, we have almost everywhere in K: ||u(t) ||r = C||w*(t,t)x||fp 11(14) u¡ (t) = K(t)[W*(t,t)x]-^r 11(15) and sign Uj(t) = sign[W*(t, t)x]-/r 11(16) From 11(8') and 11(14) we may determine the constant C. Indeed, r P = ||û||tip = ci^J/ ||w*(t,T)x||,P a-p d s\1/p j = C(||w*x||Siq)4 / P to Hence, С (||w*x||Siq)q/pP _=______(||w*x||Siq)p-iP_ Since (see 11(4) or 11(8')) the vector x is determined up to a multiplicative real constant, and since we suppose, as always, x 1 - W[t)x0 f 0 , we may determine x such that the scalar product l^x.x! - V(t1 )x0> | is equal to one. In correspondence to such x, we have from 11(8') „ 1 P ‘ II W*x ||Si q and, therefore, C = (||w*¿||s>q n (17> To determine the function K(t) let us take the finite-dimensional norm of u(t), using 11(15), IAEA-SMR-n/37 273 F rom 11(15), 11(17), and 11(18) we have K(t) ||w*(7,t)x|| 11(19) |W*x||s4q From 11(15), 11(16), 11(19), we eventually have ' q _ s Ûj (t) ||w*(t,t)x|| p r |w*(t, t)x|. sign(W*(t, t)x)¡ 11(20) l|W*x||Siq which is the general expression for the optimal control when the norm of u is the general norm f p/r \ l/ p ||г- р = ( Л Е |и‘(т)|Г]Р dT) — + — = 1 , — + — = 1 , and x is given by 11(8 ), 11(8 ’ ) with the additional constraint r s p q |x, x i - V(t)x о I = 1. Many particular cases of the general formula 11(29) are interesting. For example, if u e L 2|2 ancl its norm becomes 1 m 1/2 IIй ll2,2 = ( JYj lUi(T) |2dT) so as to have the intuitive meaning of "energy", the optimal control function is the following: u¡(t) (W*(t,t)x)¡ J Y |w*(t, t)x|j dr to ‘ if u £ L» . and its norm is u ess sup max j Uj (t) I t e К i = 1 ... ш the optimal control function is 1 (t) = — sign[W*(t, t)x]j t m J 2^ |w*(t, t)x j j d 7 In general, the type of norm to be chosen is given by physical considerations, as is obvious. Let us consider the case ne Lr», or 274 MARZOLLO IIU||r „ = ess sup ; (1 s r s °°) te к i = i and let us see how it is now possible to derive the well-known Pontryagin's maximum principles for this case. Let us consider the system 1(1) with u= 0: x = A(t)x 11( 2 1 ) and its adjoint equation ф = - A*(t)ф 11( 2 2 ) Let the fundamental matrix solution V(t) of 11(21) be non-singular at t, hence invertible almost everywhere. Calling \&(t) the fundamental matrix solution of 11(22) with = V - 1(t), we immediately see that **(t) = V_1 (t) (Indeed, (d/dt) [Ф*(t) V(t)] = - [A*(t)'t(t)] * V(t) + Ф*^) A(t) V(t) = 0, hence ^*(t) V(t) =1 , where I is the identity matrix). As a consequence, we may write the general vector solution of 11(2 2 ) in the form w(t) = M>(t) Ф 1(t)x = V 1 (t)V*(t)x 11(23) where x = w(t). The optimal control law u makes all inequalities of 11(13) equalities, as we have seen. In particular, |< W *(t,t)x, Û(t)>| = ü W*(t, t)x ||s (I û(t) ||r a.e. in К or, equivalently, K v ^ w v ^ t l x , B(t)u(t) > I = |< B*(t)V‘ 1*(t)V*(t)x, u(t)>| = ü B*(t) V "1*(t)V*(t)x ||s ||û(t)||r and since for every u such that m teK i=1 | For a general type of norm, this computation is a rather difficult problem which we are not going to treat here. We shall only show how the computation of x becom es an easy task when u £ L 2 2 , hence u belongs to a Hilbert space. From 11(20) we see that in this case the form of the optimal control is û(t) = —------= p2 W*(T, t)x 11(25) t J^(W*(t, r)x, W*(t, r ) x ) d T to Since û transfer x 0 into xj at t, we have, putting again z(t) = x x -V (t)x 0, 7 J W(t, r)u(r)dr = z(t) to which, by 11(24), becom es T p 2J'W(t, t)W*(t, t ) x dT = z(t) to or, with the usual position for Y(t), the determination of x reduces to the solution of the linear algebraic equation p2 Y(T)x = z(t) 11(26) with (from 11(24)) i 2 = ( W ) II(27) Let us distinguish two cases: in the first one, the system 1(1) is controllable at time t relative to any pair of points of Rn . In this case, Y(t) is non singular, therefore invertible, and Eq.II(26) has the unique solution x = j ; Y_1 (t) z(t) 276 MARZOLLO with p2 = < Y '^ t) z(t), z(t)> and from 11(25), the expression of the optimal control law û, is û(t) = B*(t) V -1* ( t lV ^ t ir 1 (t) z(t) for t G [t0, t] 11(28) In the second case, the matrix Y(t) is singular, but the system is still controllable with respect to x0, x 1( К = [t0, t ] , p, and therefore the matrix C(t) = Y(t) - (l/p 2)Z(t) is positive semi-definite. We shall see that it is still possible to find an explicit expression of the optimal control law. To do that, we need the following Theorem II, 5 Let Ra(Y(t)] be th_e range of matrix Y(t). Then, if C(t) is positive semi- definite, the vector z(t) belongs to Ra[Y(t)]. Proof Let N[Y(t)] be the null space of Y(t), that is the set of vectors x such that Y(t)x = 0. (The dimension of the null space N[Y(t)] is equal to the multiplicity of 0 as an eigenvalue of Y(t)). Since Y(t) is real and symmetric, its eigenvectors span the space Rn, and Rn = Ra[Y(tj] ® N[Y(?)] Let P n be the projection operator of Rn into N[Y(t)], and consider the vector P N z(t). Since C(t) is positive sem i-definite, we have < C(t) PNz(t), PNz(t) > = - 4 s (PN z (t j,z (t )> s 0 and therefore Pn z(t) is orthogonal to z(t), which means that z(t) does not have components in N[Y(t)l and belongs to Ra[Y(t)] , as we wanted to prove. Let now i be the dimension of N[Y(t)], so that n - I is the dimension of n Ra[Y(t)]. The vector z(t) can then be written as z(t) = ^ ¡ > where d¡ are _ i=i+l _ the eigenvectors of Y(t) corresponding to the non-null eigenvalues of Y(t). n Since x can be written as x = ^ £¡ d¿, where dx, ..., di are the vectors _ i = i spanning N[Y(t)], equation 11(26) becomes IAEA-SMR-17/37 277 p2 Y(t) Y ? i di = ^ / i d i i=l i=fi+l n n »s Z c*x‘a* ■ Z i=C + l i=i+l where are the non-null (and real, since Y(t) is real and symmetric) eigenvalues of Y(t). The solution of 11(26) is, therefore, s- y ( í f d<) i=l i=i + l and, from II<25), we see that the optimal control law is fi n 11(29) i = l i = C+l where Ç1( Çn are arbitrary constants. We may check that the values of ?i, .... ?n do n0^ affect the norm p simply by computing p, indeed, 1= I 2 N 1 /2 11(30) i= C+l Formula 11(30), with 1 = 0, is, of course, valid also when Y(t) is non-singular. E xercise II, 1 Given the system 2 2 0 2 4 2 8 x + u(t) x 0 = E(l) X= _ 4 x l = 5 2 1 2 -3 -4 and admissible controls u such that T I u(s) ds s 3 where ||u(t) [| is the Euclidean norm of u(t). 278 MARZOLLO a) Prove that it is controllable relative to x 0, x i to the interval K= [0,T] for some T> 0 , and to the given class of admissible controls; b) determine the minimal time t such that the system is controllable relative to the same quantities and construct a corresponding time optimal control; c) verify whether there exists only one time-optimal control; d) verify whether there exist admissible controls which transfer x 0 in x x, and which are time-optimal but not norm minimal; e) if the class of admissible controls is U= {u: sup ||u(t)|| s 3}, with t € [ 0, T] respect to which T is the system controllable? Solution 20 The eigenvalues of the matrix A = are Xj = 2, X2 = - 3. 2 2 The diagonal matrix A' = S^AS, with S = — and S" 1 ; - 1 is, therefore, A' , and the fundamental matrix solution V(t) of E (l) with V(0) = I is 2 2 e 2t 0 1 - 2 V(t) = - - 1 1 0 e '3t 1 2 Therefore 2 2 e 2 (T - 1) о 1 4 -1 e-3(T-t) - 1 1 1 -2 2 4 2 2 0 0 = e ‘ 3(T ' 0 E(2) 1 2 1 2 - 1 1 1 2 and the matrix product W(T, t) W*(T, t) is 2 2 0 0 2 - 1 = _ - 6 ( T - t ) E (3) - 1 1 0 5 2 1 and, therefore, IAEA-SMR" 17/37 279 2 2 0 0 2 -1 1 _ e -6T Y(T) = J W (T-s) W*(T- s)ds 6 о -1 1 0 5 2 1 2 2 0 0 2 -1 1 - e - 6T E(4) 6 -1 1 0 1 2 1 We notice that since Y(T) has 0 as eigenvalue it is not positive definite; hence the system is not controllable relative to any pair of points x 0, Xj. From the previous position z(T) = x^ - V (T )x 0 we have 8 2 2 e 2T 0 i - 2 2 1 z(T) = ’ 4 CO H -4 - 1 1 О n> i 2 -3 1 CO 2 2 1 - 2 e2T 0 E(5) - 1 1 1 2 -4 0 e“3T -4 and,therefore, 2 2 1 - e2T - 2 2 Z(T) = z(T) z*(T) - 1 1 1 2 - e 3T - 1 1 - e2T 1 2 - 1 X 2 - 1 - 2 2 - e"3T 2 1 From the definition of C(t) and from E(4), E(5) we have C(T) = Y(T) Z(T) 2 2 — (16-8e2T)2 c i(T) 2 -1 -6 T - 1 1 0 | ( l - e - 6T) c2 (T) 2 1 E(6 ) where с г(Т) and c 2 (T) are functions obtainable from E(4), but which are not relevant for our purposes. 280 MARZOLLO From E(6 ) we see that C(T) has the same eigenvalues as the matrix ^ С г (Т) ^ i e ( 1 6 - 8 e 2 T ) 2 R(T) C2 (T) and, therefore, C(T) has the same character as R(T) as far as positive or negative semi-definiteness is concerned. By inspecting R(T), we easily see that a necessary condition for it to be positive sem i-definite is that 16-8e2T be zero, and this is verified only for T = I In 2 = t*. So this is the only candidate for the system to be controllable. It turns out that the system is controllable relative to the given x 0, x 1; p = \Гз and the time interval [0,t*] . To prove that, let us first see that the equation Y(t*)x = z(t*) E(7) has a solution. From E(4), E(5), Eq.E(7) with t* = | In 2 is 2 2 0 0 2 - 1 2 0 35 2 £ = E(8 ) 48 2 - 3 /2 zl 1 0 1 2 1 - 1 1 Since is non-singular, we may pre-multiply both sides of E(8 ) -1 1 by its inverse, obtaining i—l 0 0 2 1 0 35 ✓4 X = 12 0 1 2 1 4 • 2 3/2 or, E(9) (2 *i+*2) =Ц -4‘ 2 ' 3/2 We know that any solution x of E(9) is such that the control u(t) = B*(t) XV'^MVIt^x takes x 0 into x £, at time t*. Actually, E(9) has infinitely many solutions, all with'this property and all corresponding to norm optimal controls. Among these solutions let us take one of the form x= X * i 9 _ о /л that from E(9) we have X = — 2 о D IAEA-SMR-17/37 281 The corresponding norm-minimal control is a3t u (t) 0 s t s j In 2 E(10) 35 and its squared norm is OS , V E(U) 6 dS=35 Since p - < 3, the control û is admissible and we have proved that 2 the given system is controllable relative to , time interval -3 -4 [0 , \ In 2 ] and p = ч/"з. Of course, since t* = j In 2 is the only time for which the system is controllable, then it is also the minimal time, and the control u(t) given by E(10) is not only norm-minimal, but also time-optimal. Since in E (ll) we saw that p < p = \Í3, we expect that, besides all infinitely many norm-minimal controls which effect the same transfer in the same time, there are other controls which do the same job, being also admissible but not norm-minimal. Indeed, let us add to the norm-minimal control u(t) given by E(10) any other control ux(t) such that iln2 J V (| ln 2 )V'1 (s)B(s)u 1 (s)ds = 0 E ( 1 2 ) K i Taking u(s) = , with K j and K2 constants, we have from E(2) lK 2 I In 2 2 4 К: ~2K1 + 4K 2 ln2)V ' 1 (s) B(s) ds = С = С -K2_ 1 2 K 2 _ К L - 2K2 with С constant, and therefore all controls of the type u x(t) = f"* L*2 satisfy E(12). Therefore, all infinitely many controls u of the type К l u(t) = Q(t) + 2 K 1 with |k J >0 limited by the constraint ||u(s)||ds< 3 are admissible о and transfer xo in xi in the time interval [0 , -| ln 2 ] , and are therefore time optimal but not norm minimal. Passing now to the last question e), let us verify the general controllability condition 1(1 1 ) in this specific case. 282 MARZOLLO F or the system to be controllable at time T, we must have T I 2 - 1 X l 4 • 2 e 2T 2 1 X 2 e -3T 0 1 2 -1 X 1 ds, Vxx, Vx2 E(13) 0 2 2 1 X2 2 -1 Since the matrix is non-singular, the vector 2 1 У 1 2 - 1 x l У 2 2 1 x 2 spans the space R as well as and, therefore, we may write У 1 4 - 2 e 2T 0 1 У1 / i \ S s /3 J e - 3 v y i < v y 2 I (4 - 2e2T)y 1 +e‘3T y 2 | S ^ (1 - e'3T) |y2 | v y i- УУ2 E(14) v 3 as a condition of controllability instead of E(13). A necessary condition for E(14) to hold is that the coefficient of yx be zero, i.e. T=|ln2. For such a value of T, E(14) becomes 2 " 3/2 i i 1 - 2 - 3 /2 1 У2 I s J У 2 vy2 which is satisfied. Therefore we may conclude that the system E (l) is controllable with respect to x0, x¡, class of admissible controls U = {u: sup ||u(t)|| s3}, only for the time interval К = [0, ^1п2]. te[o,iln2] IAEA -SM R-17/37 283 PART III. REACHABILITY OF SETS IN THE PRESENCE OF DISTURBANCES Given a control system subject to partially unknown perturbations, a problem which arises naturally is to find the set X 0 of initial states x 0 which may be transferred "for sure" into a given target set X N by an admissible control. Many practical examples of such a problem may be found in various fields of system sciences, of operational research and management science, when a "w orst case" or conservative philosophy is to be adopted, that is when the aim is to reach a goal for sure, not to maximize the expectation of a given event: an intuitive case is, for example, the problem of deter mining the zone starting from which a given vehicle can reach for sure a target in a noisy environment. This kind of problem, as well as the determination of the best control in the worst possible conditions, has been formally treated only recently, either in itself or in the framework of differential game theory, in which it may be embedded if the partially unknown disturbances or environmental situations are conservatively treated as the adversary of game theory. The algorithms which are presently available for the solution of the problem above, even for linear systems with admissible controls and disturbances belonging to convex compact sets, are generally quite difficult to be implemented and it seems that further research is needed in order to furnish flexible algorithms for facing practical situations. The purpose here is to sketch the key points of the methods of solution proposed until now, for different information structures available both to the controller and to the adversary disturbances, and to show some interesting simplifications which are the consequence of taking a subspace as a target set and any control and disturbance as admissible; for this case, it is possible to state some interesting results in a compact form, as we shall do. As an example of the problem mentioned, let us consider the following linear discrete time system x k + i = AxK + BuK + CwK (k=0, 1 ...... ) III ( 1 ) where the states x K belong to Rn, controls uK and disturbances wK belong to closed compact sets U of Rm and W of Rp respectively and the target is a given closed compact set XN of Rn (many types of extensions, like the one to tim e-varying systems with sets U and W varying also with time instants are possible but inessential to our purposes). We formulate the problem of reachability of X N from X 0 under disturb ances by giving two among the main possible information structures. Problem III, 1 Determine the set x j of initial states x 0 which may be transferred into X N at the time instant N by an admissible sequence u0 (x0), Ujixj), ...ui^.jfxj^.j of controls, for any admissible sequence (w0, wx, ... wN.x) of disturbances. Clearly, for each of its "m oves" u¡ the controller may take into account his perfect information about the present state x ¡, but is ignorant about the "m ove" Wj of the disturbances. 284 MARZOLLO P r o b l e m III, 2 2 Determine the set X 0 of initial states xQ which may be transferred into XN at time instant N by an admissible sequence u0 (x0, w0) , __ _ uN.t ( x ^ , w^j ) of controls for any admissible sequence w0, ..., wN_! of disturbances. Obviously, the information available to the controller is in this case "larger" than in the previous case, since in each step he knows the "m ove" of the disturbances. Even for these simple examples of the general problem, with the clearly defined information structures mentioned, the used methods for finding x j or X„ have required the use of the so-called operation of geometric difference of sets 1 and either the use of separation theorems for convex compact sets [3], or the use of support functions to describe sets and set-inclusion [4, 5] or some ellipsoid-type or polyhedrical-type approximation of sets [6 ], which have the advantage of describing sets with a finite number of parameters but can give only sufficient conditions for a point x 0 to be transfer rabie into X^ , that is can give only subsets of X j or X j. When the instant of time at which reachability occurs is of interest, game-theoretical approaches to the above kind of problems are interesting, but computational results are rather involved indeed [7, 8 ]. Referring to problem III, 1 and using Eq.III(l), we see that at the (N -l)-th step the state xN_j is "transferrable" into XN if there exists uN_x ( x ,^ ) € U such that AXn-! + Виц-! + CwN.j e X N for every e W , that is iff there exists u ^ t x ^ ) e U such that A x n - 2 + B u ^ - i + C W G X N or A x n . j + BuN.j G (XN * CW) that is iff A x N. x G (Xn Í C W ) * BU Therefore the set XN-j of states x ^ which may be transferred into X Nin one step satisfies the following equation: Yjj-i = AXJ,., = (XN * CW) - BU and, similarly, the set XN_x (i = l,...,N ) of states which maybe transferred into XN in i steps satisfies = a 'xJj.j = (YJj-i+i — AW CW) - A 1" 1 BU (i = 1...... N) 111(2) 1 Given two sets S and T, the geometric difference Z = S * T is defined asZ = {z :z + T cS }, For its properties, see Ref. [ 9]. IAEA-SMR-17/3 7 285 E q u a tio n 111(2), t o g e t h e r w ith the o b v io u s e q u a lity = X N 111(3) is a recursive algorithm for building Y -,^ , Y0 from U, W, X N and, therefore, x j which is characterized by ANXi=Yj 111(4) Proceeding in an analogous manner for Problem III, 2, and taking into account the different information structure, we have that in this case xN_x may be transferred into XN in one step iff for every wN.j e W there exists uN-l (x N-i > w N-l ) e U such that + BuN.x -t- Cw^-x £ X N that is iff Axn-i + CW e X N - BU Therefore, the set Xf,^ of states x N.j of this type is given by Y n - i = A X N -1 = (XN-BU)*CW and similarly the sets Х^.; (i = 1, ..., N) of states which may be transferred into XN in i steps by admissible controls and for every disturbance satisfy Y2.j = А*Х*М = (Y* _ i+1 - Ai_1BU)± Ai_1 CW, (i = 1...... N) 111(2') We have again found a recursive algorithm which, starting again backward from Yn = X N 111(3') 2 gives the following characterization of X 0: An,N X 2 = Y 2 111(4') As is apparent from 111(2), (3), (4) or from 111(2'), (3'), (4'), the determination of X j or of Xq essentially involves the computation of N geometric differences and N additions of sets. As we saw in Part I, the analogue of Problem III, 1 and Problem 111,2 for continuous systems, considering only open loop controls would lead to a similar computation for only one step, but the m atrices Ai_1 В and A 1' 1 С (i = 0, ..., N -l) would be substituted by linear integral operators from sets U and W of control and disturbance functions into Rn. Going back to consider Eqs 111(2), (3), (4) or 111(2'), (3'), (4'), we see that the key difficulty is the description of sets resulting from operations on sets. 286 MARZOLLO A natural tool to be used for this purpose is the one given by support functions (see, e.g. [5]), when the sets X N, U, W, and therefore also all other sets involved, are convex and closed. Support functi’ons are linear with respect to the addition of such a kind of sets; unfortunately, the operation of geo metric difference of sets does not possess such a property; if h is the support function of the set X : hy(p) = sup < p, x)>, Vp € X*, whers X is the x e x dual of the space X, then for every set S, T, we have hs* T(p) § hs (p) - h T(P) 111(5) It is therefore important to find conditions on the sets involved in 111(2), (3), (4) (or 111(2'), (31 ), (4')) for 111(5) to be held as an equality (we suppose, for sim plicity, that the geom etric differences are never the empty set). In such a case, the construction of the sets Y¿-i , ..., Yq from Y¿ and of Y^-i , ..., Yn would be straightforward and we would also have the interesting consequence that these two sequences of sets would coincide if YN = Y^; therefore x j would also coincide with x|j and the difference in the information structures of Problems 1 and 2 would not have any effect. Defining St = (S ¿ T) +S as the "regular part" of S with respect to T, we define the set S to be regular with respect to T iff Sx = S. It is easy to prove that 111(5) holds as an equality iff S is regular with respect to T ; indeed, from the regularity it follows that hs * T(p) + hT(p) = hs(p), Vp that is hs * T(p) = hs(p) +hT(p), Vp 111(6) and the regularity follows from 111(6) for any couple of sets S and T com pletely described by the support functions. These last remarks seem an important reason for research work on conditions easy to be expressed on couples of sets (representing, respectively, target sets and reachable regions) in order that the first one be regular with respect to the second one (see, e.g. Ref. [9]). A case in which the regularity conditions are trivially satisfied is when the sets under consideration are linear subspaces. Sets of this kind have, furthermore, the following absorption property: Г S if S D T S ± T = \ 111(7) Щ} if S p T An interesting consequence of this property applied to our Problems III, 1 and III, 2 when U and W are Rm and R r, respectively, and the target set XN is a linear subspace M of Rn. In this case we can define the orthogonal complement N of M in R n : Rn = N ®M and the projection operator n of Rn onto N. We have 7ГХ e N, VxGRn, 7ГХ = {0}, Vx e M IAEA-SMR-17/37 287 The target set X n is therefore characterized by MN = 7гХ n = jtM = {0} 111(8) Referring to Problem 111,1 and using Eq.III(l) we have therefore, for x to be transferrable into M in one step 7t(Axn-i + R^n-i ^ CwN-i) = Mn “ {0}, ^wn-i ^ W that is ttAXn-í = (MN * CW) + BU = Mn-i and using Eq.111(1 ) M¿.¡ = = (M¿ . i+1 ± 7rAi' 1C W ) - тгА1' 1 BU (i = 1 ,..., N) which gives an iterative algorithm for the construction of sets X^-; of states which may be transferred into X n in i steps, until M¿ = AnX 0 = (Mj * An_1 CW) - jrA^ 1 BU which characterizes x j, Recalling U = R m and W = Rr , and defining P¡ = 7Г (span A 1' 1 B) Q¡ = тг (span A 1_1 C) 111(9) we have Mj,.j = ttA^n-í = (Mj,.i+1 ±Qj) +Pj (i = 1, ..., N) 111(10) that is, using the absorption property of geometrical difference for subspaces = ^ ¡x ;, = MN. i+1 + p t if M¿ . i+1 D p t 111(1 1 ) MN-i = { 0 } if Mn- í+1 Э Pj From 111(11) we see that is not empty iff i = l Q : = {O}; and PjDQ^ (Pj + P2 ) Э Q 3 ... ^ Pi D Q¡ 111(12) i = i Proceeding in an analogous way for Problem III, 2 in this case, we have again 111(8 ) and M 2N-i = ^ x ^ i = (M2 . i+ 1 + P01QÍ 111(1 0 ') 288 MARZOLLO until м 20 = ttAn X 2 = (M2 + P n) * q n Using again the absorption property we have M 2N. ; = M 2N_i+1 + i f M ^ + P O Q ; 111(1 1 ') MN-! = {1} if Mn-1+i + P¿ 1 Qj 2 From 111(11') we see that MN.t is not empty iff i P ^ Q j Pi + P2 Э Q 2 ...... ^Pj Э Q¡ 111(12') j = i As is intuitively comprehensible, condition 111(12) for the existence of some x 0 which can be brought into Хц in i steps (for example, in i = N steps) is stricter than condition 111(1 2 '), which corresponds to an information structure more favourable for the controller. Nevertheless, it is very interesting to N notice that when 111(12) is satisfied, then M 1 = P. = M2 . j = i We may summarize the preceding results in the following Theorem III, 1 The set X j solution of Problem III, 1 for U = Rm , W = Rr and XN given by a subspace M of Rn is not empty iff Eqs 111(12) are satisfied and is characterized by An X¿ = M 1,, where M¿ is given by the recursive algorithm 111(1 1 ) starting from M¿ = 0 . Analogously, the set X 2 is not empty iff Eqs 111(12') are satisfied, and are characterized by 7rANX ¿ = M 2 2 where M0 is given by the recursive algorithm III(11') starting from M^j = {0}. Furtherm ore, if X j is not empty, then x j = X 2, with N = Mo = X Pi j = l For the particular system xk+1 = Axk + Buk + wk (k= 0, 1, ...) 111(13) we have the following IAEA-SMR-17/37 289 Corollary Considering Problem III, 2, if X q is not empty for some N, then (a) any x o G X q can be brought into M in an arbitrary number of steps, (b) X§ = R n. Proof Since 111(12') is satisfied with Q ¡ = vrl = L, we have Px Э Qx = L; on the i i- 1 i other hand ^ Pj э Pj ... ^ Pi i therefore, since all P¡ are in L, ^ Pj = L j = 1 j = 1 . J = 1 for any i, that is M2 = M2 = ... MjJ, = L. Recalling Mj = ггА'Х2, part (a) is proved. Part (b) follows from the characterization of X2: X q = {x: ttANx € M0} from the definition of тг and from M0 = L. i-i We may observe that in our case, since С = I, Pj Э ... Э Pt Э Q x = /rRn , j = i that is it span [B, AB, ...... AN 1 B] D 7rRn. This means that the system was controllable (in the classical sense), at least in the subspace L. REFERENCES [1] ANTOSIEWICZ, H ., Linear Control Systems, Arch. Rat. Mech.. Anal. 12 (1963) 313-24. [2] DUNFORD, , SCHWARZ, J.T. , Linear Operators, Parti, Interscience (1964). [3] MARZOLLO, A. , An algorithm for the determination of ¿¡-controllability conditions in the presence of noise with unknown statistics, Automation and Remote Control (February 1972). [4] WITSENHAUSEN, H. , "A MinMax Control Problem for Sampled Linear Systems", IEEE AC (february 1968). [5] WITSENHAUSEN, H .,,"Sets of possible states given perturbed observations", IEEE AC (October 1968). [6] GLOVER, J.D ., SCHWEPPE, F .C ., "Control of Linear Dynamic Systems with Set-constiained Disturbances", IEEE AC, october 1971. [7] PONTRYAGIN, L. , Linear Differential Games, I, II, Soviet Math. Doklady, 8 3 (1967): 4 (1967). [8 ] BORGEST, W ., VARAIYA, P., "Target Function Approach to Linear Pursuit Problems” , IEEE AC (October 1971). [9] MARZOLLO, A ., PASCOLETTI, A ., Computational Procedures and Properties of the Geometrical Difference of Sets” , J. Math. Anal. Appl. (to appear). IAEA-SMR-17/61 EXISTENCE THEORY IN OPTIMAL CONTROL C. OLECH Institute of Mathematics, Polish Academy of Sciences, Warsaw, Poland Abstract EXISTENCE THEORY IN OPTIMAL CONTROL. This paper treats the existence problem in two main cases. One case is that of linear systems when existence is based on closedness or compactness of the reachable set and the other, non-linear case refers to a situation where for the existence of optimal solutions closedness of the set of admissible solutions is needed. Some results from convex analysis are included in the paper. INTRODUCTION Perron paradox. A necessary condition for N to be the largest positive integer is that N = 1. Indeed if N f 1 then N2 > N. So N is not the largest integer, contrary to the definition; thus, N = 1. We have proved the theorem: "If N is the largest integer, then N = 1 ." There is nothing wrong with this statement. It is correct, except that the assumption is never satisfied. From such an assumption everything follows. This example, though it looks like a joke, has an important implication, i.e. necessary conditions in optimization may be useless if we do not know that the solution we are talking about exists, since, if it does not exist, one may derive a wrong conclusion from correct necessary condition. A simple example of such a situation is the following functional: 1 f (1 + x 2 )1/4 dt (0 . 1 ) о if we want to minimize it over all functions of class C 1 on [ 0 , 1 ] and satisfying the boundary condition x(0) = 0, x(l) = 1. This is a simple problem of calculus of variations. It is well known that the solution of the problem, if it does exist, has to satisfy the Euler equation which in this case is simply хЦ+х2 ) ' 3/ 4 = constant Therefore, any solution of the Euler equation of class C 1 has to be linear (x(t) = constant) and the one which satisfies the boundary condition is x(t) = t. The value of the functional (0.1) for x(t) = t is 21//4 and it is easy to see that it is not minimal. The infimum in this case is equal to 1 and the optimal solution does not exist. Without checking the existence of the minimum we seek, we may be led to a wrong conclusion, from the necessary condition. Consider another example. In the set of all functions x(t) of class C 1 on [ - 1 , 1 ] such that 291 292 OLECH x(-l) = x(l) = 0 and |x(t)| â 1 we wish to minimize the functional +l I(x) = J' tx(t)dt -1 The solution of this problem as stated does not exist. Indeed, it is clear that the integrand is estimated from below by - |t|. Thus, I(x) g -1 and equal to - 1 iff x(t) = 1 for t á 0 and - 1 for t g 0 while for any x(t) of class C1 the set {t |x(t) | < 1} has positive measure and, therefore, I(x) > -1. The example shows, and this is rather typical, that, to ensure the existence of the minimum, we sometimes have to enlarge or complete the set on which we wish to minimize the functional in question. In fact, the above problem has no solution in the class of C1, but it has a solution in the class of absolutely continuous functions. Hilbert said: "Every problem of the calculus of variation has a solution provided the word 'solution' is suitably understood". The meaning of this statement is as follows: If we look for a minimum of a functional and the infimum is finite, then we always have a sequence approaching the infimum - the so-called minimizing sequence. The question of the existence of a minimum reduces to the problem of when we can produce the optimal solution out of the minimizing sequence. Often this is possible if we are able to prove that the minimizing sequence converges. But, even if this is possible, then not always in such a way that the lim it belongs to the set on which we originally wished to minimize the functional. The last example is such a case. Indeed, one can prove that if x n(t) satisfies the boundary condition, is continuously differentiable and such that I(xn)-* - 1 as n — oo, then x n(t) converges uniformly on [ - 1 , 1 ] to x 0 (t) = t + 1 for t S 0 and -t + 1 for t S 0 while +l -l converges to zero but the xn(t) do not converge uniformly to x n(t). Hence, xn is not convergent in C1 space but is convergent in the space AC of absolutely continuous functions. It is, therefore, natural to adjust the notion of "solution" and to treat this problem in the space of absolutely continuous functions rather than in C1. This situation is typical and, for this reason, in control problems usually the control function is assumed to be measurable. Even in the case where the optimal solution is more regular, it is convenient to introduce this enlargement of the space in order to prove that the optimal solution exists. For the similar reason of completion, the Sobolev spaces were invented and became so successful in the theory of partial differential equations (weak solutions). In this paper, we discuss the existence of optimal solutions first to linear problems and then to a general non-linear problem. In the first case, compactness and convexity of the so-called reachable set is instrumental, while in the second case the closedness of the set of admissible solutions is important. In both cases, convex analysis is used extensively. IAEA-SMR-17/61 293 In Section 1, an existence theorem for time-optimal linear problems with constant coefficients is stated. Section 2 contains a detailed discussion of the integral of set-valued functions. Also, several propositions concerning convex sets are stated and proved in this section. Section 3, besides a proof of the theorem stated in Section 1, also contains some other applications of the integration of set-valued functions. Among these, we discuss the so- called "bang-bang" principle. The existence theorem given in Section 4 is a typical example of an extension of the direct method well-known from the calculus of variations to optimal control problems. The direct method originated by Hilbert was developed by Tonelli, McShane and Nagumo for problems in calculus of variations. Roughly speaking, it consists in establishing that the minimizing sequence converges (compactness), that the lim it is in the set in which we minimize the functional in question (closedness or completeness) and that the value of the functional at the limit is not greater than the infimum (lower semicontinuity). For optimal control problems, it has been extended in the last decade, mainly by Cesari. Theorem 4. 1 stated and proved in Section 4 is an example of possible existence theorems along these lines. The important difference (and perhaps the only one, too) between classical results from calculus of variations and those concerned with the existence of a solution of optimal control problems lies in regularity assumption. To cover problems of interest, one is led to consider problems with as weak regularity assumption as possible. For this purpose, also a theory of measurable set-valued mappings, selector theorems, etc. is used in an essential way. Mathematical control theory, among others, gave an impact to the recent development of this theory. These questions will also be briefly discussed in Sections 4 and 5. In Section 5, we discuss the necessity of the convexity assumption in the existence theorem given in Section 4. Actually, convexity is essential for lower semicontinuity or lower closure, and in Section 5 we give a result (Theorem 5.1), which shows that, in certain special cases, convexity is necessary and sufficient for weak lower semicontinuity of an integral functional. Also, as an example, another result on lower se mi continuity is stated there, but without detailed proof. It would be impossible to cover the whole subject in such a short paper, which is, therefore, far from complete. The choice of material presented was highly influenced by the author's own contribution to the. subject. A selected list of references is included, and, at the end of the paper, we also give some comments concerning the literature. The author wishes to acknowledge the help of Miss Barbara Kaskosz in preparing this paper. 1. TIME-OPTIMAL LINEAR CONTROL PROBLEM Consider the linear control system x = Ax + Bu (1.1) where x e Rn, u e Rm, A is an n x n matrix and В is an n x m matrix. Let U с Rm be a given fixed set. Let Q be a set of functions u taking values 294 OLECH from U, called admissible control functions. In this section, we take as admissible control functions which are piecewise constant. A solution of Eq. (1.1) is of the form ( 1 . 2 ) where X(t, to) is the so-called fundamental matrix of the homogeneous linear differential equation corresponding to Eq. (1.1). That is, we have 4r X(t, t0) = AX(t, t0) and X(t,t0) = E (1.3) where E is the identity n x n matrix. In the case of A constant, X (t,t0) = exp(A(t-t0)). A solution of Eq. (1. 1) is unique if the initial value x 0 at time t = t0 and the control function u = u(t) are fixed. Thus, we shall use the notation x(t;to, x o, u) for solution (1. 2) of Eq. (1. 1) to indicate this relation. Time-optimal control problem. Given an initial state x 0 at time t 0, we wish to transfer it by means of an admissible control function to a target point х1д and we want to do this in the shortest time possible, i.e . we seek an admissible a* and a time t* such that x(t*;t0, x 0, u*) = xa while x(t;t0, x 0, u) f xx for each u 6 fi if t < t*. We shall call t* and u* optimal time and optimal control, respectively. For the above optimal problem, the following existence theorem holds (due to LaSalle): Theorem 1.1. If A, В in Eq. (1.1) are constant, U is a compact polyhedron, Г2 the set of piecewise constant control functions taking values in U and x: = x(t;t0, x0, u) for some u e Г2 and t > t0, then the tim e-optim al solution exists. In Section 3, we shall present a result which would contain this result as a special case. Now, we wish to call the reader's attention to the fact that Theorem 1.1 is, in general, not true if A and В are not constant and U is not the polyhedron. In other words, the piecewise constant control function is, in general, too narrow a class for the optimum to be attained. F or the existence of a time-optimal solution, the properties of the reachable set are decisive. By reachable set of a control system we mean the set of all states which can be reached in time t by means of admissible control functions from a fixed initial condition. In the case of Eq. (1. 1 ), it is given by gt(t) = {x (t;t0, x0, u) I uGÍÍ} (1.4) The properties of í%(t) which make the above theorem true are ¿%(t) is convex and compact for each t (1.5) 5?(t) is Hausdorff continuous in t ( 1. 6) IAEA-SMR-17/61 295 The latter means that h(^?(t), &(s)) tends to 0 if |s-t| tends to 0 where h(P, S) = max (sup d(p, S), sup d(s, P)) is the Hausdorff distance between two sets. P6P ses For the existence of optimal solution only compactness in (1. 5) is needed. Convexity is decisive in deriving necessary and sufficient conditions for optimality. In the next section a proof of (1. 5) and (1. 6 ) will be given in a more general setting. 2. REACHABLE SET AND INTEGRATION OF SET-VALUED FUNCTIONS It follows from Eqs (1. 2) and (1. 4) that ¿%(t) = X (t;t0) (x0 + S(t)) where S(t) b = / X _1(t, t0 )Bu(t)dt, u(t) 6 U Putting P(t) {p I p = X _1 (t, t0) Bu, u £ U} we have S(t)={ / v(t) dt I v(t) 6 P (t)} (2.1) to The above set, if v is an arbitrary, but integrable function, is called the integral of set-valued function P(t). It is clear that properties of á*?(t) like closedness, convexity and continuity hold iff the same properties hold for S(t) given by (2. 1). Thus, in what follows we shall discuss the basic properties’ of the integral of the set-valued mapping P. For simplicity, we shall consider P to be defined on [0, 1] with values P(t) being subsets of Rn. A point-valued function v from [0, 1] into Rn is a selection of P (or a selector) if v(t) e P(t). We shall require the latter to hold almost everywhere (a. e.) since we shall deal with measurable or integrable selections of P. By K p we denote the set of all integrable selections of P. Thus the integral of P is the set l l I(P) = J' P(T)dT = v(t) dt I v(t) e P(t) a. e. in [0, l]j- о о 1 ( 2. 2) = v(t) dt I V e K p j 0 The aim of this section is to prove that I(P) is convex and to prove some other properties of I(P). The main results are stated at the end of the section. 296 OLECH Before, we recall some elementary facts concerning the extremal structure of convex sets. Let S be a convex set. A convex subset E с S is called an extremal face of S if for each s, p £S and any 0 < X <1, condition As+(1-X) p G E implies that both s and p belong to E. If an extremal face reduces to a single point {e} then e is called an extreme point of S. The set of all extreme points of S is called the profile of S and we denote if by ext(S). Our nearest goal is to give a characterization of an extremal face through some order relations. By С we denote a convex cone in Rn and we shall write x § cy iff y - x £ С. Because of the convexity of C, this relation is transitive. Below we shall o n ly deal with convex cones which satisfy the relation С и (-C) = Rn (2. 3) This implies that any two points x, у of Rn are comparable, that is either x - сУ or У - cx- Both inequalities hold iff x - y G С п (-C) = Mc. Clearly, M is a subspace of Rn. We shall write у = maxcS iff y e S and x s cy for each x e S. If S is closed and bounded then m a ^ S exists and it is unique if Mc = {0}. Two following propositions about cones satisfying (2 . 3) will be useful: (a) For each convex cone С satisfying (2. 3) there is an orthogonal sequence a*. ..., aksuch that M c = {x | If there were x G Rn, a, |3 G A such that x $ Ca, x $ Cg then Ca С C0 or C¡ c C„and x f Ca u (-CJ or x | Са и (-Ce). Thus (-C) и С = Rn. The Kuratowski-Zorn lemma completes the argument. Notice that if С is spanned by convex set S then Mc is the linear span of Sj = {x G S I there is a > 0 that -ax G S}. Sj is the extremal face of S containing zero in the relative interior. One more notation: If a G Rn then we put Sa = {x G S I <^x, a^> = max <( s, a)>} s es IAEA-SMR-17/61 297 and, inductively, S a !...... ak - (S a !...... ak.j)a k We are now able to state the announced characterization of extremal faces to a convex set. (7 ) The following conditions are equivalent for a subset E of a convex set S: (i) E is an extremal face of S (ii) There is a convex cone С satisfying (2. 3) such that E = {у | у = maxc S} (iii) There is an orthogonal sequence aa, . . ., ak such that E = Saj...... a , where n-k S dim E. P roof. If (i) holds then take as С a cone spanned by x 0 - S (see item (/3)), where x 0 is any point from relative interior of E. By definition of C, x 0 = maxc S and since x0G relative int E, E G xo+Mc thus for each y£E, y = maxc S. On the other hand if y = m axc S then y G x0 + Mc and the interior of the segment y, x 0 has non empty intersection with E. But E is an extremal face, thus y G E. Therefore (i) implies (ii). Assume now (ii). Let aa, ... ak be the orthogonal sequence corresponding by (a) to the cone C. Notice that from (ii) it follows that E = S n (x 0+Mc ), where x 0 is any point from E. This together with (a) applied to x0-S shows that E = Sai_ . . . iak. Manifestly dim E s dim Mc = n-k. Thus (iii) follows from (ii). Assume now (iii) and let z = Xx+(l~X)y GE = Saj...... ak while x, y GSa,...... a¡, i < к, and 0 < X < 1. Then Example: Consider the set Д = x¡ = 1 , x¡ S 0 i=l i.e. an (n-1)-dimensional simplex in Rn. It is convex, compact; e G Д is an extreme point of Д if at least one co-ordinate of e is equal to one (then the remaining have to be equal zero); extremal faces are subsets of all points in Д with certain fixed co-ordinates equal to zero. Notice that each x G Д can be written as a sum n n i=l i=l where {e^, , .,, en} is the profile of A. 298 OLECH Let P be a subset of Rn, The smallest convex set containing P is called the convex hull of P and it is denoted by coP. Correspondingly, a closed convex hull of P, denoted by clco P, is the smallest closed convex set containing P. Since the intersection of convex and/or closed sets is convex and/or closed thus both со P and clco P are well defined and clco P is the closure of со P. The basic result concerning convex hulls is the following: (ó) Carathéodory theorem: If P с Rn then n n СО P = U {Y XiPi I Xj ê 0, V = l j ...... Pn}cp t o The proof of this theorem can be found in many textbooks. However, for convenience of the reader we shall also prove it here. Denoting the right- hand side of the last relation by D, we clearly see that D с со P, thus only convexity of D should be established to prove (6 ). Let x, y be two points of D, that is **11n n n n n n X = Z XiPi* y=Z Miqi' x‘" °’ s°’ b = x ^ =ijpi6 p’ qigp i=0 i=0 i=0 i=0 Let z = ах+(1-а)у, 0 < a < 1. Clearly, the set П 5Ш+12n+l 2n+lzn+1 В ={ Z 7ipi+Z Tiqi-n-1 1Z 7i = ij 7i s °l i=0 i=n+l i=0 is convex and contains z. Let n zn+i2n+l zn+i2n+l A ={(I * tí' X )' i I Z 7ipi+Z 7141-"-1=z' Z 7i = 7¡ " °} i=0 i=n+l i=0 It is clear that A is a convex, closed subset of the (2 n+l)-dimensional simplex A2n+x and its dimension is, at least, n+1. A = M n Д2п+1 where M is linear manifold spanned by A. Let e be an extreme point of A; e = Wi (e)} o si s 2n+i- We claim that, at least, n+ 1 out of 2 n+ 2 co-ordinates of e are zeros. In fact, let F be the smallest extremal face of A2n+j containing e. Then F n M = {e }. Therefore, the dimension of F is at most n+1, thus 7 ¿(e) = 0 for, at least, (n+1) " i ’s ". That implies that 2n+l = Z ^i e = lim X^x^, where a 0 , ^ Xj = 1 j=o j=0 for each к and xŸ G P. If e were not in P then there would exist e > 0 such I kl that |e-Xj| g e > 0 for each j and k. In this case we could represent e as a lim it of n n ^ Mj¡ z*. where ^ г 0, J 1, z^ G со P j-0 j=0 and |e-z^ | = e. Without any loss of generality,n we may assume that both /jf-<■1 /u¡ J and Zj1 - Zj G clco P. Hence e = jUjZj and |e-Zj | = e > 0, which contradicts the j=o assumption that e is an extreme point of S. Thus e G P and (Ç) is proved. 300 О LECH We begin the study of the properties of the integrals (2. 2) with the following: Lemma 1. Let S = clco I(P) and let e G S be an extreme point of S. Then to each e > 0 there is 6 = 5(e) > 0 such that if |l(u)-e | < & and |l(v)-e | < 6 for l any pair u, v € K p then llu -v ll = J' I v(t)-u(t) | dt § e. о Proof. Let e > 0 be arbitrary. Put 77 = e /4 N/"n and let B(e, r;) be the closed ball centred at e and of radius rj. There is an open halfspace H such that e G H n S с B(e, 17). Indeed, consider the set Q beeing the intersection of S with the boundary of B(e, r}). Q is compact hence also coQ с B(e, 17) n S is compact. Manifestly e does not belong to co Q. Thus we can separate e from co Q; that is there is an open halfspace H, such that H n (S\B(e, 17)) = p because Q is equal S n 3B(e, rj). Now take 6 small enough that S n B(e, 6 ) c H n S с B(e, rj) (2.4) and assume u, v G KP are such that |l(u)-e | < 6 and |l(v)-e | < 6 . Let А с [ о, 1 ] be an arbitrary measurable set and put wj = u + x A(v-u) and w2 = v + Xa(u -v)> where xA is the characteristic function of the set A. Of course both W j and w2 belong to Kp and we have I ( W j ) = I(u) + J' (v(t)-u(t)) dT and I(w2) = I(v) - J' (v(t)-u(t)) dT A A This implies that, at least, one of those two points belongs toHnS and therefore by (2.4) ( v ( t )- u ( t )) dT I s 2rj. A The latter holds for each A с [0,1], therefore we have I J |vt (tO -u^t) |dT S 4r¡ 0 where v¿ is the i-th co-ordinate of v and consequently, 1 II u-v II = J I v ( t ) - u ( t ) I d T S 4«/ñ" rj = с which completes the proof. As a consequence of Lemma 1, we shall state two corollaries. Corollary 1. Assume P(t) is closed for each t. Let С be a convex cone such that С и (-C) = Rn and F = { y | y = maxcx if x clco I(P)} be a compact extremal face of clco I(P). Then IAEA-SMR-17/61 301 (i) The set Kp = {v G K p| I(v) G F} is not empty. 1 (ii) F = с о I(K¡T) = с о { J v (r )d r I v e K p Û (iii) v G Kp iff v G Kp and u(t) Scv(t) a. e. in [ 0, 1] for each u G K p Proof. Since F is compact there is an extreme point e of clco I(P) belonging to F. Thus by (Ç) there is{u¿} с Kp such that I(Uj) -» e and by Lemma 1 {u¡} is convergent in Lx norm to a function v, hence without any loss of generality we may assume that u^t) -* v(t) pointwise. Thus v G KP and of course I(v) = e G F hence (i) holds and I(Kp) contains the profile of F, therefore (ii) holds as well. To prove (iii) let us fixe v G Kp and u G Kp be arbitrary. Put w(t) = maxc (u(t),Y(t)). We have the inequality v(t) Sc w(t) and therefore I(v) Sc I(w ). Therefore I(w) G F. That means that w(t)-v(t) G С and 1 J (w(t)-v(t))dt G C n (-C) о which is the case only if w(t)-v(t) G С n (-C) a.e. in [0,1]. The latter set is a subspace thus v(t)-w(t) G С п (-C) also. Hence u(t) i cv(t) a.e. in [0, 1]. On the other hand, if the latter inequality holds for each u G Kp and v G Kp then I(v) = maxcI(u). By (i) there is w G Kp such that I(w) G F. I(w) s c I(v) U G Kp as well as I(v) sc I(w). Hence I(w)-I(v) G С n (-C) which implies that I(v) G F and completes the proof. The following is a specification of Corollary 1 to the case F = {e}. Corollary 2. Under the assumption of Corollary 1 for each extreme point e of clco I(P) and each convex cone С such that C n (-C) = {0}, С и (-C) = Rn and clco I(P) С e-C there exists a unique v G Kp such that i(v) = e and for each u G Kp u(t) s c v(t) a. e. in [ 0, 1 ]. Lemma 1 as well as both corollaries hold if we integrate on [ 0, t] instead of [0, 1]. Put t t lt(P) = J P(r)dr=^y v (r )d T I v e Kp^j- 0 0 and t Ft = со { J' v ( r ) dr I v G Kp о We shall prove now the following Lemma 2. Let P(t) be closed and F, С and Kp be as in Corollary 1. Then (i) Ft is Hausdorff continuous in t. (ii) Ft is a compact extremal face of clco It(P) and Ft = {y |y = maxcclcoIt(P)}. (iii) Ft can be represented as a sum P roof. From C orollary 1, (iii) it follows that if u, v GKp then both u(t)-v(t) G C and v(t)-u(t) G C. Thus if we fix u0 G Kp then each w G Kp is of the form w = u0 + v where v(t) 6 Mc = С п (-C) a. e. in [ 0, 1]. This proves (iii) of Lemma 2. Let F = {y |y = maxc clco It(P)}. It is clear that if s n 6 co It(P) then s n + / u 0 (T )dT G со I(P) and if the first sequence tends to a point of Ft then the second tends to a point of F. This shows that Ft is compact because F is compact. Therefore, by Corollary 1, Ft = co v(T)dt I u (t ) Sc v (t ), 0 S t á t for each u G K pj- o It is clear that each v in the set above is a truncation of an element of Kp. Thus, F = с о j ' J ' v(T)dT I v G K p j- = F t о Hence (ii) is also proved. Part (i) follows from the inequality |v(t) | s X(t) for each v G Kp, where X G L j. Such a X exists. Indeed, take any b G Mc different than zero. Apply Corollary 1 to С = (C\Mc) u {a GMc | (a,b )ï 0}, Kp с Kp and Kp = {v G Kp |<(v(t), b)> = /3(t) s <(u(t), b> for each u G Kp }. Note that /3 is integrable. This shows that for each b f 0, b G Mc , <(v(t), b> is uniformly bounded by an integrable function if v GKp , which together with (iii) proves the existence of a uniform integrable bound for Kp and completes the proof of Lemma 2. Definition: We shall call v G Kp an extremal element of KP or extremal selector of P if there is a lexicographical order; that is a cone С such that C n (-C) = {0} and С U (-C) = R" such that u(t) s c v(t) a. e. in t (2.5) v G Kp is piecewise extremal if there is a partition t 0 =0 Lemma 3. If P(t) closed and F is compact extremal face of clco I(P) then for each x G F there is a piecewise extremal selector vx of P such that x = J vx(t)dt. Moreover, the number of subintervals on which vx is extremal о can be made not greater than the dimension of F plus one. In particular, F c I(P). IAEA-SMR-17/61 303 P roof. We shall prove Lemma 3 by induction with respect to the dimension of F. If dim F = 0, that is F = {e}, where e is an extreme point of clco I(P), then by C orollary 2 there is an extremal v such that I(v) = e. Suppose now that Lemma 3 holds for dim F < к and let dim F = k. Let x„ G F be arbitrary. Take any extremal selector v 0 in Kp (C is the cone corresponding to F) and consider function (2 . 6 ) x(t) = x 0 - J v 0 (T)dT t By Lemma 2 (iii), x{t)- Corollary 3. The integral I(P) of P is convex. Indeed, if v1( v2 G K P then P}(t) = (v jit), v 2 (t)} с P(t) is closed and clco I(P}) is com pact thus by Lemma 2 I(v¡) G clco I(Pj ) с ЦР-, ) с I(P), i = 1, 2, which proves convexity of I(P). Remark 2. Notice the very mild assumption for P(t) in obtaining these results. Actually, what was relevant that far was the following property of set K cLj: (P) if u, v G К and A is measurable then u Xa + vX[o цча g Manifestly, if К = KP that is К is the set of measurable selections of a set valued map then property (P) holds. This only property was needed in the proof of Lemma 1. For the remaining lemmas, we needed the closedness of Kp in Lj-strong topology which is the case if P(t) is closed for each t. We are in a position to state the basic results of this section: l Theorem 2 .1. If P is a map from [0, 1] into closed subsets of Rn and / P(t)dt is bounded then 0 t S(t) = / P (t)d t 0 < t S 1 304 OLECH is convex and compact, and it is Hausdorff continuous in t. Moreover if K s;, is the class of piecewise extremal selectors of P(t) then S(t) = i / v(t)dt v £ K ;, (2.7) 0 t Proof. S = clco Tp(t)dt is compact thus Lemmas 2 and 3 applied to S give the Theorem. 0 Theorem 2.2. (the unbounded case) If P is a map from f 0, 1 ] into closed l subsets of Rn then J' P(t)dt is convex and each compact extremal face of l 0 i l cl ^ P(t)dt is contained in J P(t)dt. In particular, if cl J P(t)dt does not 0 0 J 0 contain any extremal ray then J P(t)dt is closed. In the latter case, (2 . 7) also holds. l 0 Proof. Convexity of J P(t)dt is given by Corollary 3. The remaining parts о follow from Lemma 3 and proposition (s'). Remark 3. Theorem 2. 1 is an extension of well known Liapunov theorem concerning the range of vector-valued measure, stating that the range of finite and non-atomic vector measure is closed and convex. A special case of this is when the measure is given by ц{A) = J ' i(t)àt and the general case A essentially can be reduced to this case. Without any loss of generality, we may assume that the measure ц is given on the interval [ 0 , 1 ]; then we see that l (ju(A) I A measurable subset of [0, 1]} = J' P(t)dt where P(t) is composed of two points: f(t) and 0. On the other hand, the Liapunov theorem could be used to prove the convexity part of Theorem 2 .1 and to some extent also the compactness part. However, the second part of Theorem 2.1 cannot be obtained from the Liapunov theorem. This part of Theorem 1 is the reason for a rather detailed analysis of the integral of set-valued functions presented here and perhaps for a rather long proof of Theorem 2.1. 3. SOME APPLICATIONS; EXISTENCE; BANG-BANG PRINCIPLE As a first application of the integral of set-valued function we shall give a proof of Theorem 1.1. F or the sake of simplicity, we assume that to = 0 in this theorem, and, we shall write X(t) for X(t;0). Put IAEA-SMR-17/61 305 P(t) = {p|p = x'^tjBu, u e u } (3.1) t S(t) = J P(t)dt (3.2) 0 mt) = X{t) ( X 0 + S(t)) (3. 3) It is clear that the reachable set for (1. 1) defined by (1. 4) is contained iná?(t). By Theorem 2.1, however, we have t á%(t) = {x(t)(^x 0 + J' v(t)dt^ I v £ Kp piecewise extremal j- (3.4) о By definition, v is an extremal selector of P(t) if there is a lexicographical order S c such that (2.5) holds. Take w(t) = m axc P(t). It is easy to see that w(t) = maXj-IX'-^tíBui | 1 s i S k} where uL are vertices of U. Each of the functions X"1 (t)Bui is analytic. If we restrict ourselves to only these vertices of U for which Buj f Bu¡ if i f j, then function u(t) taking values from {ux, . . ., uk} and such that X _1(t)Bu s c X _1 (t)Bu(t) for each t > 0 and u G U, is uniquely defined and piecewise constant. Indeed, notice that С n ( - C ) = {0} and, therefore, by (a), there is an orthogonal sequence ax, . . ., an such that x E С iff the first non-zero member of the sequence < a i( x> is positive. Thus for each t, u(t) = u j(t), 1 á j(t) â к and < a x, X ' 1 (t)Bu j(t) )> = m a x {(a b X _1 (t)Buj> | 1 S j S k. This condition either defines u(t) uniquely everywhere except for a finite number of values of t or the maximum is attained at u¡ . .... u¡ on. an interval. The latter, , Ji* ’ Js ’ because of the analyticity of X (t) B, is only in the case where á?(t) =mt) where ^(t) is the reachable set corresponding to piecewise constant control function andá?(t) is convex, compact and continuous in t. The existence of a time-optimal solution follows from the fact that the set {t I х г e á?(t)} (3. 5) is closed and contained in [ 0, +°o]. Indeed, by continuity of the set {(x, t) I x eá?(t)} = graph ^ is closed. Hence the set (3. 5) is closed and there is t* = min{t| Xj€^(t)} which completes the proof of Theorem 1.1. Notice that continuity was used only through the relation that the graph of is closed. The latter property is less than continuity and it is called 306 OLECH upper semicontinuity in Kuratowski sense of set-valued function. There is another definition of upper semicontinuity (u. s. c.) of set-valued maps: A set-valued map P into subsets of Rm is u. s.c. if P 'G = {t I P(t) nG f 0} is closed for each closed subset G of Rm. In the case of P(t) compact and bounded uniformly (i. e. P(t) с P0 for each t, P 0 — bounded), the above two definitions are equivalent. Notice also that we did not use the convexity of ¿%{t). This property and continuity allows the conclusion th at if t* is th e optimal time then x xe 3^ (t*) which in turn leads to necessary conditions for optimal control. Theorem 1.1 can be easily extended to a more general situation. Namely to the case when the control system is of the form x = A(t)x + f(t, u) (3.6) where A(t) is an integrable matrix-valued function and f satisfies the Carathéodory conditions: (C) f(t, u) is measurable in t for each fixed u, continuous in u for each fixed t. However, in this case the admissible control function u will be measurable selections of a given set-valued map U from [t0, T] into subsets of Rm and such that f(t, u(t)) is integrable. Under those assumptions, for each admissible u the solution of (3. 6 ) satisfying given initial condition x(t0) = x 0 is uniquely defined and the reachable set is contained in (3.7) о where P(t) = X_1 (t)f(t, U(t)). To have the opposite inclusion, we need to check that if v(t) G P(t) and is integrable then there is an admissible u such that v(t) = X _:L(t)f(t, u(t)) on the interval in question. A positive answer to this question requires some properties of U. For this purpose we recall the notion of measurable set-valued mapping. Definition. We say that a set-valued map P from [ 0, 1] into subsets of Rn is measurable if the set P'G = {t |P(t) n G, f p) is measurable for each G с Rn closed. Notice that upper semicontinuity implies measurability. The following result is referred to often in the literature as Filippov lemma and gives condition under which (3. 7) holds. Proposition 3 .1 . If f satisfies Carathéodory condition (C) and U(t) is closed and measurable in t, then for any measurable v such that v(t) e f(t, U(t)) there is a measurable selector u of U(t) such that v(t) = f(t, u(t)). We omit the proof of this proposition but only explain two main steps in it. Firstly,one proves that the set-valued map V(t) = {u £ U(t) |f(t, u) = v(t)} is 1AEA-SMR-17/61 307 measurable. Secondly, one applies a selection theorem which states that measurable closed set-valued mapping admits measurable selection. The next proposition suggests one way selection theorem can be obtained: Proposition 3.2. Let P(t) be closed and measurable in t on [0, 1] and let the cone С induces a lexicographical order in Rn. Then v(t, C) = maxcP(t) is finite on a measurable set on which it is measurable and v(t, C) is integrable l on [ 0, 1] if and only if maxc cl J' P(t)dt is finite. о This proposition gives both; existence of measurable selectors as well as characterization of extremal elements of Kp if P is measurable. The proof of Proposition 3.2 is by induction argument. In fact, it is enough to prove that Pa(t) for fixed a £ Rn is measurable if P(t) is measurable. To state an analogue of Theorem 1.1 for system (3. 6 ) we would like to introduce the notion of an extremal solution of (3.6). Namely, x(t) on [t0, t] is an extremal solution of (3. 6 ) if x(tj) is an extreme point of the reachable set¿%(t) corresponding to the initial condition x(t0) = x0. Now Proposition 3. 1 and Theorem 2.1 implies: Theorem 3. 1. If f satisfies assumption (C), f(t, U(t)) and U(t) are closed and U is measurable in t and we assume that for each u(t) G U(t) measurable, f(t, u(t)) is integrable, then: (a) the reachable set (3. 7) for system (3. 6 ) is compact and convex and depends continuously on t. (b) the problem of passing from x 0 at time t 0 into a closed subset Z of [t0, +oo) x Rn in a minimal time has an optimal solution x..;(t). Moreover, there is also a time-optimal solution of the above problem, which is piece- wise extremal. x .;. (t) is unique iff x.;.(t.;.) ( t .;. the optimal time) is an extreme point of ¿5?(t.;¡). Notice the following characterization of extremal solutions: x(t) is an extremal solution of (3. 6 ) on the interval [ a, b] iff for each t1( t2 6 [a, b], tj < t2, and any other solution x(t) of (3. 6 ) the condition holds: x(t¡) = x(t¿), i = 1 , 2 , implies that x(t) = x(t) for t e [t 1 ,t2] For system (1.1) and U = {(u1,..., um)/ | u¿ | s 1, i = 1, . . ., m} the tim e- optimal control function u* has the property that |u~;(t) | = 1 for each t. This is so because the extremal control functions have that property. Thus the equality^?(t) = ¿%(t) which we have proved, holds also if for admissible controls £2 we take all piecewise constants u(t) such that |uj(t) | = 1 for each i. This can, in other words, be expressed as follows: each point which can be reached in time t from the initial state x 0 at time t 0 by means of a measurable control function (ua (t), ..., un(t)), I u¡(t) j sla . e. in[t0, t], can also be reached, in the same time, by a solution corresponding to a control function u(t) whose co-ordinates only take on the values +1 or -1. This phenomenon of jumping from one extremity to another has been given a colorful name of "bang-bang" principle. A generalization of this principle follows from Theorem 2. 1 and will be stated in the next theorem. Before we introduce the following property (compare with property (P)), we shall say that a class К of functions of a real variable is closed with respect to "piecewise" operation if 308 OLECH (P*) for each Uj, . . ., uk G К and t 0 = 0 < t 1 ...< t k = l the function u = Itj.j.ti) UiG К Theorem 3. 2. Under the assumptions of Theorem 3. 1, any point which can be reached from a fixed initial condition in a fixed time by an admissible solution of (3. 6 ) can be also reached — and in the same time — by a piecewise extremal solution of (3. 6 ). Moreover, if К denotes the class of all admissible solutions of (3. 6 ) with fixed initial condition (tfl, x Proof. The first part follows from Theorem 2. 1. To prove the second part, we notice that any subset К с К with the property that for each x 6 К there is x G K that xft-J = xftj) has to contain an extremal solution of (3. 6 ) because, if e = xft}) is an extreme point of the reachable set.í2(tj), then x(t) is the only solution of (3. 6 ) and e can be reached at time tj. This, together with (P*), implies К... с к . Theorem 3. 2 is, in a sense, the best version possible of the "bang-bang" principle. Notice that uniqueness of admissible solutions leading to extreme points of.^(t) does not necessarily imply uniqueness of the corresponding control functions. Notice also that extremal control functions u(t) are such that f(t, u(t)) takes values from the profile of clcof(t, U(t)), sim ilarly to the linear case discussed above, where extremal (and also optimal, in some cases) control functions take on values from the set of vertices of the cube U, thus from the profile of U. However, even in this simple case it may happen that to an extremal solution there correspond two different control functions. As another application of Theorems 2.1 and 2. 2, let us consider the following more general problem: Suppose that, in the class of admissible solutions of (3. 6 ) with fixed initial condition, we wish to find a solution such that (tx, x(tj;xo, t 0, u)) G Z for some t x> t0 and that the functional 4 J ( attains its minimum. This is a Bolza type of problem. Consider the reachable set.9?(t) for an extended system y = A(t) y + F(t, u), y(t0) = (x0, 0) (3. 8) where y = (x, x°) G Rn+1, A(t) is (n+1) x (n+1) matrix of the form: A(t), 0 A(t) = a°(t), 0 and F(t, u) = (f(t, u) f °(t, u)). IAEA-SMR-17/61 309 Under similar assumptions as in Theorem 3.1, we can prove that the reachable set is compact and continuous. The problem reduces to finding a minimum of the function 3>(t, y) = min / f(t, x(t))dt (3.9) in the class of absolutely continuous functions on [ 0 , 1 ] satisfaying the boundary condition x(0) = a, x(l) = b. This is a special case of the classical Lagrange problem from the calculus of variation (the integral does not depend on x(t)). Put P(t) = graph f(t, . ) = {(x, y) I y = f(t, x), x e Rn } and Q(t) = epigraph f(t, • ) = {(x, y) | y ê f (t, x), x € Rn } As to f, we make the assumption that both P(t) and Q(t) are closed and measurable in t. The solution of the problem exists if and only if the intersection of Q(t)dt with the line {(x, y) | x = b, y arbitrary} is closed 0 from below. A sufficient condition for the existence of a solution of this problem for any finite value b can be deduced from Theorem 2. 2 . A sufficient condition is that either the closure of the integral does not have 310 О LECH proper unbounded extremal faces or that it does not contain a ray different from that the parallel to the positive у -axis. Indeed, consider the inter section ^cl J' Q(t)dt) П {(x, у) I x = b} = {(x, у) I x = b, у § /3} (only the case f} 0 P is finite is of interest). The optimal solution exists iff (b,/3) 6 / Q(t)dt. But (b, /3) belongs to an extremal face F of cl /Г Q(t)dt which is transversal0 to 0 f direction of у -axis. Thus if F were not contained in / Q(t)dt then F as well as cl /Г Q(t)dt would contain an extremal ray not parallel0 to the positive y-axis о which contradicts the assumption. For both of these cases, we shall give analytic conditions in terms of the so-called conjugate function to f. First, however, we shall discuss the support function to a convex set. If S с Rn, then cps(p) = sup 5 РФ1 + (1 -X)P2 ) ^ X On the other hand, if cp(p) satisfies (i), (ii) and (iii) then This is a consequence of Lemma 1 of the previous section. In this case, we integrate the epigraph Q(t) then the integral S = J Q(t)dt has the о property that if (x, y) £ S then (x, z) 6 S for each z > y. But S is convex, thus for each x we can define g(x) = inf{y |(x,y) € S} IAEA-SMR-17/61 311 if the latter set is not empty and +00 otherwise. One can check that g(x) is convex and if Q is closed then g is 1. s. c. In this case, the support function if>s(p) o f S is +00 if the last co-ordinate of p is positive and cps (p , -1 ) = sup(-g(x) + 1 (3 .1 1 ) 0 where f* is the conjugate function to f(t, • ). By the effective domain of g we mean the set A = {p| g *(p ) < +°°} The set A is convex and we have the following proposition: (ri) L e t Q be the e p ig r a p h o f a co nv e x and 1. s . c . fu n c tio n g : R n -» R и {+ со}. Then p belongs to the interior of the effective domain of conjugate function g* iff the set G (p) = {(x, g(x))|-g(x) + is non empty and compact. Proof. G(p) is convex and closed because g is 1. s. c. If G(p) is unbounded then it contains a half-line, i. e. there exists x 0, y 0 6 R n, yo f 0 s u c h th a t -g(x0 + Xy0) + It is clear that for q such that s u p (- g (x 0 + Xy0) + < x 0 + Xy0, p + eq>) = +00 if only e > 0. Thus, g*(P + eq) = +00 if e > 0, hence p does not belong to the interior of A. On the other hand, suppose that G(po) is compact for some p 0 fixed and let r > 0 be such that F = {(x, y) I |x |2 + |y 12 = r, -y + is disjoint from G(p0) as well as from Q. Let e = min d(z, Q) and let z 6 F zo = (x0, g(x0)) be a point from G(p0). There is 6 > 0 such that d(z, Dp) S e/2 for each z e F and any p such |p-po | < à and Dp = {(x, y) | -y + ^x , p^> = -g(x0) + <^p, x0>}. If Dp П Q were unbounded, then because of the convexity of Dp n Q, we would have a contradiction with the definition of r in F. Thus, for each p such that |p-po | < 6, S = {(q, r) € Q | -r + {p|f*(t, p) is integrable} l is open, then the integral J' P(t)dt is closed, where о P(t) = epigraph f(t, • ) l In particular, there exists min J' f(t, x(t))dt over all x(t) absolutely continuous о x: [ 0, 1 ] -> R n such that x(0) = a, x(l) = b. A particular case of this situation is that where f*(t, p) is integrable for each p. In this case, we say that f satisfies the growth condition. Notice that this assumption means that "f(t, u) + e f(t, u) g |u I + фе (t) (3. 13) From (3. 13) it follows that, roughly speaking, f grows, with respect to u, faster than any linear function. A special case of the growth condition is that where f(t, u) ê Ф( | u |) and 3>(s)/s -» + oo as s -> oo. This corresponds to (3. 12) or (3. 13) with function фp o r фс bounded or simply constant. Thus, it is a uniform growth condition. If the growth condition holds, then the only ray l contained in the integral J' P(t)dt is that parallel to the positive у-axis. This о is the direction in which all P(t) are unbounded. The last theorem can be generalized to a more complicated situation when we wish to m inim ize i I(x , u) = J ( x = A(t) x + f(t, u), u eu (t) x(0) = a, x(l) = b IAEA-SMR-17/61 313 The solution of this problem exists if f and f° satisfy a Carathéodory condition i.e . f, f°are measurable in t and continuous in u, U(t) is closed and measurable in t, F(t, U(t)) is closed and for each e > 0 there is ф€ G L j such that e f°(t, u) 2 |f(t, u) I + (M t) i f u G U ( t ) F, as before, stands for (f, f°). 4. A NON-LINEAR OPTIMAL PROBLEM In contrast to the case considered up so far, for a non-linear control system, the reachable set may be not closed, even if the set of admissible values of the control parameter is compact. Let uë1 consider the following e x a m p le : x = l/(l+y2), y = u/(l+y2), u = 1 or u = -1 (4.1) Notice that dy/dx = u, and it is clear how the trajectories of Eq. (4. 1) look like. However, the reachable set of Eq. (4.1) is not closed and the time- optimal problem of passing from (0, 0) to (1, 0) has no solution. Indeed, the infim um is 1 because for each e > 0 there is a solution of Eq. (4. 1) such that |y(t) I S e, x(t£) = 1, y(t£) = 0. But we have: 4 t£ J' á (t)d t = 1 = J' ^ ^ 2 dt 2 t£, thus te S 1 + e2 о 0 On the other hand, x travels with a speed less or equal one, thus the o p tim a l time is bounded from below by 1. Therefore, the optimal time, if it e x is ts , has to be equal one. But that would be possible if and only if x(t) = 1 he nce i f y(t) = 0, which is impossible. Another example of the sort is the following Lagrange problem: M in im iz e 1 I(x ) = J (1 + x 2) ( |x2 - 1 I + 1 ) dt (4 .2 ) 0 over the class of all absolutely continuous functions satisfying the boundary c o n d itio n : x(0) = 0, x(l) = 0 (4. 3) The infimum of I(x) over x satisfying (4.3) is 1. Indeed, taking a polygonal line xn with slope + 1 o r - 1 we see that 1 = J ( l + x 2(t))d t 0 314 OLECH and it is clear that we can do it so that |xn| s e for arbitrary e > 0. On the other hand I(x) г 1 for each x. But the value 1 is not attained because that would be possible only if (1 +x2) ( 11 - x2 | +1) = 1. Thus if x(t) = 0 and x(t) =1. A contradiction. In both those cases, the lack of convexity is the reason for the non existence of an optimal solution. If, in the first case, we convexify the right-hand side, i.e. we take the convex hull of the set {(1/(1 +y2), u /l+ y 2))| I u I = 1} which comes to allowing u to be |u| s i, or, in the second case, replace f(x, u) = (1 +x2) ( |l-u2 | +1) by cof (x, u) = (1 +x2) (max (0, u2 - 1) + 1) which is the largest function convex in u and bounded from above by f, then the optimal solution would exist. Notice that in both cases we could define sequences xn(t) of admissible trajectories such that the value of cost function on xn tends to the infimum. Such sequence is called a minimizing sequence. In both cases, the minimizing sequence is uniformly convergent but in the first case the limit function is not an admissible trajectory and in the second the functional evaluated at the limit is greater than the infimum. In the following theorem we give a condition guaranteeing that neither of the above effects holds. Problem 1. Minimize the functional ti = J f°(t, x(t), u(t))dt (4.4) 0 over the class Г2 of admissible pairs (x, u) such that x is an absolutely continuous function, u-measurable and x(t) = f(t, x(t), u(t)) a.e. in[0, t-J - x(0) = a x(tj) G Ф(^) (4. 5) u(t) G U(t) — a given set-valued function where Ф is continuous function from [0, T] into Rn, and tj is not fixed but tj S T. Theorem 4. 1. (existence theorem) Assume that f °, f are continuous in x, u for fixed t, measurable in t for x, u fixed; U(t) is closed and measurable in t; the set Q(t, x) = {(q, r) |q = f(t, x, u), r ê f°(t, x, u), u G U(t)} (4.6) is convex and closed for each (t, x); for each t fixed Q is u. s. c. in x in Kuratowski sense; and the growth condition holds, that is sup (- r + <(p, q)0 = (p(t, p, x) S 4t(t, p) fo r each x (4.7) (q,r) 6Q(t,x) where ф is integrable in t, fo r each p G Rn. Under these assumptions there exists optimal solution of the problem (4, 4) - (4. 5); that is there are x*, u..., t* which satisfy (4. 5) and such that IAEA-SMR-17/61 315 ti J f^ x ^ ít), u*(t))dt Й J f°(t, x(t),u(t))dt о о for any other admissible pair (x, u). Proof. The proof of this theorem will be split in a few steps. First we notice that because of (4, 7) of I(x, u) is finite. Therefore we have a sequence xn(t), un(t), t" of admissible pairs such that 4 х » . u a) = f f u(t, x n(t), u n(t))dt -♦ a = infimum I(x, u) (x,u) e s; Put qn(t) = x n(t) = f(t, xn(t), un(t)), r n(t) = fC'(t, x n(t), un(t)). By (4. 6) we have (qn(t), r n(t)) 6 Q(t, x n(t)) and J r n( t ) d t - a (4.8) о We assume that xn and un are defined on the interval [ 0, T ] and thus also (4. 8) holds on [ 0, T ]. This is not much of restriction. It will allow us to avoid certain technical difficulties. By (4. 8) we get T J r n(t)dt s M < + 0 0 (4. 9) о We shall prove first that both qn and rn are bounded in Lx - norm. rn(t) is bounded because of (4. 9) and the inequality below obtained from (4. 9) and (4. 7) fo r p = 0: - r n(t) S ф (t, 0). Similarly, by (4. 7) and (4. 8) T T T T J dt + J rn (t) n(t)dt - J < p»qn(t)> - r n(t) - i//(t, p) s о (4.11) 316 OLECH Denote by р а, /lis and i/a , v s absolutely continuous and singular part of ju and v, respectively. The left-hand side of (4.11) is also converging weak*, and the lim it is because of the inequality (4. 11) a non-positive measure. Thus both absolutly continuous part and singular part of the lim it are non positive, too. Therefore from (4. 10) and (4. 11) we obtained inequalities: É i L í (t )\ _ Û J Ù L <(p,/ns (A)^> - vs (A) â 0 for each A measurable (4.13) Since (4.13) holds for each p thus ц 5(А) = 0 for each A, Therefore T T f w h e re q 0(t) = for each continuous £ which implies: t t x n(t) = a + J qn(t)dt-a+ J q 0(t)dt = x 0(t) (4 .1 4 ) о 0 We shall prove now that l i m sup Q(t, x) n {(q, r) |-r + x C P(t) n {(q, r) |-r + and by {ri) the latter set is compact. Thus if there is (qn, r n) 6 Q(t, x n(t)) and s u c h th a t (p{t, p, xn (t) ) = - rn+ < p ,q n> s <¡»(t, p, x 0(t)) then (qn, r n) containes a convergent subsequence and we may assume as well that (qn, rn) converges. Then the lim it (q0, r 0), since xn(t) -> x 0(t), belongs to Q(t, x 0(t)) because of u.s.c. of Q with respect to x. Thus we have (4. 15). It is clear from (4.15) and (4. 7) that putting i//n (t) = max( we have lim ^„(t) = It follows from (4. 16) that for each denumerable set {pj} there is a set N of measure zero such that - r 0 (t) + Since both sides of Eq. (4.16') are continuous in p, if {pj} is dense in Rn, then (4.16') holds for each p and t e [0, T]\N. Hence we conclude that (ЯоМ» ro W) € Qft, xo W) a-e- in[0, T] (4. 17) It follows from (4. 1 1 ) that for each e > 0 th e r e is 6 such that J |qn(t) [dt < e A if the measure of A is less than 6, which implies, in particular, that the convergence in (4. 14) is uniform and that qn-+ q 0 weakly in L r From this follows that there is t* equal to the lim it of t" or of a subsequence of it such th a t x 0(t*) = Ф (^). The regularity assumption on f and f° are enough to conclude that there is measurable u 0(t) (Proposition 3. 1) such that Q0(t) = x Q(t) = f(t, x 0(t), u 0(t)) and r Q(t) ê f° ( t, x 0(t), u 0(t)) and u 0(t) G U ( t) . O b v io u s ly , x 0(0) = a and, therefore, x 0(t), u 0(t) s a tis fie s (4. 5), hence is admissible. Now by (4. 10) and (4. 13) for each e > 0 there is 6 > 0 s u c h th a t i î T < e /4 о о < e /4 fo r n ë N (e) о 0 T T 0 D < e /4 for n ê N(e) о The above inequalities im ply t» J r 0(t)d t S a + e о But e is arbitrary, therefore we have t t I(x0, U0) = J f° ( t, x 0(t), u 0(t))d t ^ J r 0(t)d t 0 0 318 OLECH which shows that x0, u0, t* is optimal solution and completes the proof of the theorem. Theorem 4. 1 can be extended, without much additional difficulty, to the case of multiple integrals. Problem 2. Suppose we wish to minimize the functional I(x , u) = J' f °(t, x(t), u(t))dt for (x, u) G Q G where G is a bounded dom ain in Rm, x : G -> Rk and u : G -» Rn. The set Г2 of admissible pairs (x, u) is defined by the following conditions: x belongs to the Sobolev space HX(G, Rk), u is measurable and u(t) G U(t) a.e. in G, where U is given set-valued mapping, Vx(t) = f(t, x(t), u(t)) a.e. in G and the boundary value of x are fixed in the sense that x G x 0 + H j0(G, R k ), w h e re x 0G H x is fixed and H x0 is the subspace of H x which is obtained by closing in H j the set of C°°-function with compact support contained in G. Theorem 4. 2. Assume that f°,f are continuous in x, u for fixed t, measurable in t for fixed x, u, U(t) is closed and measurable in t, the set Q(t, x) = {(q, r) I q = f(t, x, u), r ë f°(t, x, u), u G U(t)} is convex and closed for each (t, x) and for each t fixed Q is u. s. c. in x in Kuratowski sense; sup (-r + q n(t) = V xn(t) = f(t, xn(t), u n(t)) rn (t) = f°(t, xn(t), u n(t)) where (xn, un) is the m inim izing sequence. In exactly the same manner we can prove that there is a subsequence of (qn, r n) (for sim plicity still denoted by (qn, r j) such that qn~* q 0 weakly in L a and J r n( t ) d t -> a * J r Q(t)d t. i G i G T he sp ace H x0 has the property that xn G H ^ 0 and Vxn converges weakly in Lj then xn converges strongly to a function x 0 in H x0 and Vxn converges IAEA-SMR-17/61 319 weakly to Vx0. So we have the lim it function, we see that q 0 = V x 0 and we conclude that ( q 0U ), r 0(t)) G Q(t, x 0(t)) The remaining part goes exactly in the same way or even more simply since here the integration is always over a fixed set G while in the previous case we allow the right end of the integration interval to be variable. 5. LOWER CLOSURE AND LOWER SEMICONTINUITY The optimal problem which we discussed in the previous section can have the following "control-free" formulation: Let Q be a set-valued map defined on [ 0, T] x R n into subsets of R n+1 . W e assume that Q(t, x) has the property that if (q, q 0) G Q(t, x) (q 0 is the last co-ordinate of a point from Q) and r > q 0 then (q, r) 6 Q(t, x). Let Q be a certain set of pairs (x, w), called admissible pairs, both being integrable functions from [ 0, T ] into R n. P r o b le m 3: T m in im iz e о This problem under the assumptions of Theorem 4. 1 concerning f°, f and the set-valued map U is equivalent to the optimal-control problem considered in the previous section in the sense that if we define Q(t, x) by (4. 6) and an integrable (w(t), v(t)) G Q(t, x(t)) then there is u(t) measurable such that u(t) G U(t) and w(t) = f(t, x(t), u(t)) and v(t) й f 0(t, x(t), u(t)). On the other hand, the above problem can be equivalently expressed as the Lagrange problem if we consider Q(t, x) to be the epigraph of a function g(t, x, • ). More precisely g(t, x, u) = inf {v 1 (u, v) G Q (t, x)} (5 .1 ) if the latter set is non-empty and +oo otherwise. If Q(t, x) is closed then it is the epigraph of g(t, x, • ) defined by (5.1). P r o b le m 4. T M inimize / g(t, x(t), u(t))dt о in a given class П of admissible pairs (x(t),u(t)). Under some mild regularity condition concerning Q, Problem 4 is equivalent to Problem 3 in the sense that a solution of Problem 4 is the solution 3 with the same Г2 and vice versa. The assumption sufficient for this is that g(t, x(t), u(t)) is measurable for each x, u measurable and that the infimum in (5.1) is attained. The latter is the case if Q(t, x) is closed while the first assumption holds if Q is measurable in the sense that the graph of Q is a ¿¿fxá^-measurable subset of [ 0, T] x R n x R n+1 w h e re S f is the Lebesgue б-field on [0, T] a n d á is the B o r e l 6-field in Rn x R n+1 . 320 OLECH For the existence of optimal solution the following two conditions are usually established: (Cj) The set T D = {(x, u, a) I (u(t), v(t)) e Q(t, x(t)) and a = J' v (t)d tj“ о is c lo s e d . (C2) There is a such that the set: Qa = -j (x, u) I Г2 I there is v such that (u(t), v(t)) G Q(t, x(t)) on [ 0, T] and T v (t)d t s a о is non-empty and compact. Both conditions were established in the course of the proof of Theorem4.1, with strong sequential convergence for x, and weak sequential convergence for u. It should be emphasized that the topology here is not given by a problem and any topology is good provided both conditions Cj and C 2 a re established. In fact, different topologies have been used. This shows also that one can have a great variety of existence theorems, even for the same type of problem. Condition Cj is equivalent to the so-called lower closure property of the orientor field: (u(t), v(t)) GQ(t, x(t)); that is the property that i f u n - » u 0, x n ^ x 0, J vn(t)dt -> a, and (un(t), vn(t)) G Q(t, xn (t)) then there is v 0 T 0 s u c h th a t J' v0(t)dt á a and (u 0(t), v0 (t)) G Q(t, x 0(t)). Condition С 2 w hen о translated in terms of function g means lower semicontinuity of the functional T I(x , u) = J' g(t, x(t), u(t))dt. Indeed, if a sequence or a generalized sequence о (xa, ua) converges to (x0, u0) then by Cj (x0, u0, lim inf I(xa, ua)) G D. But because of (5.1) the infim um of {a | (x0, u 0, a) G D} equals J g ( t ,x 0(t), о u о(t))d t = I ( x 0, u 0), th u s I( x q , u 0) S lim inf I(xa, ua) and vice versa. Condition d C 2 gives us compactness of the m inim izing sequence. The growth condition is responsible for this condition in Theorem 4. 1. Both conditions Cx and C 2 plus closedness of Г2 imply existence of a optimal solution to Problem 3. Of course, in a specific existence theorem, a topology or a mode of convergence in x-space and u-space has to be chosen and conditions Cx and C 2 have to be established with'respect to those topologies. In choosing topologies in x-space and u-space, we are faced with a conflict between Cx and C2. The weaker the topology is which we choose the more probable is IAEA-SMR-17/61 321 1+ t h a t C 2 would hold, while, for condition C lf the situation is opposite. As in the previous section, we shall here consider strong L i -topology for x and Lx-w eak sequencial convergence for u. There are two types of existence theorem s follow ing the above pattern: One is valid when condition C 2 h o l d s , for reasons independent on the structure of Q. For exam ple, if we beforehand know the topology in w hich f 2 is compact, then the only assum ptions we need to im pose on Q should be such that C x holds with respect to this particular topology. The other type is such that the properties of Q which we seek should im ply both C j and C2. Theorem s 4.1 and 4.2 are of this type. There, the growth condition gives C 2 while the convexity assum ption is essential (and the growth condition is very helpful) to obtain Cj , We shall prove that convexity of Q(t, x) is, in a sense, necessary for Cx to hold. To be more precise, we have the following: Theorem 5. 1. Let Q(t) be a set-valued map defined on the interval [ 0, 1] with values being subsets of R n+1 space w ith the property that if (q, q 0) G Q(t) a n d r > q 0 then (q, r) € Q(t), too (q 0 is the last co-ordinate of point in R nTl). Assum e that the graph of Q, that is the set {(t, v) 11 G [ 0, 1 ], v G Q(t)} is ^-m easurable subset of [0, 1] x R n+1. Then the following conditions are equivalent: 0 each u0, and (u(t), v(t)) G Q(t) has I.e . p. with respect to weak convergence i n L j . (ii) The set К = {(u, v) |u G L a([0, l] ,R n), v G L ¡ ([ 0, 1 ], R), (u(t),v(t))GQ (t)| is convex and closed in weak topology of L j and there is G L„ and ф G L j such that for each (u, v) G К -v(t) + (iii) Q(t) is closed and convex a. e. in [0, 1] and Q(t) = П {(q, qQ) |-q 0 + w h e r e Ф С L „ ( [ 0 , 1 ] , R n ) is not empty and denum erable and ф 6 L 2 f o r e a c h c p . ( i v ) Q ( t ) = { (q, q(| ) j q 0 s f ( t , q)} w h e r e f : [ 0 , 1 ] x R n -> R и { + 0 0 } i s У х â ë - m easurable, lower semicontinuous and convex in u for each fixed t and the functional 1 0 is w ell defined on L x with values from R U {+ 00} and w eakly low er sem icontinuous. Proof, (i) =»(ii). The I.e . p. of (u(t),v(t)) G Q(t) with respect to weak topology im plies, in particular, that the set К in (ii) is w eakly closed. In fact, if{u a, va} converges w eakly to (u 0,v 0) and (uy.Vy) f К then without any loss of generality we m ay assum e that 322 OLECH i 1 J v 0(t)d t < a = in f f v (t)d t| (u 0,v ) € kJ- (5 .4 ) о о Indeed, let A = {t | ( u 0(t), v 0 (t)) G Q(t)} and be such that (u0, у х) 6 К and J' v 1 (t)d t - a < e. Then, putting (ua(t), va(t)) = (u 0 (t), Vj (t)) if t 6 A and о (ua (t), va(t)) otherwise we obtain a sequence weakly convergent to (u0, v£ ), w h e re v e (t) = V j (t) if t G A and v 0 (t) if t A. If e is small enough (5. 4) holds for ve because J' (v 2 (t) - v 0(t))dt > 0. But (5.4) contradicts the [0 ,1]\A lower closure property. The closedness of K, together with the fact that К is the set of integrable selectors of a set-valued function, im plies that К is convex. Indeed, let wj, w 2 6 K and put w\ = Xwj + (1 -X)w2, X G (0, 1). Take any se q u e n ce <рг, . . . , 1i 1i В = . f ^k> dt) i w = w i X A + w2 x [0i1]na; A measurable j 0 0 By theorem 2, 1 it is compact and convex and it does contain points 1 l1 a4 . ==ÇJ ( 0 0 1 1 Therefore, it contains also Xax + (1 -X)a2 = (^ J <^wx , ip^dt, . . ., J ( w ^ , D = -j(u, a) I (u, v) G К, a = J vdtj is closed and convex and it follows о from (i) that it is bounded from below thus there is a -a + J <( ф (t) = ess sup (-v(t) + ^ (ii) =* (iii) There is a denumerable subset {wa} dense in Lj-norm topology in K. Put Q(t) = clco{wa(t)}. Obviously, if w € K then w(t) G Q(t) a. e. in [0,1], and vice versa, if w(t) 6 Q(t) a. e. in [0,1] then w G cl{wH} = K, where the closure is in L j -norm topology. If the latter were not true then we should be able to show that w(t) on a set of positive measure is outside Q(t). Now, two measurable set-valued functions have to be equal a.e. if they have the same set of integrable selectors. Representation (5.3) follows from (5. 2) (that Ф is non-empty) and from the following facts: let AN(t) be defined: AN(t) = {p I sup ( - q 0 + -f(t, u n(t)) + < Sim ilarly, as in the proof of Theorem 4.1, we deduce that vn = f(t, un(t)) contains a subsequence converging to a measure v and v 0 = dya/dt satisfies the inequality -v0(t) + i l a о Now, (5.5) together with (5.3), im plies that (u 0(t), v 0(t))GQ(t) a.e. in [0,1], therefore v 0(t) г f(t, u 0(t)). Hence, by (5. 6), I(u0) s lim inf I(un), which was to be proved. (iv) =» (i) The infim um in (i) is equal to I(u0) if Q is the epigraph of f as in the case if (iv) holds. From 1. s. c. of I(u) it follows that the set D = {(u, a) |u G L a, a ê I(u)} is closed. But D = {(u, a) | a = J' v(t)dt, v(t) г f(t, u(t)) a. e. in [ 0, 1 ]j-. Therefore, let (un(t), vn (t)) G Q(t) a. e. in [ 0, 1] for each n, and let un(t) -> u 0 weakly and J vn(t)dt s a0, then I(u0) s lim inf I(un) ê a . Hence v 0 = f(t, u 0(t)) 0 l is such that J v 0 (t)d t S a 0 and (u0 (t), v 0 (t)) G Q(t) a. e. in [0,1]. Hence the 0 lower closure property holds and the proof of Theorem 5.1 is completed. 324 OLECH It is clear that convexity of Q(t, x) is also a necessary condition for lower closure property of orientor field (u(t).v(t)) Q(t,x(t)) (5.7) when the convergence in u-space is Lj-weak. In the proof of Theorem 5.1 (part (ii) =*• (iii)), we used the following results concerning measurable set-valued maps without proving them: The closed set-valued map Q is measurable if and only if there is a denumerable sequence of measurable selectors {p¿} such that Q(t) = clfpjt)}. In Theorem 5. 1, we assumed that Q has ,immeasurable graph. This implies m easurability of Q in the sense of Definition 3.1 and is equivalent to if values of Q(t) are closed. We have concluded in the proof above that Q(t) = Q(t) a.e. in [0, 1] from the fact that the sets of measurable selections are the same. There we used the fact that Q(t)\Q(t) or Q(t)\Q(t) have graphs âfx^-measurable. Thus A = (t|Q(t)\Q(t) f f>] is Lebesgue measurable and if the measure of this set were not zero then there would be a selection w of Q such that w(t) $ Q(t) on A. There we made a use of the following selection theorem: If Q : [0,1] -> subsets of R n has S f x ^-measurable graph then there is measurable selection of Q. For Q measurable in the sense of Definition 3. 1, such a selection theorem holds provided the values of Q are closed. Theorem 5. 2. Assume that the map Q(t, x) of [0, 1] x R n into closed and convex subsets of R n+1 has the properties: (i) if (q, q 0) £ Q(t, x) and r > qQ then (q, r) £ Q(t, x), (ii) the graph of Q is Sé’y.âë -measurable subset of [ 0,1] x Rn x R n+1, (iii) for each fixed t the graph of Q(t, • ) is closed. Assume further that there is ф: [ 0, 1] -*■ R integrable and constant M such that for each integrable x: [ 0, 1] -*■ R n there is m easurable p: [ 0, 1] -» R n such that I p (t) I â M and - r + <(q, p(t) У s ф (t) + M j I x(t) I for each (q. r) £ Q(t, x(t)) Then (5. 7) has lower closure property with respect to strong convergence in L-j for x, weak sequential convergence in L x for u. If M x = 0 then the conclusion holds also for pointwise convergence for x. In this paper we shall not include the detailed proof of the above result but only mention the main steps. S upp o se x n(t) -» x 0(t) pointwise and in Lj-norm, un^ u 0 weakly in L j, J' vn(t)dt -* a and (un(t), vn(t)) £ Q(t, xn(t)) for each n. To prove the о n Theorem 5. 2 we need to show that there is v 0 £ L x such that / v 0(t)d t S a о and (u 0(t), v 0(t)) £ Q(t, x 0(t)). To the latter set-valued function we apply Theorem 5.1, parts (ii) and (iii). In particular, for Q(t, x 0(t)), (5.3) holds. Let cp £ Ф be fixed. Using the assumption of lower semicontinuity of Q(t, x), one can show that there is cpa £ L » , фп, ф £ L x such that -v„(t) + < ?n(t), un(t)> s^n(t) й ф ( t) l i m sup 0n(t) = ^(t) cpn -*• The above conditions im ply that there is v0 independent on J' v0(t)dt â a and -v0 (t) + < â ^(t) a.e. in [0, 1 ], which together о with (5.3) proves the Theorem. Thus, the main difficulties in the proof lie in the construction of l о 6. C O M M E N T S The existence theorem 1. 1 can be found in the by now classical paper [19] of LaSalle for U being a cube and also in book [35]. Earlier results are due to R. V. Gamcrelidze for U being one-dimensional and to Bushaw for two-dimensional systems. The exposition of Section 2 follows the author's papers on the subject [22-27]. Lemma 1 is an extension of a result obtained by Blackwell [7]. It can also be found in a different form in the paper by Borges [ 8]. The m ain difference between our method of obtaining Theorems 2. 1 and 2. 2 and that of other authors is contained in Lemma 3. The simple idea of the proof of this lemma is also responsible for the form of Theorem 2.1 more general than that usually to be seen in the literature; by this we mean the second part of it, which states that we may restrict ourselves to piecewise extremal selectors without changing the integral S(t). To prove this lemma, we use the fact that the Lebesgue integral on the interval [0, t] is continuous with respect to t. As has been noted in one of our first publications on this subject [23], working with a Lebesgue measure on an interval is not much of a restriction. What is really needed in the case of an abstract measure spac e (T, ц) is a one-parameter fam ily of sets Tt such that Tt С Ts if t < s and that, for each integrable function v,the integral v(T)d/u(T) is a continuous T, function of t. In fact, if (T,/u) does not admit atoms and the measure is finite, then such a family does exist. The proofs of the lemmas in Section 2 are different from those in the author's previous papers. They are more "co-ordinate-free". This seems to make them simpler. In particular, the proof of Lemma 1 is simpler. Notice that, there, we essentially prove the well-known result that exposed points of a convex closed set are dense in the profile of this set. For propositions concerning convex sets and their extremal structure, we refer to book [37] and also to a paper by Klee [17]. In particular, proposition (e) is due to Klee. There are reasons why the set of integrals of integrable selectors to a set-valued function is called the integral of the latter: One can define an integral of a set-valued function by im itating the definitions of the integral 326 OLECH of a single-valued function. Many of such definitions have been given and it appears that, in general, they are equivalent to the one we used here and which sometimes is referred to in the literature as the Aumann integral [2]. For further references and discussion of the integral of set-valued mapping, we refer the reader to paper [27]. Concerning measurable set-valued maps, selection theorem and related question, the literature is quite extensive. The interested reader may look up the lecture notes by Parthasarathy [ 33]. The paper of Rockafellar [ 38] on the subject is well written and is a good source of information. In particular, the characterization of a closed set-valued map used in one proof of Theorem 5. 1 (part (ii) =» (iii)) (see comments following the proof) has been taken therefrom. In the same place, we have used a recent selection theorem found by Aumann [3] . Proposition 3. 1 called often the Filippov lem m a [14] has been also proved and used by Wazewski [ 44]. Proposition 3. 2 for compact set-valued maps is proved in Ref. [ 24]. One of the first generalizations of the existence theorem for linear systems of form (1 . 1 ) to non-linear systems in the control parameter of f o r m (3. 6) is due to Neustadt [21]. Theorem 3. 1, in that generality, is essensially due to the author and can be found in Ref. [ 25]. In the same paper, the extension of the "bang-bang" principle stated in Theorem 3. 2 can be found. This theorem can be used to obtain a generalization of Theorem 1. 1 to systems of form (3.6). We mean to formulate and to prove the existence and/or the "bang-bang" principle in the class of piecewise continuous control functions. Such a result was given by Levinson [20] and Halkin and Hendricks [15]. For Theorem 3.3 and sim ilar results and applications, we refer to Refs [26, 27]. The latter paper contains some more references and discussions and could be consulted in connection with the m aterial contained in Sections 2 and 3. Among more recent publications on this subject we would like to mention Berliocchi and Lasry [ 6], W a g n e r and Stone [41], Ar stein [1], and Ce sari [13]. A general reference for time- optimal control problems is a very nicely written monograph [16] by Hermes and LaSalle. The first existence theorem for non-linear control problems is due to Filippov [14]. It is concerned with the time-optimal problem and deals with the case where the reachable set is compact. Sim ilar cases were also treated by Roxin [40]. These results are connected with (or follow from) the theory of contingent equations developed in the thirties by Zaremba and Marchaud. It was Wazewski who noticed and explored this connection giving a foundation for, as he called them himself, orientor fields and which also is known under other names as differential equations with multi-valued right-hand side, differential inclusion and some others. In Ref. [43], the interested reader can find the basic theory of orientor fields of Wazewski and references to his papers. Ce sari was the first to investigate the unbounded case. He connected the Tonelli-McShane-Naguno existence theory with optimal control problems and obtained a series of quite general results. His contribution to existence theory cannot be overestimated. A selection of the multitube of papers which he published on the subject in the last ten years is included in the list of references. In particular, existence results of the type of Theorem 4. 1 can be found in Ref. [9] with uniform growth condition. This theorem also appears in the author's paper [28] and the proof is taken from that paper. IAEA-SMR-17/61 327 For Theorem 4. 2 we refer to Ref. [ 29]. In the papers quoted the reader will also find some other existence theorems. For various formulations of optimal control and its relation with classical variational problems, see Rockafellar's work, in particular Ref. [ 39 ]. Among others, integral functionals with the integrand assuming also the value +oo are treated systematically in an elegant and convincing way in this paper. Theorem 5.1 was stated in Ref. [31]; it generalizes some results contained in Poljak's paper [34]. In the latter paper, also the necessary and sufficient condition for Lj-weak sequential lower semicontinuity is obtained (equivalence between (ii) and (iv)) only for f(t, q) continuous in q, while here we are able to prove it only by assuming that it is Borel- measurable. The proof of Theorem 5.1 appears here for the first time. For related results which are, however, concerned with different weak topology, see the author's papers [30, 31]. Theorem 5.2 was also stated inRef.[31], and the detailed proof will appear in Ref. [ 32]. This theorem is essentially due to Berkovitz [ 5] and Cesari [ 10]. Our formulation is slightly more general, and the proof is different. In this contribution, we were not able to cover all results connected with existence theory. The two existence theorems in Section 4 and two results concerning lower closure in Section 5 are only examples. Moreover, also the list of references is far from being complete. The important theory of generalized solutions of Young or sliding regimes of Gamcrelidze or relaxed controls of Warga, which is very closely related with the existence of optimal solutions, has not been mentioned, at all. We refer the reader to monographs by Young [45] and Warga [42]. The idea of starting this paper with the Perron paradox was borrowed from Ref. [45]. Finally, we would also like to mention a recent book by Ioffe and Tichomirov [ 36], where the existence of optimal solutions is treated in some detail. Many more references to this subject can be found there. REFERENCES [1 ] ARTSTE1N, Z . , On a variational problem, J. Math. Anal. Appl. 45(1974)404-415. [2] AUMANN, R.J., Integrals of set-valued functions, J. Math. Anal. Appl., 22(1965) 1-12. [3] AUMANN, R.J., Measurable utility and the measurable choice theorem, Proc. Int. Coll. C.N.R.S. "La Decision", Paris(1969) 15-26. [4 ] BERKOVITZ, L .D ., Existence and lower closure theorems for abstract control problems, SIAM J. on Control, 12(1974), 27-42. [5 ] BERKOVITZ, L .D ., Lower semicontinuity of integral functionals, to appear in Trans. A .M .S . [6 ] BERLIOCCHI, H ., LASRY, J .M ., Intégrandes normales et mesures paramétrées en calcul des variations, Bull. Soc. Math. France 101 (1973). [7] BLACKWELL, D ., The range of certain vector integrals, Proc. Am. Math. Soc. 2 (1951) 390-395. [8] BORGES, R., Ecken des Wertebereiches von Vektorintegralen, Math. Annalen 175 (1967) 53-58. [9 ] CESARI, L ., Existence theorems for weak and usual optimal solutions in Lagrange problems with unilateral constraints, I and II, Trans. A .M .S ., 124(1966) 396-412 and 413-470. [10] CESARI, L ., Lower semicontinuity and lower closure theorems without seminormality conditions, Ann. Mat. Pura Appl. 98 (1974). [11] CESARI, L ., Closure theorems for orientor fields and weak convergence, Arch. Ration. Mech. A n al., to appear. [12] CESARI, L ., LA PALM, J.R ., SANCHEZ, D .A ., An existence theorem for Lagrange problems with unbounded controls and a slender set of exceptional points, SIAM J. Control, 1 (19 7 1 ), 590-605. [1 3] CESARI, L ., An existence theorem without convexity conditions, SIAMJ. on Control, 12 (1974) 319-331. 328 OLECH [14] FILIPPOV, A .F ., On certain questions in the theory of optimal control, Vestnik Moskov. U niv., Ser. Math. Astron. 2(1959) 25-32. [15] HALKIN, H ., HENDRICKS, E .C ., Sub-integrals of set-valued functions with semianalytic graphs, Proc. Nation. Acad. Sc., 59(1968) 365-367, [16] HERMES, H., LASALLE, J.P., Functional Analysis and Time Optimal Control, Academic Press (1969). [17] KLEE, V ., Extremal structure of convex sets, Arch. M ath., 8 (1957) 234-240. [18] KURATOWSKI, K ., RYLL-NARDZEWSKI, C ., A general theorem on selectors, Bull. Acad. Pol. S ci., Ser. Sci. Math. Astron. Phys. j>(1965) 397-403. [19] LASALLE, J .P ., The tim e optimal control problem, Contr. to the theory o f nonlinear oscillations, 5, Princeton Univ. Press, Princeton (1960) 1-24. [20] LEVINSON, N .. Minimax, Liapunov and "bang-bang", J. Diff. Eq. 2(1966) 218-241. [21] NEUSTADT, L. W ., The existence of optimal controls in the absence of convexity conditions. J. Math. Anal. Appl. 7 (1963) 110-117. [22] OLECH, C . , A contribution to the tim e optimal control problem, Abhandlungen der Deutschen Akademie der Wissenschaften zu Berlin, Kl. Physik und Technik 2 (1965) 438-446. [23] OLECH, C ., Extremal solution of a control system. J. D iff. Eqs. 2 (1966) 74-101. [24] OLECH, C., A note concerning set-valued measurable functions, Bull. Acad. pol. S ci., Ser. Sci. Math. Astron. Phys. 13 (1965) 317-321. [25] OLECH, C ., "Lexicographical order, range of integrals and bang-bang principle” , Math. Theory of Control (BALAKRISHNAN, A .V ., NEUSTADT, L. W ., Eds), Academic Press, New York (1967) 35-37. [26] OLECH, C ., "Integrals of set-valued functions and linear control problems", IFAC Congress Warsaw, 1969 on Optimal Control, Technical Session 7, 22-35. [27] OLECH, C ., Integrals of set-valued functions and linear optimal control problems, Colloque sur la Théorie Mathématique du Contrôle Optimal, C.B.R.M ., Vander Louvain(1970) 109-125. [28] OLECH, C ., Existence theorems for optimal problems with vector valued cost function, Trans. Am. Math. Soc. 136 (1969) 159-179. [29] OLECH, C ., Existence theorems for optimal control problems involving multiple integrals, J. Diff. Eqs j5 (1966) 512-526. [30] OLECH, C ., The characterization of the weak closure of certain sets of integrable functions, SIAM J. Control 12 (1974) 311-318. [31] OLECH, C . , Existence theory in optimal control problems - the underlying ideas, to appear in the proceedings of a conference held at the University of Southern California, Los Angeles, September 1974. [32] OLECH, C ., Weak lower semicontinuity of integral functionals, to appear in ]. Optim. Theor. Appl. [33] PARTHASARATHY, T ., Selection theorems and their applications. Lecture Notes in Mathematics 263, Springer-Verlag (1972), [34] POLJAK, B .T., Semicontinuity of integral functionals and existence theorems for extremal problems, Mat. Sbor. 28(1969) 65-84, (in Russian). [35] PONTRYAGIN, L .S ., BOLTYANSKI, V .G ., GAMCREUDZE, R .V ., MISHCHENKO, E .F ., The Mathematical Theory of Optimal Control, Moscow (1961) in Russian. English translation: Interscience, New York (1962). [36] IOFFE, A .D ., TICHOMIROV, W. M ., Theory of extremal problems, (in Russian) Moscow (1974). [37] ROCKAFELLAR, R .T ., Convex Analysis, Princeton Univ. Press(1969). [38] ROCKAFELLAR, R .T ., Measurable dependence of convex sets and functions on parameters, J. Math. Anal. Appl., 28(1967) 4-25. [39] ROCKAFELLAR, R .T ., Existence theorems for general control problems of Bolza and Lagrange, to appear in Advances in Math. [40] ROXIN, E., The existence of optimal controls, Michigan Math. J. J3(1962) 109-119. [41] WAGNER, D .H ., STONE, L .D ., Necessity and existence results on constrained optimization of seperable functionals by a multiplier rule, SIAM J. Control 12 (1974) 356-372. [42] WARGA, J., Optimal control of differential and functional equations, Academic Press, New York (1972). [4 3] WAZEWSKI, T ., On an optimal control problem, Proc. Conf. D iff. Equations and their Applications, Prague (1964) 229-242. [44] WAZEWSKI, T ., Sur une condition d*existence des fonctions implicites mesurables, Bull. Acad. Pol. Sci., Ser. Sci. Math. Astron. phys. ¿(1961) 861-863. [45] YOUNG, L .C ., Lectures on the Calculus of Variations and Optimal Control Theory, W.B. Saunders Company, Philadelphia-London-Toronto (1969^. IAEA-SMR-11/48 ASYMPTOTIC CONTROL R. C O N TI Istituto Matematico "Ulisse Dini”, Università degli Studi, Florence, Italy Abstract ASYMPTOTIC CONTROL. Asymptotic control is discussed in the framework of general control theory, special emphasis being placed on stability (including bounded-input bounded-state stability), affine control systems and stabilization problem s. 1. PRELIMINARIES We shall first recall a few more or less well-known facts about "linear" ordinary differential equations, so as to render this paper as self-contained as possible. Let us denote by: J = ] a , w [ an open interval of the real line E , with -« S a< u S + ю; A : t -► A(t) an nX n matrix function of t G J, Lebesgue measurable and locally integrable on J; X : t -* x(t) an n-vector function of t G J, continuous on J. Given any 0 G J, the Volterra integral equation t x(t) = x(t) + Í A(s) x(s) ds u e has a single solution x : t - x(t) defined on J by x(t) = lim X(t) + / A (t1) x(tx) dtj + . . . к к -1 . + A (t1) . . . A (tk) x(tk) dtk. . . dt 1 W h e n x(t) = x , a constant n-vector, this can be written x(t) = lim к t lk -1 . . . +J . . . J A ftj) . . . A(tk) dtk . . . dtx ( 1 . 1 ) e e where I denotes the n X n unit matrix. 329 330 CONTI The function t -» x(t) defined by (1.1) satisfies the condition x(fl) = X (C) It is locally absolutely continuous on J and such that - A(t) x(t) = 0 , a.e. t G J dt Therefore, we shall call it the (Carathêodory) solution of the linear ordinary differential equation x - A(t) x = 0 (E 0 ) satisfying (C). Since the lim it appearing in Eq. (1.1) exists for an arbitrary 0 we can define the nXn matrix t G(t, 0) = lim к + J ■ ■ ■ ] A ( t 1)...A(tk)dt1( ...d t 1 (G) в e which is called the transition matrix of (E ). Then we can replace (1.1) by the more compact form ula x (t) = G(t,0) x(0), t,0 G J (1.2) From (1.2) it is easy to derive the algebraic properties of G: G ( t , 0 ) G ( 0 , t ) = G(t,T) , 0, t ,t G J (1.3) G(t,t) = I, t G J (1.4) G 1 (t, 0) = G(0,t) , 0,t G J (1.5) and the differential properties ££% £). - A(t) G(t, 0) = 0 , 0 G J , a.e. t G J (1.6) dt 9G(h 8) + G(t,0) A(0) = 0 , tGJ, a.e .ie J (1.7) o Í7 The norm of the m atrix G(t, 0) is the number I G(t, 0 ) I = sup "jj G(t, 0) x |2 : í X |2 S If (1.8) w h e re | |2 is the Euclidean norm in lRn. IAEA-SMR-17/48 331 It is easy to show that (t, 6) -* G (t, 6) is a continuous function on JX J with respect to the norm. By virtue of (1.2), the number (1.8) is equivalent to I G(t, б) I = sup jj x(t) I : x € E, < ( 4 (1.9) w h e re E 0 denotes the set of solutions of (E0). Let us now recall some inequalities to be used later on. Let us denote by XH(t) and MH(t) the least and the greatest eigenvalue, respectively, of the Herm itian matrix H(t) = I A(t) + ^A*(t) (1.10) where A* is the transpose of A. Then it can be shown that t XH(t) and t -*■ juH(t) are measurable and locally integrable on J and we have t t Г 1 XH(s)ds Jp„(s)ds e® s G(t, 0) s# , 0 s t 1 11 t t ( . ) -ÍM H(s)ds - / s)ds 0 S |G(e,t) s ee , 0 S t From these inequalities it follows that I G(t, 0)| - / IH(s) I ds / 1 H(s) I ds s ев S t ( 1 . 12) I G(0, t) I and t - / 1 A(s)| ds |G (t,e)| / 1 A(s) I ds e 6 ■ s S t (1.13) I G(0, t) I For a fixed 9, (1.2) represents an isomorphism between ]Rn and the s e t E 0 of solutions of (E0), so that £ 0 is a (real) vector space of dimension Let us denote by X : t -> X(t) any n X n m atrix function whose columns x 1, . . .,xn are solutions of (E0). From (1.2) we have X(t) = G(t, 0) X(0) , t, 0 6 J hence det X(t) = det G(t, 0) det X(0) Since det G(t, 0) f 0 we have that either X(t) is non-singular for every t £ J or it is singular for every t e J. In the first case, we can write 332 CONTI G (t,9) = x(t) x'^e ), t, 0 G J This yields easily t / tr A(s)ds d et G (t, 0) = ее t, e e j (1.15) The equation z + A * (t) z = 0 (E*) is the adjoint to (EQ ). Using (1.14) it is easy to show that the transition matrix Г of (EQ) is given by r ( t , 6) = G*(0,t) = G*'1^,©) t, 0 e J (1.16) Equation (E ) is said to be autonomous when A is independent of t. In this case J = R and G(t,0) = e(t'e)A, t, € И (1.17) It follows that the solutions of (E0) are analytic functions of t E R . More precisely, let . . . ,Xk denote the distinct eigenvalues of A, np nk their respective multiplicities. Then for every x G Œn there are X . . ., xk 6 ŒN such that etA x ' Z eX|,[Í ? : 2. STABILITY I From now on, we shall assume w = +oo, i.e. J = ] a , +oo [ , -oo É a , and we shall study the behaviour of the solutions of x - A(t) x = 0 (E0) as t-* +00. B e c a u s e o f x(t) = G(t,0) x(0) (1.2) this amounts to studying the behaviour of G : (t, 0) -* G(t, 0) as t -» +oo. As a m atter of notation, we shall write x(t, 0,x)instead of x(t) to denote the solution of (E ) such that x(0) = x, so th a t ( 1 .2) w ill be replaced by x(t, 0, x) = G(t, 6)x (2.1) IAEA-SMR-17/4S 333 It is w ell Imown that the zero solution of (E0) is stable (according to Liapunov, as t -» + 00) iff for every e> 0 and r> a there exist 6 = 6 ( т , е ) s u c h t h a t x|2< 6 , a < t É t => |x(t,r, x)|2 < e (2.2) It is easy to see that this holds iff G has the property S: for every t > a there exist у(т) > 0 s u c h t h a t a < t â t => |G(t, t ) j < 7(t) ( 2 . 3 ) In fact, if (2.3) holds, for any e> 0, т > о we have | x ( t , T , x )|2 = |G(t,r)x|2< yM |x |2 , a < t é t i f ] x 12 á (T»e) = t'V)^. Conversely, if (2.2) holds, for e = 1, r> a, there exist 6 = 6 (t) > 0 s u c h t h a t I X |2 < <5 , a < T S t => I G ( t , t ) x I g < 1 w h e n c e a < t é t =* I G(t, t ) I < 6 ~1 ( t ) Property S m eans that for every fixed т > a the function t -* | G (t, т ) | is bounded for т á t. A m ore restrictive condition is satisfied when for e v e r y в > a the function (t,T)-> | G ( t , т ) | is bounded for S lrS t . T h i s can be expressed by saying that G has the property U S: for every в > a there exist 7 (6 ) > 0 s u c h t h a t в й т é t ¡G(t,T)¡ < 7 (0 ) ( 2 . 4 ) It is readily seen (by the same argum ents used to prove the equivalence between stability and property S) that property U S is equivalent to the uniform stability of the zero solution of (E 0 ) as t -► + 00. This means that for every e > 0, 0 > a , there exist 6 = 6 (0 , e ) > 0 s u c h t h a t ( 2 . 5 ) Another property of G more restrictive than property S is property AS: fo r e v e ry т > a we have lim j G(t, t) j = 0 ( 2.6) Clearly, this is equivalent to t > a , x G ® - n =* l i m | x ( t , T , x )|2 = 0 (2.7) + ' i.e . to the asym ptotic stability of the zero solution of (E 0) as t -*• + 00. 334 CONTI The zero solution of (E0) is said to be exponentially asymptotically stable as t-* +00, when (2.7) is reinforced by « < e s t s t, x e iR n =» |x(t,T,x)|2 s 7 (0) е 'р(0)(‘ " т> I x ! 2 (2 . 8) fo r s o m e 7 (0) > 0 and ц ( в ) > 0. This is clearly equivalent to property EAS of G: for every 0 > a th e re e x is t 7 (0) > 0, p ( 0) > 0 s u c h th a t » < 0 S t S t => I G(t,t ) I < 7 (0) е "^(0) (t ” T) (2.9) Looking at the definitions we see immediately that the following implications among the four properties S, US, AS, EAS of G are valid: U S ^ E A S S AS None of these are reversible and the two properties US, AS are independent of each other, as examples show. To this effect, let us consider the scalar (n = 1 ) e q u a tio n * - § f * = 0 (2 .10 ) where f : t ->■ f(t) is a function defined for t > a , positive and locally absolutely continuous. We immediately see that G is given by G (t, t ) = f (t) / f(r) E x a m p le 2 .1 L e t a = 0 and let f(t) = 1/t for 0 < t s 2, while for t a 2 the graph of f is the polygonal with subsequent vertices at (2,1/2), (3,1), (4,1/4), (5,1), . . . Such an f is bounded but it does not tend to zero so that G has the property S but not AS. Nor has it property US since G(2k + 1,2k) = 2k, к = 1 , 2 , . . . Example 2.2 L e t a = -o o , f(t) = 1 + e’1. Then G has property US, but not AS. Example 2.3 -2 + cos t L e t a = 0, f(t) = t . G has the property AS but not US, since G(2kTT, (2k- 1 ) 7Г ) = тг2 (2k - l )3 / ( 2 k ) , к = 1, 2, . . . IAEA-SMR-17/48 335 Ex.2.1 Ex.2.3 Ex. 2.2 -E x. 2.4 Ex.2.5 F IG .l. Relationship of properties S, US, AS and EAS. Example 2.4 L e t a = 0, f(t) = 1/t. Then G has both properties US and AS but not EAS. Example 2.5 L e t a = -oo, f (t) = e_t. G has the property EAS. What we have said can be visualized by a scheme showing the implications among the properties S, US, AS, EAS as is shown in Fig. 1. W h e n (E 0 ) is autonomous, then it can be shown that properties S and US coincide and are characterized by: a) all the eigenvalues of A have real p a r ts s 0, and b) those eigenvalues Xj which have zero real part (if any) have a m ultiplicity n^ equal to n-rank (A - Xj I). Furthermore, properties AS and EAS coincide and are equivalent to: c) all the eigenvalues of A have real parts < 0 (strictly). 3. STABILITY, II Let us now consider a property of G of integral type, namely property IS: fo r e v e r y в > a let there exist k(6) > 0 s u c h th a t (3.1) e This property is independent of property US. In fact, the G of Example 2.4 is defined by G(t, s) = s/t, so that it has property US but not IS since e On the other hand, the G of the next example has the property IS but does not have property US. 336 CONTI Exam ple 3 .1 L e t a = -oo and let X : t -» X(t) be a locally absolutely continuous function of t E , j 1 everywhere except on the intervals J k = [k- 2'4k, k + 2'4kl 6 Ok ^ 9U 1 . . . - ^ . w h e re 1 g X (t) s 2 and X(k) =4[2 . If we take Eq. (2.10) w ith f(t) = e /X (t) we have G(k + 2“4k,?-4k k) -= o2k22k ^-2'4ke“2 -» +oo as k-» +oo, so that US does not hold. On the other hand, for 0 < 1 - 2"4 we have t t t [t+i] J |G(t, s) | ds = f X(s) e s ds s e '1 J es ds + V J X ( s ) d s в в в к i i k S е‘* (е‘ - e e) + 2 У 22к2‘ 4к й 1 + 2/3 к=1 so that G has the property IS. Example 2.3 shows that AS IS (3.2) In fact, since cos s s 1 we have s 2 " cos s > s, hence for t s в > 0 I ,i . . -2 + cos t / 2 - cos s , G(t, s) ds = t / s ds I t ~2 + cos t l ' s d s = tcost {1 . (g/t)2 )/2 > so th a t 2kir J'j G(2k 7r, s) I ds = k 7r (l - (0/ 2к 7г)2 ) -* +oo as к -*■ +oo. e On the other hand, we have IS =*AS (3.3) To prove this, let 0 > a and let 0 1 = T ? r l(e +0)/2 if iï> -ю Let, further, t i//(t) = J 4 > [ s) ds, 0' S t (3.5) e' IAEA-S MR-17/48 337 so that ф (^ (s) ds ) G(t, t ) ip (s) G(t,T) ds t G (t, s) ds < k(0 ') by virtue of property IS. Owing to the definition of 0 ', we can replace k (0 ') by k(0) and write (Ht) From this follows (d/dtl0(t) e k *0^ 4 ^ a 0, 0 s т s t, hence, integrating between 0 and t ф(t) e 'k' 1(e)(t-T) й ф(в) th a t is i//_1(t) ê ф~1 (в) е‘к1(в)(‘ -е) S ф 1 (0) e к т\ 0 S т S t Since (3.4), (3 .6) g iv e I G (t, t ) I = ^ ( t ) S k(0) ^ ( t ) w e have j G (t, r ) I s k(0) ф '1 (в) e‘kl(0)(t"r) (3.7) from which property AS follows. It should be noted that (3.7) does not im ply property EAS since ф '1 (0), by definition, depends on т . However, if G has also the property US then fo r e v e r y 0 > a there exists y 0(6) > 0 s u c h th a t 0 S t S t -* I G (t, t ) I < 7 q(9) 338 CONTI w h en ce 0 ' a r â t => cp( t) = | G (t,r) I _1 > 7' 1(e) so th a t i//(t) = f In particular, ф(в) > -у"1 (б ') ( в - Q ') , th a t is 0_1 (0) < y Q (б ' ) / (0 - 9 '), and by definition of 0 ф~1(в)<у1 (в) (3.8) w h e re 7 X(0) > 0 is in d e p e n d e n t o f т . From (3.8) and (3.7) it follows 0 i t S t =» I G(t,T)| s k (0 )7 1(6) e k ^ ^ that is, property EAS holds with 7 (0) = k(0) 7 (0), ц(в) = k ‘ 1 (0). W e have thus proved IS plus US =*EAS (3.9) The converse is also true. In fac, EAS-* US is obvious. Also, if EAS holds we have t t J' I G (t, s)| d s < 7 (в) j ' e _,J(e) ( t" s) d s = 7 (0) /u'1 (0) [ 1 - e^^Ht-S)] 0 0 that is, IS holds with k(0) = 7 (0) /u_1 (0). T h e r e fo r e IS plus US«=*EAS (3.10) and the scheme of Section 2 is now completed as is shown in Fig. 2. 4. STABILITY,III We shall now try to characterize property EAS, i. e. the exponential asymptotic stability of the zero solution of x - A (t) x = 0 (E0) by means of a "Liapunov function". We start with the case of a constant A. IAEA-SMR-17/48 339 FIG.2. Relationship of properties S, US, AS, IS and EAS. Definition 4.1 « We say that the constant m atrix A admits a Liapunov function iff there are two nX n matrices Л, Q which are positive definite: Л = Л*>0 (4.1) Q = Q* > 0 (4.2) and such that А * Л + Л A = -Q (A) When this happens the quadratic form X - X*Ax is a Liapunov function of A and (Л) is a Liapunov matrix equation associated with A. We want to prove Theorem 4.1 If A has a Liapunov function, then G has the property EAS. P r o o f . L e t y(t) = et(A + tjI> x (4.3) w ith ц > 0, so that dy(t)/dt - (A + juI) y(t) = 0. From (Л) it follows d y* (t) A y(t)/dt = y* (t) [ -Q + 2 ¡лA ] y(t) If we denote by ц Л the greatest eigenvalue of A, from (4.1) we have Х*Л х s Мл [ X12 340 C o n t i Also, denoting by XQ the least eigenvalue of Q from (4.2) we have -x*Qx S -XqUIj T h e r e fo r e , y*(t) [ -Q + 2 ЦА] y(t) s [ -X.Q + 2 W a ] |y(t)|® and i f ^ < Й 2" (4-4) we have dy*(t)Ay(t)/dt < 0, whence integrating between 0 and t y*(t) A y(t) < x*Ax But again from (4.1) we have lyWla - y * (t) Ay(t) w h e re XA is the least eigenvalue of Л and i i2 X,|CAX Ê й л I X|2 T h e r e fo r e ^lyW lz < ^ N 2 and (4.3) gives HA. p -i* К for n satisfying (4.4). R e m a r k 4.1 Inequality (4.4) says that the exponential decay of the solutions of (E0) is not faster than - r 2- 4 t -*■ e ,л The converse of Theorem 4.1 is also valid. Theorem 4.2 If G has the property EAS then A has a Liapunov function. Proof. By assumption lAEA-SMR-17/48 341 I e tA| < 7 e~pt , te К (4.5) fo r s o m e у > 0 and ц > 0 . On the other hand, for x e E n and any Q = Q* > 0 X* I I e sA'Q esA ds ) x = / (esAx)* Q (esAx) d s + oo 2 = f (esAx)* (esAx)ds = nQJ \e Ax\2 ds s \ e sA \2 d s sj| ) x |2 о o o w h e re /jq is the greatest eigenvalue ofQ. Therefore, by virtue of (4.5), X * ( / e îA*Q esAd s ) x = о M This means that we can define A = / esA*Q esAds (4.6) and we have Л = А* > 0 by virtue of (4.2) and + 00 A *A + A A d „sA*.~. sA ds = -Q .ds e Q e i.e. Л is a solution of (Л). When A does depend on t, it is no longer reasonable to assume that the A and the Q in the definition of a Liapunov function are constant. Therefore we have Definition 4.2 We say that t -* A(t) has a Liapunov function iff for every в > a th e r e are two nX n matrix-valued functions t - A 0(t), t - Q 0(t) s u c h th a t Ae(t) = A* (t) (4.7) а (в) IS Ae(t) S ¡3 (в) I, 0 S t (4.8) 342 CONTI fo r s o m e а {в) > 0, (3(0) È а (в); Q „ (t) = Q | (t) (4.9) y(6) I S Q 0(t) S 6(0) I, 0 = t (4.10) fo r s o m e 7 (0) > 0, 6(0) ê 7 (0), and A 0(t) + A * (t)A e(t) + A 0(t) A (t) = - Q 0(t), 0 S t (Л е ) Along the same lines followed to prove Theorem 4.1 this can be extended into Theorem 4.3 If A : t -* A(t) has a Liapunov function x X*A0(t)x then G has the property EAS. The proof shows that the rate of exponential decay of | G [ is (4.11) A partial converse of Theorem 4.3 can also be proved to extend Theorem 4.2, namely Theorem 4.4 If A is bounded and G has the property EAS then A admits a Liapunov function with (4.12) 5 . AFFINE CONTROL SYSTEMS. BIBS STABILITY Let us denote by t-> f(t) an n-vector function of te J measurable and locally integrable on J. For each 0 G J, x G H", there is a unique (Caratheodory) solution of the affine ordinary differential equation x - A(t) x = f(t) (Ef) s u c h th a t x(0) = x (C) i.e. there is a unique n-vector function t -» x(t), satisfying (C), locally absolutely continuous on J and such that dx(t)/dt - A(t) x(t) = f(t), a.e. t G J IAEA-SMR-17/48 343 This solution is represented by the Lagrange formula x(t) = G(t, 6) x + /■ / G (t, s) f(s ) ds (L) e Let us now consider a fam ily of affine ordinary differential equations, n a m e ly x - A(t) x = B(t) u(t) (U) where t - u(t) is an m-vector function of te J belonging to a given set U с L l0C(J). This means that each u G U is measurable and (essentially) bounded on every finite interval CJ. Consequently, B(t) is an nX m matrix and we shall assume that В € L The family (U) of affine ordinary differential equations, depending on the index u e U is an affine control system. A solution of (U) depends on u, as well as on t; it also depends on 0 and x and using the Lagrange form ula we shall write (X) 6 Using the terminology of Control Theory the n-vector x(t, 0, x, u) is the state of the system, the n-vector function t -* u(t) is the control (or steering) function or input, the n-vector function t -► x(t, 0, x, u) is the response to u. The variable t is usually interpreted as time. If u is bounded as t-* +«> the corresponding response need not be bounded. We want to characterize such control systems for which bounded inputs yield bounded states. Definition 5.1 The control system (U) is bounded-input bounded-state (BIBS) stable as t-* -i-оо iff : a) G has the property US, i.e. for every в > a th e r e is 7 (0) > 0 s u c h th a t 0 S r S t =* I G(t,T)| < 7 (0) (5.1) b) for each u measurable and bounded on [0, +00[ there is som e k(6, u) > 0 such that (5.2) From (5.1), (5.2) and the Lagrange formula (X) it follows 0 á t ■=» |x(t, 0, x, u )|2 < 7 (0) I x|2+ k (0, u) i.e. to a bounded input u there corresponds a bounded state x. It is easy to prove: 344 CONTI T h e o r e m 5 .1 The control system (U) is BIBS stable if: a) G has the property US; b') for every в > a there exist kB(6) > 0 s u c h th a t t e s t - J j G(t, s) B(s) I ds < k B(0) (5.3) e Proof. Since b 1) clearly implies b), a) + b') im plies BIBS stability. S in c e t t J I G(t, s) B(s) I ds á J I G(t, s)| j B(s) I ds e e it follows that if В is bounded on [S, +oo[ for every в > a then the property US plus the property IS are sufficient for BIBS stability. On the other hand, the simultaneous validity of US and IS is equivalent to property EAS. T h e r e fo r e Theorem 5.2 Let В be bounded on [0,+oo[ for each 0 > a . Then (U) is BIBS stable if G has the property EAS. The converse is not true as is shown by taking A = О, В = 0. 6. STABILIZ ABILITY, I We start with Definition 6.1 We say that the control system x - A(t) x = B(t) u(t) (U) is stabilizable (to zero) iff there is an m X n-matrix valued function F : t F(t) of t € J, such that BF is measurable and locally integrable on J and such that ÿ - (A(t) +B(t) F(t))y = 0 (F) is asymptotically stable. If (F) is exponentially asymptotically stable we say that (U) is exponentially stabilizable. IAEA-SMR-17/4S 345 To explain the meaning of this Definition let us denote by yx the solution of (F) such that Ух (6) = X for a given в e J and a g iv e n x 6 lRn . Then we have y^ (t) - 0 as t - +oo, b u t s in c e yx(t) = G(t,e)x +JI' G(t, s) B(s) F(s) yx(s) ds 0 this means that also the solution of (U) corresponding to u = Fyx, i.e. t -* x (t, 6 , x, F y x ) tends to zero as t - * + o o . In other words, if (U) is stabilizable it is possible to bring the state x of the system from any initial position x to zero in an infinite time. In what follows we shall give conditions for stabilizability. We shall presently start with the case of constant A and B. Then it makes sense to look for constant F such that (F) is (exponentially) stable. A condition for exponential stabilizability arises from the solution of the so-called "regulator problem". For each initial state x one looks.for a control u such that the "cost" function C (u) = r L x + x * M u + u * M * x + u * N u] dt о is minim um. Here x = x(t, 0, x, u) while L, M, N are three matrices, respectively of types nXn, m X n and m X m s u c h th a t L = U N = N* (6 .1 ) The current assumption is ( 6 . 2 ) which means that for x e lRn , u 6 ]Rm x * L x + x* M u + u*M*x + u*N u > a (x*x + u*u) for some a> 0. From this follows (u = 0) L > 0 and (x = 0) N > 0 346 CONTI so th a t N "1 exists, and also that (u = -N^M^x) L - M N "1 M * > 0 For a given triplet L,M,N satisfying (6.1) and (6.2) we associate with the control system (U) the Kalman matrix equation (L - M N"1 M * ) + (A - B N '1 M * )* A (6.3) + Л (A - B N _1 M*) - A(B N ' 1 B * ) A = 0 When В = 0, (U) reduces to (E0) and (6.3) reduces to the Liapunov e q u a tio n (A) w ith Q = L - M N_1 M * > 0 (6.4) Assume that (6.3) has a solution A „ = A S > 0 I f w e take (6.5) (6.3) with A = AM can be written Q + (A + B F)* Aœ + Л* (А + В F) = 0 ( 6 . 6 ) so that A„ is also a solution of the Liapunov equation associated to A + В F, which means (Theorem 4.1) that (F) is exponentially stable. Therefore we have Theorem 6.1 Let A and В be constant. Then (U) is exponentially stabilizable if for some triplet L, M, N of matrices satisfying (6.1), (6.2) the Kalman equation (6.3) has a solution А» = A* > 0. In this case a stabilizing F is defined by (6.5). R e m a r k 6.1 According to Rem ark 4.1 the exponential decay of the solutions of (F) with F defined by (6.5) is not faster than е where Xq is the least eigenvalue of Q = L - M N _1 M * and ц А is the greatest eigenvalue of Aœ . IAEA-SMR-17/48 347 Remark 6.2 The last assertion of Theorem 6.1 needs a comment. In fact, if Л„ = AS > 0 is a solution of (6.3) we have Qa = L - M N 'V + a A „ (B N _1B* )AM = Q* > 0 (6.7) fo r a ll a a 0. Therefore, replacing (6.5) by ( 6 . 8) a n d (6.6) by Q a + (A + BF/A. +AJA + B F J = 0 we still have a Liapunov equation, and we conclude that not only F = F0, but also FK, o' È 0 are stabilizing matrices. R e m a r k 6.3 Theorem 6.1 has an inverse (D .L. Lukes) in the sense that if (U) is stabilizable then for every triplet L, M, N satisfying (6.1) and (6.2), the Kalman equation (6.3) must have a unique solution А = > 0. The stabilizability criterion represented by Theorem 6.1 can be extended to the case of time-dependent A and В by using Theorem 4.3 instead of Theorem 4.1. In this case, L, M, N are also time-dependent, and the Kalman matrix equation is replaced by a Riccati-m atrix differential equation À+ (L(t) - M(t)N _1 (t)M*(t)) + (A(t) - BMN^WIVPW^A + A (A (t) - BW N '^tJM -it)) - A(B(t)N_1(t)B*(t))A = 0 (6.9) and F is replaced by (6 .10) where is an appropriate solution of (6.9). 7. STABILIZABILITY, II Let us now consider another criterion of stabilizability for the case of constant A and В (D.L. L u k e s , W.A. C o p p e l). 348 C o n t i Theorem 7.1 Let A and В be constant. Let there exist some T > 0 such that T П = f e tA В В* e'tA* dt > 0 (7.1) о T h e n if F = -B* f i"1 (7.2) the equation x - (A + B F) x = 0 (F) is exponentially stable. Proof. From (7.1) we have T A fi + fi A'1' - -J e 'tA B B * e 'tA!| dt = -e‘TAB B * e TA* + В В* .dt о hence, by (7.2), (A+BF)iî + Г2 (A + B F)* = -(e’TAB В* e'™* + BB*)s 0 Let X be an eigenvalue of (A + B F)* and Ç a corresponding eigenvector, (A + B F)- Ç = X Ç W e have (X + X) Г:‘ fi? = S* [(A + BF)íi + í!( A + B F )* ] Ç = -Ç* (e'TAB B * e'TA'4 B B i:)Ç s 0 Since 0 it follows ReX s 0. Moreover,if the equality holds then Ç*BB*Ç = 0, i .e . | B* Ç = 0, i .e . B*Ç = 0, hence (A + B F ):ÏÇ = A*Ç = X Ç and therefore which gives т Щ = J e"tA B B * Ç e’ M dt = 0 0 and this contradicts (7.1) since Ç f 0. IAEA-SMR-17/48 349 Remark 7.1 If (7.1) holds for some T > 0 then it holds for every T > 0. In fact, let there exist some T > 0, Ç f 0 such that T T 0 = ïÇJ e tA BB* e"tA d t)? = J | B * e"tA* dt 0 0 T h is m e a n s B*e"tA*î= 0 , te[0,T] and, since t — B* e'tA'’' f is an analytic function, we have B * e"tA* Г = 0 , t 6 К It follows that if (7.1) holds for some T > 0 then we can define = П(Т), hence F = F(T) for every T > 0, so that we have a family, depending on T > 0, of stabilizing matrices. It can be shown that (7.1) is, in fact, equivalent with the existence of a matrix F such that A + B F has any prescribed set of eigenvalues (C. Langenhop, W.M. W o n h a m ). In other words, (7.1) is equivalent to the property that the "closed- loop transfer matrix" p - f (A + B F ) - p I ]'1 В can be assigned an arbitrary set of poles by a suitable choice of the "feedback gain matrix" F. On the other hand, (7.1) is equivalent to the complete controllability of the control system (U). This means that (7.1) is a necessary and sufficient condition in order that the state x of the system can be transferred from any initial position at t = 0 to any final position at t = T. Remark 7.1 means that if this is possible for some T > 0 then it is also possible for any T > 0. This rem ark leads to the notion of "uniform complete controllability" for time-dependent control systems as a sufficient condition to ensure a very strong kind of stabilizability for such systems. 8. UNIFORM COMPLETE CONTROLLABILITY To define the notion of uniform complete controllability we start by considering the nX n matrix T H (т, T ) = f G ( r , t) B (t) B * (t) G* ( r , t) d t (H) 350 CONTI which reduces to Г2 of Section 7 when т = 0 and A and В are constant. Of course, integr ability of В is not enough for the existence of the above integral, so we have to assume that B G Ljoc(J), i. e. the elements of В are locally square integrable functions. We shall also consider the nX n matrix T Г C le a r ly , H(t ,T) = H * ( t , T) ê 0, К(т,Т) = K*(t ,T) ê 0 These inequalities are (both) strict if and only if the control system (U) is completely controllable on [ t , T ] , i.e. iff for every pair v, w G K n, there is a control uvw G L" ( [ t , T ] ) s u c h th a t x ( t , t , v , U y J = v , x (T,t , v, uvw) = w (8.1) It is readily verified that we can take, for instance uvw(t) »B*(t)G*(T,t)H-1 (T,T) [G(t,T)w - v] = B * (t) G * (T , t) К 1 (t, T) [w - G(T,t)v] ( 8 . 2 ) Among the control functions acting the transfer from v to w the one defined by (8.2) has an important property. In fact, if u is another control function satisfying (8.1 ) w e h ave T r h e n ce T T r T h e r e fo r e IAEA-SMR-17/48 351 This means that uvw transfers the state of the system from v at t = т to w at t = T at the expense of a minimum amount of energy, namely T J I Uvw(t) j 2 dt = [G(t ,T) w - w]* H"1^ , T) [G(r,T)w-v] T = [w - G(T, t )v ] * К 1 (т, T ) [w - G(T, t )v ] (8.3) Let us now consider the following example. Let the scalar control system be defined by x + t x = ^ 2 (t - 1 )' e't + 1/2 u(t) with t e J = ] 1, +00[ . It is easy to see that G(t, s) = e sV2 " tV2 H ( T , T + CT) = е2(о-Цт+(о-1)*.е-2Т+1 К (т ,т + ct) = e’2T- 2o+1 - e-2(o+1) T + 1 - ° 2 According to (8.3) we see that the transfer from v at t = т to w = 0 at t = t + cr takes a m inim um amount of energy which - +oo as t-> + oo if ct < 1, w h ile -► 0 as t -* +oo i f ct > 1. The transfer from v =0 at t = т to w at t = t + ct takes a m inim um amount of energy which -» +oo as т -* +oo, no matter what the length ct of the time interval is. To avoid such unpleasant circumstances we need a special class of control systems, namely those which are called uniformly completely controllable, according to R. Kalman. Definition 8.1 The control system (U) is said to be uniformly completely controllable (u. c. c. ) iff there exist с т > 0 , 0 < h 1 s h 2, 0 < k j i k 2 such that for every т > a h1 1 S H(t , t + ct) S h2 I (8.4) kj I S K(t , t + ct) S k2 I (8.5) 352 CONTI These conditions imply, respectively h 2 I S H ( t , t + o ) i h j1 I k ’1 I S К "1 (t,t + ct) s k '1 I so that, according to ( 8.2 ), the amount of energy spent for the transfer of v to zero (or from zero to w) in a time interval [t,t + a] does not depend on th e in i t i a l tim e т . R e m a r k 8.1 It can be shown that if (8.4), (8.5) hold for some aQ > 0, then they are valid for every a > a 0 . R e m a r k 8.2 It can also be shown that if (8.4), (8.5) hold, then there exist p - g x (p) , P - g2 (p) - 0 < g x (p) s g 2 (p) s u c h th a t 0 < g l ( 11 - s I ) á I G (t, s) I = g 2 (|t-s|) (8.6) for t, s > a , 11 - s I > ст. S in ce K(t ,t + ct) = G(t + а,т) H(t ,t + a) G':<( t + ct,t ) (8.7) H(t ,t + ct) = G(t ,t + ct) K(t ,t + ct) G* (t , t + ct) it follows that if any two of the relations (8.4), (8.5), (8.6) hold, the remaining relation is also true. R e m a r k 8.3 (8.4) can be written (x € ft" ) 0 < h l I Xla - X* Н (т,т + ct)x S h 2| x I2 so that if we take x = G*(t + ct, t )r¡ we have ■ .2 i i2 h 1 1 x12 Ê Ч* К(т,т + ст)п È h2| x 12 lAEA-SMR-17/48 353 On the other hand 2 x\l* I G ( r + ст,т)|2 I n 2 and since r) = G *(t,t + cr)x we also have t2 .-2 t 2 X |2^ I G (t , t + a ) I I Y] 2 Therefore from (8.4) we have ( 8 . 8) which, in general, does not imply (8.5) because the bounds, left and right, w ill dep en d on т . However, if A is bounded, i.e., if A (t) I < a, 0 S t (8.9) for some a > 0, then, recalling (1.13), we have and f r o m (8.8) we have (8.5) with h la h 2 replaced by hje'30, h 2eao, respectively. Therefore when A is bounded we can suppress (8.5) from the definition o f u . с .с . We have further T+O T and since, from (1.13) again: T it follows, when A is bounded T+O r Therefore, if not only A but also В is bounded, i.e., if I B(s) I < b в < s 354 CONTI for some b > 0, the second inequality of (8.4) can be suppressed and we conclude that when both A and В are bounded the control system (U) is u.c.c. if and only if there are some ст > 0, > 0 s u c h th a t h j I S H ( t , t + cr) fo r a ll t (8.10) In particular, when A and В are constant, since G(t, s) = e(t' $)A we have H ( t , t + ct) = H(0,cr) so th a t (8.10) will reduce simply to H(0,cr) > 0 i.e. to complete controllability. 9. UNIFORM COMPLETE STABILIZABILITY M. Ikeda — H. Maeda — S. Kodama, in prosecution of Kalm an's work, have recently introduced the notion of uniformly completely stabilizable control systems. Definition 9.1 The control system (U) is uniformly completely stabilizable (u.c. s.) iff for every ju > 0 there is a matrix Pj, such that x - (A(t) + B(t) Fj (t)) x = 0 (F) is exponentially stable with exponent > ц . It can be proved Theorem 9.1 If (U ) is u.c.c. th e n it is u . с . s . u.c.c. serves to prove the existence for every ц > 0 of a solution Лр of the Riccati equation Л+ (A(t) + pi)* A + A (A(t) + Ail) -AB(t)B*(t)A = -I s u c h th a t A „(t) =A*(t) « i g A M(t) á p i , t e r n for some a > 0, (3 ë a , independent of t. Then, according to Definition 4.2, A^ is a Liapunov function for IAEA-SMR-n/48 355 w h e re F„ (t) = - ¿ В (t) Л р (t) From this follows the exponential stability of x - (A(t) + ju I + B(t) F p (t)) x = 0 hence that of (F ) with an exponent > y .. Theorem 9.1 has a partial converse, i.e. : Theorem 9.2 Let A and В be bounded. Then (U) is u.c.c. if it is u.c. s. by m e a n s of bounded F^ . Proof. According to Remark 8.3, since A and В are bounded if (U) is not u.c.c.,then for every ct > 0 and e v e r y p > 0 there w ill exist some vector X - x (p , o') f 0 and some т- > 0 such that ’х * Н (’т рт + ct)x < p 15Г12 i.e. s u c h th a t T + О j I B * ( s )G*( t , s )5T|2 ds - P I xlg Hence, using the Schwartz inequality T + a J I B * (s)G * ( t , s ) x |2 ds S \Гстр |5T|2 (9.1) On the other hand, for every тб J, x 6 ®.n, the solution x M of (F^, ) such th a t Хр(т) = x is given by t x M(t) = G (t,r)x + J G (t, s )B (s ) F j (s)xi¡(s)ds hence, denoting by the evolution matrix of A + В , we have t GM(t,T )x = G(t, t ) x + f G(t, sJBisJF^ (sjG^ (s ,t) x ds 3 56 CONTI Since x is an arbitrary vector it follows G^ft.r) = G(t,r) + J G (t, s ) B (s )Ffl(s)G/s,T) ds w h en ce G* ( t , т )G* ( T , t ) = I +J G* ( s , t )F*(s)B*(s)G* (t , s) ds If we apply this to x for т = т we obtain t G* (t,T )G * (t , t) x = x + J G* ( s , t ) F * (s) B * (s) G* ( t , s) x ds T w h en c e t |G*(tJT)G;!(TJt)x |2 = I x + s.tJF* (s)B*(s)G*(7, s)x ds |2 T* t |G*(t,r)| I G:;!(t, t)| I x|2 g I x|2 -J\G^s.t")! I Fp (s) f I B*(s)G>;<(t, s)x| t ” and using (9.1), for t = t- + a 'T + O ds I GM ( t + a ,- T '.I) I I G( - T, ' T + ст) •. 1 2 1 - > J T| G Hp ( s • , t ) • I| .| Fp H (s)• • |I N ow , I Gjj (T + I s 7 е~ца I G(t57 + a) I S e aa lFM - V for some fjj > 0 so th a t 7 e*_,J + a)0 è 1 - s/cp 7 fjj and if we take H = 2a, p = (9 T2 f 2 cr)'1 lAEA-SMR-17/48 357 we have 7 e"ao è 2/3 which is a contradiction since we can take a arbitrarily large. 10. B*IBS STABILITY In Section 5 we considered BIBS stability and we proved (Theorem 5.2) th a t property EAS ^ BIBS stability (10.1) provided that В is bounded on [0, +oo[, i.e. I B(t)I < |3, 0 ê t (10.2) for some (3 > 0. It is easy to see that the validity of (10.1) can be established under a condition different from (10 .2), lik e (10.3) e fo r s o m e .8 (0) > 0 . In fact, to prove (10.1) we have to prove (5.3) e Since EAS is equivalent to US plus IS we have 0 he nce I ! G (t,s) В (s) I ds S (I G (t, s) |2 ds0 B ( s )|2 ds 0 *0 S [ 7 (e) k (0 ) j3 (o) ] 358 CONTI B .D .О. Anderson — J. В. Moore replaced BIBS stability by B*IBS stability. This is obtained by requiring that (5.2) in Definition 5.1 is satisfied not only for all bounded u, but also for all the bounded* u, which means that th e r e a re CTu > °< Uu > 0 s u c h th a t r + о J' I u(t) |2 dt < uu for all t > a (10.4) r T h e r e fo r e B*IBS stability =>BIBS stability but the converse is not true. Anderson and Moore have studied the relationship between B*IBS stability and the property EAS and found that they are equivalent provided that (U) is u.c.c. We shall now prove the first part of this implication, n a m e ly Theorem 10.1 Let (U) be u.c.c. Then if G has the property EAS, (U) is B*IBS stable. Proof. Let 9 s t and take an integer к = k(t, 0) such that к + 1 ê (10.5) (7 where ст is the one appearing in the Definition of u.c.c. Then we have t 6 + a G (t, s) B (s ) u(s) d s = G (t, в + a) J" 0(6 + o3 s) B(s) u(s) ds + . . . 9 0 + j о . . . + G (t, в + j a ) J ' G (в + j ст, s) B(s) u(s)ds + . . . e + (j-i)o 0 + ko t . . . + G (t, 9 + kcr) J ' G (9 + кст, s) B (s ) u(s)ds + J' G(t, s) B(s) u(s) 0 + (k -l)a 0 + ко ( Ю 6 ) If u is bounded* then (10.4 is valid also with ct replacing ctu and we have f o r j = 1 , . . . , к IAEA - S MR-17/48 359 0 +jo 6+ jo G(0 + ja , s) B(s) u(s) ds J I G (0 + j ct, s) B(s) I |i ( s ) I 2 d s e + ( j- i) o 0 + (j-l)O 0 + jo 2 i 2 |G(0 + jcr, s) B (s )|2 d s ( 1 0 . 7 ) e + ( j- i) o A lso, because of (10.5), G(t, s) B(s) u(s) ds ; J I G(t, s) B (s)| I u(s ) |2 d s 0 + ko e + k o t s J I G(t, s) B(s)| I u ( s ) 12 d s á J |G(t, s) B (s )|2 d s ( 10. 8) t - О t - О From the second half of (8.5) we have r + о J I G ( t + a, s ) B ( s ) |2 d s s n k 2 ( 1 0 . 9 ) From this and (10.7), (10.8) we have, respectively 0 + jo / G ( 0 + jct, s) B(s) u(s) ds e + ( j- i) o (10.10) t G(t, s) B(s) u(s) ds 6 + ko From (10.6), for a bounded':î u it follow s t / G(t, s) B(s) u(s) ds 0 ( n k 2 u)u) ‘ 1 + I G(t, 0 + кст) I + . . . + I G(t, 0 + jct) I + . , + I G(t, 0 + ct) I 360 CONTI From property EAS we have (7 = 7 (6) > 0 , /и = ц{6) > 0) |G(t,0 + ja) I = |G(t, в + ka) G(t + ka, в + ja)| S I G (t, в + ka) I I G (в + ka, в + ja ) | S 7 1 G (t, 6 + ka) | (e_|Ja)k s 72 (е'П 'Л j = 1, ..., к H e n ce G(t, s) B(s) u(s) ds S ( n k 2 u u ) 1 + 7 2^ (e_fJO)k_j j = i 2 1 1 + 7 < (n k 2 U J 1 - е -м о i.e. (5 .2 ). Note that we used only a part of u.c.c. BIBLIOGRAPHY ANDERSON, B. D. O ., MOORE, J. B ., New results in linear system stability, SIAM J. Control, 7 (1969) 398. COPPEL, W .A ., Matrix quadratic equations, Bull. Austral. Math. Soc. 10 (1974) 377. IKEDA, М ., MAEDA, H ., KODAMA, Stabilization of linear systems, SIAM J. Control, 10 (1972) 716. KALMAN, R .E ., Contributions to the theory of optimal control, Bol, Soc. Mat. Mexicana, (2) 5 (1960) 102. LUKES, D. L ., Stabilizability and optimal control, Funkc. Ekv. 11 (1968) 39. SILVERMAN, L. M ,, ANDERSON, B. D. O ., Controllability, observability and stability of linear systems, SIAM J. Control 6 (1968) 121. WONHAM, W .M ., On pole assignment in multi-input controllable linear systems, IEEE Trans. Autom. Control, AC 12(1967) 600. IAEA-SMR-17/71 CONTROLLABILITY OF NON-LINEAR CONTROL DYNAMICAL SYSTEMS C . LOBRY Université de Bordeaux I, France Abstract CONTROLLABILITY OF NON-LINEAR CONTROL DYNAMICAL SYSTEMS. In this paper, the results obtained in the problem of controllability of non-linear systems from 1970 to 1974 are reviewed. After introduction of the idea of a control dynamical system, the abstract theory and applications are treated in a concise way, references to the original sources replacing detailed proofs to a great extent. INTRODUCTION This is a review paper on the results obtained in the problem of controllability of non-linear systems from 1970 to 1974. Let us just remark that the ideas developed during the last five years were initiated by Hermann [18], Hermes [19], Kucera [36, 37] and Markus [48] and probably some other people. The idea of this survey is not to write the history of the subject but to present a set of references available to the reader. In Chapter 1, we introduce the idea of control dynamical systems, which is a new formulation of what is currently called a non-linear control system. Chapter 2 is devoted to the abstract theory and Chapter 3 to applications. At the end, references to other topics of control theory where the use of these geometric tools seems to be useful are given. 1. C O N T R O L D Y N A M IC S Y S T E M S Here, we introduce the concept of a control dynamical system. We hope that Examples 1.3 and 1.4 given at the end of the section w ill convince the reader that control dynamical systems are standard objects in nature. 1.1. Vector fields on a manifold M A manifold is something like a "surface” but of a dimension not necessarily equal to 2 (see M ilnor [50] for a rapid introduction to the subject). A vector field is a mapping x - X (x ) which associates with x a vector in the tangent space TXM, to M at the point x. Under very reasonable assumptions the Cauchy problem 361 362 LOBRY d x - v i \ d t ” x (0) = x 0 has a unique solution whose value at time t is denoted by X t(xg). When this value is defined for every t we say that the vector field is complete. The smooth mapping (t, x) - X t(x) defines a group action. X 0(x) = x X «rlt2 (x) = X t¡ о X tz (x) 1.2. Control dynamical systems Let us consider a family S > of complete vector fields on the manifold M. If S) reduces to one element we have a group action of E on M which is defined by the integration of the differential equation d x - v , \ dt - X (x) Consider the collection of symbols of the form ■(t1,X 1)(t2, X 2) ... (tp, Xp); p G N, X 1 € ^> ,t.G и Take concatenation as the law of composition and adopt the two sim plification r u le s : i) if X 1 = X i+1 then replace (tif X 1) (ti+1, X i+1) by (t¡ +ti+1, X 1) ii) suppress terms of the form (О, X). The set of all irreducible sequences is called the control group associated to S> and denoted by G{&). We define the group action by (t1( X 1) (ta, X 2) ... (t¡, X 1) ... (tp, X p)x = X* o x 2 ... oX j. о ... o X ? (x) where the X{¡ are defined in section 1.1. The verification that is a group and that the above action is a group action is trivial (see Refs 142,46]). IAEA -S M R -17/71 363 1.2.1. Definition: A family 3 ) of vector fields on M is called a 'Control Dynamical System", the (non-commutative) group defined above, is the associated control group. In G(¿25) we consider the subset G + (&>) of those sequences: (tp X 1) (t2, X 2) (...) (tj, X 1) ... (tp, XP) for which the reals t1( t2, ..., t t are positive. The orbit G (^ ).x of a point is the set: G (S > ).x = { X ,1 o X 2 о ... oX‘ о ... oXP (x); X 1 e S)\ tj e Ж, p 6 IN} 4 h tj tp The positive orbit is the set G+(^>).x = {X* oX ,2 о ... oxj о ... oXP (x); X 1 e g ) \ t. e ® +, p e IN} 1 2 ' P The positive orbit at time t is G Л 3 > ) .х = j x .1 oxf о ... oX‘ о ... oXp (x); X 1 e 3 ) ; L i tz ‘ ¡ 'p p t¡ e * p£ t¡ = t, p e k| i — 1 The questions which we now want to answer are: What is the structure of G (^).x ? What is the structure of G M(Æ > ).x ? As we shall see, most of the essential questions are solved except two: When is G x or Gt (HZ)), x c lo s e d ? W h e n is G*(¿£>). x the whole manifold M? By an answer to these questions we mean a "computable algorithm" in terms of the known data which gives the answer. Unfortunately (or furtunately), the two last questions are the most pertinent ones for control purposes. 1 / .... r c, c2- p 1 —UUÜJUUÜ--- — u u u u u u u — L A FIG. 1. Electrical network as control dynamical system. 364 LOBRY 1.3. First example of a control dynamical system (taken from Ref.[3] ) Consider the electrical network shown in F ig.l. Here the switch is closed either on the right or on the left. What are the equations of the m o tio n ? a) The switch is on the left: b) The switch is on the right: d V i _ 1 . d t Cx 1 dV2 dt d i _ _1__ dt 2 Now assume, for simplicity, that the constants cx, c2, £^2, are equal to 1; d e n o tin g Vj b y V2 by x 2 and i by x3 we have : dx 0 d T o r dt dx. x, dt 2 For each position of the switch the velocity is orthogonal to the position vector, thus all the motions, whatever we are doing with the switch, are restricted to a sphere whose radius is the norm of the initial state. Thus, the network defines a family S> of two vector fields on a sphere S. This is a control dynamical system. If x° = (xjx 2xg) is an initial state then G +S > . x° is the set of all possible states from x° when we control the switch from right to left and vice versa without any restriction. It is also noted in Ref.[3] that in the problem of the rotation of a rigid body around its centre of mass we are dealing with a control dynamical system on the tangent bundle to the set of orthogonal matrices. 1.4. Second example of a control dynamical system Let X ‘; (i = 1, 2, ..., p ) and X o be p + 1 vector fields in ]Rn and consider the control system IAEA-SMR-17/71 365 and look for B.B. controls, that is piecewise constant controls t -* u¡(t) with value +1 or -1. Denote by A(t, x0) the accessible set at time t in the usual meaning of control theory and by A(xq) = U A(t, x0) the whole accessible set. Then we have A (t, x0) = G t (Æ > ).x0 A (x0) = G +(& > ).x 0 where the family is the family i — 1 Actually, any system defined by = f(x> u ) t x g и", u e Ш р where the controls are piecewise smooth can be considered to be a control dynamical system [42]. 2. GENERAL STRUCTURE OF ORBITS OF CONTROL DYNAMICAL SYSTEMS In this section, we consider a control dynamical system defined by a collection of vector fields on an n-dimensional, connected, para-compact, smooth manifold M. For the sake of simplicity, they are supposed to be c o m p le te . 2.1. The rank of a system If X and Y are two vector fields on a manifold M, we denote by [X, Y] their bracket. We recall that in a local co-ordinate system one has [X, Y] (x) = D Y (x)X (x) - D X (x) Y (x) where DX(x) and DY(x) denote the Jacobian matrix of X and Y at point x, which reduces to: if the vector field X is 3/Эх. If ¡2) is a family of vector fields, then [S&T denotes the smallest fam ily, closed under bracket operation, which contains SD. 366 LOBRY 2.1.1. Definition: The rank of a family of vector fields at point x is the dimension of the linear hull of the set of vectors V(x) in TM when V ranges over We denote it by r {x). W e have: r^ (x ) = dim (áf({V(x); V G 1 & > T } ) ) . The importance of Jacoby brackets and the rank of a system for the controllability of systems seems to have been noticed first by Hermann [17] and Hermes [19]. The systematic use of this tool started in 1970 (see Haynes-Hermes [16], Lobry [40]). Notice that the rank is unchanged if we replace IŒ>]°° by the algebra generated by 3 ) , 2.1.2. A few remarks: The rank r^,(x), when has just one element, is always 0 or 1. As soon as we have more than one element in S>, the rank may be very large. For instance, in R n, the rank at 0 of the system defined by the two vector fields x “9 X j Y = > i = 0 is equal to n, because one clearly has [X [X ...... X[X Y] ] ... ] (o) = p!e w h e re {ep}, p = 1 , ... n, is the canonical basis. 2.2. . Structure of orbits of C" systems We are first concerned with the structure of G ( S > ) .x . The classical Frobenius theorem tells us [40]: 2.2.1. Proposition: Assume that for every point x in M the dimension of 3 D , (dimension of £^({X(x); X G ££>}) is equal to the rank and constant equal to p. Such a control system is said to be involutive and of constant dimension. Then there exists a unique manifold structure on G(£Z>).x s u c h that the mappings (t,,.t. t .,.., t ) - X ,1 o... oX‘ о .. oXf (x) X‘ G p G H t LG E i 1 P 'i Ч tp are differentiable. With this structure G ( S > ) .x is p-dimensional. This theorem is quite inadequate for control purposes. There is no reason for the rank to be equal to the dimension. Chow's theorem [7,30,40] gives us a considerable improvement of this result which is the following proposition: IAEA-SMR-17/71 367 2.2.2. Proposition: (Chow — Frobenius) Assume that the rank of sytem is constant and equal to p. Then G ( 3 > ) .x has a unique manifold structure for which the mappings (t., ..., t., ... t j - x! o .„ o X ‘ о...оХ р(х)х‘ G ^ ; p G S ; t. G IR 1 r li 'i ‘p 1 are differentiable. With this structure G(£@ ).x is p-dimensional. Proof. Take the family {¿£>]°°, this control dynamical system satisfies the assumption of proposition 2.2.1; thus, G ( l S lf ° ) . x has a manifold structure of dimension p. It is connected; by standard argument, the conclusion derives from Theorem 2.2.3 below applied in G([¿2>]“ ).x which locally looks lik e IRp. 2.2.3. Theorem (Chow [7]). In 3Rn, consider a control dynamical system.® of rank n at the origin; then: i) in every neighbourhood of О the set G + (i29).0 has interior points, ii) the set G ( 3 > ) .0 is a neighbourhood of O. Proof. Actually, from Chow's original treatment it is not possible to obtain i), but just ii). Moreover, it does not work directly for families of vector fields but for algebras of vector fields. There are many proofs of Theorem 2.2.3 in control theory literature: Hermann [18], Lobry [40,42,46] , Sussmann-Jurdjevic [65], Stefan [55,56]. Below we give Krener's proof, which is very simple (see Ref.[30]). 2.2.4. Krener's proof of 2.2.3: L e t &> be a control dynamical system in ]Rn, assuming that the rank o f S> is n at point O. Then G + (£D) . О has interior points in every neighbourhood of O. Proof. At least one vector in is not zero at the origin, for, if not, by the form ulas [X,Y](0) = DX(0)X(0) - DY(0)X(0) every bracket would vanish at О and then the rank is 0. Take X 1 in SO w ith X 1 / 0. For e small enough , the s e t S 1 =1x^(0); 0 < t 1 < e 1| is a smooth sub-manifold of IRn. There exists a point pj = X¿ (0) in S x, and a v e c to r X 2 in 3 ) such that X 2 (pj) and X 1 (p^ are independent, for, if not, this would imply that all the vectors of 3 > are tangent of S 1 and thus the rank would be 1 . F o r e 2 sm all enough, the set is a sub-manifold of IRn. If 2 is sm aller than n, there exists a point 2 1 2 Я 1 p 2 = XMz oX ft in S and a vector X (which may be equal to X ) such that the vectors S(p2) X 2(p)2X 3(p2) are linearly independent for the same reason than before; one defines S3 in the same way, etc. up to Sn which is an open 368 LOBRY subset of IRn and is contained by construction in G 40. Since the e-^ e2... en can be chosen arbitrarily sm all, everything works in any neighbourhood of O. Theorem 2.2.3 is proved. Actually, Theorem 2.2.3 tells us much more than 2.2.2. It says: 2.2.5.Theorem: If the rank at x of the control dynamical system defined by is equal to n then interior points of G+(^).x are dense in G +(£ > ).x : G * ( 0 ) . x C Clos (Int(G 4 ( 0 ) .x ) ) Proof: Trivial from 2.2.3. This last result, in view of the stability question (Section 4) seems to be very significant in control theory despite the curious way in which it is stated. Moreover, this property is generic because the rank condition, as the next theorem states, has the following property: 2.2.6. Theorem : Let V(M) denote the set of all vector fields on M; in the Ck-topology of Whitney on (V(M))P, for p ë 2 and к large enough (it depends on p, and the dimension of M), the set of those fam ilies 3 ) of p-vector fields which satisfy the rank condition r^,(x) = n everywhere in M , contains an open dense subset. Proof: See Ref.[41] or Ref.[46], The proof is carried out by means of Thom's Lemm a and the standard techniques in differential topology. 2.2.7. Generic dynamical control systems are "trivial". By "trivial" we mean that for every x in M, G(¿29).x = M. Proof: For a generic control dynamical system r^fx) is n at every point (2.2.6) then G ( S > ) .x is a neighbourhood of x (2.2.3) and thus every orbit is open and hence must be the full space. Thus, we know the structure of the orbits in the generic case (trivial); in the case of constant rank (Chow + Frobenius), it rem ains the non-constant case which has been solved recently by Sussmann [59]. 2.2.8. Theorem (Sussmann) (Take a family Œ> of vector fields on M, without any assumption on (except that vector fields are smooth) there exists a unique manifold structure on G(i2>).x for which the mappings: (tt, ..., t¡, ..., tp) - X* о ... oxj. о ... oXp (x) X 1 G tj G IR; p G N are differentiable. Proof: This proof is completely "by hand” in the sense that it does not refer to any other classical result. Thus, it is a true generalization of the Frobenius theorem. It is too technical to be exposed here; see Ref.[59] o r R e f .[46 ]. As a conclusion, we rem ark that the theory of action of control dynamical systems, as considered from the point of view of orbits, is now complete and actually trivial. Fortunately, many problems on the positive orbits are not yet solved. IAEA-S MR-17/71 369 2.2.9. Examples. Consider in IR the control dynamical systems defined by (Fig.2); Эх a) x2 - A X " a y X 1 - — x “ Эх b) X 2 = (1 + X ) 3y 370 LOBRY FIG. 3. Proof of Proposition 2. 2, 10. In case a) G,\{2&).x is a closed set without interior, in case b), it has an interior and is closed, and in case c) it has an interior but is not closed. The question of interior is easily solved by the following: 2.2.10. Proposition: The set G t(j2?).x has a non-empty interior for every t provided that the rank at point (x,0) of the system £b in M X IRn defined below is equal to n. ^ = {x ® x 6 . A . ( x , t) = ( x « ) ) Proof: It is trivial; see Fig.3. Details can be found in Ref.[65]. The question of closedness is not yet solved in this setting. Actually, it is very closely related to questions of Bang-Bang controllability or singular arcs, and we have now a good knowledge of this problem. (See Section 4). Thus.it seems that we are going to obtain a reasonable answer in the near future (if it has not yet been obtained, by now). We conclude this section by a sort of inverse question: ''How many vector fields do we need to obtain G+(^).x ' = M? " The answer is two. See S u s s m a n n [64] and S u s s m a n n - L e v itt [39]. 2.3. Structure of orbits of analytic systems In this section, we shall assume that the control dynamical system 3) is composed of analytic vector fields on an analytic manifold M. In this case, one can expect "global results" from the knowledge of the "coefficients" of the system at point x. This is actually the case. The following theorem was proved by Nagano [51] in a completely different framework. The proof given in Ref.[40] is false. This was pointed out by P. Stefan [54]. IAEA -SMR-17/71 371 2.3.1. Theorem (Nagano): Let S > be an analytic control system. Let p be the rank of the system at point x. Then the manifold structure defined in 2. 2.8 has dimension p. Proof: See Refs [51,54]. We now turn to the study of G +(l2)).x and G t(& >).x. Suppose now that the rank of the system at point x is equal to n. This is not a restriction because from 2.3.1. this assumption is satisfied in the restriction to G(¿2? ).x. Consider now in M X 1R the system S) defined by: S> = {X œ 3/9t, X G £ 2 ) } and compute its rank at point x. A simple compu tation shows that X Э V 9 = [X,Y] ® 0 a t ' a t and, indirectly, all the brackets are of the same form. More precisely, the following holds: [â>r = J> ® [2)\@Y] and we are^'interested in the dimension of Clearly, (each v e c to r in S>(x) has a component equal to 1 ) this linear subspace is not con tained in the subspace (Çj, Ç2, . . . , Ç , 0); from this one easily deduces the following 2.3.3. Lemma: The rank of §b is n + 1 at point (x, 0) if and only if the linear space generated by the vectors of the form: p p Y XjX^x) + Y (x ); Y Xj = 0, Y € \S ) , [ 2 > Y ] i — 1 i — 1 is of dimension n. From this lemma, from 2.2.10, and from 2.3.1 we deduce immediately: 2.3.4. Proposition: Assume that the system S> has rank n at point x. Then Gt+(_22>).x has a non-empty interior (actually it has dense interior points) for every t,if and only if the linear subspace generated by the vectors of the fo r m : Р P ^ X¡ X ‘ (x) + Y (x ); X¡ = 0 Y G [ 3 > [ 3 > î i = l has dimension n. 2.3.5. We can say a little bit more about G*(£Z>).x when the dimension is not n. Let us denote by the collection of vectors of the form K F & 0 X; X ‘ (x) + Y (x ); Y Xi = 0 Y G [ M 3 > î 372 LOBRY Consider in M X IR the section at time t, denoted by Mt : M t = M x{t} From the considerations before Lemma 2.3.3, we know that the sub-manifold G ( 3 ) (x, 0) of M X К meets each M transversally. Hence, the intersection S t = M t П G (3 > ) (x, 0) can be considered to be a sub-manifold of M. From this construction, it is clear that for any X in 3 one h as X t(S0) = st and, on the other hand, St contains Gt+(^).x . Moreover, the system ( 3 ) restricted to G +( 3 ) (x, 0) is of maximum rank, thus in the topology of G { 3 ) (x, 0), the set G +(É>) (x, 0) has interior points everywhere dense and because it meets St transversally Gî( 3 ) ) ,x has an everywhere dense interior in the topology of St. To conclude, we know that at every point x in M the f a m ily 3 q is tangent to S0 and of rank n-1. This proves that S 0 = G (J 9 0)x. We have proved (see Sussmann-Jurdjevic [65] if more details are needed) the following theorem. 2.3.6. Theorem (Sussmann-Jurdjevic) Let S) be an analytic control system. Define by a new control system, by 2.3.5. i) if rank of at point x is n then interior points of G ^ (S > ).x are dense in G * {3 > ).x ii) if rank of at point x is less than n then it is n-1. The sub m a n ifo ld G ( 3 ^ ) . x has codimension 1. The set G \( S > ) .x is contained in X t ( G ( 3 0)) for any X in 3 ) and in the topology of the sub-manifold X t ( G ( 3 ^ ! interior points of G +(3 > ).x are dense in G +{ 3 ) . x There is another interesting point in Sussmann-Jurdjevic [65]. We just state it without comments on the proof; 2.3.7. Proposition: Let M be a manifold whose fundamental group has no elements of infinite order; then G +( 3 ) . x = M -> G \(3 > ) has a non-empty interior for some t. 2.3.8. Proposition: Let M be a manifold whose universal covering is compact, then G +( 3 ) . x has a non-empty interior -* G \ ( 3 ) . x has a non-empty interior for s o m e t. 3. CONTROLLABILITY OF NON-LINEAR SYSTEMS In this section we restrict our attention to systems of the following form: P = X °(x) + U i X 1, X e M . (Uj, u 2...... Up) e ]RP (1) i = l IAEA-S MR-17/71 373 There is some arbitrariness in choosing such systems where the control is a p-dimensional input which enters into the equation linearly. But it is well-known that the essential difficulties are in the space non-linearities and, moreover, this form is a good balance between reasonable generality in order to obtain interesting cases and a simple exposition. All the controls which we consider are locally bounded measurable controls and, following Jurdjevic-Sussmann [29], we denote by: Uu the set of controls which take arbitrary values in IRP: the set of "unrestricted controls". Ur the set of controls t -» (^(t), ^ 2(t), •••, Щ(Ъ), W p{t)) such that: | <®i''1(t))|ё 1. The sett of "restricted controls". Ub the set of piecewise constant controls which take just the values +1 or -1: The set of "Bang Bang controls". The accessible set at time t from some initial condition x is denoted by: A u(x ,t), A r(x ,t), A b(x,t) depending on the fact that the admissible controls are "unrestricted", "restricted" or "Bang Bang". And we introduce: A (x,[0,t] ) = U A (x,t) s e [0,0 Ar(x,[0,t] ) = U Ar(x,t) se [ 0 ,0 Ab(x,[0,t] ) = U Ab(x,t) se [o, 0 We shall study the properties of these sets when M is an arbitrary manifold, smooth or analytic, a compact Riemannian manifold and the control dynamical system conservative, a Lie group and the control dynamical system is right invariant and finally in IRn. 3.1. The case of an arbitrary manifold We consider a system (1) on a manifold M. When M is an analytic manifold and the vector fields are analytic we say that we are in the analytic c a s e . 3.1.1. Notation: Consider the system (1): P ^ = X°(x) + ^ u¡X'(x) Í - 1 We denote by L the Lie algebra generated by the family (Xo, X1, ..., XP), by L 0 the ideal generated by (X1, X2, ..., Xp) and by Я the Lie algebra generated by (X1, X 2, ..., X?). 374 LOBRY 3.1.2. Proposition: If (and only if in the analytic case) the dimension of L at point x is equal to the dimension n of the manifold M one has: For every (t > 0) interior points of AH(x,[0,t]) are dense in A a(x,[0,t]) where the symbol a can take the values u, r, or b. Proof: This follows from Theorems 2.2.3 and 2.3.1 as soon as we have realized, by a simple computation, that the rank of the family P ® = {x° + £ UjX1 ; (u1; u2, ..., up e Ub} i = l at point x (see definition 2.1.1) is exactly the dimension of L at point x. Details are left to the reader. Remark that in the analytic case of Theorem 2.3.1 the same conclusion holds in the topology of the manifold G(¡2>).x. Now we turn our attention to the reachable set at time t. 3.1.3. Proposition: If (and only if in the analytic case) the dimension of Lo at point x is equal to n we have: For every t(t > 0) interior points of A a(x,t) are dense in Aa(x,t) where the s y m b o l a can take the values u, r or b. Moreover, if the dimension of L at x is n, the dimension of L 0 at x is either n or n-l,and, in this last case, if the system is analytic, the same conclusion holds in restriction to some sub-manifold of dimension n-1 for the case a = r o r a = b. Proof: It follows from the fact that the dimension of Lo is the same as the dimension of the family described in Lemma 2.3.3 (with respect to &>). We turn now to controllability results, which are not very strong in this context. 3.1.4. Proposition: Assume that the system is homogeneous: i.e. X ° = 0. Then if (and only if in the analytic case) the dimension of Л is equal to n at every point of M (which is assumed to be connected) then For every x in M one has A r(x,[0,°o]) = A b(x,[0,°°]) = M For every x in M and t(t >0) one has Au(x,t) = M. Proof: Since in this case we can ''reverse tim e1' (the trajectory corres ponding to some constant control (u1( ..., Up) can be followed backward using the control (-Uj, -u2, ..., "Up), it turns out that this result comes directly, see 2.2.4). 3.1.4. Proposition: Assume that the rank of L is n everywhere, then: For every x in M one has Au(x,[0,«] = M. Proof: Take a point y; a neighbourhood of this point is accessible at time t, for arbitrary sm all t, from x under the homogeneous system (see above). During a very short time the introduction of the non-homogeneous term X o introduces just a "sm all error'’. One concludes by a fixed-point theorem. The correct proof of this last result can be found in Brunovsky-Lobry [6] where this idea is used to prove various results of controllability. IAEA-SMR-17/71 375 FIG. 4. Torus illustrating Proposition 3.1.5 . FIG. 5. Knowledge of E+ and L~ does not determine controllability. Controllability of A depends on whether the trajectory is above or below line ir. We conclude this paragraph by some comments taken from the work of Gerbier [11, 12]. We look for the controllability of a very simple system, i.e. the system: X°(x) + u Xx(x) x = (xv x2) G IR2 (2) xS (l+f(xi 'x2))i 4 +^ (XiX2> 4 X1=f Notice that for this system the control u = +1 leads to the vector field d/dxi and the control u = -1 leads to the vector field f(xxх2)Э/Эх г + + g(xix^d/дх2; thus, this class of systems is parametrized by the set of vector fields in the plane. Moreover, we restrict ourselves to non-vanishing vector fields: f^XjXg) + g2(x1x2) > 0. Itturns out that the two sets £ + = {(x1, x2); g(xp x2) = 0 and f(xp x2) > 0 } £" ~• {(Xp x2); g(xp x2) = 0 and f(xr x^ < 0 } play an important, but non-exclusive role for controllability, i.e. we can p ro v e : 3.1.5. Proposition: If the system (2) is controllable: (Ar(x,[0,°°]) = IR for every x) then the set £" is not empty. Notice that this depends on the Euler-Poincaré characteristic because it is false on the torus shown in Fig.4. Now, the following example shows how knowledge of and £~ is not enough to determine controllability. 3.1.6. Example (see Fig. 5). For details, see Ref.[11]. 3.2. The case of a compact manifold with a measure Suppose that we are on a compact manifold M with a measure ц. W e consider the system: P = X°(x) + JT u, X1 (x) x e M (1) i — 1 and we assume that the system is conservative; this means that the vector fields X o ± X 1, i = 1 ...p, perserve the measure ц. К((Xo ± X i)t(В)) = ¿((В) We denote by L the Lie algebra generated by (Xo, X 1, ..., XP). 3.2.1. Proposition: If (and only if in the analytic case) the rank of the dimension of L is n (the dimension of M) at point x then one has: A r(x,T) = A b(x,T) = M for some sufficiently large T. Proof: It uses the fact that ’'Poisson-stable" points are dense for con servative dynamics. See Lobry [44]. Compactness plus maximum rank is not sufficient for controllability as is shown by the following example: 3.2.2. Example: Consider, on the sphere, the system described by Fig.6. IAEA-SMR-17/71 377 FIG. 6. Compactness plus maximum rank not enough for controllability. 3.3. The case of a Lie group The case where the state space is a Lie group G has been extensively studied by Brockett [2,4], Jurdjevic [28], Jurdjevic-Sussmann [29], and Sussmann [58]. The technique consists in the use of general results from section 3.1, together with some advantages due to the group structure. The systems considered are right-invariant: P x e G (i) i = l which means that the vector fields X 1; i = 0, 1, ..., p, are right-invariant vector fields. The Lie. algebras L, L 0, and i, with notation of 3.1.1, deter mine Lie sub-groups S, S0, and s. In this case, the question of controlla bility reduces to a knowledge of whether or not A(e,[0,°°]) (where e is the neutral element of G) is a sub-group. An answer is given by the following th e o r e m : 3.3.1. Theorem: If G is compact the system (1) is completely controllable (i.e. AJe,[0,«>] ) = G; a = u, r, b) if and only if the Lie algebra L is the Lie algebra of G. Proof: Because of the compactness of G, one can prove, in this case, that Aa(e,[0,°o]) is a sub-group [29]. Another way of proving this is to rem ark that, for flows generated by right-invariant vector fields on a compact Lie group, all points are Poisson-stable and then the result of section 3.2 can be applied. Actually, in this compact case, one can say even more: 3.3.2. Theorem: Assume that G is compact and the Lie algebra L of (1) is the Lie algebra of G. Then for each class of controls Ua(a = u, r, b) there exists a T > 0 such that for every g in G A a(g,[0T]) = G. If, moreover, G is semi-simple the same statement holds with A^g.T) in place of A«(g,[0T]). Proof: see Ref.[29]. 3.3.3. The group GL(n,K) of invertible matrices turns out to be a Lie group. In this case, the tangent space at each point is identified with the set M(n,IR) of n-by-n matrices. A right-invariant vector field is of the form: X - AX X G GL(n, IR); A G M(n, B) 378 LOBRY Thus for every sub-group G of GL(n, IR) the "m atrix differential system": P = [A0 + Y UjA1] X , X e G ; A1 e L(G) i = 0, 1, p i = l is a particular case of right-invariant systems on Lie groups. In this case, the bracket of the two vector fields X - A‘.X X - A JX is the vector field X - (A1 A1’ - A J A 1 )X o r X - [A1, A '] X with the usual notation: [A, B] = AB -BA. 3.3.4. We conclude this paragraph by the following rem ark due to Hirshorn [24,25]. By a theorem of Palais [52], if the Lie algebra of a family of vector fields is finitely generated, then it can be realized as the Lie algebra of a Lie group of diffeomorphisms of the manifold. Then our problem reduces to a control problem on a Lie group. 3.4. The manifold is IRn We show how classical results on controllability in IR" are consequences of the previous results. But first we state a result by Brunovsky whose proof is quite delicate and does not reduce to 3.1, 3.2, or 3.3. 3.4.1. Proposition: The control system (1) p ^ = X°(x) + U jX l (x), i = 1, 2, p i = l is said to be "odd" in case we have X°(-x) = -X°(x). Then the set A(0,[0,T]) is a neighbourhood of the origin for every positive T if (and only if in the analytical case) the dimension of the Lie algebra L generated by (Xo, X 1, ..., Xp) is n. Proof: see Ref.[5]. Let us look now for "bilinear systems": IAEA-SMR-17/71 379 3.4.2. Definition: A bilinear system on IRn is a system of the form: P (2 ) It is clear that the origin is a rest point for systems (2): thus we look for controllability in ]Rn\ {0 }. Because the solution at time t of system (2), from initial condition xQ is given by X(t).X0, where X(t) is the solution of the matrix equation: i = l the problem of controllability reduces to the problem of controllability on Lie groups of matrices (see paragraph 3.3). We conclude by the celebrated Kalman critérium: 3.4.3. Theorem: The system: H Y = Ax + Bu, x e ]Rn, u e ]Rp dt is completely controllable if and only if the rank of the matrix (B, AB, A2 B, ...... An_1B) is equal to n. Proof: The reachable set from any point is a linear subspace with a non empty interior, i.e. the full space, iff the rank of the system is n at every point. Compute the brackets: [Ax + B u A x -B u J with u x = (1, 0, 0, 0) [Ax + Buj, Ax-Bu-j] = 2 A BUj Now compute: [Ax + Bax, A BuJ We obtain [Ax + BUj, A BuJ = A^Uj^ etc. One easily sees that the rank of the system is precisely the rank of the matrix (B, AB, ..., An_1B). 380 LOBRY 4. OTHER RELATED QUESTIONS To conclude this survey, we just mention here some related questions which are object of current research. 4.1. Closedness of G^{&).x: Consider the system: P = x°(x) + Y u ixiW ; x e IRn i = 1 With the notations of Chapter 3 we have: Ar(x,t) = Ab(x,t) and clearly if Ab(x,t) is closed this means: Ar(x,t) = Ab(x,t) that is to say, everything you can do you can do with a bang-bang control. This is a bang-bang principle. It is known that bang-bang principle, singular arcs and maximum principle are very closely related questions. On these topics, we mention papers by Hermes [20,21 ] Krener [30,34,35] , Lobry [43,45 ]. 4.2. Realization theory Assume that you have a control system of the standard form: Р = X°(x) + ^ u ¡x ‘ (x) x G M; (Uj u2 ... up)G IRP i = l on some manifold M. Let ф : M -* N be a mapping from M into some other manifold X (the observation). Let us say that two states x 0 and x x are equivalent if for any control t ->^/(t) we have: cp(x(x0, t/^O) = ф ( х ( х j , \.@/)), t fe 0 where x(x0, tf%/), x(x, Xi%/) denote, respectively, the response issued from x 0 and xi. To speak of minimal realization, it is necessary to make the quotient of M under the above equivalence and to define a manifold structure on the quotient. It turns out that this is possible by a theorem of Sussmann [60], which generalizes the classical "closed-sub-group theorem". Applications are given in Sussmann [61,62]. Another approach concerning local realizations of non-linear systems by bilinear ones can be found in Krener [33]. IAEA-SMR-17/71 381 4.3. Structural stability and classification Let us say that a control system is structurally stable if a little change in the data does not very much change the general behaviour of the system. What are structurally stable systems? Are they generic? Try to classify them. There are some attempts to work in this direction in Gerbier-Lobry [12,13]. 4.4. Connection with diffusion process Consider the diffusion process: P d?t = X°(?t) + Y X ‘ (Çt ) dB¡ i — 1 where Xo, X 1 ... Xp are vector fields and the B¡ are independent one dimensional Brownian motions. By a result of Stroock and Varadhan, th.e support of the diffusion process is connected to the controllability problem for the control system: P ^ = X°(x) + Y ui Xi (x). i =1 On these topics, see Elliot [8,9], Kunita [38], Strooke-Varadhan [57]. REFERENCES [1] BOOTHBY, W. , A transitivity problem from control theory. [2] BROCKETT, R. , Lie theory and control systems defined on spheres, SIAM J. Appl. Math. 24 5 (1973). [3] BROCKETT, R. , System theory on group manifolds and cosetspaces, SIAM J. Control 10 2 (1972). [4] BROCKETT, R., "Lie algebras and Lie groups in control theory. Geometric Methods in System Theory". Reidel (1973). [5] BRUNOVSKY, P., Local controllability of odd systems, to appear. [6] BRUNOVSKY, P., LOBRY, C ., "Contrôlabilité Bang-Bang, contrôlabilité différentielle et perturbation des systèmes non linéaires”, (a4 paraître dans Ann. Matem. pura ed appl.) [ 7] CHOW, W. L ., "Über Systeme von linearen partiellen Differentialgleichungen erster Ordnung", Math. Ann. U7_(1939) 98-105. [8] ELLIOTT, D. , A consequence of controllability, J. Diff. Equ. 10 (1971) 364-370. [9] ELLIOTT, D ., "Diffusions on manifolds arising from controllable systems", Geometric Methods in System Theory, (loc. cit. see Ref. [4]). [10] ELLIOTT, D ., TARN, J. T ., Controllability and observation for bilinear systems (unpublished). [ 11] GERBIER, Y. ."Classification des couples de systèmes dynamiques du plan. Application a' la théorie de la commande", Thèse, 3è cycle, Bordeaux 1974. [12] GERBIER, Y ., "Classification de certains systèmes dynamiques contrôlés du plan", C.R. Acad. Sc. Paris 280 (20 janvier 1975) 109-112. [13] GERBIER, Y ., LOBRY, C ., "On the structural stability of dynamical control systems". Communication to 1975 I. F. A. C . Congress. [ 14] GROTE, J., "Problems in Geodesic Control", Geometric Methods in Systems theory (loc. cit. see Ref. [4]). [15] GROTE, J., "La théorie des connections et contrôle". Publications Mathématiques de l ’Université de Bordeaux, Année 73-74, Fascicule 3, 7-13. [1 6 ] HAYNES, G .W ., HERMES, H., N onlinear con trollability via Lie theory, SIAM J. Control 8 (1970) 4 50 -46 0. 382 LOBRY [17] HERMANN, R., "On the Accessibility problem in control theory". Internat. Sym. Non-linear Differential Equations and Nonlinear Mechanics, Academic Press New York (1963). [18] HERMANN, R., Differential Geometry and the Calculus of Variations, Academic Press, New York (1968). [1 9 ] HERMES, H. , C on trollability and the singular problem , SIAM J. Control 2 (1965) 241-260. [2 0 ] HERMES, H ., On lo ca l and global controllability, SIA M J. C ontrol, 12 (1974) 252-261. [21] HERMES, H. , "On necessary and sufficient conditions for local controllability among a reference trajectory", Geometric Methods in System Theory (loc. cit. see Ref. [4]). [22] HERMES, H. , "Local controllability and sufficient conditions in singular problems" (to appear). [23] HERMES, H ., HAYNES, G .W ., On the nonlinear control problem with control appearing linearly, SIAM J. Control 1_( 1963) 85-108. [24] HIRSHORN, R., "Topological groups and the control of nonlinear Systems" (to appear). [25] HIRSHORN, R., Topological semi-groups, sets of generators and controllability, Duke Math. J. 40 4 (1973) 937-947. [26] HIRSHORN, R., "Controllability in Nonlinear Systems", Geometric Methods in System Theory (loc. cit. see Ref. [4]). [2 7 ] HIRSHORN, R ., "T o p o lo g ica l sem i groups and con trollability in bilinear system s", Ph. D. Thesis, Div. of Eng. and Appl. Phys., Harvard Univ. (Sept. 1973). [28] JURDJEVIC, V ., "Certain controllability property of analytic control systems", SIAMJ. Control 10 (1972) 354-360. [2 9 ] JURDJEVIC, V., SUSSMANN, H ., "Control systems on Lie groups", J. D iff. Eqs 12_2 (1972). [30] KRENER, A ., "A generalization of Chow’s theorem and the Bang-Bang theorem to nonlinear control systems", SIAMJ. Control 12(1974) 43-52. [31] KRENER, A ., "On the equivalence of control systems and the linearization of nonlinear control systems", SIAMJ. Control U_( 1973) 670-676. [32] KRENER, A., "Bilinear and nonlinear realizations of input output maps" (to appear), [3 3 ] KRENER, A . , "L ocal approxim ation o f control systems" (to appear in J. D iff. Eqs) [34] KRENER, A., "The high order maximum principle" (to appear). [35] KRENER, A ., "The high order maximum principle", Geometric Methods in System Theory (loc. cit. see Ref. [4]). [36] KUCERA, Y ., "Solution in large of control problem x = (A (l-u ) + B)x". Czech. Math. J. 16(91) (1966) 600-623. [37] KUCERA, Y ., "Solution in large of control problem x = (Au+ Bv)x". Czech. Math. J. 17 (92) (1967) 9 1-96. [38] KUN1TA, Sh., Diffusion process and control systems, cours de D. E. A. Univ. Paris VI, Lab. de Calcul des probabilités (1973-74). [39] LEVITT, М ., SUSSMANN, H., On controllability by means of two vector fields (to appear in SIAM). [40] LOBRY, C ., Controlabilité des systèmes non linéaires, SIAM J. Control 8 (1970) 573-605. [41] LOBRY, C ., Une propriété générique des couples de champs de vecteurs, J. Math. Czech. 22 (1972) 230-237. [42] LOBRY, C. , Quelques aspects qualitatifs de la théorie de la commande, Thëse-Grenoble (1972). [43] LOBRY, C ., Geometric structure of Dynamical polysystems. Warwick Control Theory Center report 19 (1972). [44] LOBRY, C. , Controllability of nonlinear systems on compact manifolds, SIAM J. Control 12-1 (1974) 1-4. [4 5 ] LOBRY, C . , "Deux remarques sur la com m ande Bang Bang des systèmes semi lin éa ires". (Proc. C onf. Zakopane, Poland). [46] LOBRY, C ., "Dynamical Polysystems and Control Theory", Geometric Methods in systems theory" (loc. cit. see Ref. [4]). [47] MARKUS, L., "Control dynamical systems”, Math. Systems Theory 3^(1969) 179-185. [48] MARKUS, L., SELL, G .R ., "Capture and control in conservative dynamical systems" Arch. Ration. Mech. Anal. 3J_(1968) 271-287. [49] MARKUS, L ., SELL, G .R., "Control in conservative dynamical systems: Recurrence and capture in aperiodic fields", J. Diff. Eqs 16 (1974) 472-505. [50] MILNOR, J. W ., Topology from the Differential View Point, The University Press of Virginia, Charlottesville (1965), [51] NAGANO, T ., Linear differential systems with singularities and application to transitive Lie algebras. J. Math. Soc, Japan 18 (1966) 398-404. [52] PALAIS, R,, A global formulation of the Lie theory of transformation groups, Mem. A.M .S. 22 (1957), [5 3 ] REBHUHN, D . , On the set o f attainability, PhD-thesis Univ. o f Illinois Urbana - Cham paign (1974). [54] STEFAN, P., Accessibility and singular foliations, PhD-thesis, Univ. of Warwich (1973). IAEA-SMR-17/71 383 [55] STEFAN, P., Integrability of systems of vector fields (to appear). [56] STEFAN, P., "Two proofs of Chow’s theorem" Geometric Methods in Systems Theory (loc. cit., see Ref. [4]). [57] STROOKE, P., VARADHAN, S., "On the support of diffusion Processes, with applications to the strong maximum principle", 6th Berkeley Symp. Mathematical Statistics and Probability. [58] SUSSMANN, H ., The Bang-Bang problem for certain systems in G. L. (n, IR), SIAM J. Control (to appear), [59] SUSSMANN, H ., "Orbits of families of vector fields and integrability of systems with singularities”. Trans. Amer. Math. Soc. _180( 1973) 171-188. [60] SUSSMANN, H ., "On quotients of manifolds: a generalization of the dosed subgroup theorem" (to appear in J. D iff. G e o m .). [ 61] SUSSMANN, H ., Minimal realizations of nonlinear systems, Geometric Methods in Systems Theory (loc. cit., see Ref. [4]). [62] SUSSMANN, H ., "Observable realization of finite dimensional non-linear systems" (to appear in SIAM J. control.) [63] SUSSMANN, H ., "Existence and uniqueness of minimal realizations of nonlinear systems I: Initialized systems" (to appear). [64] SUSSMANN, H ., "On the number of vector fields needed to achieve controllability"(to appear). [65] SUSSMANN, H ., JURDJEVIC, V ., "Controllability of nonlinear systems", J. Diff. Eqs 12 (1972) 95-116. [66] SWAMY-TARN, "Optimal control of discrete bilinear systems", Geometric Methods in System Theory (loc. cit., see Ref. [4]). [67] SWAMY, "Optimal control of single input bilinear systems", D.S.C. Dissertation (Dec. 1973) Washington, University St. Louis, Missouri. IAEA- SMR-17/63 INTRODUCTION TO CONVEX ANALYSIS J.P. CECCONI Centro di Studio per la Matematica e la Física Teórica, C.N.R., Genova, Italy Abstract INTRODUCTION TO CONVEX ANALYSIS. This paper considers the main elements of convex analysis in infinite-dimensional spaces, that is the most essential properties of convex functions valued in R (the extended real numbers) such as: continuity property, duality, sub-differentiability, properties concerning the minimization of such functions, and,finally, the connections of these with the monotonie-operators theory and the variational-in^qualities theory. Convexity has always played a fundamental role in the study of variational problems, but systematic studies on convexity have been carried out only in recent times. 1. CONVEX SETS, SEPARATION THEOREMS Let X be a vector space over Pi (abbreviated v. s. ) if x, y e X, the set (x,y]= {Xx+(1-A)y:0 s Xs 1} is called the closed line segment .joining x and y. A subset К of X is convex if x, у £ К imply that [x, y] G К. It is immediate that X and 0, the empty set, are convex sets. The intersection of convex sets is a convex set. The union of convex sets is not, in general, a convex set. If A is a subset of X the convex hull of A, denoted by со A, is the smallest convex subset containing A. Evidently n n Let H = {x : f(x) = a}, where f is a non-zero linear form on X and a G R, be a real affine hyperplane in X; the convex sets Fa = {x : f(x) < ar}, F “ = {x : f(x) > a} are called the algebraically open semispaces determined by H, and the convex sets Ga = {x:f(x) «»}, Ga = {x:f(x)S a) 385 386 CECCONI are the algebraically closed semispaces determined by H. Two non empty subsets A, В of X are said to be algebraically separated (respectively strictly separated) by the real hyperplane H if either A C G , ВС G“ or A c G “, В С G q (respectively, if either A с B e F a or A С F a, B e F a ). It is obvious that the hyperplane H = {x G X :f(x) = a} separates A and В if and only if f(x ) S a (or f(x) So) VxGA and f(y) S a (or f(y) s a) V y G В We recall now the Hahn-Banach theorem (in its analytical and geo metrical form) and the separation theorems for convex sets. Theorem 1 (Hahn-Banach, analytic form). In a v. s. X let p :X ■* R be a sublinear real function, i.e. a real function such that p(x + y) g p(x) + p(y); x, y G X p(a x) = a p(x); x g X , O 'G R + U{0 } Let A be a linear subspace of X and let f : A -* R be a linear real function on A with f(x) § p(x), x G A Then there exists a linear real function F :X R for which F(x) = f (x ) x G A F (x) S p(x) x e X P roof. Consider the family S of all real linear extensions g of f for which the inequality g(x) s p(x) holds for x in the domain of g. Let the relation h > g be defined in S to mean that h is an extension of g; this relation partially orders S. Let us show that every totally ordered subset of *3? = {g a :aG has an upper bound on S. Let us define В = U Aa, where Aa is the domain of ga a e g(x) = g a(x) if g a e <¡g, x G A a IAEA-SMR-17/63 387 This definition is not ambiguous, for if ga¡, ga2 G , then either g < g or go, < ga¡- At апУ rate- if x G A a¡ П Ааг, then g a i(o) = ga¡ (x). Clearly g 6 S, and it is an upper bound for ЯК in S. Then from Zorn's lemma follows the existence of a maximal extension F of f for which the inequality F(x) s p(x) holds for every x in the domain of F. It remains to be shown that the domain M of F is X. Otherwise we can prove that there would be a g G S such that, F < g and F f g and this would violate the maximality of F. Let us assume the existence of an x0£ X-M. Every vector x in the space = M ® {x0} spanned by M and x 0 has a unique representation in the form x = m + a x 0 with m e M, a G R. For any constant c, the function g defined on M by gfm +ttXj) = F (m) + o c is a proper extension of F. The desired contradiction will be made when it is shown that с may be chosen so that g G S- Let m, n be vectors in M. Then from F (m) - F (n) = F (m - n) S p (m -n ) S p(m + x 0) + p ( - n - x Q) it follows that -p(-n-x0) - F(n) 5 p(m +x 0 ) - F(m) Since the left side of this inequality is independent of m and the right side is independent of n, there is a constant с with -p(-m-xQ ) - F(m) Sc m G M с S p(m + x 0 ) - F(m) m e M We show that, with this c, it results for every x = m +ax 0 G M: gfrn+ax^) = F(m) + а с § р ( т + а х ( ) so that g G S. In fact, if a > 0, then m g(m+ox0) = oc + F(m) = aic+ F {■ Se {■lpWc + xoy-.) FP -) 388 CECCONI if a - -/3 < 0, then g(m +»x0) = F(m) +ac = F(m) - j3c = ¡3 | f - c } S/3{F( j ) + F G j ) +p(+ J - xo)} = P(+m- ^ 0) S p (m rax0) The proof of the theorem is so completed. Corollary 1. In a v. s. X let p : X -* R be a seminorm in X, i. e. a real function such that p(x +y) g p(x) + p(y) x, y £ X p (ax ) = I a I • p (x ) x e X , a e R Let A be a linear subspace of X and let f : A -» R be a linear real function on A with I f(x) I s p(x), x e A Then there exists a linear real function F : X -* R. for which F (x) = f(x), x e A IF (x) I S p(x), x G X Proof. From the theorem follows the existence of F :X -♦ R for which F(x) = f(x); x £ A F (x) S p(x); x С X Therefore -F(x) = F(-x) s p(-x) = p(x) Definition 1. Let К be a convex set in a v. s. X. A point x e X is called an internal point of К if for every v e X there exists £ € E 4 such that for every 5 6 R, ¡5 | < e it results x + 6 v e K. IAEA-SMR-17/63 389 Definition 2. Let К be a convex set on a v. s. X and let the origin 0 of X be an internal point of K. For each x € X let I(x) = {a e R.^:(x/a)G K} and k(x) = inf I(x). The function k(x) is called the gauge function of K. Theorem 2. Let К be a convex set containing the origin of X as an internal point and be к its gauge function. Then (a) к is a sublinear non-negative function in X; (b) if x 6 К then k(x) s 1; (c) the set of the internal points of К is characterized by the condition k(x) < 1 . Proof. Statement (a) follows from the fact that the origin is an internal point of K, from the fact that I(orx) = orl(x) for every x G X and a G R^,and from the fact that, for every x G X, e G R . k(x) + e e I (x) It follows that if x, y G К then ------G К к(х) + е 'к(у ) + е and from the convexity of К х + У______X ______k(x) + e + у k(y) + e______k(x) + k(y) + 2e k(x) + e k(x)+ k(y) + 2e k(y) + e k(x) + k(y) + 2e T herefore k(x) + k(y) + 2e G I(x + y) and consequently k(x + y) = inf I(x + y) S k(x) + k(y) + 2e Hence k(x + y) s k(x) + k(y) Statement (b) is self-evident. If x is an internal point of К then x + € x G К = x(l + c) G К for som e sufficiently small e G R+ so that ^ ^ ^ G I(x). Hence k(x) = inf I(x)g - - <1 390 CECCONI Conversely,if k(x) < 1 let e = 1 - k(x) and for every y G X let |ô I {к (у) + к (-y)} < е. Then k(x + бу) í k(x) + к(бу) â (1 - e) + бк(у) < (1 - e) + e = 1 k(x + 6 y) s k(x) + k(6 y) s (1 - e) - ó k (-y ) < (1 - e) + e = 1 according to 6 is positive or negative, so that x + 6 y G К. Therefore x is an internal point of K. We can prove now the following separation theorem for convex sets: Theorem 3. Let A and В be disjoint non-empty convex subsets of a v. s. X and let A have an internal point. Then there exists a hyperplane H which separates К and H. Proof. If a is an internal point of A, then the origin 0 of X is an internal point of the convex set A - a = {x G X:x = z - a, z G A }. The affine hyperplane H = {x G X :f(x) = a) separates A and В if and only if the affine hyperplane H' = {x G X :f(x ) = o' - f(a.)} separates the sets A -а and В-a. Thus it suffices to prove the theorem under the additional assumption that 0 is an internal point of A. Let b be any point of В so that -b is an internal point of A - В = {zG X :z=x-y, x G A, yGB} and 0 is an internal point of the convex set К = A - В + b = {z G X : z =x-y + b, x G A, yG B } Since A and В are disjoint,the set A-В does not contain 0; hence К does not contain b. Let к be the gauge function of the convex set K, so k(b) s 1. If for every a G R we put f 0 (ab) = ak(b) then f Q is a linear function defined on the one-dimensional subspace of X which consists of real multiples of b. Moreover, fotab) § k(ab) for all a G R, since for a 8 0 we have f 0(o-b) = к(о-Ь) while for a < 0 we have f q(orb) = crfo(b) < 0 sk(ab). By the Hahn-Banach theorem f 0 can be extended to a real function f such that f(x) £ k(x) for every x G X. It follows f(x) £ 1 for every x G К, while f(b) ê 1. Thus K = A- B+ b C {x: f(x) s 1}, f(b) s 1. For every a G A, (3 £ В we have f(a) - f(j3) + f(b) s i and also f(e )s f(0 )+ l-f(b) 5f(j3) lAEA-SMR-17/63 391 Let us call 7 = sup {f(a) : a G A}. Hence results f(a) 5 7 , a G A f (/3 ) S 7 . /3 G B Thence A C {x : f(x) §7}, BC {x:f(x)H} and the proof is completed. In the particular case where В is a linear affine subspace of X we have Corollary 2 (Hahn-Banach theorem in geometrical form). If A is a convex subset of a v. s. X, which has an internal point, and if В is a linear affine sub space of X such that А П В = 0, then there exists an affine hvperplane of X which contains В and does not intersect A. P roof. Let H = {x :h(x) = a } the affine hyperplane of X that separates A. and В in consequence of Theorem 3; and let be A С {h(x) § а}, В Э (h(x) S a}. If b is a point in the linear affine space В we have h(y) = h(b) = 7 for every y G В. In fact, if h(b’) f h(b), b 1 G В and B0 = В - b, b 1 = b +u, u G B 0, it results b + Xu G В for every A G Pi and h(u) f 0 so that h(b + Xu) = h(b) + Xb(u) < a for convenient X; a contradiction. Therefore h(x) § a s h(b) for every x G A and h(y) = h(b) for every y G В so that the hyperplane H1 = {x :h(x) = h(b)}, which contains B, separates A and B. Definition 3. A real v. s. endowed with a topology ^"for which the two axioms (1) (x, y) -> x+ y is a continuous map on X X X into X (2) (a, x) -’ ax is a continuous map on R X X into X are satisfied, is called a real topological vector' space (abbreviated t.v. s. ). If ^"= {U } is a basis of neighbourhood of the origin 0 of X in ^ a n d x 0 GX, then x 0 +U ; U G / is a base of neighbourhood of xQ in the topology If К is a convex set in ji_t.v. s. X, the closure К and the interior K° of К are convex sets and К = (K°) if К f 0;_moreover,- if x G K°, then x is an internal point of K; if x G K° and y G К, then every point in the open segment [x, у [ is interior to K. If K° / 0 and x is an internal point of the convex set K, from K= (K°) it follows that x G K°. 392 CECCONI If A is a subset of a t.v. s. X, the intersection of all closed convex sets containing A is called the closed convex hull of A and is denoted by со К If К is a t. v. s. and H = {x : f(x) = a} is an affine hyperplane of X, then H is a closed subset of X if the linear real function f is continuous on X. In this case the semispaces Fa, F“ [or Ga, Ga] are open [or closed] subsets of X. Conversely, if for anaE Й and a linear function f :X-» R F , F “ are open subsets of X , then f is a continuous function. In fa:ct we have Theorem 4. If H = {x : f(x) = a} is an affine hyperplane of X which separates two non-empty subsets A, В of X, one of which has an interior point, then the linear real function f :X -* R is continuous. Proof. Let x0 be an interior point of A. Then there exists a neighbourhood U of the origin 0 in X such that x0 + U 6 A. In the hypothesis that A С Ga, В С G“, then we have f(x 0+ U) S a and therefore, for every xë U, f(x) S a - f(x 0). Let V = U П (-U); then V and -V are neighbourhoods of the origin of X in and for every y G V it results f(y) s a -f(x Q) f(-y) Sa - f(x0) so that I f (У) I S a -î(x 0) Then, for every e G R+ , we have |Г(У)| “ € if У e ----- T,— Ñ V = V ' and V is a neighbourhood of the origin 0 of X in a - i (x0 ) Therefore f is continuous at the origin 0 of X. Since f is linear and continuous at 0 it is continuous everywhere. Theorems 3 and 4 yield the following separation theorem in t.v. s. Theorem 5. Let A and В be non-empty disjoint convex subsets of a t.v. s. X, one of them being an interior point. Then there exists an affine closed hyperplane which separates A and B. Definition 4. A t.v. s. X is called locally convex (abbreviated 1. c. s. ) if it is a Hausdorff space such that every neighbourhood of the origin X contains a convex neighbourhood of 0 . IAEA- SMR- 17/63 393 Every normed space is obviously a 1. c. s. In general,every v. s. endowed with a family {P : y G Г } of seminorms in X such that for every x G X a seminorm p^, exists such that Py (x) f 0, is a 1. c. s. ; its base of neighbourhoods of the origin 0 are all finite intersections of sets Uyi £ = {x : Pj, (x) s e}, y G Г, e G R +; this topology on X is called the topology generated by the family of seminorms Ру . In this topology each of the given seminorms Py is continuous on X. In a 1. c. s. we can derive from the preceding theorems of separation the following theorem which has become a standard tool of the theory. Theorem 6 . Let A and В be non-empty disjoint convex closed subsets of a 1. c. s. X, one of which is compact. Then there exists a closed affine hyperplane which strictly separates A and B. Proof. A-В is a convex closed set and (A-В) П{0} = 0. Therefore there exists a convex neighbourhood U of the origin in X such that (A-В ) П U = 0. From Theorem 5 it follows that there exists a closed hyperplane H which separates A-В and U. Let f:X -*R be the non-zero continuous linear function such that H = {x : f (x) = a, a G R } and f(A -B ) í a , f(U) Since f is a non-zero function, there exists an x G X such that f(x) f 0; since U is a neighbourhood of 0, there exists a 6 G R + such that kx G U, |k| S6 and consequently f(kx) § a if k| S 6 . Therefore there exists an e G R+ such that [-e , e ] С {kf(x) : к | s б} С f(U) so that a i e . Hence f(a-b) = f(a) - f(b) s a > e for every a G A, b G B. Let с = inf {f(a), a G A}, then c G R because A is compact and f is continuous on X. We have also f(a)ÈcSf(b)+o'>f(b) + e, aGA, bGB and therefore f(a) ê с > с - e >f(b), aGA, bGB Two important consequences of these theorems are Corollary 3. If К is a convex subset of a t.v. s. for which K° f 0, then through every boundary point of К passes a closed supporting hyperplane H; i.e . a closed hyperplane H such that КПН f 0 and К is contained in one of the closed sem ispaces determined by H. M oreover, К is the intersection of the closed semispaces which contain К and are determined by the supporting hyperplanes of K. Corollary 4. If К is a non-empty convex closed subset of a 1. c. s. , then К is the intersection of all closed semispaces containing K. 394 CECCONI Proof of Corollary 3. To see that through every boundary point x of К passes at least one supporting hyperplane it suffices to apply theorem 5 to the convex sets K° and x 0. Indeed there exists an affine closed hyper plane H = {x : f(x) = <*} such that f(x) s a for every x e K° and f(x0) ê a. Then for every y 6 К it results f(y) ё a and consequently f(x 0) = a. To prove the second assertion we prove prelim inarly that there exists no supporting hyperplane of К containing an interior point of K. In fact: assume that x G K° П H where H = {x : f(x) = a} is a supporting hyperplane of К such that К С G„. There exists y G K° with f(y) < a, since H cannot contain K°. Now f { x + e ( x - y )} = f(x) (1 + e) - ef(y) = e (l + e ) - ef(y) > a (1 + e) - ea = a for every e G R+. But from x G K° it follows that x+ e(x - y) G К for convenient e G R+ and therefore f(x + e(x-y)) S a for convenient e G R + . This is a contradiction. From the first assertion we know that КС 0 G„ СаеУ where ^ is the family of all supporting closed hyperplanesH of К and К С G„ with H = {x : f(x) = a}. It remains to prove that if у ^ K, then there exists a closed supporting hyperplane H = {x : f(x) = a} such that К С Ga, f(y) > a. Let уф K, x G K°; the open segment [x,y[ contains exactly one boundary point x 0 of K. There exists a closed supporting hyperplane H = {x :f(x) = a } passing through x 0 such that К С Ga; H does not contain y; otherwise it would contain x and this is a contradiction. Proof of corollary 4. It follows obviously from theorem 6 and the fact that sets containing exactly one point are compact. All the properties now proved are of fundamental importance in functional analysis. For example, ifX is a I.e. s., from the Hahn-Banach theorem follows the existence of linear continuous non-zero real functions on X: it suffices to consider two points x,y GX; x f у and to separate them by a closed hyperplane H; if H = {x :f(x) = a} then f :X -» R is continuous and f(x) f f(y). If X is a t. v. s. , we call topological dual of X the v. s. X* of all the continuous, real linear functions on X endowed with the usual structure. It is convenient to denote elements of X* with notations x*, y* and so on, and to denote the value of the linear continuous real function x* G X* on x with the notation<(x, x* У. Then the map (x, x*) -* ^x, x* У is a real bilinear function on X X X* and Xv [resp.X ] is a v. s. of linear real functions on X [resp.Xv ]; more over, every x* G X'r [resp. x G X] is the linear real function x x* )■ on X [resp. xt -|(x,x*)on X ’’], We can introduce on X* [resp. on X] the a (X*, X)-topology [resp. the a (X,X*)-topology ] by taking as a base at 0 the family of all sets of the form U(A, e) = {x* G X* : |<(x, x*)> | I.e. s. If X (with the initial topology) is a I.e . s ., then also X endowed with the a (X , X *) is a 1. c. s. ; in fact x | < x , x * > | is a seminorm for every x* 6 X* and for every pair x , y £ X , x f y there exists, in the hypothesis that X is a 1. c. s. , an x* 6 X * such that Corollary 5. In a 1. c. s. X every closed convex set is closed in the cr(X,X*) topology; therefore a convex subset in a 1. c. s. X is closed if and only if it is weakly closed. Proof. From corollary 5 it follows that К = П where S is the family of Ga es all closed semispaces Ga = {x : f(x) S <*} such that К С Ga. But every f considered in the definition of a Ga 6 S is also continuous in the ct(X,X*) topology of X; therefore every GaG S is also closed in that topology. Then from К = П Ga follows that К is closed in the cr(X,X*) topology. GqÆ S In the following we shall need also two theorems concerning normed spaces. Definition 5. Let X be a normed v. s. and f :X -» R a linear continuous function on X. We call || f || = inf F when F = {(3 6 R+: | f (x) | § |3 ||x||, x 6 X }. It is obvious that F is not empty because f is continuous. ||f || is called the norm of f. Theorem 7. Let X be a normed v. s. , M a linear subspace of x and f :M - R a continuous linear function on M. Then there exists a continuous linear function F : X R such that F (x) = f(x) on X and ||f || = ||f||. P roof. Let I f II be the norm of f : M -» R and p(x) = ||f|| ||x ||. Then p :X -> R isa seminorm on X such that |f(x) | s p(x) on M. Then by corollary 1 there exists a linear extension F :X -*■ R of f such that |F(x)| s p(x) = I f I) D x D. Thus ü F ü S ü f К. On the other hand, since F is an extension of f we must have ||f || S || F ||. This completes the proof of the theorem. Theorem 8 . (Mazur) Let X be a normal v. s. and {xn}nEN a sequence in X which converges to x in the weak topology of X. Then there exists a sequence Pn Pn whose elements are convex combinations of given xn, which converge in norm to x. Proof. For each n e N let us define An = U {x¡ }. Then x belongs to the i = n weak closure of An, for every n £ N. Therefore x belongs to the weak 396 CECCONI closure of co(An ) Vn G N. But, for corollary 5, x G co(A n) V n € N. Then for each n G N we can take a yn G co(A n) such that ||x-yn || < l/n . The theorem is proved. 2. CONVEX FUNCTIONS Let X be a (real) v. s. and R = R U {- oo} U { + oo}. Definition 1. If К is a convex set in X and f : К -> R is a function on К whose values are real numbers, either - co or +oo, we say that f is convex on К if for any x G X, y G X we have f(Xx + (1 - X)y) S Af(x) + (1 - X) f(y), X G [0, 1] (2.1) provided the second member is defined; i. e. the inequality subsists in all x, y G К with the exception of the points where f(x) = -f(y) = ± oo with the usual computation rules in R a+oo=+oo + a = + o o i f a f -oo а-оо=-со + а= -оо i f a f +00 a ■ o o - o o , a ■ (- oo) = - oo if a G R + , 0 • oo = 0 • (- oo) = 0 We say that f is strictly convex if (2.1) is a strict inequality for x / y, X G ]0, 1 [. From definition 1 it follows obviously that, if xp x 2. . . , x nG K, a. ê 0 ; i = 1 .... then / A n 1 1 provided the ^econd member is defined. If f : К -* R is a convex function on the convex set K, then for every a G R the sets G a = {x G X :f(x) s a}, Fa = {x G X : f(x) < a } are £onvex; but the fact that Ga [resp. Fa ] is a convex set for every a G R, does not imply that f is a convex function. _ For example, if К =_R, any increasing f ;R - R is such that Ga [resp.F a is convex for every a G R. IAEA -SM R-17/63 397 For any function f : X -» R we call effective domain of f the convex set dom f = (x gX :f(x) < + «} By a proper convex function on X we shall mean a convex function with values in R U {+ oo}_which is not merely the constant function f(x) = + 00 . Therefore, if f :X -» Ris a proper convex function, the dom f is a non empty convex set on which f is real-valued. Conversely, if К is a non empty convex subset of X and f is a real-valued function on К which is convex, i. e. satisfies (2.1) when x € К, у G К, then one can obtain a proper convex function on X by setting f(x) = + 0 0 for every x ^ K. A very useful example of a proper convex function is the indicator function X of a non-empty subset К of X, which is defined by It is obvious that a subset К of X is convex if and only if is a convex function on X. So the theory of convex subsets of X is a part of the theory of the proper convex functions on X. Definition 2. If f :X-*Ris an extended real-valued function on X, the set epi f = {(x, a) G X X R: f(x) § a} is called the epigraph of f. It is obvious that a subset G С X X R is the epigraph of some function if and only if for each x G X the set {a G R; (x, a) G G} is either R, 0 or a closed internal of the type [d, +oo[. Then the function f is obtained from G = epi f by f(x) = inf {a G R : (x, a) G G} with the usual rule inf 0 = + 00 . We have the following characterization of convex functions: Theorem 1. In a v. s. X a real extended function f :X -* R is convex if and only if epi f is à convex subset of X X R. Proof. Let f :X -» R be a convex function on X and let (x, a), (y, b) be in epi f. We prove that X(x, a) + (1 - X) (y, b) G epi f for every X G 10,1]. In fa ct,f(x) s a < + 00 , f(y) s b < + oo and for every X G [0, 1] it results f [Xx + (1 - X)y ] S Xf (x) + (1 - X) f(y) ë Xa + (1 - X)b 398 CECCONI s o t h a t X (x, a) + (1 - X) (y, b) = (Xx + (1 - X) y, Xa + (1 - X)b) G epi f Conversely, let us suppose that epi f is a convex subset of X X R. Then dom f = {x G X : f(x) f + oo} is convex. In fact, if x, y G dom f, then there exists a, b G R such that f(x) S a, f(y) s b and therefore (x, a) G epi f, (y, b) G epi f. From the convexity of epi f it follows that for every XG [0 , 1 ] X (x, a) + (1 - X) (y, b) = (Xx + (1 - X)y, Xa + (1 - X)b) G epi f and so f(xX + (1 - X)y) S Xa + (1 - X)b G R (2.1a) and dom f is convex. Let us prove that f : dom f -*■ R U {- oo} is convex. This follows from (2.1a)bymakingin it a = f(x), b = f(y) if f(x), f(y) G R and by passing by the limit for a -* - oo (or b -» - o o ) if f(x) = - oo (or f(y) = - oo). Now we can easily say that f :X - R is convex: in fact, (2 Л) subsists if x, y G dom f; and it follows from the computation rules in R if x, y ^ dom f. If, finally, x G dom f, y ^ dom f (2.1) is not requested if f(x) = - o o and it follows from the computa tion rules in R if f(x) G R. The class of all convex functions on a v. s. X has the following closure properties which imply moreover that the class of all convex functions is a convex class. Theorem 2. The supremum of any family { f a : a G A} of convex functions on a v. s. X is a convex function on X. The sum of two convex functions f on a v. s. X is a convex function on X if we let (f+ g)(x) = + oo when f(x) = -g(x) = - o o . If f is a convex function in a v. s. X and a G R+ then ai is a convex function on X. P roof. The first assertion follows from the fact that epi (sup fa : a G A) = П epi fa ae A and from theorem 1. To prove the second assertion, let f, g be convex functions on X, x, y G X. For every X G [0,1 ] we have f [Xx + ( l - X ) y ] S Xf(x) + (1 - X) f(y) g [Xx + (1 - X)y ] S Xg(x) + (1 - X) g(y) IAEA- SMR-17/63 399 and, by addition, the corresponding statement for f + g; in fact,this is obvious if additions are done in R, and it follows from calculation rules incluse ( + oo) + ( - oo) = + oo if additions are done in R. The third assertion is obvious. The most interesting part of the theory of convex functions is done in a 1. c. s. X as we suppose from now on. We recall that a function f :X - R is said to be lower-semicontinuous on X (abbreviated 1. s. c. ) if, for every x 0 G X, we have f(x0) S lim f(x) X ^Xo A function f is said to be upper-semicontinuous on X (abbreviated u. s. c. ) if - f is 1. s. c. on X. If f is 1. s. c. and u. s. c. on X it is continuous on X. Obviously the indicator function of a set A in X is 1. s. c. [resp. u. s. c. ] if and only if A is a closed [resp. open ] set. The following theorem gives a characterization of 1. s.c. functions. Theorem 3. Let f :X -*■ R'be a real extended function on a 1. c. s. Then the following conditions are equivalent: (1) f is 1. s. c. on X (2) the sets Fa = {x G X : f(x) > a} are open for every a G R (3) epi f is a closed subset of X X R. P roof. (1) implies (2). In fact, if y G Fa , it follows from (1) that lim f(x) È f(y) > a. Therefore there exists a neighbourhood U(y) of y x ^ y where f(x) > a. Hence U(y)C F a and Fa is open. (2) implies (1). Let XgG X ,f(x 0) f - oo and a < f(x„), a G R,then xg G Fa and Fa is open. Therefore, inf {f (x ) :x G F } f - oo and also lim f(x) ê a, and for the x ^ x 0 arbitrariness of a, lim f(x) È f(xn). Nothing must be proved if f(xn) = -o o . x -» x0 u u To prove the equivalence between (2) and (3) we define the map 0 :X X R -» R which value 0 (x, a) = f (x)- a and observe that f is 1. s. c. on X if and only if 0 is 1. s. c. on X X R. But as we have proved^ is 1. s. c. on X X R if and only if the set {(x, a) G X X R : 0(x, a) = f(x) - a S a } is closed for every a G R. But this set coincides with the set {(x, a) G X X R : f (x)-a Sa} and this set is a translation of epif and therefore it is closed if and only if epi f is closed. The proof of the theorem is so completed. Corollary 1. If f :X -» R is a convex 1. s. c. function on a 1. c. s. X, then f is 1. s. c. in the weak topology a (X,X*). For every a G R the set Ga = {x G X : f(x) S a } is convex and closed, therefore it is convex and closed in the weak u (X ,X *) topology. So f is 1. s. c. in this topology. The following theorems give us conditions under which a convex function is continuous. Theorem 4. If a proper convex function f:X -*R on a t.v.s, is bounded above by a real constant in a neighbourhood of a point x G X, then f is continuous on X. 400 CECCONI P roof. We can suppose that x = 0 and that f (0) .= 0. Let U be a neighbour hood of the origin 0 where f(x) S a G R. Let V = U <"> (-U) and observe that V is a symmetric neighbourhood of the origin. Let e 6 ]0,1|. If x E V we have, by the convexity of f, (x/e)G V and therefore f(x) = f((l - €) 0 + e ( j ) s (1 - e) f(0) + e f ( j ) s e a - | G V and therefore f(0 ) = f + x f(x) 1 + e 1+ e 1 + e f(x) We have also |f(x)| s ea if x e eV and f is so continuous in x. Theorem 5. Let f :X -* R be a proper convex function on a t. v. s. X. If there exists an open non-empty set U where f is bounded above by a real constant3then (domf)° D U and f is continuous on (dom f)°. P roof. From the preceding theorem follows that f is continuous at every x G U, moreover it is obvious'that U С (dom f)°. For every y G (dom f)° and a fixed x0G U let p > 1 be such that z = x 0 + p(y - x 0) G (dom f)°. Let t = 1 - (l/p )G ] 0, 1 [ and consider the map 4>:x E X - tx + (1 - t ) z G X. This is a continuous bijective map of X onto X whose inverse tp'1 is also continuous. Therefore For every w = f(w) = f(tx + (1 - t)z) itf(x ) + (1 - t) f(z) S tM + (1 - t)f(z) < + =o since z G (dom f)°. We have so proved that f is bounded above in the neighbourhood Theorem 6 . If f :X -*R is a proper convex function on a normed space X and if there exists a subset U of X where f is bounded above by a real M, then (dom f)° f ft and f is locally Lipschitzian on (dom f)°. Proof. By theorem 5, f is continuous on (dom f)°. Therefore, if x G (dom f)°- there exists r0E R+ such that if ||y-x|| S r„ it is |f(y)-f(x)| § 1 and con- IAEA-SMR-17/63 401 sequently there are real constants m ,M such that m § f(y) s M where ¡jy-x II < r 0 . Let us consider an r e ]0, r 0[ and a point ух such that Ilyx -x II § r. If we let G(z) = f(vj + z) - f(yj) for ||z || s r 0 - r, then G(0) = 0 and G(z) S M-m if ||z | = r 0 - r. Another consequence of theorem 4 is that, for every e e ]0,1 [, |G(z) j g e (M - m) if ||z ü S e (r0 - r). If we take y G X such that || у - ух || s r Qr then H y - y ü = (г -г) with У ~ е ] 0 ,1 L 11J J Iм r - г о rQ -r so that I f (y) - f (У1 ) I = |G(y-yi)| s ^ _Уг ~ If y2 is another point in X for which ||y - x || S r we choose on the segment [y , y ] a convenient number of points z, = y , z„, z, . . . z = y„ i ¿ 1 1 ¿ p 2 such that llz i +1 - Zi II = ~ l II У2 - У1 II < r 0 - r’ i = 1. 2 . . . p-l Then, for every consecutive pair z£, z i + 1 , we have |f(z ) - f(z. ) I s M ~ ^ ü z. - z. ü I ' 1 + 1 ' 1 1 r - r 11 1 + 1 l 11 and finally, summing up, we see that |f(y2) - f(y0)l - f ~J~ IIy2 - yjl The theorem is completely proved. In the special case where X = Rn we have Theorem 7. If f :R n -> R is a convex proper function, then it is continuous in (dom f)°. Proof. If (dom f)° f 0 it contains (n + 1) points Xj^, x 2. . . xn + 1 which are linearly independent. If x is in the open convex set n+l n+l (co {xj^ . . . x n + 1))° we have x = «jX^ <*; > 0 , ^ = 1 i l and from the convexity of f n + l f (x) = f ( Y ,aixi) S 1 i our assertion follows now from theorem 4. 402 CECCONI In the first section we proved that every convex closed set in a 1. c. s. can be obtained as intersection of all semispaces containing it. Now we investigate similar properties for convex 1. s. c. functions in a 1. c. s. Definition 3. In a t. v. s. X we call affine continuous function an f:X -R such that f(x) = (x ,x * )> + a where x* G X* and a € R, Theorem 7. A proper convex 1. s. c. function f:X -*R in a l.c.s, X is the pointwise supremum of all affine continuous functions h :X -* R such that h(x) S f (x) for every x G X. Proof. It suffices to prove that for every x G X and every real a < f(x) there exists an affine continuous function h in X such that â < h(x) and h(x) s f(x) V x G X. In order to prove it}we observe that epi f is a convex closed subset of X X R and that (x,a) ^ epi f. By means of the separation theorem we can strictly separate epi f and (x, a) using an affine closed hyperplane H = {(x, a) :<(x, x * )+ O' • a = 7 } ofX X R where x* G X*, a, 7 G R and <(x, x*^> + a a < y (2 . 2 ) Now if f(x) G R (x, f(x)) G epi f and the second inequality gives us <(x, х*У + f(x)a > y (2.4) so that (a - f(x)) a < 0 It must be a > 0 and by dividing (2.2), (2.4) by a we obtain a < ■ < f(x) V x dom f a \ ’ a / ’ a \ ' a / and this proves that the affine continuous function h(x) = has the required property. If f(x) = + 00 and a f 0 we can conclude as in the preceding case. Let now f(x) = + 0 0 and a = 0. In this case (2.2), (2.3) say that the affine continuous function K(x) = 7 - K'(x) = (3 - <(x, у*У such that b < K'(y) and K'(x) ë f(x) V x € X. Then, for every c G R + , the affine continuous function f (x) = j3 - At this point let us rem em ber that K(x) > 0 and choose с so great that i(x ) > a. Thus the theorem is completely proved. If we recall that the supremum of a family of convex 1. s. c. functions in a t.v. s. X is a convex 1. s. c. function, we have also proved Theorem 9. A proper function f :X->R in a 1. c. s. X is convex 1. s. c. , if and only if it is the supremum of a family of affine continuous functions. Let us introduce the following definition: Definition 4. In a 1. c. s. X let f : X -» R be an extended real-valued function. We call regularized of f (abbreviated F(f)) the function Г (f) = sup { cp : ip affine continuous function on X such that (x) S f(x)in X } Obviously F(f) is a convex 1. s. c. function on X and it results (rf) (x) S f(x) V x G X In theorem 8 we have also proved this. Theorem 10. If f :X -> R is a proper convex function on a 1. c. s. X, then f = r(f) if and only if f is a convex 1. s. c. function. A useful approach to the regularization is provided by the notion of the conjugate of a given function. Definition 5. In a 1. c. s. X let f :X -* R be an extended real function. For every x* G X* let f*(x*) = sup { The function f* :X* -*R so defined is called the conjugate function of f. It is immediately seen that an affine continuous function on X,h(x) = <(x, x*)> = a, satisfies the condition h(x) s f(x) on X if and only if <^x, x*)>- f(x) s a, i. e. if and only if f*(x*) S a . It is also immediate that for every x* G X* it results f*(x*) = sup { Theorem 11. If f, g : X -* R are real extended functions in a 1. c. s. X, then (i) f*(0) = -inf {f(x) : x G X} (ii) f(x) s g(x) for every x G X implies f*(x*) ê g*(x*) for every x*G X* 404 CECCONI (iii) for every X £ R + we have (\f*)(x*) = Af* (iv) for every 6 R we have (f + a)*(x*) = f*(x*) - о (v) (x,x’ ) s f(x) + f*(x*) for every x £ X, x*£ X* Once f* is defined we can consider, for every x sX , _ f**(x) = sup {Xx, x* f* (x*)} . The function f**: X -* R so defined x>;‘ e x * is called the second conjugate of f. This function f** is, as we know, a convex 1. s. c. function on X. If we compare f** with f we have Theorem 12. If f :X -> R is an extended-valued function on the 1. c. s. X, then for its second conjugate we have: f** = r(f). Therefore, if f is a proper convex 1. s. c. function, then f** = f. Proof. From the definitions of F(f) and f** we can derive (Г (f)) (x) = sup { f** (x) = sup { f*(x*) = sup {<(x, x*)> - f(x)} so that by a fixed x* £ X* it is (x ,x ‘ )-irSf(x) ctex on X if and only if f*(x*) s a. Therefore (F(f)) (x) = sup { It follows that (Ff) (x) = f**(x ) If we define f***(x*) = SUp {<^XjX*^ -f**(x) x £ X } = (f**)* (x*) then Theorem 13. For every f :X -* R it results f*(x*) = f***(x*) on X*. Proof. In fact, from f(x) ê (rf) (x) = f**(x) we obtain, by theorem 11, f*(x*) = f*** (x*) on X*. But from the definition of f**, we have also for every x £ X, x* £ X*: From the definition of we obtain therefore f***(x*) § f*(x*) on X* and the proof is complete. IAEA- SMR-17/63 405 The following theorem fixes a significant geometric property of T(f). Theorem 14. If f : X -* R is a proper real extended function on a 1. c. s. X, then epi T(f) = co (epi f). Proof. We prove preliminarly that, for the given f, there exists a g:X -*■ R such that epi g = со (epi f). In fact, if (x, a) G co (epi f) and b > a, then (x,b) G cojep i f) so that, for every x € X, if we let g(x) = = inf {X G R : (x, X) G co(epi f)} it results obviously epi g = co(epi f). Let (x, a) G co(epi f ) and b > a. Then there exists (x¡, a¡ ) G epi f : i = 1 . . . n XjG R+, such that n n ) G epi f. П n n Then X.a. + (b - a) = (x,b) G co(epi f). 1 i l Successively we prove that, for the obtained g:X R, there exists a G : X -* R such that epi G = epi g. _____ In fact, if (x, a) G epi g and b >_a, then (x,b) G epi g, and then, if we let for every x G X, G(x) = inf {X G R : (x, X) G epi g}, we have obviously epi G = epi g. To prove that if (x, a) G epi g, and b > a, then (x, b) G epi g, let us consider a generalized sequence (xa, aa); a G a, such that (xa, a j G epi g, for every a G a and (xH, aa) tends to (x, a) in X X R. Then, for a convenient â G a, it results aa < b when a > a and, since g(xa ) s aa, it results also g(xa) i b when a > a. Then (xa, b) G epi g when a > a, and therefore (x, b) G epi g. _ _ It is so proved that, for the given f :X -* R, there exists a G :X ^ R such that epi G = со (epi f). We prove that this G coincides with P(f). Since it follows from the definition of T(f) that T(f) (x) s f(x) for every x G X we have epiT(f) Э epi f and therefore epi (r(f)) = со (epi F(f)) Deo (epi f) Э epi f We obtain so epi T(f) 3epi G Depi f and therefore f**(x) = Г (f) (x) s G(x) s f(x) Vx G X Then (f**)* (x*) ïG *(x*) s f*(x*) and, using theorem 13, we see that (x*) = G*(x*) = f*(x*) v x * G X* 406 CECCONI Then (f***)* (x) (f**)**(x) = G**(x) = f**(x) for every x G X. But as G and f** are convex 1. s. c. functions on X, f**(x) = (f**)** (x) = G**(x) = G(x) Vx G X and also epi Г (f ) = epi f** = epi G = cô (epi f) The theorem is so completely proved. Let us consider a few examples. Example 1. If f(x) = + oo [resp. - uo] Vx G X*, then f*(x*) = - oo [resp. + oo] for every x* G X*. Example 2. If A is a subset of X and is the corresponding indicator function (i. e. ^ (x ) = 0 if x G A, á£(x) = + oo if x ^ A), then (■^)* (x*) = sup { Example 3. If f(x) = <х, x*X then f*(x*) = sup « x , x *^ - + 00 if x* f x;¿ 0 if x* = x* { о Example 4. Let X = R and p G R such that p > 1. Let l/p + l/q = 1 and consider f : R -* R such that f(x) = | x |P/p . Then f is a convex continuoi function on R. Obviously X* = R* = R and hence results But for every y G R, lim We have therefore f |x|p \ ( i .p- 1 |x|P f-'(y) = sup I x y ------— :,x G Rf = Ix |x | sig(x) - P у = I x|P 1 sig(x) p_ P I . 1 \ \ 1 i |P_1 1 I |Q From theorem (v) we have also I |P i |4 lx l У xyJ § p + ------q V x, y £ R the well-known Holder's inequality. 3. SUB-DIFFERENTIAL OF A CONVEX FUNCTION Definition 1. Let f:X -*R be a proper real extended function on a 1. c. s. X and x* G X*. We say that x* is a sub-gradient of f at a point x 0 e X where f(x0) G R i f f(x) ê f(x0) + (x-xQ, x*^ V x G X (3.1) If a sub-gradient exists we say that f is sub-differentiable at the point x 0. The set of all sub-gradients x* of f at x0 is denoted by 9f(x). The multi valued mapping f :x G X -* 9f(x) С X* is called the sub-differential of f. If f is not differentiable at x we have also 9f(x) = 0. Geometrically, the condition that x* is a sub-gradient of f at xn means that the affine closed hyperplane in X X R {(x, a) G X XR : Theorem 1. If f :X -* R is a real extended function on a 1. c. s. X, then (i) f (x0 ) = min {f(x) : x G X } if and only if 0 G Э f( xQ ) (ii) x* e 9f(x0) if and only if f(x0) + f*(x*) = (iii) 9f(x0) f 0 implies f(xQ) = f**(x0) (iv) 9f(x0 ) is a convex subset of X* which is closed in the Proof. To prove assertion (i) we observe that, if f(x) ê f(x0) V x G X, then f(x) s f(x0) +<(x-x0 ,O> so that 0 6 9 f(x 0). 408 CECCONI Conversely, if 0 € 9f(x0), then (3.1) holds with x* = 0 so that f(x) й f(xQ) V x €X . To prove (ii) we observe that, if x* G 3f(x0), then we have f(x) ê f(x 0 ) + < x - Xq , x * > V x e X, and also -f (x) + Theorem 2. Let f :X -* R be a proper convex real extended function on a 1. c. s. X which is continuous at a point x 0 G X. Then 9f(x) f j V x G (dom f)°. M oreover we know that x 0 G(dom f)° so that 9f(x0) = 0. Proof. Since f is continuous at x fl, there exists an open set containing x 0 where f is bounded above. By theorem 5 of section 2 we obtain that (domf)° f 0, and that f is continuous on (dom f)°. Therefore it suffices to prove that 9f(x0) f 0 . Since f is convex, epi f is a convex subset of X X R; since f is continuous at x 0, (epi f)° f 0. In fact, there exists an open subset U of X such that x 0 G U and f(x) § К on U, and therefore V = U X ]K, +oo [ is an open subset of X X R such that V С epi f. Since (x0, f(x0)) belongs to the boundary of epi f, then by corollary 3 of section 1 we can separate (x0 ,f(x 0)) and (epi f)° with an affine closed hyperplane H = {(x, a) G X X R : so that a È 0 . If a = 0 we would have <(x-x0 ,x*)> S 0 V x G dom f and, since (dom f)"i 0, x* = 0, so that (x, a) -> ^x, x*)> + « a = 0 o n X X R; this is a contradiction. Therefore it is о > 0. Then we have so that This proves that G 9f(x0). The theorem is so proved. IAEA-SMR-17/63 409 Let us recall the definition of the Gateaux differential of a real extended function in a 1. c. s. X and investigate the connections between the Gateaux differential and sub-gradients. Definition 2. Let f :X -» R be a real extended function on a 1. c. s. X, and let y EX. We call a derivative of f at x0 G X in the direction y and denote it by f 1 (x0, y): lim f(x +_tyj _з_ fQ_)_ t - o+ t provided the limit exists in R. If there exists an x* G X* such that f'(x 0 ,y) = <^y, x*)> for every y £X we say that f is Gateaux-differentiable at the point x 0 and we call x* the Gateaux derivative of f at the point x0 and denote it by f'(x 0). Obviously the Gateaux derivative is unique, provided it exists. Theorem 3. Let f :X -♦ R be a proper convex function on a 1. c. s. X. If f is Gateaux-differentiable at x0 6 X, then 8 f(x0) = {f'(x0)}. Conversely, if at x 0 G X, f(xQ ) G R, f is continuous and 9f(xQ) = {x*} , then f is Gateaux- differentiable at x0 and f'(x 0) = x*. Proof. If f is proper convex and Gateaux-differentiable at x 0 and, for every yGX, we consider \->cp(\) = f(x 0 + Xy), then cp is a proper convex function on R, differentiable at xQ, and cp '(0) = 'vXg, f'(x0))>. We have therefore f(x 0 + y) - f(x0) = cp(l) - f'(x 0 )G 8 f(x0) On the other hand,if x* G 9f(x0) we have V y GX and X G R + f(x 0 + Ху) ë f(xQ) + < Xy, x*> so that f(xp + Xy) - f(x0 ) g therefore, going to the limit for X -♦ 0+, \ 410 CECCONI It follows that < y ,f '( x 0) - x*)> =0 V y G X and therefore x* = f'(x0 ). Thu sit is proved that9f(x0 ) ~ f '(x 0 ). Conversely, let f be convex and let 9f(x0) contain a unique element. Since f is proper convex, f(x0) G R and f(x) is continuous at x0, we have V y G X, f(x 0 + Xy) - f(x0) S f '( x 0, Xy) V X such that x 0 + X y G (dom f)° and therefore this holds V X. It follows that the one-dimensional subspace A = {x 0 + Xy, f(x0) + Xf'(x0, y ):X G R } ofX X R does not intersect (epi f)° which is a convex, non-empty set of X X R since f is continuous at Xq . By corollary 2 and theorem 4 (of section 1) there exists a closed affine hyperplane H containing A such that H Л (epi f)° = 0. If H = {(x, a) G X X R : <(x, + a a = 7 }, where x* G X *, a, 7 G R, and (x, a) -» Let us observe that a f 0; in fact, if a - 0, then < x 0,x*> + X so that x* = 0 and, consequently, <^x,x*)> + aa = 0 V (x, a); a contradiction. Then we have and therefore V x G X. We have so proved that - — G 9f(x0) and therefore Let us recall that then we obtain <^y,x*^> = -Q 'f'(x(),y ) for every y G X so that f is Gateaux- differentiable with the Gateaux derivative -x * /a. For a Gateaux-differentiable function the convexity can be characterized in the following form: Theorem 4. Let f : К -* R be a real valued function on a convex non-empty subset of a 1. c. s. X which is Gateaux-differentiable at every x G К in the following sense: there exists an f'(x)G X* (not necessarily unique if x ^ K°) such that for every y G X for which there exists an e G R+ such that f(x + ty) - f(x) x+óyG K if 6 G ] 0, e [,then lim = <(x, f'(x)X Then f is t-> o+ t convex on К if and only if f(x) È f(xQ) + < x-x0, f'(x) У V xQ, x G K; f is strictly convex if and only if f(x) > f(xQ) +<(x-x0, f'(x0))> V x G K, x f x„. IAEA-SMR-17/63 41 1 Proof. If f is convex on К and Gateaux-differentiable, then the arguments used in the proof of theorem 3 also prove that f(x) ê f(xQ) + Conversely, if f is Gateaux-differentiable at every x 6 K and f(x) ê f(x0) + f(x) i f [x (1 - or) + y ] + о <^x - y, f 1 [x(l - a) + ay ] )> f(y) 5 f [x(l - ct)+ ay ] + (1 - a) < y -x , f' 1(1 -»)x + ay ]> We multiply the first inequality by (1 - a) and the second by a and sum up to obtain (1 - O') f(x) + a f (y) ë f [ ( 1 - or ) x +o so that f is convex in K. If it is strictly convex and Gateaux-differentiable we have for every xQ, x € K, o; € ] 0 , 1 [, xQ f x, < x-x0, f.(x0)> S ftxQ+?(x-^n)]-f(x a) t ,.(i.-^)f(xo), : .fUo). = f(x) _ f(Xo) Conversely, iff is Gateaux-differentiable and f(x) > f(x0 ) + (x -x 0, f'(x0)) Vx f x 0, then the argument used above proves that f [o’X + (l-o ,)y)]>o'f(x) + (l-or)f(y) if x f y, a € ] 0, 1 I The theorem is completely proved. DEFINITION 3. Let tp : X -» X*, a map of a 1. c. s. X in the space X*. We say that ip is monotone if, V x, y G X, < x - y, <¡p(x) - <¡e> (y) > SO With this definition we can prove that 412 , CECCONI Theorem 5. Let f : К -* R be a real-valued function on a convex subset К of a 1. c. s. X, which is Gateaux-differentiable at every x € K. Then f is convex if and only if f'(x) is monotone on K. Proof. If f is Gateaux-differentiable at every x G К and convex, then for every x, y G К there exist f'(x), f'(y) G X* such that f(x) È f(y) + f(y) S f(x) + Adding these inequalities yields f[x + (X+h) (y-x)] - f [x + X(y-x)] lim *'(X) h h 0 = < y -x , f'[x + X (y - x) ] > Therefore (X -p ) { = so that ip'(X) is an increasing function on [0, 1] and and therefore f [x(l - X) + Xy ] = The theorem is so proved. IAEA-SMR-17/63 413 We conclude these preliminary topics on convex-functions theory with the following theorem which, for convex functions, connects the property of being 1. s. c. with the property of being Gateaux-differentiable: Theorem 6 . If f :X - R is a proper convex function on a 1. c. s. X which is Gateaux-differentiable at x, then f is 1. s. c. in the ct(X,X*) topology at x. Proof. Let x converge to x 0 in the ct(x,x*) topology; we must prove that lim f(x) ë f(xQ). From theorem 3 follows that Vx GX x -» Xo f(x) ë f(x0) + < x -x Q, f'(x 0 )> therefore, if we go to the limit in the topology a (X,X*), we obtain lim ^ x - x 0, f'(x 0)^> = 0 and consequently X -»Xo lim f (x ) ë f (Xq ) . X ->x0 The theorem is so proved. 4. MINIMIZATION OF CONVEX FUNCTIONS In this last section we give the most essential results on the minimiza tion of real-valued convex functions on Banach spaces. Theorem 1. Let К be a closed convex set in a Banach space and let f : К -* R, a real-valued convex 1. s. c. function on K. We suppose furthermore that either (i) К is bounded, or (ii) lim f(x) = + oo llxll— X £ K Then there exists an xQ 6 К where f(x) has a minimum in K, i. e. f(xQ) S inf {f(x) : x € K } If f is strictly convex then x Q is unique in K. Proof. Let us consider X= inf {f(x) : x € K}; then X E f- » , + oo [. Let x n be a minimizing sequence in K, i. e. a sequence such that lim f(x' n ' ) = X n-> °° 414 CECCONI The set A = {x n : n G N} is bounded in X: in fact, in the case (i), A = {x n : n G N) С К and К is bounded; in the case (ii) A is bounded since the sequence {f(xn)} is bounded above. Then there exists a sub-sequence {xn } of xn which converges in the f(x0) § Um f(xn.) = X i -» .o this proves that f(x 0 ) = A = inf (f(x) : x G K} and also that X f - oo. If Xj, x 2 G X are such that f(Xj ) = f(x2) = X, then from the convexity of f we have f (xj + x 2) ] S fW x ^ + f(x2)) = X so that also - - G К is a point of minimum for f in K. Therefore, if f is strictly convex we cannot have f ^ ) = f(x2) = X with Xj , x 2 G К and therefore there exists a unique x 0 GK where f has its minimum. Theorem 2. Let f : К -* R be a convex real-valued function on a closed convex subset К of a Banach space which is Gateaux-differentiable at every x G K. M oreover, lim f(x) = +ooif К is not bounded. Then there IUII-” x e K exists an xQ G К such that f(x0) = inf {f(x) : x G К}; moreover Conversely, if xn G К satisfies (4.1) [resp. (4.2) and f' is continuous on К ], then f(xQ) = inf {f(x) : x GK}. Proof. From theorem 6 of the preceding section follows that f is 1. s. c. on К in the a(X ,X *) topology. Therefore from theorem 1 it follows that there exists an x 0 G К such that f(xQ) = inf {f(x) :x G К }. Then, for every x G К and XG [0,1 ] f(x0 ) ë f 1(1 - X )x0 + Xx ] and consequently ÜI!iîA (ü.-.xo.H-{(]!ii) g о X G [0 , 1 ] X Going to the limit for A 0 + ^we have (4.1) is so proved. But from theorem 5 of the preceding section it follows that < x -x Q, f'(x) - f'(x0)> SO V x , x qG K Addition yields and (4.2) is proved. Conversely, let x 0 E К satisfy (4.1). Then from the convexity of f in К follows V x G K, X e ] 0,1 [ f(x) - f(xQ ) г i {f [(1 - X )xQ + Ax ] - f(xQ)} and going to the limit X -* 0 + f(x) - f(xQ) S < x-x0, f'(x0)> § 0 so that f(xfl) = inf {f(x):x G К} and consequently (4.2) is satisfied. Finally let xQG К satisfy (4.2) and let f'(x) be unique V x £ K and continuous on K. By taking in (4.2) x = (1 - X)x0 + Ay, y G K, A G ] 0,1 [ we obtain <У- x0, f' [xQ(l - A) + Ay ]> SO and therefore < y -x 0, f'[x0(l - A)x0 + Ay]> S 0, y£K; AG]0,1[ In the limit A -* 0 +, from the continuity of f' we have (4.1). Remark 1. If the point x 0 in the hypothesis of the theorem is such that x 0 g K° then f'(x0) = 0 according to theorem 1 of section 3. Remark 2. If x 0 satisfies Theorem. (Minty-Browder) Let f : К -» X* be a monotone map of a closed convex subset К of X on X*, we suppose that (i) f is monotone on X (ii) V x, y G X the real function t -» < x - y, f [x +t(x т у) ] У is continuous on R. (iii) there exists a Ç G К such that Then V x* G X* there exists an x 0 G X such that <(x - x0, f(x) - x*) ^ 0, x G К Example 1. Let a(x, y) be a bilinear symmetric continuous form in a Banach space X, i. e. a map (x, y) G X X X -» a(x, y) G R which is linear in both variables x, y, and let it be such that | a(x, y) | S M || x || ||y|| V x, y G X and a convenient M in R. Let us suppose that a(x,y) is coercive, i.e. there exists an a GR + such that I a(x, x) | ê а ||x ||2 V x G X. Let x* G X* and consider the real function f(x) = a(x, x) = 2 ^x, x* У on X. Then f is a strictly convex function on X. In fact, from (4.3) a(x - y, x - y ) i 0, if x, y G X, we obtain a(x, x) - 2a(x, y) + a(y, y) ë 0 (4.3) and therefore 2a(x,y) s a(x,x) + a(y, y) (4.4) m oreover, the equality in (4.3), (4.4) holds if and only if x = y. From (4.4) follows that, V x.yG X , X G [0,1] a iXx + (1 - X)y; Xx+(1-X)y] = X2 a(x, x) + 2X(1 - X) a(x,y) + (1 - X)2 a(y, y) S X2 a(x, x) + X(1 - X) {a(x, x) + a(y, y)} + (1 - X) 2 a(y, y) S X a(x, x) + (1 - X) a(y, y) so that, from the linearity of <(x,x* on X, it follows that lAEA-SMR-17/63 417 f lAx + (1 - A.) y ] S Af(x) + (1 - A) f (y) and the equality holds if and only if x f y. Obviously the function f is continuous (and consequently 1. s. c. ) on X and such that lim f(x) = + oo. In fact, for every x G X, from the llxll-» coercivity of a(x, y) it follows that f(x) = a(x,x) - 2 ^x,x*^> s a || x ||2 - 2 ||x|| • ||x* || and since Hence lim f(x) = + oo II X II-+ » Therefore, if К is a closed convex subset of X, there exists, by theorem 1, a unique x 0 e К such that f(x 0 ) = inf (f(x) : x G к}. Since f is obviously Gateaux-differentiable at every x G X with the Gateaux-derivative f'(x) such that <(y, f'(x)> = 2 a(x, у) - <^y, x* У, theorem 2 entails that at x 0 a(x0 ,x- x0) - We have so proved that the variational inequality (4.5) has one and only one solution in K. BIBLIOGRAPHY EKELAND J., ТЕМАМ, R., Analyse convexe et problèmes variationnels, Dunod, Paris (1974). LIONS, J.L., Quelques methodes de risolution de problèmes aux limites non linéaires, Dunod, Paris (1969). ROCKAFELLAR, E.T., Convex analysis, Princeton Univ. Press (1970). ASPLUND, E., "Topics in the theory o f convex functions", Theory and Applications o f Monotone Operators, A. Ghizzetti (1965). IOFFE, A.D., TIKHOMIROV, V.M., "Duality in problems of the calculus of variations", Trudy Moskov. Obs. 18 (1968) 187. 418 CECCONI IOFFE, A.D., TIKHOMIROV, V.M., "Duality of convex functions and extremum problems", Uspekhi Mat. Nauk (1970). LIONS, J.L., "Partial differential inequalities", Uspekhi Math. Nauk (1972). ROCKAFELLAR, E.T., "Convex functions, monotone operators and variational inequalities". Theory and Applications of Monotone Operators, A. Ghizzetti (1969). IAEA-SMR-П /29 AN INTRODUCTION TO PROBABILITY THEORY J. ZABCZYK Institute of Mathematics, Polish Academy of Sciences, Warsaw, Poland Abstract AN INTRODUCTION TO PROBABILITY THEORY. The purpose of this paper is to give the probabilistic foundations of stochastic control theory and to show some applications of probability theory to functional analysis. Such topics as martingales, conditional expectations, Wiener process, linear stochastic equations and Ito's integral are treated in a rigorous way. Only the elements of integration theory and normed spaces are taken for granted. INTRODUCTION The purpose of this paper is to give the probabilistic foundations to stochastic control theory and at the same time to present some applications of probability theory to functional analysis. The formal prerequisite of the paper is familiarity with the concept of (probability) measure and Lebesgue's integral and a knowledge of such theorems (from integration theory) as Lebesgue's theorem, Fatou's lemma, Lebesgue's monotone convergence theorem, although all these theorems are formulated in the text. To make use of elementary parts of the measure theory only, we omit all kinds of "extension theorems" such as the Carathéodory or Kolmogorov theorems and instead we take for granted the Lebesgue's measure on the Euclidean space IRn. We also assume as known the elementary properties of Hilbert and normed spaces. With these exceptions, the paper is self-contained and complete although some proofs are given through problems. The text is divided into five chapters: Preliminaries, Martingales, Conditioning, Wiener Process, Ito's Stochastic Integral. Preliminaries contain basic information on a -fields, measurability and independence. Examples of normal distributions, independent random variables are also introduced. As far as applications to functional analysis are concerned the next chapter on Martingales is most important. "The basic definitions (of martingale theory) are inspired by crude notions of gambling but the theory has become a sophisticated tool of modern mathematics drawing from and contributing to other fields" [1]. We apply martingales to prove fundamental properties of Haar and Rademacher systems and the Radon-Nikodym theorem. The main results of this chapter are Doob's inequalities and the Martingale convergence theorem. Some relations between martingales and Markov chains (important in stochastic stability) are also indicated. The material of the chapter Conditioning is extensively used in stochastic control theory (see the lectures "Stochastic control of discrete-time systems"). An effort is made to link general definitions with concrete examples. Chapters 4 and 5 treat of a Wiener process and Ito's integral. They can be considered a starting point for stochastic differential equations and for 419 420 ZABCZYK stochastic control of continuous-time systems. Linear stochastic equations and their physical interpretation ("white noise") are also considered. Preparing the paper, we have used many existing books and articles on probability and martingales, among others an article on martingales by J .L . D o o b [l] and the books "Probability" by J. Lamperti, "Probability and Potentials" by P. A. Meyer, "Stochastic Integrals" by H. P. McKean Jr. [2, 3, 4 ] . 1. PRELIMINARIES 1.1. Random variables and generated g -fields Let Í2 be a set. A collection ^ ”of subsets of Í2 is said to be a ct - field (ct- algebra) if 1 ) n e ÍF 2) If A e & then Ac, the complement of A, belongs to ^ too, + 00 3) If A n e n = 1, 2, . . . then U An e !P~ n = 1 The pair (Q,,^) is called a measurable space. Let (E,<#) be two measurable spaces. A mapping X from Г2 into E such that for all A e S the set {u : X(u) e A} belongs to^ is called a measurable mapping or a random variable. Problem 1. Show that the composition of two random variables is also a random variable. Let be a collection of subsets of fi. The smallest ct -field on Ç1, which contains is denoted by о(1Ж) and is called a ct-field generated by It is obtained as the intersection of all ct-fields on Q containing Analogously, let (XjJjgj be a family of mappings from Í2 into (E, S') then the smallest ct-field on ÍÍ with respect to which all functions X¡ are measurable is called a p-field generated by (X¡)ie t and is denoted by a (X¡ : iei). Example 1. Let X be a mapping from into (E, S) then cr(X) = {Aa: ae<#} where Aa = {и : Х(ш) e a}. Let E be a metric space. The cr-field on E. generated by all open (or all closed) subset of E is called a Borel ct -field and denoted £S(E). Let IR1 be the real line. A real-valued random variable is a random variable with values in (IR1, (fô (IR1)). A random variable which takes only a finite number of values is called a simple (or elementary) random variable. The proofs of the following two propositions are easy and left to the reader (see also Ref. [5]). Proposition 1. Let X, Y, X lf X2, ... be real-valued random variables and a, /3 6 K 1, then X V Y = max (X, Y), X A Y = min (X, Y), aX + /3Y and lim sup Xfi, lim inf Xn are also random variables (in general, with values in (R ,^(E )i where E = К и {-00, + °°}). Proposition 2. If X is a non-negative random variable (r. v. ), then there exists an increasing sequence of elementary non-negative r.v .'s that converges to X. IAEA-SMR-17/29 421 Let X be a measurable mapping from into (E, <8). Very often it is necessary to consider real-valued r.v .'s which are measurable with respect to cr(X). The following lemma shows that such r .v . 's are measurable functions of X. Lemma 1. A real-valued function Y defined on Q is measurable with respect to cr(X) if and only if it is of the form Y = f(X) for some real-valued r.v. f defined on (E, S). Proof. If f is a real-valued (^-measurable function then f(X) is ct(X)- measurable as a composition of two random variables (see Problem 1). Let now Y be any ct(X)-measurable function. We can assume that X is non negative. If Y is the indicator IA of a set A € cr(X) then (see Example 1) A = {Xe a} for some a € if and, therefore, Y = Ia(X). Consequently, the lemma is true in this case. Since every simple r.v. is a linear combination of indicators, the lemma is true for all simple r.v .'s. If Y = lim fn(X) and n 11 fn are «^-measurable then Y = f(Y) where f = lim inf fn. Application of Propositions 1 and 2 finishes the proof. n Let (E^ ..., (Еш #„) be measurable spaces then by X . . . X Problem 2. Let Xi and X2 be mappings from Q into E x and E2, respectively. Show that the mapping (X 1; X 2) from Í2 intoEj X E 2 is a random variable if the mappings Xj^ and X 2 are random variables. Hint: The family of all sets A&Sl X <^2 such that {u : (X^u), Х 2(ш)) € A} 6 ^"is a a-field. Lemma 2. Let ..., be some families of subsets of E1; E2, . . ., En respectively and let ^ b e the collection of all sets AjX ... X An, AjG Sx, ___ Ane ct(^) = X ... X о ( ^ п) Proof. Since X ... X о(л?п) therefore а(^Ж)С X . . . X ‘ On the other hand let ^ be the family of all sets A j6 a (^ j) for which Ej X ... X Е Ь 1 X A¡ X Ei+1 X ... X EnS о^Ж). Then ^ is a a -field а-nd contains all sets from , thus ^ = а(^л^). From this if Ai Ga(^'i), i = 1, 2, . . ., n then A jX . . . X АП = .Г1 Ei X . . . X E¡-i X A¡ X E¡+i X . . . X En € a (.j?) and a( X . . . X а ) с ст{лt ). Corollary 1. For any natural numbers p1; p2, . . ., pn ^(IRPlX ... X HPn) =^(H Pl) X ... X &{TRn) Corollary 2. Let X = (X1( . . ., Xn), where X¡ are random variables with values in (IRP, ^(IR P)), then any real-valued function Y, a(X)-measurable, is of the form Y = f(X1; X2, . . ., Xn) where f is a Borel function on ]Rnp. A collection Л oi subsets of Г2 is said to be a я--system if, ф € Л and if A, B e J ' then А П В £ / 422 ZABCZYK Example 2. If E is a matric space then the family of all open (closed) subsets of E is a 7r-system. If E = К or E = [0, 1) then respectively {( - 00 , b] : b G IE} and {[a, b) : 0 s a s b < 1} are гг-system s. The lemma below will be used frequently in the sequel. Lemma 3. If Л is a -v-system and Si the smallest family of subsets of such that 1) ЛС Si 2) If A G Si then AcGSi +» 3) If Aj, A2, .. ., € Si and АпПАт = Proof. Since о(Л') satisfies 1), 2) and 3), SÍC To prove the opposite inclusion, we show first that ^ is а к-system . Let AG Si and define ^ д = {B: B € ^ and А Л B 6 ^ }, It is easy to check that SiA satisfies 2) and 3) and if A€^fthen the condition 1) is also satisfied. Thus for A 6 u Problem 3. If a 7r-system Si satisfies 2) and 3) then Si is a cr-field. Lemma 4 . ' Let f be a measurable mapping from (Ex X E2, i j X <£ 2) into (E ,#). For every XjGEj, f(xj, • ) is a measurable mapping from (E2, into (E, S), P roof. Assume first that E = H and S - &(IR) and let ^ be a family of all sets A e ^ X S2 such that for all XjEEj the function Ia(xj, • ) is a real valued r .v . on (E2> S2). If A = A 2 X A 2 where Al&é?1, A 26< ^ 2 then clearly A 6 ÿ . Moreover Si satisfies conditions 2) and 3) of lemma 3, therefore n Si = S X <%2. Let f be a simple function then f(Xj, • ) = ai^A-(x i’ ‘ ) ^or 1 = 1 i some disjoint sets A¡ e é>1 X <#2real numbers € IR i = 1, 2, . . ., n and all X jêE j. Taking into account that a linear combination of measurable functions is also a measurable function, we see that f(xj, • ) is measurable in this case, too. But every non-negative measurable function f is a limit of simple functions, therefore the lemma is valid for all non-negative, thus for all measurable function. If (E, #) is an arbitrary measurable space and AG S , then the composition IA(f) (1д is the indicator function of A) is a random variable on Ej X E2. Applying the first part of the proof, we see that for every xxG E the function IA(f(xj, • )) is (#2 -measurable. But then { x 2:f(x j, x 2)£ A } = {x2: Ia^X],, x2) = 1} G ^ and the proof is complete. 1.2. Integration The definition of the integral / XdP of a real-valued r.v. is taken Q for granted (see Refs [3, 5]). We only recall that the integral is well defined if either / X+dP < + °° or / X"dP < + °°, where X+ = XV 0, X ’ = (-X)VO and ii n that / XdP = / X +dP - J X*dP. If f X dP < + °° we say that X is an integrable n n fi n IAEA-SMR-17/29 423 random variable. In probability theory, the integral of a random variable X is called the expectation of X and is denoted E(X). F or the proof of the following two theorems we refer to Ref. [5] . Lebesgue's theorem. Let X^ X2, be real-valued r.v.'s. If for some integrable r.v. Y, |xn| s Y almost surely, n = 1 , 2 , . . . , and the sequence (Xn) converges (a. s.) to X, then E(Xn) -* E(X). Fatou's lemma. Let Xj, X 2, be non-negative r . v .'s , then lim inf E(Xn) s E(lim inf Xn). If, in addition, the sequence (XJ is increasing n n then E(Xn) -* E(X). This last property is called Lebesgue's monotone convergence theorem. 1.3. Independence Let {Ci, p) be a probability space and let { & eI be a family of sub-ct-fields of These ct-fields are said to be independent if, for every finite subset JC I and every family such that A ¡G ^, iGJ, P( П A-) = П P(Aj). Random variable (X^igj are independent if the i 6 J i G J ct-fields (^(Х ; ) ) 1 е 1 are independent. Problem 1. Let X j and X 2 be two real-valued r.v .' s, Xj measurable in the respect to X 2 with respect to If &\, ¡?~2 are independent ct-fields and the expectation E(X jX2) is well defined, then EfXpC,) = Е(Х1Щ{Х2) (1) Hint: Assume that X x, X 2are non-negative; show first that (1) holds for simple real-valued r.v .'s. Use Proposition 1, Section 1.1. and Lebesgue's monotone convergence theorem (Section 1.2). Example 1. Let us consider the probability space ([ 0,1 ), 3S [ 0,1), P) where P is the Lebesgue's measure on the interval [ 0,1) and define for every n = 1 , 2 , . . . and к = 1,2, . . . ,2 n intervals l£ = [ ~¡T> ~ТГ)- Random variables 0 if w G I,, к odd к х пИ =■ 1 if u G I., к even 4 к are independent P roof. It follows, by induction on n, that for every e¡ = 0 or 1, i = 1 , 2, . . . . n, {u-.X^u) = e1, . . . , X n(y) = en} = + 2 n • 2 1 2n This implies that P(Xj = e1, .. ., Xn = en) = — = П ^ P(X¡ = e¡). An application of the definition of independence and the fact that AíG ct(Xí) if A¡ = {u:X iGai} for some Borel set aj (see Example 1 (Section 1.1)) finishes the proof. 424 ZABCZYK X k(u) Problem 2. Show that for every u e [0 ,1 ), to = E ~k— * (dyadic £ = i 2 2 n L 2 1 Lemma 1. Let be a 7r-system on Q and let ^ = а(^^), i 6 I. The a-fields ( ^í)¡gi are independent if for every finite set J C I and sets i e j , P( Pi A ) = П P(Aj). P roof. Assume, without loss of generality, that I = J = (1, 2, . . . , n}. Let us fix the sets A2, A3, . . ., An and denote by the family of all sets Aj e ^ for which n n P( П Ak) = П P(Afc) (2 ) k = l k = l The family and the ж-system satisfy the conditions 1), 2) and 3) of Lemma 3 (Section 1.1) therefore &1 = Analogously let us fix sets Aj C orollary 1. Random variables X¡ with values in (E ¡, c f ^ j) ) where are 7r-systems on E¡, iGI, are independent if P( П {XjGAj} ) = П P(X¡G A¡) for any finite subset JCI and sets Aj 6 , i e j . Proof. It is sufficient to remark that a(Xj) = а({Х;еА;}: А;е а(^ ;)) = CT({Xi€Ai}: A iG ^i) Corollary 2. Since all intervals (- °°, x], x e ГО. form a ж-system which generates ^(IR) therefore real-valued r.v .’s Xx, X2 , . . ., Xn are independent if and only if n n P( n {Xj § Xj}) = П P(X¡ S x¡) i=l i=l for any real numbers Xj, x%, . . ., xn. Problem 3. Let X1( X& ... be independent random variables with values in (E, <£) and let Jlf J2, ... be a finite or infinite sequence of disjoint subsets of {1 ,2 ,...}. Then the a-fields ^ = cr(Xk; kejj ), i = 1, 2, . . . are independent. Hint: a-fields are generated by ж -systems of all sets Л {Xk€Ak} where Ake # and only for a finite number of k, Ak f- E. e 1 The lemma below is an important generalization of the Problem 1. The proof is analogous to that of Lemma 4 (Sec. 1.1). Lemma 2. Let f be a real-valued random variable defined on (E iX E2, Sx X <8 2) and let Х ъ X 2be independent random variables from IAEA-SMR-17/29 425 (Q, P) into, respectively, (Ej, S ]J, (E2, S2). If the expectation E (f(X 1,X 2)) is well defined then the function f^- ) = E (f(-, X2)) is ^-m easu rable and E(f(X 1,X ^ ) = E (f1(X1)). Proof. It is sufficient to prove Lemma 2 for functions f = IA where A E ^ X <#2 - Denote by the family of all sets A such that for f = Ia the lemma holds. Then Si where ~^is the тг-system of all sets Bi X B2 , B2 G<^2 " Since Si X S2 = oÇJ?) and the family satisfies the assumptions 2) and 3) of Lemma 3, Section 1.1; therefore, this yields & = #x X . 1.4. Distributions of random variables Let (Q, P) be a probability space, (E, S) a measurable space and X a random variable from Q into E. The distribution of the r.v. X is a probability law on (E, $) defined as mx(A) = P(u : X(u) G A), AG S. Lemma 1. Let ц and v be two probability measures on (E,а(.Ж)) where a 7r-system. If ju(A) = v(A) (1) for A e ^tthen (1) holds for all AGcr(, Proof. Denote by & the family of all sets AEoJ^j for which (1) holds. Then Si satisfies assumptions 1), 2) and 3) of Lemma 3 (Section 1 .1 ) and, consequently, & = Corollary 1. Let ц be a probability measure on (]RP, ¿^(]RP)). Then the distribution function F^ of ц defined as F^ (x) = ju(y :y s x), x E E p determines ju uniquely, ((y1( . . ., y ) s (xp . . ., Xp) means Xj S yj, . . ,,x p s yp. Proof. Since the family of all sets {y : y s x}, xGIRp is a 7r-system and 3B{W(P) = о(Л?) therefore Corollary 1 follows from Lemma 3, Section 1.1. Thus if X is a r. v. with values in Hp then the distribution function of X defined as Fx = F^ determines the distribution of X. Example 1. If X is a real-valued r.v. such that f 0 for t < 0 Fx(t) = j t for 0 s t < 1 ( 1 for 1 s t then the distribution of X is the Lebesgue's measure restricted to [0, 1] . We say, in this case, that X has a uniform distribution on [0 ,1 ]. The concept of distribution is often applied through the following lemma: Lemma 2. If f is a non-negative random variable defined on (E, S') then E(f(X)) ( 2 ) E 426 ZABCZYK Proof. Formula (2) holds if f = IA, AÇ.S and thus for any simple random variable. If f is a non-negative r.v. then f = lim f for some increasing n sequence of simple non-negative r.v. fx, f2, . . . . Since j fn(X)dP = n / fn(x)/ux(dx); therefore, (2) holds because of Lebesgue's monotone conver- E gence theorem (Section 1.2). Corollary 2. The random variable f(X) is integrable if and only if the function f is йх -integrable. Let ц be a probability measure on (IRn, ^(IEn)) such that / I x 12 az (dx) < + °o; ( I x I2 = x 2 + . . . + x 2). The column vector m with Rn 1 n components m¡ = / x;Ju(dx), i = 1 , .. ., n is called the mean vector of ц and Rn the matrix Q = (°"í j )í j =i ...... where a¡ j = / (x¡ -m¡ )(Xj-mj)/u(dx) is called the Rn covariance matrix of ¡л. Let us remark that if ц = дх is the distribution of a random variable X with components X 2, X2, . . . , X n and E(X2) < + °o, i = l , . . . , n then, by virtue of Lemma 2, m4 = Е(Х4), ctj j = E(Xj -mj)(Xj -m¡) i, j = 1, . . ., n. Vector m and matrix Q are called in this case the mean vector and the covariance matrix of X. If T is a linear mapping from Rn into Rk (or, equivalently, T is a к X n matrix) and m is a vector from Rn, then T' and m' denote, respectively, the conjugate mapping (the transpose matrix) and the transpose vector. Problem 1. Show that the mean vector and the covariance matrix of the random variable TX are equal Tm and TQT1. Hint: E(TX) = TE(X) and E(TX-Tm) (TX-Tm)' = T(E(X-m)(X-m)')T'. 1.5. Normal distributions Let ju be a measure on (IRn, âê(TRn)) and g a Borel function such that ju(A) = J g(x)dx for all Ae^(IRn) A then the function g is called a density of the measure ц (with respect to the n-dimensional Lebesgue's measure which is taken for granted). Here, we introduce the normal distribution through the following problems (see Ref. [ 6 , v o l.2]): Problem 1. Let R be an n X n sym m etric matrix, then the function exp {-(1/2) ^ Rx, x ^>}, xGIRn is integrable on IRn if and only if ^ Rx, x )> > 0 for all x f 0, i. e. if the matrix R is positive-definite. Hint: /Г e -rx2 dx, < + °° if and only if 7 > 0. Use an appropriate system of co-ordinates on ]Rn. IAEA-SMR-n/29 427 Problem 2. Let Q be an n X n positive-definite matrix and m a vector from Hn, then the function gm Q: (x) = ((27r)ndet Q ) " 2 exp {-(l/2) x2 1 f e 2xdx = 1 , X > 0 . •J 2irX J This probability law is called a (non-degenerate) normal distribution. Problem 3. The normal distribution with the density gmiQ has the mean vector m and the covariance matrix Q. Hint: See Section 1.4. Problem 4. Let a random variable X with components Xx< . . . ,X n be normally distributed, then, for every k g n, the random vector with components X^ X 2, .. ., Xk is also normally distributed on ]Rk. Hint: use induction on к = n ,n -l,. ,,, 1 , If ц is a measure on IRn and T is a linear map from ]Rn into IEtk then the measure Тд is defined by the formula Тц(А) = m(T_1A), A e ^ (IR k). If a random variable X has the distribution ц then the random variable TX has the distribution Тц. Any measure of the form Тц where ц is a normal distribution is called a general normal distribution. Problem 5. The measure Тд has the mean vector Tm and the covariance matrix T Q T'. Hint: Define o n ii = lRn the random variable X(x) = x, x£ K n and use Problem 1 (Section 1.4). Problem 6 . The general normal distribution T/и is a non-degenerate normal distribution if and only if the linear mapping T is onto or, equi valently, if the matrix TQT' is positive-definite. Hint: See Problem 4. Corollary 1. A general normal distribution is a non-degenerate normal distribution if and only if its covariance matrix is positive definite. Problem 7. The components X1( . . ., Xn of a normally distributed r.v. X are independent if and only if the covariance matrix Q is diagonal (if CTi,j = 0 for 1 f З)- Problem 8 . Let XJ.X 2 , ... be random variables normally distributed (in the general sense) with parameters (mbQx), (m2 ,Q 2), .... Assume that the sequence (Xk) converges almost surely to a random variable X, (mk) -» m, (Qk) - Q and Q is positive definite, then X is normally distributed with parameters (m,Q). 428 ZABCZYK 1.6. Sequences of independent random variables Classical probability theory deals with sequences of independent real valued random variables. In this section, we show that such sequences can be easily constructed if the Lebesgue's measure on [0, 1 ) is taken for granted. The construction goes back to H. Steinhaus. The theorem below will be used as a source of concrete examples in the next chapters. Theorem. Let /u1,/u2, ... be a sequence of probability measures on IR1. There exists a sequence (Yn) of independent real-valued random variables defined on ([0,1), â&{0,l), P) where P is the Lebesgue's measure on [0,1) such that the distributions of Yn are exactly the measures мп> n = 1, 2, . . . . Proof. Let Xb X2, ... be independent random variables constructed in Example 1, Section 1. 3 and let J¡ = { n¡ j : j = 1, 2. . .}, i = 1, 2, . . . be disjoint subsets of the set of natural numbers. Then, because of Problem 3, Section 1.3, the random variables (1 ) j = i are independent. We show that they have uniform distribution on [0, 1). Let, for instance, i = 1 and define П j = l Then P(Sn = к/ 2n) = 1/ 2n, к = 0 ,l,2 ,...,2 n-l, (compare Example 1, Section 1.3) and, therefore, for t€ [0, 1), P(Sn s t ) - t , On the other hand, P(Sn = t) - P(Zj s t). Thus P(Za st) =t. An application of Example 1 (Section 1.4) proves that Zi has the uniform distribution on [0, 1). Now let м be a probability measure on И 1 and let F = FH be its d istri bution function. Define F+(s) = inf {t : s s F(t)}, s € [0 ,l). If Z has uniform distribution on [0, 1), then the distribution of F+(Z) is exactly ju. Really, from the definition of F +, for s e [ 0 , 1 ) and t e ( - « , + « ), s s F(F+(s)) and F+(F(t)) s t. Therefore, {s :F +(s) s t} = [ 0, F(t) ] and, consequently, Р(ш : F+(Z(u)) s t) = P(u : Z(u) s F(t)) = F(t). To finish the proof of the theorem, it is sufficient to remark that if FÏ, F 2+, . . . are functions (defined as above) corresponding to the measures Mi, M2- • • • and Z i, Z2, . . . are real-valued random variables defined in Chapter 1, then the sequence F ^ Z ^ , F*(Z2), . . . has all properties required. 2. MARTINGALES 2.1. Definition of martingales and supermartingales Let T be an arbitrary subset of IR ordered by the relation s. Let (Í2, &, P) be a probability space and (&[)t gT an increasing family of sub-a-fields of IAEA-SMR-17/29 429 A family (Xt)teT of finite real-valued random variables adapted to the family (3 rt)t g t, (i.e. Xt are ^-measurable, teT), is said to be a martingale (or, respectively, a supermartingale, a submartingale) with respect to the’family (^t)t s x ^ 1) Xt are integrable random variables, teT; 2) If s S t, then for every event A A or, respectively Xt dP ê I X„ dP Every real constant (or, respectively, decreasing, increasing) function defined on T is a martingale (or, respectively, a supermartingale, a submartingale). Let (Xt)t e T be a supermartingale. The process ( -Xt )t eT 'is then a submartingale and conversely. Example 1. Let (fi, ¡P, P) be a probability space and let Yj, Y2, ... be a sequence of independent random variables (see Section 1.6) defined on Q such that E | Yn| < + a> and E(Yn) = 0, (or, respectively, SO, È 0) for all n = 1, 2, . . .; then the sequence (Xn), Xn = Yx + . .. + Yn, is a martingale (or, respectively, a super-, a submartingale) with respect to (£?~n), where = ст ( Y1,...,Y n). Really, by virtue of Problem 3, (Section 1.3), the a-fields and a (Yn + i) are independent. Therefore, if then IA and Yn+1are independent random variables. Applying Problem 1 (Section 1.3) we see that J (Xn + l - X n ) d P =/ Yn + 1dP = E(IAYn + 1) = P(A)E(Yn+1) = 0 , A A (or respectively s 0 , s 0 ). Problem 1. An urn contains b black and w white balls. Balls are drawn at random without replacement. Let bn and wn denote the numbers of black and white balls in the urn before the n-th drawing. Construct an appropriate probability space and show that the sequence b n X_ = :- , n = l , . . . , b + w n bnn + w„ n ' is a martingale with respect to ^ = crfXp . . ., X,,). Lemma 1. If (Xt) and (Yt) are supermartingales (relative to (3^)) and a, /3 are positive numbers, then the processes (aXt + /3Yt) and (Xt Л Yt ) are also supermartingales. If (Xt) is a martingale, then (|xt|) is a submartingale. 430 ZABCZYK Proof. Let s st and A £ . Since a J X dP è a f X.dP and A s A j3/YsdP ê |3/YtdP, therefore / (ffXs + /3YS )dP ê ■ f (aXt + f3Y, )dP. A A A A Obviously, J (X, AYJdP = J Xj dP + J Ys dP A AO{Xs< Yg[ An{Xsî ï J and An{Xs < Ys}£jr, AD{XsêYs}eJ^. By virtue of the definition of a supermartingale, we obtain J (Xs - X t)dP ê 0 , J (Y, - Yt )dP g 0 AO{Xs< Ys} An{Xss YJ and, consequently, X s A Ys dP ê J Xt A Y , dP + J Xt A Yt dP = J X t A Yt dP. A АП { Xs< Ys} An{Xs£Ys} A If (X t) is a martingale, then (Xt V 0) and ((-X t) V 0) are submartingales therefore |xt | = X, V 0 + (~Xt) V 0 is a submartingale, too. 2.2. Rademacher and Haar systems and martingales In this section, we introduce two martingales which are important from the point of view of functional analysis. The first martingale is connected with the so-called Rademacher functions. To define these functions, we use the same notations as in Example 1 (Section 1.3). Thus, Í2 = [0,1), âë [ 0, 1) and P is the Lebesgue's measure on [0, 1). Define as the ст-field generated by the intervals Tn - " i s i i J l к = 1, . . . , 2 , and n = 1, 2, . . . . Clearly, & С ^ , ik - 1 2 n ’ 2 "/ ’ ’ ’ ’ ’ J ’ n n + Rademacher functions rn are defined by the formula rn = 1 - 2Xn or, explicitly, 1 if m £I“, к odd rn (u) = -1 if u elf, к к even Proposition 1. For an arbitrary sequence of real numbers аг, a2, . . . the sequence (or^ + . . . + о^гп) is a martingale with respect to (^ n). Proof. It is sufficient to note that r . dP = 0 , к = 1, 2 , . . . , 2n or to ,nI n *k apply Example 1 (Section 2.1). Let now (Q, P) be an arbitrary probability space and let ( ^ ) n = o i be an increasing sequence of ct-fields contained in ^"such that every is n generated by exactly n + 1 disjoint sets A*!, . . . , An, U Af = Г2,Р(А") > 0 0 n k=Q к к IAEA-SMR-17/29 431 к = О, 1, . .., n. (То construct the n + 1 -st partition, we choose a set, say Ak , from the n-th partition, divide it into two parts of positive probabilities, n and the remaining sets AJJ» к / kn, are left unchanged). A generalized Haar system is a sequence of random variables (Hn)n=0 j adapted to the (^ ,)n = 0 x such that 1 ) H 0= 1 2) E |h I > 0 , and H = 0 in the complement of the set A" ; ' 1 n + 1 1 n + 1 r kn 3) E(Hn) = 0, E(H ®)=1. Proposition 2. For an arbitrary sequence of real numbers aQ, crlt a2> • ■ • > the sequence (a'0H0+. . . +anHn) is an -martingale. Proof. Let A £ ^ n. If ADA? = 0, then 2) implies E(IAHn+1) = 0. If АЛА$п fi f then A = BUA°n, В е Д and BDA° = j and, therefore, E(IAHn+1) = E(I n H ,) = E(H ,) = 0 because of 3). This proves the proposition. AkП 2 .3 . Martingales and densities The results of this section will be needed in Section 2.10 which is devoted to the Radon-Nikodym theorem. Let /л and P be two probability (or finite) measures on (Cl, Si). We recall that the measure /л is said to be absolutely continuous on Si with respect to P if for every A G Sisuch that P(A) = 0, ju(A) = 0 holds. Note that the definition not only depends on P and ju, but also on the cr-field Si. A finite partition of £2 is a sequence Ala Ag, ..., A^ of disjoint subsets of £2 such that n U A¡ = Cl. i = l Example 1. Let Si = crfA-j, . . ., An), where (Aj, .. ., A,,) is a finite partition of £2, and let /u be absolutely continuous with respect to P (on Si), then M(A) = J g(u)dP(u>). for A £ ^ (1 ) A where The proof is obvious. Let us remark that the function g is Si-measurable, and that (1) implies absolute continuity of /u with respect to P. More generally, if ц and P are arbitrary probability measures on (Cl, Si) then a Si-measurable function g such that (1 ) holds is called a density of ц with respect to P (on Si). If g is a density then a Si-measurable function h is a density also if and only if P(u : g(u) f h(u)) = 0. 432 ZABCZYK Let ^ b e the collection of all finite partitions of Í2. If t = (Aj, . . ., A„t we define ^ = cr(Aj, . . ., A„t). We write t s s if and only if ^ Ç ^ that means if the s-th partition is a refinement of the s-th partition. Proposition. Let ti s t 2 s ... be an increasing sequence of partitions of Q . Let (Í2, P) be a probability space and <^tnc n = 1,2, . . . . If a measure ц is absolutely continuous with respect to P on and gn is a corresponding ¿^-measurable density, n = 1 , 2 , . . . , then the sequence (gn) is ( 5 rtn)-martingale. Proof. Let A 6 & then A £ ^ . Therefore, ц(А) = f g dP and n n+l A m(A) = f gn+1dP. Thus / gndP = f gn+1dP. This proves the result. A A A Example 2. Let us consider the probability space ([0,1),^ [0,1), P) where P is the Lebesgue's measure and a Borel, integrable function g. If M is a measure with density g and ^ = cr(l", . . ., Inn), l£ = ^ 2 L 2 ; к = 1 , . . . , 2 n then the density gn is a picewise constant (on l£) function given by the formula gn(oj) = 2nJ g(s)ds for u 6 l", к = 1 , . . ., 2n Í 2 .4 . Martingales and Markov chains Let (Г2, ÎP~, P) be a probability space and (E1( S\), (E2, S%) some measurable spaces. A sequence X0, Xj, ... of random variables from Í2 into Ej is said to be a Markov chain if there exists a sequence of independent random variables Ço,|j,Ç2, ■ • • from Г2 into E 2 and measurable mappings Fn from Ej X E 2 -* Ej such that X = F (X , Ç ), n = 0,1, ... (1) n + l n 4 n’ ’ and f0, fj, Ç2, ... are independent of X0. If the mappings Fn and the distributions of Çn do not depend on n, then the Markov chain (Xn) is homogeneous (in time). Example 1. Iffo ,€ i,... are real-valued r.v.'s then the sequence (Xn), where X n + 1= X 0 + | 0 + . . . + fn = Xn + fn, is a Markov chain, so called random walk. Problem 1. Let us define the transition function (IPn) of the Markov chain (Xn) by the formula IPn (x, A) = P(Fn(x,?n)ÉA) = E(IA(Fn (x, fn ))) then (for n = 1 , 2 ,...): If A 6 #j, IPn(*,A)is (^-measurable (2 ) If x£Ej, IPn (x, • ) is a probability measure on Sj (3) Hint: Apply Lemma 4 (Section 1.1) and Lemma 2 (Section 1.3). IAEA-SMR-П /29 433 Usually, Markov chains are defined starting from a transition function (B?n), (see, e.g. Ref.[ 6 , V ol.l]) but models of stochastic control theory are described often by Eqs (1) (see Refs [7, 8 ]). From the mathematical point of view, these two approaches are equivalent. In this direction, we propose to solve the following problem: Problem 2. Let Ex be 1) a countable (or finite) set and é[the family of all subsets of E 2 or 2) E j = IR1and S\ = ^(IR 1) and let a function IP satisfy (2) and (3). Then there exists a measurable function F from Ex X [0,1) into ТГ* o i i n V i +Vi q + -fz-.v» а п т т т » а л Н r v m t t q r»i a K l a C n ni invm br Hi cf ributoH on Í П 1 \ F(x, t) = Fj(t), as in Section 1.6, where Fx(t) = IP(x, ( - °°, t ] ). Let us now consider a homogeneous Markov chain (Xn) with the transition function IP. A real-valued (^-measurable function h is said to be harmonic (sub-, superharmonic) for (Xn) if 1) IFh = h, (respectively, IFh ë h, IFh ë h); 2) For every initial point X 0 = xeE x, E | h(Xn) |<+°°, n = 0,1, 2, . . . . Here the operator IP is defined as IFh(x) = / h(y)P(x, dy), x£Ej The theorem below has many applications to stability theory. Theorem. If h is a harmonic (sub-, superharmonic) function for (Xn), then the sequence (h(Xj,)) is a martingale (sub-, supermartingale) with respect to ^ = g(X0, . . . ,X n) for any initial point X 0 = x. Proof. Let A € ct(X0, . . . , Xn) then (see Example 1, Section 1.1) A = {u : (X(}(u), . . . , Xn(u))€a} for some a £ ^ X S\ X . . . X S\. Therefore, independent of and, consequently, by virtue of Lemma 2, (Section 1.3), E(IAh(X n+1)) = E(Ia(X 0...... Xn) IRi (Xn)) = E(IAIHi(Xn)). Since IFh = h (IFh ê h orlP h s h) we obtain E(IAh(Xn+1)) = E(IAh(Xn)), (E(IAh(Xn+i)) ê E(IAh(Xn)), or, respectively â). Example 2 (coin tossing). Let E: = {. . . , -1, 0, 1, . . .} E2 = { -1, 1} and let ?o> ?i« • • • are independent r .v .'s such that P(Sn = 1 ) = p, P(?n = "^ ) = 1 "P* 0 < p < 1. The function h: h(x) = x, if p = j is harmonic for random walk Xn = X 0 + f 0 + . . . + fn. Thus, the sequence (Xn) in the form er and the sequence ((l-p )/p )x" in the latter case are martingales. 434 Z.ABCZ.YK For the sake of completeness we propose solution of the following problem (see Chapter 3): Problem 3 (Markov property). Show that for any bounded, á’j-measurable real function h: E(h(Xn+i)| X o,...,X n) = E(h(Xn+1)|Xn) = JPnh(Xn), a .s . Hint: E(h(Xn+1) IX0, .. ., Xn) = E(h(Fn(X„, f n) |X0, . . ., X*). But |n is independent of ^ = ct(X0, . . ., XJ; therefore, using Lemma 1, Section 3.2, E(h(Xn+1)| J^) = IPnh(Xn) a. s. 2.5. Stopping times Let (Q ,^ ) be a measurable space and (^ )teTan increasing family of ст-fields, tGT. A function S : £2 -» T is said to be a stopping time (relative to the if for every tGT the set {u : S(u) s t} belongs to ^ , i.e. the condition S s t is a condition involving only what has happened up to and including time t. Let S be a stopping time. By we denote the collection of events A 6 J such that An{S s t}G^"t for all teT. It is easy to verify that is a ct-field, so called a-field of events prior to S. Proposition 1 a). If T is a finite or a countable subset of IR then S: 57 -» T is a stopping time if and only if {u : S(u) = t}G ^ for tGT. b) If Si and S2 are stopping times then SiAS2 and S!VS2 are again stopping times, c) Any stopping time S is ^-measurable, d) If S1 s S2, then Proof. The properties a), b), c) follow directly from the definitions. To prove d) assume that A G , then ÍS2 s t} П A = {S 2 s t}D{Si s t} П A because {S2 s t}D{Si s t} = {S2 s t}. Since {Si § t}r iA G ^ and {S2 ^ t i e ^ j therefore {S2 s t}n {Si s t } n A G ^ t, Example 1. Let (Xn) be a sequence of real-valued r.v. 's adapted to ( ^ n), (that means Xn are ^ -m ea su ra b le) and let ! Then the least n such that Xn i a + 00 if Xn < a for all n = 1 , 2 ,.,. is a stopping time. To see this fix a natural number к then {S = k} = {X i< a ...... Х к_! < а, X k 6 a }G a (X i,------Xk)C S ^ . The following special case of a theorem proved by Courrege and Priouret [ 9] is helpful in considering concrete examples. Proposition 2. Let Xj.Xj, ... be random variables with values in a countable set EC IR1. Define ^ rn = a(Xi, . . . ,X n), = a(X1,X 2, , . . A mapping S : £2 - {1 ,2 ,...,+ »} is a stopping time relative to (3^n) if and only if 1) S is ^.-measurable 2) If S(u) = n and X k(u) = X k(w') for к = 1, 2, .. ., n then S(u') = n.. IAEA-SMR-17/29 435 Proof. We prove only that 1) and 2) imply: S is a stopping time. (The opposite case is obvious). Let us define A ={u; S(u) = n} and a = {(x1( . . ., x^GE"; x¡ = X;(u) for i = 1, ..., n and some u€A}, The set a is at most a countable set, therefore it is a Borel one. The property 2) implies A = {u : (X-^u), . . ., X n(u)) Ga} and, therefore, АЁа(Хр . . ., Хд) = ^ . 2.6. More on stopping times. Marriage problem. In this section, we try to consider stopping times in a more intuitive way. Let П be the set of all sequences u> = (e±, e2, .. ., eN) where en = -1 or 1, n = 1, . .., N. Any sequence u £ Í2 can be interpreted as the record of N successive tosses of a coin: -1 stands for heads and 1 for tails. Let us introduce, for any n = 1, . . ., N, a random variable Yn and a ст-field ^ by the formulas Yn(e1( . . ., eN) = en, ^ = (Yb . . ., Yn). Thus Yn represents the outcome of coin tossing at the moment n and ^ the class of all past events up and including time n ( ^ is generated by the sets {Yj = el3 . . ., Yn = en} ). Any stopping time S relative to can be interpreted as a (non- anticipating) rule for stopping gambling: stop at the moment n if the event S = n has just occurred. Since the information available to the gambler at the moment n is contained in the sequence of outcomes Y2, Y2, . . ., Yn, we require {S = n} G3&. Conversely any rule for stopping gambling based on the past history only (non-anticipating rule) defines a stopping time. For instance, if we have the following rule: stop at moment 1 if Y j = - 1 , continue if Yj = 1; stop at moment 2 if Yj = 1 and Y 2 = -1, continue if Y2 ■= 1; stop at moment 3 if Y* = 1 and Y2 = 1 and Y3 = -1 o r Y 3 = 1. Then the corresponding stopping time has to be defined as follows: S(w) = 1, for all и such that Yx (to) = -1; S(u) = 2 for all и such that Yj (u) = 1 and Y 2 (cj) = -1; S(u) = 3 otherwise. To define a rule that anticipates the future (or, equivalently, a "stopping time" S which does not satisfy {S = n} € ^ ) we introduce a sequence Xj, X2, . . ., XN, which describes the fortune of the gambler at moments 1, 2, . . ., N i. e. let Xn = 1 + Yj + . .. + Yn. The following rule: stop at the first moment n such that X n = max (X1( . . ., XN) anticipates the future. If for instance N = 3 then {S = 1} = {Х г ё X 2 and Xi â Хз}= {Y 2 = -1} To stop the game at moment 1 , the player should know the outcome at moment 2 . Problem 1. (The space Г2 and the sequence {¡Pn) as above). Let r^j and dN denote respectively the number of all stopping times S s N and the number of all random variables T with values in {1, 2, . . ., N} and ^ ”N- measurables. Show that rx = 1, rN + 1 = (1 + r N)2, dN= (N)sN, N = 1, . . . and 0N that (rN/dN) á (1/2) (2/N) . Hint: Use Proposition 2, Section 2.5 and the 2 N -1 estimate r„r ё 2 . N We propose also to consider the following "Marriage Problem . It can be formulated as follows: In an urn, there are N balls. Inside, each ball contains a piece of gold which we do not see and for different balls the amounts of gold are different. We may draw (at random) one and keep it and, of course, we want to draw the one with more gold inside. But we are not allowed to check all balls for the amount of gold they contain. Instead, we are to follow the following rule. We are allowed to draw and check one ball a time and decide either to keep it or to have another drawing. However, 436 Z.ABCZYK all balls which have been drawn, checked and rejected, are not available any longer. Keeping the к-th ball we choose, the probability of maximal gain is 1/ N for к = 1, 2, . .., N. Here, we do not use the information available from previous choices. However, it may be shown that, using the information available, one can find a strategy which increases the probability of the best choice to pN where pN -*■ 1/e as N -» °o. This problem one faces in life if one wishes to get married and believes that one will be given N possibilities of ch oice. The above problem can be formalized as an optimal stopping-time problem. All details are given in the book [ 10]. 2.7. Doob's optional sampling theorem Theorem. Let (X n) n=1 k bea supermartingale (a martingale) relative to (3^)n = l,...,k. Let S1( S2, ------Sm be an increasing sequence of (^ti) stopping times. The sequence (X s )i = 1 ...... m is then also a supermartingale (a martingale) with respect to и-fields , ¡P'c • S m Proof. Let A e ^ . We shall prove that f (X. -X_ ) dP â 0 in the 1 A bl b2 supermartingale case and J (Xs -X s ) dP = 0 in the martingale case. If for A 1 2 every u, S2 (to) -Sj^io) § 1 then к / ( W dP- I / lx- ’x',dP A i = l A n {Sj =r} n {S2 > r} but An{Si = r} and {Эг> r}, belong to therefore An{Sx = r}n{S2> r} The definition of a supermartingale (a martingale) implies that the desired inequality (equality) follows. Let r = 0, 1, . . .,k and define stopping times Rr = S2 A(S 1 + r ). Then S i= R o S R 1 S ... s Ht = S2 and R ¡ +i - R¡ s 1, i= 0 ,...,k -l. By virtue of the first part of the proof J X Sid P ê J X Rj dP È . .. й J X RkdP =J X s.dP A A A A In the martingale case, the above inequalities should be replaced by the equalities and, thus, the theorem is proved. Let us consider some simple applications of the above Doob's optional sampling theorem (more serious applications will be given in Sections 2.8 and 2 .9 ). Example 1. (Ruin problem). Let (XJ be the random walk (defined in Section 2.4) which starts at the origine (X0= 0). Thus Xn + 1= Y0 + . . . + Yn where Y0, Yx, are independent random variables P(Yn = -1) = 1 - p, P(Yn = 1) = p, n = 0, 1, 2, .. . . Let a and b be natural numbers and let S = inf {n :X n = -a or Xn = b}. In gambling language: a gambler with initial capital a plays against an adversary with initial capital b; in each game, he can win or lose a dollar with probability p and q = 1 -p, respectively, and the games are independent; X nis the gambler's total winnings after the n-th game and the game ends exactly at S, when he or his adversary lost all his IAEA-SMR-H/29 437 money. The ruin problem consists in finding the probability that the gambler will be ruined, i.e. the probability that Xs = -a. Problem 1. Show that P(S <+<»)= 1. Hint: See Remark 3, Section 2. 9. Assume that p = 1/2 (fair game) and define stopping times Sn = SAn, n = 1, 2, . .. . Since (Xn) is a martingale, by Doob's theorem (applied to stopping times: 0 and Sn) E(Xsn) = E(X0) = 0 . But S < + (a. s.) and, there fore, the sequence Xg^ is convergent: Xs - -a on {Xs = -a} X„ - b on {X. = b} Lebesgue's theorem (Section 1.2) implies 0 = E(XS ) - E(lim X s ) = -aP(Xs = -a) + b(l - P(XS = -a)) n n n Consequently, P(XS = -a) = b/(a + b), P(XS = b) = a/(a+b) Problem 2. Consider the case p f 1/2. Hint: Sequence (q/p) n is a martingale (Section 2.4). Answer P(XS = -a) = (1 - (q/p)b)/(p/q)a - (q/p)b Example 2 (Strategy for a schoolboy). Let us consider the martingale introduced in Example 1, Section 2.1. Again by Doob's theorem, we obtain that, for any stopping time S (relative to (^ ,)), E(XS) = E(X0) = b/(b+w ) The above identity can be interpreted as follows: During an examination, every schoolboy draws at random (without replacement) one question he has to answer. At the beginning of the examination, there are b + w questions, and some pupil is able to answer in a satisfactory way exactly w of them. At any moment, he knows which questions have been drawn and he wants to find an optimal moment for drawing question to minimize the probability of choosing a "bad" question. But for any rule S, E(XS) = b/(b+w); there fore, all possible "strategies" are of the same value for him. 2 . 8 . Fundamental inequalities Theorem 1. Let (Xn)n_.[ jb e a supermartingale and с a non-negative constant. Then we have 1 ) cP(sup Xn ê c) S E(Xx) - J X kdP {sup x n < c} 2) g EfXJ + E(Xjj 438 Z.ABCZ.YK 3) cP(inf Xn § -c) g - J X kdP {inf xns -c} 4) â E (3Q Proof. To prove the inequalities 1), 2), define S(u) = inf {n:X n(u) è с}, or S(u) = к if sup Xn(u) < c. S is a stopping time and S i 1, Thus, by Doob's optional sampling theorem, we obtain E (X j) ё E (X S) = J X sdP + J' XsdP ê cP (sup X n ê c) + J' X kdP { sup x n > c } { sup Xn < c} {sup X n < c} or equivalently cP(supXnÈc)sE(X1) - J X ^P {sup x n < c} This is exactly inequality 1). Since -Xk s Xk, inequality 2) follows, too. To establish the relations 3) and 4), we introduce an analogous stopping time S, S = inf {n:X n s -c} or S = к if inf Xn > - c. Since S § k, therefore E (X k) § E (X g) = J X sdP + J X sdP {inf Xn < -c} {infXn>-c} s -cP(inf Xn s -c) + J X kdP {inf x n > -c} These inequalities imply 3) and 4). Problem 1. (Doob-Kolmogorov 1 s inequality). Let (Xn)n = 1 k be a martingale then for every с > 0 , , Е(М P(sup IXn j ê c) s ------n Hint: If (Xn) is a martingale then (|xn|) is a submartingale (Lemma 1, Section 2.1). Use 4), Theorem l . To formulate and prove the next theorem, we have to introduce some new definitions and notations. Let x = (xj, . . ., x„) be a sequence of real numbers and let a < b. Let Sj be the first of the numbers 1 , 2 , . .., n such that XSl s b or n if there exists no such number. Let S^be, for every even (respectively odd) integer к > 1 the first of the numbers 1, 2, . .., n such that Sk > S ^ and xs^ s a (respectively x s^ ë b). If no such number exists, we set sR=n. In this way, the sequence Si,S 2 ,S 3 , ... is defined. The number ££>n{n-, (a,b)) of down- crossings by the sequence x of the interval (a,b) is defined as the greatest integer к such that one actually has x„ ê b and x s a. If not such 2 k-l 2 k integer exists, we put ^ n(x; (a,b)) = 0. Let us remark that the "intervals": IAEA-SMR-И /29 439 [ Sj, S2] , . .., [S 2 k-i>S2kl represent the periods of time when the sequence x is descending from b to a. Analogously the number (a,b)) of upcrossings by the sequence x of the interval (a,b) can be defined. In fact ^„(x; (a,b)) = &n(-x, (-b,-a)). Numbers S^ Sg, .. . which correspond to the sequence -x and the interval (-b, -a) will be denoted as Ri, R2, ... . Theorem 2 (Doob's inequalities). Let (X m) m = 1 ...... n be a supermartingale relative to ... n and let a < b be two real numbers. Then the following inequalities hold: 1) E(Æ>n(a ,b )) S ~(ЩХгЛЪ) - E(X„Ab)) 2) E(^n(a,b)) S E((a -Xn)+) Herei2>n(a,b) ^ „((X j ...... Xn); (a, b)), «^(a.b) = № ...... X J ; (a ,b )), (¿£>n(a,b) and Proof. 1) Since (XmAb)m = 1...... n is a supermartingale and the numbers of downcrossings corresponding to (Xm )m = 1...... n and (X mAb) m = 1 ...... n are the same we can assume that Xm = XmAb, m = 1, .. ., n. Let Si, ..., S^, 21 > n be moments defined above and corresponding to the sequence (Хг, .. . , Xn). Then E » Chj >} =m ,Q is a density and a probability law on ]Rn. Hint: Use the formula