SIAM J. CONTROL Vol. 13, No. 5, August 1975
ON THE SOLUTIONS OF A STOCHASTIC CONTROL SYSTEM. II*
TYRONE DUNCAN'}" AND PRAVIN VARAIYA:
Abstract. This paper presents generalizations of the work in [1], [2] to include controlled stochastic processes which take values in a certain class of Fr6chet spaces. The crucial result is an extension of Girsanov's technique for defining solutions of stochastic differential equations by an absolutely continuous transformation of measures. The result is used to prove existence results for stochastic control problems and for a class of two-person zero sum games. 1. Introduction and summary. Consider a controlled stochastic process represented by the stochastic differential equation dX(t) f(t, X, u(t, X)) dt + dB(t), [0, 1], where B(t) is a Fr6chet-valued Brownian motion, X(t) is the state process. The drift f, and the control u, depend at any time on the past of the state process, {X(s), s __< t}. For the case where the Fr6chet space is Rn, a satisfactory theory dealing with the problem of existence of solutions of the differential equation and existence of optimal control laws is now available [1], [2]. A crucial building block in this theory consists in defining a solution of the differential equation via an absolutely continuous transformation of measures. Each control thereby de- fines a solution characterized by its (unique) probability law which is absolutely continuous with respect to Wiener measure. Thus the influence of a control law upon the system is captured in the Radon-Nikodym derivative of the resulting probability law with respect to Wiener measure. Questions dealing with the existence of an optimal control can then be converted into questions about the compactness (in an appropriate sense) of the set of Radon-Nikodym derivatives. The measure transformation technique mentioned above is due originally to Girsanov 16]. This paper deals with these same questions for the case where the state space is infinite-dimensional. The problem of characterizing Brownian motion in infinite-dimensional spaces is a difficult one and has been resolved for certain Fr6chet spaces only. This is described in the next section, where some additional properties of such Brownian motion as sample continuity and stochastic inte- gration are established also.. In 3 the result of Girsanov is extended to cover the differential equation under consideration. Once this has been achieved the tech- niques of [1], [2] apply without change and the existence of optimal controls and saddle points follows easily. This is sketched in 4. 2. Preliminaries for defining Brownian motion. To define probability measures on Fr6chet spaces the results of Dudley-Feldman-LeCam [3] are used. The
Received by the editors September 25, 1973, and in revised form August 29, 1974. f Department of Applied Mathematics and Statistics, State University of New York at Stony Brook, Stony Brook, New York 11790. This research was supported in part by the National Science Foundation under Grants GK-24151 and GK-32136. :I: Department of Electrical Engineering and Computer Science and Electronics Research Lab- oratory, University of California, Berkeley, California 94720. This research was supported in part by Downloaded 09/10/14 to 129.237.46.100. Redistribution subject SIAM license or copyright; see http://www.siam.org/journals/ojsa.php the National Science Foundation under Grant GK-10656X3. 1077 1078 TYRONE DUNCAN AND PRAVIN VARAIYA
latter are generalizations of the work of Gross [4]. While the generality of the immediately following discussion is not used subsequently, it does serve to indicate some directions along which the results reported here can be further pursued. Some notation and definitions are introduced first. For a locally convex Huasdorff topological (real) vector space X let X* denote its topological dual. A dual system or duality over the reals consists of two vector spaces X, Y and a bilinear form (.,. X Y--* R that separates points for both X and Y. For a pair X, Y in duality, let FD(X) denote the collection of all finite-dimensional subspaces of X. For G c X, let///(Y, G) be the a-algebra of Y generated by the sets (y: (x, y) B}, where x G, B is a Borel set in R. Thus /J/(Y, G) is the smallest r-algebra on Y for which each x G, regarded as a function on Y, is measurable. Finally let (g(Y, X) U (/J/(Y, G)IG FD(x). A cylinder set measure on Y is any nonnegative, finitely additive set function m on (g(Y, X) with m(Y) such that m is countably additive on '(Y, G) for each G FD(X). The following notion of a measurable seminorm given in [3] is crucial to the analysis. DEFINITION 1. Given a duality X, Y, (.,. , and a cylinder set measure m on Y, a seminorm I" on Y is said to be m-measurable if for each e > 0 there is a G FD(Y) such that if F FD(X) and F d_ G (i.e., (x, y 0 for x e F, y G), then m{y:ly- F+/-I _-< e} _-> 1-e or equivalently m*{y:iy- Gl
This means that if C e (H, is of the form C P- (E), where P is the orthogonal projection of H onto a subspace L FD(H) and E is a Borel subset of L, then pt(C) (2t)-"/Z feexp (- lxl dx, norm on its inner product. Downloaded 09/10/14 to 129.237.46.100. Redistribution subject SIAM license or copyright; see http://www.siam.org/journals/ojsa.php where n is the dimension of L, and ]. lu denotes the H induced by STOCHASTIC CONTROL SYSTEM 1079
H, and define the maps i'H F, j'F* H*, as the canonical injections. Finally let N(F) denote the Borel sets of F. Throughout, H and F denote the spaces introduced here. The collection #t, e R/, will be used now to define a Brownian motion with values in F. For eacl e R/ let F be a copy of F. For each finite collection t < < t,, let tl,'",trt be the measure on (]-[7= Ft" HT=l (Ft')) defined by fAn d#tl'""t" d#,._t._,(x,,) dl.tt=_t,(x2) dYt,(Xl) for Ai (F"), 1, ..., n. Since sets of the form l--I'= Ai generate the product r-algebra ]-I'-1 (F") the measure #,1,...,,, is defined. To show that the family of measures #,1,...,,, is a projective system, it is necessary to verify consistency. Since F is separable in the topology determined by the countable seminorms I. j, the measure #,1,...,,, is determined by sets of the form Ai (I P)F @ Fi, where P is an orthogonal projection with finite dimensional range and F is a Borel subset of PF. Let A1, "'", A, be such sets, and suppose Aj F. Then d'''''"= dl'tt"-t"-'(xn)'"d#t*(x1) fA fA fA fA 2--Xl y.- 1Xi @,._,._ ,(x,)
dp,._,._
..dp,,+ =-t,+ dpt,+ ,-t,_,(x)
[l,',[j l[j+ 1,",n j-1 j+l In the above p is the measure of a Brownian motion with values in the finite- dimensional space PF and so the third equality above follows from the Markov property of such a process. Since each of the measures #,, R/, is a regular Borel measure, the projective system of measures admits a projective limit [6, p. 49]. The projective limit thus obtained is denoted (f2, ff, P). Evidently, (f2, )= 1-[,e+ (U, (F')). It will be assumed that (f2, if, P) is complete. For each let X denote the F-valued evaluation map on f2 at t. Let o be the smallest completed a-algebra with respect to which X is measurable for s =< t. Then (Xt, t, P)te is a Brownian motion as defined below. DEFINITION 2. A stochastic process (Xt, ,, P)tzR+ (or simply (X,, P)tzR or (X,) if there is no ambiguity) is said to be a Brownian motion with values in the Frdchet space F induced from a family of canonical normal distributions on a Hilbert space H, if for each e F* (the topological dual of F) the real-valued process ((1, X,), ,P)tzR+ is a real-valued Brownian motion with E
For Fr6chet-valued Brownian motion there is a modification which has continuous sample paths as shown by the following lemma. motion. LEMMA 1. Let (Xt, , P)tR +be a Frchet-valued Brownian There is a modification of Xt with continuous sample paths. Proof. Since a countable number of seminorms determine the topology of a Fr6chet space it suffices to verify the continuity of the sample function of Brownian motion with respect to each of these countable number of seminorms and there- fore it is enough to prove the lemma for a Banach-valued Brownian motion. Fernique [7] has shown that for a Gaussian random variable X on a topo- logical vector space with a measurable seminorm I. there is an > 0 such that (1) E exp lSl 2 < Combining (1) with a result of Nelson [8, Thm. 2] as used by Gross [9, p. 134] it follows that the Banach valued Brownian motion has a modification with con- tinuous sample paths. Since stochastic integrals will be used subsequently, a family of processes has to be described that can serve as integrands. The following definition gives such a family. DEFINITION 3. Let / be a separable Hilbert space. An/-valued stochastic process ()g on (f, , P) that is adapted to ()tR is said to be predictable if the map (og)'R+ f-/ is measurable with respect to the a-algebra on R+ f generated by the left-continuous/-valued processes adapted to ()R Real-valued stochastic integrals will now be defined from F-valued Brownian motion and predictable H-valued processes. LEMMA 2. Let (d/)tR be a predictable H-valued process with
(2) E I,lr dt < ,
and let (Bt,tt, P)teR be an F-valued Brownian motion. Then the real-valued process Yt, , P)tR+
(3) Y (Os, dBs), is a square integrable martingale which has a modification with continuous sample paths. (The integral (3) is defined in the course of the proof.) Proof. Let e, > 0 be a sequence decreasing to zero and for each n let Pc. be a projection on H with finite-dimensional range P.H c jF* such that
Let 11, "", lu be an orthonormal basis for P.H. The process (P,) defined by
(4) Downloaded 09/10/14 to 129.237.46.100. Redistribution subject SIAM license or copyright; see http://www.siam.org/journals/ojsa.php STOCHASTIC CONTROL SYSTEM 1081
is a (continuous) martingale since by the definition of the process (B), ((/i, B)) is a real-valued Brownian motion. By Doob's inequality [10, p. 353] the sequence of continuous martingales (Y(P,)) converges uniformly on compact subsets of R+ because EIY,(P,) Y(P)I 2 E I(P, P,,)&I ds
which vanishes as m, n increase to infinity. The integral (3) is defined as the limit of the martingales Y(P.). Clearly the limit does not depend on the particular choice of the projections P. and Y is evidently square integrable. U COROLLARY 1. Let ()R be a predictable H-valued process with
(5) 112 ds < o a.s. for all t. Then the real-valued process Z defined by (6) Z, (,dB)
is a locally square integrable martingale which has a modification with continuous sample paths. 2 The following representation of square integrable functionals on the F-valued Brownjan motion probability space will be useful subsequently. For R"-valued Brownian motion, K. It6 [11] has obtained this representation by describing results of Wiener [12] and Cameron-Martin [13] in terms of stochastic integrals. PROPOSITION 1. Let (g), ,, P) be the probability space for an F-valued Brownian motion (B, ')tR /. Let f be a real-valued square integrable functional on (f, if, P). Then f can be represented as (7) f= c + (,dB),
where c Ef and (lt)teR is a predictable H-valued process with e Itl2ndt < . Proof. Since H is separable so is F, hence by the Hahn-Banach theorem, there is a countable family F c F* that separates points of F. Consider the random variables (7,j' dBs), where 7 e F, e R+ and let be the algebra of real-valued random variables formed from these and the constant random variables. Since the random variables (7, j' dBs) are jointly Gaussian, it follows that c L2(p) and furthermore L2(p)
mt is said to be a locally square integrable martingale if there exists a sequence of stopping
Downloaded 09/10/14 to 129.237.46.100. Redistribution subject SIAM license or copyright; see http://www.siam.org/journals/ojsa.php times T. T a.s. such that mt r. is a square integrable martingale for each n. 1082 TYRONE DUNCAN AND PRAVIN VARAIYA
for ai R and f . Since F is separable, the family of random variables (7,odB), 7 F, R+, generates the a-algebra -. By a result of Segal [14, Lemma 2.1] it follows that is dense in L2(p). Let f L2(p). There exists then a sequence g, in such that Elf g,I 2 O. By properties of finite-dimensional Brownian motion, g, has a representation of the form (7) (see [15]), i.e.,
(8) g. c. + <', dB>,
where c, Eg, and (') is a predictable process with values in some finite- dimensional subspace L of H and with L c iF*. Since LZ(P)-convergence implies Ll(P)-convergence it follows that c, converges to c. Furthermore the stochastic integrals in (8) must be Cauchy; hence ElO' O'12n dt 0 as m, n . Since the sequence of processes (') are predictable, there must exist a predictable process (fit) such that
and evidently (7) is satisfied. 3. Transformation of measures. Theorem below describes how Fr6chet- valued Brownian motion is transformed by changing the probability measure by an absolutely continuous substitution of the measure. This result was first estab- lished by Girsanov [16] for the case of R"-valued Brownian motion. The result has been presented in [17] and a related result is given in [18]. THEOREM-1. Let (Bt,,P)tO,ll be an F-valued Brownian motion and let (tt)tto, be a predictable H-valued process such that (9) | IGlat (10) M exp (d/z, dB> -} IGI2 ds Then (11) E(M) 1. Suppose tkat (12) (M,) ; then the process (, , P)to, is an F-valued Brownian motion where Bt Bt iO ds, Downloaded 09/10/14 to 129.237.46.100. Redistribution subject SIAM license or copyright; see http://www.siam.org/journals/ojsa.php STOCHASTIC CONTROL SYSTEM 1083 and the probability measure is given by dp M1. Proof. Define the increasing sequence of stopping times T, by inf{tlM > n}, T, if the set above is empty. Because of (9) M has (a modification with) continuous sample paths, so T, ]" a.s. P, and in particular, (13) lim Mt^r. Mt a.s.P. By the main result in [19], the processes (M,) and (M, ^r.) satisfy (14) M, + Ms( /s, dBs) (15) M, ^T. + M(0s, dB). Since (M, ^T.) is bounded, (15) implies that it is a martingale and hence E(M,^T. E(MT. 1. An application of Fatou's lemma to this result and (13) yields (11). From now on suppose that (12) holds, so that from (14) it follows that (M,, , P)o,x is in fact a martingale. Let H be fixed and consider the process (N,, o, P),t0,l, where N, (1, ,) ( l, B,) (1, Os) ds. The theorem will be proved once it is shown that (16) (N,, t, P) is a martingale, (17) /(N N[)= [/[(t- s), where/ denotes expectation with respect to the measure P. Now to prove (16) it is sufficient to show instead that (18) (M,N,, , P) is a martingale. Because suppose that (18) is true. Then using the fact that (M,, P) is a martingale, E(M,N,J) (N,[) (by [20, p. 345]) E(M,I) (by (18)) Downloaded 09/10/14 to 129.237.46.100. Redistribution subject SIAM license or copyright; see http://www.siam.org/journals/ojsa.php 1084 TYRONE DUNCAN AND PRAVIN VARAIYA which is equivalent to (16). Now MtN Mr(l, Bt) M (1, d/) ds. Applying the differentiation formula for continuous martingales 15] gives MtNt MoNo Ms(l, dBs) Ms(l, Ps) ds + N dM + Ms(l, /s) ds Msl, dBs) + gsdMs, which clearly implies (18). It remains to prove (17). To this end note that (19) /(Nt2 N2I)-- and apply the differentiation formula to MtNt to obtain MtN2t MoN N2 dM + 2 MsNs(l, dBs) 2 MsNs(l, s) ds + 2 NsMs(1, Os) ds + MsIll ds. Substitution of this into (19) gives /(N2 N2])= Ill(t- s), and so (17) is proved, l-I In many applications of the transformation of measures technique of Theorem the crucial difficulty is to verify that/3(f) 1. The next result gives a sufficient condition for/() 1. It is due to V. Beneg. LEMMA 3 (Beneg). Let (Bt, t, P)tto,ll be an F-valued Brownian motion. Let 0Pt)t[o,ll be a predictable H-valued process such that (20) IP(t, B)IH K + K' sup IBI, =< sz[O,t] where I" is a seminorm on F and K, K' are constants. Then there is > and M < o, depending only on K and K', such that (21) E exp a (q, dBs) ]d/si2n ds < M. In particular, E exp Fix ($,) satisfying (20) and let T=inf IOl ds- > n Evidently T T a.s. and let 0' 0 ^r. Let e > 1. Let B,- o[.o iO' ds. Since j' I01,2 dt <= n, it is easy to use the proof of [16, Lemma' 1] to show that Eexp(eO")= Hence by Theorem 1 ,, P)tto, is a Brownian motion, 11 where dP" [exp (e0")] dP. Now IO'ln K + K' sup IBl --< s[O,t] so that IBt] I/'[ + o K + K' sup IB.I ds, u[0,s] sup IBI <_- sup Itl + Kt + K' sup IB,,I ds, ss[0,tl se[0,tl ue[0,sl and hence by Gronwall's inequality, IIBII <- (eK + II/11) exp eK', where Ilxll supto,1 IXl. From (20), [0l 2 _< 2K2 + 2K'IIBI[ 2, so that Iq,71z as 2K 2 + 4K'(o2K z + II/"ll =)exp 2xK'. By the differentiation formula, E exp e(0) E exp (0@) + IGI ds < E exp (0@")exp e-2 + 4K'(02K 2 + 2)exp 2K'] 2 z)[2K2 IIB"I NE" exp exp 2K')II"I] 2-a)(4K' 21 where N is a constant and E" denotes expectation with respect to P". Since (,,/3,) is a Brownian motion by a result of Fernique [7] there exists 7 > 0 such that E" exp IIB"II 2 A < independent of n. Let > be so small that ((2 e)/2)(4K' exp 2eK') < 7. Then Downloaded 09/10/14 to 129.237.46.100. Redistribution subject SIAM license or copyright; see http://www.siam.org/journals/ojsa.php E exp (") < NA, 1086 TYRONE DUNCAN AND PRAVIN VARAIYA and so (21) follows by Fatou's lemma. The final assertion is then immediate l-] because (21)implies that exp [ (ks, dBs 1/2'o I1 ds is a martingale, 4. Preliminaries for optimization. The system to be controlled is represented by the stochastic differential equation (22) dX(t) if(t, X, u(t, S)) dt + dB(t), [0, 1], where B is an F-valued Brownian motion, X e F is the state with X o 0 a.s., and u is the control law taking values in a prespecified compact subset U F called the control set. The functionftakes values in H and :H F is the canonical injection. The first difficulty to be resolved is to define the solution of the differential equation (22) for a large class of control laws. This is achieved in the following manner. One starts with a process (X, , Po)to, which is an F-valued Brownian motion. For a given control law u an F-valued process B' is defined by B' X, if(s, X, u(s, X)) ds. Next the probability measure Po is replaced by another probability measure pu such that the process B is an F-valued Brownian motion. The process (X,, , W),to,1 is then regarded as the solution of (22) corresponding to the control law u. To make this procedure precise the following notations and definitions are useful. DEFINITION 4. (a) is the linear space of all F-valued continuous functions, denoted by z, on [0, 1]. (b) For [0, 1], is the smallest a-algebra of subsets of cg which contain all sets of the form {z e lz(s) A}, where s [0, t] and A is a (topological) Borel subset of F. 6e . Throughout the remainder of this paper f is a fixed probability space with an increasing family of a-algebras , [0, 1]. ff . It will be necessary to consider different probability measures on the space (fL if). If Y, is a family of measurable functions on (f, ) and if P is a probability measure on (fL if), the stochastic process corresponding to P and the family Y will be denoted by (Y, , P)to,. Then the same family Y generates different stochastic processes corresponding to different probability measures. Finally let Po be a distinguished probability measure, and let X be a distinguished family of F-valued measurable functions on (fL ), [0, 1], such that the process (Xt, , Po)tto, is an F- valued Brownian motion with continuous sample paths. Unless mentioned other- wise the process X refers to this process. Also Eo will denote expectation with respect to Po. The measure induced on and will be called the Wiener measure on (, 6). The following conditions are imposed on the function f in (22). fl. f is a map from [0, 1] cg U into H and f is measurable with respect to the product a-algebra (R) (R) u, where (') is the family of Borel subsets of [0, 3 (u). I"2. For [0, I], f(t, .,. is (R) )-measurable. t3. For (t, z) [0, 1] x cg, f(t, z,. ): U H is continuous. Downloaded 09/10/14 to 129.237.46.100. Redistribution subject SIAM license or copyright; see http://www.siam.org/journals/ojsa.php f4. There is an increasing function fo-:R/ R/, and a seminorm I. on F STOCHASTIC CONTROL SYSTEM 1087 such that for (t, z, u) e [0, 1] x cg x U, If(t, z, u)lt fo(llzII), where Ilzll max,to,1] Iz(g)l. Throughout the remainder the symbols l" and I1" will denote the seminorms assumed here. f5. For (t, z) e [0, 1] x cg, f(t, z, U) f(t, z, u)lu U} is a closed and convex subset of H. DEFINITION 5. (a) An admissible control (law) is a map u:[0, 1] x ---, U which is (R) ,9-measurable and, further, is such that for each e [0, 1], u(t,. is ,-measurable. //denotes the set of all admissible controls. (b) The drift corresponding to u e //is the function gu given by gu given by g,(t, z) f (t, z, u(t, z)). {g, lu U}. (c) For g e and positive integer n, g" is the function given by z) if Iz(s)l n for s e [0, t], g"(t, z) { g(t,o otherwise. DEFINITION 6. A function 4)'[0, 1] x c H is causal if it is (R) 5 measurable and if qS(t,. is -measurable for [0, 1]. DEFINITION 7. (I) is the collection of all causal functions such that Ib(t, z)ln- --< fo( ]z 1) for (t, z) [0, 1] cg. , {b 11q(t, z)l/ _-< n for all (t, z)}. The next result which follows immediately from [21, Lemma 1] gives a very useful characterization of . LEMMA 4. A causal function g is in Y/ if and only if g(t, z) f(t, z, U)for all t Z. DEFINITION 8. Let q5 be a causal function such that (23) | Ic/)(t, z)lEn dt< for all z d 0 Then ((t(b), , Po))tto,l denotes the continuous real-valued process defined by ,() ((s, x), dx) I(s, X)l ds. Let ((O) a(). Note that (23)is always satisfied for The problem of the existence of solutions of (22) is resolved in the following result which follows immediately from Theorem 1. THEOREM 2. Let u be such that (24) Eo exp ((g.) , where E o denotes expectation with respect to Po. Define the probability measure P, by P. exp ((gu)Po. Then the process (B,, , P,),to,l defined by B, X, if(s, X, u(s, X)) ds Downloaded 09/10/14 to 129.237.46.100. Redistribution subject SIAM license or copyright; see http://www.siam.org/journals/ojsa.php is an F-valued Brownian motion. 1088 TYRONE DUNCAN AND PRAVIN VARAIYA The next result shows that (24) must be satisfied by every solution of (22). LEMMA 5. Let d? O. Let (Yt, , P)ttO,Xl be any process with continuous sample paths such that the stochastic process (Bt, , P)tto,xl defined by dB dY ice(t, Y)dt is an F-valued Brownian motion. Then exp (dp(s, Y), dBs) Ib(s, Y)12u ds dP 1. Proof. Define the function b. :[0, 1] x cg H by j" (t, z) if Iz(s)l _-< n for s [0, t], z) - 0 otherwise. From the definition of it follows that [b.(t, Y(o))ln __< fo(n) for e [0, 1], By Lemma 3, exp '(-.) dP 1, where ( dp.) ( dp.(s Y) dBs) ]dp.(s, Y)12n ds By Theorem the process (Y.(t), ., P.),[o,, where Y.(t) idp.(s, Y) ds -+ B(t), dP. exp ((-b.)dP, is a Brownian motion with continuous sample paths and hence induces Wiener measure # on fig, 5). Let e > O, and n be so large that P,,(gl,,) P.(II Y,,II < n)= ,u(z 1 Ilzll < n) 1 e. Now it is clear that Y(co)l] n only if Y()ll n and hence Y.(co, t) Y(oo, t), .(, t) b(co, t) for co so that P.(gl.) fn exp(-b.)dP fn exp((-b)dP__> e. Since e > 0 is arbitrary, the result follows. U COROLLARY 2. Let O and (Y, , P),[o,x satisfy the hypothesis of Lemma v measure on Downloaded 09/10/14 to 129.237.46.100. Redistribution subject SIAM license or copyright; see http://www.siam.org/journals/ojsa.php 5. Let be the indaced by the process (Y, , P) the measurable space STOCHASTIC CONTROL SYSTEM 1089 (c, ). Then v is mutually absolutely continuous with respect to # and dl dv(Y(o)) exp ((-$)(o)). Remark. This corollary implies that the solutions of (22) are unique in a weak sense, i.e., all solutions of (22) which have continuous sample paths must induce the same measure on ((, 5). The qualification "weak" is inserted because only the uniqueness of the probability law has been proved. Recall that (X t, t, Po)tto, 11 is an F-valued Brownian motion with continuous sample paths. Also recall Definition 8. DEFINITION 9. For any subset A c tI) define (A) c L l(fl, if, Po) by @(A) {exp (b)lb e A}. The corollary to Lemma 3 implies the next assertion. PROPOSITION 2. (O") is a bounded subset of LE(f), o, Po). LEMMA 6. :((I)") is a closed subset of L2(fl, o, Po). Proof. Let bl, b2, be a sequence from tI)" and let p be such that (25) lim Eol p exp (b.)l / 0, and (26) lim exp (b.)= p a.s. Po- Since Eo exp (b,)= for all n, Eop 1, and so by Proposition 1 there is a functional such that Eo IO(t, x)12 dt < oz and (27) p 1 + (d/(t, X), dX) a.s. Po. 0 Define the martingale Pt by Pt Eo{Plt}, [0, 1]. By taking modifications if necessary it can be assumed that the martingales Pt and t(b,) have continuous sample paths so that, by Doob's inequality [10, p. 353], it follows from (25) that (28) Pt lim exp (t(b.) uniformly on [0, 1] a.s. Po. Next exp ((b,) -4- exp (t(b,)(,(t), dX(t)) so that from (25), (26), lim Eo lexp '(4)4(t) 0(t)l dt 0, and hence by taking subsequences if necessary it can be assumed that Downloaded 09/10/14 to 129.237.46.100. Redistribution subject SIAM license or copyright; see http://www.siam.org/journals/ojsa.php (29) 0(t) lim exp (4,)4,(t) a.s. l(R) Po, 1090 TYRONE DUNCAN AND PRAVIN VARAIYA where denotes Lebesgue measure on [0, 1]. Now p > a.s. Po, because if Po(A) Po{P 0} > 0, then (26), together with the fact that 14,lu _-< n, implies that lim (O,(t),dX(t)) o on A. But then E Id.(t)12 dt N 2 =Eofjl(dp,(t),dX(t)) <= 0 so that to avoid the contradiction one must have p > 0 a.s. Po. It follows that Pt > 0 a.s., and hence combining (28) and (29) gives lim b.(t) a.s. (R) Po. Thus there is a causal map b e q)" such that th(t, X) lim b,(t, X) a.s. (R) Po and evidently p exp ((b). The proofs of the next two results are identical respectively with the proofs of [2, Lemma 4] and [2, Thin. 2] with some obvious notational changes. Hence the proofs are omitted. LEMMA 7. (") is a convex subset of L2(fL , Po). THEORE 3. Let o {g lEo exp ((g) 1}. Then (o:) is a closed, convex subset of L l(fL ._, Po). 5. Applications. The results developed above immediately imply the existence of optimal control laws for a broad class of problems. Consider the control system dX(t) if(t, X, u(t, X)) dt + dB(t), e [0, 1], with X(0) 0 a.s. Suppose that f satisfies the assumptions fl to f5 and in addition the function fo in f4 satisfies assumption f6. f6. There exists K, K' such that fo(n) <= K + K'n for all n e R +. Let L : R be a fixed bounded 5-measurable function. For each u e q/ the cost incurred by u is defined to be (30) J(u) Eo[(exp (g,))L(X)] fn L(X(o))) exp (g,(o))) dPo. THEOREM 4. Under assumptions fl to f6, there exists an optimal control u* Downloaded 09/10/14 to 129.237.46.100. Redistribution subject SIAM license or copyright; see http://www.siam.org/journals/ojsa.php J(u*) <__ J(u), u e STOCHASTIC CONTROL SYSTEM 1091 Proof. From Lemma 3 it follows that Eo exp (g,)= for all u q/ and furthermore there exists > such that sup Eo exp (g,) < . Hence by [6, Chap. 2], () is a uniformly integrable subset of L l(fL , Po). By Theorem 3, @() is convex and strongly closed in L l(f,,Po). So that by [6, Chap. 2] () is weakly compact in L l(f, , Po). Hence the linear functional exp (gu) EoL(X) exp (gu) attains a minimum. In a manner corresponding exactly to the argument developed in [2], the results presented above can be used to obtain the existence of a saddle point for a class of two-person zero-sum games. Since there is nothing new here the details are omitted. As an example of the class of the Brownian motions described here let (B,,0,,)to, 112 be a biadditive Gaussian process, i.e., a zero mean Gaussian process with independent increments in each coordinate of the index set such that E[B(tl,Z)B(t2, "/72)]--(/71 A tz)(Z A q72). This stochastic process has continuous sample paths and when it is considered as indexed by one coordinate of the index set, it is a Brownian motion with values in the Banach space of continuous functions, C[0.1]. Optimal control results can be obtained by the previous results for stochastic differential equations with this Brownian motion. Remark on [2]. It is necessary to make the assumption of uniform integrability of the family of densities to obtain the results in Theorems 4 and 5 of [2]. One sufficient condition for this uniform integrability is that the growth of the drift term is at most linear. This fact is shown by Bene [1] by verifying that the family of densities have a bounded th. moment for some > 1. REFERENCES l] V. E. BENE, Existence of optimal stochastic control laws, this Journal, 9 (1971), pp. 446-472. [2] T. DUNCAN AND P. VARAIYA, On the solutions ofa stochastic control system, this Journal, 9 (1971), pp. 354-371. [3] R. M. DUDLEY, J. FELDMAN AND L. LECAM, On seminorms and probabilities, and abstract Wiener spaces, Ann. of Math., 93 (1971), pp. 390-408. [4] L. GROSS, Abstract Wiener spaces, Proc. Fifth Berkeley Symposium on Math. Stat. and Prob., University of California Press, Berkeley, Calif., 1965. 5] I. E. lEGAL, Distributions in Hilbert space and canonical systems of operators, Trans. Amer. Math. Soc., 88 (1958), pp. 12-41. [6] P. A. MEYER, Probability and Potentials, Blaisdell, Waltham, Mass., 1966. [7] X. FERNIQUE, IntegrabilitO des vecteurs gaussians, C.R. Acad. Sci. Paris S6r. A-B, 270 (1970), pp. 1698-1699. [8] E. NELSON, An existence theorem for second order parabolic equations, Trans. Amer. Math. Soc., 88 (1958), pp. 414-429. [9] L. GROSS, Potential theory on Hilbert space, J. Functional Analysis, (1967), pp. 123-181. [10] J. L. DOOB, Stochastic Processes, John Wiley, New York, 1953. I1 l] K. ITtS, Multiple Wiener integral, J. Math. Soc. Japan, 3 (1951), pp. 157-169. ['12] N. WIENER, The homogeneous chaos, Amer. J. Math., 60 (1938), pp. 897-936. [13] R. H. CAMERON AND W. T. MARTIN, Transformation of Wiener integrals under a general class of linear transformations, Trans. Amer. Math. Soc., 58 (1945), pp. 184-219. I. E. over Downloaded 09/10/14 to 129.237.46.100. Redistribution subject SIAM license or copyright; see http://www.siam.org/journals/ojsa.php 14] lEGAL, Tensor algebras Hilbert space I, Ibid., 81 (1956), pp. 106-134. 1092 TYRONE DUNCAN AND PRAVIN VARAIYA [15] H. KUNITA AND S. WATANABE, On square integrable martingales, Nagoya Math. J., 30 (1967), pp. 209-245. 16] I. V. GIRSANOV, On transforming a certain class of stochastic processes by absolutely continuous substitution of measures, Theor. Probability Appl., 5 (1960), pp. 285-301. [17] T. E. DUNCAN, Transforming Frdchet-valued Brownian motion by absolute continuity of measures, unpublished. 18] A. BENSOUSSAN, Gdndralization du thdorbme de Girsanov, to appear. [19] C. DOLIANS-DADE, Quelques applications de la formule de changement de variables pour les semimartingales, Z. Wahrscheinlichkeitstheorie und Verw. Gebiete, 16 (1970), pp. 181-194. [20] M. LolvE, Probability Theory, 2nd ed., Van Nostrand, Princeton, N.J., 1960. [21] V. E. BENE, Existence of optimal strategies based on specified information for a class of stochastic decision problems, this Journal, 8 (1970), pp. 179-188. Downloaded 09/10/14 to 129.237.46.100. Redistribution subject SIAM license or copyright; see http://www.siam.org/journals/ojsa.php