The Monotone Case Approach for the Solution of Certain Multidimensional Optimal Stopping Problems
Sören Christensen∗, Albrecht Irle∗
June 4, 2019
This paper studies explicitly solvable multidimensional optimal stopping problems of sum- and product-type in discrete and continuous time using the monotone case approach. It gives a review on monotone case stopping using the Doob decomposition, resp. Doob-Meyer decomposition in con- tinuous time, also in its multiplicative versions. The approach via these decompositions leads to explicit solutions for a variety of examples, includ- ing multidimensional versions of the house-selling and burglar’s problem, the Poisson disorder problem, and an optimal investment problem.
Keywords: Monotone Stopping Rules; Optimal Stopping; Explicit Solutions; Multi- dimensional; Doob Decomposition; Doob-Meyer Decomposition; House-Selling Problem; Multiple Buying-Selling; Burglar’s Problem; Poisson Disorder Problem; Optimal Invest- ment Problem Mathematics Subject Classification: 60G40; 62L10; 91G80
1 Introduction
In multidimensional problems, optimal stopping theory reaches its limits when trying to find explicit solutions for problems with a finite time horizon or an underlying (Marko- vian) process in dimension d ≥ 2. In the one-dimensional case with infinite time horizon, the optimal continuation set usually is an interval of the real line, bounded or unbounded, so it remains to determine the boundary of that interval, which boils down to finding arXiv:1705.01763v2 [math.PR] 3 Jun 2019 equations for one or two, resp., real numbers. A wealth of techniques has been developed to achieve this, see Salminen (1985), Dayanik and Karatzas (2003) for one dimensional
∗Christian-Albrechts-Universität, Mathematisches Seminar, Ludewig-Meyn-Str. 4, 24098 Kiel, Ger- many
1 diffusions, or Mordecki and Salminen (2007) and Christensen et al. (2013) for jump processes, to name but a few. The notion of a multidimensional stopping problem is employed in this article in the following sense. It is used for such problems where the general theory of optimal stop- ping prescribes that the optimal stopping time is given by the first entrance time of a stochastic process into a d-dimensional optimal stopping set with (Euclidean) dimen- sion d ≥ 2. Some typical cases are as follows: The underlying stochastic process is a Markovian process with d-dimensional state space; under certain explicit time depen- dencies of the pay-off, in particular for finite time horizon, the space-time process arises with d = 2. Particularly in discrete time problems multidimensional problems arise due to history dependence for the optimal stopping time, e.g. for a Markovian process of degree k, or even full history dependence as in the well-known best choice problem of Robbins, see Bruss (2005). There are a few multidimensional problems, see Dubins et al. (1994), Margrabe (1978), Gerber and Shiu (1996), Shepp and Shiryaev (1995), which, by some transformation method, may be transferred to a one-dimensional problem. Let us call a multidimensional problem truly multidimensional (as a manner of speech) if such a transformation seems hardly possible, at least does not seem to be available in the current literature (as we know it). Explicit solutions for such problems seem to be rare, but, of course, many techniques have been developed to tackle such problems, either semi-explicitly using nonlinear integral equations, see the monograph Peskir and Shiryaev (2006) or the more recent article Christensen et al. (2018) for an overview, or numerically, see Chapter 8 in Detemple (2006) and Glasserman (2004). The purpose of this note is to provide some examples of seemingly truly multidimen- Pm i sional problems with an explicit solution where the payoffs take the form i=1 Xn n Qm i 1 m or i=1 Xn n for m stochastic processes (Xn)n,..., (Xn )n in discrete time as well as i in continuous time. Knowing the individual solutions for the (Xn)n-problems in gen- eral does not seem to lead to the explicit solution for the sum or product problem. Here, we present a class of examples for which this is possible, the key being the notion of monotone stopping problems. The class of monotone stopping problems has been used extensively in the solution of optimal stopping problems, in particular in the first decades starting with Chow and Robbins (1961, 1963). A long list of examples can be found in Chow et al. (1971) and, more recently, in Ferguson (2008). The extension to continuous time problems is not straightforward. This was developed in Ross (1971), Irle (1979), Irle (1983), and Jensen (1989). Although these references are not very recent, it is interesting to note that the solution to certain “modern” optimal stopping problems is directly based on the notion of monotone case problems. Here, one may look e.g. at the odds-algorithm initiated in Bruss (2000) and extended by Ferguson (2016), or at Christensen (2017) where, to solve the original problem, an auxiliary problem for a two-dimensional process consisting of the underlying Markov process and its running maximum with suitable monotonicity properties is introduced. See also Christensen and Irle (2019) for related results. This paper aims at presenting the monotone case approach to optimal stopping by a systematic use of the Doob (in continuous time Doob-Meyer) decomposition of the
2 pay-off process, and shows how it may be used to solve some multidimensional stopping problems. The Doob(-Meyer) decomposition is a well-known tool in the treatment of super-/sub-martingales, in particular in optimal stopping it is applied to the Snell enve- lope. In the theory of continuous time finance, the decomposition, applied to the Snell envelope, is used to obtain duality results for option pricing, see Jamshidian (2007) for an overview and further discussions. Here we review monotone case problems in terms of the Doob(-Meyer) decomposition with regard to the optimality of the myopic stopping time in Section 2. Although this approach seems to be natural and is mathematically straightforward, we could not locate it in the literature in this form, so we present it in a survey style. Due to the simplicity of this approach, the very short proofs are given. Almost sure finiteness of stopping times is not needed in this approach. The multiplicative Doob(-Meyer) decomposition is included, and our treatment covers the discrete and continuous time case in a unified way such that it can be used for the application in multidimensional problems in Sections 3 and 4. In Section 3 we show that, under certain assumptions, the monotone case property of the individual stopping problems carries over to the sum- and product-type problems providing an explicit solution. For this, we use the Doob(-Meyer) decomposition. We discuss a variety of examples in Section 4. We start with multidimensional versions of the classical house-selling and burglar’s problem. Here, the original one-dimensional problems are well-known to be solvable using the theory of monotone stopping. Also the multidimensional house-selling problem with recall was already solved in Bruss and Ferguson (1997). The last two examples are multidimensional extensions of continuous- time problems: the Poisson disorder problem and the optimal investment problem, which in one dimension are usually solved using other argument. The more involved arguments concerning the Doob-Meyer decomposition are treated there in detail.
2 Monotone Stopping Problems
2.1 Monotone stopping problems in discrete time and the Doob decomposition Let us first stay in the realm of discrete time problems with infinite time horizon. For a se- quence X1,X2,... of integrable random variables, adapted to a given filtration (An)n∈N, we want to find a stopping time τ ∗ such that
EXτ ∗ = sup EXτ . (1) τ
Here τ runs through all stopping times such that EXτ exists. We include a random variable X∞, so that the stopping times may assume the value ∞. A natural choice for our problems below is X∞ = lim infn→∞ Xn, see the discussion in Subsection 2.2.3. There is a certain class of such problems for which we can easily solve this. Call the above problem a monotone case problem iff for all n ∈ N it holds that
E(Xn+1|An) ≤ Xn =⇒ E(Xn+2|An+1) ≤ Xn+1.
3 Using sets in the notation this may be written as
{E(Xn+1|An) ≤ Xn} ⊆ {E(Xn+2|An+1) ≤ Xn+1} for all n ∈ N.
A particularly simple sufficient condition for the monotone case is that the differences
Yn = E(Xn+1|An) − Xn are non-increasing in n, hence Yn ≤ 0 =⇒ Yn+1 ≤ 0. This condition turns out to be fulfilled in many examples of interest and allows for the treatment of multidimensional problems of sum type discussed in the following section. If we only want to consider stopping times ≥ k, e.g. if stopping in {1, ..., k − 1} is clearly suboptimal, then we may formulate the monotone case condition only for n ≥ k. 0 Of course, using Xn = Xn+(k−1), this may be subsumed in the case k = 1, so we shall look at n ≥ 1 in the following and, similarly in the latter continuous time case, at t ≥ 0. Comparing the current gain with what to expect in the next step leads to the stopping time ∗ τ = inf{n : Xn ≥ E(Xn+1|An)} = {n : Yn ≤ 0}. It is called the one-step look ahead rule or, as we will use in this paper, the myopic rule. In general, this does not yield an optimal rule, but in monotone case problems it is the natural candidate for an optimal one. The discussion of the optimality of τ ∗ for the monotone case is a well-known topic, see the references mentioned in the introduction. We, however, find it enlightening to provide a short review using the Doob decomposition, which leads to a shortcut to optimality results without the usual machinery of optimal stopping theory. This approach also provides a unifying line of argument for both discrete and continuous time. For every n let
n−1 X Mn = (Xk+1 − E(Xk+1|Ak)),M1 = 0, k=1 n−1 n−1 X X An = (E(Xk+1|Ak) − Xk) = Yk,A1 = 0, k=1 k=1 so that by telescoping expectation terms we have the Doob decomposition
Xn = X1 + Mn + An
∗ with a zero mean martingale (Mn)n∈N. For the myopic stopping time τ
∗ E(Xk+1|Ak) − Xk > 0 for k = 1, . . . , τ − 1
∗ – valid for all k ∈ N if τ = ∞ – and in the monotone case
∗ ∗ E(Xk+1|Ak) − Xk ≤ 0 for k = τ , τ + 1,....
4 Thus we have Aτ ∗ = sup An n ∗ ∗ and, using τL = min{τ ,L} for L ∈ N,
Aτ ∗ = sup An. L n≤L So, for any stopping time τ, not necessarily finite a.s.,
A ≤ A ∗ ,A ≤ A ∗ . τ τ min{τ,L} τL Basically these simple inequalities are the foundation for the optimality properties of the myopic stopping time. We provide sufficient conditions for this optimality, called (V1) and (V2), which are suitable for our classes of problems, in Subsection 2.2. Remark 2.1. It is remarkable that the myopic stopping rule immediately provides optimal stopping times for all possible time horizons in the monotone case. The same observation also holds true in the continuous time case discussed below. See also Ferguson (2008) and Irle (2017). This is in strong contrast to most Markovian-type optimal stopping problems, where infinite time problems are often easier to solve as the stopping boundary is not time dependent.
2.2 Optimality of The Myopic Rule Based on the Doob Decomposition We continue the treatment of Subsection 2.1 and assume that we are in the monotone case.
2.2.1 Optimality for finite time horizon
Let L ∈ N and τ ≤ L a bounded stopping time. Then, using the martingale property, EMτ = 0, valid for bounded stopping times,
EX = EX + EA ≤ EX + EA ∗ = EX ∗ . τ 1 τ 1 τL τL ∗ This implies optimality of τL for the finite time horizon L.
2.2.2 Optimality for infinite time horizon
The extension from the finite to the infinite case uses the approximation τ = limL→∞ τL,
τL = min{τ, L}, so that Xτ = limL→∞ XτL on {τ < ∞}, but on {τ = ∞} we need a specific definition of X∞. Here, we use
X∞ = lim inf Xn, n→∞ so that Xτ = lim infL→∞ XτL . We introduce the following conditions:
lim EXτ ∗ ≤ EXτ ∗ (V1) L→∞ L
sup{EXτ : τ bounded} = sup EXτ (V2) τ Note for (V1) that EX ∗ is increasing in L. τL
5 Proposition 2.2. Under (V1) and (V2), τ ∗ is optimal, i.e.
EXτ ∗ = sup EXτ . τ Proof. We have from the previous considerations that
EXτ ∗ ≥ lim sup EXτ = sup{EXτ : τ bounded}, L→∞ τ≤L proving the claim by (V2).
2.2.3 Discussion of Assumptions (V1) and (V2) The validity of (V2) is of course a well-known topic in optimal stopping, independently of the monotone case context. We only remark that, under E(infn Xn) > −∞, Fatou’s Lemma shows that for any τ
lim inf EXτ ≥ E(lim inf Xτ ) = EXτ . L→∞ L L→∞ L 0 0 The same holds if we add costs of observation, e.g. Xn = Xn−cn, assuming E(infn Xn) > −∞. (V1) follows from the condition E(supn Xn) < ∞, which is the standard assumption in optimal stopping theory, see e.g. Theorem 4.5, 4.5’ in Chow et al. (1971). This needs a short argument.
Proposition 2.3. E(supn Xn) < ∞ =⇒ (V1). Proof. Using Fatou’s Lemma again, we have
E lim sup Xτ ∗ ≥ lim sup EXτ ∗ . L→∞ L L→∞ L
Due to our definition of X we have to show that lim sup X ∗ = lim inf X ∗ ∞ L→∞ τL L→∞ τL ∗ on {τ = ∞}. On this set, (An)n∈N is increasing, hence limn→∞ An exists. Furthermore, ∗ (Mτn )n∈N is a martingale fulfilling the boundedness condition
Mτ ∗ = Xτ ∗ − X1 − Aτ ∗ ≤ sup Xn + |X1|, n n n n
∗ since Aτn ≥ 0. We may thus invoke the martingale convergence theorem and obtain the ∗ ∗ ∗ convergence of Mτn to some a.s. finite random variable. Since τn = n on {τ = ∞}, this shows the convergence of (Mn)n, hence of (Xn)n, on this set.
2.3 Monotone stopping problems in continuous time and the Doob-Meyer decomposition
To find the extension of the discrete time case to continuous time processes (Xt)t∈[0,∞) we may use the Doob-Meyer decomposition. Under regularity assumptions, not discussed here, we have Xt = X0 + Mt + At,
6 where (Mt)t∈[0,∞) is a zero mean martingale and (At)t∈[0,∞) is of locally bounded varia- tion. Now assume that we may write Z t At = YsdVs 0 where (Vt)t∈[0,∞) is increasing. Then the myopic stopping time – here often called in- finitesimal look ahead rule – becomes
∗ τ = inf{t : Yt ≤ 0}. In this situation, we say that the monotone case holds if
∗ Yt ≤ 0 for t > τ .
If (Yt)t∈[0,∞) is non-increasing in t, then again the monotone case property is immediate. The discussion of optimality is essentially the same as in the discrete time case, so is omitted. As no confusion can occur, we keep the notations (V1) and (V2) for the continuous-time versions of the optimality conditions.
2.4 Monotone stopping problems in discrete time and the multiplicative Doob decomposition
For processes (Xn)n∈N with Xn > 0 we may also consider the multiplicative Doob decomposition Xn = MnAn where, with X0 = 1, A0 = {∅, Ω}, n X M = Y k , n ≥ 1, is a mean 1-martingale, n E(X |A ) k=1 k k−1 n E(X |A ) A = Y k k−1 , n ≥ 1. n X k=1 k−1 Optimality of the myopic stopping time may also be inferred from this multiplicative decomposition in the monotone case. As in Subsection 2.1, we have for any τ
Aτ ≤ Aτ ∗ = sup An,Amin{τ,L} ≤ Aτ ∗ . n L The multiplicative decomposition leads, however, to different sufficient conditions for optimality. There is also a connection to a change of measure approach. Both is discussed in Subsection 2.5 below. We furthermore observe that we have a monotone case problem in particular if Xn+1 E An is non-increasing in n, Xn which turns out to be a basis for the treatment of product-type problems in the following section.
7 2.5 Optimality of The Myopic Rule based on the Multiplicative Decomposition We assume the setting of Subsection 2.4 and work under the assumption that we are in the monotone case.
2.5.1 Optimality for finite time horizon For any bounded stopping time τ ≤ L
EX = EM A = EM A ≤ EM A ∗ = EX ∗ , τ τ τ L τ L τL τL so we arrive as in Subsection 2.2.1 at
EXτ ∗ = sup EXτ . L τ≤L
2.5.2 Optimality for infinite time horizon To extend this argument to infinite time horizon, first note that, due to the positivity property of the Xn, (V2) is valid due to the discussion in Subsection 2.2.3. Condi- tion (V1) has to be taken care of for the specific problem at hand (and we know from Subsection 2.2.3 that E(supn Xn) < ∞ is sufficient). We now present a measure-change approach leading to another sufficient condition for dQ|An optimality. We use a probability measure Q such that = Mn for each n, invoking dP |An the Kolmogorov extension theorem for the existence of Q. Then for any stopping time τ
EXτ 1{τ<∞} = EQAτ 1{τ<∞}.
Proposition 2.4. Assume that
Q(τ ∗ < ∞) = 1. (W1)
∗ Then, τ is optimal, i.e. EXτ ∗ = supτ EXτ Proof. For any stopping time τ
EXτ ≤ lim inf EXmin{τ,L} = lim inf EQAmin{τ,L} L→∞ L→∞
≤ EQAτ ∗ = EQAτ ∗ 1{τ ∗<∞}
= EXτ ∗ 1{τ ∗<∞} ≤ EXτ ∗ .
As (W1) does not seem to be very handy for applications, we now give a sufficient condition. Using Z Z Z ∗ Xn Q(τ > n) = MndP = dP ≤ XndP, {τ ∗>n} {τ ∗>n} An {τ ∗>n}
8 we see that we obtain optimality for τ ∗ if Z XndP → 0 as n → ∞. {τ ∗>n} Note the similarities to the approach of Beibel and Lerche as presented, e.g., in Beibel and Lerche (1997) and Lerche and Urusov (2007). There, in the continuous time case, the decomposition Xt = AtMt and EXτ 1{τ<∞} = EQAτ 1{τ<∞} is used for the case At = g(Zt) for some diffusion Z. Then, the stopping time
∗ σ = inf{t : g(Zt) = sup g(z)} z ∗ has on {σ < ∞} the property Aσ∗ = supt At, as the myopic stopping time.
2.6 Monotone stopping problems in continuous time and the multiplicative Doob-Meyer decomposition Also in continuous time, a multiplicative Doob-Meyer-type decomposition of the form
Xt = MtAt can be found in the case of a positive special semimartingale X, see Jamshidian (2007). For the ease of exposition, we now concentrate on the case of continuous semimartingales to have more explicit formulas. Using ibid, Theorem 4.2, M is a local martingale and R t if ( 0 YsdVs)t∈[0,∞) denotes the process in the additive Doob-Meyer decomposition as in Subsection 2.3, the process A here is given by t Z Ys At = exp dVs . 0 Xs The optimality may be discussed as in the discrete time case. We again remark that the problem can be identified to be monotone in particular if the process Y t is non-increasing. Xt t∈[0,∞)
3 Multidimensional Monotone Case Problems
We now come to the main point of this paper: Can we use the monotone case approach to find truly multidimensional stopping problems with explicit solutions? The answer is yes as shown in the following section by several non-trivial examples.
3.1 The sum problem
Already for sequences of real numbers, supn(an + bn) ≤ supn an + supn bn, usually with strict inequality. For optimal stopping,
1 2 1 2 sup E(Xτ + Xτ ) ≤ sup E(Xτ ) + sup E(Xτ ), τ τ τ
9 with strict inequality as a rule; this means that being able to solve the stopping problems 1 2 for (Xn)n and (Xn)n does not imply that we are able to solve the stopping problem for 1 2 (Xn + Xn)n.
3.1.1 Discrete time case
1 m Now let us look at m sequences (Xn)n∈N,..., (Xn )n∈N, adapted to a common filtration (An)n∈N, with Doob decompositions
i i i i Xn = X1 + Mn + An, i = 1, . . . , m,
i Pn−1 i where An = k=1 Yk as in Subsection 2.1. Then, the Doob decomposition for the sum process is m m m n−1 m X i X i X i X X i Xn = X1 + Mn + Yk , i = 1, . . . , m. i=1 i=1 i=1 k=1 i=1 i Now, if for each i the stopping problem for (Xn)n∈N is a monotone case problem it does not necessarily follow that we have a monotone case problem for Pm Xi , see i=1 n n∈N i Example 4.2 below. But in the special case that all the (Yk )k∈N are non-increasing in k the monotone case property holds. We formulate this as a simple proposition:
Proposition 3.1. Assume that the processes X1,...,Xk have Doob decompositions
n−1 i i i X i Xn = X1 + Mn + Yk , i = 1, . . . , m, k=1
i such that all the sequences (Yk )k∈N are non-increasing in k. Then: (i) The sum problem for Pm Xi is a monotone case problem with myopic rule i=1 n n∈N m ∗ X i τ = inf{k : Yk ≤ 0}. i=1
(ii) If (V1) and (V2) hold for Pm Xi , then the myopic rule τ ∗ is optimal. i=1 n n∈N Proof. For the Doob decomposition
m m m n−1 m X i X i X i X X i Xn = X1 + Mn + Yk , i = 1, . . . , m, i=1 i=1 i=1 k=1 i=1 the sequence Pm Y i is non-increasing in k by assumption, yielding (i). i=1 k k∈N (ii) now follows from (i) using Proposition 2.2.
10 3.1.2 Continuous time case
1 m Now let us look at m continuous time processes (Xt )t∈[0,∞),..., (Xt )t∈[0,∞), adapted to a common filtration (An)n∈N, with Doob-Meyer decompositions
i i i i Xi = X0 + Mt + At, i = 1, . . . , m,
i R t i where At = 0 Ys dVs for an increasing V independent of i. (The typical case is dVs = ds.) Then, the Doob decomposition for the sum process is
m m m Z t m X i X i X i X i Xt = X0 + Mt + Ys dVs, i = 1, . . . , m, i=1 i=1 i=1 0 i=1
i so that for non-increasing (Yt )t∈[0,∞), i = 1, . . . , m, the monotone case property holds. We obtain as in the discrete time case:
Proposition 3.2. Assume that the processes X1,...,Xk have Doob decompositions
Z t i i i i Xi = X0 + Mt + Ys dVs, i = 1, . . . , m, 0
i such that all the processes (Yt )t∈[0,∞) are non-increasing in t. Pm i (i) The sum problem for i=1 Xt t∈[0,∞) is a monotone case problem with myopic rule m ∗ X i τ = inf{t : Yt ≤ 0}. i=1