Special Issue Paper Received 2 July 2016, Revised 15 November 2016, Accepted 19 November 2016 Published online 13 January 2017 in Wiley Online Library (wileyonlinelibrary.com) DOI: 10.1002/asmb.2222 Clinical trial design as a decision problem Peter Müllera*†, Yanxun Xub and Peter F. Thallc The intent of this discussion is to highlight opportunities and limitations of utility-based and decision theoretic arguments in clinical trial design. The discussion is based on a specific case study, but the arguments and principles remain valid in general. The exam- ple concerns the design of a randomized clinical trial to compare a gel sealant versus standard care for resolving air leaks after pulmonary resection. The design follows a principled approach to optimal decision making, including a probability model for the unknown distributions of time to resolution of air leaks under the two treatment arms and an explicit utility function that quan- tifies clinical preferences for alternative outcomes. As is typical for any real application, the final implementation includessome compromises from the initial principled setup. In particular, we use the formal decision problem only for the final decision, but use reasonable ad hoc decision boundaries for making interim group sequential decisions that stop the trial early. Beyond the discussion of the particular study, we review more general considerations of using a decision theoretic approach for clinical trial design and summarize some of the reasons why such approaches are not commonly used. Copyright © 2017 John Wiley & Sons, Ltd. Keywords: Bayesian decision problem; Bayes rule; nonparametric Bayes; optimal design; sequential stopping 1. Introduction We discuss opportunities and practical limitations of approaching clinical trial design as a formal decision problem. Using a case study as a running example keeps the argument focused and specific. We review a study that was set up to compare a hydrogel sealant (Progel) against standard care for patients who develop air leaks after pulmonary resection. The main features of the design are the elicitation of a utility function that quantifies clinical preferences for time to resolve the air leaks and a nonparametric Bayesian prior for the distributions of the resolution time under the two treatment arms. A Bayesian nonparametric (BNP) model is a prior for an unknown probability measure that is not restricted to a specific parametric family. Both features are important. The utility function is only meaningful if the probability model allows learning about detailed features of the event time distribution, and the nonparametric model is only needed when the decision hinges on such details. In the upcoming discussion, we focus mainly on the features of the decision problem. A complete discussion of the design and the trial appears in [1], including extensive simulations to evaluate the design’s operating characteristics (OCs) under alternative scenarios. The use of decision theoretic approaches in Bayesian clinical trial design is rare. Commonly used methods use Bayesian inference to compute posterior probabilities of clinically meaningful events or inference summaries for key parameters but then use these summaries for reasonable, but ad hoc designs. See, for example, [2] or [3] for a recent review. Beyond clinical trial design, Bayesian decision theoretic approaches are not widely used for optimal design in general. The review by [4] discusses commonly used Bayesian approaches to optimal design, focusing mainly on design problems related to learning about unknown parameters but clearly recognizing the importance of more general problems. To date, this paper remains one of the most cited and most comprehensive reviews of Bayesian optimal design. In their final discussion, Chaloner and Verdinelli write ’It is clearly helpful, in the design process, to carefully consider the reason the experiment is being done and to consider what utility should be used. [...] it would also be interesting to see alternatives constructed and explored in future research’. In the following discussion, we review one such construction which is in the spirit of what Kathryn Chaloner might have wished to see. aDepartment of Mathematics, University of Texas at Austin, USA bDepartment of Applied Mathematics and Statistics, Johns Hopkins University, USA 296 cDepartment of Biostatistics, University of Texas, M.D. Anderson Cancer Center, USA *Correspondence to: Peter Müller, Department of Mathematics, University of Texas at Austin, USA †E-mail: [email protected] Copyright © 2017 John Wiley & Sons, Ltd. Appl. Stochastic Models Bus. Ind. 2017, 33 296–301 P. MÜLLER ET AL. The discussion is restricted to the particular decision problem of clinical trial design and even further focused by fol- lowing a particular example. The arguments are meant to highlight the benefits of following a formal decision theoretic setup and also to indicate the practical limitations and challenges that are involved. 2. Decision problem 2.1. Framework We set up the design problem as a Bayesian decision problem, following a formal, principled approach. See, for example, [5] or [6] for a description of the general framework. Briefly summarized, the ingredients of a Bayesian decision problem are a probability model for all relevant unknown quantities, including data, future data, and unknown parameters of interest and a utility function that formalizes relative preferences of a decision maker under hypothetical data and assumed parameters. The optimal action, known as the Bayes rule, is the action that maximizes utility, in expectation over all unknown variables and conditional on all known variables. To be specific, let y denote the data. Often, some data may already be observed at , the time of decision making. In that case, we partition the data vector into (y y0) denoting the observed data by y0 (but we will simply use y when all data is observed). Let denote unknown parameters, and let d denote the decision. The Bayesian , , probability model usually is given as a sampling model for the data, p(y y0 ∣ d) and a prior model, p( ). Including d here in the conditioning set is a slight abuse of notation and only indicates that the sampling model can be indexed by d. In general, the prior also could be indexed by d, but this usually is not performed. Finally, let u(d,,y) denote the utility, defined as a function of a possible action d, assumed parameters , and hypothetical data y. The utility function represents the decision maker’s relative preferences for actions under an assumed truth and data. Let A denote the action set of all possible decisions under consideration. The Bayes rule is ⋆ ,, , , . d = arg max u(d y) p( y ∣ y0 d) dy d (1) d∈A ∫ ∫ ,, , , ⋆ Letting U(d ∣ y0)= u(d y) p( y ∣ y0 d) dy d , we can write d = arg max U(d ∣ y0). Here, the conditioning bar in ⋅ U( ) indicates the conditioning on y0 in the expectation. One can argue, from first principles, that a rational decision maker should act as if he or she were maximizing expected utility U(d ∣ y0) [7]. An important detail in the setup is the statement of the action set A. Usually, the set of possible actions is highly restricted, to avoid unintuitive, unreasonable, or impractical decisions. A good choice of action set avoids awkward solutions d⋆. Mathematically, a choice of a probability model and a utility function implies an optimal decision d⋆. But the mapping is very indirect through the integration and maximization in (1), and technical details in the choice of the probability model and utility function could lead to unintended solutions if care is not taken to restrict A suitably. We will give examples of this later on. 2.2. Terminal decision The design of the Progel study involves two different types of decisions. After each patient cohort is treated, and their , outcomes are observed, we decide whether to continue accrual (Sequential Stopping Decision). Let ac ∈{0 1} denote the , , continuation decision, with ac = 0 indicating continuation and ac = 1 for early stopping. We index cohorts by c = 1 … C, with a1 denoting the continuation decision after the first cohort, etc. The study includes a maximum sample size. That is,we , restrict aC = 1. Upon stopping, we decide whether or not to report Progel as superior (Terminal Decision). Let d ∈{0 1} denote the terminal decision, with d = 0 for recommending standard care versus d = 1 for Progel. In summary, the action , , set is ac ∈{0 1} and d ∈{0 1}. Ideally, all decisions should be made as Bayes rule with respect to the same underlying utility function. However, this is where the need to construct a practicable implementation that actually can be used to conduct clinical trial parts with the principled approach. For the Progel trial, only the terminal decision d is implemented as a Bayes rule. In contrast, the stopping decisions are based on a reasonable group sequential decision boundary. Details of the latter are discussed in the next section. We first discuss the terminal decision and explain the terminal decision graphically, in Figure1. Posterior inference. Index j = 1 for Progel and j = 0 for standard care, with sample sizes n1 and n0 Let yji denote the , , . time until resolution of air leaks for patient i = 1 ··· nj in treatment arm j Let Gj(y) denote the distribution of time to , , , . resolution of air leaks, j = 0 1. That is, yji ∼ Gj independently for all i = 1 ··· nj We will use a BNP model for Gj. That is, Gj itself is the unknown parameter, for which the BNP model defines a prior p(Gj). Figure 1(a) shows hypothetical posterior means for G and G , conditional on all patients up to a particular time during the trial. For some patients, actual 0 1 297 event times are recorded, while for others the time until resolution of air leaks is censored at the current time.