Set Valued Dynamic Treatment Regimes by Tianshuang Wu A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy (Statistics) in The University of Michigan 2016 Doctoral Committee: Professor Susan A Murphy, Chair Professor Xuming He Assistant Professor Ambuj Tewari Associate Professor Lu Wang c Tianshuang Wu 2016 All Rights Reserved To my parents Jing and Wei To my wife Yingyi and my daughter Audrey ii ACKNOWLEDGEMENTS I would like to give my thanks to my advisor Prof. Susan Murphy. For the past five years, she turned me from a student whose only advantage was taking exam to a researcher by teaching me how to discover a scientific problem, evaluate it and find ways to deal with it. However, the help and inspiration from her includes far more than knowledge and research. As a role model, she showed me almost all the virtue of a researcher: solid mathematic background, the enthusiasm of diving into the problem, the ability of communicating with people both inside and outside Statistics field and work with people in the research lab. I learned these from her not from her word, but her act. I feel really lucky to have her as my mentor. Everything that I learned from her has been, and will always stay with me and help me for the rest of my life. I would thank my committee members, Prof. He, Prof. Tewari and Prof. Wang, for their kind help and advise during these years. I could always gain fresh and new ideas when discussing with them. Also I would thank Prof. Almirall and the former postdoc Ashkan Ertefaie, for their cooperation for the paper we wrote together. They are both great researchers and good friends and I have had a good time working with them. Of course, I would thank everyone in our lab. They are perfect friends, colleagues and I'm grateful that I have them around during my time at University of Michigan. Finally I would give my most sincere thanks to my parents Wei and Jing, my wife Yingyi and my lovely daughter Audrey. None of my work could have been iii done without their support and encouragement, and none of my work would be so meaningful to me without them. iv TABLE OF CONTENTS DEDICATION :::::::::::::::::::::::::::::::::: ii ACKNOWLEDGEMENTS :::::::::::::::::::::::::: iii LIST OF FIGURES ::::::::::::::::::::::::::::::: vii LIST OF TABLES :::::::::::::::::::::::::::::::: viii LIST OF ABBREVIATIONS ::::::::::::::::::::::::: ix ABSTRACT ::::::::::::::::::::::::::::::::::: x CHAPTER I. Introduction ..............................1 II. Comparison methods .........................4 2.1 Introduction . .4 2.2 Review of the comparison methods . .6 2.2.1 Union-intersection principle (UIP) method . .6 2.2.2 Likelihood Ratio Test (Likelihood ratio test (LRT)) 13 2.2.3 Bayesian approach . 20 2.2.4 Comparison of these methods . 24 2.3 Step-up and step-down methods . 33 2.3.1 Introduction . 33 2.3.2 Form of the two methods . 34 2.3.3 Simulation Study . 37 2.3.4 Discussion . 37 III. Set valued DTR ............................ 40 3.1 Introduction . 40 3.2 Set-valued DTRs and the construction of the recommended sets 43 v 3.3 Role of the ACI in the construction of the recommended set 50 3.4 Simulation study . 58 3.5 Analysis of the ADHD study . 61 3.6 Conclusion and future work . 66 3.7 Three treatment per stage case . 67 3.7.1 Introduction . 67 3.7.2 Formulation of the problem . 68 3.7.3 Simulation Study . 81 3.7.4 Discussion . 82 IV. Identifying a set that contains the best DTR .......... 83 4.1 Introduction . 83 4.2 Preliminaries . 85 4.2.1 Sequential, Multiple Assignment, Randomized Trials 85 4.2.2 Data Structure . 86 4.2.3 Embedded Dynamic Treatment Regimes . 86 4.3 Estimation . 87 4.4 Multiple Comparison with the Best . 90 4.5 Simulation Study . 93 4.5.1 SMART Design: Example 1 . 94 4.5.2 SMART Design: Example 2 . 96 4.6 Illustrative data analysis . 98 4.7 Discussion . 100 4.8 Comparison with the modified version of ACI method . 100 V. Discussion and future work ..................... 104 5.1 Discussion . 104 5.2 Future work . 105 APPENDIX :::::::::::::::::::::::::::::::::::: 106 A.1 The proof of theorem III.5 . 107 A.2 The proof of lemma III.8 . 118 A.3 Proof of theorems in chapter IV . 121 A.4 Tables for chapter IV . 124 A.5 Discussion and tables for the simulation results in section 4.8 125 BIBLIOGRAPHY :::::::::::::::::::::::::::::::: 132 vi LIST OF FIGURES Figure 2.1 The rejection regions of Perlman's test. 14 2.2 The rejection regions of Berger's two new tests. 15 2.3 The acceptance regions from Gupta and LRT. 17 2.4 The difference of expected set sizes between set from Gupta's method and the step-up method. 38 2.5 The difference of expected set sizes between set from Gupta's method and the step-down method. 38 2.6 The difference of expected set sizes between set from Gupta's method and the step-down method. 39 3.1 The design of the ADHD study. 63 A.1 Simulation SMART design Example 1: The vertical axes is the estimated set (of best) size (ESS) and horizontal axes is the difference between the best and the second best EDTR. .................... 128 A.2 Simulation SMART design Example 2: The vertical axis are the estimated set (of best) size (ESS) and horizontal axes are the difference between the best and the second best EDTR. .................... 131 vii LIST OF TABLES Table 3.1 Description of the simulation models using ACI . 60 3.2 Simulation results of the ACI . 61 3.3 Descriptions of variables in ADHD . 63 3.4 Second-stage results for ADHD . 64 3.5 First-stage result for AHDH . 65 A.1 Simulation SMART design Example 1: Inference about the param- eters β using IPW, AIPW and AIPWm where the latter represents the misspecified scenario. 124 A.2 Simulation SMART design Example 2: Inference about the param- eters β using IPW, AIPW and AIPWm where the latter represents the misspecified scenario. 125 A.3 Extend trial: Inference about the parameters β using IPW and AIPW.126 A.4 Results of Scenario One . 126 A.5 Results of Scenario Two . 127 A.6 Results of Scenario Three . 128 A.7 Results of Scenario Four . 129 viii LIST OF ABBREVIATIONS FWER Family-Wise Error Rate LRT Likelihood ratio test MCB Multiple Comparisons with the Best MCC Multiple Comparison with the Control SMART Sequential Multi-Assignments Randomized Trials UIP Union-intersection principle ix ABSTRACT Set Valued Dynamic Treatment Regimes by Tianshuang Wu Chair: Susan Murphy Dynamic Treatment Regimes (DTR)s are composed of sequences of decision rules, one per stage of treatment. Each decision rule inputs patient information and outputs a single recommended treatment. While the majority of present studies are focused on finding the optimal DTR, we take another approach. Instead of trying to determine the true best DTR, we aim to construct a set of DTRs such that the true best DTR is contained in this set with a desired probability. The reasons are as follows: (1) Usually we do not have enough data to identify the best DTR and (2) we want to give patients and clinicians more options. To discuss the second reason in more detail, patients and clinicians might have treatment preferences related to cost, side effects or convenience, etc. Thus, our goal is to provide a recommended set of DTRs, such that the DTRs contained in the set are those we cannot distinguish from the best, while the DTRs we exclude are those that are certain to be inferior with high confidence. This idea comes from decision support: we do not tell patients and clinicians what to do; we do not offer treatments known to be inferior. Rather we offer a set of treatments that excludes inferior treatments. In this thesis we develop a set valued DTR in which the decision rules at each stage can output a set of treatments. Second we develop x an approach for constructing a recommended set of DTRs. In the appendix we prove the relevant theorems. xi CHAPTER I Introduction Comparisons of treatments is always crucial when we have more than one available candidate at hand at a certain situation, e.g., at a certain time point for a certain kind of patients. For the comparison of two populations, there are a lot of well developed methods from parametric methods like t-test to non-parametric methods like U-test. However, in practice, we often face the situation when more than two treatments are available at hand and we would like to compare them together. One naive way is N to compare them pairwise and produce 2 confidence intervals of the difference s of the effects between all pair of treatments. Naturally this approach fails to consider the Family-Wise Error Rate (FWER) (see Shaffer (1995) for more details). Rescues have been made by different approaches, like the Bonferroni correction proposed by Bonferroni (1936) and developed by Dunn (1959, 1961),the Sidak correction which is credited to Sid´akˇ (1967) by Seidler et al. (2000). If there is one \standard effect” that serves as a control and we want to compare the effect of each treatment with this control, a simultaneous comparison method called Multiple Comparison with the Control (MCC) (Dunnett, 1955) can be applied. If we are only interested in whether all effects of the treatments are equal, ANOVA (Box et al., 1954a,b) could be applied. But if we want further know the relationship between the effects of the treatments, the Multiple Comparisons with the Best (MCB) method proposed by Gupta (1965) and 1 developed by Hsu (1996) can be applied.