Decision Theory
Total Page:16
File Type:pdf, Size:1020Kb
Decision Theory Principles and Approaches Giovanni Parmigiani Johns Hopkins University, Baltimore, USA Lurdes Y. T. Inoue University of Washington, Seattle, USA with contributions by Hedibert F. Lopes University of Chicago, USA A John Wiley and Sons, Ltd., Publication Decision Theory WILEY SERIES IN PROBABILITY AND STATISTICS Established by WALTER A. SHEWHART and SAMUEL S. WILKS Editors DAVID J. BALDING,NOEL A. C. CRESSIE,GARRETT M. FITZMAURICE,IAIN M. JOHN- STONE,GEERT MOLENBERGHS,DAVID W. SCOTT,ADRIAN F. M. SMITH,RUEY S. TSAY, SANFORD WEISBERG,HARVEY GOLDSTEIN. Editors Emeriti VIC BARNETT,J.STUART HUNTER,JOZEF L. TEUGELS A COMPLETE LIST OF THE TITLES IN THIS SERIES APPEARS AT THE END OF THIS VOLUME. Decision Theory Principles and Approaches Giovanni Parmigiani Johns Hopkins University, Baltimore, USA Lurdes Y. T. Inoue University of Washington, Seattle, USA with contributions by Hedibert F. Lopes University of Chicago, USA A John Wiley and Sons, Ltd., Publication This edition first published 2009 c 2009 John Wiley & Sons, Ltd. Registered office John Wiley & Sons Ltd, The Atrium, Southern Gate, Chichester, West Sussex, PO19 8SQ, United Kingdom For details of our global editorial offices, for customer services and for information about how to apply for permission to reuse the copyright material in this book please see our website at www.wiley.com. The right of the author to be identified as the author of this work has been asserted in accordance with the Copyright, Designs and Patents Act 1988. All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, except as permitted by the UK Copyright, Designs and Patents Act 1988, without the prior permission of the publisher. Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic books. Designations used by companies to distinguish their products are often claimed as trademarks. All brand names and product names used in this book are trade names, service marks, trademarks or registered trademarks of their respective owners. The publisher is not associated with any product or vendor mentioned in this book. This publication is designed to provide accurate and authoritative information in regard to the subject matter covered. It is sold on the understanding that the publisher is not engaged in rendering professional services. If professional advice or other expert assistance is required, the services of a competent professional should be sought. Library of Congress Cataloging-in-Publication Data Parmigiani, G. (Giovanni) Decision theory : principles and approaches / Giovanni Parmigiani, Lurdes Inoue. p. cm. Includes bibliographical references and index. ISBN 978-0-471-49657-1 (cloth) 1. Statistical decision. 2. Axiomatic set theory. 3. Experimental design. I. Inoue, Lurdes. II. Title. QA279.4.P37 2009 519.542—dc22 2009008345 A catalogue record for this book is available from the British Library. ISBN 978-0-471-49657-1 Set in 10/12pt Times by Integra Software Services Pvt. Ltd, Pondicherry, India Printed in the UK by TJ International, Padstow, Cornwall To our advisors: Don Berry, Morrie De Groot, and Jay Kadane; and to their advisors: Jay Kadane, Jimmy Savage, and Herman Chernoff Contents Preface xiii Acknowledgments xvii 1 Introduction 1 1.1 Controversies 1 1.2 A guided tour of decision theory 6 Part One Foundations 11 2 Coherence 13 2.1 The “Dutch Book” theorem 15 2.1.1 Betting odds 15 2.1.2 Coherence and the axioms of probability 17 2.1.3 Coherent conditional probabilities 20 2.1.4 The implications of Dutch Book theorems 21 2.2 Temporal coherence 24 2.3 Scoring rules and the axioms of probabilities 26 2.4 Exercises 27 3 Utility 33 3.1 St. Petersburg paradox 34 3.2 Expected utility theory and the theory of means 37 3.2.1 Utility and means 37 3.2.2 Associative means 38 3.2.3 Functional means 39 3.3 The expected utility principle 40 3.4 The von Neumann–Morgenstern representation theorem 42 3.4.1 Axioms 42 3.4.2 Representation of preferences via expected utility 44 3.5 Allais’ criticism 48 3.6 Extensions 50 3.7 Exercises 50 viii CONTENTS 4 Utility in action 55 4.1 The “standard gamble” 56 4.2 Utility of money 57 4.2.1 Certainty equivalents 57 4.2.2 Risk aversion 57 4.2.3 A measure of risk aversion 60 4.3 Utility functions for medical decisions 63 4.3.1 Length and quality of life 63 4.3.2 Standard gamble for health states 64 4.3.3 The time trade-off methods 64 4.3.4 Relation between QALYs and utilities 65 4.3.5 Utilities for time in ill health 66 4.3.6 Difficulties in assessing utility 69 4.4 Exercises 70 5 Ramsey and Savage 75 5.1 Ramsey’s theory 76 5.2 Savage’s theory 81 5.2.1 Notation and overview 81 5.2.2 The sure thing principle 82 5.2.3 Conditional and a posteriori preferences 85 5.2.4 Subjective probability 85 5.2.5 Utility and expected utility 90 5.3 Allais revisited 91 5.4 Ellsberg paradox 92 5.5 Exercises 93 6 State independence 97 6.1 Horse lotteries 98 6.2 State-dependent utilities 100 6.3 State-independent utilities 101 6.4 Anscombe–Aumann representation theorem 103 6.5 Exercises 105 Part Two Statistical Decision Theory 109 7 Decision functions 111 7.1 Basic concepts 112 7.1.1 The loss function 112 7.1.2 Minimax 114 7.1.3 Expected utility principle 116 7.1.4 Illustrations 117 7.2 Data-based decisions 120 7.2.1 Risk 120 7.2.2 Optimality principles 121 CONTENTS ix 7.2.3 Rationality principles and the Likelihood Principle 123 7.2.4 Nuisance parameters 125 7.3 The travel insurance example 126 7.4 Randomized decision rules 131 7.5 Classification and hypothesis tests 133 7.5.1 Hypothesis testing 133 7.5.2 Multiple hypothesis testing 136 7.5.3 Classification 139 7.6 Estimation 140 7.6.1 Point estimation 140 7.6.2 Interval inference 143 7.7 Minimax–Bayes connections 144 7.8 Exercises 150 8 Admissibility 155 8.1 Admissibility and completeness 156 8.2 Admissibility and minimax 158 8.3 Admissibility and Bayes 159 8.3.1 Proper Bayes rules 159 8.3.2 Generalized Bayes rules 160 8.4 Complete classes 164 8.4.1 Completeness and Bayes 164 8.4.2 Sufficiency and the Rao–Blackwell inequality 165 8.4.3 The Neyman–Pearson lemma 167 8.5 Using the same α level across studies with different sample sizes is inadmissible 168 8.6 Exercises 171 9 Shrinkage 175 9.1 The Stein effect 176 9.2 Geometric and empirical Bayes heuristics 179 9.2.1 Is x too big for θ? 179 9.2.2 Empirical Bayes shrinkage 181 9.3 General shrinkage functions 183 9.3.1 Unbiased estimation of the risk of x + g(x) 183 9.3.2 Bayes and minimax shrinkage 185 9.4 Shrinkage with different likelihood and losses 188 9.5 Exercises 188 10 Scoring rules 191 10.1 Betting and forecasting 192 10.2 Scoring rules 193 10.2.1 Definition 193 10.2.2 Proper scoring rules 194 10.2.3 The quadratic scoring rules 195 10.2.4 Scoring rules that are not proper 196 x CONTENTS 10.3 Local scoring rules 197 10.4 Calibration and refinement 200 10.4.1 The well-calibrated forecaster 200 10.4.2 Are Bayesians well calibrated? 205 10.5 Exercises 207 11 Choosing models 209 11.1 The “true model” perspective 210 11.1.1 Model probabilities 210 11.1.2 Model selection and Bayes factors 212 11.1.3 Model averaging for prediction and selection 213 11.2 Model elaborations 216 11.3 Exercises 219 Part Three Optimal Design 221 12 Dynamic programming 223 12.1 History 224 12.2 The travel insurance example revisited 226 12.3 Dynamic programming 230 12.3.1 Two-stage finite decision problems 230 12.3.2 More than two stages 233 12.4 Trading off immediate gains and information 235 12.4.1 The secretary problem 235 12.4.2 The prophet inequality 239 12.5 Sequential clinical trials 241 12.5.1 Two-armed bandit problems 241 12.5.2 Adaptive designs for binary outcomes 242 12.6 Variable selection in multiple regression 245 12.7 Computing 248 12.8 Exercises 251 13 Changes in utility as information 255 13.1 Measuring the value of information 256 13.1.1 The value function 256 13.1.2 Information from a perfect experiment 258 13.1.3 Information from a statistical experiment 259 13.1.4 The distribution of information 264 13.2 Examples 265 13.2.1 Tasting grapes 265 13.2.2 Medical testing 266 13.2.3 Hypothesis testing 273 13.3 Lindley information 276 13.3.1 Definition 276 13.3.2 Properties 278 13.3.3 Computing 280 13.3.4 Optimal design 281 CONTENTS xi 13.4 Minimax and the value of information 283 13.5 Exercises 285 14 Sample size 289 14.1 Decision-theoretic approaches to sample size 290 14.1.1 Sample size and power 290 14.1.2 Sample size as a decision problem 290 14.1.3 Bayes and minimax optimal sample size 292 14.1.4 A minimax paradox 293 14.1.5 Goal sampling 295 14.2 Computing 298 14.3 Examples 302 14.3.1 Point estimation with quadratic loss 302 14.3.2 Composite hypothesis testing 304 14.3.3 A two-action problem with linear utility 306 14.3.4 Lindley information for exponential data 309 14.3.5 Multicenter clinical trials 311 14.4 Exercises 316 15 Stopping 323 15.1 Historical note 324 15.2 A motivating example 326 15.3 Bayesian optimal stopping 328 15.3.1 Notation 328 15.3.2 Bayes sequential procedure 329 15.3.3 Bayes truncated procedure 330 15.4 Examples 332 15.4.1 Hypotheses testing 332 15.4.2 An example with equivalence between sequential and fixed sample size designs 336 15.5 Sequential sampling to reduce uncertainty 337 15.6 The stopping rule principle 339 15.6.1 Stopping rules and the Likelihood Principle 339 15.6.2 Sampling to a foregone conclusion 340 15.7 Exercises 342 Appendix 345 A.1 Notation 345 A.2 Relations 349 A.3 Probability (density) functions of some distributions 350 A.4 Conjugate updating 350 References 353 Index 367 Preface Goals The goal of this book is to give an overview of fundamental ideas and results about rational decision making under uncertainty, highlighting the implications of these results for the philosophy and practice of statistics.