Confidence Distribution Xie and Singh (2013): Confidence distribution, the frequentist distribution estimator of a parameter: A Review Céline Cunen, 15/09/2014 Outline of Article ● Introduction ● The concept of Confidence Distribution (CD) ● A classical Definition and the History of the CD Concept ● A modern definition and interpretation ● Illustrative examples ● Basic parametric examples ● Significant (p-value) functions ● Bootstrap distributions ● Likelihood functions ● Asymptotically third-order accurate confidence distributions ● CD, Bootstrap, Fiducial and Bayesian approaches ● CD-random variable, Bootstrap estimator and fiducial-less interpretation ● CD, fiducial distribution and Belief function ● CD and Bayesian inference ● Inferences using a CD ● Confidence Interval ● Point estimation ● Hypothesis testing ● Optimality (comparison) of CDs ● Combining CDs from independent sources ● Combination of CDs and a unified framework for Meta-Analysis ● Incorporation of Expert opinions in clinical trials ● CD-based new methodologies, examples and applications ● CD-based likelihood caluculations ● Confidence curve ● CD-based simulation methods ● Additional examples and applications of CD-developments ● Summary CD: a sample-dependent distribution that can represent confidence intervals of all levels for a parameter of interest ● CD: a broad concept = covers all approaches that can build confidence intervals at all levels Interpretation ● A distribution on the parameter space ● A Distribution estimator = contains information for many types of inference ● An estimator FOR the parameter of interest, instead of an inherent distribution OF the parameter ● Purely frequentist: – The parameter is a fixed, non-random quantity – 95% CI: the true parameter value will be covered by the CIs 95% of the time Fiducial Inference ● “Fisher's biggest blunder”? ● Fiducial – Faith – A fiducial distribution: describes the level of “faith” attached to different values of the unknown parameter ● Sometimes normalized likelihood function are interpreted as distributions of the parameter – ~ the parameters are both fixed and random quantities ● Interpretation of 95% CI: there is a 95% probability that the parameter lies inside the CI. 2.2 Modern definition ● A function Hn() is called a confidence distribution for a parameter θ if: – R1: Hn() is a cummulative distribution function on the parameter space – R2: at the true parameter value θ = θ0 Hn(θ0) as a function of the sample x follows the uniform distribution U[0,1] ● R2 is important! 2.3.1: Basic parametric examples ● A sample: – With σ² known ● The function satisfies the requirements in the CD-definition – R1: it clearly is a cdf – R2: 2.3.2 p-value functions ● One-sided test H0: θ ≤ b vs H1: θ > b ● P-value function: ● Usually CDs or (aCDs): – Because cdf – And because when b=θ0, H0 is true and p-values are uniformly distributed when H0 is true. 2.3.3 Bootstrap distributions ● True (unknown) parameter ● Original sample, with estimator ● Bootstrap sample: a sample of equal size as the original sample, sampled with replacement from the original sample ● Compute the estimator on each bootstrap samples (get many ) --> the Bootstrap Distribution ● The Bootstrap Distribution is an aCD! 2.3.4 Likelihood functions ● Under some mild conditions: normalized likelihood functions are density functions of asymptotic normal CDs – Method for obtaining CDs from likelihoods ● CD-based inference ≈ likelihood inference 3.1 CD-random Variable, ... ● The CD-random variable: – The CD is not a distribution of θ! ● CD-random variable = Bootstrap estimator, and this is useful because it: – Help understanding CD-inference and develop new methods – Clarifies the interpretation of CDs: CDs are not distributions of θ, so therefore it is not a problem that a transformation g(θ) of θ does not generally lead to a CD for g(θ) 3.2 CD, Fiducial Distribution... ● Methods from fiducial reasoning (and from CDs) are supposed to have good statistical performance in the frequentist sense – CIs should have the exact coverage property – For tests: the actual rate of type I error is equal to the specified level of the test ● Many fiducial distributions are CDs ● Fiducial reasoning: provides a procedure for finding CDs 3.3 CD and Bayesian Inference ● Bayesian credible intervals to not possess the exact coverage property – But asymptotically they can obtain it – Then: posterior distributions are aCDs – Bayes methods can produce CDs! ● Benefits with CBs compared to bayesian methods: – Nuisance parameters 4 Inferences using a CD ● CDs contain information for any type of frequentist inference 4.1 Confidence Intervals ● CDs allows us to construct CIs for all levels of α ● CI constructed from CDs have the exact coverage property: 4.2 Point Estimation ● Median: ● Mean: ● Mode: ● Under some conditions, these are consistent estimators (= converging in probability to the true value) 4.3 Hypothesis Testing ● One-sided test: – The support on C: – Reject H0 if the support on C is less than α – The rejection region corresponds to a level α test – Support = p-value (often) ● Two-sided test – Same story, but with a more complicated rejection region 5 Optimality of CDs ● Can have multiple CDs for the same parameter ● A better CD = a CD more concentrated around the true parameter value 6.1 Combination of CDs ... ● k independent studies, estimate the same parameter of interest θ ● Study i with sample xi and CD Hi(.) ● Propose a general recipe for combining k independent CDs: Combining k p-values - “Fisher's method” ● k p-values from k independent studies ● Under H0: ● Remember that ● So that the teststatistic ● Will be distributed under H0 Example ● 2 studies aim to estimate the μ parameter from a normal model with known sigmas ● Study 1: n1=30, ● Study 2: n2=40, ● With CDs: ● We choose ● And then 6.2 Incorporation of Expert Opinions ● How can one incorporate existing knowledge in an analysis? – Bayesian approaches: prior = existing knowledge – In a frequentist setting: CD! ● CD-approach: – CDe: summarizes the existing information/opinions – CDd: from the data – Combines these two by the methods in the last section ● Advantages: – Easy to implement/ computationally cheap – No need for priors on nuisance parameter! – Avoids “the discrepant posterior phenomenon” 7 New methodologies, Examples... ● Different CD-related methods: – Obtaining likelihood functions from CDs – Presenting CDs: the confidence curve – CD-based simulation methods 8 Summary ● CDs is a broad concept which “contains” many well-known notions and results ● Most types of frequentist inference can be derived from CDs ● Advantages of the CD-approach: – Handles nuisance parameters well – Easy to combine information ● Problems (need of further study): – Multivariate CDs – Cases where it difficult to obtain CDs – Model uncertainty/ diagnosis/ selection .