The Negative Aspects of Blinding in Trials

BMJ

Confidential: For Review Only Opening the blinds and shedding light on the risks and problems of blinding in clinical trials

Journal: BMJ

Manuscript ID BMJ-2019-050313

Article Type: Analysis

BMJ Journal: BMJ

Date Submitted by the 18-Apr-2019 Author:

Complete List of Authors: Anand, Rohan; Queen's University Belfast, Wellcome-Wolfson Institute for Experimental Medicine Norrie, John; University of Edinburgh, Usher Institute of Population Health Sciences and Informatics Bradley, Judy; Queen's University Belfast, Wellcome-Wolfson Institute for Experimental Medicine McAuley, Daniel; Queen's University Belfast, Wellcome-Wolfson Institute for Experimental Medicine Clarke, Mike; Queen's University Belfast, Northern Ireland Methodology Hub

blinding, placebos, clinical trials, pragmatic trials, clinical trial Keywords: methodology, RCT, PROBE, bias

https://mc.manuscriptcentral.com/bmj Page 1 of 25 BMJ

1 2 3 1 Full Title: 4 5 6 7 2 Opening the blinds and shedding light on the risks and problems of blinding 8 9 3 10 in clinical trials 11 12 Confidential: For Review Only 13 4 Standfast: Blinding can have negative implications for a clinical trial. 14 15 16 5 Author Positions, Names and Emails: 17 18 19 6 Rohan Anand1, [email protected], (ORCiD: 0000-0002-1957-5336); 20 21 7 Professor John Norrie2, [email protected], 22 23 24 8 Professor Judy M Bradley3, [email protected], (ORCiD: 0000-0002-7423-135X) 25 26 9 Professor Danny F McAuley4, [email protected], (ORCiD: 0000-0002-3283-1947) 27 28 29 10 Professor Mike Clarke5, [email protected], (ORCiD: 0000-0002-2926-7257) 30 31 11 32 33 34 12 Author Affiliations and Addresses: 35 36 13 1 Doctoral Research Student in Clinical Trial Methodology; Wellcome-Wolfson Institute for 37 38 39 14 Experimental Medicine; School of Medicine, Dentistry and Biomedical Sciences; Queen's 40 41 15 University Belfast; Belfast; BT9 7BL; United Kingdom. 42 43 16 44 45 46 17 2 Professor of Medical Statistics and Trial Methodology; Usher Institute of Population Health 47 48 18 Sciences and Informatics; The University of Edinburgh; Edinburgh; EH16 4UX; United 49 50 51 19 Kingdom. 52 53 20 54 55 56 57 58 59 60

Page 1 of 25 18APR2019 https://mc.manuscriptcentral.com/bmj BMJ Page 2 of 25

1 2 3 21 3 Director of the Wellcome Trust-Wolfson Northern Ireland Clinical Research Facility; 4 5 6 22 Wellcome-Wolfson Institute for Experimental Medicine; School of Medicine, Dentistry and 7 8 23 Biomedical Sciences; Queen's University Belfast; Belfast; BT9 7BL; United Kingdom. 9 10 11 24 12 Confidential: For Review Only 13 25 4 Clinical Professor and Consultant in Intensive Care Medicine; Wellcome-Wolfson Institute 14 15 16 26 for Experimental Medicine; School of Medicine, Dentistry and Biomedical Sciences; Queen's 17 18 27 University Belfast; Belfast; BT9 7BL; United Kingdom. 19 20 28 21 22 23 29 5 Director of the Northern Ireland Clinical Trials Unit and the Northern Ireland Methodology 24 25 30 Hub; Centre for Public Health; School of Medicine, Dentistry and Biomedical Sciences; 26 27 28 31 Queen's University Belfast; Belfast; BT12 6BJ; United Kingdom. 29 30 32 31 32 33 33 Correspondence to [email protected] (ORCiD: 0000-0002-2926-7257) 34 35 34 36 37 38 35 Contributors and sources: 39 40 41 36 The objective of this paper is to highlight the potential harm of blinding and placebos when 42 43 37 used in clinical trials. As blinding is such an established and prominent feature of past and 44 45 46 38 current trials, there is a need for a comprehensive paper that describes the potential 47 48 39 negative consequences of using it in some trials. The authors have combined extensive 49 50 40 51 experience in the design, conduct, management and analysis of clinical trials both from a 52 53 41 clinical and methodological aspect. Professor Mike Clarke is the Director of the Northern 54 55 42 Ireland Clinical Trials Unit and the Northern Ireland Methodology Hub, and Co-ordinating 56 57 58 43 Editor of the Cochrane Methodology Review Group, and has over 30 years’ experience in 59 60 44 trials and systematic reviews. John Norrie is Professor of Medical Statistics and Trial

Page 2 of 25 18APR2019 https://mc.manuscriptcentral.com/bmj Page 3 of 25 BMJ

1 2 3 45 Methodology and is Director of Edinburgh Clinical Trials Unit. Professor Judy Bradley is a 4 5 6 46 physiotherapist and Director of the Wellcome Trust-Wolfson Northern Ireland Clinical 7 8 47 Research Facility and Co-Lead of the Northern Ireland Clinical Research Network for 9 10 11 48 Respiratory Health. Professor Danny McAuley is a Consultant in Intensive Care Medicine at 12 Confidential: For Review Only 13 49 the Royal Victoria Hospital Belfast, Director of the MRC/NIHR Efficacy and Mechanism 14 15 16 50 Evaluation (EME) Programme and Co-Director of Research for the Intensive Care Society. 17 18 51 Rohan Anand is a PhD candidate exploring how trial methods, such as the use of placebos, 19 20 52 can affect the outcomes. 21 22 23 53 24 25 54 26 27 28 55 Contributor and guarantor information: 29 30 56 The corresponding author attests that all listed authors meet authorship criteria and that no 31 32 33 57 others meeting the criteria have been omitted. All authors contributed to conceptualisation 34 35 58 and writing of the paper. Rohan Anand prepared the original draft. Mike Clarke is guarantor. 36 37 38 59 39 40 60 The Corresponding Author has the right to grant on behalf of all authors and does grant on 41 42 61 behalf of all authors, a worldwide licence to the Publishers and its licensees in perpetuity, in 43 44 45 62 all forms, formats and media (whether known now or created in the future), to i) publish, 46 47 63 reproduce, distribute, display and store the Contribution, ii) translate the Contribution into 48 49 50 64 other languages, create adaptations, reprints, include within collections and create 51 52 65 summaries, extracts and/or, abstracts of the Contribution, iii) create any other derivative 53 54 55 66 work(s) based on the Contribution, iv) to exploit all subsidiary rights in the Contribution, v) 56 57 67 the inclusion of electronic links from the Contribution to third party material where-ever it 58 59 68 60 may be located; and, vi) licence any third party to do any or all of the above.

Page 3 of 25 18APR2019 https://mc.manuscriptcentral.com/bmj BMJ Page 4 of 25

1 2 3 69 4 5 6 70 7 8 71 Competing interests declaration: 9 10 11 72 All authors have read and understood the BMJ Group policy on declaration of interests and 12 Confidential: For Review Only 13 73 declare no conflicts of interest. 14 15 16 74 17 18 75 Patient and Public Involvement: 19 20 76 It was not appropriate or possible to involve patients or the public in this work 21 22 23 77 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Page 4 of 25 18APR2019 https://mc.manuscriptcentral.com/bmj Page 5 of 25 BMJ

1 2 3 4 5 78 ABSTRACT 6 7 8 79 Controlled trials with blinding of patients and researchers to interventions are complex to 9 10 80 implement. There is debate about the actual impact that blinding and the commonly 11 12 Confidential: For Review Only 13 81 associated placebo can have on the results of trials. Blinding is not always possible or may 14 15 82 be challenging to setup and implement. Here, we highlight problems associated with the 16 17 18 83 practicality, the potential harm to patients and the applicability of a trial’s results, using 19 20 84 both real world and hypothetical examples. There are many instances where blinding might 21 22 23 85 actually damage the quality and reliability of a study, which is especially true in 24 25 86 effectiveness trials. Adequate randomisation, blinding of outcome assessment and use of 26 27 28 87 objective outcomes should reduce the main causes of bias which blinding of patients and 29 30 88 practitioners is intended to limit. This, coupled with doubts about whether the results 31 32 89 obtained when blinding is used are a good estimate of what will happen in routine practice, 33 34 35 90 suggests that it might be better to reduce the emphasis on patient and participant blinding, 36 37 91 especially in pragmatic randomised trials exploring effectiveness. 38 39 40 92 41 42 93 43 44 45 94 46 47 48 49 95 INTRODUCTION 50 51 52 96 Blinding (also referred to as masking) in clinical trials is used with the intention of reducing 53 54 97 various forms of bias, in the hope that this will increase the reliability of the trial’s results. 55 56 th 57 98 First documented in the 18 century (1), the process of blinding is essentially the 58 59 99 withholding of information from people involved in the trial relating to the interventions 60

Page 5 of 25 18APR2019 https://mc.manuscriptcentral.com/bmj BMJ Page 6 of 25

1 2 3 100 that are being compared. Double or triple blind trials are usually regarded as the “gold 4 5 6 101 standard” of clinical research and evidence (2, 3). However, its illustrious reputation means 7 8 102 there is a danger that blinding is held to be an unreasonable prerequisite for a “good” 9 10 11 103 clinical trial, without a sound rationale for using it. Furthermore, if trials without blinding are 12 Confidential: For Review Only 13 104 inappropriately judged to be of lower quality or value than blinded trials, opportunities to 14 15 16 105 use robust data from unblinded trials to improve health care will be missed. 17 18 19 106 20 21 22 107 In this analysis we present a counter-view on the benefits of blinding to stimulate thinking 23 24 108 and debate. We describe weaknesses and adverse consequences of blinding associated with 25 26 27 109 patient safety and the applicability of a trial’s results to routine practice. There are cases 28 29 110 where attempts at blinding would be inappropriate or damage the quality and reliability of a 30 31 32 111 study, especially in pragmatic effectiveness trials. 33 34 35 112 36 37 38 113 The purpose of blinding 39 40 41 114 The primary purpose of blinding is to cancel out the placebo effect and reduce bias. 42 43 115 Cancelling out the placebo effect identifies the “true effect” of the new treatment, as 44 45 116 46 distinct from any effect arising simply from the patient’s knowledge or expectation of having 47 48 117 the intervention without its “active ingredient”. In placebo-controlled trials, any placebo 49 50 118 effect of the new treatment would be discounted when comparing the intervention and 51 52 53 119 control group to determine the effects of the active properties. Blinding is also used to 54 55 120 reduce bias in which the measured effect is not the true effect (4). Blinding aims to minimise 56 57 58 121 performance bias that can manifest specifically as response and observer bias. Response 59 60 122 bias is the process of responding inaccurately, either intentionally or unintentionally, in a

Page 6 of 25 18APR2019 https://mc.manuscriptcentral.com/bmj Page 7 of 25 BMJ

1 2 3 123 way that systematically differs from what should be the true response. A simple example is 4 5 6 124 patients providing answers that they think the researcher would like to hear. Observer bias 7 8 125 occurs when researchers assessing the effects of the interventions being studied have 9 10 11 126 presumptions about the research aim or the interventions. They might then introduce 12 Confidential: For Review Only 13 127 biases by inaccurately measuring outcomes and lead to estimates of effect that differ from 14 15 16 128 the true effect. Again this can happen either intentionally or unintentionally, an example 17 18 129 being how previous positive experience with a drug might lead them to exaggerate the 19 20 130 benefits. Another form of bias that blinding might help to minimise is co-intervention bias, 21 22 23 131 in which trial or non-trial interventions may be taken differently if patients know what they 24 25 132 have been allocated in the trial. 26 27 28 29 133 30 31 134 Empirical evidence showing the impact of blinding and its ability to cancel out the placebo 32 33 34 135 effect and reduce bias is highlighted from comparisons between selected trials (5) and from 35 36 136 systematic reviews of methodology research (6-10). 37 38 39 137 40 41 42 43 138 THE NEGATIVE ASPECTS OF BLINDING IN TRIALS 44 45 46 139 47 48 140 49 Practical negative implications of blinding and placebos 50 51 141 Recruitment, retention and resentful demoralisation: 52 53 54 142 The significant challenges in recruitment and retention in clinical trials have been 55 56 143 highlighted as priorities for methodology research (11, 12). Poor recruitment leads to 57 58 59 144 prolonged study timelines, underpowered results and even failure to reach completion. 60

Page 7 of 25 18APR2019 https://mc.manuscriptcentral.com/bmj BMJ Page 8 of 25

1 2 3 145 Research has shown that matters are made worse by blinding. Trials with nested 4 5 6 146 components that were blinded and open found that the blinded design discouraged 7 8 147 participant recruitment (13-15). Key reasons given by patients for not wanting to enrol in 9 10 11 148 these trials were that they wanted a named medication or wanted to know what was in the 12 Confidential: For Review Only 13 149 tablets. This suggests that achieving blinding and using a non-active comparator, such as a 14 15 16 150 placebo, discourages people from joining a trial. A study exploring recruitment to a 17 18 151 prospective hypertension trial found a similar result, with approximately 25% of a total of 19 20 152 216 patients expressing concerns about receiving placebos (16). 21 22 23 153 24 25 154 Successful retention of patients is equally important to recruitment (17) and the use of a 26 27 28 155 placebo might be damaging, especially if patients who suspect that they have been 29 30 156 allocated to receive it become discouraged and withdraw from the study. For instance, a 31 32 33 157 meta-analysis of retention in trials of antipsychotic interventions concluded that a placebo- 34 35 158 controlled design significantly increased dropout (18). The issue of patient preference or 36 37 38 159 resentful demoralisation can occur if patients in the placebo group become resentful upon 39 40 160 suspicion or discovery of not receiving an active treatment. This can lead to behavioural 41 42 161 changes that could result in bias via differential loss to follow up between treatment groups 43 44 45 162 (19). Patients can feel frustrated because they believe they are receiving inadequate 46 47 163 treatment and so exaggerate negative answers on questionnaires or even withdraw from 48 49 50 164 the trial (20). 51 52 53 165 54 55 56 57 58 59 60

Page 8 of 25 18APR2019 https://mc.manuscriptcentral.com/bmj Page 9 of 25 BMJ

1 2 3 166 Production of the placebo, tell-tale side effects and patients’ desires: 4 5 6 167 If a placebo is used, there are specific issues relating to its production and packaging. Not 7 8 168 only does it need to look identical to the intervention, any characteristic taste, texture, 9 10 11 169 smell, colour or viscosity of the intervention will need to be replicated in the placebo. Given 12 Confidential: For Review Only 13 170 these requirements, it is no surprise that the financial cost can be high (21). Lost 14 15 16 171 opportunity costs can also occur if funding spent on blinding cannot be spent on optimising 17 18 172 other features that help more with the trial’s robustness, such as the training of trial staff, 19 20 173 boosting the sample size and comprehensively measuring outcomes (22). Even if the 21 22 23 174 placebo is designed to be physically identical to the intervention, any signature and 24 25 175 revealing side effects associated with the intervention may lead to unblinding (23). 26 27 28 176 Examples come from the IMOP (24) and IMAGES studies (25), which both had high 29 30 177 occurrences of side effects in the intervention arms. Even if no formal unblinding of 31 32 33 178 researchers and patients occurred, those involved in the trial may have had a good idea 34 35 179 about the groups that patients were in. In addition to this passive association, those 36 37 38 180 involved with the trial might actively look for signs that they believe to be linked to the 39 40 181 interventions. The origins of the online community group PatientsLikeMe highlight this (26, 41 42 182 27). Patients who had been enrolled in blinded clinical trials congregated their outcomes, 43 44 45 183 including side effects, on online platforms outside of the official protocol or any trial 46 47 184 regulations, even before the trial’s completion (28). Their aim was to help each other 48 49 50 185 deduce their allocated intervention, showing their frustration in the placebo-blinded 51 52 186 approach. Researchers have also been shown to break blinding by comparing pills and 53 54 55 187 searching through the restricted notes of patients (29). 56 57 188 58 59 60

Page 9 of 25 18APR2019 https://mc.manuscriptcentral.com/bmj BMJ Page 10 of 25

1 2 3 189 Additional problems that can arise due to blinding in trials relate to emergency unblinding 4 5 6 190 and testing the actual success of blinding. These are briefly described in Table 1. 7 8 191 9 10 11 192 Table 1: Problems associated with emergency unblinding and testing for the success of 12 Confidential: For Review Only 13 193 blinding. 14 15 16 Emergency unblinding: 17 18 Should a clinical need arise for unblinding, there is the potential that an individual 19 20 21 unblinding can cascade and unblind others in the trial. A simple example would be an 22 23 adverse event needing treatment that is reported by blinded trial staff who then code 24 25 break to identify which intervention the patient received. Although the trial staff are 26 27 28 officially unblinded to just this single case, they might now associate this event or related 29 30 symptoms with the specific intervention. Even worse, if all the interventions had been 31 32 33 coded in the same way (such as “Drug A” and “Drug B”) those who unblind themselves to 34 35 one patient, effectively unblind themselves to all patients. Even in the absence of such 36 37 38 coding, unblinding of patients in a trial that used blocked randomisation might reveal the 39 40 allocations of patients from the same block or strata (30). 41 42 43 Testing for blinding: 44 45 Testing for the success of blinding in trials has been reported in approximately 2% of trials 46 47 (31), usually by asking those blinded to guess treatment allocation (32-35). In theory, any 48 49 50 significant difference over chance suggests that blinding was compromised. However, 51 52 measuring blinding is highly challenging. Asking people to say which treatment was 53 54 55 allocated after outcomes have been accumulated makes them likely to base their answer 56 57 on assumptions related to the effects of the intervention. This was observed in a 2x2 58 59 60

Page 10 of 25 18APR2019 https://mc.manuscriptcentral.com/bmj Page 11 of 25 BMJ

1 2 3 factorial trial of aspirin and sulfinpyrazone for stroke prevention where blinded clinicians 4 5 6 were asked to guess treatment groups and did significantly worse than chance (36). It 7 8 appears that their guesses were influenced by their prior assumptions that sulfinpyrazone 9 10 11 was more effective than aspirin and that patients who did well must have been on 12 Confidential: For Review Only 13 sulfinpyrazone, when in fact the trial showed the opposite (37). This confounds testing for 14 15 16 the success of blinding with expectations about treatment efficacy. 17 18 194 19 20 21 22 195 Blinding and the risks to patient safety: 23 24 196 If blinding might compromise patient safety, it is paramount to consider whether it is 25 26 197 necessary. Safety issues can be illustrated by various recent trials. For example, a double 27 28 29 198 blind, placebo-controlled trial of fibrinogen for postpartum haemorrhage required a 30 31 199 moratorium on the use of any new treatments for 15 minutes after the randomly allocated 32 33 200 treatment was given, with the sole purpose of maintaining the blind (38). Similarly, there 34 35 36 201 are difficulties in making dose adjustments in blinded trials were a fixed dose would 37 38 202 compromise patient safety. Such dose adjustments are needed for trials of medications with 39 40 41 203 a narrow and volatile therapeutic range in order to provide safe and optimal treatment. For 42 43 204 example, clinical trials with anticoagulants (39) and antipsychotics have been historically 44 45 46 205 difficult to blind due to dose adjustments (23). 47 48 206 49 50 51 207 If a placebo or other sham therapy is used it might lead to adverse effects that would not 52 53 208 otherwise have happened if an open control group had been used. These could be direct 54 55 209 harms from the procedures intended to ensure blinding, such as infection from piercing the 56 57 58 210 skin to give a placebo injection or muscular problems from sham physiotherapy. 59 60 211

Page 11 of 25 18APR2019 https://mc.manuscriptcentral.com/bmj BMJ Page 12 of 25

1 2 3 212 The nocebo effect is when patients experience more of a negative effect than they should 4 5 6 213 due to negative expectations such as those regarding side effects (40). It is essentially the 7 8 214 opposite of the placebo effect. A systematic review found high rates of such events in 9 10 11 215 placebo groups of anti-migraine trials, including anorexia, dizziness and memory difficulties; 12 Confidential: For Review Only 13 216 some of which led to patient withdrawals (41). Placebo controls can also induce nocebo 14 15 16 217 effects in invasive surgical procedures with a systematic review of surgical trials finding that 17 18 218 adverse events were likely to be associated with the placebo in 9 of the 53 trials. Two trials 19 20 219 within the review demonstrated harms directly related to the placebo (42). Furthermore, 21 22 23 220 the evidence for the argument that placebos have a positive therapeutic effect is limited. A 24 25 221 systematic review comparing placebo to no treatment in 114 trials across 40 clinical 26 27 28 222 conditions found poor evidence of the placebo having any significant clinical effect on 29 30 223 outcomes (43). 31 32 33 224 34 35 225 In considering these concerns about patient safety, Franklin G. Miller outlined three key 36 37 38 226 questions that might help when deciding whether to use placebos in surgical trials (44, 45). 39 40 227 It seems reasonable to apply these questions to trials considering placebo-blinds in all 41 42 228 clinical areas: 43 44 45 229 1. Is blinding or a placebo needed for a scientifically sound result? 46 47 230 2. Are the potential harms to participants excessive? 48 49 50 231 3. Can the anticipated social value of the study results justify these potential harms? 51 52 232 53 54 55 233 These questions are entirely context dependent and are essentially determined by those 56 57 234 planning the trial, but if the answer to one or more of these questions is “no”, a blinded 58 59 235 60 placebo-controlled trial might not be appropriate.

Page 12 of 25 18APR2019 https://mc.manuscriptcentral.com/bmj Page 13 of 25 BMJ

1 2 3 236 4 5 6 237 7 8 9 238 Pragmatism and what happens in the real world 10 11 239 At its simplest, a randomised trial is a means to obtain as unbiased an estimate as possible 12 Confidential: For Review Only 13 14 240 of how different the outcomes observed for patients in the treatment group are to what 15 16 241 would have happened if they had been in the control group. Beyond this, the ultimate aim is 17 18 19 242 to generate evidence that can be used to make assumptions about what will happen to 20 21 243 patients in the future who receive either the treatment or the control after the trial. 22 23 244 Blinding might help with reducing bias but hamper the evidence generated. Minimising 24 25 26 245 biases with blinding might weaken the ability to predict the future accurately, because 27 28 246 blinding itself is unlikely to be used for future patients in routine practice. Some of the types 29 30 31 247 of blinding that would only be contemplated in a research setting are inconsistent with the 32 33 248 desire for pragmatism in large, phase III pragmatic effectiveness trials. Pragmatic trials strive 34 35 36 249 to generate situations that are as close as possible to routine practice, when patients and 37 38 250 practitioners will not be blinded to the intervention. Outside of the trial setting the 39 40 41 251 intervention is known and this will have a legitimate impact on behaviour including use of 42 43 252 co-interventions, concerns about side effects, and decisions about continuing or stopping 44 45 253 the therapy. Some interventions will be marketed for over the counter and prescription use, 46 47 48 254 and both patients and clinicians will be susceptible to brand psychology, meaning choices 49 50 255 will be determined by facets surrounding brand loyalty (46). Clinicians might pay particular 51 52 53 256 attention to assessing patients for side effects and act if they observe them. Both patients 54 55 257 and clinicians might choose to continue with a therapy they believe to be active and 56 57 58 258 beneficial and stop taking therapies they believe to have completed their action, or switch 59 60

Page 13 of 25 18APR2019 https://mc.manuscriptcentral.com/bmj BMJ Page 14 of 25

1 2 3 259 from those that do not seem to be working. Hypothetical examples highlighting these 4 5 6 260 concerns are presented in Table 2. 7 8 9 261 Table 2: General examples of clinical trial research questions where blinding and placebos 10 11 262 would damage pragmatism. 12 Confidential: For Review Only 13 14 15 1. Does a cream reduce facial acne? 16 17 The uncertainty faced by someone looking at the array of acne creams in a 18 19 pharmacy is probably not “should I use one of these creams or their base material?”, 20 21 22 but “should I use one of these creams or not?” A double blind trial of two creams to 23 24 see which would cause a greater reduction on acne, with independent blinded 25 26 27 outcome assessment, would likely determine whether the active ingredients really 28 29 are active. But it will not test what would happen if the patient uses the cream 30 31 32 rather than taking a completely different approach to treating their skin. The trial 33 34 would not account for a patient’s or clinician’s perception on how the creams are 35 36 branded and marketed and their subsequent behaviour. The biases related to 37 38 39 psychological attachment to a brand, combined with the patient’s own self- 40 41 assessment of their acne, could be very different to the result of the blinded trial. In 42 43 44 addition, even if a placebo was used, it might be difficult to match it to the 45 46 intervention with regards to texture, viscosity, smell and colour. 47 48 49 50 51 2. Does cognitive behaviour therapy delivered by a highly skilled and experienced 52 53 54 therapist improve the quit rates of smokers? 55 56 Whenever the effects of personally delivered therapies as diverse as psychotherapy, 57 58 teaching and surgery are assessed, we might wish to compare whether there are 59 60

Page 14 of 25 18APR2019 https://mc.manuscriptcentral.com/bmj Page 15 of 25 BMJ

1 2 3 differences between those delivered by practitioners with high levels of training or 4 5 6 experience and those delivered by relative novices. This might have important 7 8 implications for the costs of the therapy or for the rate at which patients can receive 9 10 11 it and waiting lists can be shortened. A blinded trial in which the patient is not 12 Confidential: For Review Only 13 informed of the therapist’s skill and experience would remove the effect of this 14 15 16 knowledge, but in routine practice they will be aware of this information and it may 17 18 even influence their decision to seek out a particular therapist. Having access to the 19 20 information might bias the patient by having a positive effect of helping them to 21 22 23 benefit from the therapy, or, conversely, it might have a negative effect by raising 24 25 their expectations of benefit which, if not met, could worsen their outcomes. These 26 27 28 biases are real and should be part of the pragmatic trial. This challenge also raises 29 30 the difficult issue of whether therapies that the patient would pay for outside of a 31 32 33 trial should be paid for by them in the trial, as the outcome in the real-world setting 34 35 could be influenced by the costs of the two procedures available, regardless of the 36 37 38 proven efficacy of each intervention. 39 40 3. Does physiotherapy airway clearance reduce acute exacerbations in bronchiectasis? 41 42 43 Clinical trials of airway clearance are difficult to fully blind due to the physical and 44 45 complex nature of the intervention. It would be hard to define exactly what the 46 47 sham physiotherapy consists of and how to implement it if was to be used as a 48 49 50 placebo control. The trial would also be at a risk of unblinding if patients that are 51 52 well experienced with functioning of the active airway clearance intervention were 53 54 55 allocated to the placebo arm. Physiotherapists would likely be aware when 56 57 implementing a sham procedure and so further weakening the blind. The results of 58 59 60

Page 15 of 25 18APR2019 https://mc.manuscriptcentral.com/bmj BMJ Page 16 of 25

1 2 3 such a fully blinded trial would be unlikely to translate into the real world because 4 5 6 biases surrounding clinician preference and co-intervention bias are neglected. Even 7 8 if patients were blinded, the physiotherapists may not be confident in applying a 9 10 11 sham procedure. Similar problems would arise in a trial exploring a physiotherapy 12 Confidential: For Review Only 13 airway clearance regime versus a pharmacologic agent because, even with outcome 14 15 16 assessors blinded, it raises additional issues with the practicality around the 17 18 communication between patients and staff. For example, the scheduling of patient 19 20 visits would be complex as medications could not be seen in the possession of 21 22 23 patients. A more pragmatic approach would be to blind external outcome assessors 24 25 or to use an objective primary outcome for the trial along with a blinded end-point 26 27 28 committee. 29 30 263 31 32 33 34 264 METHODS THAT INCREASE TRIAL INTEGRITY 35 36 37 265 38 39 40 266 Prospective Randomised Open Blinded Evaluation of Outcomes 41 42 43 267 Techniques such as randomisation with secure concealment in advance of the allocation 44 45 268 being revealed (47) and blinded outcome assessment and adjudication are important. These 46 47 48 269 techniques essentially form the Prospective Randomised Open Blinded End-point evaluation 49 50 270 (PROBE) methodology for trials (48, 49), which allows for lower trial costs and results more 51 52 53 271 applicable to routine practice whilst balancing for potential confounders. This methodology 54 55 272 focuses on blinding the evaluation of defined endpoints during a trial. PROBE, along with the 56 57 273 58 blinding of outcome assessors, can be implemented in most trials and especially in 59 60 274 pragmatic effectiveness trials in which outcomes are either subjective or objective.

Page 16 of 25 18APR2019 https://mc.manuscriptcentral.com/bmj Page 17 of 25 BMJ

1 2 3 275 Outcome assessor blinding throughout the trial or the blinded evaluation of end-points by a 4 5 6 276 committee at set points, reduces the impact of observer and response bias, which can be a 7 8 277 significant cause of differences reported between treatments. Such methodology would 9 10 11 278 increase rigor when blinding of patients and healthcare providers is dropped, might be 12 Confidential: For Review Only 13 279 simpler to deliver and can directly avoid the many challenges outlined in this paper. 14 15 16 280 17 18 281 Objective outcomes 19 20 21 282 In instances that blinded outcome assessment cannot be used in a trial, it is preferable to 22 23 283 use objective rather than subjective outcomes, to avoid bias. This is supported by a large 24 25 26 284 meta-epidemiological study that found little evidence of bias in unblinded trials that used 27 28 285 objective outcomes for both drug and non-drug interventions (50). Another option to 29 30 31 286 reduce bias is to modify the outcome to make it less subjective. This can include avoiding 32 33 287 surrogate markers and limiting the change to a given clinical parameter by limiting the 34 35 36 288 assessment of effect (51). 37 38 289 39 40 41 42 290 CONCLUSION 43 44 45 291 Overall, blinding in clinical trials has been proposed as a means for increasing the reliability 46 47 48 292 of a trial’s results. However, there are significant issues with its implementation. As we 49 50 293 showcase in this paper, the use of blinding and any associated placebos has consequences 51 52 53 294 for the practicality, safety and results of some trials. We suggest that the key elements for 54 55 295 clinical trials seeking to minimise bias when comparing the effects of interventions should 56 57 296 be adequate randomisation, allocation concealment, use of objective outcomes, blinded 58 59 60 297 outcome evaluation and, when possible, blinded outcome assessment. The blinding of

Page 17 of 25 18APR2019 https://mc.manuscriptcentral.com/bmj BMJ Page 18 of 25

1 2 3 298 participants and practitioners should be a secondary issue and should not be regarded as a 4 5 6 299 gold standard to strive for and should only be used if the negative impacts are considered 7 8 300 and are rationally outweighed by positive benefits. 9 10 11 301 12 Confidential: For Review Only 13 14 15 302 KEY MESSAGES BOX 16 17 18 303  Blinding is a feature used in clinical trials to avoid forms of bias. However, there are 19 20 304 potential harms associated with blinding and placebos that could be detrimental to 21 22 23 305 the integrity of some trials. 24 25 306  Potential damaging effects of blinding relate to the practicality of a trial, recruitment, 26 27 28 307 retention, potential harm to patients and the applicability of the results to routine 29 30 308 practice. 31 32 33 309  Blinding of patients and clinicians is not always the ideal setup and those conducting 34 35 310 trials should rationalise the need for blinding and placebos. 36 37  38 311 Adequate randomisation, using objective outcomes, blinded outcome assessment 39 40 312 and independent blinded adjudication of outcomes when possible should reduce the 41 42 313 main forms of bias in a clinical trial. 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Page 18 of 25 18APR2019 https://mc.manuscriptcentral.com/bmj Page 19 of 25 BMJ

1 2 3 4 314 REFERENCES 5 6 315 7 8 316 1. Best M, Neuhauser D, Slavin L. Evaluating Mesmerism, Paris, 1784: the controversy over 9 10 317 the blinded placebo controlled trials has not stopped. Quality & safety in health care. 11 12 Confidential: For Review Only 13 318 2003;12(3):232-3. 14 15 16 17 319 2. Jones DS, Podolsky SH. The history and fate of the gold standard. Lancet. 18 19 320 2015;385(9977):1502-3. 20 21 22 23 321 3. Bothwell LE, Greene JA, Podolsky SH, Jones DS. Assessing the Gold Standard — Lessons 24 25 322 from the History of RCTs. N Engl J Med. 2016;374(22):2175-81. 26 27 28 29 323 4. Tavel ME. The Placebo Effect: the Good, the Bad, and the Ugly. Am J Med. 30 31 324 2014;127(6):484-8. 32 33 34 35 325 5. Curfman G. Rigor in Biomedical Science. In: Blinding as a Solution to Bias. San Diego: 36 37 326 Academic Press; 2017. p. 41-3. 38 39 40 41 327 6. Hróbjartsson A, Thomsen ASS, Emanuelsson F, Tendal B, Hilden J, Boutron I, et al. 42 43 328 Observer bias in randomised clinical trials with binary outcomes: systematic review of trials 44 45 46 329 with both blinded and non-blinded outcome assessors. BMJ. 2012;344:e1119. 47 48 49 330 7. Hróbjartsson A, Thomsen ASS, Emanuelsson F, Tendal B, Hilden J, Boutron I, et al. 50 51 52 331 Observer bias in randomized clinical trials with measurement scale outcomes: a systematic 53 54 332 review of trials with both blinded and nonblinded assessors. CMAJ. 2013;185(4):E211. 55 56 57 58 333 8. Hróbjartsson A, Thomsen ASS, Emanuelsson F, Tendal B, Rasmussen JV, Hilden J, et al. 59 60 334 Observer bias in randomized clinical trials with time-to-event outcomes: systematic review

Page 19 of 25 18APR2019 https://mc.manuscriptcentral.com/bmj BMJ Page 20 of 25

1 2 3 335 of trials with both blinded and non-blinded outcome assessors. Int J Epidemiol. 4 5 6 336 2014;43(3):937-48. 7 8 9 337 9. Hrobjartsson A, Emanuelsson F, Thomsen ASS, Hilden J, Brorson S. Bias due to lack of 10 11 12 338 patient blindingConfidential: in clinical trials. A systematic For review Review of trials randomizing Only patients to blind 13 14 339 and nonblind sub-studies. Int J Epidemiol. 2014;43(4):1272-83. 15 16 17 18 340 10. Savović J, Jones HE, Altman DG, Harris RJ, Jüni P, Pildal J, et al. Influence of reported 19 20 341 study design characteristics on intervention effect estimates from randomized, controlled 21 22 23 342 trials. Ann Intern Med. 2012;157(6):429-38. 24 25 26 343 11. Bower P, Brueton V, Gamble C, Treweek S, Smith CT, Young B, et al. Interventions to 27 28 29 344 improve recruitment and retention in clinical trials: a survey and workshop to assess current 30 31 345 practice and future priorities. Trials. 2014;15:399. 32 33 34 35 346 12. Kearney A, Daykin A, Shaw ARG, Lane AJ, Blazeby JM, Clarke M, et al. Identifying 36 37 347 research priorities for effective retention strategies in clinical trials. Trials. 2017;18(1):406. 38 39 40 41 348 13. Treweek S, Pitkethly M, Cook J, Fraser C, Mitchell E, Sullivan F, et al. Strategies to 42 43 349 improve recruitment to randomised trials. Cochrane Database Syst Rev. 2018(2):MR000013. 44 45 46 47 350 14. Hemminki E, Hovi S, Veerus P, Sevón T, Tuimala R, Rahu M, et al. Blinding decreased 48 49 351 recruitment in a prevention trial of postmenopausal hormone therapy. J Clin Epidemiol. 50 51 52 352 2004;57(12):1237-43. 53 54 55 353 15. Avenell A, Grant AM, McGee M, McPherson G, Campbell MK, McGee MA. The effects of 56 57 58 354 an open design on trial participant recruitment, compliance and retention--a randomized 59 60

Page 20 of 25 18APR2019 https://mc.manuscriptcentral.com/bmj Page 21 of 25 BMJ

1 2 3 355 controlled trial comparison with a blinded, placebo-controlled design. Clin Trials. 4 5 6 356 2004;1(6):490-8. 7 8 9 357 16. Halpern SD, Karlawish JHT, Casarett D, Berlin JA, Townsend RR, Asch DA. Hypertensive 10 11 12 358 patients'Confidential: willingness to participate in placebo-controlled For Review trials: implications Only for recruitment 13 14 359 efficiency. Am Heart J. 3 00;146(6):985-92. 15 16 17 18 360 17. Daykin A, Clement C, Gamble C, Kearney A, Blazeby J, Clarke M, et al. ‘Recruitment, 19 20 361 recruitment, recruitment’ – the need for more focus on retention: a qualitative study of five 21 22 23 362 trials. Trials. 2018;19(1):76. 24 25 26 363 18. Kemmler G, Hummer M, Widschwendter C, Fleischhacker WW. Dropout Rates in 27 28 29 364 Placebo-Controlled and Active-Control Clinical Trials of Antipsychotic Drugs: A Meta- 30 31 365 analysis. Arch Gen Psychiatry. 2005;62(12):1305-12. 32 33 34 35 366 19. Torgerson D, Sibbald B. Understanding controlled trials: What is a patient preference 36 37 367 trial? BMJ. 1998;316(7128):360. 38 39 40 41 368 20. Onghena P. Resentful Demoralization. In: Encyclopedia of Statistics in Behavioral 42 43 369 Science. American Cancer Society; 2005. 44 45 46 47 370 21. Christensen M, Knop FK. The unobtainable placebo: control of independent clinical 48 49 371 research by industry? Lancet. 2012;379(9810):30. 50 51 52 53 372 22. Williamson PR, Altman DG, Bagley H, Barnes KL, Blazeby JM, Brookes ST, et al. The 54 55 373 COMET Handbook: version 1.0. Trials. 2017;18(3):280. 56 57 58 59 60

Page 21 of 25 18APR2019 https://mc.manuscriptcentral.com/bmj BMJ Page 22 of 25

1 2 3 374 23. Leucht S, Heres S, Hamann J, Kane JM. Methodological Issues in Current Antipsychotic 4 5 6 375 Drug Trials. Schizophrenia Bulletin. 2008;34(2):275-85. 7 8 9 376 24. Bollapragada SS, MacKenzie F, Norrie JD, Eddama O, Petrou S, Reid M, et al. Randomised 10 11 12 377 placebo-controlledConfidential: trial of outpatient (at For home) cervicalReview ripening with Only isosorbide 13 14 378 mononitrate (IMN) prior to induction of labour clinical trial with analyses of efficacy and 15 16 17 379 acceptability. The IMOP Study. BJOG. 2009;116(9):1185-95. 18 19 20 380 25. Muir KW, Lees KR, Ford I, Davis S. Magnesium for acute stroke (Intravenous Magnesium 21 22 23 381 Efficacy in Stroke trial): randomised controlled trial. Lancet. 2004;363(9407):439-45. 24 25 26 382 26. Wicks P. Chapter 6 - Clinical Trial Blinding in the Age of Social Media. In: Blinding as a 27 28 29 383 Solution to Bias. San Diego: Academic Press; 2016. p. 97-106. 30 31 32 384 27. Wicks P, Massagli M, Frost J, Brownstein C, Okun S, Vaughan T, et al. Sharing health data 33 34 35 385 for better outcomes on PatientsLikeMe. J Med Internet Res. 2010;12(2):e19. 36 37 38 386 28. Wicks P, Vaughan T, Heywood J. Subjects no more: what happens when trial participants 39 40 41 387 realize they hold the power? BMJ. 2014;348:g368. 42 43 44 45 388 29. Schulz KF. Subverting randomization in controlled trials. JAMA. 1995;274(18):1456-8. 46 47 48 389 30. Ayala N, MacKillop E. Educating Investigators to Understand When to Break the Blind. 49 50 51 390 Applied Clinical Trials. 2001;8(11). 52 53 54 391 31. Hróbjartsson A, Forfang E, Haahr MT, Als-Nielsen B, Brorson S. Blinded trials taken to the 55 56 57 392 test: an analysis of randomized clinical trials that report tests for the success of blinding. Int 58 59 393 J Epidemiol. 2007;36(3):654-63. 60

Page 22 of 25 18APR2019 https://mc.manuscriptcentral.com/bmj Page 23 of 25 BMJ

1 2 3 394 32. Boutron I, Estellat C, Ravaud P. A review of blinding in randomized controlled trials 4 5 6 395 found results inconsistent and questionable. J Clin Epidemiol. 2005;58(12):1220-6. 7 8 9 396 33. Fergusson D, Glass KC, Waring D, Shapiro S. Turning a blind eye: the success of blinding 10 11 12 397 reportedConfidential: in a random sample of randomised, For placebo Review controlled trials. Only BMJ. 13 14 398 2004;328(7437):432. 15 16 17 18 399 34. Bang H, Ni L, Davis CE. Assessment of blinding in clinical trials. Control Clin Trials. 19 20 400 2004;25(2):143-56. 21 22 23 24 401 35. James KE, Bloch DA, Lee KK, Kraemer HC, Fuller RK. An index for assessing blindness in a 25 26 402 multi-centre clinical trial: disulfiram for alcohol cessation--a VA cooperative study. Stat Med. 27 28 29 403 1996;15(13):1421-34. 30 31 32 404 36. The Canadian Cooperative Study Group. A Randomized Trial of Aspirin and 33 34 35 405 Sulfinpyrazone in Threatened Stroke. N Engl J Med. 1978;299(2):53-9. 36 37 38 406 37. Sackett DL. Commentary: Measuring the success of blinding in RCTs: don’t, must, can’t 39 40 41 407 or needn’t? Int J Epidemiol. 2007;36(3):664-5. 42 43 44 45 408 38. Wikkelsø AJ, Edwards HM, Afshari A, Stensballe J, Langhoff-Roos J, Albrechtsen C, et al. 46 47 409 Pre-emptive treatment with fibrinogen concentrate for postpartum haemorrhage: 48 49 410 randomized controlled trial. Br J Anaesth. 2015;114(4):623-33. 50 51 52 53 411 39. Büller HR, Halperin JL, Bounameaux H, Prins, M. [=Martin H. ]. Double-blind studies are 54 55 412 not always optimum for evaluation of a novel therapy: the case of new anticoagulants. J 56 57 58 413 Thromb Haemost. 2008;6(2):227-9. 59 60

Page 23 of 25 18APR2019 https://mc.manuscriptcentral.com/bmj BMJ Page 24 of 25

1 2 3 414 40. Faasse K, Petrie KJ. The nocebo effect: patient expectations and medication side effects. 4 5 6 415 Postgrad Med J. 2013;89(1055):540-6. 7 8 9 416 41. Amanzio M, Corazzini LL, Vase L, Benedetti F. A systematic review of adverse events in 10 11 12 417 placebo Confidential:groups of anti-migraine clinical trials.For Pain Review. 2009;146(3):261-9. Only 13 14 15 418 42. Wartolowska K, Judge A, Hopewell S, Collins GS, Dean BJF, Rombach I, et al. Use of 16 17 18 419 placebo controls in the evaluation of surgery: systematic review. BMJ. 2014;348:g3253. 19 20 21 420 22 43. Hróbjartsson A, Gøtzsche PC. Is the placebo powerless? An analysis of clinical trials 23 24 421 comparing placebo with no treatment. N Engl J Med. 2001;344(21):1594-602. 25 26 27 28 422 44. Miller FG. Chapter 7 - The Ethics of Single-Blind Trials in Biomedicine. In: Blinding as a 29 30 423 Solution to Bias. San Diego: Academic Press; 2016. p. 107-14. 31 32 33 34 424 45. Miller FG. Sham surgery: an ethical analysis. Sci Eng Ethics. 2004;10(1):157-66. 35 36 37 425 46. Costa-Font J, Rudisill C, Tan S. Brand loyalty, patients and limited generic medicines 38 39 40 426 uptake. Health Policy. 2014;116(2-3):224-33. 41 42 43 427 47. Suresh KP. An overview of randomization techniques: An unbiased assessment of 44 45 46 428 outcome in clinical research. J Hum Reprod Sci. 2011;4(1):8-11. 47 48 49 429 48. Ford I, Norrie J. Pragmatic Trials. N Engl J Med. 2016;375(5):454-63. 50 51 52 53 430 49. Hansson L, Hedner T, Dahlöf B. Prospective randomized open blinded end-point (PROBE) 54 55 431 study. A novel design for intervention trials. Prospective Randomized Open Blinded End- 56 57 58 432 Point. Blood Press. 1992;1(2):113-9. 59 60

Page 24 of 25 18APR2019 https://mc.manuscriptcentral.com/bmj Page 25 of 25 BMJ

1 2 3 433 50. Wood L, Egger M, Gluud LL, Schulz KF, Jüni P, Altman DG, et al. Empirical evidence of 4 5 6 434 bias in treatment effect estimates in controlled trials with different interventions and 7 8 435 outcomes: meta-epidemiological study. BMJ. 2008;336(7644):601-5. 9 10 11 12 436 51. KahanConfidential: BC, Cro S, Doré CJ, Bratton DJ, For Rehal S, ReviewMaskell NA, et al. ReducingOnly bias in open- 13 14 437 label trials where blinded outcome assessment is not feasible: strategies from two 15 16 17 438 randomised trials. Trials. 2014;15(1):456. 18 19 20 439 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Page 25 of 25 18APR2019 https://mc.manuscriptcentral.com/bmj