18-Aug-2016

Dear Dr. Nagendran

Manuscript ID BMJ.2016.034309 entitled "Do randomised trials with very large treatment effects make further trials unnecessary? An empirical assessment." We sent it for external and discussed it at our manuscript committee meeting. We recognise its potential importance and relevance to general medical readers, but I am afraid that we have not yet been able to reach a final decision on it because several important aspects of the work still need clarifying.

We hope very much that you will be willing and able to revise your paper as explained below in the report from the manuscript meeting, so that we will be in a better position to understand your study and decide whether the BMJ is the right journal for it. We are looking forward to reading the revised version and, we hope, reaching a decision.

Yours sincerely,

Georg Roeggla [email protected]

**Report from The BMJ’s manuscript committee meeting**

These comments are an attempt to summarise the discussions at the manuscript meeting. They are not an exact transcript.

Manuscript meeting 18.08.2016

Jose Merino (chair), Angela Wade (stats), Elizabeth Loder, Georg Roggla, Wim Weber, Tiago Villanueva, Jessamy Bagenal, Amy Price

Decision: Ask for revision

The committee was interested in the topic of your research. The following concerns were mentioned:

• Were patients involved in discussion of this paper at outset? • There's a bit about patient recruitment in the PI statement, as if recruitment and involvement were synonymous. • What does this add to the recent JAMA paper (http://jama.jamanetwork.com/article.aspx?articleid=1386610) • The committee agreed with the issue about absolute risks vs relative risks raised by the reviewer Siegerink. • This is interesting and thought-provoking but the committee had concerns about the methods you've used to identify the initial VLE study and the subsequent larger study. It's not until Table 2 that we get a sense of the actual clinical questions and topics that are involved. One of them, we note is rizatriptan (where the initial VLE was upheld by a subsequent trial although it wasn't quite as large). We could not, however, locate which small study they referred to, or which subsequent large rizatriptan trial. However, having this concrete example made us realize that the authors don't take into account the context of a VLE. Riza was a me-too drug and there was every reason to think its effects would be in line with previous triptan results. It doesn't seem quite right to consider it in isolation from the known effects of similar drugs. We question whether a subsequent large trial was needed (or even ethical, if it involved placebo), yet the authors would say it was. • Unless we are interpreting table 2 incorrectly, it seems more VLEs were upheld than refuted. • Could you define what you mean by empirical? It appears in the title and then in the text. Also need to define "agreed with." • We thought this is an interesting read, although a bit abstract.

First, please revise your paper to respond to all of the comments by the reviewers. Their reports are available at the end of this letter, below. Please also respond to the additional comments by the committee.

In your response please provide, point by point, your replies to the comments made by the reviewers and the editors, explaining how you have dealt with them in the paper.

** Comments from the external peer reviewers**

Reviewer: 1

Recommendation:

Comments: Peer review report BMJ.2016.034309

Bob Siegerink, PhD [email protected] / bobsiegerink.com

To the editor and authors I have read with great interest the paper by Nagendran et al with the title “Do randomised trials with very large treatment effects make further trials unnecessary? An empirical assessment”, aka BMJ.2016.034309. The authors ask whether a very large treatment effect (RR >5 or its inverse) in a trial can be seen as an indication of no need for follow up trials. This question, linked to the day-to-day practice that small trials provide an “obvious benefit” and therefore clinical equipoise is not applicable anymore, is a relevant question as it addresses concepts like reproducibility of research, potential source of publication bias as well as introducing new treatments that potentially could harm patients. The authors use data from the Cochrane reviews to which they apply several methods to identify RCT with very large effects and subsequent large trials that either refute or uphold that effect. The results indicate very large effects are not a reliable marker for true clinical benefit. The authors even draw the conclusion that therefore these smaller trials might better be skipped and that a new treatment should be investigated with a large trial at the end. The paper is well written, clear in its approach, results well reported and the topic is both relevant as well as attractive and relevant for readers of the BMJ. Nonetheless, I have several comments and question, which I will provide below in a point-by-point fashion. Please note that I have no conflict of interest with regard to this paper and that I am very happy to relinquish my anonymity in this peer review process. Bob siegerink

// Major comments. 1. relative risks vs absolute risks. All calculations, selections and definitions are based on relative risks. Even though these measures are the most used measured to estimate the effect size of an intervention, they do not convey the true impact of an intervention. For this, a form of absolute measure is needed, which can either be an absolute risk reduction, number needed to treat or otherwise. This is not only relevant to understand what “very large effects” indeed mean in clinical practice, but is also important as the initial VLE trial and subsequent large trial can have similar RR while the NNT is different, and vice versa. The figure below shows a numeric example, in which only the absolute risk and total number of persons in the subsequent trials is changed. IN each of the subsequent trials, either the RR or the ARR is similar to the initial VLE trial even though the other measure is then not the same. This shows that there is a problem in interpretation of the RR between trials when there is no information on the absolute risk. It is not unrealistic to think that the subsequent analyses trial could have a different absolute risk compared to the VLE trial, either due to chance, different inclusion criteria, change in co-treatment over time periods, change in geographic location etc. I am not sure to what extend whether such scaling of the RR indeed hampers the interpretation of the results provided by the authors. Details on the absolute risk in both the VLE trials and subsequent trial (e.g. extra row in table 2) would help the reader to understand the impact of this scaling problem.

<< please refer to pdf for table>>

In line with this question on the absolute risk, where the authors able to ensure that there was sufficient overlap in the patient populations of the initial and subsequent trials (overlapping external validity?) 2. upheld/refute. The definitions of “upheld” and “refute” are unclear to me. The authors describe their two methods on page 10 line 4 to 14. The first approach is clear, but the second approach is unclear, as it speaks of “agreed with the index trial” in which agreed which is not defined. After repeatedly reading the second approach, I think the authors mean that when multiple large trials are present, a meta analyses is performed and the result of this meta analyses is checked based on the same criteria as the first approach (effect in opposite direction or non-statistically significant effect). But if this is the case, does approach 1 not rule out that approach 2 is even possible? The figures provide some indication on the definitions, i.e. that everything that stays significant is upheld. Although understandably when it comes to clear definition, I doubt whether a statistical significant of RR 1.1-2 in a subsequent large trial should indeed be seen as “upheld”. It is quite dependent on the research question (PICO) whether the finding of this second trials should next to statistically significant also be seen as ‘just as clinically relevant” as the finding of the index trial. 3. Selection bias. I wonder whether this analyses suffers from selection bias due to the definition of index trial. As is stated at page 7 line 35 an index trial is a VLE trial followed by at least one large trial. If this is the case selection bias might be introduced through this selection: consider all VLE trials that are possible. It is likely that the characteristics of those who are followed by a large trial are indeed different from those VLE trials who are not followed by large trials – and as such never reach the status of index trial-. Perhaps VLE trials who are followed by a large trial (i.e. index trial) are more “disbelieved” by the medical community, perhaps due subject matter knowledge of problems with the initial VLE trial. VLE trials who are not followed by a large trial are could indeed be more solid in terms of effect and these are, as far as I can judge, not part of this analyses. If this is the case, it is no wonder that VLE in the index trials are indeed a poor predictor of success in a subsequent large trial. There is some information in the paper which makes the it not easier for the reader to understand what is going on, being the reference in the flowchart which reads “index RCT not followed by RCT=2”. Does this mean that there are only 2 VLE trials which are not followed by a large and therefore cannot be coded as index trial or is this referring to the situation where the manual screening process have led to the exclusion of two VLE trials because the computerized algorithm falsely categorized these as index trials? If the former if the case, the described problem of selection bias is negligible, whereas the latter does not provide any information on the size and direction of this potential selection bias. // Minor comments 1. Is there information whether trials where stopped early for “success”? 2. PPV is a relevant and useful concept, but only introduced in the discussion section. Suggestion: Introduction of PPV somewhat earlier in the text might be useful for the reader; provide it with confidence limits and explanation as the use of PPV in this meta science concept might not be directly clear to all readers. 3. Page 13 line 40 ‘one case...’ this is unclear to me, perhaps also because the definitions used are not clear to me. 4. Page 17, last sentence “avoid small trials”: I would be a bit more cautious with this statement, especially of the above described selection bias is present – the word perhaps is in my view not cautious enough. The analyses on small trials with VLE do convey their point, but the reasoning mirrored to all small trials (including those without VLE) is not warranted and therefore the conclusion should not be drawn so general? How about small trials that help to identify those that will not have large enough of an effect to continue that line of research? (i.e. not VLE trials) also, how about trials in the so-called ‘rare-diseases’ with only a couple of hundred patients available for trial participation worldwide? 5. // Editing: 1. Page 18 line 34: new paragraph superfluous? // Tables and figures: Flowchart: It is unclear to me how one can have 45 cochrane reviews and only 44 forect plots. Where the presence of forst plots not needed for the automatic screening process? does the answer lie on page 12 line 24-33? Figures in general: please consider lay out that also is readable when plots are printed in black and white by using different shapes. Table 1: no comments Table 2: interesting table. The comprehensibility would be increased if the definition of “uphold” and “refute” would be repeated at this point. Next to this table, it would be interesting to see the refute table (perhaps as suppl file?)

Additional Questions: Please enter your name: Bob Siegerink

Job Title: group leader

Institution: Center for Stroke research Berlin, Charite Universitatsmedizin Berlin

Reimbursement for attending a symposium?: No

A fee for speaking?: No

A fee for organising education?: No

Funds for research?: No

Funds for a member of staff?: No

Fees for consulting?: No

Have you in the past five years been employed by an organisation that may in any way gain or lose financially from the publication of this paper?: No

Do you hold any stocks or shares in an organisation that may in any way gain or lose financially from the publication of this paper?: No

If you have any competing interests (please see BMJ policy) please declare them here: no conflict of interest

Reviewer: 2

Recommendation:

Comments: In the present paper, the authors performed a systematic review of trials with very large effects, followed by larger trials, to assess whether the former provide enough evidence such that further (bigger) trials are unnecessary.

I am not really surprised by the conclusion but I do think that these "negative" findings are worth publishing. I particularly appreciated the discussion. I still have a couple of comments below.

1. I don't really get the argument why the relative risk was preferred over the odds-ratio (p.7, l 20-23). Actually I think that both outcome measures could have been included in the systematic review (an OR > 5 is a big effect irrespective of the frequency of the outcome.)

2. It would be nice to have a more systemic display of the PPV for different cutoff values, either in a table or in a graphic.

3. The authors note that "there were only rare instances where an initial VLE in a trial from a primary outcome forest plot was followed by a large trial" (p. 16, l. 6-9).

Do the authors think it would be worth it to perform a similar systematic review including more outcome measures, e.g., hazard ratio, odds-ratio > 5 or other cutoff?

4. p.13, l. 2-4, "Very forest plots assessed mortality."

Mortality is more likely assessed using survival analysis techniques

Additional Questions: Please enter your name: Arthur Allignol

Job Title: Statistician

Institution: Ulm University

Reimbursement for attending a symposium?: No

A fee for speaking?: No

A fee for organising education?: No

Funds for research?: Yes

Funds for a member of staff?: No

Fees for consulting?: No

Have you in the past five years been employed by an organisation that may in any way gain or lose financially from the publication of this paper?: No

Do you hold any stocks or shares in an organisation that may in any way gain or lose financially from the publication of this paper?: No

If you have any competing interests (please see BMJ policy) please declare them here:

Reviewer: 3

Recommendation:

Comments: This is excellent work and the authors are to be congratulated for addressing this question. We are often misled by effect size and consider this as stronger evidence, disregarding the fact that evidence needs to be concluded with corroborative findings from several sources. We are therefore often challenged as to why we need a second trial if the first one was so significantly positive; this is especially true in surgery. The authors have helped us support the need for further trials in any research program. The only issue that I have is that only a very small number of studies ended up in the analysis. I am suggesting that the authors address this some more in their discussion with emphasis on whether including trials that did not fulfill the exact inclusion criteria would alter the results. Another question may be answered by another study in which the subsequent studies would have variable sizes to determine whether the difference in results is influenced by the difference in size between the index and subsequent studies. This can be expanded to more than one subsequent study as well. Another issue that could be addressed is whether it makes sense to conduct subsequent studies when well designed and adequately powered index studies are negative nto only by statistcal significance but also by clinical importance. I would recommend publication of this paper

Additional Questions: Please enter your name: Dr. John S Sampalis

Job Title: Professor Faculty of Medicine Department of Surgery

Institution: McGill University

Reimbursement for attending a symposium?: No

A fee for speaking?: No

A fee for organising education?: No

Funds for research?: No

Funds for a member of staff?: No

Fees for consulting?: No

Have you in the past five years been employed by an organisation that may in any way gain or lose financially from the publication of this paper?: No

Do you hold any stocks or shares in an organisation that may in any way gain or lose financially from the publication of this paper?: No

If you have any competing interests (please see BMJ policy) please declare them here:

Reviewer: 4

Recommendation:

Comments: Dear editors.

The manuscript ‘Do randomised trials with very large treatment effects make further trials unnecessary? An empirical assessment.’ raises the question about the necessity of further trials after a randomized interventional trial has shown a very large treatment effect. The authors conducted a - what they call - ‘empirical assessment’, which is methodologically close to a systematic review, where certain sources were searched according to pre-defined inclusion criteria in order to extract data. Specifically, systematic reviews were searched for forest plots with one trial showing a very large effect, followed by a large trial with the same objective. The results of this assessment indicate that the very large effect seen in one trial is rarely replicated in following trials. The authors conclude that randomised trials with very large treatment effects do not make further trials unnecessary.

The manuscript is well written and illustrative. The issues raised in this manuscript are interesting and important. They are highly related to the problem of studies with a small sample size, and the manuscript adds to existing knowledge. I agree with the authors’ methods and conclusions, and I would recommend the publication of the manuscript.

I have listed some minor issues about the manuscript below.

1) Page 8, row 15-25 and row 31-45; page 9, row 20-29: Inclusion and exclusion criteria are presented in different sections of the methods chapter, with other methodological issues presented in-between. Maybe restructuring would be helpful presenting all inclusion and exclusion criteria together.

2) Page 8, row 34-38: The authors state that they ‘excluded forest plots … not including the year of publication of each trial (as it would not be possible to determine if the trial was followed by a large trial)’. a. Would it not have been possible to extract the year of publication from the reference list of the review? b. How many potentially eligible forest plots were excluded because of this restriction?

3) Page 8, row 38-44: I feel that this exclusion criterion does not fit well to the heading of this paragraph (‘topics’).

4) Page 10, row 9-14: When there was more than 1 large trial following the index trial, I would find it useful to know the number of subsequent large trials with upheld and with refuted results (in addition to the meta-analysis).

5) Page 10, row 36: Please explain what is meant by the ‘difference between proportions’. If my understanding is correct, only relative risks are considered, not absolute differences.

6) Page 10, row 49-57: Please explain the selection of statistical tests (Fisher’s exact, Mann-Whitney U, and Kruskal- Wallis tests). E.g. which comparisons were performed with Mann-Whitney U and Kruskal-Wallis tests?

7) Table 1 (page 26): a. Row 27/28: I am not sure I understand what is meant by ‘No of large trials’. Is it ‘large trials among index trials’ or ‘‘large trials within each forest plot’ or ‘large trials for each index trial’? How is the number 1 (overall) compatible with the sum of 1 (upheld) and 2 (not upheld)? b. Row 28-34: I am not sure I understand what is meant by ‘… largest trial’. Which trial is ‘the largest’?

8) Table 2 (page 27-28): a. In Table 1 (page 26), the relative risks (RR) of all trials have been ‘normalised’ in order to be combined. While this is not technically necessary for Table 2, I would still suggest doing so (e.g. RR of all index trials expressed as above one), because here the reader compares the RR for the index and the subsequent trial, and I would help if all comparisons would work in the same direction (instead of towards 1 from both directions). b. Table 2 presents only data for the 25 upheld index trials and their subsequent trial(s). I would also find it interesting to see the numerical results for the 19 refuted trials (including the information about upheld or not). c. For reasons of transparency and replicability, I would suggest to include the references of all 44 forest plots and/or all 44 index trials plus their respective subsequent trial(s) in the manuscript. Maybe Table 2 would be a convenient place for doing so. This would also show the number of subsequent trial(s) for each index trial, which I think should be given. d. I fully agree with the authors’ conclusion that ‘Caution should be taken when interpreting small studies with very large treatment effects.’ To strengthen this conclusion, it would be useful to present the sample size of the included trials, especially for the index trial (but why not also for the subsequent large trial). Again, maybe Table 2 would be a convenient place for doing so. e. The order of rows in Table 2 seems to be arbitrary (or at least not easy to identify). A content-related ordering could be useful. For example, ordering regarding the magnitude of the RR of the index trial; or by (absolute) discrepancy in RR between index trial and subsequent trial(s).

9) Page 10, row 2-13: This maybe only a theoretical issue, but if there were for example four subsequent trials, one with a statistically significant effect in the opposite direction, and three with an effect in the same direction (significant or not), and a meta-analysis of these four trials is significant in the same direction: Does condition 1 hold (‘at least one subsequent large trial presented a statistically significant effect in the opposite direction’) or condition 2 (‘where there was more than 1 large trial … meta-analysis performed’). Maybe a rephrasing of the definition of refuted/upheld trials might increase clarity.

10) Page 16, row 36-40: A ‘rule of thumb’ is mentioned here, but I find it difficult to understand which rule is meant. In addition, the ‘positive predictive value (PPV) of the rule’ is unclear. The PPV is mentioned several times in the discussion section, but it would be helpful if it was clearer related to the results.

11) Figure 1 (page 29): In the lowest box, the number of forest plots (n=44) is smaller than the number of Cochrane reviews (N=45). Is that correct?

12) Figure 2 (page 30) is nicely done and informative. The grid is useful to grasp the RR-values of the plotted studies. a. Maybe an additional diagonal line would help as an indicator of discrepancy in RR between index trial and subsequent trial(s). b. As mentioned before, I would find it helpful to plot the ‘normalised’ RRs here (e.g. all equal to or above 1). Especially in this figure, the split of results in 2 areas could be avoided.

13) Discussion section (page 16-19): The work presented here is purely empirical without considering the setting, populations, test interventions, control interventions, etc. of the trials. It might be worthwhile to check if discrepancies in results between index trial and subsequent trial(s) might be due to differences in study characteristics.

14) Entire manuscript (including tables and figures): It seems that the terms ‘refuted’ and ‘not upheld’ are used synonymously to indicate subsequent large trials with disagreeing results compared to the index trial. Clarity could be increased if only one term would be used consistently throughout the manuscript.

15) Entire manuscript (including tables and figures): The term ‘effect size’ is used to denote relative risks. I would suggest using the term ‘relative risks’ instead, as for some readers ‘effect size’ might be considered the standardized mean difference (SMD).

16) Entire manuscript (including tables and figures): I would suggest to ‘normalise’ all RR in all tables, figures, and in the entire manuscript to improve clarity.

Dr. Stephanie Roll Scientist / statistician Charité - Universitätsmedizin Berlin Institute for Social Medicine, Epidemiology, and Health Economics Luisenstr. 57, 10117 Berlin, Germany [email protected] http://epidemiologie.charite.de

Additional Questions: Please enter your name: Stephanie Roll

Job Title: Scientist / statistician

Institution: Charité - Universitätsmedizin Berlin. Institute for Social Medicine, Epidemiology, and Health Economics

Reimbursement for attending a symposium?: No

A fee for speaking?: No

A fee for organising education?: No

Funds for research?: No

Funds for a member of staff?: No

Fees for consulting?: No

Have you in the past five years been employed by an organisation that may in any way gain or lose financially from the publication of this paper?: No

Do you hold any stocks or shares in an organisation that may in any way gain or lose financially from the publication of this paper?: No

If you have any competing interests (please see BMJ policy) please declare them here:

**Information for submitting a revision**

Deadline: Your revised manuscript should be returned within one month.

How to submit your revised article: Log into http://mc.manuscriptcentral.com/bmj and enter your Author Center, where you will find your manuscript title listed under "Manuscripts with Decisions." Under "Actions," click on "Create a Revision." Your manuscript number has been appended to denote a revision.

You will be unable to make your revisions on the originally submitted version of the manuscript. Instead, revise your manuscript using a word processing program and save it on your computer. Once the revised manuscript is prepared, you can upload it and submit it through your Author Center. When submitting your revised manuscript, you will be able to respond to the comments made by the reviewer(s) and Committee in the space provided. You can use this space to document any changes you make to the original manuscript and to explain your responses. In order to expedite the processing of the revised manuscript, please be as specific as possible in your response to the reviewer(s). As well as submitting your revised manuscript, we also require a copy of the manuscript with changes highlighted. Please upload this as a supplemental file with file designation ‘Revised Manuscript Marked copy’. Your original files are available to you when you upload your revised manuscript. Please delete any redundant files before completing the submission.

When you revise and return your manuscript, please take note of all the following points about revising your article. Even if an item, such as a competing interests statement, was present and correct in the original draft of your paper, please check that it has not slipped out during revision. Please include these items in the revised manuscript to comply with BMJ style (see: http://www.bmj.com/about-bmj/resources-authors/article-submission/article-requirements and http://www.bmj.com/about-bmj/resources-authors/forms-policies-and-checklists).

Items to include with your revision (see http://www.bmj.com/about-bmj/resources-authors/article-types/research):

1. What this paper adds/what is already known box (as described at http://resources.bmj.com/bmj/authors/types-of- article/research)

2. Name of the ethics committee or IRB, ID# of the approval, and a statement that participants gave informed consent before taking part. If ethics committee approval was not required, please state so clearly and explain the reasons why (see http://resources.bmj.com/bmj/authors/editorial-policies/guidelines.)

3. Patient confidentiality forms when appropriate (see http://resources.bmj.com/bmj/authors/editorial- policies/copy_of_patient-confidentiality).

4. Competing interests statement (see http://resources.bmj.com/bmj/authors/editorial-policies/competing-interests)

5. Contributorship statement+ guarantor (see http://resources.bmj.com/bmj/authors/article-submission/authorship- contributorship)

6. Transparency statement: (see http://www.bmj.com/about-bmj/resources-authors/forms-policies-and- checklists/transparency-policy)

7. Copyright statement/licence for publication (see http://www.bmj.com/about-bmj/resources-authors/forms-policies- and-checklists/copyright-open-access-and-permission-reuse)

8. Data sharing statement (see http://www.bmj.com/about-bmj/resources-authors/article-types/research)

9. Funding statement and statement of the independence of researchers from funders (see http://resources.bmj.com/bmj/authors/article-submission/article-requirements).

10. Patient involvement statement (see http://www.bmj.com/about-bmj/resources-authors/article-types/research).

11. Please ensure the paper complies with The BMJ’s style, as detailed below: a. Title: this should include the study design eg "systematic review and meta-analysis.” b. Abstract: Please include a structured abstract with key summary statistics, as explained below (also see http://resources.bmj.com/bmj/authors/types-of-article/research). For every clinical trial - and for any other registered study- the last line of the abstract must list the study registration number and the name of the register. c. Introduction: This should cover no more than three paragraphs, focusing on the research question and your reasons for asking it now. d. Methods: For an intervention study the manuscript should include enough information about the intervention(s) and comparator(s) (even if this was usual care) for reviewers and readers to understand fully what happened in the study. To enable readers to replicate your work or implement the interventions in their own practice please also provide (uploaded as one or more supplemental files, including video and audio files where appropriate) any relevant detailed descriptions and materials. Alternatively, please provide in the manuscript urls to openly accessible websites where these materials can be found. e. Results: Please report statistical aspects of the study in line with the Statistical Analyses and Methods in the Published Literature (SAMPL) guidelines http://www.equator-network.org/reporting-guidelines/sampl/. Please include in the results section of your structured abstract (and, of course, in the article's results section) the following terms, as appropriate: i. For a clinical trial: Absolute event rates among experimental and control groups; RRR (relative risk reduction); NNT or NNH (number needed to treat or harm) and its 95% confidence interval (or, if the trial is of a public health intervention, number helped per 1000 or 100,000.) ii. For a cohort study: Absolute event rates over time (eg 10 years) among exposed and non-exposed groups; RRR (relative risk reduction.) iii. For a case control study:OR (odds ratio) for strength of association between exposure and outcome. iv. For a study of a diagnostic test: Sensitivity and specificity; PPV and NPV (positive and negative predictive values.) v. For a systematic review and/or meta-analysis: Point estimates and confidence intervals for the main results; one or more references for the statistical package(s) used to analyse the data, eg RevMan for a systematic review. There is no need to provide a formal reference for a very widely used package that will be very familiar to general readers eg STATA, but please say in the text which version you used. For articles that include explicit statements of the quality of evidence and strength of recommendations, we prefer reporting using the GRADE system. f. Discussion: To minimise the risk of careful explanation giving way to polemic, please write the discussion section of your paper in a structured way. Please follow this structure: i) statement of principal findings of the study; ii) strengths and weaknesses of the study; iii) strengths and weaknesses in relation to other studies, discussing important differences in results; iv) what your study adds (whenever possible please discuss your study in the light of relevant systematic reviews and meta-analyses); v) meaning of the study, including possible explanations and implications for clinicians and policymakers and other researchers; vi) how your study could promote better decisions; vi) unanswered questions and future research g. Footnotes and statements

Online and print publication: All original research in The BMJ is published with . Our open access policy is detailed here: http://www.bmj.com/about-bmj/resources-authors/forms-policies-and-checklists/copyright-open- access-and-permission-reuse. The full text online version of your article, if accepted after revision, will be the indexed citable version (full details are at http://resources.bmj.com/bmj/about-bmj/the-bmjs-publishing-model). The print and iPad BMJ will carry an abridged version of your article. This abridged version of the article is essentially an evidence abstract called BMJ pico, which we would like you to write using the template downloadable at http://resources.bmj.com/bmj/authors/bmj-pico. Publication of research on bmj.com is definitive and is not simply interim "epublication ahead of print", so if you do not wish to abridge your article using BMJ pico, you will be able to opt for online only publication. Please let us know if you would prefer this option. If your article is accepted we will invite you to submit a video abstract, lasting no longer than 4 minutes, and based on the information in your paper’s BMJ pico evidence abstract. The content and focus of the video must relate directly to the study that has been accepted for publication by The BMJ, and should not stray beyond the data.

Date Sent: 18-Aug-2016