CORE Metadata, citation and similar papers at core.ac.uk

Provided by PubMed Central PLoS CLINICAL TRIALS Essay Publishing Results: The Future Beckons Elizabeth Wager

Search for the truth is the noblest and from spin-offs such as reprints and inadvertently including the same results occupation of man; its publication is a advertising [4]. more than once (which can bias system- duty. —Madame de Stael (1766–1817) atic reviews and meta-analyses) will thus Problems with Existing Publishing be reduced [6]. Reviewer (and reader) Formats for reporting results from Models access to information about trial designs randomized clinical trials in peer-re- The traditional system of publishing trials and planned outcomes, which was re- viewed journals have remained virtually in medical journals reflects many of the corded at the start of each trial, will unchanged and unchallenged for the limitations of paper-based systems despite permit detection of selective reporting past 50 years [1]. However, a number of the advent of electronic publishing. In since reports of results can be compared developments, some technological and general, new media and the Internet have with the original [9,10]. some political, provide exciting oppor- been used to modify existing systems Suppression of unfavourable findings tunities to question whether we are rather than for dramatic redesign of the will be apparent if all studies are regis- using the most efficient and effective architecture of the medical evidence base. tered at inception. But trial registration methods of publication. Here, I suggest For example, papers may now be accessed can prevent only if it is how methods of reporting clinical trials from journal Web sites but the format of coupled with a commitment to make could be improved and consider the articles and the ways in which results from all studies available and a implications for trial sponsors and findings are presented remain the same. mechanism for implementing and polic- While some journals allow additional ing this. Guidelines for pharmaceutical medical journals. material to be displayed electronically, companies [11] and investigators [12] Why Do We Publish Clinical Trial most still impose strict size limits on have long emphasized the importance Results? research reports and simply use Web sites (and some would argue moral obligation) as a means of accessing documents that are of publishing results from all studies, Publication (and non-publication) of trial identical to the print version. The result is regardless of their outcome. But if such results can be motivated by several that, while there have been some major guidelines are to be enforced, we need a factors. Research reports may affect changes such as the development of ‘‘open clear definition of what is meant by clinical practice, inform patients, influ- access’’ journals, information remains publication. ence future research, and prevent dupli- hard (or expensive) to access, scattered cation of effort. Many believe there is a across many places, and difficult to syn- moral obligation to report results of all thesize. Producing and disseminating trials that involve patients [2]. printed material can be slow, and journal Publication also has important secon- peer-review often contributes further de- dary effects. These may explain existing ...... lays [5]. patterns of publication and resistance to Another problem with the existing Funding: The author received no specific funding for change—despite evidence that current system is that non-publication of negative this article. systems are not the best. Authorship of trials and non-reporting of negative out- Competing Interests: Elizabeth Wager is a freelance papers in journals can establish reputa- comes, coupled with redundant publica- medical writer and publications consultant who tions and enhance career prospects. often works for pharmaceutical companies. Before tion of positive findings, has led to Institutional or corporate reputations becoming self-employed she worked for Janssen- systematic publication bias, which can may also benefit. The productivity of Cilag, and then GlaxoSmithKline. She is a co-author of undermine the reliability of medical the Good Publication Practice guidelines for academic departments is judged on their evidence [2,6,7]. pharmaceutical companies and the EMWA guidelines publication output, which also affects on the role of professional medical writers, and occasionally receives payment for running their chances of obtaining future fund- Benefits of Alternative Models workshops on publication ethics based on these ing. The peer-reviewed (and indexed) One of the most important initiatives guidelines. journal has thus become part of the aimed at reducing publication bias is trial Citation: Wager E (2006) Publishing clinical trial process of academic appointments and registration (i.e., making details of study results: The future beckons. PLoS Clin Trials 1(6): e31. promotions, which, in turn, has spawned designs publicly available at the start of a DOI: 10.1371/journal.pctr.0010031 the discipline of bibliometrics [3]. And, study) [8]. One potential advantage of DOI: 10.1371/journal.pctr.0010031 of course, there are financial interests: electronic publication coupled with trial drug companies use publications to Copyright: Ó 2006 Elizabeth Wager. This is an open- registration is that study results can be access article distributed under the terms of the increase sales while publishers make linked to the original protocol, or at least Creative Commons Attribution License, which money both directly from journal sales permits unrestricted use, distribution, and to a summary of its main features. Clear reproduction in any medium, provided the original study identification, e.g., by including a author and source are credited. trial register number, should highlight The Essay section contains opinion pieces on topics Elizabeth Wager is Publications Consultant at of broad interest to a general medical audience. redundant publications. Secondary anal- Sideview, Princes Risborough, Buckinghamshire, yses will be easy to identify, and the risk of United Kingdom. E-mail: [email protected]

www.plosclinicaltrials.org 0001 October | 2006 | e31 Table 1. Comparison of Various Methods of Results Publication

Format Length Public Designed to Detail of Standard Guidelines/Evidence Review Process Accessibility Be Read by Study Design Format for Contents

Conference 250–500 Variable Specialists (clinicians Low Usually No External review (accept/reject) words and researchers) (IMRAD structure) but no chance to revise ICH E3 summarya 2–3,000 Only if posted Regulators Low Yes Weak Extensive internal and some words by company external (investigators) Summary posted Variablea Yes Public Moderate No No Internal oncompany Web (4–25 pages) (unless ICH E3) siteb Traditional 2,500–5,000 Depends on Journal Moderate IMRAD Depends on journal Traditional journal article words journal policy readers and if CONSORT is peer-review enforced CONSORT- 4,000–7,000 Yes for open Journal/site High Yes Strong Traditional based articlec words access journals users (CONSORT) peer-review ICH E3 full reporta 20–50,000 No Regulators High Yes Yes (but not Extensive internal and some (main text) words evidence-based) external (investigators) ICH E3 reporta Huge (hundreds No Regulators High Yes Yes (but not Extensive internal and some and fulllistings of pages) evidence-based) external (investigators) Databank Unlimited Yes Specialists High Yes No Post-publication (researchers) aFormat proposed by the International Conference on Harmonization E3 guidelines for preparing clinical trial reports for regulatory authorities. bGSK usually 4–5 pages (1,500–2,000 words); Eli Lilly usually 10–25 pages. cBased on PLoS Clinical Trials. DOI: 10.1371/journal.pctr.0010031.t001

What Do We Mean by Publication? suggested that the summary format other, possibly better, forms of In the era when print technology was used for regulatory reports might be publication. dominant, publication was easy to define. used for posting results [16]. (This The role of peer-review and The need for expensive equipment and summary is sometimes termed ICH E3, traditional journals. Publication in expertise restricted entry into large-scale referring to the recommendations of peer-reviewed journals has, until now, publishing activities. This was the golden the International Conference on been the main mechanism for age of the publisher. Within science and Harmonisation, which sets reporting disseminating findings to clinicians medicine, also became estab- standards.) This format is now used by and researchers. Yet the evidence that lished (despite limited evidence of its several companies on their Web sites, peer review in medical journals acts as effectiveness and lack of agreement about an effective measure of quality control its primary purpose) [13]. Until recently, although critics point out that it has not is, at best, extremely limited [13]. publication therefore meant being been tested and no guidelines exist Although they have not been through printed in a peer-reviewed journal. How- about the amount of detail that should ever, a combination of new technology be provided under each heading [17]. journal peer review, trial summaries prepared for regulatory reports are and increasing public calls for transpar- However, since most journals will not usually subjected to stringent quality ency means that different modes of publish material that has already been control mechanisms and are scru- publication are now being explored. A published elsewhere, some companies tinised by both company personnel comparison of some publication methods have stated that results summaries will and external investigators. It could be is shown in Table 1. not be posted if the trial is pending Results posting on Web sites. argued that these processes are at least publication in a peer-reviewed journal Following legal action against as good as traditional peer review, but [18]. Journals have always accepted that companies accused of suppressing others will argue that self-regulation by publication of conference abstracts unfavourable findings, several phar- the industry is never sufficient. would not jeopardize subsequent full maceutical companies have an- Despite the fact that traditional nounced plans to make study findings publication; however, summaries papers are considered by many to be available via Web sites [14,15]. The posted by Eli Lilly and Glaxo- the ‘‘gold standard’’ for clinical trial American industry association PhRMA SmithKline range from three to 20 publication, their effectiveness in (the Pharmaceutical Research and pages, and therefore contain conveying information has rarely Manufacturers of America) has considerably more detail than been tested [1], they fail to provide created a Web site for this purpose abstracts, so the debate continues enough detail for some users [19], and (http://www.clinicalstudyresults.org). about whether journal editors should selective reporting of outcomes is The World Health Organization consider such postings to be ‘‘full’’ common [10]. (WHO), which has been coordinating publications [19]. It is ironic that The standards for reporting discussion about trial registries, has also medical journals, for so long the randomized trials in journals have discussed guidelines for the posting of bastion of publishing research been raised by guidelines such as the results. An initial working group findings, may now prevent or delay CONSORT statement [20], yet there is www.plosclinicaltrials.org 0002 October | 2006 | e31 benefits of peer review is weak, its requires not only technical feasibility but Box 1. Reporting Results from ability to ‘‘tone down’’ conclusions also political and commercial acceptabil- Clinical Trials and remove unfounded claims has ity. Individuals, academic institutions, been clearly shown [13]. But peer- and commercial companies usually con- What’s the problem? review has a poor record in detecting sider data as a precious possession, not  Clinical trial results are scattered and often something to be shared with potential difficult to access. incorrect or fabricated data [5]. If the rivals. Many researchers are concerned  Non-publication of unfavourable trials or functions of reporting results and that public access to raw data could negative results causes systematic discussing their implications are encourage inappropriate analyses and publication bias. separated, one might argue that peer misinterpretation. However, Senn has  Standard journal articles often omit review is more helpful for the latter suggested that providing computable data important details. than the former. A new model might  Traditional peer-review and journal might become ‘‘the ultimate test and therefore be for investigators or promoter of statistical quality’’ [25]. He publishing delays the publication of sponsors to make results available on findings. argues that ‘‘every statistician will have to publicly accessible Web sites using  Enforcing commitments to publish all prepare reports knowing that they will be results are hampered by a lack of a clear standard templates and for journals to scrutinised in the finest detail by a definition of publication. add value by publishing peer-reviewed frequently hostile posterity’’. This, surely, What are the opportunities? commentary and synthesis. is the ultimate form of peer review.  Trial registers/linking protocols with This model might also help resolve However, although databanks offer excit- publications. the ethical arguments about access to ing possibilities, researchers’ reluctance  Online publication on Web sites with research. Funders, researchers, and to expose their raw data to the world may minimal running costs. readers increasingly demand access to hold back change.  Data banks are a possibility for the future. clinical trial results for free, yet What’s the solution? producing peer-reviewed journals is a Problems Ahead  If all trials were registered at inception, costly business [24]. If results were Publicly accessible, searchable electronic redundant publication would be easy to results dissemination coupled with trial freely accessibly from publicly funded spot and non-publication could be registers could solve many of the prob- Web sites, then the costs and delays challenged. lems that limit the current system of involved with peer review would not  Links between trial protocols (or registers) research reporting via peer-reviewed and results would highlight selective affect the primary dissemination of journals. However, such a new system reporting and post hoc analyses. results. Publishers and editors could could also create problems of its own. Electronic templates for results reporting  justify journal access charges by For a start, trial registers and publica- should raise standards and improve producing lively and informative tion vehicles need to be funded. Cur- searchability. reviews and critiques of the latest rently, some registers and Web sites  Results could be posted on publicly findings. Journals (or electronic accessible Web sites at minimal cost. charge an administration fee for entries. media) could also specialize in  Peer-reviewed journals should concentrate This may be a barrier for unfunded on interpretation, synthesis, commentary, providing information and inter- research and projects from the develop- and discussion. pretation for different audiences, for ing world. It is therefore vital that example, to meet the different needs of sustainable funding systems are devised, researchers, clinicians, and patients. and these must allow inclusive policies for trial registration and results dissemina- Looking Further into the Future tion. still considerable room for im- While free text may serve readers well, it Researchers will need to be equipped provement [21,22]. In particular, isnotanefficientelectronicstorage to produce reports or data in the re- statistical errors continue to be a medium. The most powerful, and some quired format. This might be facilitated problem [23]. In considering the would argue the most efficient and trans- by open-access software and protocols for merits of various forms of publication, parent, method of making results avail- data transfer, but human effort will also it would be interesting to compare able would be to provide raw (but be required for education and support. journal articles with ICH E3 study anonymized) datasets for public scrutiny. The proposed system of posting results reports in terms of their completeness, It is now technically possible to store is largely self-policing and relies to a compliance with CONSORT, and results in a computable databank as in the greater extent on internal systems of quality of statistical analyses. It would Trial Bank Project (http://rctbank.ucsf. quality control (within companies or also be interesting to compare peer edu/). Based on such research, a Global academic institutions) than traditional review as performed by journals with Trial Bank has recently been established peer-reviewed publication. Performing and aims to be ‘‘the most advanced the quality-control mechanisms used by trials and handling data according to computable repository of trial protocol Good Clinical Practice requires consid- companies when preparing reports— and results information to promote bio- erable expertise and resources. This is for example, in terms of their ability to medical discovery as well as transparency currently reinforced by oversight from detect errors. and accountability in clinical trial re- regulatory bodies (e.g., random audits) in Journal articles using the traditional search’’ [16]. Companies already provide the case of the pharmaceutical industry. IMRAD format (i.e., Introduction, such data to the US Food and Drug The new model of publication might Methods, Results, And Discussion) Administration, proving that this is tech- require similar policing systems, espe- combine objective study findings with nically possible and that standard data cially for non-industry-funded studies, subjective interpretation. While transfer protocols are available. and such systems would require consid- evidence of many other supposed However, adoption of such an initiative erable resources. www.plosclinicaltrials.org 0003 October | 2006 | e31 Editors and publishers may be con- tional Institutes for Health (in the US), to 12. World Medical Association (2004) World cerned about what will happen to their rise to this challenge. We also need to Medical Association . Ethical principles for journals if they are no longer the primary consider the best funding models for such involving human subjects. 52nd WMA General vehicle for publishing clinical trial results. systems that would optimize access to Assembly; October 2000; Edinburgh, Scotland. Some medical editors have already con- findings without creating barriers to non- Available: http://www.wma.net/e/policy/b3.htm. Accessed 28 September 2006. sidered and even prophesied this scenario commercially funded research. Changes 13. Jefferson TO, Alderson P, Wager E, Davidoff F [26–28]. Under the new system, successful in research reporting systems will have (2002) Effects of editorial peer review: A journals will be those brave enough to implications for the future of medical . JAMA 287: 2784–2786. focus on other functions. These could journals and for current systems of 14. Dyer O (2004) GlaxoSmithKline to set up comprehensive online clinical trials register. include secondary publications providing measuring research productivity. Given BMJ 329: 590. synthesis, commentary, opinion, and in- the established position of peer-reviewed 15. (2004) Drug company to make its trial results terpretation, along with other roles such journals, there is likely to be a gradual public. BMJ 329: 366. 16. Sim I, Detmer DE (2005) Beyond trial as stimulating debate, campaigning for transition from the current model. This registration: A global trial bank for clinical change, educating, and even entertaining transition may be accelerated by influ- reporting. PLoS Med 2: e365. DOI: 10.1371/ their readers [26]. Freed from the respon- encers such as journal editors (if they are journal.pmed.0020365 sibility of publishing clinical trial results, brave enough to embrace the change 17. Grimes DA, Hubacher D, Nanda K, Schulz KF, Moher D, et al. (2005) The Good Clinical journals might also develop new and rather than resisting it), regulators, and Practice guideline: A bronze standard for more effective models of peer review. legislators. With wide consultation and clinical research. Lancet 366: 172–174. The current system of academic credits cooperation from all key players (includ- 18. Eli Lilly and Company (2005) Principles of medical research—Clinical trial registry. and research assessment based on author- ing clinicians, drug companies, and bio- Available: http://www.lillytrials.com/docs/ ship of peer-reviewed articles would need informatics experts), we could produce a ctr_policy.pdf. Accessed 28 September 2006. to be changed. Entries on trial registers system that greatly enhances the effi- 19. Rockhold FW, Krall RL (2006) Trial summaries and results databases are currently anon- ciency, transparency, and power of the on results databases and journal publication. Lancet 367: 1635–1636. ymous. If electronic posting becomes the medical knowledge base. This is an 20. Moher D, Schulz KF, Altman DG (2001) The primary (or, for some studies, the sole) opportunity we must not miss. n CONSORT statement: Revised method of publication, a fair and trans- recommendations for improving the quality of reports of parallel-group randomised trials. parent system for acknowledging contrib- REFERENCES Lancet 357: 1191–1194. utors will need to be devised. This should 21. Ioannidis JPA, Lau J (2001) Completeness of be viewed as an opportunity to replace 1. Smith R (2004) Commentary: Scientific articles safety reporting in randomized trials. JAMA have hardly changed in 50 years. BMJ 328: traditional journal authorship systems, 285: 437–443. 1533. 22. Mills EJ, Wu P, Gagnier J, Devereaux PJ (2005) which are open to abuse [29]. Better 2. Chalmers I (1990) Underreporting research is The quality of randomized trial reporting in methods of assessing academic perform- scientific misconduct. JAMA 263: 1405–1408. leading medical journals since the revised 3. Garfield E (1999) Journal impact factor: A brief ance than relying on journal impact CONSORT statement. Contemp Clin Trials 26: review. Can Med Assoc J 161: 979–980. 480–487. 4. Hopewell S, Clarke M (2003) How important is factors should also be developed [30]. 23. Neville JA, Lang W, Fleischer AB (2006) Errors the size of a reprint order? Int J Technol Assess Use of study identifiers presents possibil- in the Archives of Dermatology and the Health Care 19: 711–714. Journal of the American Academy of ities to develop more accurate citation 5. Wager E, Jefferson TO (2001) Shortcomings of Dermatology from January through December measures relating to the original study peer review in biomedical journals. Learned 2003. Arch Dermatol 142: 737–740. Publishing 14: 257–263. rather than its publication in a journal— 24. Tamber PS, Godlee F, Newmark P (2003) Open 6. Trame` r MR, Reynolds DJM, Moore RA, for example, by measuring the number of McQuay HJ (1997) Impact of covert duplicate access to peer-reviewed research: Making it times that a study is re-analysed, synthe- publication on meta-analysis: A case study. happen. Lancet 362: 1575–1577. 25. Senn S (2000) Statistical quality in analysing sized, or referred to in journals. BMJ 315: 635–640. 7. Dickersin K (1990) The existence of pharmaceutical clinical trials. Good Clinical publication bias and risk factors for its Practice Journal 7: 22–26. Conclusions occurrence. JAMA 263: 1385–1389. 26. Smith R, Roberts I (2006) We are at the threshold of a new era of 8. Sim I, Chan AW, Gu¨ lmezoglu AM, Evans T, requires a new way to publish clinical trials. Pang T (2006) Clinical trial registration: PLoS Clin Trials 1: e6. DOI: 10.1371/journal. reporting clinical trial results. We need to Transparency is the watchword. Lancet 367: pctr.0010006 develop an evidence-based system to 1631–1633. 27. Smith R (2005) Medical journals are an ensure that data storage and dissemina- 9. Melander H, Ahlqvist-Rastad J, Meijer G, extension of the marketing arm of pharmaceutical companies. PLoS Med 2: e138. tion technology are harnessed to produce Beerman B (2003) Evidence b(i)ased medicine—Selective reporting from studies DOI: 10.1371/journal.pmed.0020138 the best outcome for all users. Given the sponsored by pharmaceutical industry: Review 28. Abbasi K, Butterfield M, Connor J, Delamothe huge sums spent on medical research, it is of studies in new drug applications. BMJ 326: T, Dutton S, et al. (2002) Four futures for scientific and medical publishing. BMJ 325: a scandal that we have invested so little in 1171–1173. 10. Chan AW, Altman DG (2005) Identifying 1472–1475. determining the most effective and effi- outcome reporting bias in randomised trials 29. Rennie D (1997) When authorship fails. A cient ways of disseminating findings. I on PubMed: Review of publications and survey proposal to make contributors accountable. urge funders, especially non-commercial of authors. BMJ 330: 753. JAMA 278: 579–585. 11. Wager E, Field EA, Grossman L (2003) Good 30. Seglen P (1997) Why the impact factor of ones such as the Wellcome Trust, Medical Publication Practice for pharmaceutical journals should not be used for evaluating Research Council (in the UK), and Na- companies. Curr Med Res Opin 19: 149–154. research. BMJ 314: 498–502.

www.plosclinicaltrials.org 0004 October | 2006 | e31