Evidence-Based Dentistry (2002) 3, 32 ± 34 ã 2002 Evidence-Based Dentistry All rights reserved 1462-0049/02 $25.00 www.nature.com/ebd editorial Systematic review in scientific controversy: the example of water fluoridation, an open access review D Richards1, P Mansfield2 and J Kleijnen3 1Director, Centre for Evidence-based Dentistry, Oxford, UK, 2Director of Templegarth Trust, Louth, UK, 3NHS Centre for Reviews and Dissemination, University of York, York, UK

protocol, and of the subsequent meet- The systematic review of the safety and effectiveness of water fluoridation was ings to discuss progress and drafts of the an open-access systematic review. This has overcome some of the difficulties reports. Members of the advisory panel encountered in a highly contentious area, but it did not work perfectly. Parties also had frequent discussions with the on each side of the controversy were reluctant to abandon their previous review team during the progress of the positions and endorse the review result whole-heartedly. Surprising progress review. A website (http://www.york.ac.uk/ was made, however, which should encourage greater confidence in future inst/crd/fluorid.htm) was established reviews of this kind. shortly after the first advisory panel Evidence-Based Dentistry (2002) 3, 32 ± 34. doi:10.1038/sj.ebd.6400121 meeting. This initially contained the draft protocol and the approved min- utes for the advisory meeting. The The systematic review published in their reliability. Rather than reflecting objective of providing the website was 20001,2 on the safety and effectiveness the views of the authors or being based to allow interested groups and indivi- of water fluoridation is the first of its only on a (possibly biased) selection of duals open access to the process of the type in many respects. We believe that the published literature, they contain a review and give them the opportunity important lessons can be learned from comprehensive summary of the avail- to comment or provide additional the conduct of this review that can be able evidence. information, which could be relevant used to improve the quality of scientific Following the decision of the UK to the review. A paper detailing the reviews in areas where strong views are Government to conduct a review of impact of this on the methodology of held. the evidence relating to the safety and the review is in preparation. Systematic reviews3,4 locate, appraise effectiveness of water fluoridation, the An important element in the process and synthesise evidence from scientific Centre for Reviews and Dissemination of the advisory panel meetings was the studies in order to provide informative (CRD) in York was contracted to carry discussion of the protocol. This discus- empirical answers to scientific research out the review. One of the early sion focused on identifying the key questions. They are therefore valuable decisions was to appoint an advisory objectives of the review and the types of sources of information for decision- panel to the review. The advisory panel evidence required to address these makers. In addition, by identifying was balanced to represent views from objectives. This resulted in the defini- what we know and don't know, they both sides of the fluoridation debate, as tion of specific inclusion criteria for are an invaluable first step before well as including those who are neutral. each objective and what confounding carrying out new primary research. This included doctors, dentists, scien- factors needed to be addressed in the Systematic reviews differ from other tists, consumers, epidemiologists and analysis. The review team felt that these types of review in that they adhere to a representatives of the water industry. discussions were very helpful in pre- strict scientific design in order to make The review team met with the advisory paring the protocol for the review. In them more comprehensive, to mini- panel on three occasions. The purpose addition, the focus on the structured mise the chance of bias, and so ensure of the first meeting was to clarify the methodology of the systematic review 33 was a binding factor in the heteroge- the York review. Contributors from far greater grasp in advance of the neous advisory panel. both sides of the argument whose review process and the reasons for it. These discussions and the open pro- comments were not included will un- Future research can be more efficient cess of the performance of the review derstandably be disappointed. Many and sharply focused as a result of open `on the web' convinced the members of were rejected because they covered areas reviews of this kind. A consensus on the advisory panel that the review team which, although relevant to the overall future research priorities was one result was independent and not working fluoride debate, were outside the focus of this review. The eventual findings of towards predetermined conclusions. of the review. Others were rejected future work are more likely to be In the end, this led to agreement about because they did not meet the inclusion debated rationally and achieve wide the protocol and the results of the criteria set at the outset of the review, consensus than if the review had not review. Despite a measure of success, most usually for lack of baseline infor- taken place. This prospect is a strong however, there was still a large measure mation or of an adequate control study. argument for rapid extension of the of suspicion and rumour. Some of this Drafts of the results of the review open systematic review process into could be related to a lack of complete appeared on the web before the comple- other controversial areas. understanding of the systematic review tion of the report. Although this helped process, which existed on both sides of improve both the quality and the read- Conclusion the argument. Despite the availability of ability of the report it has inevitably led Willingness to work openly through all clear information on both the questions to speculationas to its eventual findings. the evidence in a contentious issue is being covered by the review and the Withtheresultsofthereport,butnotthe essential for efficient use of resources in process of systematic reviewing itself on conclusions, available at an early stage the widest public interest. Water fluor- the website, there was a lack of under- some `spinning' occurred.23,24 The ad- idation was a hard test of any debating standing of the ground rules and the visory panel and review team have tried process. Open-access, independent, reason for them. This was clearly a hard to ensure that the report is a factual systematic scientific review proved to deep-seated problem because it was account of the best availableevidence on be a giant leap in the right direction. evident in comments from senior aca- the questions addressed. (Summaries of demics and professionals as well as the evidence related to each of the members of the public. questions addressed by the report ap- 1. NHS Centre for Reviews and Dissemination. A Systematic Review of Water Fluoridation. The remit from the UK Department of pear later in the journal.) York: NHS Centre for Reviews and Health was to conduct a systematic Although the advisory panel and Dissemination, University of York; 2000. review of the evidence for the safety and review team agreed the results of the 2. McDonagh MS, Whiting PF, Wilson PM, effectiveness of water fluoridation review, agreement was not reached Sutton AJ, Chestnutt I, Cooper J, et al. based on the currently available evi- about the conclusions or, more impor- Systematic review of water fluoridation. BMJ 2000; 321:855±859. dence from population-based studies. tantly, the implications of the review. As 3. NHS Centre for Reviews and Dissemination. This is only a small section of the issues predicted by the review leader at the Undertaking Systematic Reviews of that need to be considered when mak- first advisory board meeting, this was Research on Effectiveness: CRD's Guidance ing the decision of whether or not never going to be a once-and-for-all for Carrying Out or Commissioning public water supplies should be fluori- assessment that would settle the matter Reviews. 2nd Edition. York: NHS Centre for Reviews and Dissemination, University of dated. By limiting the scope of the about water fluoridation. Systematic York; 2001. review to a manageable area a thorough reviews are simply not policy decisions. 4. Chalmers I, Altman DG (Eds). Systematic and high-quality assessment of the They can only inform such decisions Reviews. London: BMJ; 1995. available evidence could be conducted. along with many other relevant factors. 5. Ministry of Health, Department of Health Attempting to review the whole fluor- This open-access systematic review for Scotland, Ministry of Housing and Local Government. The Fluoridation of Domestic idation issue in one systematic review has overcome some of the difficulties Water Supplies in North America as a would be a huge undertaking. In the encountered in a highly contentious Means of Controlling Dental Caries. Report past other non-systematic reviews5±22 area, but it did not work perfectly. of the United Kingdom Mission. London: have attempted this, which inevitably Parties on each side of the controversy HMSO. 1953 led to a compromise on quality and were reluctant to abandon their pre- 6. World Health Organisation. Report of Expert Committee on Water Fluoridation. accusations of bias, the probity of which vious positions and endorse the review Technical Report Series No 146. Geneva: cannot be checked. result whole-heartedly. Surprising pro- World Health Organisation; 1958. The open-access format certainly pro- gress was made, however, which should 7. Ministry of Health, Scottish Office, Ministry vided an opportunity for a wide range of encourage greater confidence in future of Housing and Local Government. The groups and individuals to submit in- reviews of this kind. Prerequisites for conduct of the fluoridation studies in the United Kingdom and the results achieved formation, which in some instances this are adequate assurance of the after five years. Reports on Public Health amounted to scholarly reviews working independence of the review team Ð and Medical Subjects. No. 105. London: from other ground rules than those of well borne out in this instance Ð and a HMSO; 1962.

Evidence-Based Dentistry 34 8. Department of Health and Social Security, 19. National Research Council, National Jan Cooper, Dental School, University of Scottish Office, Welsh Office. The Academy of Sciences Committee on Wales College of Medicine, Cardiff, fluoridation studies in the United Kingdom Toxicology. Health Effects of Ingested and the results achieved after eleven years. Fluoride. Washington DC: National Wales, UK Reports on Public Health and Medical Academy Press; 1993. Elizabeth Treasure, Dental School, Subjects. No. 122. London: HMSO; 1969. 20. Expert Committee on Oral Health Status University of Wales College of 9. World Health Organisation. Fluorides and and Fluoride Use. Fluorides and oral health. Medicine, Cardiff, Wales, UK human health. World Health Organisation WHO Technical Report Series. No. 846. Monograph Series. No. 59. Geneva: World Geneva: World Health Organisation; 1994. Health Organisation; 1970. 21. Royal Melbourne Institute of Technology 10. Royal College of Physicians. Fluoride, Teeth Ð Key Centre for Toxicology and the Systematic Review Advisory Panel and Health. Bath: Pitman Medical; 1976. Monash University Medical School's Centre Chair: Trevor Sheldon, York Health 11. Clemmesen J. The alleged association for Epidemiology and Preventive Medicine. between artificial fluoridation of water Review of Water Fluoridation and Fluoride Policy Group, University of York, York, supplies and cancer: a review. Bull World Intake from Discretionary Fluoride UK Health Organ 1983; 61:871±883. Supplements; 1999. Earl Baldwin of Bewdley, House of Lords, 12. Lord Jauncey. Opinion of Lord Jauncey in 22. National Research Council: National London, UK causa Mrs Catherine McColl (AP) against Academy of Sciences Committee on Iain Chalmers, UK Cochrane Centre, Strathclyde Regional Council. Edinburgh: Toxicology. Health effects of ingested Court of Session; 1983. fluoride. Washington DC: National Oxford, UK 13. Knox E G. Fluoridation of Water and Academy Press 1993. Sheila Gibson, Glasgow Homeopathic Cancer: a Review of the Epidemiological 23. Anon. Fluoride: the state of the art and Hospital, Glasgow, Scotland, UK Evidence. London: HMSO; 1985 science. REPORT: Newsl UK Public Health Sarah Gorin, Help for Health Trust, 14. Ad Hoc Subcommittee on Fluoride of the Assoc Summer 2000; Issue 4:21±22. Winchester, UK Committee to Coordinate Environmental 24. British Dental Association. British Dental Health and Related Programs. Review of Association News Release 20 Jul 2000 Mike Lennon, Chairman of the British Fluoride: Benefits and Risks. Washington Fluoridation Society, Department of DC: Public Health Service, US Department Clinical Dental Sciences, University of of Health and Human Services; 1991. Liverpool School of Dentistry, Liverpool, 15. Murray JJ, Rugg-Gunn AJ, Jenkins GN. UK Fluorides in Caries Prevention. 3rd Edn. Oxford: Wright; 1991. York Fluoride Systematic Review Team Peter Mansfield, Director of Templegarth 16. National Health and Medical Research Jos Kleijnen, NHS Centre for Reviews Trust, Louth, UK Council. The Effectiveness of Water and Dissemination, York, UK John Murray, Dean of Dentistry, Fluoridation. Canberra: Commonwealth of Marian McDonagh, NHS Centre for University of Newcastle, Newcastle Australia; 1991. Reviews and Dissemination, York, UK upon Tyne, UK 17. Medical Research Council Physiological Medicine and Infections Board. Report of Kate Misso, NHS Centre for Reviews and Jerry Read, Department of Health, the Working Group on Fluoridation of Dissemination, York, UK London, UK Drinking Water Ð Link With Osteoporosis, Penny Whiting, NHS Centre for Reviews Derek Richards, Centre for Evidence- 17 December 1993. London: Medical and Dissemination, York, UK Based Dentistry, Oxford, UK Research Council; 1993. Paul Wilson, NHS Centre for Reviews George Davey Smith, Department of 18. New Zealand Public Health Commission. Fluoridation of Water Supplies. Draft Policy and Dissemination, York, UK Social Medicine, University of Bristol, Statement, May 1993. Wellington: Public Ivor Chestnutt, Dental Public Health Bristol, UK Health Commission; 1993. Unit, Cardiff, Wales, UK Pamela Taylor, Water UK, London, UK

Evidence-Based Dentistry