CRD_mcdaid05_Poster.qxd 13/6/05 5:12 pm Page 1

Is it worthwhile including observational studies in systematic reviews of effectiveness? The experience from a review of treatments for childhood retinoblastoma Catriona Mc Daid, Suzanne Hartley, Anne-Marie Bagnall, Gill Ritchie, Kate Light, Rob Riemsma Centre for Reviews and Dissemination, University of York, UK Background • Overall there were considerable problems with quality Figure 1: Mapping of included studies (see Table). Without randomised allocation there was a Retinoblastoma is a rare malignant tumour of the retina and high risk of selection bias in all studies. Studies were also usually occurs in children under two years old. It is an susceptible to detection and performance bias, with the aggressive tumour that can lead to loss of vision and, in retrospective studies particularly susceptible as they were extreme cases, death. The prognoses for vision and survival less likely to have a study protocol specifying the have significantly improved with the development of more intervention and outcome assessments. timely diagnosis and improved treatment methods. Important • Due to the considerable limitations of the evidence clinical factors associated with prognosis are age and stage of identified, it was not possible to make meaningful and disease at diagnosis. Patients with the hereditary form of robust conclusions about the relative effectiveness of retinoblastoma may be predisposed to significant long-term different treatment approaches for childhood complications. retinoblastoma. Historically, enucleation was the standard treatment for unilateral retinoblastoma. In bilateral retinoblastoma, the eye ■ ■ ■ with the most advanced tumour was commonly removed and EBRT Chemotherapy EBRT with Chemotherapy the contralateral eye treated with external beam radiotherapy ■ Enucleation ■ Local Treatments (EBRT). However, EBRT may be associated with the risk of Table 1: Summary of study quality serious long-term complications. There has been an increasing trend in the use of chemotherapy and local Quality criteria Studies meeting Comments conservative treatments such as cryotherapy, criteria photocoagulation and thermotherapy and a decrease in the Description of assignment to n=14 Allocation was on the basis of disease severity, time period in which use of radiotherapy and enucleation, where possible. intervention treatment was given, the clinic which provided the treatment or generally the treatment protocol in use. Objective Groups balanced by design n=2 Identification of relevant prognostic n=21 Generally the only variable reported was Reese-Ellsworth •To conduct a systematic review of the evidence-base for variables classification or type of retinoblastoma. treatments for retinoblastoma. Matched for relevant prognostic n=4 Some studies also reported outcomes by Reese-Ellsworth •To consider the problems and benefits of including variables or effect of any group classification difference evaluated in a valid observational studies in a systematic review of the analysis clinical effectiveness of treatments for retinoblastoma. Intervention groups comparable at n=4 In five studies there was evidence that groups were not comparable baseline and in the remaining studies it was unclear or the relevant Methods information was not reported • Seventeen electronic databases were searched from Number of patients lost to follow-up n=2 Twenty-four studies included only patients on whom follow-up reported and rates similar across information was available. In four studies loss to follow-up was inception to April 2004. groups reported only for both treatment groups combined. In one study • Studies of participants diagnosed with childhood rates were reported for both groups but loss to follow-up was not similar across groups. retinoblastoma were eligible for inclusion. Any Follow-up period reported for both n=13 Mainly through use of Kaplan-Meier analysis intervention, or combination of interventions, and all treatment groups clinical outcomes were eligible. Where controlled trials Analyses adjusted for different n=10 were not available, prospective and retrospective cohort lengths of follow-up studies with clear comparisons between treatment Follow-up long enough for the n=20 groups were eligible. outcomes to occur Treatment clearly specified n=12 •Two reviewers independently assessed titles and Clearly defined criteria for measuring n=15 abstracts and full papers. outcomes • One reviewer carried out data extraction and quality assessment and this was checked by a second reviewer. Review conclusions Was it worthwhile including •Anarrative synthesis was conducted. • The conclusion of the review was that good quality observational studies? randomised controlled trials assessing the • Stricter study design inclusion criteria would have led Results effectiveness of different treatment options for to the same conclusions in a more efficient way. childhood retinoblastoma are required. • Thirty-one studies met the inclusion criteria. Apart from • It could be argued that it was inappropriate to include one study where some of the participants were • Where controlled trials are not feasible, only high observational studies of treatment interventions in a randomised, only observational comparative studies were quality prospective, non-randomised studies should systematic review due to the potential for bias. be given consideration, due to the generally higher identified. Twenty-seven studies were retrospective. • However the review would have been less risk of bias in retrospective studies. • The studies were diverse. comprehensive. Specific problems with the study designs used in this field were identified. We were • Most of the studies investigated EBRT or chemotherapy able to make specific recommendations for (see Figure). There were very few studies available on This project was funded by the UK National Health improvement. Hopefully this will be useful for local treatments with only plaque radiotherapy or Service Health Technology Assessment (HTA) clinicians evaluating treatments for retinoblastoma. brachytherapy and photocoagulation being assessed Programme (project number 03/63). The full results of • This review is currently being updated as a Cochrane as individual treatments. There were no comparative this systematic review will be published as a HTA review. Only controlled studies will be eligible for studies assessing the effectiveness of cryotherapy, monograph. The views and opinions expressed here inclusion. It is unlikely that any studies will be found thermotherapy or chemothermotherapy. No studies are those of the authors and do not necessarily reflect that meet these stricter inclusion criteria. compared different local treatments. those of the Department of Health.

For further details contact Dr. Catriona Mc Daid [email protected]

Promoting the use of research based knowledge Centre for Reviews and Dissemination