Usefulness of Systematic Review Search Strategies in Finding Child Health Systematic Reviews in MEDLINE

ARTICLE Usefulness of Systematic Review Search Strategies in Finding Child Health Systematic Reviews in MEDLINE

Nicole Boluyt, MD, PhD; Lisa Tjosvold, BA, MLIS; Carol Lefebvre, MSc; Terry P. Klassen, MD, MSc, FRCPC; Martin Offringa, MD, PhD

Objective: To determine the sensitivity and precision to 618 053. Precision for the combined filters ranged from of existing search strategies for retrieving child health sys- 2% to 52%. Because of poor reporting of specific system- tematic reviews in MEDLINE using PubMed. atic review criteria in both titles and abstracts, in 25% of the records screened we were unsure whether the article Design: Filter (diagnostic) accuracy study. We identi- concerned a systematic review according to our definition. fied existing search strategies for systematic reviews, combined them with a filter that identifies articles relevant Conclusions: The high numbers of records yielded by sen- to child health, and applied the combination in MEDLINE sitive search strategies and the low precision threaten the to a reference set of child health systematic reviews. use of systematic reviews for clinical decision making and guideline development. Reporting of specific systematic re- Main Outcome Measures: Total number of records view criteria in titles and abstracts is poor, and reporting retrieved, sensitivity, and precision. recommendations given by Quality of Reporting of Meta- analyses (QUOROM) should be used more strictly. To make Results: We tested 9 search filters. Sensitivity of the systematic review filters combined with the child filter ranged identification using MEDLINE easier, there is an urgent need from 68% to 96%; sensitivity of the child filter alone was to set minimal criteria that any review should fulfill for it 98%. The number of records retrieved with PubMed (lim- to be indexed as a systematic review. ited to January 1990-January 2006) by the systematic review filters combined with the child filter ranged from 7861 Arch Pediatr Adolesc Med. 2008;162(2):111-116

ELL-CONDUCTED SYS- views). Journal reviews can also be found tematic reviews in the Cochrane Library Database of Ab- provide the best evi- stracts of Reviews of Effects (DARE) and dence to guide clini- are available from the Centre for Reviews cal practice, are cor- and Dissemination Web site.5 The DARE nerstonesW for the recommendations of database provides quality assessments of evidence-based practice guidelines, and systematic reviews published in journals af- should be an integral part of the planning ter 1994, identified by regular searches of of future research.1,2 In contrast to tradi- important bibliographic databases such as tional narrative reviews, systematic re- MEDLINE, EMBASE, and CINAHL and by Author Affiliations: Center for views of the literature address a well- hand searching some of the major general Pediatric Clinical Epidemiology, defined question, use an explicit search journals but no specific pediatric jour- Emma Children’s Hospital, strategy to locate all relevant evidence, nals. Bibliographic databases such as Academic Medical Centre, evaluate the retrieved studies using pro- MEDLINE index Cochrane reviews and University of Amsterdam, spectively defined methodological crite- can be used to identify other systematic Amsterdam, the Netherlands ria, and formally synthesize the results.3,4 reviews, but only those indexed in the re- (Drs Boluyt and Offringa); Clinicians and researchers should be able spective database. Finding systematic re- Alberta Research Centre for to reliably and quickly find systematic reviews in MEDLINE poses 2 challenges. Child Health Evidence, views. Two types of systematic reviews can First, only a fraction of all citations in Department of Pediatrics, be identified in current bibliographic da- MEDLINE are for systematic reviews and, University of Alberta, Edmonton, Alberta, Canada tabases: (1) the Cochrane systematic re- second, the MEDLINE indexing proce- (Ms Tjosvold and Dr Klassen); views, which can be found in the Coch- dures do not include “systematic review” and United Kingdom Cochrane rane Database of Systematic Reviews, the as a “publication type.” To limit the search Centre, Oxford, England Cochrane Library and (2) systematic re- results from a query in MEDLINE, there- (Ms Lefebvre). views published in journals (journal re- fore, it is recommended that a method-

(REPRINTED) ARCH PEDIATR ADOLESC MED/ VOL 162 (NO. 2), FEB 2008 WWW.ARCHPEDIATRICS.COM 111

©2008 American Medical Association. All rights reserved. Downloaded From: https://jamanetwork.com/ on 09/27/2021 EXISTING SEARCH STRATEGIES FOR Table 1. Reference Standard: Number of Systematic SYSTEMATIC REVIEWS AND CHILD STUDIES Reviews Found by Hand Searching 7 Pediatric Journals With a Range of Impact Factors for 1994, 1997, 2000, 2002, To identify articles reporting on the development and valida- and 2004 and Number of Child Health Systematic Reviews tion of systematic review search filters in MEDLINE, we searched Included in DARE and Indexed in MEDLINE MEDLINE from January 1995 to January 2006 with the following MeSH terms: MEDLINE, Information Storage and Retrieval/ No. of Methods, and Review, literature. In addition, reference lists of Systematic relevant articles were reviewed and content experts were con- Impact Reviews or Journal Factor Meta-analyses tacted to find additional studies. To improve precision, we combined the systematic review filters with a sensitive child filter Pediatrics 3.4 36 developed by the Cochrane Child Health Field10 to retrieve only Journal of Pediatrics 3.2 6 studies in children. Pediatric Infectious Diseases Journal 2.4 10 Archives of Diseases in Childhood 2.1 34 Archives of Pediatric and 2.1 20 REFERENCE STANDARD Adolescent Medicine European Journal of Pediatrics 1.2 5 To measure the sensitivity of the search strategies, a reference Journal of Perinatal Medicine 0.9 4 standard set of systematic reviews was established by search- Total found by hand searching 115 ing for child health systematic reviews in DARE5 and by hand DAREa 298 searching several pediatric journals for systematic reviews. All Overlapb 26 titles and abstracts in DARE (Cochrane Library, Issue 2, 2004) Total Reference Standard 387 were screened to find child health systematic reviews also indexed in MEDLINE. We hand searched 7 pediatric journals in- Abbreviation: DARE, Database of Abstracts of Reviews of Effects. a dexed in MEDLINE with a range of impact factors and for which Cochrane Library, Issue 2, 2004; reviews also indexed in MEDLINE. full-text electronic copies were available from our medical li- b Records found in both DARE up to Issue 2, 2004, and 1 of the journals hand searched. In 2006, 52% of the journal reviews on prevention, brary (Table 1). All issues of each journal were searched for intervention, or diagnosis were also indexed in DARE. the following 5 years: 1994, 1997, 2000, 2002, and 2004. In addition, we were interested as to whether DARE contained all child health systematic reviews found by our hand search of pediatric journals. Inasmuch as DARE includes only system- ological filter be used consisting of text words and Medi- atic reviews on prevention, intervention, or diagnosis, we se- cal Subject Headings of the US National Library of lected systematic reviews covering these domains. Next, we Medicine (MeSH) directed to general indicators of sys- searched DARE in July 2006 using the Cochrane Library in- tematic reviews in the MEDLINE record. terface to determine how many child health systematic reviews on prevention, intervention, or diagnosis found by our Several search strategies for locating systematic re- hand search were also included in DARE. views in MEDLINE have been developed and vali- dated6-9; however, to our knowledge, the performance of SENSITIVITY these filters has never been evaluated in the universe of all articles included in MEDLINE, nor have they been Because the child filter has not been validated, we measured tested for sensitivity and precision in finding child health the sensitivity of the child filter separately. The sensitivity of a systematic reviews. search strategy was defined as the percentage of child health The objective of the present study was to assess the systematic reviews retrieved from our reference standard set usefulness of existing search filters in finding child health of child health systematic reviews. systematic reviews in MEDLINE using the PubMed in- Sensitivity=(Number of systematic reviews retrieved from terface by applying them to a reference standard of child reference standard/Total number of systematic reviews in health systematic reviews and by determining whether reference standard)ϫ100 these filters focus the search strategy sufficiently to be Sensitivity was measured for all systematic review filters sepa- practical in the universe of MEDLINE. To this goal, the rately and in combination with the child filter. percentage of child health systematic reviews retrieved from our reference standard (sensitivity) was measured PRECISION and the number of true child health systematic reviews yielded by the searches (precision) was estimated. The precision of a combined search strategy was defined as the percentage of true child health systematic reviews found in METHODS MEDLINE (limits 1990-2006) divided by the total number of records retrieved by the search. INCLUSION AND EXCLUSION CRITERIA Precision=(Number of true systematic reviews/ All records in MEDLINE retrieved by the search)ϫ100 Herein, we use the term systematic review to refer to any lit- We estimated precision rather than calculated precision be- erature review, meta-analysis, or other article that explicitly in- cause the yield of each combined search strategy was large, rang- dicates the use of a strategy for locating evidence by mention- ing from 7861 records (Montori 2 search filter) to 618 053 rec- ing at least the databases that were searched and reviewing the ords (Montori 1 search filter). To estimate precision, we selected empirical evidence on children (aged Ͻ18 years). Excluded were a random sample of 100 records from the yield of each com- systematic reviews that assessed the effects on the fetus of in- bined search strategy. The random sample was generated using terventions in pregnant women. R software11 and stratified by year of publication to account for

(REPRINTED) ARCH PEDIATR ADOLESC MED/ VOL 162 (NO. 2), FEB 2008 WWW.ARCHPEDIATRICS.COM 112

©2008 American Medical Association. All rights reserved. Downloaded From: https://jamanetwork.com/ on 09/27/2021 Table 2. Reported Performance and Performance in Our Reference Standard of Combined Systematic Review Filter With Child Filter

Performance in Reported Performancea Our Reference Standard Performance In PubMed

Search Filter Sensitivity, % Precision, % Sensitivity, % (95% CI) Precision, % (95% CI)b Shojania plus child 93 50c 74 (69-78) 45 (36-55) Boynton plus child 98 20 95 (92-97) 3 (1-9) White 1 plus child 100 4 93 (91-96) White 2 plus child 94 10 94 (91-96) 2 (1-7) Montori 1 plus child 100 3 96 (93-97) Montori 2 plus child 71 57 68 (64-73) 52 (42-62) Montori 3 plus child 98 14 94 (91-96) 3 (1-9) Montori 4 plus child 74 69 72 (67-76) PubMed plus child Unknown Unknown 76 (72-80) 32 (24-42)

Abbreviation: CI, confidence interval. a Reported performance without child filter. Sensitivity of child filter, 98% (95% confidence interval, 96-99). All numbers have been rounded. b Precision not estimated for White 1, Montori 1, and Montori 4 filters because other filters with similar sensitivity have lower numbers of hits. c Applied to the following 3 clinical topics: screening for colorectal cancer, thrombolytic therapy for venous thromboembolism, and treatment of dementia.

changes in MeSH indexing with time. Each sample was screened of 431 child health systematic reviews were identified in independently by 2 investigators (N.B. and L.T.) for true sys- DARE (Cochrane Library, Issue 2, 2004; N=4645), of tematic reviews. We considered an article a true systematic re- which only 298 were indexed in MEDLINE. Hand search- view only if the title, abstract, author-supplied key words, or ing MEDLINE-indexed pediatric journals yielded 115 publication type terms explicitly identified the article as a sys- child health systematic reviews, of which 26 records were tematic review or meta-analysis or if the article abstract indi- cated a strategy for locating the literature reviewed and in- also included in DARE (Cochrane Library, up to Issue cluded children. We classified records as unsure if the title or 2, 2004). Of our hand-searched 115 reviews, 73 re- abstract did not specifically state whether children were in- ported on prevention, intervention, or diagnosis. A 2006 cluded, it was stated that a literature search was performed but search of DARE revealed that 38 of these 73 systematic without stating which databases were searched, or it was stated reviews on prevention, intervention, or diagnosis (52%) that the article was an evidence-based overview. In all of these were also indexed in DARE. unsure cases, the full text of the article was retrieved and a fi- nal decision was made as to whether it was a true systematic SENSITIVITY review. Next, we applied the search strategies to 7 priority topics in child health, developed by the Cochrane Child Health Field, that necessitate systematic reviews. The number of rec- Sensitivity of the child filter when run separately in ords retrieved for the most sensitive and the most precise sys- MEDLINE and tested against our reference standard was tematic review search strategies is reported, as is the precision 98% (380/387; 95% confidence interval, 96%-99%). Sen- for the most precise filter. The search strings for each clinical sitivity of the systematic review filters ranged from 68% topic were created by using the MeSH browser function.12 We to 96% (Table 2). reported the time it takes a pediatrician (N.B.) and a clinical librarian (L.T.) to screen 100 MEDLINE records. PRECISION

RESULTS Precision Not Limited to Clinical Topics

EXISTING SEARCH STRATEGIES Figure 1 shows the number of records retrieved with the FOR IDENTIFYING SYSTEMATIC REVIEWS systematic review filters in PubMed (1990-2006) with and without the child filter added. The number of records re- We identified 4 studies reporting on the development and trieved with the systematic review filters ranged from more validation of several systematic review search strategies than 38 000 (Montori 2) to more than 3 million (Montori for MEDLINE.6-9 We evaluated 8 of the reported search 1) and for the combined filters from more than 8000 (Mon- strategies with varying sensitivity and precision and the tori 2) to more than 618 000 (Montori 1). PubMed Clinical Queries systematic review filter.13 De- We screened 600 titles and abstracts for systematic re- tails of the systematic review search strategies (trans- views (a random sample of 100 records retrieved with 6 lated into PubMed format where necessary) and the child systematic review filters combined with the child filter) health search strategy are provided in the online supple- (Table 2). We were unsure in 150 records whether the ment (eBox at http://www.archpediatrics.com). article was a true systematic review and in 15 of those whether children were included in the review. For all of REFERENCE STANDARD these unsure results, full text was retrieved, which even- tually led to a yield of 137 of the 600 reviews being true Our reference standard contained 387 child health sys- systematic reviews in children. Precision ranged from 2% tematic reviews indexed in MEDLINE (Table 1). A total to 52% in our study with the combined systematic re-

(REPRINTED) ARCH PEDIATR ADOLESC MED/ VOL 162 (NO. 2), FEB 2008 WWW.ARCHPEDIATRICS.COM 113

× 618 550 500

400 381 324

No. of Records, 300

200 169 151 91 92 100 50 38 10 8 16 18 0 Shojania Boynton 121234 White Montori PubMed Type of Search Filter

Figure 1. Number of records retrieved through PubMed (January 1990-January 2006) for the systematic review filters with and without the child filter. All numbers have been rounded.

4000 Shojania and child 3178 filters 3000 Montori 3 and child 1913 2000 filters 1216 1000 608

400 352

300 261 No. of Records 232 222 200

129 120 100 61 29 24 20 0 Asthma Obesity ADHD UTI Enuresis Bronchiolitis Constipation Topic

Figure 2. Number of records retrieved when applying a precise (Shojania) and a sensitive (Montori 3) systematic review filter, combined with the child filter, in PubMed (January 1990-January 2006) to clinical topics. ADHD indicates attention-deficit/hyperactivity disorder; UTI, urinary tract infection.

view and child filters, compared with 3% to 69% re- COMMENT ported in the original studies (Table 2). On average, it took the pediatrician 34 minutes and the clinical librarian 47 minutes to screen the titles and abstracts of 100 Although the DARE database contains quality assess- retrieved records. ments of systematic reviews published in peer-reviewed journals, DARE includes only 52% of our hand- Precision Limited to Clinical Topics searched child health journal reviews. However, DARE uses more stringent inclusion criteria for systematic re- Figure 2 shows the number of records retrieved when views than we did.5 Although DARE was created in Feb- applying a sensitive and a precise systematic review fil- ruary 1995, all hand-searched child health journal re- ter in combination with the child filter to 7 different clini- views published in 1994 were indexed in DARE. cal topics. Adding a MeSH term for asthma yielded the The Cochrane Database of Systematic Reviews and largest number of records, and adding a MeSH term for DARE are already prefiltered in their focus on system- constipation yielded the lowest number (Shojania filter, atic reviews; thus, the only task is to find topic-relevant 352 vs 20; Montori 3 filter, 3178 vs 222). Precision with systematic reviews in these databases. In general medi- the combined Shojania plus child filter ranged from 49% cal databases such as MEDLINE, systematic reviews for obesity to 93% for enuresis. On average, it took the should ideally be indexed using an appropriate and spe- pediatrician 18 minutes to screen the titles and ab- cific publication type term. Until there is an agreed defi- stracts of 100 topic-specific records for systematic nition of a systematic review, indexers will be unable to reviews. introduce or apply such a term. Pending this, we will need

(REPRINTED) ARCH PEDIATR ADOLESC MED/ VOL 162 (NO. 2), FEB 2008 WWW.ARCHPEDIATRICS.COM 114

©2008 American Medical Association. All rights reserved. Downloaded From: https://jamanetwork.com/ on 09/27/2021 more precise search strategies to find systematic re- It was often difficult to determine from the title and views in MEDLINE to reduce the number of irrelevant abstract whether an article was a true systematic review. records. Abstracts often lacked descriptions of important criteria The usefulness of a systematic review filter is ex- for systematic reviews. Instead, general terms were used, pressed by sensitivity and precision. The trade-off be- such as “A literature review was performed,” “We tried tween these 2 features will determine the choice of fil- to collect all published literature,” “This is an evidence- ter. Researchers conducting a new systematic review or based overview,” and “We performed a qualitative as- guideline developers would best be served by the most sessment of the literature.” Therefore, it is to be ex- sensitive search. This search will have the highest prob- pected that the most sensitive systematic review filters ability of retrieving all relevant reviews, but it will have also retrieve many irrelevant records. In 1999, the Qual- low precision, retrieving many irrelevant articles. Those ity of Reporting Meta-analyses (QUOROM) statement was with little time on their hands (for example, clinicians published to improve the quality of reporting meta- looking for answers to patient care questions) will likely analyses or systematic reviews of clinical, randomized, be best served by a more precise search strategy. The sen- controlled trials.14 To be able to differentiate narrative sitivity of the systematic review filters as reported in the reviews from systematic reviews, we advise that authors original studies and compared with our reference stan- follow these recommendations when writing abstracts. dard were similar except for the Shojania filter (93% vs In addition, when children are included, this should be 74%, respectively). Shojania’s reference standard set of explicitly reported in the title or abstract. To make iden- papers mainly included high-quality systematic reviews tification using PubMed easier, there is an urgent need that probably contained consistent descriptions of sys- to set minimal criteria that any review should fulfill for tematic review criteria in the abstracts’ “Methods” sec- it to be indexed as a systematic review. Ideally, a data- tions. A filter developed in such a sample will probably base of child health systematic reviews should be cre- consist of another combination of text words and MeSH ated. Pending this, validated child health filters are needed headings than would be optimal to retrieve systematic to be able to retrieve all child health–relevant articles. Since reviews in our reference standard, which includes both this study was conducted, a study has been published that low- and high-quality systematic reviews. determined the retrieval characteristics of age-specific Because the sensitivity of the child filter alone was ex- terms in MEDLINE (Ovid interface; Ovid Technologies, cellent (98%), we combined the systematic review fil- Inc, New York, New York) for pediatric and neonatal ters with the child filter to improve precision. Inasmuch medicine.15 as few systematic reviews were published before 1990, we limited our search to 1990 and after. The number of LIMITATIONS records retrieved with the various systematic review filters was in the hundreds of thousands. Adding the child We used a broad definition of a systematic review, and filter reduced the number of records by a factor range of a number of these reviews will not fulfill some of the 5 to 7. Of the sensitive search filters, the Montori 3 has stringent criteria for systematic reviews used by others. the best trade-off between sensitivity (94%), number of Given the amount of time needed to perform a system- records retrieved (151 227), and precision (3%). Of the atic review and the methodological skills required, our more precise filters, the Shojania filter has the best trade- goal was not to exclude potentially relevant systematic off between sensitivity (74%), number of records re- reviews. The explicit mention of a strategy for locating trieved (10 188), and precision (45%). Although we added evidence seems to be the most basic and least contro- a child filter to improve precision and used a broad defi- versial.7 nition of a systematic review, precision was lower than Our reference standard contains a subset of the reported in the original studies for which we have com- MEDLINE database because it is impossible to hand search parative precision data. However, this is to be expected all journals indexed in MEDLINE for child health sys- because precision changes with the prevalence of posi- tematic reviews, and sensitivity could have been overes- tive results (or richness of the database). The preva- timated or underestimated. We tried to compose a rep- lence of child health systematic reviews in MEDLINE is resentative reference standard of systematic reviews by lower than the prevalence of adult systematic reviews. hand searching 7 pediatric journals with a range of im- In addition, most existing search strategies were devel- pact factors. This should help avert selection bias in our oped from a small, selective subset of systematic research filter accuracy study because of higher standards views published in high-impact journals. Precision was of study description (eg, by the use of a structured ab- also estimated from their subsets of journals. Clinicians stract with an explicit “Methods” section) in high- searching for systematic reviews in MEDLINE using the impact journals. In an attempt to avert possible biases filters search the universe of MEDLINE and not a spe- occurring in any one year, we sampled records from vari- cific set of journals. Therefore, we believe that clini- ous years within a 10-year range. cians will be interested in precision in the universe of All systematic review filters except the Shojania filter MEDLINE. Shojania and Bero8 calculated precision in were developed and tested using the Ovid interface. Be- MEDLINE for 3 clinical topics (Table 2). As we have cause PubMed is the only MEDLINE free interface avail- shown, precision varies considerably among different able worldwide and is widely used, we converted the Ovid clinical topics and seemed to be higher than precision search filters into PubMed format.6,7,9 Translating a search not limited to clinical topics. Therefore, the results will filter from Ovid format to PubMed format is a factor that not be generalizable. may influence the performance of the filter.

(REPRINTED) ARCH PEDIATR ADOLESC MED/ VOL 162 (NO. 2), FEB 2008 WWW.ARCHPEDIATRICS.COM 115

©2008 American Medical Association. All rights reserved. Downloaded From: https://jamanetwork.com/ on 09/27/2021 and Klassen. Critical revision of the manuscript for impor- Precise Systematic Review Search Strategy tant intellectual content: Boluyt, Tjosvold, Lefebvre, and Shojania Filter Offringa. Administrative, technical, and material support: ((meta-analysis [pt] OR meta-analysis [tw] OR metaanalysis [tw]) OR ((review [pt] OR guideline [pt] OR consensus [ti] OR guideline∗ [ti] OR literature [ti] OR overview Boluyt and Tjosvold. Study supervision: Boluyt, Klassen, [ti] OR review [ti]) AND ((Cochrane [tw] OR MEDLINE [tw] OR CINAHL [tw] OR and Offringa. Information science expertise: Lefebvre. (National [tw] AND Library [tw])) OR (handsearch∗ [tw] OR search∗ [tw] OR searching [tw]) AND (hand [tw] OR manual [tw] OR electronic [tw] OR bibliography∗ Financial Disclosure: None reported. [tw] OR database∗ OR (Cochrane [tw] OR MEDLINE [tw] OR CINAHL [tw] OR Additional Information: The eBox is available at http: (National [tw] AND Library [tw]))))) OR ((synthesis [ti] OR overview [ti] OR review [ti] OR survey [ti]) AND (systematic [ti] OR critical [ti] OR methodologic [ti] OR //www.archpediatrics.com. quantitative [ti] OR qualitative [ti] OR literature [ti] OR evidence [ti] OR Additional Contributions: Marjan Loep, MLIS (de- evidence-based [ti]))) BUT NOT (case∗ [ti] OR report [ti] OR editorial [pt] OR comment [pt] OR letter [pt]) ceased), Marcel van der Paardt, PhD, and Arnold Leen- ders, MLIS, Medical Library, Academic Medical Center, Sensitive systematic review search strategy Montori 3 Filter Amsterdam, the Netherlands, searched DARE and hand Meta-analysis [ptyp] OR meta-analysis [tiab] OR meta-analysis [MESH] OR searched the pediatric journals; Grace Liang, PhD, Al- ∗ review[pryp] OR search [tiab] berta Research Centre for Child Health Evidence, Uni- Child Search Strategy versity of Alberta, Edmonton, Alberta, Canada, devel- Infant[MeSH] OR Infant∗ OR infancy OR Newborn∗ OR Baby∗ OR Babies OR Neonat∗ oped a program to enable random sample selection; and OR Preterm∗ OR Prematur∗ OR Postmatur∗ OR Child[MeSH] OR Child∗ OR Schoolchild∗ OR School age∗ OR Preschool∗ OR Kid or kids OR Toddler∗ OR Liza Bialy, BSc, Alberta Research Centre for Child Health Adolescent[MeSH] OR Adoles∗ OR Teen∗ OR Boy∗ OR Girl∗ OR Minors[MeSH] OR Evidence, University of Alberta, Edmonton, Alberta, Minors∗ OR Puberty[MeSH] OR Pubert∗ OR Pubescen∗ OR Prepubescen∗ OR Pediatrics[MeSH] OR Paediatric∗ OR Paediatric∗ OR Peadiatric∗ OR Schools[MeSH] screened records for true systematic reviews. None re- OR Nursery school∗ OR Kindergar∗ OR Primary school∗ OR Secondary school∗ OR ceived compensation for their work. Elementary school∗ OR High school∗ OR Highschool∗

Figure 3. Recommended precise and sensitive PubMed systematic review and child search strategies. REFERENCES

RECOMMENDATIONS FOR CLINICIANS 1. Haines A, Jones R. Implementing findings of research. BMJ. 1994;308(6942):1488- AND RESEARCHERS 1492. 2. Scholten RJ, Kremer LC. Systemic reviews as a basis for guidelines [in Dutch]. Ned Tijdschr Geneeskd. 2004;148(6):262-264. We advise busy clinicians searching for systematic re- 3. Bero LA, Grilli R, Grimshaw JM, Harvey E, Oxman AD, Thomson MA; Cochrane views on health care interventions to search the Coch- Effective Practice and Organization of Care Review Group. Closing the gap be- rane Library (Cochrane Database of Systematic Reviews tween research and practice: an overview of systematic reviews of interventions to promote the implementation of research findings. BMJ. 1998;317(7156): and DARE) for high-quality systematic reviews. Several 465-468. studies have shown that Cochrane reviews are more rig- 4. Oxman AD, Cook DJ, Guyatt GH; Evidence-Based Medicine Working Group. orous than journal reviews.16,17 If no relevant systematic Users’ guides to the medical literature: VI, How to use an overview. JAMA. 1994; reviews are found in the Cochrane Library or the inter- 272(17):1367-1371. est is other than prevention, treatment, or diagnosis, other 5. Centre for Reviews and Dissemination, University of York, York, England. http: //www.york.ac.uk/inst/crd/crddatabases.htm. Accessed December 1, 2006. databases such as MEDLINE should be searched next. 6. Boynton J, Glanville J, McDaid D, Lefebvre C. Identifying systematic reviews in For a specific search, the Shojania filter combined with MEDLINE: developing an objective approach to search strategy design. J Inf Sci. the child filter can be used. For topics for which not many 1998;24:137-157. doi:10.1177/016555159802400301. reviews are available (eg, constipation), it may be more 7. Montori VM, Wilczynski NL, Morgan D, Haynes RB; Hedges Team. Optimal search strategies for retrieving systematic reviews from Medline: analytical survey. BMJ. efficient to search with the sensitive Montori 3 filter com- 2005;330(7482):68-73. bined with the child filter. Researchers and guideline de- 8. Shojania KG, Bero LA. Taking advantage of the explosion of systematic reviews: velopers interested in finding nearly all systematic re- an efficient MEDLINE search strategy. Eff Clin Pract. 2001;4(4):157-162. views can use the Montori 3 filter combined with the child 9. White V, Glanville J, Lefebvre C, Sheldon T. A statistical approach to designing filter. Readers can save the search filters (Figure 3) they search filters to find systematic reviews: objectivity enhances accuracy. J Inf Sci. 2001;27:357-370. doi:10.1177/016555150102700601. wish to use in PubMed and use them in combination with 10. Cochrane Child Health Field. www.cochranechildhealth.ualberta.ca/reviews a clinical topic. .html. Accessed December 1, 2006. 11. R Development Core Team R version 2.3.1. Vienna, Austria: R Foundation for Accepted for Publication: July 13, 2007. Statistical Computing; 2006. http://www.R-project.org. Accessed June 16, 2006. 12. Shaw RL, Booth A, Sutton AJ, et al. Finding qualitative research: an evaluation Correspondence: Nicole Boluyt, MD, Center for Pedi- of search strategies. BMC Med Res Methodol. 2004;4:5. doi: 10.1186 atric Clinical Epidemiology, Room H3-145, Emma Chil- /1471-2288-4-5. dren’s Hospital, Academic Medical Centre, PO Box 22700, 13. PubMed Clinical Queries: Systematic Reviews. http://www.ncbi.nlm.nih.gov/entrez 1100 DD Amsterdam, the Netherlands (n.boluyt@amc /query/static/clinical.shtml#reviews. Accessed December 17, 2006. .uva.nl). 14. Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D, Stroup DF. Quality of reporting of meta-analyses. Lancet. 1999;354(9193):1896-1900. Author Contributions: Dr Boluyt had full access to all 15. Kastner M, Wilczynski NL, Walker-Dilks C, McKibbon KA, Haynes B. Age- of the data in the study and takes responsibility for the specific search strategies for Medline. J Med Internet Res. 2006;8(4):e25. integrity of the data and the accuracy of the data analy- 16. Jadad AR, Cook DJ, Jones A, et al. Methodology and reports of systematic re- sis. Study concept and design: Boluyt, Tjosvold, Lefebvre, views and meta-analyses: a comparison of Cochrane reviews with articles published in paper-based journals. JAMA. 1998;280(3):278-280. Klassen, and Offringa. Acquisition of data: Boluyt and 17. Jadad AR, Moher M, Browman GP, et al. Systematic reviews and meta-analyses on Tjosvold. Analysis and interpretation of data: Boluyt, treatment of asthma: critical evaluation [published corrections appear in BMJ. 2000; Lefebvre, and Offringa. Drafting of the manuscript: Boluyt 320(7240):984 and 2000;321(7256):275]. BMJ. 2000;320(7234):537-540.

(REPRINTED) ARCH PEDIATR ADOLESC MED/ VOL 162 (NO. 2), FEB 2008 WWW.ARCHPEDIATRICS.COM 116

Systematic Review Search Strategies and Child Search Strategy

SYSTEMATIC REVIEW SEARCH STRATEGIES Shojania ((meta-analysis [pt] OR meta-analysis [tw] OR metaanalysis [tw]) OR ((review [pt] OR guideline [pt] OR consensus [ti] OR guideline* [ti] OR literature [ti] OR overview [ti] OR review [ti]) AND ((Cochrane [tw] OR Medline [tw] OR CINAHL [tw] OR (National [tw] AND Library [tw])) OR (handsearch* [tw] OR search* [tw] OR searching [tw]) AND (hand [tw] OR manual [tw] OR electronic [tw] OR bibliographi* [tw] OR database* OR (Cochrane [tw] OR Medline [tw] OR CINAHL [tw] OR (National [tw] AND Library [tw]))))) OR ((synthesis [ti] OR overview [ti] OR review [ti] OR survey [ti]) AND (systematic [ti] OR critical [ti] OR methodologic [ti] OR quantitative [ti] OR qualitative [ti] OR literature [ti] OR evidence [ti] OR evidence-based [ti]))) BUTNOT (case* [ti] OR report [ti] OR editorial [pt] OR comment [pt] OR letter [pt]) Boynton (Most Sensitive Strategy) ((Meta[tiab] NOT meta[ti]) OR (synthesis[tiab] NOT synthesis[ti]) OR (literature[tiab] NOT literature[ti]) OR (randomized[tw] NOT randomized[tiab]) OR published[tiab] OR Meta-Analysis[ptyp] OR extraction[tiab] OR (Trials[tw] NOT Trials[ti]) OR (controlled[tw] NOT controlled[tiab]) OR (search[tiab] NOT search[ti]) OR (medline[tiab] NOT medline[ti]) OR (selection[tiab] NOT selection[ti]) OR (sources[tiab] NOT sources[ti]) OR (review[tiab] NOT review[ti]) OR review[ptyp] OR articles[tiab] OR (reviewed[tiab] NOT reviewed[ti]) OR (english[tiab] NOT english[ti]) OR (language[tiab] NOT language[ti])) NOT (Letter[ptyp] OR comment[ptyp] OR editorial[ptyp]) White 1 (Most Sensitive Strategy) ((Controlled[tiab] NOT controlled[ti]) OR (design[tiab] NOT design[ti]) OR (evidence[tiab] NOT evidence[ti]) OR (extraction[tiab] NOT extraction[ti]) OR “randomized controlled trials”[MeSH] OR Meta-Analysis[ptyp] OR Review[ptyp] OR (sources[tiab] NOT sources[ti]) OR (studies[tiab] NOT studies[ti])) NOT (Letter[ptyp] OR comment[ptyp] OR editorial[ptyp]) White 2 (Most Precise Strategy) ((Review[tiab] NOT review[ti]) OR Review[ptyp] OR meta-analysis[tiab] OR Meta-Analysis[ptyp]) NOT (Letter[ptyp] OR comment[ptyp] OR editorial[ptyp]) Montori 1 (Most Sensitive Strategy) search*[tiab] OR meta analysis[ptyp] OR meta-analysis[tiab] OR meta analysis[MeSH] OR review[ptyp] OR diagnosis[MeSH Subheading] OR associated[tiab] Montori 2 (Most Precise Strategy) Medline[tiab] OR (systematic[tiab] AND review[tiab]) OR meta-analysis[ptyp] Montori 3 (Minimizing the Difference Between Sensitivity and Specificity) Meta-analysis[ptyp] OR meta-analysis[tiab] OR meta-analysis[MeSH] OR review[ptyp] OR search*[tiab] Montori 4 (Combining Most Precise Term With Most Sensitive Terms) Cochrane Database Syst Rev [ta] OR search[tiab] OR meta-analysis[ptyp] OR; medline[tiab] OR (systematic[tiab] AND review[tiab]) PubMed (Systematic review* [tiab] OR systematic literature review* OR meta-analysis [ptyp] OR meta-analysis [ti] OR metaanalysis [ti] OR meta-analyses [ti] OR evidence-based medicine OR (evidence-based AND (guideline [tiab] OR guidelines [tiab] OR recommendations)) OR (evidenced-based AND (guideline [tiab] OR guidelines [tiab] OR recommendation*)) OR consensus development conference [ptyp] OR health planning guidelines OR guideline[ptyp] OR cochrane database syst rev OR acp journal club OR health technol assess OR evid rep technol assess summ OR evid based dent OR evid based nurs OR evid based ment health OR clin evid) OR ((systematic [tiab] OR systematically OR critical [tiab] OR (study [tiab] AND selection [tiab]) OR (predetermined OR inclusion AND criteri*) OR exclusion criteri* OR “main outcome measures” OR “standard of care” OR “standards of care”) AND (survey [tiab] OR surveys [tiab] OR overview* OR review [tiab] OR reviews [tiab] OR search* OR handsearch OR analysis [tiab] OR critique [tiab] OR appraisal OR (reduction AND risk AND (death OR recurrence))) AND (literature [tiab] OR articles [tiab] OR publications [tiab] OR publication [tiab] OR bibliography [tiab] OR bibliographies [tiab] OR published [tiab] OR unpub- lished OR citation OR citations OR database [tiab] OR internet [tiab] OR textbooks [tiab] OR references OR trials [tiab] OR meta- analysis [mh] OR (clinical [tiab] AND studies [tiab]) OR treatment outcome)) NOT (case report [ti] OR editorial [ti] OR editorial [ptyp] OR letter [ptyp] OR newspaper article [ptyp]) Child Search Strategy Infant[MeSH] OR Infant* OR infancy OR Newborn* OR Baby* OR Babies OR Neonat* OR Preterm* OR Prematur* OR Postmatur* OR Child[MeSH] OR Child* OR Schoolchild* OR School age* OR Preschool* OR Kid or kids OR Toddler* OR Adolescent[MeSH] OR Adoles* OR Teen* OR Boy* OR Girl* OR Minors[MeSH] OR Minors* OR Puberty[MeSH] OR Pubert* OR Pubescen* OR Prepubes- cen* OR Pediatrics[MeSH] OR Paediatric* OR Paediatric* OR Peadiatric* OR Schools[MeSH] OR Nursery school* OR Kindergar* OR Primary school* OR Secondary school* OR Elementary school* OR High school* OR Highschool*

(REPRINTED) ARCH PEDIATR ADOLESC MED/ VOL 162 (NO. 2), FEB 2008 WWW.ARCHPEDIATRICS.COM E1