Systematic Reviews: Consolidating Research Evidence for EBP 51

© Jones & Bartlett Learning, LLC CHAPTER© Jones & Bartlett Learning, LLC NOT FOR SALE OR DISTRIBUTION NOT FOR SALE OR DISTRIBUTION 3 © Jones & Bartlett Learning, LLC © Jones & Bartlett Learning, LLC NOT FOR SALE OR DISTRIBUTION NOT FOR SALE OR DISTRIBUTION Systematic Reviews: Consolidating © Jones & Bartlett Learning, LLC © Jones & Bartlett Learning, LLC NOT FOR SALEResearch OR DISTRIBUTION EvidenceNOT for FOR EBPSALE OR DISTRIBUTION

THEODORA D. KWANSA

© Jones & Bartlett Learning, LLC © Jones & Bartlett Learning, LLC NOT FOR SALE OR DISTRIBUTION NOT problem/phenomenonFOR SALE OR DISTRIBUTION and the related review question(s) might relate to the effects of clinical intervention, healthcare CHAPTER oBJECTIVES policy, or the effectiveness of specifi c aspects of professional The main objectives of this chapter are to: practice. Identifying, compiling, critiquing, summarizing, and condensing the best available evidence and synthesiz- • Explore the significance, rationale, and benefits © Jones & Bartlett Learning, LLCing these compose the stages© ofJones systematic & Bartlett review. As Learning, the LLC of systematic reviews NOT FOR SALE OR DISTRIBUTIONname suggests, systematic reviewsNOT areFOR rigorous SALE in ORterms DISTRIBUTION of • Examine the process of reviewing, condensing, following exact processes, transparent in terms of being and summarizing research evidence applicable to different contexts in actual situations, and • Describe the processes of data extraction and repeatable. They provide cumulative evidence from the abstraction fi ndings of pertinent studies that policy makers and clinical ©• Jones Identify the& Bartlettprocesses ofLearning, quantitative LLCresearch practitioners© canJones draw &on. Bartlett Thus, systematic Learning, reviews LLC can be NOTsynthesis, FOR SALEmeta-analysis, OR DISTRIBUTION and the PRISMA used to employNOT evidence-based FOR SALE bestOR practice,DISTRIBUTION designed to statement achieve positive outcomes and patient satisfaction. Other • Explore the approaches to synthesizing qualitative benefi ts include helping to inform and direct policy, and research evidence and the related terminologies clinical decision making, and the development of standards • State the need for systematic reviews, meta- to ensure correct implementation of recommended guidelines. © Jones & Bartlettanalysis, Learning, and qualitative LLC research synthesis © JonesHemingway & Bartlett and Learning, Brereton (2009)LLC identifi ed specifi c NOT FOR SALE OR DISTRIBUTION NOT professionalsFOR SALE and OR stakeholders DISTRIBUTION who may require sound and trustworthy information at varied times, on a considerable range of interventions and aspects of healthcare delivery. Introduction Healthcare practitioners—clinicians, nurses, therapists, and healthcare managers—as well as policy makers, patients, and This chapter focuses on systematic reviews of multiple © Jones & Bartlett Learning, LLCconsumer representatives may,© at Jones some time & Bartlettor another, Learning,seek LLC studies with similar design(s) and method(s) and combining specifi c evidence-based information on clinical interventions and condensing the fiNOT ndings FOR for evidence-based SALE OR DISTRIBUTION support NOT FOR SALE OR DISTRIBUTION or aspects of the healthcare policy, such as the effects, prac- of professional practice. Meta-analysis, which allows for ticality, signifi cance, and the relevance of certain aspects of pooling and interpreting emerging concepts to create new professional practice or a specifi c treatment. These authors theories, is also explored. note that systematic reviews help to ease the additional demand © Jones & Bartlett Learning, LLC on practitioners© Jones who may & Bartlettbe inundated Learning, with several LLC reports TheNOT Rationale FOR SALEand Advantages OR DISTRIBUTION and emergingNOT new FOR fi ndings SALE from ORnumerous DISTRIBUTION research studies. of Systematic Reviews for EBP Practitioners constantly face the challenge and professional expectation to keep up to date with ongoing advances in sci- Generally, systematic reviews are informative overviews of entifi c techniques and new procedural interventions. Emerging carefully selected primary research studies that have rigor- new ideas often involve implementing changes in policy, clini- © Jones &ously Bartlett applied Learning,specifi c research LLC design(s), methodology, and© Jonescal decisions, & Bartlett and guidelines Learning, (Brown LLC et al., 2006; Crombie & NOT FORmethods SALE to OR investigate DISTRIBUTION a clinical problem/phenomenon. ThisNOT Davies,FOR SALE2009; Petticrew, OR DISTRIBUTION 2003; Petticrew & Roberts, 2006).

9781284114942_CH03.indd 49 06/11/17 2:24 pm 50 PART I Exploration of the Concept of Evidence-Based Practice and Related Practical Challenges

© Jones & Bartlett Learning, LLC © Jones & Bartlett Learning, LLC Without a doubt, information from a large amount of Despite the above observations, as Petticrew (2003) NOT FORresearch SALE is easierOR DISTRIBUTION to grasp when condensed in a systematicNOT remarked, FOR SALE systematic OR DISTRIBUTIONreviews are often criticized for not review (Hemingway & Brereton, 2009). Earlier proponents, incorporating precise guidance on what indicates evidence including Oxman (1993), maintained that the pooling of of effective or ineffective interventions. However, reviews several studies yields better assurance, in terms of more of healthcare interventions may not yield enough evidence comprehensive evidence,© about Jones the &effectiveness Bartlett ofLearning, specific LLCto answer precise questions ©on Jones the effectiveness & Bartlett or inef Learning,- LLC interventions. Systematic reviews allow for determining and fectiveness of specific interventions. Perhaps practitioners confirming consistencyNOT in studies FOR that SALE are conducted OR DISTRIBUTION across might find it less daunting toNOT conduct FOR systematic SALE ORreviews DISTRIBUTION different clinical contexts and targeting comparable patient if they could find review guidelines that state parameters populations. A single independent study, by contrast, may and instructions for applying them to ensure standard- fall short of being generalizable to other population groups. ization. Arguably, non-systematic review is a quick way Systematic© Jones reviews & Bartlett have the Learning, additional benefit LLC of thor- to put forth© introductory Jones & Bartlettpapers on Learning,specific opinions LLC on oughness as long as the reviewer(s) use well-designed and policy and practice. Nonetheless, that process falls short approvedNOT frameworks. FOR SALE Conducted OR DISTRIBUTION correctly, systematic of providingNOT an all-inclusive FOR SALE combination OR DISTRIBUTION of best avail- reviews can provide a clearer picture of the overall effects, able evidence. The general view is that a non-systematic both positive and adverse, of particular clinical interven- review carried out carelessly and incorrectly can yield tions. In addition to confirming what is already known, serious misrepresentation of detail and cause confusion. © Jonessystematic & Bartlett reviews Learning, also help LLCto identify the deficits or gaps© JonesPetticrew & Bartlett (2003) examined Learning, the reasonsLLC for the lack of in professional knowledge and practice and thus provide specific guidance for systematic reviews of social and NOT FORa guideSALE for OR future DISTRIBUTION research Brown et al., 2006; Petticrew,NOT healthcare FOR SALE interventions. OR DISTRIBUTION He highlighted Millward, Kelly, 2003; Petticrew & Roberts, 2006). Crombie and Davies and Nutbeam’s 2001 work, which showed a limitation in (2009) note that a key feature of systematic reviews is the the number of trials in social and health care and reviews unbiased critical appraisal of all the available relevant studies. involving outcome assessments. A concise summation of© the Jones rationales, & Bartlett main benefits, Learning, and LLC © Jones & Bartlett Learning, LLC the significance of systematic reviews is outlined in Box 3-1. NOT FOR SALE OR DISTRIBUTIONNon-Systematic ReviewsNOT Examined FOR SALE OR DISTRIBUTION

Box 3-1 Summary of the Emphasis on Non-systematic reviews do not necessarily aim to identify Systematic Reviews for EBP all the relevant published studies that applied a specific research design or methodology to investigate a specific © Jones & Bartlett Learning, LLC © Jones & Bartlett Learning, LLC Provide cumulative evidence from relevant avail- clinical problem. Rather than critically appraising and NOTable FOR studies SALE to inform OR clinical DISTRIBUTION practitioners and synthesizingNOT the research FOR SALEfindings, OR non-systematic DISTRIBUTION reviews policy and decision makers characteristically present a broad discussion of the find- Provide sound evidence basis for guiding organi- ings from some studies and substantiate with relevant zational policy decisions on standards of health references. Additionally, non-systematic reviews do not care and practice require adherence to an exact and precise review protocol. © Jones & BartlettReveal theLearning, impact of specific LLC policy regulations © JonesConsequently, & Bartlett without Learning, compliance LLCor application of specific NOT FOR SALEon OR particular DISTRIBUTION aspects of professional practice NOTpre-set FOR criteriaSALE toOR the DISTRIBUTION study selection, a non-systematic Yield better assurance, more comprehensive and review might be carried out in a rather haphazard manner. consistent evidence about the effects, practi- Thus, a non-systematic review may largely represent the cability, significance, and suitability of specific reviewer’s subjective interpretations based on fixed ideas treatment interventions and lead to distorted conclusions (Sandelowski, 2008). Provide a clearer picture© Jones with better& Bartlett insight and Learning, AnLLC outline of the principles and© Jones essential & components Bartlett Learning, of LLC understanding aboutNOT the FOR overall SALE outcomes, OR bothDISTRIBUTION systematic reviews is presentedNOT in Box FOR 3-2 SALE. OR DISTRIBUTION positive and adverse, of clinical interventions Help to identify the deficits or gaps in professional knowledge and practice thus providing a guide Evaluating the Quality of Each Selected to future research and appropriate contexts for Research Study © Jonesparticular & investigations Bartlett Learning, LLC © Jones & Bartlett Learning, LLC The process should involve critical examination of each (DataNOT from FORPetticrew, SALE M. (2003). OR Why DISTRIBUTION certain systematic re- NOT FOR SALE OR DISTRIBUTION views reach uncertain conclusions. BMJ, 326(7392), 756–758; study to assess the methodology for thoroughness and cor- Kitchenham, B. (2004). Procedures for performing systematic rect application, data analysis techniques, level of detail in reviews: Joint technical report. Keel, United Kingdom: Keel the presentation of the research results, degree of accuracy University; Petticrew, M., & Roberts, H. (2006). Systematic reviews in the social sciences: A practical guide. Malden, MA: of the measurements, and outcomes/results. Moreover, the © Jones & BartlettBlackwell; Brown, Learning, P., Brunnhuber, LLC K., Chalkidou, K., Chalmers, © Jonesprocess & should Bartlett involve Learning, critically examining LLC the methodology NOT FOR SALEI., Clarke, OR C., Fenton, DISTRIBUTION M., . . . Young, P. (2006). How to formu- NOTfor FOR possible SALE replication OR DISTRIBUTION and generalization of the findings. late research recommendations. BMJ, 333(7572), 804–806. Some experts recommend blinding the quality assessors to hide the identity of the researchers/authors of the study

9781284114942_CH03.indd 50 06/11/17 2:24 pm CHAPTER 3 Systematic Reviews: Consolidating Research Evidence for EBP 51

© Jones & Bartlett Learning, LLC © Jones & Bartlett Learning, LLC The Need for an Extensive Systematic Review NOT FOR SALE OR DISTRIBUTION NOT FOR SALE OR DISTRIBUTION Box 3-2 Key Considerations and Actions Readers may find it useful to access the Database ofAbstracts in Systematic Reviews of Reviews of Effects (DARE) at www.crd.york.ac.uk /crdweb/. The Cochrane Database of Systematic Reviews Constructing pertinent review questions: (CDSR) at http://www.cochranelibrary.com/cochrane These should ©be Jonesfocused, well-defined& Bartlett andLearning, LLC-database-of-systematic-reviews/index.html© Jones & Bartlett also provides Learning, LLC unambiguous.NOT FOR SALE OR DISTRIBUTIONhealthcare researchers and practitionersNOT FOR fullSALE and regularlyOR DISTRIBUTION Developing a review protocol: The purpose of updated texts of systematically reviewed healthcare inter- this is to outline The process of constructing vention effects. Policy and protocols also undergo similar the review question, the search and retrieval of high standard review processes conducted by the Cochrane evidence materials, the eligibility criteria, the Review Groups in the Cochrane Collaboration system. The © Jonesstudy selection, & Bartlett information Learning, extraction, LLC quality U.S. National© Jones Library &of BartlettMedicine (NLM),Learning, which LLC is part assessment, synthesis, interpretation, conclu- NOT FOR SALE OR DISTRIBUTION of the NationalNOT Institutes FOR SALE of Health OR (NIH), DISTRIBUTION also produces sion, and preparation of the review report. extensive data on all aspects of medicine and health care, Extensive search of the literature and other rel- which is accessible at http://www.nlm.nih.gov/. evant sources: The process for locating the per- Other useful sources of systematically reviewed studies tinent studies should be comprehensive. include the National Institute for Health and Care Excellence © Jones & BartlettDefining Learning, and using criteria LLC for selection of eligible © Jones(NICE), & Bartlettthe Scottish Learning, Intercollegiate LLC Guidelines Network studies (inclusion/exclusion criteria): Criteria NOT FOR SALE OR DISTRIBUTION NOT (SIGN),FOR SALE and the OR Evidence DISTRIBUTION for Policy and Practice Informa- should be clearly reasoned and applicable to tion (EPPI) Centre, which provides reviews of policy and the identified problem/phenomenon and the protocols relating to social welfare and social care, health review questions. The PICO model (Patient/Pop ulation/Problem, Intervention, Comparison, Out- education, health promotion, and public health. The World come) may be used and the selection should Health Organization (WHO) is another good source, as target full research© Jones reports. & Experts Bartlett recommend Learning, LLCit has the reputation of ensuring© Jones that &its Bartlettguidelines Learning, and LLC involvement ofNOT at least FOR two reviewersSALE ORto ascer- DISTRIBUTIONrecommendations are basedNOT on meticulouslyFOR SALE reviewed OR DISTRIBUTION tain consistency in the application of the stud- research studies, meta-analysis, and meta-synthesis on ies’ inclusion/exclusion criteria. interventional effects and other forms of substantiated Extracting pertinent details from the reports of the evidence. The Centre for Reviews and Dissemination selected studies: The process of data extraction (2009) provides a detailed guide for systematic review. © Jonesshould be & consistent.Bartlett Both Learning, electronic LLCand script For copyright© Jones reasons, & readers Bartlett are encouraged Learning, to LLC explore NOTpro-formas FOR SALE are available. OR DISTRIBUTION the CRD documentNOT FOR themselves. SALE OR DISTRIBUTION

Selecting the Team of Reviewers A carefully selected review team should represent all the disciplines in the particular area of clinical practice. This © Jones &reports, Bartlett the institutions,Learning, andLLC the particular journals of© Jones & Bartlett Learning, LLC publication. Subjective preferences in the research report ensures joint participation in the review and ultimately, NOT FORin SALE terms ofOR potential DISTRIBUTION biases should also be given carefulNOT sharedFOR ownership,SALE OR support, DISTRIBUTION and cooperation in implementa- consideration. tion of EBP. Experts advise that implementation of change should be facilitated by a team of professionals who have Initial Preparation the relevant backgrounds of specialism (Lo Biondo-Wood & Haber, 2013). The review team should advise not only Because systematic reviews involve considerable rigor, © Jones & Bartlett Learning, LLCpractitioners with a range ©of Jonesknowledge & Bartlett and expertise Learning, LLC preparation is important and should be carefully thought NOT FOR SALE OR DISTRIBUTIONin the clinical specialism, butNOT also FOR in systematic SALE OR review DISTRIBUTION out. It is important to establish what reviews, both previ- methods, information retrieval, research, health econom- ous and current, have been conducted on the particular ics, and statistics. clinical problem and what review questions were asked. Another motive for systematic reviews may be obliga- tory ©response Jones to &a commission.Bartlett Learning, The commissioning LLC body The Role of© aJones Consultative & Bartlett Committee Learning, LLC may NOTdemand FOR verification SALE OR of the DISTRIBUTION best available evidence The consultativeNOT FORcommittee SALE represents OR DISTRIBUTION an advisory group through systematic reviews of such issues as organizational comprising professionals from the healthcare organization, policy, emerging new procedures/treatment interventions, expert researchers, and stakeholders from the public sector(s) or the prevalence and pattern of spread of an infection. (who may be patient representatives and service users). For more detailed information, readers are encouraged to This team of advisers has the responsibility to examine and © Jones &explore Bartlett both theLearning, Cochrane LLC Handbook (Higgins & Green,© Jonescomment & Bartlett on the review Learning, protocol LLC and the final report, and NOT FOR2011) SALE and OR the DISTRIBUTION Centre for Reviews and DisseminationNOT toFOR provide SALE support, OR clarification, DISTRIBUTION and guidance. In essence, (CRD) publication, Guidance for Undertaking Reviews in the advisory group ensures that the review is pertinent for Healthcare (Centre for Reviews and Dissemination, 2009). meeting the needs of all relevant stakeholders. Therefore, this

9781284114942_CH03.indd 51 06/11/17 2:24 pm 52 PART I Exploration of the Concept of Evidence-Based Practice and Related Practical Challenges

© Jones & Bartlett Learning, LLC © Jones & Bartlett Learning, LLC group should have the authority to examine the feasibility produced with accompanying narrative guides or instructions, NOT FORof implementingSALE OR DISTRIBUTION the recommended guidelines (Gagan &NOT others FOR are SALE considered OR DISTRIBUTIONrelatively self-explanatory and easy Hewit-Taylor, 2004). It is important that such groups are to apply. A copy of the review framework should be enclosed established prior to commencing the systematic review. separately or incorporated in the protocol (Crombie & Davies, 2009; Hemingway & Brereton, 2009; Kitchenham, Developing the Review© ProtocolJones & Bartlett Learning, LLC2004). Other key stages are examined© Jones here. & Bartlett Learning, LLC Lo Biondo-Wood and Haber (2013) recognize the advantage NOT FOR SALE OR DISTRIBUTIONRelevance of the PICO(S) FormatNOT FORand Criteria SALE OR DISTRIBUTION of putting evidence-based policies, procedures, and guidelines in writing. Public and organizational/administrative The process for selecting relevant studies for systematic dynamics that may potentially hinder the use of research reviews traditionally considers population, intervention, findings should be carefully examined. Appropriate mea- comparators, and outcome (PICO). Study design has been sures should© Jones be taken & Bartlett to address Learning, these before LLC embarking added to the© process Jones to &create Bartlett PICO SLearning,. LLC on theNOT implementation FOR SALE of EBP.OR Brown,DISTRIBUTION Wickline, Ecoff, NOT FOR SALE OR DISTRIBUTION and Glaser, (2009) and other authors cite barriers such as Application of the PICO(S) Criteria: An Exemplar inadequate staffing capacity, varied levels of competence, The main purpose for the review should be stated, specifying unequal and inadequate management support, neglected the exact clinical issue that is to be explored. For example: research training, and experience. Other limitations To explore the impact of specific campaigns, © Jonesinclude & Bartlett deficient Learning, research LLCactivities, disinterest and lack© Jones & Bartlett Learning, LLC including sexual health promotion and patient of motivation to conduct studies, and poor application NOT FOR SALE OR DISTRIBUTION NOT FOReducation, SALE sexual OR DISTRIBUTION health screening, safer sex, and of credible findings to clinical practice in certain areas. condom usage designed to target young adults The tendency to consistently apply unsubstantiated and between 18–24 years of age in X area(s). The intuitive practices could lower the standard and quality review would cover all clinical settings within of care delivery. the . . . locales and/or geographical areas. The protocol should© outline Jones clear, & practicable,Bartlett Learning,reasoned, LLC © Jones & Bartlett Learning, LLC and achievable actions to ensure correct and efficient NOT FOR SALE OR DISTRIBUTIONP: Population/Patients or NOTStudy FORParticipants SALE DetailsOR DISTRIBUTION implementation. An important advantage of developing a about the characteristics of the participants should be protocol is to avoid omissions and inconsistencies that may indicated in the statement or review question. Relevance result in inaccurate conclusions being drawn. In addition, to the target population should be recognizable such as a protocol provides a practical guide to the review process gender, age range, type of medical condition or specific and the© intendedJones use& Bartlett for the findings Learning, (Crombie LLC & Davies, © Jones & Bartlett Learning, LLC health problem, degree of severity, and specific status of 2009; Hemingway & Brereton, 2009; Kitchenham, 2004). NOT FOR SALE OR DISTRIBUTION the problemNOT or condition. FOR SALE OR DISTRIBUTION In reality, while most of the decisions about the systematic review and meta-analysis are incorporated in the protocol, I: Intervention, for example, Specific Medication additional decisions may become necessary as the review or Therapeutic Procedure The particular treatment progresses. The following is a sample protocol derived intervention(s) should reflect those specified in the review © Jonesfrom & Bartlett existing protocols.Learning, LLC © Jones & Bartlett Learning, LLC question. Multiple treatment interventions should be clearly NOT FOR SALE OR DISTRIBUTION NOTitemized FOR SALEaccording OR to relevanceDISTRIBUTION to the review. For example: Key Components of the Protocol Sexual health promotion/education The protocol should provide a schedule and venue for the Sexual health screening for chlamydia and other STIs review meetings, specify the processes to be employed in Promotion of condom use the review, and serve as© a Jonesguide to deciding& Bartlett on the Learning, review LLC © Jones & Bartlett Learning, LLC question. This helps to refine the question and make clear C: Comparators The nature of the comparators should which intervention is beingNOT explored FOR SALE and why. OR DISTRIBUTION NOT FOR SALE OR DISTRIBUTION be explained in terms of the study/control groups or the The protocol should also include an outline of the study types of treatment interventions and modes of delivery. selection process. To avoid bias, criteria for inclusion and For example: exclusion should be clearly defined. The protocol should also outline© Jones the processes & Bartlett of data Learning, extraction and LLC assessment Comparisons© Jones of an &intervention Bartlett groupLearning, of sexually LLC of the quality of the research studies in terms of design and active young adults who regularly attend sexual methodology.NOT FOR The techniqueSALE OR for DISTRIBUTIONdata synthesis should be health clinicsNOT FORto those SALE who seldom OR DISTRIBUTION ever or never concisely stated together with the strategy for disseminat- attend clinics. ing the findings from the review. Any modifications to these Or, processes should be clearly documented in relation to the © Jonesreview & Bartlett question. Learning, LLC © JonesImplementation & Bartlett Learning, of specific behavior LLC change inter- The use of an approved and formal systematic review ventions with guidance and instructions provided NOT FORframework SALE OR facilitates DISTRIBUTION these processes. While some areNOT FORin the SALE group ORcontext DISTRIBUTION as well as on individual basis.

9781284114942_CH03.indd 52 06/11/17 2:24 pm CHAPTER 3 Systematic Reviews: Consolidating Research Evidence for EBP 53

© Jones & Bartlett Learning, LLC © Jones & Bartlett Learning, LLC O: Outcome This relates to the effectiveness of the treatment While it is important to consider a varied range of NOT FORintervention(s). SALE OR TheDISTRIBUTION outcomes should be recorded as a directNOT studies,FOR SALE it is crucial OR toDISTRIBUTION determine whether the studies are result of the intervention(s) but if not, the compounders directly or indirectly related to the topic. Therefore, in a or co-interventions that may have been in place should preliminary search, particular attention should be paid to be clearly specified. Additionally, the exact nature of each potential limitations such as publication preferences and observed and measured© outcome Jones should & Bartlett be clearly Learning, recorded. LLCother deficiencies that may© influence Jones &the Bartlett range of studiesLearning, LLC For example: published. Because these can influence the selection of NOT FOR SALE OR DISTRIBUTIONstudies for reviews, those problemsNOT FOR are considered SALE OR here. DISTRIBUTION Clear evidence of improvement in the sexual behaviors of the research intervention group. Use of Approved Set of Eligibility Criteria for Clear evidence that increased proportion of the Objective Selection of the Studies intervention© Jones &group Bartlett of young Learning, adults demonstrated LLC The stages of© studyJones selection & Bartlett outlined Learning, here are derived LLC from behaviorsNOT FOR indicative SALE ofOR being DISTRIBUTION better informed various sources;NOT in FOR particular, SALE Kitchenham OR DISTRIBUTION (2004), Heming- about and practicing safer sex with fewer sexual way and Brereton (2009), and Higgins and Green (2011). risk-taking behaviors. • Stage one: Abstract selection. Based on the abstracts, Negative outcomes should also be clearly reported studies that are not relevant to the problem are and more exact descriptors and terminology specifying excluded. Studies deemed relevant to the problem © Jones &the Bartlett nature of Learning,each type of outcomeLLC should be stated. The© Jones &of Bartlett interest are Learning, retained. LLC challenge is whether the outcomes were observable, mea- NOT FOR SALE OR DISTRIBUTION NOT FOR• StageSALE two: OR Population. DISTRIBUTION Studies are examined for surable, and recordable to achieve objectivity in each of relevance of population/patient characteristics. the selected studies. • Stage three: Intervention. The studies are examined S: Study Design A design that allows for objectivity and for the nature of the intervention and its delivery. consistency is likely to© have Jones been carefully& Bartlett developed. Learning, For LLC The exact content and© modeJones of delivery& Bartlett should Learning, be LLC example: “randomizedNOT controlled FOR trials SALE” with OR appropriate DISTRIBUTION stipulated. Other factsNOT could FOR include SALE specifying OR who DISTRIBUTION attention to detail. This could prove to be a challenge in delivered the intervention and where that intervention some areas of sexual healthcare where empirical quantitative took place. Once again, studies that fall short of the or other seminal studies may be relatively limited. specifications are excluded at this stage. The above suggestions draw on various guidelines for • Stage four: Comparison. This stage examines the systematic© Jones reviews &including Bartlett Petticrew Learning, (2003) and LLC Petticrew and comparison© Jones and/or & Bartlett control processesLearning, and LLC further RobertsNOT (2006). FOR SALE OR DISTRIBUTION exclusionsNOT FORcarried SALE out as OR necessary. DISTRIBUTION Comparators should be specifically defined (Hemingway & The Literature Search and Retrieval of Published Brereton, 2009; Kitchenham, 2004). Evidence Materials • Stage five: Expectation. The indicators of the intervention outcome are used to determine the size, © Jones &Comprehensive Bartlett Learning, systematic LLC search and retrieval of pub©- Jones & Bartlett Learning, LLC lished evidence materials forms an essential requirement nature, and extent and significance of the effects, NOT FORin SALE systematic OR reviews. DISTRIBUTION Documentation provides an auditNOT FOR bothSALE positive OR andDISTRIBUTION adverse, of a specific intervention. trail and helps the reviewers, the target audience, and other The importance of clearly defining the relevant set readers to judge the exhaustiveness of the search. It also of outcomes such as measures of mortality and allows for determining the extent of inclusion of the range morbidity, quality of life, and specifically related experiences of physical function should be empha- of research studies that© were Jones accessed & Bartlett (Kitchenham, Learning, 2004). LLC © Jones & Bartlett Learning, LLC Determining the time frame over which the search should sized. In the case of the participants’ experiences, cover depends on the natureNOT and FOR complexity SALE of OR the problem/DISTRIBUTIONthe review may involveNOT qualitative FOR SALE studies OR or DISTRIBUTION a phenomenon under review and on how rapidly that field of combination of quantitative and qualitative studies. clinical practice has undergone progressive development and From that assessment, further exclusion is carried change. Therefore, the extent to which the particular problem out as necessary (Hemingway & Brereton, 2009; Kitchenham, 2004). and the© relatedJones intervention & Bartlett have Learning, been investigated LLC could be an © Jones & Bartlett Learning, LLC influence (Scottish Intercollegiate Guidelines Network [SIGN], • Stage six: Study design. This stage assesses whether 2011).NOT The searchFOR for SALE reports OR on seminalDISTRIBUTION work should lead to the NOTdesign FORemployed SALE for ORthe investigationDISTRIBUTION in each identification of empirical studies if the topic has been explored study accords with the stipulated indicators for those by experienced researchers. Other high-quality studies from designs. The designs are meticulously examined, both primary and secondary sources may also be identified. followed by more sensitive analysis at the stage of © Jones &An Bartlett all-inclusive Learning, exploration LLC is advantageous to obtain all© Jones &synthesis Bartlett or integrationLearning, of LLC findings (Hemingway & available pertinent studies on the topic (Crombie & Davies, Brereton, 2009; Kitchenham, 2004). Robustness of NOT FOR2009; SALE Hemingway OR DISTRIBUTION & Brereton, 2009; Polit & Beck, 2008).NOT FOR theSALE study OR design DISTRIBUTION is often the determining factor for

9781284114942_CH03.indd 53 06/11/17 2:24 pm 54 PART I Exploration of the Concept of Evidence-Based Practice and Related Practical Challenges

© Jones & Bartlett Learning, LLC © Jones & Bartlett Learning, LLC including a study in the review. However, studies methodology, and techniques of data collection and NOT FOR SALEof the OR same DISTRIBUTION design may not necessarily be of theNOT FORanalysis. SALE Contact OR DISTRIBUTION with the original authors may be same high standard and quality. A particular cau- considered for further clarification at the stage of tion relates to the risk of omitting studies that have data analysis (National Institute for Health and direct relevance within the context of the review if Clinical Excellence [NICE], 2006; Noyes et al., 2001; the set criteria were© Jones too narrow & Bartlett and/or too Learning, rigidly LLC Thomas & Harden, 2008).© Jones & Bartlett Learning, LLC adhered to. However, unrealistically broad criteria NOT FOR SALE OR DISTRIBUTION• Description of the generalNOT baseline FOR SALEcharacteristics, OR DISTRIBUTION that allow for varied interpretations and application social and economic circumstances directly relating could result in poor and/or complicated compari- to the setting and context, delivery of the interven- sons and syntheses (Hemingway & Brereton, 2009; tion, and the subsequent interpretation and synthesis Kitchenham, 2004). of the findings is provided. Additionally, the total At© each Jones of these & Bartlett stages, the Learning, specific reason LLC for the number© Jonesof participants & Bartlett and how Learning, recruited, andLLC the exclusionNOT should FOR be SALE documented OR DISTRIBUTION in the appropriate col- total numberNOT FOR in the SALE treatment OR intervention DISTRIBUTION and con- umn indicating the nature of the limitation or deficiency trol groups are also recorded (NICE, 2006; Noyes (Hemingway & Brereton, 2009; Kitchenham, 2004; SIGN, et al., 2001; Thomas & Harden, 2008). 2011). • The type of treatment interventions and co-interventions, related conceptual/theoretical framework and prin- © Jones & Bartlett Learning, LLC © Jones & Bartlett Learning, LLC Extracting Pertinent Data ciples, development and mode of delivery, duration, NOT FOR SALE OR DISTRIBUTION NOT FORcontrol, SALE and OR the DISTRIBUTION staff responsible for administering Data extraction is the process of pulling out essential these are stated (NICE, 2006; Noyes et al., 2001; information about the main aspects and key attributes of Thomas & Harden, 2008). individual research studies. Because studies of different designs present variations in methodology, the amount • The predicted outcomes, types, how defined, and cal- and content of data to be extracted is bound to vary. culated, the related baseline measurements, subgroups, © Jones & Bartlett Learning, LLC exclusions, and withdrawals© Jones are stated & Bartlett (NICE, 2006; Learning, LLC Key considerations forNOT data FOR extraction SALE depend OR DISTRIBUTION on the NOT FOR SALE OR DISTRIBUTION purpose of the review and the specific design of the Noyes et al., 2001; Thomas & Harden, 2008). selected research studies. An appropriately devised set of • Framework approach for establishing the direct extraction criteria with a clear and methodical system of relationships between findings and review question(s) documentation helps to achieve the desired consistency should show the key elements extracted. and accuracy© Jones of extraction.& Bartlett Furthermore, Learning, the LLC technique • Finally,© Jonesthe social & andBartlett economic Learning, implications LLC of for analyzingNOT FOR the data SALE and theOR data DISTRIBUTION presentation depend the findings,NOT FOR potential SALE impact OR DISTRIBUTIONof change in clinical on the key issues conveyed in the review question and contexts, health policy, and cost effectiveness are objectives (Higgins & Green, 2011; Petticrew & Roberts, stated (NICE, 2006; Noyes et al., 2001; Thomas 2006; Polit & Beck, 2008). A copy of the data extraction & Harden, 2008). form should be included in the review protocol. Because © Jonesreviews & Bartlett are carried Learning, out on several LLC studies, it is pragmatic© JonesClearly, & Bartlett the use Learning, of an approved LLC framework with a to use a well-devised purposeful, tested, and approved pre-specified set of criteria helps to achieve efficient and NOT FORdata SALE extraction OR DISTRIBUTIONform. NOTwell-organized FOR SALE recording OR DISTRIBUTION of the extracted data (Petticrew & Various formats are available, such as the Cochrane Roberts, 2006; Polit & Beck, 2008). Moreover, stan- Study Selection, Quality Assessment & Data Extraction dardization increases confidence in the data extraction Form. Two examples of application that may be of in- process (Hemingway & Brereton, 2009). The selected terest to readers are the© Data Jones Extraction & Bartlett Form Learning,for HIV/ frameworkLLC or template must© be Jones up to standard & Bartlett and easy Learning, LLC for the team to use. While certain versions may be useful AIDS Provider Training,NOT and FORthe Quality SALE assessment OR DISTRIBUTION for NOT FOR SALE OR DISTRIBUTION Intervention Studies of HIV/AIDS Provider Training. The for introductory purposes, practitioners and educators are successive steps in the process of data extraction presented urged to explore more comprehensive, formally designed here are applicable to general review purposes. The impor- formats and current electronic and updated versions that tant thing is for the team of reviewers to critically examine continue to emerge. the content© Jones then adapt& Bartlett as necessary Learning, to suit theLLC particular As the team© Jones gains more & Bartlett familiarity Learning, with the process LLC of purpose of the systematic review. data extraction, modification and pilot testing of the identified NOT FOR SALE OR DISTRIBUTION data extractionNOT forms FOR becomes SALE more OR practical DISTRIBUTION and achiev- • Identification comprises the reviewer’s initials/ able. The sample shown in Box 3-3 can be further adapted: allocated identification code and the assigned review for example, the inclusion and exclusion criteria could be reference number. structured to allow for recording indicators relating to the © Jones & Bartlett• For each Learning, study, the title,LLC author(s), and publica©- Jonespopulation, & Bartlett the design, Learning, the intervention, LLC and the outcomes tion specifics are recorded. The type of systematic with an additional column for specifying the elements that NOT FOR SALEreview OR is DISTRIBUTIONrecorded together with the study designs,NOT justify FOR exclusion SALE OR(Khan, DISTRIBUTION Kunz, Kleijnen, & Antes, 2011).

9781284114942_CH03.indd 54 06/11/17 2:24 pm CHAPTER 3 Systematic Reviews: Consolidating Research Evidence for EBP 55

Review author’s ID or initial: Date: Code & unique reference number for the study: Author(s): © Jones & Bartlett Learning,Date: LLC © Jones & Bartlett Learning, LLC Study title on the researchNOT FORreport: SALE OR DISTRIBUTION NOT FOR SALE OR DISTRIBUTION Publication journal: Year / Vol. / Page numbers: Study type / characteristics: Aim: ©Study Jones design: & Bartlett Learning, LLCDuration: © Jones & Bartlett Learning, LLC NOTInclusion: FOR SALE OR DISTRIBUTIONExclusion: NOT FOR SALE OR DISTRIBUTION Process of randomization / allocation of participants: Total number / sample size: Intervention group: Control group: © Jones & BartlettParticipant Learning, description: LLC © Jones & Bartlett Learning, LLC NOT FOR SALEAge OR / age DISTRIBUTION range: Gender:NOT FOR SALE OREthnic DISTRIBUTION origin: Social & economic background: Medical diagnosis: Related health problems: The study setting / context of interventions: Details of specific intervention(s):© Jones & Bartlett Learning, LLC © Jones & Bartlett Learning, LLC Specific outcomes: NOT FORHow SALEdefined: OR DISTRIBUTIONType of measurement: NOT FOR SALE OR DISTRIBUTION 1. 1. 2. 2. Technique(s) of analysis: R©esults Jones per sample & Bartlett group: Learning, LLC © Jones & Bartlett Learning, LLC Findings:NOT FOR SALE OR DISTRIBUTION NOT FOR SALE OR DISTRIBUTION Comments:

© Jones &Assessment Bartlett Learning,of the Meticulousness LLC © Jonesfor. Each & Bartlett predicted biasLearning, should be LLC clearly recorded together NOT FORof SALE Each Study OR DISTRIBUTIONDesign NOT withFOR the SALE relevant OR elements DISTRIBUTION of quality assessment (Khan et al., Because the process of data extraction in systematic reviews 2011). The impact of specific risk biases may be rated as high, and quality assessment of the research studies are linked, low, or unclear in relation to causing over- or undervaluation these processes tend to be undertaken concurrently. and misrepresentation of the true effects of the intervention. To gain a better understanding of this concept, readers are Assessment of Risk Bias© Jones & Bartlett Learning, LLCencouraged to explore Higgins© &Jones Altman & (2008) Bartlett in the Learning, Co- LLC chrane Handbook for Systematic Reviews of Interventions. The strength and truthfulnessNOT FORof the SALEfindings OR from DISTRIBUTION research NOT FOR SALE OR DISTRIBUTION studies become doubtful when flaws occur in the design. Assessment of the Suitability of the Statistical Shortcomings in research design that adversely affect findings Techniques and the Degree of Accuracy may occur in the form of selection, performance, measure- The statistical techniques and methods of analyzing data ment,© and Jones attrition & biasesBartlett (Khan Learning, et al., 2011). LLC These can be may depend© on Jones the adequacy & Bartlett of the sample Learning, size. The LLC sample explained as inadequate allocation procedure; dissimilarities NOT FOR SALE OR DISTRIBUTION should be NOTappropriately FOR SALE representative OR DISTRIBUTION of the population in the characteristics of the participant groups; lack of clarity in its components, subgroups, strata, and other relevant of the blinding process applied to the participants, the care characteristics (Higgens & Green, 2011; NICE, 2006). deliverers, and those who assessed the outcomes; and imbal- ance in the sample sizes caused by unpredicted dropouts or • Assessment of generalizability. © Jones &other Bartlett losses. Further Learning, shortfalls LLC may be attributed to inadequate© Jones• &In Bartlett the appraisal Learning, of findings, LLC two factors are crucial: (1) management and lack of clarity of the intention to treat, the study sample should be representative and should NOT FORthe SALE related OR analysis, DISTRIBUTION and how missing data were accountedNOT FOR characterizeSALE OR the DISTRIBUTION target population, and (2) the findings

9781284114942_CH03.indd 55 06/11/17 2:24 pm 56 PART I Exploration of the Concept of Evidence-Based Practice and Related Practical Challenges

© Jones & Bartlett Learning, LLC © Jones & Bartlett Learning, LLC should be generalizable to other populations beyond provided on how to summarize and interpret the overall NOT FOR SALEthe study OR populationDISTRIBUTION or participants. The study shouldNOT grade, FOR the SALE aggregate OR rate,DISTRIBUTION or the total score. have been conducted to closely represent standard Readers are encouraged to explore different methods current practice. Differing views are proposed about before making a choice (Higgins & Green, 2011; NICE, assessment of the generalizability of findings from 2006). Suggested sources of additional reading are included quantitative and qualitative© Jones studies, & Bartlett taking accountLearning, of LLCin the reference list, and readers© Jonesare encouraged & Bartlett to explore Learning, LLC the specific designs employed (Higgins & Green, 2011). the National Collaborating Centre for Methods and Tools NOT FOR SALE OR DISTRIBUTION NOT FOR SALE OR DISTRIBUTION • Assessment of the quality of the research report. (NCCMT) Quality assessment tool for quantitative studies (2008). Khan et al. (2011) and Armijo-Olivo, Stiles, The research report should be clear and comprehen- Hagen, Biondo, and Cummings’s (2012) Assessment of sible. There should be adequate detail on all important Study Quality for Systematic Reviews are also useful. aspects addressed in a full quality appraisal to allow for © Jones & Bartlett Learning, LLC Tables 3-1© andJones 3-2 represent & Bartlett simple pro-formasLearning, that LLC can be possible replication of the research process. Reporting on adapted and applied to assessment of quantitative and qualita- a particularNOT FORcriterion SALE should OR clearly DISTRIBUTION indicate whether it tive research NOTquality FORin conjunction SALE with OR data DISTRIBUTION extraction. These was met or unmet or lacking in clarity. should be regarded as rather broad formats that may require While some reporting systems only focus on a scoring adjustments to meet the purpose for the particular review. system, others require the assessors to provide brief statements on specific aspects of the study. In order to deal with © Jonesthe & largeBartlett number Learning, of selected studies LLC in systematic reviews, the© JonesData &Abstraction: Bartlett Learning, Relevance LLC in Systematic NOT FORuse SALE of quality OR assessment DISTRIBUTION checklists has become quite popular.NOTReview FOR SALE Data ORSynthesis DISTRIBUTION and Meta-Analysis The more elaborate assessment forms may have all the criteria stated for each key element assessed. Additionally, Data abstraction is an important process in systematic re- instructions on exactly how the scoring should be done may view and reviewers/meta-analysts should provide a detailed accompany the assessment forms. Guidance may also be explanation of the process in the full report. Standardized/ © Jones & Bartlett Learning, LLC © Jones & Bartlett Learning, LLC NOT FOR SALE OR DISTRIBUTION NOT FOR SALE OR DISTRIBUTION Table 3-1 Exemplar of Pro-Forma for Recording Quantitative Research Quality to Accompany the Data Extraction Process Quality Criteria/ Rating System and/ Notes (Quality Implications Key Elements Assessed Appraisal Question Quality Status or Method of Scoring for the Systematic Review) © Jones & Bartlett Learning, LLC © Jones & Bartlett Learning, LLC SpecificNOT quantitative FOR SALE Sampling: OR DISTRIBUTION NOT FOR SALE OR DISTRIBUTION design: paradigm Allocation concealment RCT Blinding CCT Confounders Other © Jones Unclear& Bartlett Learning, LLC © Jones & Bartlett Learning, LLC NOT FOR SALE OR DISTRIBUTION NOT FOR SALE OR DISTRIBUTION Study selection Risk bias Intervention(s) Data collection © Jones & Bartlett Learning, LLC © Jones & Bartlett Learning, LLC Withdrawals/dropouts NOT FOR SALE OR DISTRIBUTION NOT FOR SALE OR DISTRIBUTION Specific outcomes: How defined Specific measures Statistical© Jones techniques & Bartlett of Generalizability Learning, of theLLC © Jones & Bartlett Learning, LLC data analysisNOT FOR SALEresults OR DISTRIBUTION and findings NOT FOR SALE OR DISTRIBUTION Internal validity External validity The final report

© Jones Data& Bartlett from National Learning, Collaborating LLC Centre for Methods and Tools. (2008).© Jones Quality assessment & Bartlett tool forLearning, quantitative studies LLC. Hamilton, ON: NOT FORMcMaster SALE University. OR DISTRIBUTION (Updated 30 August 2017). Retrieved from http://www.nccmt,ca/knowkedge-repositories/search/14;NOT FOR SALE OR DISTRIBUTION Khan, K. S., Kunz, R., Kleijnen, J., & Antes, G. (2011). Systematic reviews to support evidence-based medicine: How to review and apply findings of health care research (2nd ed.). London, United Kingdom: Hodder Arnold.

9781284114942_CH03.indd 56 06/11/17 2:24 pm CHAPTER 3 Systematic Reviews: Consolidating Research Evidence for EBP 57

© Jones & Bartlett Learning, LLC © Jones & Bartlett Learning, LLC NOT FOR SALETable 3-2OR DISTRIBUTION Exemplar of Pro-Forma for Recording QualitativeNOT FOR Research SALE Quality OR toDISTRIBUTION Accompany the Data Extraction Process The Quality Status (Adequate Notes Appraisal Quality Inadequate (Quality Implications for Key Elements Assessed© Jones &Question Bartlett Learning,Indicators LLCUncertain) © Jonesthe Systematic & Bartlett Review) Learning, LLC NOT FOR SALE OR DISTRIBUTION NOT FOR SALE OR DISTRIBUTION Specific qualitative design: paradigm

Risk bias assessment

Quality© Jonesof the study & Bartlettinterventions Learning, LLC © Jones & Bartlett Learning, LLC SpecificNOT outcomes: FOR SALE OR DISTRIBUTION NOT FOR SALE OR DISTRIBUTION How defined Specific measures applied

Specific methods of analysis © Jones & Bartlett Learning, LLC © Jones & Bartlett Learning, LLC NOT FOR GeneralizabilitySALE OR DISTRIBUTION NOT FOR SALE OR DISTRIBUTION Quality of the research report

(Data from Spencer et al. 2006)

© Jones & Bartlett Learning, LLC © Jones & Bartlett Learning, LLC NOT FOR SALE OR DISTRIBUTION NOT FOR SALE OR DISTRIBUTION Table 3-3 A Simplified Format for Data Abstraction Inclusion and Specific Key Quality of Exclusion of Elements Research List or Study Studies Based on About the Design and Specific Count© ofJones Date &of BartlettCode or Learning, LLCStudy Year of Defined© JonesCriteria &Treatment Bartlett Learning,Methods ClinicalLLC StudiesNOT FORAbstraction SALENumber OR DISTRIBUTION Author(s) Title Publication and ReasonsNOT FORIntervention SALE OR(Score) DISTRIBUTIONDetails

Total number of Techniques participants of analysis, © Jones & Bartlett Learning, LLC © Jones & Bartlett Learning,*Intervention LLC type and group NOT FOR SALE OR DISTRIBUTION NOT FOR SALE OR DISTRIBUTIONpresentation *Control of the group results

formal data abstraction© forms Jones can &be Bartlettadopted for Learning, use or an LLCThe scoring systems also© vary. Jones Raw figures,& Bartlett risk ratios,Learning, LLC original version created,NOT if properly FOR SALEpilot tested. OR The DISTRIBUTION com- and the results from intention-to-treatNOT FOR analyses SALE should OR DISTRIBUTION all pleted abstraction form should be incorporated in the final be reported. The latter relates to analysis of participants report. The information provided in the data abstraction regardless of whether they received the intervention or not should convincingly show that the selected and included and regardless of what happened later. studies are appropriate for synthesizing and combining the An example of a data abstraction form deriving from results© (GrimshawJones & et alBartlett., 2003). Learning, LLC various sources,© Jones including & GrimshawBartlett et alLearning,. (2003), is LLCprovided ExpertsNOT FOR suggest SALE that ORthe processDISTRIBUTION of abstraction be in Table 3-3NOT. This FOR example SALE is intended OR DISTRIBUTION simply to illustrate carried out by at least two members of the review team possible development of an abstraction format that draws who should work separately. Additionally, they should be on existing ones. Depending on the type of review and the blinded to the original authors of the studies and the insti- specific purpose, review teams might prefer to develop a tutions in which the studies took place. After comparing more suitable abstraction format de novo. © Jones &the Bartlett abstractions, Learning, any discrepancies LLC should be resolved and© Jones & Bartlett Learning, LLC • In the column relating to the inclusion and exclu- NOT FORclearly SALE documented OR DISTRIBUTION (Grimshaw et al., 2003). PractitionersNOT FOR SALE OR DISTRIBUTION sion criteria, the specific factors should be clearly and systematic reviewers are urged to explore and critically defined with precise statements or scored against examine selected data abstraction forms. benchmarks. The reasons for exclusion should be © Jones & Bartlett Learning, LLC. NOT FOR SALE OR DISTRIBUTION.

9781284114942_CH03.indd 57 06/11/17 2:24 pm 58 PART I Exploration of the Concept of Evidence-Based Practice and Related Practical Challenges

© Jones & Bartlett Learning, LLC © Jones & Bartlett Learning, LLC concisely stated and the degree of diversity or het- NOT FOR SALEerogeneity OR DISTRIBUTION also stated. NOT FOR SALE OR DISTRIBUTION Box 3-4 Main Assumptions and Rationales for • In the column relating to the study design and Synthesizing Qualitative Research Evidence methods it may be useful to refer to an existing pro-forma for quality appraisal to determine relevant ●● Synthesizing qualitative research evidence po- key elements to ©look Jones for and & allocateBartlett appropriate Learning, LLC tentially helps to broaden© Jones the scope & andBartlett depth Learning, LLC scoring for specificNOT study FOR characteristics SALE OR including, DISTRIBUTION of existing professionalNOT knowledge. FOR SALE OR DISTRIBUTION for example, randomization, allocation concealment, ●● Qualitative research synthesis enables review- blinding, follow-up, analysis of intention-to-treat, ers to create more contemporary concepts and sampling technique, sample size, intervention group, develop new interpretations from the synthesis control group, baseline characteristics, and observed of available research evidence. This strength- ©outcomes Jones against & Bartlett the projected Learning, outcomes. LLC ens© the Jones basis for & ongoing Bartlett research. Learning, LLC • NOTIn relation FOR to the SALE methods, OR the DISTRIBUTION type(s) of data, techniques ●● ApartNOT from FOR gaining SALE deeper ORinsight DISTRIBUTION about the of analysis and presentation of the results, the raw effects of clinical interventions and aspects of healthcare policy, the synthesis may also data, the effect size—ratios of odds/risks, and other key help to identify issues requiring more in-depth results should be examined and appropriately scored. exploration. • The impact of heterogeneity among the studies and © Jones & Bartlett Learning, LLC © Jones &●● Furthermore,Bartlett Learning, synthesizing researchLLC data helps related subgroup data and sensitivity analysis should to establish better understanding about specific NOT FOR SALEbe recorded OR DISTRIBUTION for assessing the impact of the researchNOT FOR SALEproblems OR and DISTRIBUTION phenomena encountered in clini- quality on the results. cal practice. • The identified key strengths and the identified key ●● The role of qualitative research synthesis in limitations of the study should be stated and the helping determine the factors that enhance or overall quality score recorded. inhibit the effectiveness of specific therapeutic © Jones & Bartlett Learning, LLC or interventional services© Jones is now recognized. & Bartlett Learning, LLC • In the column relatingNOT toFOR specific SALE clinical OR details, DISTRIBUTION the NOT FOR SALE OR DISTRIBUTION type of clinical settings, locations, patient charac- ●● Synthesis of qualitative research evidence has the potential for generalizing findings in clinical teristics and participant characteristics, age range, practice and policy decision-making. gender, and specific problem of clinical/public health interest relating to the study may be indicated. Data from Brown, P., Brunnhuber, K., Chalkidou, K., Chalmers, I., Clarke, C., Fenton, M., . . . Young, P. (2006). How © Jones & Bartlett Learning, LLC to formulate© researchJones recommendations. & Bartlett Learning, BMJ, 333(7572), LLC Data Synthesis Explored NOT FOR SALE OR DISTRIBUTION 804–806;NOT Finfgeld-Connett, FOR SALE D. (2010). OR Generalisability DISTRIBUTION and By and large, studies focusing on healthcare interventions es- transferability of metasynthesis research finding. Journal sentially quantify the degree and measure of efficacy. Thus, the of Advanced Nursing, 66(2), 246–254; Petticrew, M. (2003). Why certain systematic reviews reach uncertain conclu- positive or negative impact of the specific intervention on the sions. BMJ, 326(7392), 756–758; Petticrew, M., & Roberts, H. outcomes tends to be documented in numerical or statistical (2006). Systematic reviews in the social sciences: A practical © Jonesformats & Bartlett as summarized Learning, and LLCpresented by the researcher(s)© Jonesguide &. Malden,Bartlett MA: Learning,Blackwell. LLC NOT FORin theirSALE report. OR Quantitative DISTRIBUTION research studies tend to involveNOT FOR SALE OR DISTRIBUTION application of comparable processes of data collection and similar techniques in analyzing the data. Moreover, synthesizing quantitative data involves application of approved specific assessment of the quality of the research studies. Neverthe- processes and techniques with strict statistical structuring and less, the approaches to qualitative research synthesis and exactness in documenting the results. Therefore, combining the processes involved tend to be more complex, involving © Jones & Bartlett Learning, multipleLLC interconnected elements,© Jones underpinning & Bartlett thoughts, Learning, LLC the findings across studiesNOT can FOR be achievedSALE ORmeaningfully DISTRIBUTION NOT FOR SALE OR DISTRIBUTION (Ring, Ritchie, Mandava, & Jepson, 2010). Primary studies deductions, and reasoning toward the anticipated goals. included in reviews of quantitative data are likely to be ho- The findings tend to be presented in various ways and the mogenous in their focus on the same topic and application interpretations also vary rather than reaching a common of comparable research design. Importantly, their similarities understanding. Synthesizing primary qualitative research also include© Jones application & Bartlett of comparable Learning, methodology LLC in terms data presents© particularJones & challenges Bartlett (Ring Learning, et al., 2010). LLC of the NOTtype of FOR data, theSALE analysis, OR and DISTRIBUTION the results. NOT FOR SALE OR DISTRIBUTION Various approaches have been proposed for identi- Determining an Appropriate Method for Synthesizing fying and selecting qualitative studies for synthesizing Specific Data based on the application of a specific set of criteria for Randomized controlled trails (RCT) remain the preferred inclusion (Box 3-4). As Ring et al. (2010) remarked, some methodology and are therefore the more commonly used © Jonesapproaches & Bartlett applied Learning, to the synthesis LLC of qualitative research© Jonesresearch & designBartlett for Learning,evaluating healthcare LLC interventions. NOT FORstudies SALE share OR similarities DISTRIBUTION to the synthesis of quantitativeNOT Meta-analysis FOR SALE is ORthe most DISTRIBUTION frequently applied quantitative studies, including the search for identifying relevant primary technique for data synthesis. However, qualitative research studies, the use of criteria for inclusion and exclusion, and synthesis involving narrative process may also be applied as

9781284114942_CH03.indd 58 06/11/17 2:24 pm CHAPTER 3 Systematic Reviews: Consolidating Research Evidence for EBP 59

© Jones & Bartlett Learning, LLC © Jones & Bartlett Learning, LLC an acceptable alternative if quantification of the data is not • Development of a protocol outlining all stages in- NOT FORconsidered SALE OR practicable. DISTRIBUTION Also, RCTs may not be applicableNOT FOR cludingSALE theOR technique DISTRIBUTION of combining the results by to many systematic review questions on healthcare issues. meta-analysis. Determining which comparable factors, Moreover, many systematic reviews do not encompass outcome measures, and summary effect measures to statistical data and, therefore, meta-analysis may not be be applied should be stated in the protocol. applicable for combining© Jones results from& Bartlett the studies. Learning, LLC• Thorough search of the© literatureJones & and Bartlett identification Learning, LLC of relevant studies. The Benefits and KeyNOT Elements FOR SALEof Meta-Analysis OR DISTRIBUTION NOT FOR SALE OR DISTRIBUTION • Construction and application of eligibility criteria. Meta-analysis allows for generalizing the findings from related studies to the target population indicated in those • The selection of pertinent studies guided by prede- studies. Other benefits are that meta-analysis helps to termined set of eligibility criteria. It is important that the operational definitions provided in the resolve© Jonesthe problem & Bartlett of professionals Learning, missing LLC significant © Jones & Bartlett Learning, LLC findings from high-standard and quality empirical studies. research report are carefully noted and a scoring Nevertheless,NOT FOR in determining SALE OR whether DISTRIBUTION combining the results systemNOT applied. FOR SALE OR DISTRIBUTION from multiple research studies would be feasible, account • Methodical data extraction to determine comparabil- should be taken of the clinical and statistical significance. ity of corresponding elements in the selected studies. Meta-analysis is not always possible or necessary (Khan These include the type of intervention, outcomes, © Jones &et al.,Bartlett 2011; SuttonLearning, & Higgins, LLC 2008). © Jones &intention-to-treat, Bartlett Learning, and the LLC particular results that can be effectively combined in the meta-analysis. NOT FORSummary SALE OR of Main DISTRIBUTION Arguments for Quantitative NOT FOR SALE OR DISTRIBUTION • Thorough examination of the studies to ensure that Meta-Analysis pertinent and seminal studies are selected for inclu- • Meta-analysis offers unbiased synthesis of data sion while studies with dubious and non-significant from carefully selected experimental or other high- results are rejected. The exact quality standards must quality quantitative© Jones studies. & The Bartlett technique Learning, allows LLC be defined for deciding© Jones which studies& Bartlett to select Learning, for LLC for integration NOTof multiple FOR studies, SALE including OR DISTRIBUTION large meta-analysis: NOT FOR SALE OR DISTRIBUTION empirical research and relatively smaller studies that ◆◆ Type of research design/paradigm and thor- may have been considered as inconclusive. ough critical examination of the conduct of • The process offers more detailed statistical calcula- the study. The sampling technique should be tions with precision, which ensures that relatively examined for the statistical basis for calculat- ©minor Jones details & Bartlett that may Learning, be significant LLC for clinical ing© theJones sample & Bartlettsize. Learning, LLC decisions in treatments can also be identified and NOT FOR SALE OR DISTRIBUTION ◆◆ RiskNOT bias FOR assessment SALE ORshould DISTRIBUTION be detailed and taken into account. methodical. Potential systematic errors and • The merging of results from studies with consistently the strategies for dealing with them includ- small numbers of participants effectively increases the ing the system of randomization—allocation/ total sample size and, therefore, improves the statistical selection bias. © Jones & Bartlettpower andLearning, generalizability LLC of the findings and results.© Jones & Bartlett Learning, LLC ◆◆ The degree of thoroughness in the implementation NOT FOR SALE• The OR pooled DISTRIBUTION results from larger numbers of patients/NOT FOR SALEof the researchOR DISTRIBUTION intervention should be ascertained clients provide more valid and convincing informa- and there should be planned procedures for deal- tion about particular clinical problems and the effects ing with performance bias. of treatment interventions that may not have been ◆◆ Procedures for reporting inconsistencies in the extensively investigated. In this case, pooling may outcome measures and the intervention effects— also afford greater© Jonesconfidence & Bartlett in the related Learning, clinical LLC © Jones & Bartlett Learning, LLC detection bias. decisions. NOT FOR SALE OR DISTRIBUTION NOT FOR SALE OR DISTRIBUTION ◆◆ Publication bias can occur in the process of locating • Rather than simply reporting the odds ratio of oc- and selecting relevant studies for meta-analysis. currence of a condition among the study group com- These may depend on the direction of the results parative to the control group, meta-analysis allows and whether these are statistically significant. Ad- for calculating the risk ratio or relative risk ratio as © Jones & Bartlett Learning, LLC ditionally,© Jones language & Bartlett and potential Learning, interpretation LLC well. (Khan et al., 2011; Sutton & Higgins, 2008) NOT FOR SALE OR DISTRIBUTION issuesNOT can FOR cause SALE publication OR DISTRIBUTION bias.

Logical Stages of Meta-Analysis ◆◆ The statistical techniques employed. Appraisal of • Because meta-analysis forms a key component of the degree of precision applied in the data collection systematic review, the initial stage begins with that. process, the data analysis, and degree of accuracy of the results. © Jones & Bartlett• Clearly Learning, formulated questionLLC of clinical significance.© Jones & Bartlett Learning, LLC NOT FOR SALEThis OR is important DISTRIBUTION because unfocused and broad quesNOT- FOR ◆SALE◆ Generalizability. OR DISTRIBUTION tions result in ambiguous criteria and inconsistency ◆◆ The completeness and quality of the research report. in the selection of studies. (Khan et al., 2011; Sutton & Higgins, 2008)

9781284114942_CH03.indd 59 06/11/17 2:24 pm 60 PART I Exploration of the Concept of Evidence-Based Practice and Related Practical Challenges

© Jones & Bartlett Learning, LLC © Jones & Bartlett Learning, LLC Amalgamating Research Study Results: Effect depth of the education and counseling support provided to NOT FOREstimates SALE andOR WeightedDISTRIBUTION Average NOTthe FOR study SALE groups. OR Other DISTRIBUTION variations in the results may arise Meta-analysis allows for combining the effects from all the from the duration and persistence of the health promotion relevant studies to calculate the overall mean effect or the campaigns targeting different population groups within summary estimate. Combining effects from a number of colleges and places of employment. The random effects studies requires that they© Jonesare expressed & Bartlett in the same Learning, units. LLCmodel allows for determining© Jonesthe distribution & Bartlett of effects Learning, LLC among the various studies. The effect size may be affected An estimation of the effectNOT size FOR represents SALE theOR extent DISTRIBUTION of NOT FOR SALE OR DISTRIBUTION the intervention effect. The effect estimates are based on depending on how consistently and accurately the outcomes data obtained from the individual studies, and each study are measured in each study. produces a different estimate of the magnitude of the The random effects model accepts variations in the results intervention effect. While individual effects refers to the between studies and within studies so that the weighting observed© Joneseffects in & separate Bartlett studies, Learning, summary LLCeffects refers of studies combines© Jones both & Bartlettof these variances. Learning, The LLCstudies included in the meta-analysis represent random samples to the NOTpooling FOR of the SALE effects OR from DISTRIBUTION each of the studies in a NOT FOR SALE OR DISTRIBUTION meta-analysis (Khan et al., 2011). of the particular effects. This model aims to estimate the Each study is weighted according to the exactness of mean of the true effects across the studies; therefore, the the statistical calculation of the sample size. Weighting also combined effects would be calculated to determine the mean takes account of differences in measurement error between in that distribution. The argument is that while large studies © Jonesstudies. & Bartlett Studies Learning, that represent LLC more rigorous precision are© Jonespresent & more Bartlett precise Learning, estimates than LLC smaller studies, variations also occur in the effect sizes among these categories of NOT FORgiven SALE weightier OR effectDISTRIBUTION estimates than those that do not meetNOT FOR SALE OR DISTRIBUTION the expected level of rigor and precision. The pooled effects studies. Each effect size represents a separate sample from observed across the studies are statistically calculated as a the population whose mean value is estimated. Therefore, weighted average effect. The process allows for determining in the allocation of weights, balancing is achieved whereby which studies’ results contributed more significantly to the the larger studies might not overpower and unbalance the pooled or sum total—the© summary Jones effect.& Bartlett The contribution Learning, LLCanalysis and small studies might© Jones not be & underestimated. Bartlett Learning, LLC In the random effects model, pooling is feasible when the is proportional to the amountNOT FORof information SALE OR in the DISTRIBUTION study. NOT FOR SALE OR DISTRIBUTION The more empirical studies with larger sample sizes attain range and size of variations are noticeably diverse and heavier weighted average when pooled or amalgamated unexplainable with discrete and disparate effect sizes. than those with smaller sample sizes (Crombie & Davies, Individual studies would be found to have yielded differing 2009). Further clarification of the weighting process is values or ratios of intervention effects (Borenstein et al., provided© Jones here, together & Bartlett with the Learning, combined effect. LLC 2007; Crombie© Jones & Davies, & 2009;Bartlett Sutton Learning, & Higgins, LLC 2008). Understanding the relevance of these models is necessary NOT FOR SALE OR DISTRIBUTION NOT FOR SALE OR DISTRIBUTION The Fixed-Effect Model to enable reviewers to choose the most appropriate model for analyzing selected studies for the systematic review. This model is based on the assumption that there is one true effect size and that all the included studies estimate the same effect size. Therefore, the combined effect represents Sensitivity Analysis © Jonesthat & Bartlett common trueLearning, effect size. LLC The weight allocated to each© JonesSensitivity & Bartlett analysis allows Learning, for comparing LLC the findings from NOT FORstudy SALE would OR represent DISTRIBUTION the amount of information generatedNOT given FOR systematic SALE ORreviews DISTRIBUTION and meta-analyses to determine by that particular study so that the larger studies yielding if these realistically substantiate rigor and validity in the more information would be more heavily weighted and methods applied. The sensitivity tests may help to establish the relatively smaller studies less weighted (Borenstein, the impact of excluding certain studies categorized as outli- Hedges, & Rothstein, 2007; Khan et al., 2011). This model ers with distinctly divergent results. Thus, the initial process upholds that consistency© persistsJones in & all Bartlett the studies Learning, with no wouldLLC involve analysis of the ©findings Jones from & Bartlettall the selected Learning, LLC variations in the size ofNOT treatment FOR effects, SALE thus OR implying DISTRIBUTION studies that meet the criteria forNOT inclusion FOR (Khan SALE et al., OR 2011). DISTRIBUTION that there is no statistical heterogeneity in the treatment Following the initial meta-analysis, a repeat process is carried effects among the studies. Any emerging variation occur- out involving sensitivity testing. In that repeat analysis, those ring within a particular study can be attributed merely to studies that were originally rejected or excluded because of chance. Therefore, the estimated ratios, for example, as poor or questionable quality and/or lack of detailed reports calculated© Jones in each & of Bartlett the studies Learning, would show LLC equivalent would be included© Jones in order & Bartlett to compare Learning, the results. LLC Thus, valuesNOT (Borenstein FOR SALEet al., 2007; OR PolitDISTRIBUTION & Beck, 2008). sensitivity testsNOT afford FOR a means SALE of ORtesting DISTRIBUTION how sensitive the results are to changes in the conduct of the meta-analysis. Random-Effects Model of Meta-Analysis A further test may involve the intervention effects across This model holds that the estimation of treatment effects the subgroups identified in the selected studies. This may could vary between one study and another and that varia- require conducting separate reviews and meta-analyses for © Jonestions & Bartlett can also Learning,occur within LLCthe same study. Variations in© Jonessubgroups & Bartlett of patients Learning, who are likely LLC to respond differently NOT FORresults SALE may OR be influenced DISTRIBUTION by how vigorous the campaigns,NOT to FOR the treatment SALE intervention OR DISTRIBUTION because of different preexisting the type and substance of information, and the amount and medical conditions or specific characteristics. Thus, sensitivity

9781284114942_CH03.indd 60 06/11/17 2:24 pm CHAPTER 3 Systematic Reviews: Consolidating Research Evidence for EBP 61

© Jones & Bartlett Learning, LLC © Jones & Bartlett Learning, LLC analysis allows for taking into consideration the ambiguities Differences in the clinical trials may occur in the form of NOT FORand SALE lack of OR confidence DISTRIBUTION created by information that is lost,NOT crossoverFOR SALE or parallel OR DISTRIBUTION group studies. Variations may occur omitted, or vague and indistinct. More detailed clarification in the thoroughness of allocation concealment or blind- about the related statistical processes of sensitivity analysis ing against detection bias. Varied durations of treatment can be explored in the Cochrane Collaboration Handbook interventions and analysis of intention-to-treat may also and other texts by various© Jones meta-analysts & Bartlett (www.cochrane. Learning, LLCreveal differences among studies.© Jones See Cochrane & Bartlett Handbook Learning, LLC org; Higgins & Green, 2011; Khan et al., 2011). for Systematic Reviews of Interventions: Cochrane Training NOT FOR SALE OR DISTRIBUTION(http://training.cochrane.org/handbook).NOT FOR SALE OR DISTRIBUTION Statistical heterogeneity refers to variations in the treat- The Relevance of Homogeneity and Heterogeneity ment effect. Because meta-analysis is applied to estimate Homogeneity in meta-analysis is based on the assertion the combined effect of numerous studies, it is necessary that the statistical quantification of the pooled results from © Jones & Bartlett Learning, LLC to carefully© examine Jones the & treatmentBartlett effect Learning, in the individual LLC the selected research studies should, for practical purposes, studies. To establish statistical heterogeneity, the estimates of representNOT the FOR combined SALE effect OR from DISTRIBUTION numerous comparable the treatmentNOT effect FOR in the SALE individual OR studies DISTRIBUTION are calculated studies. Many meta-analysts hold the view that it is crucial to ascertain that they are adequately comparable to justify to ensure that the observed effects from the individual studies a combined estimate of effect (Higgins et al., 2003; Khan are adequately comparable, equivalent, and consistent. This et al., 2011). Meta-analysis essentially seeks to combine enables the analyst to convincingly assert that the combined © Jones & Bartlett Learning, LLC © Jonesstudies & thatBartlett have yielded Learning, similar LLC intervention effect. The estimate of results through meta-analysis realistically produces intervention effect may yield positive/favorable or negative/ NOT FORa SALEtrue representation OR DISTRIBUTION across the selected studies (Crombie &NOT unfavorableFOR SALE results. OR DISTRIBUTION Statistical heterogeneity indicates Davies, 2009; Deeks, Altman, & Bradburn, 2001). diversities in the interventions across the studies. These Heterogeneity is commonly examined and tested for reflect clinical as well as methodological heterogeneity with regard to the range of variation and bias that occurs (Polit & Beck, 2008). in the conduct of research studies and their results. Het- © Jones & Bartlett Learning, LLCMoreover, analysts look© for Jones excessive & Bartlettvariation in Learning, the LLC erogeneity relates to the inevitable occurrence of multiple estimate of the treatment effect. The presence of excessive differences among studies.NOT It is FOR important SALE to explore OR DISTRIBUTION in what variation in the observed treatmentNOT FOR effects SALE would OR suggest DISTRIBUTION ways the studies differ, identify the specific types of differ- statistical heterogeneity. In that case, the systematic reviewer ences, and establish how those differences might influence may choose not to carry out meta-analysis, as the result could the effectiveness of a treatment intervention. Examples of lead to drawing conclusions that are ambiguous and misrep- diverse factors that may impact on the nature and extent © Jones & Bartlett Learning, LLC resent the observed© Jones effects & Bartlett in the studies Learning, (Khan et al LLC., 2011). of intervention effect include the type of disease, serious- Alternative processes of random effects model, subgroup ness NOTof the FORcondition, SALE and ORthe availableDISTRIBUTION resources. Other analysis, andNOT meta-regression FOR SALE may OR be applied.DISTRIBUTION These allow factors include the environment of care delivery, quality of for establishing the reasons for the occurrence of variations care provided, and the major consequence of the disease in treatment effects in different studies. Brief overviews of or medical condition, such as death or disability of varied these processes are provided. durations. Importantly, interpretations from this technique © Jones &depend Bartlett to a large Learning, extent on theLLC number of studies examined© Jones & Bartlett Learning, LLC NOT FOR(Higgins, SALE Thompson,OR DISTRIBUTION Deeks, & Altman, 2003). NOT Meta-RegressionFOR SALE OR and DISTRIBUTION Subgroup Analysis Depending on the nature and degree of diversity, the Calculation of the combined effect size would be question- factors may be categorized as clinical, methodological, able and probably lead to inaccurate clinical decisions if or statistical heterogeneity. These effectively describe the wide-ranging heterogeneity across the studies is disregarded. identified source of the diversity among the studies. Clinical Therefore, it is crucial that review teams and practitioners heterogeneity (diversity)© Jonesrefers to &patient-related Bartlett Learning, factors. LLCshould ascertain if the treatment© Jones effect is& likelyBartlett to vary Learning, in LLC Thus, gender, age, the NOTexact diagnosisFOR SALE and severityOR DISTRIBUTION of the different circumstances and NOTin what FOR ways. SALE The disparity OR DISTRIBUTION in condition, and other prescribed medications the patients the effects could make the appropriateness of this stage of the may be using at the time constitute clinical heterogeneity meta-analysis questionable and unjustified. Meta-regression (Higgins et al., 2003; Khan et al., 2011). allows for examining the types and nature of diversities and Differences in the research treatment interventions in the extent to which particular factors influence the intervention terms© of Jones the exact & Bartletttypes, dosage, Learning, mode(s) ofLLC administra- outcomes and© Jones indeed, the& Bartlett effect size Learning,(Polit & Beck, LLC 2008). tion, NOTvariations FOR in theSALE intensity OR of DISTRIBUTION the regimen also fall into SubgroupNOT analysis FOR SALEfocuses ORthe meta-analysis DISTRIBUTION on the the category of clinical heterogeneity. Varied definitions of specific subgroups of participants involved in the studies. In outcomes; the processes employed to detect, record, and order to avoid errors and drawing ambiguous conclusions, measure them; and varied types of clinical settings and early determination of what groupings will be investigated available resources have also been classified as clinical and analyzed is important. Experts maintain that carrying © Jones &heterogeneity Bartlett Learning, (Higgins et al LLC., 2003; Khan et al., 2011). © Jonesout subgroup & Bartlett analysis Learning, at the later LLCstage of the meta-analysis NOT FOR SALEMethodological OR DISTRIBUTION heterogeneity refers to diversity in theNOT andFOR after SALE the results OR haveDISTRIBUTION been calculated could create bias study designs in terms of how the studies were conducted. in the reporting of the results.

9781284114942_CH03.indd 61 06/11/17 2:24 pm 62 PART I Exploration of the Concept of Evidence-Based Practice and Related Practical Challenges

© Jones & Bartlett Learning, LLC © Jones & Bartlett Learning, LLC Subgroup analysis may not give clear or full information NOT FOR SALE OR DISTRIBUTION NOT FOR SALE OR DISTRIBUTION about how specific variations in the mode of administration Reflective Considerations of the intervention treatment affected the outcomes (Khan et al., 2011; Noyes et al., 2011). Additionally, subgroup analysis The following reflective activity can be undertaken may not reveal or confirm exactly how varied are those ob- on individual basis, in pairs, or in small groups. The served outcomes among© the Jones subgroups & Bartlettfrom the population Learning, LLCsummary outline is intended© Jones to provide & Bartlettideas that Learning, LLC studied. Any observed variations could be attributable to may serve as directions and/or broad objectives for particular factors in theNOT characteristics FOR SALE of one OR subgroup DISTRIBUTION or systematic review teams orNOT colleagues FOR SALEin the multi OR- DISTRIBUTION another. Experts suggest that conclusions from subgroup disciplinary team. analysis should be viewed with caution and interpreted Taking account of the setting, participants, inter- tentatively because they derive from subdivisions of studies ventions, outcomes, research designs, and methods and contrasts rather than precise scientific statistical tests. employed in the studies, carefully reflect on the © Jones & Bartlett Learning, LLC following:© Jones & Bartlett Learning, LLC While meta-analysis allows for investigating the extent NOT FOR SALE OR DISTRIBUTION NOT FOR SALE OR DISTRIBUTION to which specific study characteristics might be associated ●● Consider the problem of hospital-acquired with specific intervention effects, meta-regression goes a infection and formulate a problem title, stage further. For example, meta-analysis could be used questions, and objectives to be explored for for investigating the impact of carefully planned and systematic review with a view to implementing a feasible intervention in the context of EBP. © Jonesmethodically & Bartlett deliveredLearning, behavior-change LLC program for one© Jones & Bartlett Learning, LLC group of study participants and the observed results may ●● Consider the preliminary stages of literature NOT FORshow SALE greater OR degree DISTRIBUTION of effective positive outcome. HowNOT- FOR SALEsearch and OR perusal DISTRIBUTION activities. ever, a participant group (a control group) who were not ●● Consider the development of a protocol and exposed to that behavior-change program would show develop an outline of the key components different outcome effects. This means that two subgroups that should be incorporated for an achievable under investigation may© yieldJones different & Bartlett treatment Learning, effects LLC in-house EBP implementation.© Jones & Bartlett Learning, LLC in meta-analysis due to certain factors in the study design. ●● Consider which members of the multidisci- Meta-regression providesNOT FOR a means SALE to ORidentify DISTRIBUTION the plinary team could effectivelyNOT FOR participate SALE in theOR DISTRIBUTION specific factors that may have contributed to disparities systematic review team. (Borenstein, Hedges, Higgins, & Rothstein, 2009). How- ●● Consider which clinical leads could be involved ever, meta-regression may not be entirely appropriate for in an advisory committee. assessing© Jones differences & Bartlettin treatment Learning, effects. LLC ●● Discuss© Jones the stages & Bartlett of the actual Learning, literature LLC Various processes have been proposed such as determin- review and possible components of a simple ing theNOT relationship FOR SALE between OR specific DISTRIBUTION factors, study and/or formatNOT for FORdata extraction SALE andOR consider DISTRIBUTION a clear patient characteristics, and the magnitude of effect observed format for data abstraction. in each study. Nevertheless, systematic reviewers, practitio- ●● Consider the selection of the pertinent stud- ners, and decision-makers are cautioned that these tests are ies; outline the main criteria for inclusion and exclusion. © Jonesnot & Bartlettentirely flawless. Learning, They LLCmay have low statistical power© Jones & Bartlett Learning, LLC and may fall short of revealing all the disparities present ●● Consider the key components of quality ap- NOT FORin theSALE studies OR and DISTRIBUTION the results. Meta-regression requires thatNOT FOR SALEpraisal ofOR the DISTRIBUTIONselected research studies for each study be allocated an appropriate estimated weight. classification of the levels of evidence. Moreover, careful decision should be made regarding an ap- ●● Consider what factors may influence meta- propriate effects model that is applicable to meta-regression. analysis or meta-synthesis of the findings from the included studies. Many experts consider© the Jones random-effects & Bartlett model Learning, to be LLC © Jones & Bartlett Learning, LLC more appropriate for analyzing variations between studies ●● Consider the question, “Is the review team for the purposes meta-regressionNOT FOR (Borenstein SALE ORet al .,DISTRIBUTION 2009). confident that combiningNOT and FOR meta-analyzing SALE OR DISTRIBUTION Summary data incorporates the averages of effect size, the study results/findings would be pragmatic, severity of disease and length of follow-up. Meta-regression meaningful, and consequential for guiding clini- seeks to explore the influence of study characteristics on cal decisions?” the size© of Jones the effects & Bartlettobserved in Learning, systematic reviews LLC and/or ●● Consider© Jones possible & Bartlettintervention(s) Learning, that might beLLC meta-analysis. However, meta-regression is unable to directly feasible to implement in this scenario. NOT FOR SALE OR DISTRIBUTION NOT FOR SALE OR DISTRIBUTION link specific patient factors to the size of treatment effect. ●● Consider what additional steps might be Nonetheless, the size of treatment effect may be lost if con- necessary to produce substantial informa- tinuous data with constant/countless number of values were tion for drawing conclusions and making converted to dichotomous data with only two possible values recommendations to inform relevant clinical decisions and action plans by the policy/ © Jonesof & presence/existence Bartlett Learning, or absence/non-existence. LLC Therefore,© Jones & Bartlett Learning, LLC regression based on individual patient data may have to be decision-makers. NOT FORemployed SALE to OR address DISTRIBUTION that limitation (Borenstein et al., 2009).NOT FOR SALE OR DISTRIBUTION

9781284114942_CH03.indd 62 06/11/17 2:24 pm CHAPTER 3 Systematic Reviews: Consolidating Research Evidence for EBP 63

© Jones & Bartlett Learning, LLC © Jones & Bartlett Learning, LLC Reporting and Appraising the Meta-Analysis it useful to read Popay et al.’s (2006) Guidance to the NOT FOR SALE OR DISTRIBUTION NOT FOR SALE OR DISTRIBUTION The following section considers the importance of detailed Conduct of Narrative Synthesis in Systematic Reviews and reporting of completed meta-analysis. The framework Rodgers et al.’s (2009) Testing Methodological Guidance on Preferred Reporting Items for Systematic Reviews and the Conduct of Narrative Synthesis in Systematic Reviews. Meta-Analysis (PRISMA) is examined here. Practitioners The framework for narrative synthesis comprises four are encouraged to explore© Jones and to & carefully Bartlett examine Learning, the LLCkey elements, each of which© are Jones addressed & Bartlett by employing Learning, LLC particular tools and techniques (Popay et al., 2006). The full PRISMA statementNOT together FOR with SALE the ORunderpinning DISTRIBUTION NOT FOR SALE OR DISTRIBUTION principles, recommendations, and correct application. important thing is to carefully consider which tool would The following key elements derive from the PRISMA be appropriate for fulfilling the purpose of the review checklist and may usefully guide correct development, question and the evidence being synthesized. reporting, and evaluating to finalize completion of in-house© Jones systematic & Bartlett review projects Learning, and meta-analysis. LLC Transforming© Jones Data & Bartlett Learning, LLC Detailed,NOT accurate, FOR SALE and transparent OR DISTRIBUTION reporting is essential Different statisticalNOT FOR counts SALE and measurements OR DISTRIBUTION are used in if effective judgment is to be made for possible appli- presenting the data from research studies, and, in quantitative cation to evidence-based clinical decision making and studies, the type of data is predominantly numerical. The care provision. initial figures may be in the form of raw data, which may Box 3-5 presents concise outlines of the main com- be calculated into summary data or expressed in different © Jones &ponents Bartlett of the Learning, PRISMA frame LLC and draws on the model© Jonesstatistical & Bartlett figures. InLearning, order to enhance LLC the accuracy of the NOT FORdeveloped SALE OR by Moher, DISTRIBUTION Liberati, Tetzlaff, and Altman (2009).NOT descriptionFOR SALE of theOR effects, DISTRIBUTION devising a common system of Practitioners are encouraged to examine these critically and measures to transform the data is necessary. For example, compare with the original authors’ model. dichotomous data may be converted or transformed into odds ratio, risk ratio, relative risk, and risk difference. Continuous data may be presented in the form of weighted The Role of Narrative© Jones Synthesis & Bartlett Learning, LLCmean difference or standard© Jonesmean difference. & Bartlett For Learning, the LLC NOT FOR SALE OR DISTRIBUTIONpurposes of meta-analysis,NOT results FOR from SALE studies ORmay DISTRIBUTIONbe Narrative synthesis involves synthesis/integration of evidence combined to produce an estimate of effect (Popay et al., relating to effectiveness and other questions in a narrative. 2006; Rodgers et al., 2009). Transforming data helps pres- The process involves compiling the findings from studies ent results in easy-to-understand ways, and the range of included in a systematic review and summarizing them in effects from research interventions can be measured and textual© Jonesform (Rodgers & Bartlett et al., Learning,2009). The current LLC recom- calculated ©more Jones accurately. & Bartlett This statistical Learning, technique LLC is not mendationNOT isFOR that all SALE numeric OR tables DISTRIBUTION presented in systematic applicable NOTto narrative FOR synthesis.SALE OR DISTRIBUTION reviews should be accompanied by narrative text. Apart from providing an explanation and summary of the main Vote-Counting characteristics of the results and findings of the included Vote-counting provides another tool for producing descrip- studies, it also provides an analysis of the variations within tive summary data and involves counting up and calculating © Jones &and Bartlett between Learning, studies (Noyes, LLC Popay, Pearson, Hannes,© Jones & Bartlett Learning, LLC the frequency of different types of research results from the & Booth, 2008). An overall judgment of the strength of NOT FOR SALE OR DISTRIBUTION NOT studiesFOR SALEincluded OR in the DISTRIBUTION review. Vote-counting can be used the evidence should also be stated. Narrative synthesis to establish patterns across the studies. In reviews relating is necessarily carried out when the studies are found to to evaluation of intervention effects, vote-counting may be be overly diverse in terms of clinical and methodological used for tabulating significant and non-significant results variations to allow for combining the data in meta-analysis. (Popay et al., 2006). Thus, this tool is useful for recording However, some amount© of Jones narrative & description Bartlett is Learning, necessary LLC © Jones & Bartlett Learning, LLC findings that may be tabulated according to the direction to integrate and interpret the evidence. Nevertheless, it is NOT FOR SALE OR DISTRIBUTIONof the effect. More currentNOT techniques FOR ofSALE vote-counting OR DISTRIBUTION contended that narrative synthesis is not strictly objective continue to be proposed, and the idea of counting and but can be potentially subjective compared to the rigor and calculating by categories and allocation of weights and precision of meta-analysis. To avoid or minimize potential scores have emerged as alternative concepts. Despite its bias, the process should be performed in a strictly methodical potential usefulness, this tool tends to be considered ques- manner,© Jones detailed &and Bartlett transparent Learning, enough to LLCbe repeatable, © Jones & Bartlett Learning, LLC tionable, overly simplistic, and not very informative to although this could prove to be difficult to achieve (Noyes NOT FOR SALE OR DISTRIBUTION provide adequateNOT FOR statistical SALE detail OR and DISTRIBUTION may not be chosen et al., 2011; Rodgers et al., 2009). for high-level synthesis. Nonetheless, some prefer to use it as a way of establishing occurrence or non-occurrence Framework for Narrative Synthesis and for counting frequencies. Therefore, it may be useful © Jones &Different Bartlett frameworks Learning, have LLC been proposed as a guide for© Jonesin supporting & Bartlett specific Learning, observations LLC made in the studies. carrying out narrative synthesis and specific formal guid- Similar to the different practices of vote-counting, var- NOT FORance SALE should OR be DISTRIBUTIONexplored. In particular, readers would findNOT iedFOR interpretations SALE OR of DISTRIBUTION the counts have also been proposed.

9781284114942_CH03.indd 63 06/11/17 2:24 pm 64 PART I Exploration of the Concept of Evidence-Based Practice and Related Practical Challenges

© Jones & Bartlett Learning, LLC © Jones & Bartlett Learning, LLC NOT FOR SALE OR DISTRIBUTION NOT FOR SALE OR DISTRIBUTION Box 3-5 Concise Outline of the Key Elements in the Appraising and Reporting of Meta-Analysis Using the PRISMA Frame

PRISMA: The principles© Jones and guidelines & Bartlett Learning, LLC●● Additional analyses in terms© Jones of pre-specified & Bartlett meth- Learning, LLC Title ods such as sensitivity analysis, meta-regression, NOT FOR SALE OR DISTRIBUTIONsubgroup analysis NOT FOR SALE OR DISTRIBUTION This should inform the wider target audience what this evaluation is about. If both systematic review and meta- Results analysis had been carried out these should be clearly ●● Study selection with use of a flow diagram indicated in the title. © Jones & Bartlett Learning, LLC ●● Study characteristics© Jones & including Bartlett PICOS, Learning, study size, LLC Abstract and duration of follow-up and relevant citations NOT FOR SALE OR DISTRIBUTION NOT FOR SALE OR DISTRIBUTION The abstract should be competently written to convey what ●● Risk of bias within studies and outcome level the report is about in terms of the focus, context, and the assessment processes involved in the meta-analysis. ●● Results of individual studies in terms of simple sum- Introduction mary data with compliance to the recommended © Jones & BartlettThe introduction Learning, should clearly LLC state the rationale and the objec©- Jonesuse & ofBartlett forest plot Learning, LLC NOT FOR SALEtives for ORthe review DISTRIBUTION and meta-analysis if both were performed.NOT FOR●● Synthesis SALE of ORresults DISTRIBUTION with presentation of the main review results and if meta-analysis done inclusion Methods of the confidence intervals and measures of consis- The required details for this section should include the tency for each following: ●● Risk of bias across studies with presentation of the ●● Protocol and details of registration if applicable © Jones & Bartlett Learning, LLCresults of the assessment ©or Jonesthe risk bias & Bartlett Learning, LLC ●● Eligibility criteria ●● NOT FOR SALE OR DISTRIBUTIONAdditional analysis relatingNOT to the FOR presentation SALE OR DISTRIBUTION ●● Information sources and databases used and direct of the results of the subgroup analysis, sensitivity contacts with researchers analysis, and meta-analysis ●● Search in terms of full search strategy Discussion ●● Study selection in terms of the processes of screen- ●● Summary of the key findings and the significance ing and inclusion © Jones & Bartlett Learning, LLC and relevance© Jones of each & Bartlettmain outcome, Learning, to the major LLC NOT●● Data FOR collection SALE process OR in termsDISTRIBUTION of the data extraction, groups NOTof stakeholders FOR SALE OR DISTRIBUTION pilot testing, and measures taken to obtain and verify ●● Limitations in terms of examining the risk of bias required details from relevant original researchers at study and outcome level, biases relating to in- ●● Data items in terms of all the key variables for complete study retrieval and reporting bias at the which data were sought review level

© Jones & Bartlett●● Risk of Learning, bias in individual LLC studies, assessed and at © Jones●● Conclusion & Bartlett conveying Learning, a general LLC interpretation of NOT FOR SALEwhat OR levels DISTRIBUTION and how applied to the synthesizing of NOT FORthe SALEresults in OR the contextDISTRIBUTION of other evidence and im- the data plications for future research ●● Summary measures employed to risk ratio or differ- Funding ence in mean ●● ●● Synthesis of results in terms of the detailed Funding should be addressed in terms of the sources methods employed© Jones in amalgamating & Bartlett the results Learning, LLCfor the systematic review, the© Jonesrole of the & funders, Bartlett and Learning, LLC of the studies andNOT consistency FOR SALE testing inOR the DISTRIBUTIONenvisaged implications forNOT the supply FOR of dataSALE. OR DISTRIBUTION meta-analyses Readers are encouraged to explore the full document ●● Risk of bias across studies in terms of selective report- for more detailed information. (Partly draws on Moher ing and publication bias and the potential impact et al., 2009) Modified from Moher, D., Liberati, A., Tetzlaff, J., Altman, D.G., & The Prisma Group. (2009). Preferred reporting items for systematic reviews© Jones and meta-analyses: & Bartlett The Learning, PRISMA Statement. LLC PLoS Med, 6(7), e1000097. © Jones & Bartlett Learning, LLC NOT FOR SALE OR DISTRIBUTION NOT FOR SALE OR DISTRIBUTION

In relation to vote-counting by categories, the interpretation et al., 2009). However, this concept is questionable be- is based on the number of studies in each category so that cause of equal weighting of studies with dissimilar sample the category with the highest count of studies is placed sizes and effect sizes. Contrariwise, the category with the © Jonestopmost. & Bartlett The topLearning, category LLCis considered as showing the© Jonesleast vote-counts & Bartlett is consideredLearning, as LLC carrying low statistical NOT FORtrue SALE size of OR effect DISTRIBUTION estimate and carrying a higher level ofNOT significance FOR SALE and OR is therefore DISTRIBUTION placed at the bottom. Thus, statistical significance as compared to the other categories interpretation of results from vote-counting should be with a lesser number of studies (Popay et al., 2006; Rodgers considered only tentatively in the synthesis of data.

9781284114942_CH03.indd 64 06/11/17 2:24 pm CHAPTER 3 Systematic Reviews: Consolidating Research Evidence for EBP 65

© Jones & Bartlett Learning, LLC © Jones & Bartlett Learning, LLC NOT FOR SALETable 3-4OR DISTRIBUTIONTools and Techniques to Explore RelationshipsNOT WithinFOR andSALE Among OR Studies DISTRIBUTION Tool/Technique Examples/Notes

Graphs Frequency of distribution—# of times a variable appears Forest plots—results of individual studies © Jones & Bartlett Learning, FunnelLLC plots—establish biases ©in smallerJones studies & Bartlett Learning, LLC NOT FOR SALE OR DISTRIBUTION NOT FOR SALE OR DISTRIBUTION Other graphical tools: • Conceptual models • Idea webbing • Concept mapping • Investigator,© Jones methodologies, & Bartlett andLearning, conceptual LLC triangulation © Jones & Bartlett Learning, LLC NOT FOR SALE OR DISTRIBUTION NOT FOR SALE OR DISTRIBUTION Translation Comparable accounts (reciprocal) Opposing accounts (refute translations) Qualitative case descriptions Describes differences in statistical findings among studies © Jones & Bartlett Learning, LLC © Jones & Bartlett Learning, LLC NOT FOR SALE OR DISTRIBUTION NOT FOR SALE OR DISTRIBUTION

Translation of Data familiarity with the related theorization and concepts Thematic analysis allows for identifying and analyzing the enables the researcher to apply qualitative principles in key concepts and themes that repeatedly occur among several determining and describing the relevant categories of the selected studies (Thomas© Jones& Harden, & Bartlett2008). The Learning, technique LLCkey elements in the findings© (Flemming, Jones & 2010b).Bartlett Learning, LLC is conventionally usedNOT in qualitative FOR SALE research OR analysis DISTRIBUTION but Table 3-4 lists the main toolsNOT and FOR techniques SALE that OR can DISTRIBUTION be many researchers also use this in mixed qualitative and used for exploring relationships within and among studies. quantitative research studies and even in some quantitative studies (Flemming, 2010a). Just as conceptual themes are Multiple Terminologies and Models of identified in qualitative research data, thelabels applied to Synthesizing Qualitative Research Evidence specific© Jones variables & in Bartlett quantitative Learning, research can LLC also be iden- © Jones & Bartlett Learning, LLC tifiedNOT as themes FOR or SALE concepts. OR Therefore, DISTRIBUTION although thematic There is anNOT assumption FOR SALE that the OR different DISTRIBUTION approaches to analysis was originally developed for application to primary synthesizing share the common principle of pooling findings qualitative research studies, the data may be converted into from primary qualitative research studies (Finfgeld-Connett, frequencies. While this technique involves an inductive pro- 2010). However, while some approaches focus simply on the cess, the conclusions drawn and the conceptual assumptions process of synthesizing the data, others adopt a wider scope. © Jones &must Bartlett derive directly Learning, from the LLC research findings (Noyes et ©al., JonesThe latter & Bartlett approaches Learning, tend to encompass LLC the essential elements NOT FOR2011; SALE Popay OR et al DISTRIBUTION., 2006; Rodgers et al., 2009). NOT andFOR processes SALE from OR problem DISTRIBUTION identification to appraisal of the Some systematic reviewers question the degree of quality of research studies and writing a full report. Some transparency in thematic analysis because it may not be consider the idea of synthesizing the findings from several entirely straightforward to determine how the themes and qualitative research studies as questionable and not possible concepts were developed or at what stages of the study. The to achieve. They maintain that the varied methodologies with results of the synthesis ©involving Jones thematic & Bartlett analysis Learning, may not LLCdifferent theoretical basis that© Jonestypify qualitative & Bartlett research Learning, LLC be similar enough to synthesizeNOT FOR based SALE on a theory-guidedOR DISTRIBUTION could present potential flawsNOT in the FOR process SALE of synthesizing. OR DISTRIBUTION approach. Therefore, to strengthen the transparency of Systematic reviewers and professional practitioners are en- thematic analysis, details of the entire analysis must be couraged to read Barnett-Page and Thomas’s (2009) critical clearly and meticulously explained in the researcher’s report review of the methods for the synthesis of qualitative research. (noyes et al, 2011; Popay et al., 2006; Rodgers et al., 2009). Synthesizing and integrating the findings from mixed and Content© Jones analysis & Bartlett has been Learning, explained asLLC a technique varied qualitative© Jones methodologies & Bartlett can proveLearning, useful for LLC making for condensingNOT FOR the SALE amount OR of content DISTRIBUTION in a text into much and supportingNOT clinical FOR decisions SALE OR(Finfgeld-Connett, DISTRIBUTION 2010). reduced content. The textual descriptive data is condensed and structured into fewer categories by applying specific rules and coding system (Popay et al., 2006; Rodgers et al., Alternative Approaches to Qualitative 2009). In this way, the technique of content analysis serves Research Synthesis © Jones &the Bartlett purpose of Learning, arranging, categorizing, LLC and summarizing the© Jones & Bartlett Learning, LLC NOT FORresearch SALE findings, OR DISTRIBUTION which should be carried out methodically.NOT TheFOR various SALE descriptive OR DISTRIBUTION terms applied to the different ap- The quantitative element of content analysis requires that proaches are quite numerous and may prove to be confus- the data should be converted into frequencies. However, ing, and as many as 15 have been proposed. Some approve

9781284114942_CH03.indd 65 06/11/17 2:24 pm 66 PART I Exploration of the Concept of Evidence-Based Practice and Related Practical Challenges

© Jones & Bartlett Learning, LLC © Jones & Bartlett Learning, LLC the characteristics of qualitative methodology while others the choice of practice or the process should be guided by NOT FORadvocate SALE converting OR DISTRIBUTION qualitative results and findings toNOT the FOR review SALE question, OR theDISTRIBUTION quantity of relevant studies, and quantitative form such as content analysis. However, this the knowledge and competence within the team for con- practice is considered unnecessary. Some terms are used ducting systematic reviews and synthesis of the research ambiguously or interchangeably, such as meta-synthesis and findings. A range of practices are explored in Table 3-5. meta-study to describe any© Jones form of qualitative& Bartlett synthesis. Learning, In LLCThe challenge of qualitative© Jones research & Bartlett synthesis Learning, in LLC particular, the use of the term meta-synthesis is challenged systematic reviews is choosing an approach that will help because it is not specificNOT to qualitativeFOR SALE research OR andDISTRIBUTION often to produce pertinent conclusionsNOT that FOR are convincing.SALE OR It DISTRIBUTION is used incorrectly. Experts argue that this depends on the level crucial that findings from the synthesis are presented in a at which this term is applied, whether the synthesizing or way that would be easy for policy makers, decision-makers, the level of included studies (Dixon-Woods, Agarwal, Jones, and practitioners to make sense of and translate into the Young,© & Jones Sutton, &2005). Bartlett Despite Learning, all these considerations, LLC context of clinical© Jones practice. & Bartlett Learning, LLC NOT FOR SALE OR DISTRIBUTION NOT FOR SALE OR DISTRIBUTION

Table 3-5 Alternative Approaches to Qualitative Research Synthesis Alternative Approach Examples/Notes

© Jones Meta-ethnography& Bartlett Learning, LLC Comparing and synthesizing© Jones study results& Bartlett to establish Learning, if the same LLC concepts occur in NOT FOR(reciprocal SALE ORtranslation DISTRIBUTION analysis) different studies NOT FOR SALE OR DISTRIBUTION Grounded theory synthesis Develop concepts from the qualitative data by carrying out simultaneous coding and (constant comparative method) analyzing Four stages: © Jones & 1. BartlettComparing Learning, occurrences thatLLC relate to individual categories© Jones & Bartlett Learning, LLC NOT FOR 2. SALE Integrating OR theDISTRIBUTION categories according to their properties NOT FOR SALE OR DISTRIBUTION 3. Delimiting the emerging theory 4. Reporting in detail Meta-study Three phases: © Jones & Bartlett Learning, 1. Meta-theory LLC © Jones & Bartlett Learning, LLC NOT FOR SALE OR DISTRIBUTION 2. Meta-method NOT FOR SALE OR DISTRIBUTION 3. Meta-data analysis Critical interpretive synthesis Draws on meta-ethnography and incorporates elements of grounded theory Thematic synthesis Three stages: © Jones & Bartlett Learning, LLC 1. Developing codes of© themes Jones from & the Bartlett word-based Learning, findings LLC NOT FOR SALE OR DISTRIBUTION 2. Organizing the codesNOT into appropriateFOR SALE categories OR DISTRIBUTION of descriptive themes 3. Producing analytical themes Textual narrative Suitable for organizing the studies into homogenous sets Meta-narrative Synthesize research findings addressing a variety of theories © Jones & Bartlett Learning, LLC © Jones & Bartlett Learning, LLC Meta-summary NOT FORInvolves SALE converting OR DISTRIBUTION qualitative findings into quantitative formNOT andapplying FOR SALE statistical OR DISTRIBUTION techniques for analyzing data Meta-interpretation Ultimate goal is to establish meaning through methodical interpretation Qualitative cross-case analysis A typical feature of this approach it the use of tables or matrices to summarize the data © Jones & Bartlett Learning,across qualitative LLC and quantitative research© Jonesstudies & Bartlett Learning, LLC RealistNOT synthesis FOR SALE OR DISTRIBUTIONApplications of this approach focus on complexNOT socialFOR interventions SALE OR to DISTRIBUTION establish the reasons why these may be successful or unsuccessful in certain contexts Framework synthesis Organizes qualitative data using numerical codes for indexing and presents the data in graphs or diagrams © Jones Ecological& Bartlett triangulation Learning, LLC Aims to establish associations© Jones between & individuals,Bartlett populationLearning, groups, LLC behaviors, age, gender, ethnicity, interventions, outcome, environments, and specific settings NOT FOR SALE OR DISTRIBUTION NOT FOR SALE OR DISTRIBUTION

9781284114942_CH03.indd 66 06/11/17 2:24 pm CHAPTER 3 Systematic Reviews: Consolidating Research Evidence for EBP 67

© Jones & Bartlett Learning, LLC © Jones & Bartlett Learning, LLC The following references provide additional sources implementing, and evaluating systematic reviews are NOT FORfor SALE more detailOR DISTRIBUTION on approaches to synthesizing qualitativeNOT carefullyFOR SALE selected OR toDISTRIBUTION produce practical and realistic research evidence: conclusions and recommendations. • Barnett-Page, E., & Thomas, J. (2009). Methods Clearly, efficient implementation of evidence-based for the synthesis of qualitative research: A critical practice is crucial and it is important that practitioners review. BMC Medical© Jones Research & Bartlett Methodology, Learning, 9, 59. LLCadopt EBP with adequate understanding© Jones & Bartlettand familiarity Learning, LLC with the related concepts, principles, and processes. This • Campbell, R., Pound, P., Morgan, M., Daker-White, NOT FOR SALE OR DISTRIBUTIONpractice should be in placeNOT for examiningFOR SALE all in-houseOR DISTRIBUTION G., Britten, N., Pill, R., . . . Donovan, J. (2011). reports from systematic reviews and meta-analysis as well Evaluating meta-ethnography: Systematic analysis as published complete reports that have relevance to the and synthesis of qualitative research. Health Tech- particular field and area of practice. nology Assessment, 15(43), 1–164. © Jones & Bartlett Learning, LLC The need© Jonesto pursue & appropriate Bartlett Learning, training has LLCbecome • Garside, R. (2008). Methods for synthesizing quali- inevitable in contemporary professional practice for acquiring NOTtative FOR evidence SALE [Online OR workshop]. DISTRIBUTION Retrieved from adequate knowledgeNOT FOR and SALE insight ORabout DISTRIBUTION systematic reviews Centre for Knowledge Translation for Disability linked to evidence-based practice. Review teams and their and Rehabilitation Research (KTDRR) website: colleagues in multidisciplinary settings should engage in at http://www.ktdrr.org./training/workshops/qual formally organized and informal reflective activities prior /index.html © Jones & Bartlett Learning, LLC © Jonesto and & following Bartlett every Learning, in-house systematicLLC review project. • Pawson, R., Greenhalgh, T., Harvey, G., & Walshe, Therefore, the significance of appropriate preparation NOT FOR SALEK. OR(2004). DISTRIBUTION Realist synthesis: An introduction.NOT andFOR efficient SALE implementation OR DISTRIBUTION must also receive careful Manchester, United Kingdom: ESRC Research consideration. Methods Programme. Retrieved from http://www .researchgate.net/publication228855827_Realist REFERENCES _Synthesis_An_Introduction.© Jones & Bartlett Learning, LLCArmijo-Olivo, S., Stiles, C. R.,© Hagen,Jones N. & A., Bartlett Biondo, P.Learning, D., LLC • Williams, T. L., NOT& Shaw, FOR R. L. SALE (2016). OR Synthesising DISTRIBUTION Cummings, G. G. (2012). AssessmentNOT FOR of studySALE quality OR DISTRIBUTIONfor qualitative research: Meta-synthesis in sport and systematic reviews: A comparison of the Cochrane Collabora- exercise. Routledge Handbook of Qualitative Res tion Risk of bias tool and the Effective public health practice Sport and Exercise. Chapter 21. Williams, T. L. & project quality assessment tool: Methodological Research. Journal of Evaluation in Clinical Practice, 18(1), 12–18. Shaw R. L. Abingdon, United Kingdom: Routledge. © Jones & Bartlett Learning, LLC Barnett-Page,© E., Jones & Thomas, & Bartlett J. (2009). Methods Learning, for the LLCsynthesis of qualitative research: A critical review. London, United Kingdom: NOT FOR SALE OR DISTRIBUTION Evidence forNOT Policy FOR and Practice SALE Information OR DISTRIBUTION and Co-ordinating Conclusion (EPPI) Centre, Social Science Research Unit. Borenstein, M., Hedges, L. V., & Rothstein, H. R. (2007). Systematic reviews, meta-analysis, and synthesis of quali- Meta-analysis: Fixed effect vs. random effects. Retrieved from tative research evidence are crucial for the implementa- https://www.meta-analysis.com/downloads/M-a_f_e_v_r © Jones &tion Bartlett of research Learning, findings LLC to substantiate all aspects ©of Jones _e_sv.pdf & Bartlett Learning, LLC NOT FORevidence-based SALE OR DISTRIBUTION practice. Many healthcare practitionersNOT Borenstein,FOR SALE M., Hedges, OR DISTRIBUTION L. V., Higgins J. P. T., & Rothstein, H. share the view that systematic reviews are intended to R. (2009). Introduction to meta-analysis. Hoboken, NJ: John advise, update, and substantiate policy decisions and Wiley & Sons Ltd. professional practice. Recommendations to guide profes- Brown, P., Brunnhuber, K., Chalkidou, K., Chalmers, I., Clarke, C., sional practice are based on extensive and thoroughly Fenton, M., . . . Young, P. (2006). How to formulate research examined evidence from© high-qualityJones & Bartlettempirical studiesLearning, and LLCrecommendations. BMJ, 333(7572),© Jones 804–806. & Bartlett Learning, LLC are therefore consideredNOT to beFOR trustworthy SALE OR for supportDISTRIBUTION- Brown, C. E., Wickline, M. A.,NOT Ecoff, FOR L., & SALEGlaser, D. OR (2009). DISTRIBUTION ing decisions about care and service provision. However, Nursing practice, knowledge, attitudes and perceived barri- many practitioners are challenged by a deluge of new ers to evidence-based practice at an academic medical centre. research evidence, new information, and new additional Journal of Advanced Nursing, 65(2), 371–381. professional knowledge. Centre for Reviews and Dissemination. (2009). Systematic reviews: This© Jones chapter & set Bartlett out to exploreLearning, systematic LLC reviews CRD’s Guidance© Jones for undertaking & Bartlett reviews Learning, in health care. LLCYork, and NOTthe synthesizing FOR SALE of ORquantitative DISTRIBUTION and qualitative United Kingdom:NOT Author. FOR SALE OR DISTRIBUTION research evidence. A range of related methods and Crombie, I. K., & Davies, H. T. O. (2009). What is meta-analysis? processes have been examined including meta-analysis London, United Kingdom: Hayward Medical Communications, and content analysis applied to quantitative and mixed- Hayward Group Ltd. method research as well as different approaches to Deeks, J. J., Altman, D. G., & Bradburn, M. J. (2001). Statistical © Jones &qualitative Bartlett research Learning, synthesis. LLC It is important that the© Jonesmethods & forBartlett examining Learning, heterogeneity LLC and combining results from NOT FORmethods SALE ORselected DISTRIBUTION for exploring, explaining, planning,NOT severalFOR studiesSALE in meta-analysis.OR DISTRIBUTION In M. Egger, G. D. Smith, & D. G.

9781284114942_CH03.indd 67 06/11/17 2:24 pm 68 PART I Exploration of the Concept of Evidence-Based Practice and Related Practical Challenges

© Jones & Bartlett Learning, LLC © JonesAltman & Bartlett (Eds.), Systematic Learning, reviews LLC in health care: Meta-analysis in for inclusion of qualitative research in cochrane systematic NOT FORcontext SALE. (2nd ORed.). London,DISTRIBUTION United Kingdom: BMJ Publishing Group.NOTreviews FOR ofSALE interventions. OR DISTRIBUTION Cochrane Collaboration Qualitative Dixon-Woods, M., Agarwal, S., Jones, B., Young, B., & Sutton, A. Methods Group. Retrieved from http://cqrmg.cochrane.org (2005). Qualitative and quantitative evidence: a review of possible /supplemental-handbook-guidance methods. Journal of Health Services Research & Policy, 10(1), 45–53. Noyes, J., Popay, J., Pearson, A., Hannes, K., & Booth, A. (2008). Qualitative research and cochrane review. Chichester, United Finfgeld-Connett, D. (2010).© JonesGeneralisability & Bartlett and transferability Learning, LLC © Jones & Bartlett Learning, LLC of metasynthesis research finding. Journal of Advanced Nursing, Kingdom: John Wiley & Sons. 66(2), 246–254. NOT FOR SALE OR DISTRIBUTIONOxman, A. D. (1993). Meta-statistics:NOT FORhelp or SALEhindrance? OR ACP DISTRIBUTION Flemming, K. (2010a). The synthesis of qualitative and quantita- Journal Club, 118, A13. doi:10.7326/ACPJC-1993-118-3-A13 tive research: Its role in producing an evidence base for practice. Petticrew, M. (2003). Why certain systematic reviews reach London, United Kingdom: Royal College of Nursing. uncertain conclusions. BMJ, 326(7392), 756–758. Flemming,© Jones K. (2010b). & TheBartlett synthesis Learning, of qualitative LLC and quantita- Petticrew, M.,© & Jones Roberts, & H. Bartlett (2006). Systematic Learning, reviews LLC in the tive research: an example of critical interpretive synthesis. Journal social sciences: A practical guide. Malden, MA: Blackwell. NOT FOR SALE OR DISTRIBUTION NOT FOR SALE OR DISTRIBUTION of Advances in Nursing, 66(1), 201–217. Polit, D.F., & Beck, C.T. (2008). Translating research evidence into Gagan, M., & Hewitt-Taylor, J. (2004). The issues of nurses nursing practice: Evidence-based nursing. D.F. Polit & C.T. Beck involved in implementing evidence in practice. British Journal of (Eds.), Nursing research: Generating and assessing evidence for Nursing, 13(20), 1216–1220. nursing practice (8th ed., pp. 28–54.) London, United Kingdom: © JonesGrimshaw, & Bartlett J., McAuley, Learning, L., Bero, LLC L.A., Grilli, R., Oxman, A.D.,© JonesLippincott & Bartlett Williams & Learning, Wilkins. LLC Ramsay, C., . . . Zwarenstein, M. (2003). Systematic reviews of Popay, J., Roberts, H., Sowden, A., Petticrew, M., Arai, L., NOT FORthe SALE effectiveness OR of DISTRIBUTION quality improvement strategies and programs.NOT Rodgers, FOR SALEM., et al. OR (2006a). DISTRIBUTION Guidance on the conduct of Narra- Quality & Safety in Health Care, 12(4), 293–303. tive Synthesis in Systematic Reviews: A product from the ESRC Hemingway, P., & Brereton, N. (2009). What is systematic review? Methods Programme. Retrieved from http://www.lancs.ac.uk/shm Kings Hill, United Kingdom: Hayward Group Ltd. /research/nssr/research/dissemination/publications.php Higgins, J. P. T., & Altman,© D.Jones G. (2008). & Bartlett Assessing risk Learning, of bias LLCRing, N., Ritchie, K., Mandava, L.,© & Jones Jepson, R. & (2010). Bartlett A guide Learning, to LLC in included studies. In J.P.T. Higgins & S. Green (Eds.), Cochrane synthesising qualitative research for researchers undertaking health handbook for systematicNOT reviews FOR of interventions SALE OR (Ver. DISTRIBUTION 5.0.0). technology assessments and systematicNOT FORreviews. SALE Retrieved OR from DISTRIBUTION Retrieved from www.cochrane-handbook.org. http://www.healthcareimprovementscotland.org/our_work/technolo gies_and_medicines/programme_resources/synthesising_research.aspx Higgins, J. P. T., & Green, S. (Eds.). (2011). Cochrane Handbook for systematic reviews of interventions (Ver. 5.1.0). Retrieved Rodgers, M., Sowden, A., Petticrew, M., Arai, L., Roberts, H., from www.cochrane-handbook.org Britten, N., & Popay, J. (2009). Testing methodological guidance © Jones & Bartlett Learning, LLC on the conduct© Jonesof narrative & Bartlettsynthesis reviews: Learning, Effectiveness LLC of Higgins, J. P. T., Thompson, S. G., Deeks, J. J., & Altman, D. G. interventions to promote smoke alarm ownership and function. (2003).MeasuringNOT FOR inconsistency SALE in OR meta-analysis. DISTRIBUTION BMJ, 327, 557–560. NOT FOR SALE OR DISTRIBUTION Evaluation, 15(1), 47–71. Khan, K. S., Kunz, R., Kleijnen, J., & Antes, G. (2011). Systematic Sandelowski, M. (2008). Reading, writing and systematic review. reviews to support evidence-based medicine: How to review and Journal of Advanced Nursing, 64(1), 104–110. apply findings of health care research (2nd ed.). London, United Kingdom: Hodder Arnold. Scottish Intercollegiate Guidelines Network (SIGN). (2011). SIGN 50 A: guideline developer’s handbook. Edinburgh, United © JonesKitchenham, & Bartlett B. Learning,(2004). Procedures LLC for performing systematic© Jones & Bartlett Learning, LLC Kingdom: Author. NOT FORreviews: SALE Joint OR technical DISTRIBUTION report. Keel, United Kingdom: KeelNOT FOR SALE OR DISTRIBUTION University. Spencer, L., Ritchie, J., Lewis, J., Dillon, L. (2003). Quality in qualitative evaluation: A framework for assessing research Lo Biondo-Wood, G., & Haber, J. (2013). Nursing research: evidence. London, United Kingdom: Government Chief Social methods and critical appraisal for evidence-based practice. Researcher’s Office. (8th ed.). Philadelphia, PA: Elsevier-Mosby. © Jones & Bartlett Learning, Sutton,LLC A. J., & Higgins, J. P. (2008).© Jones Recent & developments Bartlett Learning, in LLC Millward, L., Kelly, M., & Nutbeam, D. (2001). Public health meta-analysis. Statistics in Medicine, 27, 625–650. intervention research: TheNOT evidence FOR. London, SALE United OR Kingdom: DISTRIBUTION NOT FOR SALE OR DISTRIBUTION Health Development Agency. Thomas, J., & Harden, A. (2008). Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Moher, D., Liberati, A., Tetzlaff, J., & Altman, D. G. (2009). Pre- Medical Research Methodology, 8(45), 1–10. ferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. Open Medicine, 3(3), 123–130. © Jones & Bartlett Learning, LLC © Jones & Bartlett Learning, LLC National Collaborating Centre for Methods and Tools. (2008). SUGGESTED FURTHER READING QualityNOT assessment FOR tool SALE for quantitative OR DISTRIBUTION studies. Hamilton, ON: NOT FOR SALE OR DISTRIBUTION McMaster University. (Updated 30 August 2017). Retrieved from For additional detailed information, readers may wish to http://www.nccmt,ca/knowkedge-repositories/search/14 explore other useful sources related to the references listed National Institute for Health and Clinical Excellence (NICE). here. The following may also prove to be useful texts for (2006). Methods for development of NICE public health guid- further reading and additional resources. © Jones & Bartlett Learning, LLC © Jones & Bartlett Learning, LLC ance. London, United Kingdom: Author. Altman, D. (2013). Systematic reviews: Key principles of their NOT FORNoyes, SALE J., Booth OR A., DISTRIBUTION Hannes, K., Harden, A., Harris, J., Lewin, S.,NOT development FOR SALE and reporting.OR DISTRIBUTION The Equator Network Workshop. & Lockwood, C. (Eds.). (2011). Supplementary guidance Geneva, Switzerland: World Health Organization.

9781284114942_CH03.indd 68 06/11/17 2:24 pm CHAPTER 3 Systematic Reviews: Consolidating Research Evidence for EBP 69

© Jones & Bartlett Learning, LLC © Jones Doody,& Bartlett C. M., Learning,& Doody, O. LLC(2011).Introducing evidence into framework for analysing public involvement in health services NOT FORnursing SALE practice: OR DISTRIBUTIONusing the IOWA model. British Journal of NursNOT- research.FOR SALE Health Expectations,OR DISTRIBUTION 11(1), 72–84. ing, 20(11), 661–664. Paterson, B.L., Thorne, S.E., Canam, C., & Jillings, C. (2001). Edwards, M., Davies, M., & Edwards, A. (2009). What are Meta-study of qualitative health research: A practical guide to the external influences on information exchange and shared meta-analysis and meta-synthesis. Thousand Oaks, CA: Sage. decision-making in healthcare consultations: A meta-synthesis of © Jones & Bartlett Learning, LLCPopay, J., Roberts, H., Snowden,© Jones A., Petticrew, & Bartlett M., Arai, Learning, L., LLC the literature. Patient Education and Counseling, 75(1), 37–52. Rodgers, M., et al. (2006b). Guidance on the conduct of narrative Flemming, K. (2007).TheNOT knowledge FOR SALEbase for ORevidence DISTRIBUTION based synthesis in systematic reviews:NOT Final FORReport. SALE Swindon, OR United DISTRIBUTION nursing: a role for mixed methods research. Advances in Nursing Kingdom: ESRC Methods Programme. Science, 30(1), 41–51. Pope, C., Mays, N., & Popay, J. (2007). Synthesising qualitative Garside, R. (2014). Should we appraise the quality of qualitative and quantitative health evidence: A guide to methods. Maidenhead, research reports for systematic reviews, and if so how? Innovation: United Kingdom: Open University Press; McGraw-Hill. © Jones & Bartlett Learning, LLC © Jones & Bartlett Learning, LLC The European Journal of Social Science Research, 27(1), 67–79. Rees, C. (2010). Understanding evidence and its utilisation in Gough,NOT D. (2007). FOR WeightSALE of OR evidence: DISTRIBUTION A framework for the nursing practice.NOT In FOR K. Holland SALE (Ed.), OR Nursing: DISTRIBUTION Evidence-based appraisal of the quality and relevance of evidence. Research Papers practice skills. Oxford, United Kingdom: Oxford University Press. in Education, 22(2), 213–228. Robson, C. (2011). Real world research (4th ed.). Chichester, Gough, D. A., Oliver, S., & Thomas, J. (2012). An introduction United Kingdom: John Wiley and Sons Ltd. © Jones &to systematicBartlett reviews. Learning, London, LLC United Kingdom: Sage. © JonesRodgers, & M.,Bartlett Arai, L., Learning,Britten, N., Petticrew, LLC M., Popay, J., Roberts, Hannes, K. (2011). Chapter 4; Critical appraisal of qualitative H., & Sowden, A. (2006). Narrative synthesis in systematic reviews: NOT FORresearch. SALE In: OR Noyes, DISTRIBUTION J., Booth, A., Hannes, K., Harden, A., Harris,NOT AnFOR ESRC SALE research OR methods DISTRIBUTION programmes project. Retrieved J., Lewin, S., Lockwood, C. (Editors), Supplementary Guidance for from http://www.lancaster.ac.uk/shm/research/nssr/research Inclusion of Qualitative Research in Cochrane Systematic Reviews /dissemination/publications.php of Interventions. Version 1 (Updated August 2011). Cochrane Roen, K., Arai, L., Roberts, H., & Popay, J. (2006). Extending Collaboration Qualitative Methods Group, 2011. Retrieved from: systematic reviews to include evidence on implementation: Meth- http://methods.cochrane.org/qi/supplemental-handbook-guidance© Jones & Bartlett Learning, LLCodological work on a review of© communityJones & based Bartlett initiative Learning, to LLC Harris, R.P., Helfand, M.,NOT Woolf, FOR S.H., Lohr, SALE K.N., OR Mulrow, DISTRIBUTION C.D., prevent injuries. Social ScienceNOT & Medicine, FOR 63SALE(4), 1060–1071. OR DISTRIBUTION Teutsch, S.M., . . . Third US Preventive Services Task Force (2001). Sackett, D.L., Richardson, W.S., Rosenberg, W., & Haynes, R.B. (2001). Current methods of the US Preventive Services Task (1997). Evidence-based medicine: How to practice and teach Force: A review of the process. American Journal of Preventive EBM (1st ed.). London, United Kingdom: Churchill Livingstone. Medicine, 20(Suppl. 3), 21–35. Sackett, D. L., Straus, S. E., Richardson, W. S., Rosenberg, W., & Jesson,© J., Jones Matheson, & L., Bartlett Lacey, F. M. Learning, (2011). Doing LLC your literature Haynes, R. B.© (2000). Jones Evidence-based & Bartlett medicine: Learning, How to LLC practice review:NOT traditional FOR andSALE systematic OR DISTRIBUTIONtechniques. London, United and teach EBMNOT (2nd FOR ed.). Edinburgh, SALE OR United DISTRIBUTION Kingdom: Churchill Kingdom: Sage. Livingstone. Johnston, L. (2005). Critically appraising quantitative evidence. Sandelowski, M., & Barroso, J. (2007). Handbook for synthesiz- In B.M. Melnyk & E. Fineout Overholt (Eds.), Evidence-based ing qualitative research. New York, NY: Springer. practice in nursing and healthcare: A guide to best practice. Sandelowski, M., Borroso, J., & Voils, C. I. (2007). Using qualitative © Jones &Philadelphia, Bartlett PA: Learning, Lippincott WilliamsLLC & Wilkins. © Jonesmeta-summary & Bartlett to synthesise Learning, qualitative LLC and quantitative descrip- NOT FORKolb, SALE S. M. OR(2012). DISTRIBUTION Grounded theory and the constant comparativeNOT tiveFOR findings. SALE Research OR DISTRIBUTIONin Nursing & Health, 30(1), 99–111. method: Valid research strategies for educators. Journal of Emerging Uman, L. S. (2011). Systematic reviews and meta-analyses. J Can Trends in Educational Research and Policy Studies, 3(1), 83–86. Acad Child Adolesc Psychiatr. 20(1): 57-59. Journal of the Cana- Liberati, A., Altman, D.G., Tetzlaff, J., Mulrow, C., Gotzsche, P.C., dian Academy of Child and Adolescent Psychiatry, 20(1), 57–59. Ioannidis, J.P.A., . . . Moher, D. (2009). The PRISMA statement for Webb, C., & Roe, B. H. (2007). Reviewing research evidence for nurs- reporting systematic reviews© andJones meta-analyses & Bartlett of studies Learning, that evalu- LLCing practice: systematic reviews. Oxford,© Jones United & Kingdom: Bartlett Blackwell. Learning, LLC ate health care interventions: Explanation and elaboration. PLoS NOT FOR SALE OR DISTRIBUTIONWeed, M. (2005). Meta-interpretation:NOT FOR A SALE method OR for DISTRIBUTIONthe Medicine, 6(7), e1000100. doi:10.1371/ Journal.pmed.1000100 interpretive synthesis of qualitative research. Qualitative Social McCormack, B., Wright, J., Dewar, B., Harvey, G., & Ballantine, Research, 6(1), 1–17. K. (2007). A realist synthesis of evidence relating to practice Weed, M. A. (2007). A potential method for interpretive synthesis development. Practice Development in Health Care, 6(1), 5–24. of qualitative research. International Journal of Social Research Melnyk,© B.Jones M., & Fineout-Overholt, & Bartlett Learning, E. (Eds.). (2014). LLC Evidence-based Methodology,© 11Jones(1), 13–28. & Bartlett Learning, LLC practice in nursing and healthcare: A guide to best practice. NOT FOR SALE OR DISTRIBUTION Wong, G., Greenhalgh,NOT FOR T., SALE& Pawson, OR R. DISTRIBUTION(2010). Internet-based Philadelphia, PA: Lippincott Williams & Wilkins. medical education: A realist review of what works, for whom Noblit, G. W., & Hare, R. D. (1988). Meta-ethnography: Synthe- and in what circumstances. BMC Medical Education, 10(12). sizing qualitative studies. Newbury Park, CA: Sage. Wright, R. W., Brand, R. A., Dunn, W., & Spindler, K. P. (2007). Oliver, S., Rees, R., Clarke-Jones, L., Milne, R., Oakley, A., How to write a systematic review. Clinical Orthopaedics and © Jones &Gabbay, Bartlett J., . . . Learning,Gyte, G. (2008). LLC A multidimensional conceptual© JonesRelated & Research, Bartlett 455, Learning, 23–29. LLC NOT FOR SALE OR DISTRIBUTION NOT FOR SALE OR DISTRIBUTION

9781284114942_CH03.indd 69 06/11/17 2:24 pm © Jones & Bartlett Learning, LLC © Jones & Bartlett Learning, LLC NOT FOR SALE OR DISTRIBUTION NOT FOR SALE OR DISTRIBUTION

9781284114942_CH03.indd 70 06/11/17 2:24 pm