<<

J Am Board Fam Med: first published as 10.3122/jabfm.2011.05.110067 on 7 September 2011. Downloaded from

SPECIAL COMMUNICATION Quasi-Experimental Designs in Practice-based Research Settings: Design and Implementation Considerations

Margaret A. Handley PhD, MPH, Dean Schillinger, MD, and Stephen Shiboski, PhD

Background: Although randomized controlled trials are often a for determining interven- tion effects, in the area of practice-based research (PBR), there are many situations in which individual is not possible. Alternative approaches to evaluating interventions have received in- creased attention, particularly those that can retain elements of randomization such that they can be considered “controlled” trials. Methods: Methodological design elements and practical implementation considerations for two quasi- experimental design approaches that have considerable promise in PBR settings – the stepped-wedge design, and a variant of this design, a wait-list cross-over design, are presented along with a from a recent PBR intervention for patients with diabetes. Results: PBR-relevant design features include: creation of a cohort over time that collects control but allows all participants (clusters or patients) to receive the intervention; staggered introduction of clusters; multiple points; and one-way cross-over into the intervention arm. Practical considerations include: randomization versus stratification, training run in phases; and extended time period for overall study completion.

Conclusion: Several design features of practice based research studies can be adapted to local cir- copyright. cumstances yet retain elements to improve methodological rigor. Studies that utilize these methods, such as the stepped-wedge design and the wait-list cross-over design, can increase the evidence base for controlled studies conducted within the complex environment of PBR. (J Am Board Fam Med 2011;24: 589–596.)

Keywords: Clinical Trials, Implementation Science, Practice-based Research, Quasi-Experimental Design

Although randomized, controlled trials (RCTs) are situations where individual randomization is not http://www.jabfm.org/ often a gold standard for determining intervention possible.1 For example, an RCT may not be con- effects, in the area of practice-based research, qual- sidered possible for any of the following reasons: ity improvement, and public health, there are many (1) there is a commonly held view among stake- holders that evidence is sufficient in some settings of an established intervention benefit, and it would This article was externally peer reviewed.

be unethical to have control groups; (2) the inter- on 26 September 2021 by guest. Protected Submitted 25 February 2011; revised 16 June 2011; ac- cepted 30 June 2011. vention delivery is underway already (e.g., a policy From the University of San Francisco, Depart- change sets in motion new clinical procedures); or ment of and (MAH, SS) and Department of Medicine, Division of General Internal (3) assignment to a control group is unacceptable to Medicine, San Francisco General Hospital Center for Vul- some groups that would potentially be controls. nerable Populations (MAH, DS). Funding: This work was funded by AHRQ grant R18 HS Consequently, alternative approaches to evaluating 017261, Harnessing Health for clinical and community interventions have received Self-Management Support and Activation in a Medicaid Health Plan, McKesson Foundation. increased attention, particularly those that can re- Conflict of interest: none. tain some elements of randomization such that they Corresponding author: Margaret A. Handley, PhD MPH, 2–6 Division of General Internal Medicine, San Francisco can be considered “controlled” trials. Such de- General Hospital Center for Vulnerable Populations, 995 signs are consistent with discussions of “practical Potrero Avenue, Ward 83, San Francisco, CA 94110 (E- 7 mail: [email protected]). clinical trials” in that these designs adapt to local doi: 10.3122/jabfm.2011.05.110067 Quasi-Experimental Designs in Practice-based Research 589 J Am Board Fam Med: first published as 10.3122/jabfm.2011.05.110067 on 7 September 2011. Downloaded from considerations, such as those raised above, and studies), because it is possible to control the roll- therefore are likely to achieve better outcomes. In out and to include elements of randomization that light of the recognition that evidence-based prac- can reduce biases. However, the best design for a tice must be informed by practice-based evidence,8 particular intervention must be determined within practice-based research (PBR), in particular may the local context, through determining what types benefit from approaches that do not require ran- of evidence exist to support other designs and what domization of individuals in settings where it may are the relative merits of alternatives to RCT in not be feasible. Such alternative designs, often different contexts.1 called “quasi-experimental” designs, are increas- ingly used for the of clinical and/or prac- Methods tice-based interventions applied under real world We present a review of design features and practi- circumstances where individual-level RCT designs cal considerations for PBR implementation for the are not suitable. stepped-wedge and wait-list design, along with a This paper reviews design elements and practi- discussion of published examples from studies of cal implementation considerations for two quasi- clinic-based interventions using these designs. A experimental designs that have considerable prom- PubMed search was conducted using the terms ise in PBR settings: the stepped-wedge design, and quasi-experimental, stepped-wedge, quality improve- a variant of this design, described herein as a wait- ment, and wait-list design to identify methodological list cross-over design, currently used in a large as well as clinic-based publications that describe PBRN project. The stepped-wedge design’s rele- these design approaches. The examples described vance to PBR lies in: (1) it is a cluster-based design in this paper are derived from the literature review suitable to clinic-level interventions that enables all and also the author’s experiences in PBR work sites to receive the intervention, yet (2) it avoids conducted within the San Francisco Bay Collabor- some of the methodological pit-falls associated

ative Research Network, a PBRN located in the SF copyright. with before and after designs, as it retains con- Bay Area and affiliated with the University of Cal- trolled data elements.5 The stepped-wedge and ifornia San Francisco, with which the authors are wait-list variant allow all patients to receive the long-time members and research collaborators. intervention while also contributing control time. This differs from a traditional cluster randomized trial done in parallel time, in which some sites are Results randomized to control at the outset and do not have Stepped-Wedge Designs an opportunity to cross over into an intervention Stepped-wedge designs have been in use for several arm. decades, often in the context of settings in which http://www.jabfm.org/ A variant of the stepped wedge, the wait-list there are strong objections to individual level ran- cross-over trial, is also particularly well suited to domization.1 A 2006 review of stepped-wedge de- PBR and quality improvement studies involving signs indicated this design was frequently used in clinic or systems-based registries, as it en- developing countries, often in the context of HIV ables staggered implementation of patient-level in- treatment interventions.2 More recently, the liter- terventions over time. Staggered implementation is ature indicates stepped-wedge designs are increas- highly relevant to counseling-based programs and ingly being implemented in clinic-based settings on 26 September 2021 by guest. Protected those requiring considerable staff training, such as world-wide, for example, in a trial of guideline some guideline-based interventions, as the design implementation for therapeutic hypothermia in can alleviate staffing burdens (and costs) that would post-cardiac patients in a network of Emergency be encumbered without staggered implementation. Departments in Canada10 and in evaluating imple- The staggered implementation creates a waiting mentation of a clinician-based psychosocial inter- period for some patients, and this wait time pro- vention for improving treatment of patients with vides useful “control” information for inclusion in in Australia.11 Stepped-wedge designs the data analysis. are a cluster-randomized type of cross-over design These designs have stronger methodological in which clusters are all initially assigned to the rigor in comparison with other well-known quasi- control group, then switch to the intervention experimental design options (such as pre and post group at randomly assigned time points. All clus-

590 JABFM September–October 2011 Vol. 24 No. 5 http://www.jabfm.org J Am Board Fam Med: first published as 10.3122/jabfm.2011.05.110067 on 7 September 2011. Downloaded from

Figure 1. Stepped-wedge design. Adapted from Brown that control data are collected multiple times for and Lilford, 2006. some units. In Figure 1, cluster 5 contributes data at five time points before receiving the interven- tion. (4) Every site gets the intervention eventually and every site contributes control data across one or more time periods (with the exception of the first cluster). (5) Clusters cross over from control to interven- tion (one-way cross-over). (6) Ability to control for time trends by allowing contemporaneous comparisons across clusters, at different time periods. Figure 2 provides a comparison of the cluster assignments over time when using: (a) a parallel ters receive the intervention by the last time inter- time allocation as in a traditional cluster random- val.3 The stepped-wedge design derives its name ized trial in which control clusters do not receive from the shape of the staggered roll out over time the intervention within the study observation pe- periods of intervention units (clinics or other units riod and there is only one time period allowed; (b) for clustering such as schools), which resembles a a cross-over trial in which clusters can cross over stepped or stacked blocks (see shaded blocks or from intervention to control (two time periods al- steps in Figure 1). The stepped wedge design is lowed); and (c) a stepped wedge, in which clusters particularly useful for evaluating the population each contribute multiple data points to the cohort impact (or effectiveness) of an intervention that was and are staggered in the order they receive the previously found efficacious in an individually ran- intervention, but only cross over into intervention copyright. domized trial.3 Because stepped-wedge designs in- from control (multiple time periods allowed). clude a cross-over component, data analysis options are flexible and include between and within-cluster Practical Implementation Considerations for comparisons as well as temporal variations in inter- Stepped-Wedge Designs in PBR vention effects.2–4 Randomization of Clusters If it is possible, of start times Design Features of a Stepped Wedge for clusters is preferred. However, there are often (1) Creation of a cohort comprised of control data circumstances for which this is not feasible or suit- http://www.jabfm.org/ and intervention data with contributions from each able and stratified or matched approaches may be site to both cohorts, such that two samples can be used to reduce the biases associated with non-ran- compared. In Figure 1, this would be that outcomes dom assignments (see below). Several recent exam- in the white units are compared with those in the ples have used stratified approaches to account for: shaded units, and both cohorts are created across time periods (as compared with a before and after

Figure 2. Comparison of treatment schedules for on 26 September 2021 by guest. Protected study that doesn’t enable adjustments over time). intervention within parallel time, cross-over, and (2) Staggered introduction of intervention (clus- stepped-wedge patterns. Reproduced from ters) over time. Following an initial baseline data Contemporary Clinical Trials, Vol. 28, Michael A. collection period in which all clusters contribute Hussey and James P. Hughes, Design and Analysis of control data, the order in which clusters receive the Stepped Wedge Cluster Randomized Trials, pp.182– intervention may be determined at random, or may 191, 2007, with permission from Elsevier. involve accounting for practical considerations (such as size of clusters), which may necessitate Parallel Crossover Stepped Wedge Time Time Time matched of stratified approaches to assigning inter- 1 12 12345 vention timing. Cluster 1 1 Cluster 1 1 0 Cluster 1 0 1 1 1 1 21 210 200111 (3) Multiple data collection points. As described 30 301 300011 in (1), each block of data requires data collection, so 40 401 400011 doi: 10.3122/jabfm.2011.05.110067 Quasi-Experimental Designs in Practice-based Research 591 J Am Board Fam Med: first published as 10.3122/jabfm.2011.05.110067 on 7 September 2011. Downloaded from

Figure 3. Stepped-wedge design integrating training component prior to intervention period.

(1) The varying size of clusters, such that not all instead were allocated within strata based on vol- the largest clusters (e.g., hospitals or community ume of patients. The authors used 4 strata levels of clinics with large patient volume) would be ran- numbers of patients per site (1 ϭ least patients, 4 ϭ domized to intervention at the start of the study, most patients) using 1, 2, 3, 4, then 4, 3, 2, 1 and not all the small ones randomized at the end.13 roll-out. This enabled a slower introduction of the (2) The intervention was already underway in training in use of the ARV and clinic some sites but not others (e.g., guideline imple- procedures, while enabling some controlling for mentation in circumstances where some hospital the non-random allocation using stratified analysis sites beginning to implement a new , while techniques. others have not yet begun). (3) Logistic implementation factors such as geo- graphical distance from training or referral re- Extended Observation Period Associated With Stepped- sources or seasonal variations that are related to the Wedge Designs 13 copyright. outcome may also limit the feasibility of a com- Stepped-wedge designs usually take longer than pletely randomized allocation of clusters. traditional RCTs or traditional cluster randomized trials. Therefore, it is important that funding and Training or Phasing-In of Intervention-Related staffing are secured to allow for the extended study Components length. Additionally, community clinic partners Because it is often the case that PBR interventions need to be aware before agreeing to participate that require clinician or staff training components that the results may take longer to obtain and the im- must take place before the intervention start date pact on the clinic will last longer. With an extended (such as for adopting a new clinic guideline or other time period, several concerns may arise. Most http://www.jabfm.org/ practice change initiative), a stepped-wedge design prominently, there are concerns about bias, that if can build in a training-related “run-in” phase that sites serving for longer in control periods have to takes place after the control time has been com- provide repeated data then it is possible that the pleted and before the intervention period begins, as in Figure 3. In this way, each of the participating quality of the data may change over time. As well, clinical sites can be allocated in random order from bias could arise from differential drop-out rates due control condition to training then delivery of the to delays in receiving the intervention in clinics that on 26 September 2021 by guest. Protected intervention.12 This training period cannot con- have waited longer. In the Killan example above, tribute data to the cohorts, and has to be excluded electronic medical records data were used, so that in subsequent analyses, which must be taken into additional patient interviews were not necessary account when developing sample size estimates. An reducing potential biases associated with multiple example of a completed clinic-based study that used data collection that is not part of routine care. this stepped-wedge approach with a phased in in- While the possibility of differential drop out rates tervention examined the impact of integration of is real within longer studies of this kind, to our HIV testing and anti-retroviral (ARV) knowledge, there have not been any published as- within prenatal care services in 8 primary care prac- sessments examining these rates within time peri- tices in Zambia on rates of ARV therapy before ods. Instead, time periods are presented as adjust- delivery.12 Sites were not randomly allocated but ments in the data analysis of the clustered data.

592 JABFM September–October 2011 Vol. 24 No. 5 http://www.jabfm.org J Am Board Fam Med: first published as 10.3122/jabfm.2011.05.110067 on 7 September 2011. Downloaded from

Repeated Observations Before Intervention Exposure design that would enable them to study a variety of With the stepped-wedge design, clusters (or in the outcomes. The SFHP thought that a RCT with a case of the wait-list design, individual participants) traditional parallel assigned control group would not yet crossed over to intervention are evaluated not be ethical because our recent studies had shown multiple times, which could introduce a bias. In the the effectiveness of the ATSM Program (the mod- case in which clinics are repeatedly evaluated, the ified ATSM that was implemented was called same individuals are rarely included in each repeat SMART STEPS) within the same clinic population assessment, and in many cases, the assessments are covered by the health plan. The SFHP decided to conducted using existing data, such as medical re- provide SMART STEPS as a covered member cords, rather than based on interviews. However, benefit, with wait-list patients receiving the inter- interview data may be used as in the case study vention after 6 months. The University of Califor- described below, and multiple observations per nia San Francisco research team was asked to assist person will be included. with the evaluation strategy that involved patient- level consent and randomization procedures, for which we received an Agency for HealthCare Re- Wait-List Design Variant of Stepped Wedge search and Quality (AHRQ) PBRN R18 grant.22 A stepped-wedge wait-list design is a version of the As is the case for many regional health plans such cluster based stepped-wedge approach described as SFHP and clinic systems or PBRNs, it was not above in which individuals (rather than clusters like feasible to scale-up the ATSM intervention across all clinics or community centers) are randomized from clinics and to all eligible patients at once, without wait list to intervention over a series of time stages. incurring huge costs for staffing. This is because This design variant is well adapted to contexts in SMART STEPS involves a care manager/counselor which there is a large registry of patients who are to call patients with some to solve diabetes eligible for participation in an intervention (such as self management problems reported by patients or disease-specific registries or health plan member- copyright. that were identified through review of electronic data, ship rosters), when an RCT is not possible, and such as not picking up or laboratory when it is not feasible to assign all patients to values, such as elevated hemoglobin A1c. Conse- receive the intervention at the same time. quently, rolling out the intervention in a staggered fashion, but controlling the roll-out through random- Case Study: SMART STEPS Project izing a sizable proportion of the patients to wait-list at In our experience in PBR within the SF Bay Col- each time interval, would create a cohort that has laborative Research Network, a wait-list variant of retained design elements of randomization, but that the stepped-wedge design was selected to evaluate would be practical for staffing purposes (see Figure 4). http://www.jabfm.org/ the implementation of a diabetes self-management The staggered intervention implementation meant support program, in which a regional Medi-Caid the health plan could hire only one health coach for managed care plan that maintains a registry of di- the majority of the intervention period, with some abetes patients has enrolled active members with additional staffing needed at certain time intervals. diabetes into a self-management support diabetes The evaluation and data collection involve both intervention across 4 PBRN clinic sites.14 The de- individual interviews and electronic medical re- velopment of the project started when the San cord-derived outcomes, such as laboratory values, on 26 September 2021 by guest. Protected Francisco Health Plan (SFHP), approached the au- medication uses, and primary care and hospital vis- thors (DS and MH), for help in adapting a health its. The SMART STEPS Project is nearing com- IT diabetes self-management support program, the pletion with over 350 patients enrolled. Automated Telephone Self Management Support Represented in the figure are 130 patients from (ATSM), which we had previously developed, the 4 clinics who were included in the diabetes tested and found efficacious in a RCT within our registry who were randomized to intervention and PBRN.15–21 The intervention was to become a cov- 130 who were randomized to wait list, over the ered benefit to their patients for a trial period, for entire study period. Each 6-month enrollment which the evaluation was conducted (2009 to 2011) phase (the boxes identified as waves) in this project with the goals of delivering the intervention to has patients going both directly into an interven- several hundred patients using an implementation tion arm (INT) or going into wait list for 6 months doi: 10.3122/jabfm.2011.05.110067 Quasi-Experimental Designs in Practice-based Research 593 J Am Board Fam Med: first published as 10.3122/jabfm.2011.05.110067 on 7 September 2011. Downloaded from

Figure 4. Wait-list design application for SMART Steps diabetes intervention.

(WL). Each wave of the wait-list patients is then be eligible, or there could be new diabetes patients crossed over into intervention after 6 months (WL- who would become eligible and need to be added to INT). The dots represent the cross-over for indi- the eligible patient pool. It was necessary to review viduals on wait list into the active SMART STEPS the diabetes registry data on a monthly basis and intervention arm (WL-INT). This resulted in sev- remove some participants from the wait list and eral waves of control data as well as intervention intervention arm, when they lost their health plan data that take into account possible variations over membership, or became ineligible for other rea- time that may affect the study results in the out- sons. Although this flux was small and did not affect comes analyses. overall study sample size considerably in this proj- copyright. To conduct the wait-list evaluation, data collec- ect, it did require active registry surveillance and tion in the form of interviews was done multiple the development of study criteria to determine if times. For example, participants in both arms re- patients who became ineligible had participated for ceived interview 1 at baseline, before getting “acti- enough time to qualify as ‘exposed’ to the interven- vated” to begin SMART STEPS or begin the con- tion or to the wait list. trol wait-list 6-month period. Wait-list patients again received interview 1 just before crossing over Differential Attrition From Wait-List Groups into intervention. After completing the interven- We did not find that patients who had been on the http://www.jabfm.org/ tion, all patients receive the follow up interview. wait list were less likely to participate in the inter- vention once they were crossed over, but there is a Practical Implementation Considerations concern that the duration of the wait list could There are additional challenges in implementing affect subsequent participation. We will be con- the wait-list design in addition to those described ducting a variety of fidelity assessments to examine above, but they reflect many of the realities of PBR, potential differential engagement across waves, and there are often strategies to overcome them. clinics, and study arms to examine this possibility in on 26 September 2021 by guest. Protected more detail. Changing Eligibility Based on Registry and Health Plan Membership Criteria Statistical Analysis Considerations One challenge pertains to any study that uses an Stepped-wedge designs can be analyzed using a active enrolment strategy, for example from health variety of techniques for longitudinal outcomes plan membership, such that patients’ eligibility can that allow for clustering and covariate effects.3–5 change over time, or from a disease-based registry Most statistical software programs such as SAS or that must be updated to include newly diagnosed STATA can accommodate data analysis strategies patients. For example, in this cohort, participants relevant to stepped-wedge design, such as mixed could switch clinics and become ineligible, they effects or generalized estimating equation regres- could lose health plan membership and no longer sion models. A key point is that while there are

594 JABFM September–October 2011 Vol. 24 No. 5 http://www.jabfm.org J Am Board Fam Med: first published as 10.3122/jabfm.2011.05.110067 on 7 September 2011. Downloaded from efficiencies in study power associated with cross- ity in follow-up outreach, and increases in sample over designs such as the stepped-wedge,3 there are sizes to account for losses. also costs associated with its use that must be eval- uated before the design is used. The role of con- Sample Size Planning for Stepped-Wedge Designs founding, the impact of extended follow-up time, Sample size estimation and power calculations for and the impact of clustering in the data analysis stepped-wedge designs require specification of the strategy are each discussed below, followed by a number of clusters, number of time steps, number summary of sample size estimation for stepped- of participants per cluster per step, the desired wedge designs. Detailed treatments of relevant sta- , and the expected variability of responses tistical issues are provided by Hussey and Hughes,3 at both the individual and cluster level. The vari- Cousens et al,4 and Li and Frangakis.6 ances of cluster-level responses are often expressed as a function of a “ inflation factor” reflect- Examining in Stepped-Wedge Designs ing the impact of the intra-class correlation be- The estimate of overall intervention effect from a tween individual responses within a given cluster. If stepped-wedge design is based on a comparison of we denote this quantity by ␳, and by N, the number average responses between treatment and interven- of individuals in each cluster, the following expres- tion groups. The model for cluster-level responses sion gives the approximate cluster-level variance in typically includes cluster-specific random compo- the case in which responses do not vary with time: nents and also allows for separation of intervention and time effects. The latter represent an important ͑␶2 ϩ ␴2͒ e ͓1 ϩ ͑N ϩ 1͒␳] potential source of confounding due to the partially N randomized nature of this design, and must be accounted for in data analyses. Further the com- The expression before the brackets represents the mon assumption of no between inter-

variability in the case of independent individual re- copyright. vention effects and time needs to be evaluated in sponses, and the bracketed quantity is the variance analyses.3 If there are no significant time trends inflation factor. This makes it clear that the sample detected, then analyses can frequently be simplified size required for stepped-wedge trials will increase to a paired t test comparison of responses between with both the cluster size and the intraclass correla- groups. Other potential confounders need to be tion between individual level responses. Sample size accounted for in adjusted regression modeling.4 estimates typically assume constant treatment effects. Individual responses can be analyzed using gener- alizations of these methods, including hierarchical models investigating group and individual-level ex- Conclusion http://www.jabfm.org/ planatory covariates. This paper presents a summary of key methodolog- ical and implementation elements of two quasi- Implications of Extended Follow-Up Periods in experimental designs that have particular relevance Stepped-Wedge Designs for practice-based research. Additional studies that Sample size planning for stepped-wedge studies utilize these methods and offer variants that adapt should also take into account the extended length to important local considerations can increase the of follow-up that may be needed to roll out each of evidence base for controlled studies conducted with on 26 September 2021 by guest. Protected the steps. When it is possible to include several in the complex environment of PBR. steps and decrease the time intervals for each step, then study power is increased. This extended time The authors acknowledge the Agency for Health Care Research for the overall study observation period may reduce and Quality funding (R18 HS 017261, Harnessing Health In- overall retention, result in and reduce formation Technology for Self-Management Support, and Medication Activation in a Medicaid Health Plan). overall power from the associated decreases in sam- ple size. Although this is a risk in any cohort design, References those that prolong the observation period as with 1. Bonell CP, Hargreaves J, Cousins et al. Alternatives the stepped wedge, may be more likely to experi- to randomization in the evaluation of public health ence participant attrition unless steps are taken to interventions: design challenges and solutions. J Epi encourage sustained participation, such as continu- Comm Health 2009. doi: 10.3122/jabfm.2011.05.110067 Quasi-Experimental Designs in Practice-based Research 595 J Am Board Fam Med: first published as 10.3122/jabfm.2011.05.110067 on 7 September 2011. Downloaded from

2. Brown CA, Lilford RJ. The stepped wedge trial use therapeutic food with standard therapy in the design: a . BMC Med Res Meth- treatment of malnourished Malawian children: a odol 2006;6:54. controlled, clinical effectiveness trial. Am J Clin 3. Hussey M, Hughes JP. Design and analysis of Nutr 2005;81:864–70. stepped wedge cluster randomized designs. Contemp 14. Ratanawongsa, N, Handley M, Quan J, et al. Scaling Clin Trials 2007;182–91. up a health information technology intervention for 4. Cousens S, Hargreaves J, Bonell C, et al. Alternatives patients with diabetes in a Medi-Cal managed care to randomisation in the evaluation of public-health plan: the SMART Steps Program. HRQ Health IT interventions: statistical analysis and causal infer- Grantee Conference. Washington, DC: 2010. ence. J Epidemiol Community Health July 13, 2010 15. ATSM Fact Sheet. Available at: http://www.cvp-sf. [Epub ahead of print]. com/_media/ideall-fact-sheet.pdf. 5. Fan E, Laupacis A, Pronovost P, et al. How to use an 16. Schillinger D, Hammer H, Wang F, et al. Seeing in article about quality improvement. JAMA 2010;304: 3-D: examining the reach of diabetes self-manage- 2279–87. ment support strategies in a public healthcare sys- 6. Li F, Frangakis CE. Designs in partially controlled tem. Health Educ Behav 2008;35:664–82. studies: messages from a review. Stat Methods Med 17. Schillinger D, Handley MA, Hammer H, et al. Ef- Res 2005;14:417–31. fects of self-management support on structure, pro- 7. Tunis SR, Stryer DB, Clancy CM. Practical clinical cess, and outcomes among vulnerable patients with trials: increasing the value of for diabetes: a three-arm practical clinical trial. Diabetes decision making in clinical and health policy. JAMA Care. 2009;32:559–66. 2003;290:1624–32. 18. Handley MA, Hammer H, Schillinger D, Navigating 8. Green LW. Making research relevant: if it is an the terrain between research and practice: a Collabor- evidence-based practice: where’s the practice-based ative Research Network (CRN) case study in diabetes evidence? Fam Pract 2008;25(Suppl 1):i20–4. research. J Am Board Fam Med 2006;19:85–92. 9. Available at: http://developmentalmedicine.ucsf.edu/ 19. Handley M, Shumway M, Schillinger, D. Cost-effec- research/research_programs/crn/crn.aspx. tiveness of an automated telephone self management 10. Dainty K, Scals D, Brooks S, et al. A knowledge support intervention with nurse care management

collaborative to improve use of therapeu- among vulnerable patients with type-2 diabetes. Ann copyright. tic hypothermia in post-cardiac arrest patients: pro- Fam Med 2008;6:512–8. tocol for a stepped wedge randomized trial. Imple- 20. Sarkar U, Handley MA, Gupta R, et al. Use of an ment Sci 2011. interactive, telephone-based self-management sup- 11. Turner J, Kelly B, Clarke D, et al. A randomised trial port program to identify adverse events among am- of a psychosocial intervention for cancer patients inte- bulatory diabetes patients. J Gen Intern Med 2008; grated into routine care: the PROMPT study (Promot- 23:459–65. ing Optimal Outcomes in Mood through tailored Psy- 21. Agency for Healthcare Research and Quality. Auto- chosocial Therapies). BMC Cancer 2011;11:48. mated, telephone-based interactive, language-appro- 12. Killam WP, Tambatamba BC, Chintu N, et al. priate monitoring engages and improves health be-

Antiretroviral therapy in antenatal care to increase haviors of low-income diabetes patients. Innovations http://www.jabfm.org/ treatment initiation in HIV-infected pregnant Exchange 2010. women: a stepped-wedge evaluation. AIDS 2010; 22. Schillinger D. R18 HS 017261, Harnessing Health 24:85–91. Information Technology for Self-Management Sup- 13. Ciliberto MA, Sandige H, MacDonald LN, et al. port and Medication Activation in a Medicaid Health Comparison of home-based therapy with ready-to- Plan. Awarded 2007. on 26 September 2021 by guest. Protected

596 JABFM September–October 2011 Vol. 24 No. 5 http://www.jabfm.org