Background: Depression Is the Second Most Common Cause of Disability World-Wide

Improving Care for Depression in VA:

Regional Expansion of the TIDES Initiative (ReTIDES) Summary

Lisa Rubenstein, MD and Edmund Chaney, PhD

7/24/05

Background: A team of VA investigators, under the overall leadership of the

VA Mental Health Quality Enhancement and Research Initiative (QUERI) in Little Rock, is applying the findings from the prior VA and non-VA effectiveness1-12, cost effectiveness 13-21 , and quality improvement studies22-25 for depression to promote VA system improvement. These findings show, based on a series of randomized trials, that a new care model (collaborative care for depression) is both necessary and sufficient for improving depression care and outcomes, including symptom reduction, recovery, reduced job loss, and satisfaction with care. Simpler quality improvement interventions

(QIIs), such as clinician education, screening and feedback, and computer reminders alone, have no effect on major depression22-29. Studies show similar effects among elders8, adolescents10, and ethnic minorities30-33, and endure two to five years34-36. These findings can be replicated in small practices9 and in staff model and network model managed care1,2,4,5,6,8,10,11, in VA7 and in rural and urban settings4,9,10. Detailed tools and information about how to implement collaborative care have been well-documented and are easily accessible37-44.

The collaborative care model fills the existing gap between mental health specialty and primary care in caring for depression. The model systematically brings together mental health specialty and primary care resources and perspectives to ensure that patients with depression are assessed, educated, and followed for symptoms and 2 treatment adherence in accordance with AHRQ guidelines. It is similar in its outlines to the chronic illness care model45,46, and requires a committed staff person (the depression care manager) to support primary care clinicians and their collaboration with mental health specialists.

The VA team is carrying out a series of studies under the rubric of Translating

Initiatives in Depression into Effective Solutions (TIDES) to assess how collaborative care for depression can best be implemented by the VA nationwide. These studies use quality improvement methods to assist VA clinical and administrative leaders to adapt collaborative care to the specifics of VA practice, and develop the necessary tools, policies, procedures, and staffing to make the model part of usual care. In this paper, we describe in detail the latest of these projects, which focuses on learning about regional sustainability, spread, and national rollout needs for VA depression collaborative care

(Regional TIDES, or ReTIDES).

Previous Work (TIDES)

In the initial TIDES studies, we aimed to tailor and implement collaborative care in two to three large primary care practices in each of three multi-state VA administrative regions (Veterans Integrated Service Networks, or VISNs), using Evidence-Based

Quality Improvement . We built on the previous study results showing that CQI premised on facilitating evidence-based care model designs improved results compared to classical CQI 24,25, and focused this study’s QI on achieving collaborative care models.

We had also observed that practices participating in QI for depression often faltered on their inability to support technical and communication functions necessary for completing an effective design, and we provided research team support for these functions.

Lisa Rubenstein Page 2 5/4/2018 D:\Docs\2018-04-09\056fe2a23928596798fe12872ef3f1db.doc 3

We engaged senior leaders from primary care, mental health, nursing, and quality management in three VISNs in designing evidence-based but VISN-specific collaborative care for depression. We directed our QI design process at VISNs rather than medical centers because in the VA, care models that are not endorsed by VISNs are unlikely to last. This is because money for clinical care passes through VISN administration on its way to individual medical centers, and through medical centers to center-based and community outpatient clinics. Formulas for allocating these resources emphasize comparability across centers and clinics. Once VISN leaders had identified design specifications, we used rapid-cycle Plan-Do-Study-Act (PDSAs) to assist VISN personnel and local leaders to develop the specific tools and approaches implied by the overall VISN designs. Tools included templates for policies, procedures, and hiring; training for care managers; assistance with monitoring of patient and system outcomes; and collaboration between VISN staff and clinicians. We convened and provided infrastructure for VISN/researcher partnership work groups including: Informatics;

Patient Education and Care Management; Senior Leaders; and Collaboration.

VISNs chose nearly identical basic design features for VA collaborative care. In the final TIDES collaborative care model, as developed by the VISNs, primary care clinicians refer their patients to a depression care manager through the usual electronic consult mechanism. Emphasis is placed on referring screen-positive patients identified through the VA’s yearly depression screening program. Care management is carried out by telephone, with mailed materials and web support. Visit documentation and workload recording follow established mechanisms and conventions. Most clinicians and patients in involved clinics assume that TIDES is part of usual care.

Because TIDES is a QI project, and patients participate as part of an enhancement of their usual care, patients do not complete a research consent process to participate in

TIDES. The research team therefore has no access to individually identified information on TIDES patients. Similarly to other non-research QI interventions, we evaluated the initial TIDES care model in participating practices based on aggregated clinical results for patients referred to depression care management. We summarized the results for the panel (registry) of patients entering care management at each site and across all sites every three months as a quarterly report. We showed, for example, depression symptom graphs for the panel, percentages of people completing six months of care management, and percent of patients followed in primary care (Appendix 1). With our support, VISNs and practices used these results and others in continuous improvement, or Plan-Do-

Study-Act cycles. These cycles in turn served as the basis for midcourse corrections.

Our design is summarized below in Figure 1. In this figure, the horizontal axis represents time, progressing from left to right, and the volume of the large cube represents patients with depression visiting the practices.

POPULATION OF INTERVENTION IN DEPRESSED PLACE PATIENTS VISITING EXPERIMENTAL PRACTICES

CONTINUOUS PRE-POST CLINICAL F/U OF PATIENTS INTERVEN- RECEIVING INTERVENTION TION START-UP QI DESIGN PROCESS QUARTERLY REPORTS Clinician Intervention

Researcher Intervention Researcher Support PDSA CYCLES

Fig. 1: Design for formative, PDSA evaluation

This design is typical for real-world quality improvement evaluations, such as the evaluations carried out by practices in the Breakthrough Series45. The design aims to assess whether the care model provides the expected level of benefit to patients actually exposed to it through naturalistic means as part of a practice’s usual care. The design does not show results for representative patients, nor for any comparison group (it assesses experimental practices only), reducing generalizability. This formative evaluation-type design tells more, however, about the actual clinical workings of the care model intervention as applied by local care settings than would the more generalizable designs. Here we see how clinicians and patients, without any researchers directing their actions, actually behave when exposed to the new care model in all its parts. Rather than trying to eliminate selection bias, this design might be viewed as learning about selection

Lisa Rubenstein Page 5 5/4/2018 D:\Docs\2018-04-09\056fe2a23928596798fe12872ef3f1db.doc 6 bias, both positively and negatively. For example, patient outcomes might be much worse than expected based on prior effectiveness studies because clinicians selected patients who were too sick to benefit. On the other hand, clinicians might achieve better than expected results because they were able to be much more precise than study enrollment criteria in identifying patients who would benefit.

Another consideration in using a QI-type formative evaluation in TIDES is that if collaborative care is implemented as a routine clinical model, ongoing tracking of registry patient outcomes is essential for preventing model deterioration. The TIDES formative evaluation method will become the basis for standard outcome reporting during future TIDES spread.

The main threat to the face validity of this formative design when used to track the quality of an implemented evidence-based intervention, rather than to assess the intervention’s generalizable level of efficacy or effectiveness, is that it will not by itself take account of patients who could have benefited but did not access the model. The main corrective for this is the implementation of high quality performance measures that assess outcomes across the entire depressed population, and thus can signal any problems with model access.

Qualitative Studies to Prepare for Roll-Out: During TIDES, we identified additional informational needs. The Cost and Value (COVES) study is directed at understanding stakeholder perspectives on collaborative care. In this study, TIDES clinics are evaluated for system level impacts on utilization and cost. Senior leaders, clinicians, and patients from these clinics are undergoing semi-structured interviews that are being coded using rigorous qualitative methods. The Creating HealtheVet

Informatics Applications for Collaborative Care (CHIACC) project meets the need to

Lisa Rubenstein Page 6 5/4/2018 D:\Docs\2018-04-09\056fe2a23928596798fe12872ef3f1db.doc 7 improve the functionality of future VA clinical informatics for supporting proactive panel management. Results from these studies will be translated into tools for national rollout.

Evaluating the effectiveness of TIDES for representative patients: We also carried out a randomized sub-study of TIDES (the Well-Being Among Veterans

Enhancement Study, or WAVES). The timing was such that prior to identifying TIDES intervention practices, we were able to construct several matched sets of three midsized community-based outpatient clinics within each VISN based on size, location, and academic affiliation. VISN leadership chose among these sets of matched clinics, and we then randomized two clinics within each triplet to intervention and one to usual care in preparation for WAVES.

When WAVES began, we screened patients in experimental and usual care clinics for depression, and referred them to the ongoing TIDES intervention. Analysis of six- month follow-up for WAVES patients is in progress. In addition to assessing intervention effectiveness, we will be comparing the characteristics of the representative patients in WAVES to the group of patients referred by their clinicians to TIDES. This will help us to understand the implications of carrying out randomized trials of representative patients visa vis what is likely to happen when TIDES is operating under usual clinic circumstances. WAVES will also evaluate the cost-effectiveness of TIDES collaborative care. For example, so far it appears that clinicians refer fewer of those who are severely ill or previously treated by mental health specialists to care management than does our WAVES enrollment process.

Regional Roll-Out of TIDES (ReTIDES): Once we had shown positive clinical outcomes for TIDES patients, and had developed VA-specific policy, informatics,

Lisa Rubenstein Page 7 5/4/2018 D:\Docs\2018-04-09\056fe2a23928596798fe12872ef3f1db.doc 8 decision support, and care management tools, we aimed to carry out and evaluate regional spread of VA collaborative care.

Purpose: The purpose of ReTIDES is to prepare for national implementation of

TIDES by 1) demonstrating and evaluating spread to additional regions and sites, and 2) preparing the full set of necessary VA tools, policies, procedures, methods, and political links. Local impacts to be assessed in ReTIDES include performance improvement, system costs, long-term cost-effectiveness, and changes in clinician collaboration, knowledge, and skills.

Intervention: ReTIDES uses the same Evidence-Based Quality Improvement method as TIDES used, but begins with the tools and approaches already developed in

TIDES. Design decisions that were made in TIDES and were successful form the starting point for ReTIDES EBQI.

One additional feature of the ReTIDES intervention is a focus on national policy contacts. Through the Mental Health QUERI Center ReTIDES seeks to partner with national VA leadership in nursing, primary care, mental health specialty, employee education, performance measurement, guidelines, and system strategic planning (See

Template).

Intervention Targets: We expect to improve system performance for depression care through our organizational collaborative care intervention, and thereby to improve the outcomes for depressed patients visiting intervention practices. More specifically, our intervention targets VISN leadership and resources in order to change healthcare provider behavior in practices. These changes, including institution of care management, are expected to translate into improved clinical treatment of depression and hence into improved depression clinical outcomes (see Template).

Settings: The intervention is taking place in about 27 outpatient practices in four

VA VISNs. This represents between two and nine additional medical centers in each of the three original TIDES VISNs, and the addition of a new VISN. Each VISN identified new practices in ReTIDES in a different way. One VISN designated substantial funding for TIDES implementation. This VISN created a steering committee with representatives from a wide range of practices to foster site participation. In a second VISN, a VISN- wide primary care council is active in promoting TIDES. A third VISN is spreading

TIDES through the VISN mental health leader. Finally, in a fourth VISN the VISN director designated the TIDES sites from the top.

Outcome Measures: 1) Is collaborative care successfully adopted and sustained in experimental VISNs? (as measured by qualitative data); 2) Do patients referred by clinicians to care management adhere to treatment? Do their depression symptoms improve? Does their depression remit? (as measured by clinical data collected by care managers); 3) Does depression care performance improve in experimental compared to matched comparison clinics? (as measured by electronic data); 4) Do provider collaboration, knowledge and skill improve? (as measured by web-based survey) 3)

Does system cost-effectiveness improve? (as measured by electronic data); 4) Is the intervention cost-effective at 18 months (as measured by survey and electronic data); 5)

Does ReTIDES affect national VA perspectives, plans, and policies? (as assessed using qualitative methods)?

Study Design: The study’s comparative-change design (nonequivalent group design with unique pre-test and post-test samples) is depicted below in Figure 2. Again, time is depicted horizontally moving from left to right, and the volume of the large cube

Lisa Rubenstein Page 9 5/4/2018 D:\Docs\2018-04-09\056fe2a23928596798fe12872ef3f1db.doc 10 represents patients visiting the practices. The volume of the lighter “Care Model in Place” cube represents patients actually participating in the full care model.

PRE- POST- INTERVENTION INTERVENTION POPULATION OF DEPRESSED PATIENTS CROSS CROSS VISITING EXPERIMENTAL PRACTICES SECTIONAL SECTIONAL SAMPLES SAMPLES Care Model In Place

EXPERI- EXPERI- EXPERI- EXPERI- MENTAL MENTAL MENTAL MENTAL VISNS VISNS VISNS VISNS AND AND AND AND CLINICS CLINICS CLINICS CLINICS CARE MODEL START-UP POPULATION OF QI DESIGN DEPRESSED MATCHED MATCHED MATCHED MATCHED PATIENTS USUAL USUAL PROCESS USUAL USUAL VISITING USUAL CARE CARE CARE CARE CARE PRACTICES VISNS AND VISNS AND Clinical VISNS AND VISNS AND CLINICS CLINICS Partner Intervention CLINICS CLINICS

Researcher Intervention Researcher Support

CLINIC PERFORMANCE CLINIC PERFORMANCE MEASURES MEASURES

Fig. 2: Regional Roll-Out Comparative Change Design

As shown, in ReTIDES researchers initiate and structure a quality improvement process (Evidence-Based Quality Improvement) at the VISN level (the researcher intervention is to structure the VISN QI process). Clinical leaders from the VISN implement the design (the clinical partner intervention) with technical and communications support from the research team. Once the care model is established in the experimental clinics it reaches some proportion of the population of depressed patients attending those clinics. For the evaluation, ReTIDES will select a series of cross-sectional samples of depressed patients, some of whom, as depicted, have been exposed to the TIDES care model and others (those outside the box labeled “Care Model in Place”) who have not. We will monitor quality of care (e.g., completion of appropriate

Lisa Rubenstein Page 10 5/4/2018 D:\Docs\2018-04-09\056fe2a23928596798fe12872ef3f1db.doc 11 treatment) over the six months following each patient’s entry into one of our evaluation cohort.

Comparison Group: We are matching intervention VISNs and their practices with usual care VISNs and their practices for comparison. The main matching method is through existing structure surveys, administrative data, and performance measure data.

We will also compare matched clinic performance to intervention clinic performance for the pre-intervention period.

Analytic Approach: We will identify relevant cohorts of patients from electronic data bases and measure their care, also electronically. For example, we will identify patients who meet the denominator requirements for HEDIS measures; patients who screen positive for depression; patients newly started on any antidepressant, and patients with a new ICD 9 diagnosis of depression given by a primary care clinician. We will determine whether, for example, treatment completion is greater among patients in

TIDES practices than in matched usual care practices. We hope to apply these measures every six months, and to compare across four time periods--two pre-TIDES implementation and two post-TIDES implementation to compare. Detailed analytic plans are in process. We expect to do a subanalysis to examine disparities in care.

Results

Results are in process. Spread has been achieved or is in active process, however, in one new VISN and 15 new practices, and we have learned an enormous amount about how spread occurs. For example, we learned that the ability of TIDES to impact performance measures is critical. TIDES will not save VISNs major dollars in the short run (although it is relatively cheap); this is because the consequences of untreated depression are more likely to be, for example, job loss (affecting the patient only) than hospitalization (a cost to the health plan). To justify investment in TIDES, therefore,

VISNs and practices will need a very visible metric showing TIDES progress and effects.

We also learned that care model “tweaking” is continuous and that the Mental Health

QUERI role in both supporting innovation and maintaining tight links to the evidence base will be critical to ongoing effectiveness of the model.

Conclusions and Discussion of Design Choices

To a major extent, our choice of a non-randomized evaluation design was conditioned on what we already knew about both about provider behavior change in general and about depression care and care improvement in specific, based on high quality prior literature, much of it in the form of randomized trials. Our task was not to test effectiveness, but to determine how a large body of effectiveness literature and accompanying tools could be implemented to become usual care in VA. To facilitate eventual system-wide adoption of collaborative care, we needed to evaluate the process by which our study VISNs spread TIDES care. Our evaluation methods needed to continue to assess the clinical integrity of the model, while identifying potential spread pitfalls, resources, and methods for dealing with both. This required a non-randomized design.

The consequences of randomization would have been inimical to our study goals.

First, our organization and practice-level intervention demands cluster randomization.

Given the heterogeneity of practices on dimensions that we know have major effects on performance such as size, rural location, academic affiliation, and others, randomization at the practice level requires matching. This reduces the pool of possible sites, and leaves out any type of practice that cannot be easily matched. Thus, within a VISN, randomization substantially constrains leadership choice regarding spread. This in turn takes authority for TIDES away from the clinical leadership whose decisions we want to study. More subtly, randomization would firmly identify TIDES as research during a phase in which we want to study it as quality improvement. Finally, randomization would add substantially to the cost of the study, in no small part because of added IRB requirements.

We use matched practices in non-TIDES VISNs as our comparison group because

TIDES is quite likely to spread in unpredictable ways within TIDES VISNs. We are fairly confident of our ability to match practices because of the relatively large amount of data we have on all VA practices. In prior studies, our basic matching approach has been validated by the comparability of detailed patient and patient care data on matched sites.

While we recognize the limitations of matching in assuring internal validity we also recognize its strengths when our purpose is to understand a complex naturalistic phenomenon.

Another important design choice for this study was to use only administrative data for assessing patient outcomes. We chose this approach for two major reasons. One is that the VA will use electronic data to evaluate any future TIDES rollout. The second is cost. This data has a major limitation for assessing depression care in that it cannot

Lisa Rubenstein Page 13 5/4/2018 D:\Docs\2018-04-09\056fe2a23928596798fe12872ef3f1db.doc 14 accurately identify patients with major depression by DSM-IV criteria. TIDES care is directed at patients with major depression based on a systematic assessment. We are attempting to minimize this problem by defining the cohorts for our assessments in a variety of ways that range from inclusive to narrow in relationship to the likely level of match with DSM-IV criteria. We will also be limited in terms of adjusting for casemix.

We hope that these limitations will be mitigated by our ability to identify full population cohorts at multiple time points.

A major challenge for this project is to align system incentives with collaborative care for depression--simultaneously pushing forward regional expansion and national agenda-setting. This requires a continual top-down and bottom-up approach. For example, to gain top-down support, we must demonstrate that our intervention improves performance on the imperfect measures in use (HEDIS), and that costs are offset by the improved performance. This cannot be accomplished without a large-scale intervention.

On the other hand it is hard to spread and sustain a large-scale intervention without top- down support, especially given the current VA budget environment (most medical centers have frozen FTEEs). Similarly, to improve current performance measures, we must gain national VA support. To gain national VA support, we must integrate TIDES with strategic planning initiatives, and we must update national VA guidelines for depression care. To achieve this, we must already have some evidence of success. Finally, to achieve seamlessness in usual care, we must change VA policy. To provide information on which such a policy would be based, we need to demonstrate that TIDES is broad- based and successful. Our study design must be realistic enough and flexible enough to accomplish all of these goals.

Conclusions Regarding Improving the Science Base for QII evaluation research

Although final lessons learned are pending, we include the following as illustrations: 1) The process of translating randomized trial research evidence on collaborative care for depression into improved system performance is non-linear, and cannot be accomplished by “handing off” research evidence to clinical leadership even in a relatively top-down system like VA . National implementation requires a multi- pronged and coordinated approach that links current system incentives to data on how the model performs in actual VA settings, while at the same time working to improve the fit of those incentives to the model. This in turn requires a series of different study designs depending upon the specific questions managers, clinicians, and policy makers have about the model. 2) Policy makers and clinical managers want to refer to a rigorous evidence base, but want business case analysis of the care model as implemented in their own system. So far, we find VA leaders accepting of the prior strong evidence base on collaborative care for depression as a basis for change, and interested in further implementation research on TIDES. The information these leaders now demand for decision-making relates to the business case. The business case VA managers want researchers to assemble includes an assessment of the impact of the program, when implemented by VA employees in VA practices, on economic and performance measures. 3) For sustainability- and spread-oriented research, involvement of a steering committee that includes high-level system managers is essential. We believe it is neither ethical nor practical to carry out large-scale research that uses healthcare system resources without oversight and input from key stakeholders. The clinical organization, not the researchers, owns the project, and should have final authority over it. 4)

Implementation research requires willingness at all phases to adapt scientific methods creatively to the needs of the field. At the same time, representing the true level of

Lisa Rubenstein Page 15 5/4/2018 D:\Docs\2018-04-09\056fe2a23928596798fe12872ef3f1db.doc 16 generalizability of available evidence, and what questions any particular design will or will not answer, is a critical part of the role of the scientist in the clinical/health services research partnership.

Funding Source

VA VISNs and medical centers (primarily to support the intervention as well as part of the evaluation); Health Services Research and Development (HSR&D) QUERI (to support part of the intervention and most of the evaluation).

Was the Evaluation Plan Subject to Peer Review and IRB Approval

Yes, ReTIDES is subject to both peer review and IRB approval. Other than the

18 month cost-effectiveness piece which is based on individual patient information,

ReTIDES is exempt from human subjects consent requirements. It must, however, be reviewed by IRBs.

1. Katon WJ, Von Korff M, Lin E, et al. Collaborative management to achieve treatment guidelines: Impact on depression in primary care. JAMA. 1995;273(13):1026-1031 (orig trial) 2. Katon W, Von Korff M, Lin E, et al. Stepped collaborative care for primary care patients with persistent symptoms of depression: a randomized trial. Arch Gen Psychiatry. Dec 1999;56(12):1109-1115. (persistent dep trial) 3. Von Korff M, Gruman J, Schaefer J, Curry SJ, Wagner EH. Collaborative management of chronic illness. Ann Intern Med. 1997;127(12):1097-1102 (orig trial). 4. Wells KB, Sherbourne C, Schoenbaum M, et al. Impact of disseminating quality improvement programs for depression in managed primary care: a randomized controlled trial. JAMA. 2000;283(2):212-220. (PIC) 5. Hunkeler EM, Meresman JF, Hargreaves WA, et al. Efficacy of nurse telehealth care and peer support in augmenting treatment of depression in primary care. Archives of General Medicine. 2000;9(8):700- 708. 6. Simon GE, VonKorff M, Rutter C, Wagner E. Randomized trial of monitoring, feedback, and management of care by telephone to improve treatment of depression in primary care. Br Med J. 2000;320(7234):550-554. 7. Hedrick SC, Chaney EF, Felker B, et al. Effectiveness of collaborative care depression treatment in Veterans' Affairs primary care. J Gen Intern Med. Jan 2003;18(1):9-16. 8. Unutzer J, Katon W, Callahan CM, et al. Collaborative care management of late-life depression in the primary care setting: a randomized controlled trial. JAMA. 2002;288(22):2836-2845. (IMPACT) 9. Rost K, Nutting P, Smith J, Werner J, Duan N. Improving depression outcomes in community primary care practice: a randomized trial of the quEST intervention. Quality Enhancement by Strategic Teaming. J Gen Intern Med. 2001;16(3):143-149. (QuEST) 10. Asarnow JR, Jaycox LH, Duan N, et al. Effectiveness of a quality improvement intervention for adolescent depression in primary care clinics: a randomized controlled trial. JAMA. Jan 19 2005;293(3):311- 319. 11. Dietrich AJ, Oxman TE, Williams JW, Jr., et al. Re- engineering systems for the treatment of depression in primary care: cluster randomised controlled trial. Br Med J. Sep 11 2004;329(7466):602. (RESPECT) 12. Dietrich AJ, Oxman TE, Williams JW, Jr., et al. Going to scale: re-engineering systems for primary care treatment of depression. Ann Fam Med. Jul-Aug 2004;2(4):301-304. (RESPECT) 13. Schoenbaum M, Unutzer J, Sherbourne C, et al. Cost-effectiveness of Practice-Initiated Quality Improvement for Depression: Results of a Randomized Controlled Trial. JAMA. 2001;286(11):1325-1330. (PIC)

14. Pyne JM, Rost KM, Zhang M, Williams DK, Smith J, Fortney J. Cost-effectiveness of a primary care depression intervention. J Gen Intern Med. Jun 2003;18(6):432-441. (QuEST) 15. Pyne JM, Smith J, Fortney J, Zhang M, Williams DK, Rost K. Cost-effectiveness of a primary care intervention for depressed females. J Affect Disord. Mar 2003;74(1):23-32. (QuEST) 16. Simon GE, Katon WJ, VonKorff M, et al. Cost- effectiveness of a collaborative care program for primary care patients with persistent depression. Am J Psychiatry. Oct 2001;158(10):1638-1644. (persist dep) 17. Von Korff M, Katon W, Bush T, et al. Treatment costs, cost offset, and cost-effectiveness of collaborative management of depression. Psychosom Med. Mar-Apr 1998;60(2):143-149. (orig trial) 18. Rost K, Smith BS, Elliott CE, Dickinson LM. The Effect of Improving Primary Care Depression Management on Employee Absenteeism and Productivity: A Randomized Trial. Med Care. In press. (QuEST) 19. Simon GE, Revicki D, Heiligenstein J, et al. Recovery from depression, work productivity, and health care costs among primary care patients. Gen Hosp Psychiatry. 2000;22(3):153-162. 20. Schoenbaum M, Unutzer J, McCaffrey D, Duan N, Sherbourne C, Wells KB. The effects of primary care depression treatment on patients' clinical status and employment. Health Serv Res. 2002;37(5):1145-1158. (PIC) 21. Liu CF, Hedrick SC, Chaney EF, et al. Cost- effectiveness of collaborative care for depression in a primary care veteran population. Psychiatr Serv. May 2003;54(5):698-704. (VA) 22. Solberg LI, Fischer LR, Wei F, et al. A CQI intervention to change the care of depression: a controlled study. Eff Clin Pract. 2001;4(6):239-249. 23. Goldberg HI, Wagner EH, Fihn SD, et al. A randomized controlled trial of CQI teams and academic detailing: can they alter compliance with guidelines? J Qual Imp. 1998;24(3):130-142. 24. Rubenstein LV, Parker LE, Meredith LS, et al. Understanding team-based quality improvement for depression in primary care. Health Serv Res. 2002;37(4):1009-1029. 25. Rubenstein LV, Meredith LS, Parker LE, et al. Impacts of Evidence-Based Quality Improvement on Depression in Primary Care: A Randomized Experiment. Submitted (revise and resubmit). 26. Rubenstein LV, McCoy JM, Cope DW, et al. Improving patient quality of life with feedback to physicians about functional status. J Gen Intern Med. Nov 1995;10(11):607-614. 27. Rollman BL, Hanusa BH, Lowe HJ, Gilbert T, Kapoor WN, Schulberg HC. A randomized trial using computerized decision support to improve treatment of major depression in primary care. J Gen Intern Med. Jul 2002;17(7):493-503.

28. Gerrity MS, Cole SA, Dietrich AJ, Barrett JE. Improving the recognition and management of depression: is there a role for physician education? J Fam Pract. Dec 1999;48(12):949-957. 29. Thompson, C., A. L. Kinmonth, et al. (2000). Effects of a clinical-practice guideline and practice-based education on detection and outcome of depression in primary care: Hampshire Depression Project randomised controlled trial. Lancet 355(9199): 185-91.

30. Rost K, Pyne JM, Dickinson LM, LoSasso AT. Cost-effectiveness of enhancing primary care depression management on an ongoing basis. Ann Fam Med. Jan-Feb 2005;3(1):7-14. (QuEST) 31. Miranda J, Schoenbaum M, Sherbourne C, Duan N, Wells K. Effects of primary care depression treatment on minority patients' clinical status and employment. Arch Gen Psychiatry. Aug 2004;61(8):827-834. (PIC) 32. Miranda J, Duan N, Sherbourne C, et al. Improving care for minorities: can quality improvement interventions improve care and outcomes for depressed minorities? Results of a randomized, controlled trial. Health Serv Res. Apr 2003;38(2):613-630. 33. Schoenbaum M, Miranda J, Sherbourne C, Duan N, Wells K. Cost- effectiveness of interventions for depressed Latinos. J Ment Health Policy Econ. Jun 2004;7(2):69-76. (PIC) 34. Sherbourne CD, Wells KB, Duan N, et al. Long-term effectiveness of disseminating quality improvement for depression in primary care. Arch Gen Psychiatry. Jul 2001;58(7):696-703. (PIC) 35. Wells K, Sherbourne C, Schoenbaum M, et al. Five-year impact of quality improvement for depression: results of a group-level randomized controlled trial. Arch Gen Psychiatry. Apr 2004;61(4):378-386. (PIC) 36. Rost K, Pyne JM, Dickinson LM, LoSasso AT. Cost-effectiveness of enhancing primary care depression management on an ongoing basis. Ann Fam Med. Jan-Feb 2005;3(1):7-14. (QuEST) 37. Dietrich AJ, Eisenberg L. Better management of depression in primary care [editorial]. J Fam Pract. 1999;48(12):945-946. 38. Oxman TE, Dietrich AJ, Williams JW, Jr., Kroenke K. A three-component model for reengineering systems for the treatment of depression in primary care. Psychosomatics. Nov-Dec 2002;43(6):441-450. 39. Rubenstein LV, Parker LE, Meredith LS, et al. Understanding team-based quality improvement for depression in primary care. Health Serv Res. 2002;37(4):1009-1029. 40. Rubenstein LV, Jackson-Triche M, Unutzer J, et al. Evidence-based care for depression in managed primary care practices. Health Affairs (Millwood). 1999;18(5):89-105. 41. Rost K, Nutting PA, Smith J, Werner JJ. Designing and implementing a primary care intervention trial to improve the quality and outcome of care for major depression. Gen Hosp Psychiatry. Mar-Apr 2000;22(2):66-77. 42. Depression Patient Outcomes Research Team (PORT-II). Partners In Care. RAND. Available at:

http://www.rand.org/health/partners.care/portweb/. 7/1/01. Accessed August, 2005. 43. TIDES/WAVES/COVES VA HSR&D Website. Available at: http://www.va.gov/tides_waves/. Accessed August, 2005. 44. MacArthur Initiative on Depression in Primary Care Website. Available at:http://www.depression-primarycare.org. Accessed June, 2005. 45. Wagner EH, Glasgow RE, Davis C, et al. Quality Improvement In Chronic Illness Care: A Collaborative Approach. Jt Comm J Qual Improv. 2001;27(2):63-80. 46. Institute of Medicine. Crossing the Quality Chasm : A New Health System for the 21st Century. Washington: National Academy Press; 2001.

Lisa Rubenstein Page 20 5/4/2018 D:\Docs\2018-04-09\056fe2a23928596798fe12872ef3f1db.doc