Specialty Referral Patterns and Curricula Across I3 Collaborative

ORIGINAL ARTICLES

What’s the Right Referral Rate? Specialty Referral Patterns and Curricula Across I3 Collaborative Primary Care Residencies Mark Gwynne, DO; Cristen Page, MD, MPH; Alfred Reid, MA; Katrina Donahue, MD, MPH; Warren Newton, MD, MPH

BACKGROUND AND OBJECTIVES: Specialty physician visits ac- million specialty referrals annually.3 count for a significant portion of ambulatory visits nationally, con- In many settings, specialty consulta- tribute significantly to cost of care, and are increasing over the tions and referrals lack coordination past decade. Marked variability in referral rates exists among pri- and may generate testing redundan- mary care practices without obvious causality. We present data cy, disagreement on the role of the describing the referral process and specialty referral curriculum specialty physician, and lateral or within the I3 collaborative. cross-referrals among subspecialty 4 METHODS: Residency directors were surveyed about residency care. The result can be poor qual- characteristics related to referrals. Specialty physician referral rates ity, overutilization, and subsequent were obtained from each program and then correlated to pro- increased cost of care, and a patient gram characteristics referral rates in four domains: presence and experience that has been described type of referral curriculum, process of referral review, faculty pre- as “a perilous journey through the ceptor characteristics, and use of referral data for administrative Health Care System.”5 However, cost processes. and quality in primary care can be dramatically improved by carefully RESULTS: The survey response rate was 87%; 10 programs sub- defined relationships with specialty mitted complete referral data. Three programs (23%) reported a consultants.6 formal curriculum addressing the process of making a referral, There is increasing evidence that and four programs (31%) reported a curriculum on appropriateness of subspecialty referrals. Specialty referral rates varied from residency training has a long-term 7%–31% of active residency patients, with no relationship to age, impact on how graduates practice 7 payor status, or race. throughout their career; however, little information exists to describe DISCUSSION: Marked variability in referral rates and patterns how family medicine residencies exist within primary care residency training programs. Specialty train the process of referrals. Re- referral practices are a key driver of total cost of care yet few cur- ferring to specialty care is a funda- ricula exist that address appropriateness, quantity, or process of mental skill within primary care, specialty referrals. Practice patterns often develop during residen- and management of the process is cy training, therefore an opportunity exists to improve training and a central role of the patient-centered practice around referrals. medical home. Yet there is little re- (Fam Med 2017;49(2) 91-96.) ported about residency referral rates, appropriateness of referrals and curriculum to address these skills with- he consultation and referral frequency of referrals, defined as an in primary care training programs process from primary care outpatient referral made by a pri- Tto specialty care is complex, mary care provider to a subspecial- varies widely among primary care ty physician service, is increasing as providers, and is a key driver of well and has almost doubled in the From the Department of Family Medicine, 1,2 the total cost of health care. The last decade, resulting in almost 100 University of North Carolina at Chapel Hill.

FAMILY MEDICINE VOL. 49, NO. 2 • FEBRUARY 2017 91 ORIGINAL ARTICLES

and few standards exist to guide the program reported initial practice Results process of specialty referrals within characteristics, which included size Characteristics of primary care. This lack of training and setting of the residency, num- Participating Programs clarity may contribute to special- ber of providers and staff, electron- Table 1 summarizes the characteris- ty referral rate variability reported ic health record (EHR) used, faculty tics and distribution of participating among practicing primary care phy- physician involvement in quality I3 residency programs. The majority sicians.8,9 It is unlikely that we can and program priorities for improve- of residencies were family medicine, reduce cost as a health care system ment. For this project, a referral was mostly urban community and uni- unless we address the process and defined as an outpatient ambulato- versity-based programs, although a training around subspecialty refer- ry referral made by a primary care wide distribution of training environ- rals. provider to a subspecialty physician ments were represented. We set out to evaluate referral service. Referral rate was defined as Surveys were distributed to 24 curriculum and referral processes the percentage of annual outpatient residency program directors with an among 24 primary care residencies visits for empaneled patients result- 87% response rate (21/24). Of the 21 in North Carolina, South Carolina, ing in an ambulatory subspecialty residency programs who responded and Virginia. referral. This definition focused our to the survey, 13 had also submit- evaluation to subspecialty physician ted baseline specialty referral rate Methods care and therefore procedures and data. Table 2 summarizes the rela- Setting and Participants ancillary services such as physical tionship between question respons- The I3 collaborative consists of 24 therapy, nutritionist, diabetes edu- es within the four domains and the family medicine, internal medicine, cation, and were not included. Pro- relationship to referral rates. Three and pediatric residencies in North grams reported referral rates as the programs (23%) had a formal curric- Carolina, South Carolina, and Vir- sum total of referrals to their 10 ulum addressing the process of mak- ginia designed to improve quality most common specialties. With re- ing a referral, and four programs of care in residency practices. Prior spect to the survey, focus group dis- (31%) had a formal curriculum for phases of I3 have focused on chronic cussions of residency directors, the the appropriateness of subspecialty care and PCMH recognition.10,11 I3 academic collaborative, and others referrals. Figure 1 shows the com- POP, the current phase, focused on helped to frame the questions and bined specialty referral rates to the improving population health through calibrate the answers. I3 residency 10 most common subspecialties as implementing the triple aim of im- directors were surveyed using Qual- reported by participating practices. proving quality, the patient experi- trics software11 about four domains Although the referral frequency to ence, and health care utilization for of referral curricula: presence and each subspecialty varied, the most active patients in participating resi- type of referral curriculum, referral common specialties represented in- dencies. Methods are similar to those review process, faculty preceptor cluded GI, ophthalmology, orthope- described in prior phases10,11 and in characteristics, and use of referral dics, surgery, behavioral health, ENT, the description of baseline data.12 The data for administrative processes. dermatology, neurology, cardiology, I3 POP Collaborative used the Insti- Complete objective data on subspe- and OB-Gyn. The combined referral tute for Health Care Improvement cialty referrals were submitted from rates varied from 7% to 31% with a Breakthrough Series Collaborative 13 residencies participating in the median of 25%. There was no corre- design to assess practice transfor- I3 collaborative. Programs submit- lation between university relation- mation and rapidly spread change ted objective referral data abstracted ship, payor status or percentage of specifically in primary care residen- from their electronic medical records minority patients and rate of refer- cies. Collaborative participants met or other practice management soft- rals. There were no statistically sig- through monthly webinars as well ware. In total, 10 programs submit- nificant differences in referral rates as face to face collaborative meetings ted both baseline referral data and within the four domains assessing every six months. Baseline data from survey response data for compara- the relationship between curriculum the practices, including specialty re- tive analysis. Using chi-square test- and referral rates in the question- feral data, were reported annually ing, we assessed the likelihood and naire, but there were trends toward and for some metrics monthly and significance that programs with decreased referral rates in programs quarterly. This project was approved referral rates less than the medi- that had formal review processes for by the I3 POP Academic Collabora- an of the collaborative rate (25%) referrals from advanced practice pro- tive. would answer “yes” or “no” to each viders, used report cards for provid- question. This study was reviewed ers that included individual referral Data Collection and Analysis and exempted by the University of data, had curriculum that addressed As a requirement of participation North Carolina’s Institutional Re- the appropriateness of subspecial- in the I3 POP collaborative, each view Board. ty referrals, and had a greater

92 FEBRUARY 2017 • VOL. 49, NO. 2 FAMILY MEDICINE ORIGINAL ARTICLES

Table 1: Characteristics of I3 Collaborative Residency the first available data comparing Programs Participating in Referral Survey referral curricula and rates across many residencies. Another limita- I3 Collaborative Residency Specialties tion of this work is that residency n (%) program directors responding to the Total residency practices 30 survey may have had different un- derstandings of what “referral cur- Family medicine 25 (83) riculum” meant. Also the survey did Internal medicine 2 (7) not define the process of making referrals nor define appropriateness. Pediatrics 3 (10) More detailed interviews and focus Program Characteristics With Both Survey and Baseline Referral Data groups will be necessary to distin- guish the key elements of context n (%) and curriculum that are important to note. Finally, these data are lim- University department 1 (10) ited to referrals for subspecialty care Community setting 9 (90) or other clinical services and do not address other high cost issues such Rural setting 1 (10) as high end radiology or specialty medication use. These are also im- PCMH certification 8 (80) portant drivers of cost and quality I3 Collaborative Characteristics and should be addressed in future 109,227 studies. Empaneled patients Within these limitations, then, Yearly visits 314,373 what are the implications of these data? We believe that the process of 249 Current residents in training consultations and referrals, both ap- Current faculty clinicians 183 propriateness and volume, are foun- dational to a central role for primary care in the health care systems of proportion of internal faculty precep- It is important to acknowledge the the future. They are essential to im- tors, defined as core campus residen- limitations of our study. First, par- plementing the triple aim.13 This cy faculty as opposed to community ticipating residency programs were reality is reflected by the rapid emer- preceptors. We also asked for narra- from the Southeast which, given re- gence of narrow networks in which tive description of referral curricu- gional variation in referral patterns, primary care providers are incentiv- lum. Programs that described having may limit the generalizability of our ized to refer to lower cost and high- a referral curriculum, for both pro- findings. Moreover, we only had data er quality specialists. There is also cess and appropriateness, identified on referral rates from 10/24 partici- increasing evidence that residency these sessions during resident orien- pating residencies, raising the possi- programs impart long-term patterns tations or grand rounds and resident bility of selection bias and limiting of care on their residents, including didactics related to specific clinical the power of the study. This limi- choice of medications, complication topics. tation speaks to the barriers that rates for procedures,14 and overall ex- many programs continue to face in perience of care. While we do not yet Discussion accessing and using reliable data know if referral behaviors learned Our data indicate a striking lack of about internal processes and prac- in residency persist over the long formal curricula or operational man- tice patterns. Barriers included ac- term, there is no reason to assume agement of referrals within our pri- cessing data, expertise in running that they are unlike other complex mary care residencies. Less than 1/3 non-routine reports, and prioriti- behaviors of physicians. In this con- of the residencies had formal curri- zation of resources to address this text, then, the lack of attention to cula, trained preceptors, or system- specific question. Despite these limi- referrals as an explicit and formal atically assessed the appropriateness tations, however, the survey response part of the curriculum in these resi- of referrals, quantity, or outcomes. rate of program directors was very dencies is an area for development. Likewise, there is dramatic variabil- high, and the I3 residencies reflect What would the ideal “refer- ity in specialty referral rates among the diversity of Family Medicine res- ral curriculum” consist of? Figure residency programs within the I3 idencies and vary by size, university 2 outlines key aspects of the refer- collaborative. relationship, and urban/rural geogra- ral process and illustrates the pos- phy. Moreover, these data represent sible curricular targets. We believe

FAMILY MEDICINE VOL. 49, NO. 2 • FEBRUARY 2017 93 ORIGINAL ARTICLES

Table 2: Association of Factors in Four Domains With Specialty tracking will be important. Our Referral Rate Less Than the Median Rate of the Sample* data, with its relatively low number of residencies, were unable to show that experience of preceptors % with influence the rate of referrals, how- referral ever did show a potential relation- rate < P Value ship between type of preceptor and Factor median n (χ2) referral rates. There may be vari- Curriculum able practice patterns between fac- Curriculum on referral process Yes 33 3 .49 ulty preceptors who primarily see No 57 7 patients within residency practic- Curriculum on referral appropriateness Yes 67 3 .49 es versus those who work in community practices. Whether primary No 43 7 practice environment affects facul- Formal referral review process ty precepting around referrals is a For residents Yes 20 5 .06 question worth future study. Our No 80 5 data suggest a third area of focus should be developing some degree For mid-level providers Yes 100 1 .24 of “formal curriculum” within core No 38 8 didactics that address appropriate- Use of referral data ness of referrals. Didactic curricu- To monitor process internally Yes 50 6 1.00 la vary greatly across residencies. When and how to incorporate refer- No 50 4 ral curricula is an appropriate topic To give feedback Yes 0 2 .11 for discussion and debate. Several No 63 8 programs reported embedding dis- To track trends Yes 67 3 .49 cussion about referral appropriateness in major disease topic didactics. No 43 7 This may be a reasonable approach. To monitor process externally Yes 33 3 .49 Fourth, ur data suggest that pro- No 57 7 viding regular feedback to residents For formal reporting to institution Yes 67 3 .49 about the volume and appropriateness of their referrals may impact No 43 7 referral rates. Scorecards, like other Process in place to audit referrals Yes 50 2 .81 comparative data including clinical No 40 5 quality, may drive physician behav- For report cards Yes 100 1 .23 ior change more than other interventions. How to provide this feedback No 43 7 efficiently yet with adequate fre- Faculty preceptors quency is challenging operationally. > median number of internal preceptors Yes 75 4 .12 Our findings suggest a number of No 33 6 fruitful areas for further work. First, in order to develop impactful curri- > median total number of preceptors Yes 40 5 .53 cula we will need to further explore No 60 5 and describe referral patterns to common subspecialties and develop * n refers to the number responding “yes” or “no” for each factor; % is the percentage of n with best practices. Second, among pro- referral rates below the sample median. gram directors and residency faculty, we should define the goals of a cur- that a starting point should be the the level of precepting, explicit guide- riculum to address what appropriate good of the patient, meaning ap- lines and formal review for referrals referrals mean for different clinical propriateness of referrals, with a may be appropriate, particularly for problems and we should develop bias towar taking care of as much advanced practice providers. guidelines to best manage the con- as possible within the patient-cen- If implemented, residents will sultation and referral process. Third, tered medical home. Beyond this then need to learn skillsets to man- in the spirit of family medicine for general stance, our data and expe- age this process in future practice. America’s Health,16 we should debate rience suggest four tactics: first, at Second, preceptor development and what the right rate of referrals is. A

94 FEBRUARY 2017 • VOL. 49, NO. 2 FAMILY MEDICINE ORIGINAL ARTICLES

Figure 1: Baseline Subspecialty Referral Rates of I3 Collaborative Residency Programs

Figure 2: Targets for Interventions: A Schematic of the Referral Process

Stars represent potential points to implement formal review of referral.

first step is agreeing upon a defini- address appropriateness. We do not these factors we can then evaluate tion of denominator, along with the know what the right rate is; howev- effectiveness of a curricula designed attributes necessary for risk adjust- er, it is important that the primary to address the referrals process. ment, such as age, RAF (risk adjust- care of the future continue to refer In conclusion, appropriate re- ed factor) score, comorbidities, and patients who need referral and ad- ferrals are a foundation for im- payer status. Once we have the best just scope of primary care practice plementing the triple aim. This estimates of rates we then need to for those who do not. If we can define is particularly true with the

FAMILY MEDICINE VOL. 49, NO. 2 • FEBRUARY 2017 95 ORIGINAL ARTICLES

consolidation of health networks and 2009. Arch Intern Med 2012;172(2):163-70. increasing attention and focus on the 4. Schoen C, Osborn R, Huynh PT. Taking the 12. Donahue KE, Reid A, Lefebvre A, Stanek M, health of a population. The wide pulse of health care systems: experiences of Newton WP. Tackling the triple aim in primary patients with health problems in six countries. care residencies: the I3 POP Collaborative. variation of referral rates among Health Aff 2005 Suppl Web Exclusives: W5- Fam Med 2015;47(2):91-7. these primary care residencies, along 509-W5-525. 13. Qualtrics software, Version 2015 of Qualtrics. with the lack of formal attention on 5. Bodenheimer T. Coordinating care—a perilous Qualtrics and all other Qualtrics product or how to teach referrals suggest a ma- journey through the health care system. N service names are registered trademarks or Engl J Med 2008;358:10. trademarks of Qualtrics, Provo, UT, USA. jor opportunity to improve primary 6. Milstein A, Gilbertson E. American medical http://www.qualtrics.com. care residency training. home runs. Health Aff 2009;28(5):1317-26. 14. Berwick DM, Nolan TW, Whittington J. The 7. Chen C, Petterson S, Phillips R, Bazemore triple aim: care, health, and cost. Health Aff CORRESPONDING AUTHOR: Address corre- 2008 May-Jun;27(3):759-69. spondence to Dr Gwynne, University of North A, Mullan F. Spending patterns in region of Carolina at Chapel Hill, Department of Family residency training and subsequent expendi- 15. Asch DA, Nicholson S, Srinivas Sindhu, Her- Medicine, Campus Box 7595, Chapel Hill, NC tures for care provided by practicing physicians rin J, Epstein AJ. Evaluating obstetrical resi- 27514-7595. 919-966-3711. Fax: 919-966-6125. for Medicare beneficiaries. JAMA 2014 Dec dency programs using patient outcomes. JAMA [email protected]. 10;312(22):2385-93. 2009;302(12):1277-83. 8. Mehrotra A, Forrest CB, Lin CY. Dropping the 16. Phillips RL Jr, Pugno PA, Saultz JW, et al. ACKNOWLEDGMENTS: Financial support was baton: specialty referrals in the United States. Health is primary: family medicine for Ameri- received from the Duke Endowment, Fullerton Milbank Q 2011;89(1):39–68. ca’s health. Ann Fam Med 2014 Oct;12:S1-S12. Foundation, North Carolina AHEC. 9. Forrest CB, Majeed A, Weiner JP, Carrol Kl, Bindman AB. Comparison of specialty referral References rates in the United Kingdom and the United States: retrospective cohort analysis. BMJ 1. Franks P, Zwanziger J, Mooney C, Sorbero M. 2002;325:370–1. Variations in primary care physician referral rates. Health Serv Res 1999 Apr;34(1 Pt 2): 10. Newton WP, Baxley E, Reid A, Stanek M, Rob- 323-9. inson M, Weir S. Improving chronic illness care in teaching practices: learnings from the I3 2. Franks P, Williams GC, Zwanziger J, Mooney Collaborative. Fam Med 2011;43(7):495-502. C, Sorbero M. Why do physicians vary so widely in their referral rates? J Gen Intern 11. Reid A, Baxely E, Stanek M, Newton WP. Med 2000 Mar;15(3):163-8. Practice transformation in teaching settings: lessons from the I3 PCMH Collab- 3. Barnett ML, Song Z, Landon BE. Trends in orative. Family Medicine, 2011;43(7):487-94. physician referrals in the United States, 1999-

96 FEBRUARY 2017 • VOL. 49, NO. 2 FAMILY MEDICINE