Chapter 3 Ministry of Health and Long-Term Care Section 3.02 Health Human Resources

care and education systems, and better manage the Background supply of health human resources. Total expenditures for the strategy grew from $448 million in the 2006/07 fiscal year Health human resources—physicians, nurses to $738.5 million in the 2012/13 fiscal year, an and other health-care providers—are crucial to increase of about 65%. These amounts included the delivery of health services. They represent $431 million for physician and nursing initiatives in the single greatest asset, as well as cost, to the 2006/07 and $728 million for them in 2012/13, as health-care system. Acting to address concerns over well as ministry operating expenses of $17 million provincial physician and nursing shortages, long in 2006/07 and $10.5 million in 2012/13, as shown wait times and an increasing number of patients in Figure 1. without family doctors, the Ministry of Health and

Chapter 3 • VFM Section 3.02 Long-Term Care and the Ministry of Training, Col- leges and Universities jointly developed a strategy called HealthForceOntario in the 2005/06 fiscal Audit Objective and Scope year. As part of the strategy, the Ministry of Health and Long-Term Care established the HealthForce­ The objective of our audit was to assess whether the Ontario Marketing and Recruitment Agency Ministry of Health and Long-Term Care, in conjunc- (Agency) in 2007. The Agency’s activities focus on tion with the Agency, had adequate systems and recruitment and retention of health professionals. procedures in place to: The strategy’s goal is to ensure that Ontarians identify and assess the appropriateness of have access to the right number, mix and distribu- • the mix, supply and distribution of qualified tion of qualified health-care providers, now and in health-care professionals to help meet the cur- the future. Responsibility for its implementation rent and future needs of Ontarians across the lies with the Health Human Resources Strategy province; Division of the Ministry of Health and Long-Term ensure that strategy initiatives were delivered Care (Ministry), but its Assistant Deputy Minister • in accordance with established regulatory reports to the Deputy Ministers at both ministries. requirements, applicable directives and poli- This is meant to establish a link between the health cies, and agreements; and

82 Health Human Resources 83

to representatives of other jurisdictions—Manitoba, Figure 1: Health Human Resources Strategy Division Alberta and British Columbia—to gain an under- Expenditures, 2006/07–2012/13 ($ million) standing of how health human resource planning is Source of data: Ministry of Health and Long-Term Care done in those provinces. 800 Operating expenses 700 Physician and nursing initiatives 600 Summary 500

400 Over the last six years, the Ministry of Health and 300 Long-Term Care (Ministry) has spent $3.5 billion 200 through its HealthForceOntario strategy to address

100 the shortages of physicians, nurses and other health professionals across Ontario. In 2012/13 the Min- 0 istry directed $738.5 million toward this strategy:

2006/07 2007/08 2008/09 2009/10 2010/11 2011/12 2012/13 $505 million for physician initiatives, $151 million for nursing initiatives, $72 million for other health • measure and report regularly on the progress human resource initiatives and the remaining of the strategy’s objectives. $10.5 million for operating expenses. The Ministry and Agency senior management Overall, Ontario has seen an 18% increase in reviewed and agreed to our audit objective and physicians from 2005 to 2012 and a 10% increase criteria. in nurses from 2006 to 2012. While the initiatives Our audit focused on physician and nurse human increased enrolment, created more postgraduate resources. In conducting our audit, we reviewed training positions and attracted more doctors and relevant legislation, administrative policies and pro- nurses from other jurisdictions, Ontario has not met cedures, and interviewed staff at the Ministries of its goal of having the right number, mix and distri-

Health and Long-Term Care and Training, Colleges bution of physicians in place across the province to Chapter 3 • VFM Section 3.02 and Universities. We visited three Local Health Inte- meet the population’s future health-care needs. gration Network (LHIN) offices and three hospitals Specifically, we noted the following: in the North West and South West regions, and we • The province spends an average of about contacted two hospitals in the Greater Toronto Area $780,000 (including $375,000 for resident region to interview staff and obtain relevant docu- salaries and benefits) to educate one special- ments. We also obtained information related to vari- ist who completes a four-year undergraduate ous nursing initiatives from Greater Toronto Area degree and up to five years of postgraduate hospitals. To gain an overall understanding and residency training. For a specialist who perspective of the health human resources area, we enters Ontario at the postgraduate level from spoke with a number of external stakeholders such outside the province, this cost is $225,000. as the College of Family Physicians, the Ontario However, many specialists trained in Ontario Hospital Association, the Registered Nurses Associa- do not stay and practise here. Retention tion of Ontario, the Registered Practical Nurses statistics show that, on average, 33% of Association of Ontario, the Professional Association Ontario-funded surgical specialist graduates of Residents of Ontario (formerly the Professional left the province each year between 2005 Association of Internes and Residents of Ontario), and 2011. The lack of full-time employment and the Ontario Medical Association. We also spoke opportunities for graduating residents of 84 2013 Annual Report of the Office of the Auditor General of Ontario

certain surgical specialties may lead to more • Although the physician forecasting model physicians deciding to leave the province, built in partnership with the Ontario Medical despite long wait times for these services. For Association was a positive step in determining example, wait-time data for the three-month physician workforce requirements, it is ham- period from June to August 2013 showed pered by the limited reliability and availability waits of 326 days for forefoot surgery and of data. These limitations make planning the 263 days for cervical disc surgery. optimal number, mix and distribution of phys- • The Agency provides temporary physician icians with appropriate funding, training and or “locum” coverage in eligible communities deployment difficult. As well, a simulation across the province to support access to care. model being developed by the Ministry to help However, vacancy-based locum programs plan for future nursing education positions meant as short-term measures continued to and to help formulate nursing policies aimed be used for long periods of time. At the time at recruitment and retention determines only of our audit there were about 200 specialist what the supply of nurses will be without vacancies in Northern Ontario, and of those considering how many nurses will be needed hospitals using locum services, one-third that to meet the population’s needs. had been using the Emergency Department Coverage Demonstration Project before Janu- OVERALL MINISTRY RESPONSE ary 2008 had been continuously using its The Ministry of Health and Long-Term Care locum services from as early as 2007, and one (Ministry) and the HealthForceOntario Market- hospital had been using them since 2006. ing and Recruitment Agency acknowledge and Over the four fiscal years from 2008/09 to • thank the Auditor General for the timely audit 2011/12, $309 million was dedicated to hir- and the recommendations in this report. ing 9,000 new nurses. Our review showed In a Canadian first, the province launched that while the system was unable to hire that the HealthForceOntario strategy in May 2006. many nurses in the four years, it had increased This was an innovative response to existing

Chapter 3 • VFM Section 3.02 the number of nurses by more than 7,300 and critical shortages in health human resources, the Ministry was on track to achieve its goal and it aimed to ensure that existing gaps would within five years. not worsen. At the end of 2011, 66.7% of nurses were • The strategy has led to a significant improve- working full-time in Ontario, which was just ment in the health human resource capacity of slightly under the Ministry’s goal of 70% of Ontario. Shortages of health providers, includ- nurses working on a full-time basis. However, ing physicians and nurses, are no longer the the Ministry needed to improve its oversight primary barrier to access or cause of wait times. and assessment of the effectiveness of its nurs- The strategy has mitigated the shortages and ing programs and initiatives. For example, improved the province’s ability to plan, train funding for the Nursing Graduate Guarantee and support its health workforce, with some key Program is provided for up to six months with results since May 2006 including: the expectation that organizations will offer more than 35,000 new regulated providers, permanent full-time employment for partici- • including an 18% increase in physician sup- pating new graduate nurses. However, only ply and a 10% increase in nurse supply; about one-quarter of program participants in expanded first-year undergraduate enrol- 2010/11 and one-third in 2011/12 actually • ment in medical schools (up by 22%) and obtained permanent full-time positions. first-year postgraduate trainees (up by 60%); Health Human Resources 85

professionals across Ontario. While the province 15,644 more nurses working full-time, a • was able to increase the number of physicians, some 23% improvement; Ontario communities face shortages of health-care 25 nurse practitioner-led clinics providing • providers, especially physicians. Primary-care care to over 36,000 patients; physicians, also known as family physicians, are more than 15,100 employment opportun- • not always available in small, rural or remote com- ities for new Ontario nursing graduates; munities. In Northern Ontario, general specialists new health-care provider roles including • (for example, in the areas of general surgery, inter- physician assistants, clinical specialist radia- nal medicine and psychiatry) also remain in high tion therapists and five new nursing roles; demand despite a significantly improved provincial creation of evidence capacity to inform • physician supply. planning; Although the significant amount of funds that legislative and regulatory changes increas- • the Ministry has expended over the last six years ing the quality and safety of patient care, has increased the supply of physicians in the prov- expanding scopes of practice and regulating ince, shortages remain in certain specialties and new health professions; and geographical areas even as physicians in those spe- establishment of the HealthForceOntario • cialties are unable to obtain full-time employment. Marketing and Recruitment Agency. Ontario is now able to focus health human resource activities on health-system transforma- Increased Supply of Physicians in Ontario tion rather than responding to critical shortages Medical education is funded jointly by the Min- of providers. The Ministry’s work continues istry of Training, Colleges and Universities and to evolve to address today’s challenges. The the Ministry of Health and Long-Term Care. The Ministry is renewing the HealthForceOntario Ministry of Training, Colleges and Universities strategy so that it: funds universities for undergraduate positions, builds on the successes of previous • while the Ministry of Health and Long-Term Care accomplishments;

funds most aspects of postgraduate training. The Chapter 3 • VFM Section 3.02 aligns with the goals of Ontario’s Action Plan • majority of the $485 million the Ministry of Health for Health Care; and and Long-Term Care spent on physician initiatives advances evidence-informed planning and • in the 2011/12 fiscal year was in two areas: total decision-making. payments of $315 million to medical schools and The recommendations in this audit will hospitals for the salaries and benefits of residents inform the strategy renewal. who provide clinical services across Ontario; and $107 million paid to medical schools to support academic activities such as teaching, educational infrastructure and related administrative costs for Detailed Audit Observations clinical education of medical learners. From 2005 to 2012, the Ministry of Health and PHYSICIANS Long-Term Care worked with the Ministry of Train- ing, Colleges and Universities to increase enrol- Over the last six years, the Ministry of Health and ment in physician training programs. First-year Long-Term Care (Ministry) has spent $3.5 billion undergraduate enrolment in medical schools went through its HealthForceOntario strategy to address up by 22% and first-year postgraduate trainees by the shortages of physicians, nurses and other health 60%. In family medicine, the number of first-year 86 2013 Annual Report of the Office of the Auditor General of Ontario

postgraduate trainees went up by 67% and spe- north was twice as high. Access has been a long- cialists by 56%. In addition, the number of inter- standing issue in many rural, remote and northern national medical graduates who entered residency communities in Ontario with chronic physician training went up by 48%. As seen in Figure 2, shortages. Geographic isolation, long travel dis- between 2005 and 2012, the number of physicians tances, low population densities and inclement increased by 18%, or about 4,100. At the same weather conditions are just some of the challenges time, the number of family doctors per 100,000 to providing health care in these areas. people went from 84.9 to 91, and specialists from A 2011 Canadian Institute for Health Informa- 92.9 to 104.3. The total number of doctors per tion report showed that 95% of physicians in 100,000 people went from 177.8 to 195.3. Accord- Ontario practised in urban areas while the remain- ing to the Canadian Institute for Health Informa- ing 5% practised in rural areas. This number falls tion, the number of specialists per 100,000 people short of urban-rural population distribution in in Ontario in 2011 was in line with the Canadian Ontario: according to Statistics Canada’s 2011 average, while the number of family physicians per census, 86% of the population lived in urban areas 100,000 people in Ontario was about 10% below while 14% lived in rural areas. To help assess the average. the accessibility of health care in rural areas, the Ministry uses the Rurality Index of Ontario (RIO), developed by the Ontario Medical Association. The Sub-optimal Distribution of Physicians in RIO incorporates data on population and physicians Ontario practising in rural and northern areas, including Despite the overall increase in primary health-care large urban centres in the north. The RIO indicates providers and specialists, access to health care is that in 2011, 8.1% of physicians in Ontario prac- still a problem for some Ontarians. According to the tised in these areas, which contained 11.6% of the Ministry, based on data collected between October province’s population. 2012 and March 2013, 6% of Ontarians lacked a Although the Ministry acknowledged that family physician. Although more recent regional physician distribution across Ontario was still not

Chapter 3 • VFM Section 3.02 data is not available, in 2010 the percentage in the optimal, it cited factors that could account for the

Figure 2: Increase in the Number of Physicians and Physician Trainees in Ontario, 2005–2012 Source of data: Ministry of Health and Long-Term Care

2005 2012 Increase (%) First-year undergraduate enrolment 797 972* 22 Medical school graduates 663 875 32 First-year postgraduate trainees 757 1,213 60 Family medicine — first-year postgraduate trainees 305 508 67 Specialty — first-year postgraduate trainees 452 705 56 International medical graduates 171 253 48 Family physicians 10,641 12,296 16 Specialists 11,636 14,086 21 Total physicians 22,277 26,382 18 Family medicine physicians per 100,000 population 84.9 91.0 7 Specialists per 100,000 population 92.9 104.3 12 Total physicians per 100,000 population 177.8 195.3 10

* Latest data available for undergraduate enrolment is from 2011. Health Human Resources 87 way physicians are distributed. For example, some training programs may attract trainees from juris- highly specialized health-care services are delivered dictions where these programs are not offered, and in tertiary care units, which means patients in some some may complete residency training in Ontario communities must travel to large urban centres to and return to their home province afterward. receive specialized care. However, others may leave Ontario because they have difficulty finding stable employment after graduation. Medical Specialties Facing Employment The Royal College of Physicians and Surgeons of Problems Canada (College) and the National Specialties Soci- On average, the province invests about $780,000 eties conducted a mini-study from July to Novem- (including $375,000 for resident salaries and ber 2010 that found physician unemployment and benefits) to educate one specialist for a four-year underemployment were common in the following undergraduate degree and up to five years of post- areas: cardiac surgery, nephrology, neurosurgery, graduate residency training. For a specialist who plastic surgery, public health and preventative enters Ontario from outside the province at the medicine, otolaryngology (ear, nose and throat postgraduate training level, this cost is $225,000. specialists) and radiation oncology. In light of these In 2011, the province spent a total of $438 million results, the College expanded its research in April on specialist education—a 63% increase since 2011 to conduct a multi-phase national study of 2005, when the amount spent was $269 million. medical specialist employment in Canada. The final However, many specialists trained in Ontario report was released in October 2013. The report do not stay and practise here. Figure 3 shows indicated that the specialties affected included that, on average, about 33% of surgical specialist orthopaedic surgery, urology, gastroenterology, graduates (including neurosurgeons and cardiac, hematology, critical care, general surgery, ophthal- orthopaedic, paediatric and general surgeons) who mology, neurosurgery, nuclear medicine, otolaryn- were funded by the Ministry left Ontario each year gology and radiation oncology. In our discussions between 2005 and 2011. with medical associations, we repeatedly heard that

Not every graduating specialist who leaves graduating specialists face employment difficulties Chapter 3 • VFM Section 3.02 Ontario does so because of employment difficul- in various surgical specialties, including many of ties. The size and breadth of Ontario postgraduate the above.

Figure 3: Surgical Specialists Leaving Ontario, 2005–2011 Source of data: Ministry of Health and Long-Term Care retention data extracted from Canadian Post–M.D. Education Registry (CAPER) Annual Census of Post-M.D. Trainees

Average for 2003 2004 2005 2006 2007 2008 2009 2003–09 Surgical specialists graduating in Ontario 120 122 120 125 111 114 147 123 Average for 2005 2006 2007 2008 2009 2010 2011 2005–11 Surgical specialists practising in Ontario two years 79 79 79 87 73 70 108 82 after graduation Surgical specialist graduates leaving Ontario (41) (43) (41) (38) (38) (44) (39) (41) % of surgical specialist graduates leaving Ontario 34 35 34 30 34 39 27 33 Surgical specialists coming into Ontario to practise 22 8 12 19 18 14 9 15 Net number of specialists leaving Ontario 19 35 29 19 20 30 30 26 88 2013 Annual Report of the Office of the Auditor General of Ontario

The College’s 2011–12 study found that about choosing specialties over general practice because 20% of new specialists and subspecialists in Ontario of the perception that specialists have more pres- (compared to 16% in Canada) could not find a job tige and higher earning potential. According to the after completing their residency training periods of Future of Medical Education in Canada Postgrad- two to five years following medical school. It also uate Project funded by Health Canada, a 50/50 noted that employability was impacted by personal balance of generalists and specialists is needed to factors and preferences such as jobs not being based provide optimal care to patients. In 2011, Ontario in new graduates’ preferred locations; hospital had about 1,700 more specialists than general- budgetary restrictions; and delayed retirements. ists. The Ministry has worked with the faculties of Such factors could result in physicians choosing to medicine to increase the number of family medi- prolong their studies or make do with contract and cine residency positions by 119% from 2003/04 temporary work, losing skills, leaving Ontario or to 2011/12. Since 2012, the Ministry and medical having to work in non-surgical practice. At the same schools moved to implement a more structured, time, there are specialist vacancies in some areas of annual planning cycle to better support decision- the province. For example, in the north, almost all making and fine-tuning of the size and composition (99%) of the $13 million spent on temporary special- of the postgraduate training system. ist coverage in 2011/12 was for covering specialist The Ministry acknowledged that graduating vacancies while recruitment was being pursued. residents faced a number of employment concerns The Ontario Medical Association also collected and that unemployment and underemployment employment data in a 2011 survey. It described were concentrated in specific specialties, particu- some of the barriers new graduates face in finding larly those requiring hospital resources. However, positions in anaesthesiology, cardiac and thoracic we found that it had not collected data from surgery, general surgery, neurosurgery, orthopaedic hospitals nor analyzed existing data to identify the surgery and vascular surgery: causes or to develop solutions. For example, the • Many cardiac surgeons were working as Ministry had not examined how hospital funding surgical assistants because they could not find might affect areas such as operating-room capacity,

Chapter 3 • VFM Section 3.02 jobs in their specialty; 34% of those graduates and how this in turn might impact employment in who were working as cardiac surgeons still some surgical specialties. It also had not collected considered themselves underemployed. data on factors such as the volume of individual • More general surgeons were choosing to do physicians’ surgical bookings, the allocation of fellowships in surgical subspecialties, which surgical bookings among physicians by level of decreased the number of general surgeons in experience, or the available operating room cap- the health-care system. acity across the system. The Ministry indicated that • There was competition among orthopaedic it continues to work with stakeholder partners such surgeons for operating room time; older as the Ontario Medical Association and the Ontario surgeons were reluctant to relinquish operat- Hospital Association to better understand how ing room time to enable new physicians to profession-specific challenges, including hospital practise. operating practices, affect physician employment • Senior vascular surgeons were working past and underemployment. the normal retirement age, which meant they The Ministry told us that once the final results of were holding on to operating room time and the College’s national study are available with juris- hospital clinic resources. dictional results, it will help inform it of the current Naturally, individual job preferences also affect status of and the multiple factors that contribute to employment patterns. More students may be Health Human Resources 89 physician unemployment and underemployment as neurosurgery, orthopaedic surgery, psychiatry, across Canada. paediatrics, obstetrics/gynaecology, geriatrics and emergency medicine, but had difficulty recruiting physicians to meet their needs. They indicated Lengthy Wait Times for Specialist Services that graduating physicians often prefer to work Ministry data that we examined for the three- in large urban centres rather than rural, remote month period from June to August 2013 showed and northern areas. Practising in non-urban areas long waits for certain surgical services, as shown presents challenges that may be quite different in Figure 4. We found that some of the procedures from those encountered during physicians’ medical with long wait times were in the same surgical training or posed by practising in an urban centre. specialties in which graduating residents faced There are differences in the level of back-up, the unemployment and underemployment. For extent of on-call work and the types of illnesses that example, for some orthopaedic surgeries, waits need treating. There may also be fewer social and following a specialist’s assessment were as long as cultural activities available and limited employment 326 days (forefoot) and 263 days (cervical disc). opportunities for physicians’ partners. Patients often wait months just to see a specialist after the family physician’s referral. Physician Initiatives Our discussions with hospitals we visited and the Ontario Medical Association suggested that A number of studies have suggested that one factor long wait times could be related to factors such affecting a physician’s practice location decision as hospital funding. Reduced capacity caused by is where he or she receives a significant portion budget constraints could mean long wait times of postgraduate medical training. For this reason, for some services; if funding is constrained then rural, remote and northern communities may have operating room hours and/or the resources who more trouble attracting physicians than urban cen- staff them could be cut, resulting in unemployment tres that are close to medical faculties and teaching and underemployment among the specialists who hospitals. Figure 5 shows a number of programs

provide these services. and initiatives the Ministry has funded to help Chapter 3 • VFM Section 3.02 The hospitals we spoke to in rural areas said those communities recruit and retain physicians. they needed resources in various specialties, such The Ministry also manages a Return of Service Program that requires international medical gradu- Figure 4: Provincial Wait Times* in Surgical ates and participants in certain other physician Specialties with High Unemployment/ postgraduate training programs to practise in Underemployment, June–August 2013 eligible communities in Ontario, generally for a Source of data: Ministry of Health and Long-Term Care period of five years. (The program is covered in a subsequent section of this report.) The Ministry, Type of Service/Procedure Wait Time (days) through separate divisions, also funds a number of Neurosurgery (overall) 134 related initiatives, such as the Northern and Rural Orthopaedic surgery (overall) 192 Recruitment and Retention Initiative, which offers Cervical disc 263 financial incentives to physicians who establish a Forefoot 326 Hip replacement 186 full-time practice in an eligible northern, rural or Knee replacement 220 remote community. Lumbar disc 251 At the time of our audit, the effectiveness of these initiatives had yet to be evaluated. Some of * Wait time is calculated as the number of days from when 9 out of 10 patients see their specialist to when they undergo surgery. the initiatives had only recently been implemented. 90 2013 Annual Report of the Office of the Auditor General of Ontario

Figure 5: Selected Ministry Initiatives for Physician Recruitment and Retention in Rural and Remote Communities Source of data: Ministry of Health and Long-Term Care

Funding Received in 2011/12 Initiative Description ($ million) Northern Ontario School of Medicine Rural-distributed, community-based medical school that seeks to recruit students coming from Northern Ontario or rural, remote, aboriginal or 12.7 francophone backgrounds (started 2005) Distributed Medical Education Organizations co-ordinate clinical teaching placements in small urban and rural communities for undergraduate and postgraduate learners 11.7 (started 1995) Northern and Rural Recruitment and Financial incentives to physicians who establish a full-time practice in an 3.4 Retention Initiative eligible community (started 2010) Hospital Academic and Operating Funding to cover academic and operating costs for hospitals affiliated Costs for hospitals affiliated with the with the Medical Education Campuses (started 2008) 3.5 medical education campuses

The 2010 evaluation of the Return of Service Physician Locum Programs Program found that it was not meeting the needs The Agency provides support for temporary phys- of most northern and remote communities. In ician or “locum” coverage in eligible communities response, the Ministry implemented changes to across the province. Locum support targets two that program, including expanding the eligible geo- specific types of need: graphic boundaries and providing other targeted • Respite coverage is an ongoing retention sup- funding for certain geographic areas that were port to physicians who work in northern and underserviced. rural communities. Because there are fewer The Northern Ontario School of Medicine, physicians in these communities, there are

Chapter 3 • VFM Section 3.02 opened in September 2005, has not been operating limited options for local replacements. Respite long enough for a meaningful evaluation, but a locum coverage provides these rural and five-year tracking study from 2010/11 to 2014/15 northern physicians with back-up when they is under way to determine the extent to which the are temporarily away on leave, continuing school’s undergraduate and postgraduate programs medical education or vacation. improved the supply and distribution of physicians • Vacancy-based coverage is intended as a in northern and rural communities. short-term solution to provide access to care In addition, as noted in Figure 5, the Ministry in areas where there are physician vacancies funds the Distributed Medical Education Program, while long-term recruitment is pursued. in which organizations co-ordinate clinical teaching As well as providing access to physician care in placements in small urban and rural communities communities with temporary physician absences or for undergraduates and postgraduates. The Min- vacancies, these programs are also meant to sup- istry informed us that it has been working with port the retention of rural and northern physicians. medical schools since autumn 2012 on an approach In the 2011/12 fiscal year, the Ministry spent a total to evaluate this initiative. of about $22 million on three physician locum pro- grams administered by the Agency. The programs include the Northern Specialist Locum Programs, the Emergency Department Health Human Resources 91

Coverage Demonstration Project and the Rural arium—about four times the average amount paid Family Medicine Locum Program. for respite coverage. In addition to fees for service, We focused our audit work on the Northern physicians receive eligible travel and accommoda- Specialist Locum Programs and the Emergency tion expense reimbursement for respite coverage, Department Coverage Demonstration Project which amounts to $241 per day on average. In some because these programs specifically targeted phys- Northern Ontario communities, physician short- ician vacancies while permanent recruitment was ages and recruitment challenges might have con- pursued. tributed to the extended use of physician locums to support ongoing access to care for patients. Our Costly Long-term Use of Northern Specialist Locum review showed that using locums has become a Programs service delivery model. Almost all (99%) of the Northern communities can access up to 26 locum $13 million spent on locum coverage in 2011/12 specialty services. The Northern Specialist Locum was for covering specialist vacancies. Programs incurred $13 million in expenditures in We looked at locum programs in other Canadian 2011/12 to provide temporary physician specialty jurisdictions and found that they generally provide coverage through two sub-programs that provide only respite coverage. The Ministry indicated that short-term coverage for specialist physician vacan- Ontario is unique in that it provides large-scale cies and ongoing respite coverage to support hospital-based services to five Northern Urban retention. Referral Centres and has a medical school based According to the Agency, the latest available in the north. The mass of critical services in the data at the time of our audit indicated that about north combined with physician vacancies and 30% of specialist positions in Northern Ontario recruitment challenges in some communities and/ were vacant. This translates to a total of about 200 or specialties creates a need for locum support that specialist vacancies, or 40,000 work days that need may not exist in other jurisdictions. The Agency’s coverage. Data that we examined for the years long-term goal is to transition to a predominantly from 2009 to 2011 showed that the specialties respite program in Ontario and eliminate use of the requiring the greatest number of locum days in

locum program as a service delivery model. The Chapter 3 • VFM Section 3.02 Northern Ontario were internal medicine, diagnos- Agency indicated that it is working to implement tic imaging, general surgery and psychiatry. Over new eligibility criteria, with full implementation by the past five fiscal years, from 2008/09 to 2012/13, 2014/15. four large northern cities—Sault Ste. Marie, We found extensive reliance on locum programs Thunder Bay, Timmins and Sudbury—received to deliver needed health-care services to some more than 80% of specialist locum coverage days. rural, remote and northern communities. For According to the Ministry, the four larger northern example, at a number of the hospitals we visited city hospitals have the highest usage because they we reviewed the on-call locum usage for a single act as critical referral centres to the smaller rural month in 2012 and found that locum coverage was northern communities where low population and as high as 94% for internal medicine at one hospital other factors would not support specialist practice. and 72% for diagnostic imaging at another hospital. Also, they provide teaching and research to the Although hospitals with specialist vacancies Northern Ontario School of Medicine. receiving coverage by the Northern Specialist In addition to fees for services or claims for Locum Programs are required to post the positions work sessionals for daily clinical work, payments to on the Agency’s HealthForceOntario jobs website, physicians for vacancy coverage averaged $1,017 they are not required to report on their progress per day for travel, accommodation and honor- in recruiting for and filling their vacancies. The 92 2013 Annual Report of the Office of the Auditor General of Ontario

Agency informed us that it was in the process of 20 hospitals in meeting the emergency needs of developing new criteria for locum coverage eligibil- communities at any one time, the amount that ity. In a phased approach over the next two fiscal has been spent is not significant overall, averaging years starting in 2013/14, the Agency will require about $200,000 per hospital. hospitals to regularly complete a form, which was being piloted at the time of our audit, to inform it Problems With Retaining Physicians in Northern of recruitment efforts that have been made to fill and Rural Areas vacancies by specialty area. The Return of Service Program is intended to pro- Emergency Department Coverage Demonstration vide greater access to physicians in smaller urban Project communities and underserviced northern and rural The Ministry and the Ontario Medical Association communities by allowing them to recruit from a designed the Emergency Department Coverage pool of physicians. The Ministry funds postgraduate Demonstration Project to be a measure of last resort training and assessment for international medical for hospitals to prevent unplanned emergency graduates and other physicians seeking to qualify to department closures due to physician unavailabil- practise in Ontario in return for a commitment from ity that otherwise would result in patients being them to provide services for a period of time (usu- unable to access critical emergency services in their ally five years) in an Ontario community (except own community. the cities of Ottawa and Toronto and adjacent About 50 of 164 emergency departments across municipalities). As of August 2013, the Ministry Ontario have used the project’s locum services since had active return of service agreements with 550 it started in 2006. According to the Agency, about physicians who had completed their training. 20 hospitals use this resource at any one time, on According to the Ministry, before 2010 under the average. The Agency’s total expenditures on the Underserviced Area Program, physicians could only project for the 2011/12 fiscal year were approxi- complete their return of service commitment in a mately $4 million. community designated as underserviced. Origin- We found at the time of our audit that of those ally, the Underserviced Area Program was focused Chapter 3 • VFM Section 3.02 hospitals using the locum services since before on northern and rural communities. However, over January 2008, one-third had been continuously time more communities outside of the north were using the services from as early as 2007. One designated as underserviced to attract return of hospital had been using them from 2006 up to the service physicians, with the result that designated time of our audit. These hospitals received a total of communities close to major population centres in about 9,000 hours of locum coverage in 2011/12. the south outnumbered those in Northern Ontario, We reviewed a sample of the monthly emergency which made it even more difficult for northern department schedules between local and locum and most rural communities to compete for phys- doctors at one of the hospitals we visited and found ician resources. From 2010 onward the Ministry that more than half of the emergency department attempted to improve access to care by expanding shifts had been covered by locum physicians. eligible practice areas to include all but the cities of These findings imply that some hospitals Ottawa and Toronto and adjacent municipalities. facing physician shortages and recruitment chal- These areas were excluded so they would not be lenges need long-term use of the locum support able to out-compete smaller urban, rural, northern to maintain access to services while permanent and remote communities for physician resources. recruitment is pursued. However, given that In our discussions with staff at the rural and approximately $4 million was able to cover about northern hospitals we visited, we heard repeatedly Health Human Resources 93 that they were in need of more physicians. They Some of the unsuccessful candidates may go on told us that it was challenging to keep return of to consider a field of practice outside their original service physicians on after their five-year commit- profession, such as physiotherapy, dietetics and ment was up. The Ministry does not keep track of chiropractics. In the 2012/13 fiscal year, the Agency international medical graduate physicians who stay helped 27 international medical graduates who had on and practise in eligible communities after their been unsuccessful in obtaining residencies to tran- return of service commitments have been met, but sition to other careers in the health sector. information we obtained from one of the hospitals Although its current primary mandate does not we visited showed that only one of three physicians include helping international medical graduates stayed on after completing their service commit- find employment in alternative fields, the Access ments. The other hospitals we visited did not keep Centre informed us that it plans to further develop information on retention. services to assist internationally educated health The Ministry has not evaluated the Return of professionals who are unable to practise in their Service Program to assess its effectiveness since it field to transition to an alternative health career was redesigned in 2010. There are currently no per- consistent with their education and experience. At formance measures or metrics to measure the pro- the time of our audit, the first module of an Alterna- gram’s success. The Ministry has only a quarterly tive Career Toolkit was available on the Access update on the number of international medical Centre’s website. graduate physicians in the program, broken down by LHIN and specialty at a snapshot date. RECOMMENDATION 1

To better meet the health-care needs of Ontar- Alternative Health Careers for International ians, the Ministry of Health and Long-Term Care, Medical Graduates in conjunction with the HealthForceOntario Marketing and Recruitment Agency, should: The Access Centre is a unit within the Agency to compare the existing mix and distribution help physicians and all other regulated health • of physicians across the province to patient

professions who want to practise in Ontario with Chapter 3 • VFM Section 3.02 needs and consider what measures it can the licensing, certification and regulatory process. take to reduce any service gaps; Many international medical graduates use the assess how various factors, including hospi- Access Centre for assistance with navigating the • tal funding and capacity and health-delivery system and competing for Canadian Resident models, affect patients’ access to needed Matching Service (CaRMS) residency positions. services and physician employment, and CaRMS is a national not-for-profit organization develop cost-effective solutions where con- that provides an electronic matching service for cerns are identified; postgraduate medical training in Canada. It enables continue to work with medical schools and applicants to decide where they wish to train in • associations to encourage more medical Canada and enables programs to indicate which students to select fields of study and geo- applicants they wish to enrol in postgraduate med- graphic areas in which to practise that are in ical training. In the 2011/12 fiscal year, approxi- demand; and mately 2,100 international medical graduates were assess the effectiveness of its various phys- registered with the Agency’s Access Centre, but only • ician initiatives in meeting the health-care 173 obtained residencies in Canada; of those, 156 needs of underserved areas. obtained residencies in Ontario. 94 2013 Annual Report of the Office of the Auditor General of Ontario

practitioners (NPs)—in Ontario to improve access MINISTRY RESPONSE to care. RNs usually obtain their education through The Ministry welcomes this recommendation a four-year university degree. RPNs usually obtain as it is consistent with and supports work that it education through a college program. A nurse prac- has undertaken. Working with the Agency and titioner is a registered nurse who has acquired the other key health-system partners, the Ministry’s knowledge base, decision-making skills and clinical HealthForceOntario strategy strives to continu- competencies for a practice that extends beyond ally advance evidence-informed health human that of an RN. resource planning that is responsive to the From 2006 to 2012, the number of nurses health-care needs of the people of Ontario. in Ontario increased by 10%, from 138,583 to The Ministry will continue working with 153,073. The number of nurses employed per partners to better understand these factors 100,000 people in Ontario increased by 3.5%, from and their impact on access to health care and 1,106 in 2006 to 1,145 in 2012. Figure 6 provides a health human resource requirements, including detailed breakdown of the increase. physicians. We found that the Agency played a small role in The Ministry is actively engaged with the nursing sector. Agency regional advisers who medical schools and professional associations operate from various LHINs across Ontario focused to identify and promote a stable supply of phys- mainly on physicians, and very few Agency advisers icians across the continuum of specialties with worked at the Access Centre to counsel internation- a focus on those that are required for harder-to- ally trained nurses. service patient populations or geographic areas. Although the Nursing Strategy has successfully The Ministry is committed to ongoing assess- increased the number of nurses in Ontario, we ment of the effectiveness of initiatives launched found that improvements were generally needed in to address specific needs, and looks forward ministry oversight and assessment of the effective- to the outcome of such evaluations to inform ness of its nursing programs and initiatives. future planning.

Chapter 3 • VFM Section 3.02 Nursing Initiatives NURSES According to data from the College of Nurses of Ontario, 66.7% of members employed in nursing A comprehensive strategy on nursing was an in this province said they worked full-time in 2012. integral part of the HealthForceOntario strategy The rest were categorized as part-time or casual. launched by the Ministry in May 2006. The Nursing Figure 7 breaks down the working status for nurses Strategy is a collection of programs and initiatives in Ontario from 2008 to 2012, and Figure 8 shows intended to achieve the right number and mix of employment by practice sector for full-time, part- nurses in Ontario now and in the future. It addresses time and casual employment, where the nurse does issues such as work environments, full-time employ- not have a set number of hours and is called in to ment, and recruitment and retention of nurses. work as needed. At the time the Nursing Strategy was launched, The Ministry has implemented several nursing the province was experiencing a shortage of nurses, initiatives aimed at stabilizing the workforce and and further shortages were anticipated with increasing full-time opportunities and retention of imminent retirements. The Ministry’s focus was on nurses across Ontario. The Health Human Resour- increasing the number of nurses—registered nurses ces Strategy Division spent $151 million on nurs- (RNs), registered practical nurses (RPNs) and nurse ing initiatives in 2012/13. This amount does not Health Human Resources 95

Figure 6: Number of Nurses in Ontario, 2006–2012* Source of data: College of Nurses of Ontario

Type of Nurse 2006 2012 Increase Increase (%) Registered nurse 108,185 112,194 4,009 4 Registered practical nurse 29,706 38,859 9,153 31 Nurse practitioner 692 2,020 1,328 192 Total 138,583 153,073 14,490 10

* The data provides a “point-in-time” snapshot of the available labour supply of nurses (at the end of the prior year or beginning of the stated year).

Figure 7: Overall Working Status of Nurses in Ontario, 2008–2012* Source of data: College of Nurses of Ontario

2008 2009 2010 2011 2012 Working Status # % # % # % # % # % Full-time 75,649 62.9 78,694 63.9 80,356 63.8 83,972 66.4 85,010 66.7 Part-time 34,820 29.0 34,371 27. 9 34,939 27. 8 32,316 25.6 32,712 25.6 Casual 9,796 8.1 10,026 8.2 10,549 8.4 10,117 8.0 9,889 7. 7 Total 120,265 100.0 123,091 100.0 125,844 100.0 126,405 100.0 127,611 100.0

* The data provides a “point-in-time” snapshot of the working status of nurses (at the end of the prior year or beginning of the stated year).

Figure 8: Employed Nurses1 by Practice Sector, 20102 Source of data: College of Nurses of Ontario

Registered Registered Nurses Practical Nurses Total

Working Status # % # % # % Chapter 3 • VFM Section 3.02 Full-time Hospital 41,030 44 7,544 25 48,574 40 Long-term care 4,851 5 6,690 22 11,541 9 Community 10,947 12 2,636 9 13,583 11 Other & not specified 4,656 5 771 2 5,427 4 Part-time Hospital 15,576 16 4,669 15 20,245 16 Long-term care 2,201 2 3,597 12 5,798 5 Community 4,757 5 1,409 4 6,166 5 Other & not specified 2,052 2 468 1 2,520 2 Casual Hospital 4,352 5 1,168 4 5,520 4 Long-term care 552 1 804 3 1,356 1 Community 2,134 2 506 2 2,640 2 Other & not specified 808 1 180 1 988 1 Total 93,916 100 30,442 100 124,358 100

1. Not including nurse practitioners. 2. Most recent year that data was available. 96 2013 Annual Report of the Office of the Auditor General of Ontario

include funds spent by other program areas within Specific data was not available to determine the the Ministry for nursing initiatives. number of full-time, part-time or casual positions We examine a few of the Ministry’s more signifi- that the funded organizations had created through cant initiatives in the following sections. the 9,000 Nurses Commitment.

Meeting the 9,000 Nurses Commitment Inadequate Assessment of the Nursing Graduate In 2007, the government committed to hiring 9,000 Guarantee Program’s Effectiveness more nurses over a four-year period. It also com- Announced in February 2007, the Nursing Graduate mitted to a goal of 70% of nurses working full-time. Guarantee Program’s objective was to support new However, in fall 2008, the province indicated that it Ontario nursing graduates (both RNs and RPNs) would take longer to achieve the goal of 9,000 new in finding full-time employment immediately upon hires. From 2008/09 to 2011/12, $309 million was graduation. Some of the program’s other objectives dedicated to this initiative. Our review of the initia- include facilitating recruitment in all nursing sec- tive showed that: tors; transforming employer practices to make more • From year-end 2007 (reported as 2008 by the full-time nursing positions available; and increasing College of Nurses of Ontario), the number the total supply of nurses by providing full-time of nurses in the province increased by more employment to nurses who may have otherwise than 7,300 over four years. At the time of our sought work in other jurisdictions or professions. audit, CNO nursing data was not available for The program provides funding for temporary 2013 (which would represent 2012 figures), full-time, above-staffing-complement positions for but it appeared likely that the goal of hiring 26 weeks with the expectation that these bridg- 9,000 new nurses would be achieved by the ing positions will lead to permanent full-time end of 2012. employment. Employers must commit to funding • The province was slightly under its goal of an additional six-week full-time position for the having 70% of nurses working on a full-time new graduate nurse if he or she is not bridged to basis. As of the year ended 2011, reported by permanent full-time employment by the end of Chapter 3 • VFM Section 3.02 CNO as 2012 numbers, 66.7% of nurses were the 26 weeks. The program is open to employers working full-time. From the year ended 2007 in all health sectors (hospitals, long-term-care (reported as 2008) to the year ended 2011 homes and community care organizations); all (reported as 2012), there were almost 9,400 Ontario-educated new graduate nurses are eligible more nurses working full-time, representing a to take part as long as they register on the Agency- 12% increase. administered new graduate guarantee job website We reviewed five Nursing Secretariat programs and accept a job offer within six months of complet- that reported creating 1,316 nursing positions. Two ing their studies. In 2011, there were approximately of the programs we reviewed had been in place 1,200 potential employers. In the 2011/12 fiscal since 2008/09 and the other three had been imple- year, about $66 million of ministry funding was mented in 2011/12. The Ministry indicated that provided to about 210 participating health-care 1,125 of the 1,316 new positions (85.4%) had been organizations (representing an 18% employer par- filled by March 2013. We also noted that the 1,316 ticipation rate), which employed 2,235 new gradu- nursing positions created were not all full-time, but ate nurses under the program. included part-time and casual positions. The Nurs- According to the Ministry’s January 2011 ing Secretariat did not have detailed information Guidelines for Participation in the Nursing Gradu- to determine the type of employment obtained by ate Guarantee for New Graduate Nurses, program some of the positions funded through the Division. funding is provided with the expectation that the Health Human Resources 97 bridging positions offered by organizations will 24% of its graduates had transitioned to full- lead to permanent full-time jobs. In the transfer- time employment and 59% to part-time. payment agreements that they enter into with the • In 2007/08, another organization reported Ministry, participating organizations commit to that 85% of its graduates had transitioned making their “best effort” to place a new graduate to full-time employment and 2% went to in a permanent full-time position after he or she has part-time. By 2011/12, the same organization worked with the employer for at least 12 weeks, but reported that 19% of its graduates had tran- the Ministry does not assess program data against sitioned to full-time employment and 47% to established program targets to determine how well part-time. organizations are doing with bridging participants • In 2011/12, an organization reported that to permanent full-time employment. We conducted 40% of its nursing graduates had voluntarily our own analysis of 2010/11 and 2011/12 program left the program early. Sixty percent of them data and found that only about one-quarter of left because they found employment else- participants had been transitioned to permanent where, while no explanations were given for full-time employment after the six-month period the others who left. in 2010/11 and one-third in 2011/12; about an The Ministry has allocated funds ranging from additional one-third of participants had been tran- about $86 million to almost $100 million per year sitioned to part-time employment in both 2010/11 for the program since it began in 2007/08. We and 2011/12. found that for each of the past five fiscal years We also found that the Ministry had not (from 2007/08 to 2011/12), program expenditures reviewed employment trends at participating were less than the amount of funds allocated. As health-care organizations to determine whether shown in Figure 9, other than for 2009/10, the employers were making their best efforts to transi- amounts of unspent funds varied considerably, tion program participants to full-time permanent ranging from $17.2 million (20%) to $33.6 million employment. We reviewed a sample of health-care (37%) of total funds allocated for the year; they organizations that received funding from the totalled $105.7 million over the five-year period.

program in 2007/08 to 2011/12 to look for employ- Figure 9 also shows how declining program Chapter 3 • VFM Section 3.02 ment patterns and found the following: expenditures are related to a one-third drop in • In 2011/12, one organization reported that program participation by eligible graduate nurses. 15% of its graduates had transitioned to Participation was as high as 62% in 2007/08 but full-time employment while 65% went to part- declined to 35% in 2011/12. When we asked time. In 2007/08, the same employer reported the Ministry about the reasons for this decline,

Figure 9: Nursing Graduate Guarantee Program Funding and Participation Source of data: Ministry of Health and Long-Term Care

Allocated Actual Unused # of New Nurse % of New Nurse Amount Expenditure Portion # of New Nurse Graduates Graduates Fiscal Year ($ million) ($ million) ($ million) Graduates Participating Participating 2007/08 88.9 71.7 17. 2 4,300 2,660 62 2008/09 94.2 72.3 21.9 4,902 2,825 58 2009/10 85.8 85.5 0.3 5,139 2,598 51 2010/11 87. 5 54.8 32.7 5,555 1,804 32 2011/12 99.6 66.0 33.6 6,386 2,237 35 Total 456.0 350.3 105.7 98 2013 Annual Report of the Office of the Auditor General of Ontario

it indicated that not every nursing graduate will the people of Ontario. Patients at these clinics see secure a job through this program. Some move a nurse practitioner (NP) as their primary health into positions in the same organization where they care provider, consulting with a physician only completed their clinical placement, but outside the when needed. The first clinic opened in Sudbury in program, while others may decide to continue their 2007 and served as a pilot project for the initiative. studies or take time off. Nevertheless, the signifi- In November 2007, the Ministry announced that cant decline in graduate participation in this pro- it would establish 25 more NP-led clinics across gram raises questions that need to be considered in the province. In 2011/12, the 26 clinics received evaluating the program’s effectiveness. $29 million in ministry funding for development An external party has evaluated the Nursing and implementation. Graduate Guarantee Program annually since Each clinic can have up to four NPs who operate 2007/08 using ministry data supplemented by a in collaboration with an interprofessional team survey of participants and employers. The 2011/12 (such as RNs, dietitians and social workers) to evaluation resulted in nine recommendations that provide increased access to primary health care. To included, for example, continuing to promote the achieve this, the clinic may, for example, focus on participation of long-term-care and community providing: employers in the program; supporting the partici- • family health care for people previously with- pation of northern, rural and small organizations out a primary care provider; in the program; and examining the differences in • chronic disease detection and management, employment status, retention and transition into such as obesity programs, smoking cessation the nursing profession across sectors. The Min- and cancer screening; and istry indicated that it was working to address the • faster access to care through house calls and recommendations. same-day or next-day appointments and However, the evaluator has not assessed the extended hours. overall effectiveness of the program. For example, We looked at information from a sample of clin- it has not reported on the total percentage of nurse ics to determine whether the majority of patients

Chapter 3 • VFM Section 3.02 graduates who have transitioned to permanent full- had indeed previously been without primary care time employment through the program. providers. We found that only two of the five clinics The Ministry has set performance targets for we contacted had taken steps to document whether the number of temporary full-time positions to be their patients had family physicians. Two of the bridged—the number of participants—but has not other clinics informed us that patients were asked set an outcome-based performance target— the if they had a family physician. The remaining clinic number of participants to achieve permanent full- did not begin collecting this information until April time positions—for the Nursing Graduate Guaran- 2013, when it began having patients complete an tee Program. In the program’s first year (2007/08), intake form that included a question about family the target was 3,000 positions to be bridged; by physicians. 2011/12 the target had decreased to 2,500 pos- At the time of our audit, the Ministry‘s overall itions. The Ministry met its targets in only one of target was to have a total of 40,000 registered the five years—2008/09. patients (who do not have regular access to a family health-care provider) at all 25 clinics. As of January 2013, the 24 NP-led clinics that were open (one NP- Inadequate Assessment of Nurse Practitioner– led clinic is targeted to open late in the 2013/14 fis- led Clinics cal year) reported having about 33,000 registered The purpose of nurse practitioner–led clinics is to patients, or 82% of the program target. Given that provide increased access to primary health care to Health Human Resources 99 the clinics are a new model of primary health-care monitor the nurse practitioner-led clinics delivery and that many of them had been open less • more closely to ensure that they are meeting than two years, not all clinics are at full capacity, as program requirements and achieving their it takes time to establish a patient roster. patient targets and program objectives. Clinic budgets setting out operational and one- time costs are approved annually by the Ministry. MINISTRY RESPONSE Clinics are required to submit quarterly and annual financial and performance information and statis- The Ministry acknowledges the recommenda- tics to the Ministry for review as well as an audited tion and comments regarding the Nursing statement of revenues and expenditures for the Graduate Guarantee (NGG) program and the year. Our review of ministry documentation from a nurse practitioner–led clinics. sample of clinics indicated that the Ministry gener- Recognizing the importance of monitoring ally followed up with clinics on matters during and evaluating the NGG program, work has their development phase. Ministry staff informed been under way since early 2012 to enhance its us that they reviewed clinic operating costs, com- online management. A new tool was launched paring actual operating expenditures to those set in April 2013 that allows the Ministry to collect out in the clinic’s budget, approving funding levels and analyze information that was previously and discussing any variances with the appropriate unavailable, including monitoring aspects of clinic staff. NGG participation rates by both nursing gradu- The Ministry’s performance measure for ates and employers, as well as employment this program is the establishment of 25 nurse outcomes. practitioner–led clinics, which has been met. The annual program evaluation will con- However, this measure does not assess whether the tinue and will be enhanced to examine the clinics are effectively meeting program goals. At variations in participation rates. the time of our audit, the Ministry had evaluated The Ministry will also implement a targeted only one clinic (Sudbury) for its effectiveness in communication strategy to promote increased

meeting program objectives. uptake of NGG participants across health-care Chapter 3 • VFM Section 3.02 sectors, with particular attention to the com- RECOMMENDATION 2 munity—for example, in home care, long-term care, primary care and public health sectors. To provide an appropriate level of nursing ser- The recommendation about nurse vices and thereby improve access to care across practitioner–led clinics is timely as the clinics the health sector, the Ministry of Health and transition from startup to full operations. Long-Term Care should: The Ministry will review how it can apply monitor nursing employment trends and • greater oversight to these clinics and ensure assess the outcome of its nursing initiatives accountability for outcomes including in transitioning graduating nurses to perma- achievement of patient targets and program nent full-time employment; objectives. The Ministry will continue to assess the reasons for declining participa- • take timely, appropriate action when non- tion rates of nurse graduates in its Nursing compliance with agreements is identified. Graduate Guarantee Program, and take steps to improve program effectiveness, including encouraging participation in the program across sectors; and 100 2013 Annual Report of the Office of the Auditor General of Ontario

Untimely Recovery of Unspent Funds recovering from the clinics. The Ministry also informed us that it is working to complete the Organizations that receive transfer payment fund- reconciliations that remained for 2009/10, ing from the Ministry are required to submit annual 2010/11 and 2011/12. financial statements. Subsequent to year-end, the In October 2012, the Ministry of Finance’s Ministry reviews year-end financial statements to internal audit department issued a report on assess whether it is owed any surplus funds. Any oversight and monitoring of transfer payment payables to the Ministry are recovered from the recipients that made observations similar to ours. transfer payment recipients. It recommended establishing outcome-based For the nursing initiatives we examined in our performance measures and guidelines for review audit, we found that some related transfer payment and analysis of financial reporting to increase agreements did not set out time requirements for consistency and enhance efficiency. The Ministry submitting or completing the financial reconcilia- informed us that the focus of the strategy to date tion. For some organizations that received fund- has been to establish capacity in the province by ing through the 9,000 Nurses Commitment, the increasing the supply of providers, which relates information provided was lumped into broader to an output-based measure. It has plans to estab- programs instead of being broken down by initia- lish outcome-based performance measures and tive. For example: guidelines where appropriate. • Our review of a sample of the organizations that received 9,000 Nurses Commitment RECOMMENDATION 3 funding found 36 programs for which many of the year-end reconciliations had not been To improve financial oversight of funded completed on a timely basis. organizations and recover unspent funds, the • At the completion of our audit, the Ministry Ministry of Health and Long-Term Care should had completed reconciliations up to only perform timely reviews of relevant financial 2009/10 for the Nursing Graduate Guarantee statements. Program. Pending completion of the other Chapter 3 • VFM Section 3.02 years, the total amount of recoveries identi- MINISTRY RESPONSE fied to date was at least $7.3 million. The The Ministry is committed to enhancing finan- Ministry was still in the process of recovering cial management systems to ensure optimal the funds. use of financial resources. In 2013/14, the We looked at a sample of five nurse • Ministry is implementing processes that will practitioner–led clinics and identified total ensure emphasis is placed on in-year review operating funds of about $1.3 million owed to and analysis of financial reports submitted by the Ministry for the 2011/12 fiscal year. This funding recipients to support timely recovery or amount represented about 30% of the total payment adjustments. funding provided to these clinics. In addition, For the HealthForceOntario strategy, the the audited statements of two of the clinics Ministry has made significant progress over the showed about $360,000 owing to the Ministry past two fiscal years through dedicated efforts in for the 2010/11 fiscal year. Shortly after we the area of reconciliations. Its goal is to have all completed our audit fieldwork, the Ministry reconciliations current by March 31, 2014. indicated that it had identified approximately $3.4 million in recoveries related to 2009/10 and 2010/11, which it was in the process of Health Human Resources 101

HEALTH HUMAN RESOURCE able quantifiable data on physician productivity. FORECASTING MODELS Physician productivity is an important component, and even small improvements in productivity can Forecasting models are recognized as one import- have a significant impact on the number of human ant component of evidence-based health human resources required in the system. resource planning. Good information and proper During development, the consultant who built health human resources planning are essential if the model defined productivity as the number of the Ministry and health system stakeholders are to patients seen in the physician’s practice for a given work together to determine an appropriate number period of time, noting several factors that affect and mix of health professionals to meet the health physician productivity, including: needs of Ontarians across the province. • information and communication technology (electronic health records, telemedicine); Better Physician Forecasting Data Needed • health system change (new or different primary health-care models, such as Family Some Canadian jurisdictions are engaged in Health Teams); physician forecasting and modelling. For example, non-physician clinicians (other health-care Alberta and Nova Scotia have developed needs- • providers working with physicians, such as based physician forecasting models similar to NPs); and Ontario’s to plan their physician supply require- funding and compensation models. ments. This type of planning generally involves • Even for these four factors, specific quantifiable estimating the health services required to meet the data for only one category of non-physician clin- needs of the population and then translating them icians—NPs—was available and incorporated into into the number and type of physicians required to the model. deliver those services. Also, while physician human resource forecast- Historically, physician human resources plan- ing reflects factors such as workforce demographics ning in Ontario has been supply- or utilization- and changing population health needs, it is also sig- based; however, this method does not provide a

nificantly affected by broader economic, social and Chapter 3 • VFM Section 3.02 complete picture because it does not consider the health-system trends, as well as health technology population’s health needs. In 2009, the Ministry advancements. Many of these other factors—the partnered with the Ontario Medical Association availability of diagnostic and laboratory equipment, and used a tendering process to select an external operating-room time and space to perform surger- party to develop a new, needs-based model. The ies, physician preferences for certain specialties and new model works by examining the population’s practice locations, and employment opportunities health needs and translating them into needs for across the province—can affect access to and physician services, then comparing these needs to delivery of health-care services, but they are not the supply of physician services currently available. easily incorporated into the model. Although the Service gaps are quantified and converted into the needs-based simulation model does make it pos- number of physicians required to meet the needs. sible to test “what-if” simulations that may help to Ontario’s physician forecasting model is a assess the impact of some of these factors, it cannot positive step in determining physician workforce incorporate all of them. requirements. However, the model is hampered by For these reasons, the results obtained from the limited reliability and availability of data. We Ontario’s physician forecasting model can be found that Ontario’s model does not account for considered only one of many tools and pieces of some important variables because of a lack of avail- evidence available to support policy formulation. 102 2013 Annual Report of the Office of the Auditor General of Ontario

The model’s limitations make it difficult to use in conduct a simulation model analysis that would planning for the optimal number, mix and distri- accurately estimate the requirements for this nurs- bution of physicians with appropriate funding, ing group. training and deployment across the province. During the course of our audit, the Ministry was in the process of working with the Ministry of Training, Colleges and Universities to develop Lack of Forecasting of Demand for Nursing a supply-based nursing simulation model for RNs Services and RPNs, with results expected to be available by In 2008, the Ministry also engaged an external late 2013. consultant to develop a needs-based nursing The Ministry advised us that the purpose of model that would be able to project the gap the supply-based simulation model is to help between the need for and the actual supply of the government plan properly for future nursing RNs, RPNs and NPs in Ontario for each year over a education positions and formulate nursing recruit- 10-year period. Separate simulation models to test ment and retention policies. The initial model various health human resources policy scenarios is to provide projections for the future supply of were also developed for both RNs and RPNs. nurses (RNs and RPNs) at the regional level and Although the model cost about $435,000 to will enable planners to test the potential impact of develop, it was initially not used to specifically policy changes on the supply, such as changes to inform any nursing policies because of concerns enrolment numbers and percentages of full-time about the accuracy of its predictions. nurses, the introduction of incentives for working In 2012, as part of its ongoing evidence in rural and remote areas, shifting distribution development work, the Ministry found that the of nurses by employment sector, recruitment model had understated data for first-year enrol- and retention rates and other attrition factors. ment of RNs in 2007 and incorrectly assumed However, a supply-based model cannot assess that all RNs provided the same rate of direct whether the supply is appropriate because it does patient care regardless of their years of experi- not take into account the population’s need for

Chapter 3 • VFM Section 3.02 ence; in practice, younger RNs just entering the nurses. In addition, the new supply-based model profession may be providing different amounts will not include NPs because some historical data of direct patient care than RNs near the end of specific to them is not available from the College their careers. The model had also applied an of Nurses of Ontario. The Ministry informed us estimated percentage of total RNs providing direct that the initial focus is on the future supply of patient care based on previous outdated data. nurses, which is important work that needs to be The Ministry corrected the first-year enrolment completed to support future needs-based model- data and updated the direct patient care data and ling considerations. completed two simulations. However, assump- To get better information to inform future tions regarding attrition, retirement and workload policy work, the Ministry entered into an agree- were not updated, and no other data reviews were ment with a large hospital in late 2012 to create a conducted. one-time snapshot of the current supply, distribu- The model has other limitations. For example, tion and predicted shortfall or surplus of RNs, it forecasts the gap between supply and need at RPNs and NPs working in selected primary health- the provincial level but not at a regional level. care organizations, long-term-care homes and At the time of model development, data on the acute-care hospitals across the province for the patterns of service delivery for NPs was not avail- next three months. Each surveyed health organ- able, and therefore the consultant was unable to ization is to report on current staffing, vacancies, Health Human Resources 103 details about leaves of absence and predictions of MINISTRY RESPONSE short-term staffing changes. There will also be a comprehensive analysis of the overall staffing situa- The Ministry agrees with the audit findings that tion in each area. This data will be used to identify its physician forecasting model is a positive step which organizations and geographical areas are in determining physician workforce require- having difficulty in recruiting and retaining nurses ments. Similarly, the Ministry concurs with the in Ontario. audit observations regarding forecasting models as one of many types of tools that are required RECOMMENDATION 4 to support health human resource planning. As such, the Ministry is actively engaged To provide reasonable and reliable forecasts with the health sector to improve evidence for of the requirements for physicians and nurses decision-making. Over the summer of 2013 and to better ensure effective health human the Ministry has been meeting with the field resources planning, the Ministry of Health and regarding the current health human resource Long-Term Care should: environment, including how we continue to conduct assessments of employment trends, • evolve and develop evidence. This work will the supply and projected needs for health contribute important information to inform services, and the associated health workforce future HealthForceOntario work. requirements to best meet those needs cost- The HealthForceOntario strategy will effectively; and continue to provide innovative health human for physicians and nurses, further refine its • resource solutions to meet patient-care needs. forecasting models and their capabilities Evidence for decision-making will continue to to assess the impact of various factors on be a key aspect of the strategy and the Ministry service-provider productivity. will seek to enhance and expand tools, including forecasting models, to improve planning. Chapter 3 • VFM Section 3.02