2013 Annual Report

ARCHIVED - Archiving Content ARCHIVÉE - Contenu archivé

Archived Content Contenu archivé

Information identified as archived is provided for L’information dont il est indiqué qu’elle est archivée reference, research or recordkeeping purposes. It est fournie à des fins de référence, de recherche is not subject to the Government of Canada Web ou de tenue de documents. Elle n’est pas Standards and has not been altered or updated assujettie aux normes Web du gouvernement du since it was archived. Please contact us to request Canada et elle n’a pas été modifiée ou mise à jour a format other than those available. depuis son archivage. Pour obtenir cette information dans un autre format, veuillez communiquer avec nous.

This document is archival in nature and is intended Le présent document a une valeur archivistique et for those who wish to consult archival documents fait partie des documents d’archives rendus made available from the collection of Public Safety disponibles par Sécurité publique Canada à ceux Canada. qui souhaitent consulter ces documents issus de sa collection. Some of these documents are available in only one official language. Translation, to be provided Certains de ces documents ne sont disponibles by Public Safety Canada, is available upon que dans une langue officielle. Sécurité publique request. Canada fournira une traduction sur demande.

2013 Annual Report

Ofﬁce of the Auditor General of Ontario

To the Honourable Speaker of the Legislative Assembly

In my capacity as the Auditor General, I am pleased to submit to you the 2013 Annual Report of the Office of the Auditor General of Ontario to lay before the Assembly in accordance with the provisions of section 12 of the Auditor General Act.

Bonnie Lysyk Auditor General

Fall 2013 Copies of this report are available for $9.00 from Publications Ontario: (416) 326-5300 or toll-free long distance 1-800-668-9938. An electronic version of this report is available on the Internet at www.auditor.on.ca

Ce document est également disponible en français.

ISSN 1719-2609 (Print) Cover photograph credits: ISBN 978-1-4606-3017-4 (Print, 2013 ed.) top right: © iStockphoto.com/Renphoto middle left: © iStockphoto.com/Photawa ISSN 1911-7078 (Online) bottom right: © iStockphoto.com/SolStock ISBN 978-1-4606-3018-1 (PDF, 2013 ed.) Table of Contents

Reflections 5 Chapter 1 Summaries of Value-for-money Audits 12 Chapter 2 Public Accounts of the Province 24 Chapter 3 Reports on Value-for-money Audits 51 Section 3.01 Autism Services and Supports for Children 52 Section 3.02 Health Human Resources 82 Section 3.03 Healthy Schools Strategy 104 Section 3.04 Land Ambulance Services 121 Section 3.05 Ontario Power Generation Human Resources 152 Section 3.06 Private Schools 180 Section 3.07 Provincial Parks 202 Section 3.08 Rehabilitation Services at Hospitals 221 Section 3.09 ServiceOntario 247 Section 3.10 Violence Against Women 269 Chapter 4 Follow-up to 2011 Value-for-money Audits and Reviews 293 Section 4.01 Auto Insurance Regulatory Oversight 294 Section 4.02 Electricity Sector—Regulatory Oversight 302 Section 4.03 Electricity Sector—Renewable Energy Initiatives 308 Section 4.04 Electricity Sector—Stranded Debt 318 Section 4.05 Forest Management Program 321 Section 4.06 Funding Alternatives for Family Physicians 332 Section 4.07 Funding Alternatives for Specialist Physicians 339 Section 4.08 LCBO New Product Procurement 343 Section 4.09 Legal Aid Ontario 347 Section 4.10 Office of the Children’s Lawyer 354 Section 4.11 Ontario Trillium Foundation 361 Section 4.12 Private Career Colleges 368 Section 4.13 Student Success Initiatives 376 Section 4.14 Supportive Services for People with Disabilities 383 4 2013 Annual Report of the Office of the Auditor General of Ontario

Chapter 5 Review of Government Advertising 393 Chapter 6 The Standing Committee on Public Accounts 406 Chapter 7 The Office of the Auditor General of Ontario 409 Exhibit 1 Agencies of the Crown 434 Exhibit 2 Crown-controlled Corporations 435 Exhibit 3 Organizations in the Broader Public Sector 437 Exhibit 4 Treasury Board Orders 441 Reflections

Introduction Value-for-money Audits

I was appointed by the Legislature as Auditor There were several common threads in this year’s General of Ontario effective September 3, 2013. value-for-money audits, including the importance This Annual Report is the result of the dedicated of good information for sound decision-making, work by staff in the Office of the Auditor General of and the fact that data to help fully assess program Ontario, with input from employees of the public effectiveness is often unavailable. As well, we noted and broader public sectors who co-operated with instances where ministries could improve their our audit teams. We all share the same goal: to monitoring and delivery of programs. However, four make a positive difference for all Ontarians. broad themes also emerged from this year’s audits: I would especially like to recognize the work of • obtaining full value from programs focused on my predecessor, Jim McCarter, who served as the helping vulnerable people; Auditor General of Ontario for the past 10 years; • improving co-ordination for cost-effective Deputy Auditor General, Gary Peall; and the experi- service delivery; enced Directors in this Office who guided our teams • meeting public expectations; and in performing the value-for-money and financial • increasing public awareness. statement audits and follow-up work for this year’s Annual Report. In addition, I want to express my OBTAINING FULL VALUE FROM thanks to all management and staff for their profes- PROGRAMS FOCUSED ON HELPING sionalism and hard work. And to everyone in the VULNERABLE PEOPLE Office, a big thank you for your welcoming acceptance of me since my arrival in September. This year, we audited two programs that focus on I feel privileged to have the opportunity to helping vulnerable people with a variety of needs: serve as the Auditor General of Ontario, and I look autism treatment and support for children and forward to working with Members of the Legisla- youth, and help for women and their children who tive Assembly, the Standing Committee on Public have experienced violence and abuse. Autism ser- Accounts, Deputy Ministers and their staff, and vices currently being offered do not meet identified management in broader-public-sector organizations. needs. The extent of current needs for services for victims of domestic violence is not known.

5 6 2013 Annual Report of the Office of the Auditor General of Ontario

Autism Services and Supports for Children women secure more permanent housing. In all, the Ministry funds more than 200 not-for-profit agen- Autism is growing more prevalent in Ontario and cies in local communities to deliver supports and around the world. Ontario does not have a formal services to abused women and their children. provincial autism strategy. However, the Ministry Overall, we found that the Ministry did not of Children and Youth Services funds a variety of have sufficient information to properly assess the autism-specific services and supports for children effectiveness of the programs and services offered up to age 18 and their families at a cost of about to victims of violence, and therefore know whether $182 million in the 2012/13 fiscal year. Despite services were sufficient to meet the needs of abused that, more children with autism are actually women and their children. waiting for government-funded services than are receiving them. It can take 3 to 12 months to obtain a diagnosis IMPROVING CO-ORDINATION FOR of autism, depending on where in the province COST‑EFFECTIVE SERVICE DELIVERY a person lives. Due to long waiting lists, most Our audits of both the Healthy Schools Strategy children in Ontario are almost seven years old and Rehabilitation Services at Hospitals identi- when they start Intensive Behaviour Intervention fied a need for improved program co-ordination. (IBI). Research shows that children who start IBI In the case of the Healthy Schools Strategy, this before age 4 have better outcomes than those who will involve the Ministry of Education and school start later. Wait times and service levels can vary boards better integrating their activities with between regions in the province. other ministries and organizations to work toward Although research also shows that children healthy child and youth development. With respect with milder forms of autism have better outcomes to rehabilitation services, this will involve the with IBI, the program is currently available only Ministry of Health and Long-Term Care working to those assessed as having severe autism. There is with the Local Health Integration Networks and also limited funding and support to help children service providers to establish a province-wide co- with autism transition into adolescence and high ordinated system for rehabilitation and restorative school. The Ministry needs to re-evaluate its pro- inpatient services, along with all community-based gram design to optimize services and outcomes for outpatient services. children with autism.

Healthy Schools Strategy Violence Against Women The number of overweight children and youth in The Ministry of Community and Social Services Canada has increased dramatically in the past 30 provides funding for community programs and years, with nearly one in three being overweight services to help women and their children who and almost 12% considered obese. The Healthy are victims of domestic violence find safety and Kids Panel recently reported that obesity alone rebuild their lives. These programs also serve adult cost Ontario about $4.5 billion in 2009, including survivors of childhood sexual abuse. In the 2012/13 $1.6 billion in direct health-care costs. The Ministry fiscal year, the Ministry spent $142 million in this of Education established policies to support student area, with about $82 million of that going to the learning and growth through proper nutrition, and operation of 95 shelters and the remaining $60 mil- to set nutrition standards for food sold in public lion for other support services, including commun- schools. In 2005, the Ministry also revised the ity- and telephone-based counselling, and helping school curriculum to require that all elementary Reflections 7 students get 20 minutes of daily physical activity help people transition from acute care to rehabilita- during instruction time. tion and to ensure patients receive cost effective We noted that school food is currently not rehabilitation where and when they need it. monitored to ensure compliance with nutrition standards in the School Food and Beverage Policy, MEETING PUBLIC EXPECTATIONS and the Ministry does not know how successful the Policy has been or whether it has helped students In a number of the areas we audited, public eat healthier foods. We also found that not all ele- expectations are pretty clear. With respect to land mentary students were getting 20 minutes of daily ambulance services, for example, the public expects physical activity during instruction time as required an ambulance to arrive quickly, stabilize a patient by ministry policy. The Ministry and school boards and then get that patient to hospital fast. Ontar- need to ensure compliance with their requirements, ians expect provincial parks to be well-maintained work more effectively across government to better and the wildlife and natural surroundings in those integrate and align student nutrition programs, parks to be protected for the enjoyment of people explore best practices elsewhere, and work with today and tomorrow. The public also expects there other organizations and stakeholders, including to be employment opportunities for doctors in parents, to promote healthy eating and physical Ontario when their training has been paid for by activity for students. the province, in order to reduce patient wait times. And ratepayers expect to pay a reasonable price for electricity. However, these expectations are not Rehabilitation Services at Hospitals always fully met. The Ministry of Health and Long-Term Care funds rehabilitation services for eligible Ontarians, Land Ambulance Services including all hospital rehabilitation inpatients and hospital-registered outpatients. Demand for these In the 2011/12 fiscal year, the total cost of land services is expected to grow significantly as the ambulance services was an estimated $1.1 billion, population ages. Rehabilitation services include with $627 million of that funded by the Ministry physiotherapy, occupational therapy, speech- of Health and Long-Term Care, and $477 million language pathology, social work and nursing. These funded by municipalities. Under the Land Ambu- services assist people who have had certain types of lance Act, the Ministry must ensure the existence surgery, and those with injuries, chronic conditions across Ontario of a balanced and integrated system and disabilities, to help them regain, maintain or of ambulance services, including the communica- improve their health. tion services used to dispatch those ambulances. There is currently no provincially co-ordinated Municipalities are responsible for ensuring the rehabilitation system in Ontario. Each hospital has proper provision of land ambulance services within its own processes, and a patient deemed eligible their municipal boundaries in accordance with the for a service at one hospital might not be eligible needs of people in their municipality. for the same service at another. These services Ministry funding of land ambulance services have evolved across the province over many years almost doubled between the 2004/05 and 2011/12 such that there are now significant variations in the fiscal years, but the number of patients transported availability and type of services provided, which in that same period rose by only 18%. The Ministry can affect patient access to care. Many stakeholder does not know whether the additional funding associations have called for better provincial co- produced better service levels and patient out- ordination of rehabilitation programs in order to comes. In 2012, only about 60% of municipalities 8 2013 Annual Report of the Office of the Auditor General of Ontario

responded to 90% of their emergency calls within day-use areas and overnight camping. In 2012/13, the target of 15 minutes. Furthermore, there is no these 114 operating parks, which charge user fees, patient-centred measure and analysis of the time attracted more than nine million visitors. Our audit from receipt of an ambulance call to the time an notes that there has been significant environmental ambulance arrives at a patient’s location. damage to parks, but no meaningful strategies to address this damage. As well, there is little or no enforcement of hunting and fishing regulations in Health Human Resources significant portions of the provincial parks system; In the last six years, the Ministry of Health and enforcement of the prohibition of activities such Long-Term Care spent $3.5 billion through its as commercial timber harvesting and mining is HealthForceOntario strategy to address the short- also weak. Further, the Ministry estimates that it ages of physicians, nurses and other health profes- will need about $590 million to improve buildings, sionals across Ontario. Overall, Ontario has seen an bridges, roads, drinking water systems and other 18% increase in the number of physicians from 2005 park assets in poor or defective condition. to 2012, and a 10% increase in the number of nurses from 2006 to 2012. While provincial initiatives have Ontario Power Generation (OPG) Human increased enrolment in training programs, created Resources more postgraduate training positions, and attracted more doctors and nurses from other jurisdictions, OPG has relatively generous employee compensa- Ontario has not met its goal of having the right tion and benefit practices. About two-thirds of number, mix and distribution of physicians across OPG’s operating costs are human resources–related, the province to meet the population’s current and and reached $1.7 billion in 2012. It is therefore future health-care needs. For example, the province critical that OPG’s human resources expenditures be has spent significantly to train many specialists— effectively managed. A number of reviews of OPG who then leave the province because there are no have highlighted concerns over high staffing and full-time employment opportunities for them here. compensation levels. OPG offers its employees gen- One-third of Ontario-funded surgical specialist erous pension plan benefits, questionable relocation graduates left the province each year from 2005 allowances and compensation significantly higher to 2011. This is happening at a time when patients than comparable positions in the Ontario Public Ser- continue to experience long wait times for services vice, as well as a generous annual incentive bonus these specialists could provide. plan for non-unionized employees. Although OPG has been undergoing a business transformation process since 2010, there still are many areas relating to Provincial Parks compensation and benefit practices, staffing levels, It has been a challenge for the Ministry of Natural recruitment practices, performance management, Resources to meet its legislated mandate within its succession planning, outsourcing arrangements, funded resources, to protect Ontario’s provincial overtime usage, absenteeism and staff training that parks system and provide ecologically sustainable need further improvement. recreation because of the growth of the parks system and the Ministry’s expanded responsibil- INCREASING PUBLIC AWARENESS ities under the Provincial Parks and Conservation Reserves Act, 2006. Ontario has 334 provincial Two of our audits this year address the importance parks, of which about one-third are operating parks of public awareness—making the public more aware that provide such recreational opportunities as of services provided directly by the government, and Reflections 9 giving people a clearer and fuller picture of areas private school to educate their children. All private where the government provides less oversight than schools must be registered with the Ministry of the public might reasonably expect. In the case of Education. During the 2012/13 school year, there ServiceOntario, it is important that people be aware were over 1,000 registered private elementary of the varied services it offers, and how they can and secondary schools in Ontario that informed access those services. The public should also know the Ministry that they had enrolled about 110,000 that the Ministry of Education has only limited students. These schools are not required to follow involvement with private schools. policies developed for publicly funded schools, and do not have to follow the Ontario curriculum unless they offer credits toward the Ontario Secondary ServiceOntario School Diploma. ServiceOntario provides centralized service deliv- Based on the Ontario secondary school lit- ery to people and businesses seeking government eracy standardized test results that we reviewed, information. It administers programs for birth, a greater percentage of public-school students marriage and death certificates; business services, than private-school students met the provincial including company registrations; personal property standard, with private-school results varying from security registrations, such as liens on vehicles; well below average to excellent. We also noted that and land registrations, searches and title services. there is a risk that some private schools may be ServiceOntario also provides driver’s licence renew- operating unlicensed child-care centres and that als, vehicle registrations, and health-card renewals the Ministry needs to control and monitor its issu- and registrations. In 2012/13, ServiceOntario had ance of blank grade 12 diplomas to private schools approximately 2,000 staff, spent $289 million and to guard against diploma fraud. collected $2.9 billion in revenues. Notwithstanding its success in centralizing services, ServiceOntario could still make more operational improvements. In the 2012/13 fiscal year, only 30% of Special Audits ServiceOntario transactions were done online, well short of its forecast of 55% to 60%. An effective strat- Under the Auditor General Act, we perform assign- egy that includes heightened public awareness of the ments as requested by the Legislature, by a resolu- availability of these services, and pricing incentives, tion of the Standing Committee on Public Accounts, would help ServiceOntario meet this forecast and by the Premier, or by a Minister of the Crown. reduce costs. In addition, the risk of fraud exists with The reports of Special Audits are normally tabled the continued use of the 3.1 million remaining older upon completion, separately from our Annual red-and-white health cards still in circulation as well Report. This year, the Office issued the following as from frequent transaction processing errors. Special Audit reports (available on our website at www.auditor.on.ca): Mississauga Power Plant Cancellation Costs; and Private Schools • • Oakville Power Plant Cancellation Costs. Ontario has one of the least regulated private- At the time of writing, two other special audits school sectors in Canada. The Ministry provides requested by the Standing Committee on Public very little oversight to ensure that private-school Accounts were well in progress: the divestment of students receive satisfactory instruction. On its the Ontario Northland Transportation Commission website, the Ministry cautions parents to exercise and the modernization plan of the Ontario Lottery due diligence before entering into a contract with a and Gaming Corporation, including cancellation of 10 2013 Annual Report of the Office of the Auditor General of Ontario

the slots at racetracks program. The Office is start- So far, the impact of this on the consolidated ing a third special audit on the cost of the revised financial statements of the province has not been collective agreements for teachers in Ontario. considered material, in that it would not change a reader’s interpretation of the financial position and operations of the government. However, I share my predecessor’s concerns about legislating accounting The Government’s treatments that depart from generally accepted Consolidated Financial accounting principles established by the independ- Statements ent standard-setter, CPA Canada (formerly the Institute of Chartered Accountants of Canada). This could, in the future, put the Auditor General The objective of a financial statement audit is to in the position of concluding that, although the express an opinion on whether the financial state- accounting complies with legislation, the financial ments have been fairly presented. I am pleased to statements are not fairly presented under Canadian report that for the 20th straight year, my Office generally accepted accounting principles. has concluded that the government’s consolidated financial statements were fairly presented. This year the audit opinion was signed by Deputy Auditor General Gary Peall, who served as Acting The Province’s Financial Auditor General following the retirement of former Condition Auditor General Jim McCarter on May 1, 2013.

Our updated analysis of the province’s financial condition is discussed in Chapter 2. The govern- Financial Audits ment will need to continuously monitor and take action to manage its debt in a sustainable manner.

The Office spends considerable resources on conducting and overseeing financial statement audits. In the case of the financial statement audits of Crown Implementation of Our agencies completed this year, we concluded that Recommendations all were fairly presented. Our Office also assisted several agencies with their transition to public-sector A key part of the work of my Office has to do with accounting standards during the past year. follow-ups; each year, the Office revisits each of the value-for-money audits performed two years earlier to assess the progress that auditees have made on Legislating Accounting our recommendations. Many of the recommendations in our 2011 Annual Report were either substantially or partially In the past few years, the government has chosen implemented, although additional work remains to to legislate how certain transactions should be be done in several areas, where we will continue to accounted for in either the consolidated financial monitor progress. Follow-ups are discussed in detail statements of the province or other public-sector in Chapter 4. entities. Reflections 11

Responsibilities Under the Standing Committee on Government Advertising Act, Public Accounts 2004 Over the years, Ontario has had a diligent and active Public Accounts Committee. Since my arrival, I have The Government Advertising Act, 2004 requires our seen that this is still very much the case. Just as my Office to review most proposed government adver- predecessor did, I believe that from a pragmatic tising in advance of it being broadcast, published or perspective, the Committee’s support of our work displayed. We are responsible for ensuring that such encourages the implementation of our recom- advertisements meet certain prescribed standards mendations. Members of the Committee play a vital and do not promote the governing party’s partisan role in ensuring that any needed improvements in political interests by fostering a positive impression operational cost-effectiveness and service levels are of the government or a negative impression of any made by the public and broader public sectors. person or group critical of the government. In the 2012/13 fiscal year, we reviewed 572 advertisements in 130 submissions with a total value of $30.1 million. We noted again this year Acknowledgements that the government spent significantly on Internet advertising, which is not covered by the Act and On behalf of my Office, I want to thank the many thus is beyond our review to ensure it is not parti- people in the public and broader public sectors san. A full discussion of this issue and our advertis- involved in our work for their assistance and co- ing work can be found in Chapter 5. operation in the completion of this year’s Annual Report. We look forward to continuing to serve the Legislative Assembly and, through it, the citizens of Ontario. Chapter 1 12 on IBIeligibilityordischarge. providers regarding decisions families andservice between disagreements to dealwith mechanism ing IBI,anditintroduced anindependentreview governmenttentious hasfacedsurround issuesthe panelto con giveexpert itadviceonsomeofthe $182 million. totalled andsupports payments for autismservices not-for-profit organizations. In2012/13, transfer are agencies usuallyity orhospital-based that delivered approximately through 90commun exclusively autismaresupports with for children andother Theseservices (ABA)-based services. (IBI)andappliedbehaviourintervention analysis ortherapies—intensive behaviour tion services autism. with andprovided exclusivelyces (Ministry) to children ofChildren andYouth Ministry funded by the Servi andsupports onservices audit focused primarily Our ministries. fundedby various services, health therapy,speech andmental occupationaltherapy including andsupports access general services autismmay diagnosedwith children In Ontario, The prevalence ofautismhasbeenincreasing. FOR 3.01 • Chapter 1 Chapter Some of our key observations are asfollows:Some ofourkey observations In December2012, convened Ministry an the fundstwo typesofautisminterven The Ministry

rupled autism funding. Despite this, there there autismfunding.Despite this, rupled Over the last decade, the Ministry hasquad Ministry last decade,the Over the CHILDREN AUTISM SERVICES Value-for-money Audits Summaries of AND SUPPORTS - - -

- - - • • • •

dren with milder forms ofautismhave milderforms dren better with receiving them. children are there than government-funded services autismwaiting for with are more children across the province.across the resources areequitably beingdistributed available next spot.the wait list for endofthe to returns the child the onegoal, or goalsto achieving achieve. After whohavethose many behavioural problems mightnot besufficient at atime,and,thus, for autism, allow to work achild ononly onegoal mildto of moderateto with forms children incially available fundedtherapy inOntario four years. regions we three visitedces inthe was almost The medianwait timefor accessingIBIservi factors. risk SocietyPediatric with for children Canadian by endorsed the screening period tle over 18-to-24-month the 3years, later than atamedianage ofalit diagnosed inOntario IBIuntilalmost age 7.start due to donot longwait lists, children typically later. whostart those outcomes than However, IBIbefore age whostart 4havechildren better severe alsoshows autism.Research that available more only to assessedwith children iscurrently program IBI,the outcomes with Although scientific research shows that chil showsthat scientific research Although The Ministry hasnot assessedwhether The Ministry only typeofprov the ABA-based services, We autismare with children estimated that

- - - - • • • • •

planned to includeautism. is strategy that developmental services committee to work onacomprehensive legislature passedamotion creating aselect strategy. However, inMay 2013, provincial the autism. to educate with children are beingeffectivelymethods usedinschools are onawait list forothers services. ofspecialtreatmentpractice continueswhile system typically receives.regular This service inthe achild amountthat twicethe than represents more Per this vice program. child, regular ser ongoing cost ofIBIoutsidethe to 60peoplefor atotal of$21 million for the ments cannot berescheduled. isbecausemissedappoint arises this that were approved for. key Oneofthe reasons received fewer they than oftherapy hours option direct-service receiving IBIunderthe Ingeneral,versus children 25months). option (35months direct-service under the those than almost oneyear more ofservices direct-funding option received onaverage in 2012/13 province-wide, underthe those situationwas reversed. region, the another option. In age direct-service wait underthe the optionaver wasthan longer five months direct-funding underthe wait for IBI services regions. Inoneregion in2012, average the cantly between two the options andamong Wait signifi candiffer timesfor IBIservices agency services. to private purchase service family getsing, where the lead fundsfrom the provider atnocost;a service ordirect fund directly from receives child service where the options: direct service, IBI service-delivery how fundingbetween to allocate ministry two It is up to the lead service agency to decide leadservice It isupto the Ontario doesnot haveOntario aprovincial autism More ABA to work ensure that isnecessary up hasreimbursed Ministry Since 2006,the discharged from IBIservices children Of the - - - - - Agency (Agency) in2007. TheAgency focuses on Marketing HealthForceOntario the andRecruitment Strategy.on the hasspent$3.5 billion Ministry last sixyears, the the million spentin2006/07. $448 65% from the Over 2012/13 were $738.5 million, anincrease ofabout providers. Total Strategy for expenditures the in ber, ofqualified health-care mixanddistribution num right the maintains Ontario is to ensure that andlongwait times.Theintent inOntario nurses overaddress concerns ofphysicians shortages and (Strategy) in2005/06to Strategy ForceOntario and Universities jointly developedHealth the ofTraining, Ministry andthe Colleges(Ministry) andLong-Term ofHealth The Ministry Care following: from 2006toin nurses 2012. in physicians from 2005to 2012 anda10% increase factithasseenan18%needs, despite the increase care professionals to meet itsfuture health-care number, right the ofhealth- mixanddistribution andretention professionals. recruitment of health 3.02 • • As part of the Strategy, of the As part established Ministry the Our most significant observations include the include Our most significantobservations hasnotOverall, met itsgoalofhaving Ontario

2005 to 2011 are longwait eventhere though province year the from each left graduates surgical specialist age, 33%ofOntario-funded ing specialists. Statistics show onaver that, employment graduat for these opportunities are few because there full-time primarily fits)—do not stay inOntario inandpractise $375,000 andbene for resident salaries at acost ofabout$780,000 (including each areas. tion lived inrural sametime,14%areas. At the popula ofthe inurbanareaspractised and5%inrural ince. Asof2011, 95%ofphysicians inOntario prov areas ofthe remoterural, andnorthern wholive particularly in those some Ontarians, Many specialists whoare inOntario— trained Access care isstill toaproblem for health HEALTH HUMAN RESOURCES Summaries of Value-for-money Audits ------13

Chapter 1 Chapter 1 14 2013 of Ontario of the OfficeGeneral of the Auditor Annual Report • • • •

to determine the supply doesnotto ofnurses determine the employment difficult. and funding,training appropriate with ofphysicians mixanddistribution numbers, optimal make These limitations planningthe especiallyof data, onphysician productivity. limitedpered by reliability andavailability the physician workforce requirements, itisham Association was apositive step indetermining Medical Ontario the with built inpartnership full-timejobs. given permanent 2010/11 in2011/12 andathird actually were in nurses new graduate a quarter ofthese However, nurses. ing new graduate only about full-timejobsto participat permanent, offer will they that expectation the with six months provides fundingfor upto organizations with Nursing Graduatethe Guarantee Program andinitiatives.ing programs For example, effectiveness ofitsnurs and assessmentofthe needsto improve Ministry the itsoversight workingnurses onafull-timebasis.However, goalof70% Ministry’s slightly underthe wasworking just which full-timeinOntario, since 2007. continuously tion Project hadbeenusingthem CoverageEmergency Department Demonstra ofthe aspart physiciantemporary services were that hospitals using ofthe one-third timeofouraudit, At the Ontario. Northern positionsfor specialists vacantin permanent were there audit indicated about200 that The latesttimeofour available data atthe instead oftime. beingusedfor longperiods to cover vacancies,are Ontario, short-term eligible communities,particularly inNorthern professionalsvide accessto in health-care physicians are to trained these provide.gery) forefoot andkneereplacement sur surgery as (such sameservices times for someofthe At the endof2011,At the were 66.7%ofnurses are meantto which pro Locum programs, As well, the model currently being developedAs well, model currently the physician forecasting the model Although ------nutrition andphysical activity.nutrition endeavour, Inthis proper through andgrowth learning students’ port Strategy HealthySchools to the has established sup responsibility for publicly fundedschools, primary 20% infive years. ment set obesityby agoalofreducing childhood late 1970s.as inthe In2012, govern Ontario the 12% are considered obese—almost twiceasmany Nearly students isoverweight. oneinthree Almost Canada hasbecomeasignificantpublicconcern. The increasing incidenceofoverweight in children increased physical activity throughout the system. the increased physical activitythroughout ive for practices healthylivingand encouraging and stakeholders, includingparents, to share effect to work organizations more other effectively with requirements, need complying these andthey with are schools that into ensuring to putmore effort however, boards need andschool Ministry the that minutes day. ofphysical activityeach We found, students getto 20 require allelementary that curriculum school Ithasalsorevised the schools. soldin requirementsnutritional offood anddrinks Strategy.Schools to 2011/12 onactivitiesrelated Healthy to the lion annually fiscal over 2009/10years three the told usitspentabout$4 mil dents. TheMinistry anenrolment ofapproximatelywith 2 millionstu schools andsecondary operate 4,900elementary Care andChildren andYouth Services. andLong-Term asHealth such ment ministries, govern ofother support relies onthe Ministry the 3.03 • The Ministry of Education (Ministry), which has which ofEducation(Ministry), The Ministry Our key observations wereOur key asfollows: observations hasdeveloped policiesfor the The Ministry 72 boards publicly fundedschool Ontario’s

the population’s needs. the neededto numberofnurses meet consider the complied with the nutrition standards in the standards inthe nutrition the complied with soldinschools food anddrinks ensure that visited hadeffective strategies monitoring to Neither the Ministry nor the school boards we school northe Ministry the Neither HEALTHY SCHOOLS STRATEGY ------cation services used in dispatching ambulances.” usedindispatching cation services andcommuni system ofambulanceservices grated ofabalancedandinte Ontario existence throughout and Long-Term must ensure “the Care (Ministry) Under the 3.04 • • • •

secondary school cafeteria sales at the three three cafeteria school salesatthe secondary policy,drink someto asignificantdegree. food and inthe did not criteria meet nutrition that board identifiedanumberofitemsschool standards. met the they cafeterias to soldintheir ensure and drinks boards we visited hadnot reviewed food the policy. three school Ministry’s the Officialsat activity perweek. inatleast 150participate minutes ofphysical Columbia, students must ate, andinBritish credits to fourdents must obtain such gradu four InManitoba, years stu ofhighschool. their andphysical educationduring in health students must complete only onecredit course InOntario, jurisdictions. it isinsomeother level lower highschool ismuch ity atthe than utes aday. didnot get schools requiredtheir the 20 min students at whoresponded saidthat of those representatives, half of school andmore than boards weschool visited conducted surveys Two curriculum. school three tary ofthe timeasrequired elemen instruction by the 20 minutes ofdaily physical activityduring to 1to ensure 8hadthe students ingrades istry, we boards andschools school visited to eatatnearby fast-food restaurants. saidmanyspoke students with now preferred dropped between we 70% and 85%. Principals 25% and45%.Vending revenue machine also boards weschool visited decreased between The Ministry’s requirement for physicalThe Ministry’s activ Min byThere monitoring was the noformal were food choices healthier introduced, After Our review ofasample ofmenuitems atone LAND Ambulance Act AMBULANCE , the Ministry of Health ofHealth Ministry , the SERVICES ------for providing medical support to paramedics with for to providing with paramedics medicalsupport Physicians inseven are responsible basehospitals four by municipalitiesandoneby operator. aprivate in Ontario—11 Ministry, by the run sixby hospitals, patients. port butdonot paramedics trans vehicles, carry which emergency830 ambulancesand300other response viding landambulanceservices. agents)designated are delivery responsible for pro Municipalities (42municipalitiesandeightother providers. ambulanceservice reviews, certifying service standards and,through those pliance with ambulance equipment com standards, monitoring Itisresponsible for setting Ontario. patient-care and in oversees landambulanceservices The Ministry of service, and it does not determine the reasons anditdoesnot determine the of service, relationship betweenlyze the fundingandlevels doesnot provide. ana they TheMinistry service spendmore, level regardless that ofthe services provides year more to fundingeach ambulance ter patientoutcomes. funding hasresulted infaster response timeorbet doesnot increased know the The Ministry whether increased byported only 18% time. that during doubled. However, numberofpatientstrans the nearlymunicipalities for landambulanceservices and$477Ministry million by municipalities. lion, $627 were million ofwhich fundedby the For 2011/12, total ambulancecosts were $1.1 bil remote other areas. incertain andfor those services 100% 10 cost for the ofthe First Nations ambulance funds TheMinistry centres andbasehospitals. patch well as100% ofapproved costs for ambulancedis plusanincrease for inflation,as ambulance services, prior-year municipality’s about 50%ofeach costs for about 15% for since2008. both 970,000 patientswere anincrease of transported, 1.3 million ambulanceswere andabout dispatched medicalprocedures.complex In2012, orrisky about There are 22 Ministry-controlled dispatch centres centres dispatch There are 22Ministry-controlled municipalitieshaveIn total, 50Ontario about The Ministry’s funding formula automatically fundingformula The Ministry’s From to 2004/05 2011/12, fundingto Ministry hasfunded last fewOver Ministry the years, the Summaries of Value-for-money Audits ------15

Chapter 1 Chapter 1 16 including the following:including the 15gency callswithin minutes. emer 50 municipalitiesresponded to 90%oftheir compared In2012, to others. only about60%ofthe somemunicipalitiesspendandreceivethat more 2013 of Ontario of the OfficeGeneral of the Auditor Annual Report • • • • • We noted areas where other actionisneeded,

designed to over-prioritize callswhenthere centre. dispatch the rates ranged from 70%to 90%,dependingon required centres’ result, compliance dispatch two within minutes.need to Asa dispatch percentage ofurgent the callsitwouldchoose of 2013, centre was allowed dispatch each to andahalfminutes. As three callswithin these two minutes. However, 90%of alldispatched target the of 90%ofcallswithin dispatching policyof Ministry’s the calls complied with timeto respondmeasure to their emergency used aretrospective response timemeasure. Wepatched. that found jurisdiction noother timeacallisdis availableinformation atthe response timesbasedon report jurisdictions Most other timeofdispatch. by atthe callers provided oninformation ively) than rather apatient(i.e.,retrospect reach they after patient urgency measured by paramedics usedisto method bebasedon reporting the in2014,ambulance response timesstarting location. patient’s the call receipt at to whenanambulancearrives overall timeof is,from the response time,that system’s ambulanceservice measure ofthe eightminutes. to 85%within (rural) significantly, vary patients andthey from 9% ity sets itsown response-time targets for these urgent asstrokes. municipal cases,such Each butnot cardiac for arrest, other experiencing or whoare choking asthose patients, such most time-sensitivetime standards for the While the Ministry expects to expects publicly report Ministry While the doesnot haveThe Ministry apatient-centred hasset meaningfulresponse- The Ministry While dispatch protocolsWhile dispatch are generally In 2012, centres that 20dispatch noneofthe - - - - • • • • • •

ensure proper patient care for such heart ensure proper heart patientcare for such equipment andappropriate paramedics to someambulancesdidnot havethat trained patients, andaJune2013 indicated survey attack heart careensure ofcertain appropriate more cost-effective. would dispatch centralized be or whether centres isoptimal, numberofdispatch current urgent.respond are to truly new callsthat can leave few ornoambulances available to actually required anurgent response. This urgent level, whenonly about25%ofpatients most- two-thirds ofcallsatthe more than protocol dispatch prioritized Ministry’s the condition, aboutapatient’s is uncertainty 20% of the hospitals funded. hospitals 20% ofthe hasactually hospital increased at tioned atthe mented, ambulancewaiting time whilesta We was imple program sincethis found that totalled program $40 million. funding for this Between 2008/09and2012/13, ministry for value-for-money. program offload nurse underlying reasons. hasnot assessedthe patient. TheMinistry a didnot transport ambulances dispatched proper oversight. mayviders to not provide have expertise the pro cated municipallandambulanceservice physicians basehospital havethough indi most patient-care paramedic activities,even ambulance paramedics. later, relying instead from onverbal briefings have records accessto untiladay such ortwo donot often emergencyhospital room staff Asaresult, hospitals. patient records with cannot electronically that share software patients. attack The Ministry hasnoprovincial policytoThe Ministry hasnot the assessedwhether The Ministry The Ministry hasnot patient evaluated the The Ministry In 2012, over 25%(orabout350,000)of Municipalities are responsible for overseeing Municipalities acquired patient-care record - - - - anonymous survey of more than 800 OPG staff. 800OPGstaff. anonymous ofmore than survey were byour concerns respondents echoed to our needimprovement.and benefitpractices Many of its humanresource management and compensation overall levels, staffing we found several areas where inreducing its While OPGhasmadesomeprogress by 2,000employees by 2015. attrition through ject in2010, atarget with ofreducinglevels staffing ance andadministration. ofitstotalor 64% costs for mainten operations, in2012labour costs, which were about$1.7 billion, cost ofelectricity,the respect to particularly with city, costs itsoperating have asignificantimpact on OPG still generates province’s electri 60%ofthe Given have that inOntario. prices beenrising tricity power generation. ismore private-sector involvementthere innew has decreased, coal-fired plantshave closedand demandfor electricity last decadebecausethe the of power OPGproduces hasdecreased by 23%over However, America. generators inNorth amount the owned province, largest by the isoneofthe power PowerOntario (OPG),acorporation Generation HUMAN 3.05 • • • Some of our key observations wereSome ofourkey asfollows: observations OPG initiated itsBusinessTransformation Pro demand,elec decliningelectricity Despite the

torial and custodial services was still staffed at at was still staffed andcustodialtorial services in2011,starting area ofmaintenance, jani the left. in lump they sumsafter pensionsand somehadalready drawn their and AnnualIncentive Plan(AIP)awards, continued to allowances receive significant transfer andsuccession planning.Some OPG, indicatingineffective knowledge left they after shortly almost allofthem 58% (to 238in2012 from 152 in2005). senior management hasincreased by group 12,100 sizeofitsexecutive in2005),the and down about8.5%(to 11,100 in2012 from OPG rehired some former employees,OPG rehired someformer overallWhile OPG’s levels staffing have gone Even after staff reductions at nuclear facilities reductions atnuclear facilities Even staff after ONTARIO RESOURCES POWER GENERATION

- - - - - • • • •

had expired clearances. clearances. had expired required or clearances the security obtained hadneverconfidential nuclearinformation, accessto with including senior staff staff, process. Aswell, 50%ofOPG more than recruitment normal the been hired through to had show family ofstaff whether members related. However, OPGhadnodocumentation sameaddresses andappearlikelyat the to be ofemployees orgroups 700pairs About live processes. clearance andsecurity recruitment functionswereated support overstaffed. associ whiletheir nuclear plantoperations, were including significantly understaffed, 2013. Meanwhile, functions someoperational a level 170% benchmarkin above industry the in housingandmoving allowances. from hisnew work locationreceived $80,000 employeeAnother away whomoved further saleofhisoldresidence. of $354,000 from the the proceeds tion benefitsfrom OPG,on top of an employee received over $392,000inreloca that seem questionable. benefits example,For valuation. in itslatest actuarial was about$555 million pension deficit,which OPG isalsosolely responsible for financing its OPS. for 1:1ratio the the cantly higherthan hasbeenaroundtion ratio 4:1to 5:1, signifi employer-employeeOPG’s pensioncontribu most deputyministers. As well, since2005, seniorexecutivesOPG’s more than earned (OPS)andmany of PublicService Ontario for comparableat OPGthan positionsinthe score achieved. the todocumentation support were limitedthere anumberofcaseswith frequently inseniorpositionsand to staff However, highscores were given more much andAIPscore onascaleof0to 4. base salary to $1.3 joblevel, million, dependingonthe from ranging $1,600all non-unionizedstaff, We found areas ofnon-compliance inOPG’s Some of OPG’s employeesSome ofOPG’s received generous wereEarnings significantly more generous OPG gives AnnualIncentive Planawards to Summaries of Value-for-money Audits - - - - 17

Chapter 1 Chapter 1 18 children at a private school. school. ataprivate children before into entering to acontract educate their website, cautionsparents to exercise duediligence and,onits students instruction receive satisfactory littleoversight school very private to ensure that provides sectors inCanada.The Ministry school students. school provincial private dents achieved the standard than reviewed, stu percentage agreater ofpublicschool excellent. Basedonstandardized test results we from well varies below schools private average to quality ofeducationprovidedthe by participating results ofstandardized academictests suggest that are notschools inspected Ministry. by the The atnon-credit-granting offered loma. Theprograms dip school credits toward secondary anOntario offer registeredthat at only schools those private diploma. school its toward secondary Ontario the cred offers school unlessthe curriculum Ontario the developed for orto publicly follow fundedschools organizations, andare not required to follow policies are considered to These schools beindependent atotal reported enrolment of110,000they students. were schools private registered and and secondary 2012/13 year, school 1,000 more than elementary Inthe ofEducation(Ministry). Ministry the with must beregistered inOntario schools Private 2013 of Ontario of the OfficeGeneral of the Auditor Annual Report 3.06 • • Our major observations included the following: includedthe Our major observations least regulated private hasoneofthe Ontario inspections conductsprogram The Ministry

or her grade 12or hergrade diploma. toward credits granted the actually earned his to credit integrity, astudent meaning whether related loma credits. concerns Many ofthese dip highschool offer that 400schools of the inspection andmaintenance technicians. resulted inhighovertime pay, especially for since 2003.Plannednuclearoutages have $50,000 inovertime pay peryear haddoubled The number of OPG staff earning more than more than earning The numberofOPGstaff The Ministry noted at significantconcerns The Ministry 100 PRIVATE SCHOOLS - - - - • • • •

registered before June1993 to operate child- allows schools private TheMinistry schools. age school enrolled inprivate compulsory 15,000be more than younger children than may there information, According to ministry centres. unlicensedchild-care be operating enrolment data. hadnot required submitted they the though evenissued 2,300diplomasto 50schools, also 12 grade enrolment. TheMinistry their received atotal of1,500 more diplomasthan and,for example, schools schools 30private ever, procedure isnot appliedto this private 12 to grade schools student enrolments. How reconciles blankdiplomarequests from public these schools. these ensures somelevel ofeducationquality at Ministry the that could bemisledinto thinking Ministry. public Parents, students andthe hadbeenaccredited by programs the their were schools where that private advertising However,program. we identifiedseveral cases hasapproved academic Ministry their the are not to schools permitted stateprivate that noted.cerns oversightto other inform agencies ofany con andsafetyhealth issues;andhasnoprocess for quality orcontent; for any doesnot check doesnot curriculum evaluate the Ministry again.place to The everschools visitthese isnoprocess in registered butthere schools, visit to validate conductabrief newlyofficers education schools, secondary credit-granting required to have qualifications. child-care are not staff requirements, school andprivate canoversee, are nofire safetystaff there ofany school ber ofchildren private age that licensed daycare, num isnolimitto the there alicence.Incontrast to care facilitieswithout There is a risk that some private schools may schools someprivate that There isarisk To helpprevent Ministry the diplomafraud, Given the limitations of the validation process, validation process, ofthe limitations Given the andnon- 600elementary For more than the - - - Ministry’s ability within itsfundedresources to ability within Ministry’s Act haveresponsibilities inthe challenged the park expanded system of the andthe The growth are protected. parks the valueswithin natural the requirements that the forexpanded ensuring grown innumberandsize.The2006 Act park infrastructure. funded solely province, alsofunds by the which planningandprotection park systemthe ofthe are Expenditures difference. related tomaking upthe province the costs, operating with parks’ of the ated by userfees have recovered 80% more than totalled $80 million.Historically, revenues gener expenses, includingheadoffice expenses, operating generated about$69 millioninrevenues, while onlyand offer limited facilities.)Provincial parks public,havestill onsite accessibleto the no staff 9 millionvisitors. (Non-operating while parks, than attracted more them, within offered and services andfacilities parks charge fees useofthe for the ecologically recreation. sustainable for andprovide heritage and cultural opportunities natural significantelementsofOntario’s contain that reserves system ofprovincial andconservation parks Act. the ance with andmanagingprovincialoperating inaccord parks isresponsible for establishing, Resources (Ministry) and overnight ofNatural camping. TheMinistry likevide recreationalday-use opportunities areas pro that are parks operating ofwhich about athird 334 provincialand management parks, ofOntario’s (2006) (Act) development, governs the operation The 3.07 PROVINCIAL • Over the last 10Over the years, provincial have parks In 2012/13, 114 the which parks, operating Act isto protect ofthe permanently a The purpose Provincial Parks and Conservation Reserves Act Reserves Provincial Parks andConservation

had ended. June 2013, year school the afullyear after 2011/12students for the year by school not submitted required ontheir information Approximately hadstill schools 250private PARKS - - - sustainable recreation.sustainable system for andprovide ecologically opportunities meet itslegislated mandate to protect park the • • • • Specifically, the following:we found

every managementevery direction reviewed noted the park. Infact, inmanaging priority first the was ecologicalintegrity clear statement that a nonecontained Act andconcludedthat the haddeemedto beconsistent with Ministry the audit reviewed directions asample that ofthe and values.Anecologist we retained for the park’sment andmanagement resources ofthe provides protection, policiesfor the develop to have inplaceamanagement direction that Act requires Asaresult, park the parks. each the inmanaging priority first isthe integrity condition require $590 million more than listed asbeingin“poor”or“defective” water systemsdrinking andseptic systems asbuildings,roads, assets bridges, such that assets. oncapital expenditures We estimated andmining. timber harvesting activitiesascommercial prohibition ofsuch the regulations onhuntingandfishing, of tem are subject to littleornoenforcement of by ateam oftechnicians. ence team ofatleast oneecologist supported federal park system hasasci each inthe that Asacomparison, Parks Canadatoldparks. us in20to andmonitoring research 50provincial few park biologists, mightberesponsible for ogist, aidedonly by anda aseasonalassistant Act. Weof the oneecol noted inOntario that itrequires standard that to rigorous meet the the park system on baselinescientificdata the lacked Ministry intendents revealed the that ecologists, biologists andparkners, super meaningful strategies to address them. park, butnoneputforward the tions within significant damage to environmental condi According Act, ecological to maintaining the The Ministry has a significant backlog for hasasignificantbacklog The Ministry the provincial of park sys Significant portions own 2011The Ministry’s ofpark plan survey Summaries of Value-for-money Audits ------19

Chapter 1 Chapter 1 20 from the province.from the who are eligible for socialordisabilityassistance homes;and people at homeorinlong-term-care and 65over; peoplewho require physiotherapy forthose qualified people,including 19 andunder alsofundscommunity-based services The Ministry tion inpatients andhospital-registered outpatients. rehabilita Thisincludeshospital eligible Ontarians. tion bedsoradmissions. total numberofrestorativeable onthe rehabilita availinpatient There isnoinformation programs. people were admitted to regular rehabilitation ments) andstroke were most commonreasons the pedic conditions(includinghipandkneereplace 30,000 patientswere admitted in2012/13. Ortho moreregular bedsto than rehabilitation which longer The61 term). have hospitals almost 2,500 andrestorativeterm) (slower-paced andover a regularrehabilitation: (frequent sessionsfor ashort Networks (LHINs).There are two kindsofinpatient in 61 14 through hospitals Integration LocalHealth fundsinpatient services rehabilitation (Ministry) inpatientsrehabilitation were over 75 years ofage. 75. turn In 2012/13,boomers abouthalfofregular cantly, 2021 especially after baby first whenthe therapy, isexpected to increase signifi inOntario asphysiotherapy andoccupational such services, demandfor rehabilitation In comingyears, the 2013 of Ontario of the OfficeGeneral of the Auditor Annual Report HOSPITALS 3.08 • The Ministry funds rehabilitation services for services fundsrehabilitation The Ministry andLong-Term ofHealth The Ministry Care

revenue than the Ministry receives. Ministry revenue the than generate approximately $6.7 millionmore in cantly below fairmarket valueandshould We leasepayments are noted the signifi that ties heldunderleaseintwo provincial parks. approximately $160 million. hasincreased by backlog in2002,the parks to replace. Sinceourlast auditofprovincial There are nearly cottage 600private proper REHABILITATION SERVICES AT

------co-ordination of rehabilitation programs, toco-ordination help programs, ofrehabilitation holder associationshave calledfor better provincial atanother.eligible for similarservices Many stake mightnot atonehospital be eligible for services and providing care. Asaresult, apatientdeemed patients prioritizing eligibility for itsservices, policies andprocedures for determining patient itsown establishes hospital each means that willoffer, they ifany. services rehabilitation This inpatientdetermine which and/or outpatient input LHIN—generally from with their tals—some Instead, individualhospi tion system inOntario. outpatientprograms. hospital-run numberofpatientswhouse oronthe tion services total publicfundingspentonrehabilita able onthe include the following:include the needed. and to ensure patientsreceive when rehabilitation peoplefromtransition acute care to rehabilitation • • • There is currently noco-ordinatedThere iscurrently rehabilita doesnot have availThe Ministry information Some of our other significant observations significantobservations Some ofourother

were available. lesscostly services if these patient programs mighthavethey beenbetter inout served Thissuggestsmild functionalimpairment. assessed by anacute-care ashaving hospital we stroke hadbeen visited programs with twoinpatient atthe hospitals rehabilitation are effective. not know services ifthose does Ministry meansthe patient rehabilitation of restorative inpatient oronout rehabilitation provincial average is18 bedsper100,000. six per100,000 West Central inthe LHIN. The Torontopeople inthe LHINto Central only number ofbedsranged from 57 per100,000 The to travel LHINfor services. outsidetheir patientshaveprovince, couldmeanthat which inpatientlar rehabilitation bedsacross the Approximately ofpatientsadmitted to athird useoroutcomes onthe ofinformation The lack supply ofregu inthe There iswidevariation ------meeting its service level targets,meeting itsservice butitneedsto andisgenerally service plishments incentralizing centres. service telephone centres and7%atin-person contact the were ofthese rals—55% madeonline, 38%through about 12 million requests andrefer for information alsohandled Internet 30%.ServiceOntario the centres accountingfor 70%and service in-person with 35 milliontransactions, handled more than registrations. vehicle renewals registrations andhealth-card and most significantly licensing driver renewals and high-volume routineministries, transactions, andlandregistration services. services; registration and security property personal trations; includingcompanycates; businessservices, regis certifi anddeath marriage events, asbirth, such involvingbusinesses for anumberof programs vital to individualsand delivery service centralized hasamandate toGovernment provide Services, of Ministry ofthe aseparate part ServiceOntario, 3.09 • • ServiceOntario has made substantial accom hasmadesubstantial ServiceOntario 2012/13In the fiscal year, ServiceOntario alsoprocesses, for 14ServiceOntario other

care (which includesrestorativecare (which rehabilita bedoracomplexlar rehabilitation continuing 25%weremade. Ofthese, waiting for aregu bedsforcare to hospital arrangements be ready to bedischarged were waiting inacute- alternate-level-of-care patientswhowere 2013, asofMarch that reports about2,300 Association Hospital patients. TheOntario care may beoccupying bedsneededby other hospitals wehospitals visited. frequency atthe oftreatment. varied Practices and shouldstart matters aswhentherapy for such best-practice standards inOntario are few there rehabilitation, tions requiring tion) bed. With the exception the ofstroke,With for most condi Patients whonolonger required hospital SERVICEONTARIO ------areas: tions, registrations andpermits. inissuingandmanaginglicences,certifica risks levels andcustomer satisfaction,andreduce its costs, effectively delivery monitor service service to itssystems andprocedures strengthen to reduce improve inseveral key areas. Itneedsto continue • • • • • • Specifically,the following actionisneededin

by approximately $2.9 millionannually if50% costs operating wouldestimated that decrease online instead For of inperson. we instance, encourage to peopleto doingbusiness switch could could beachieved ifServiceOntario would beonlineby 2012. savings Further 55%to2008 forecast 60%oftransactions that actions were doneonline,well ofits short health card, which has no expiry date, with the the date, with hasnoexpiry card, which health red-and-white by replacing the costs andrisks government announceditsplantothe reduce 51% to 77%, compared to itsgoalof80%. targetedin answering callswithin timeswas dards for answering calls.Therange ofsuccess stan telephone centres met contact itsservice target timeof15 minutes. farexceeded often its centres, which service customer waitthe atpeaktimesorspecific application. oneonthe vehicle the to than anameother awrong licenceplate numberto transferring applicationsto health-card renewing the chargesfinancial to missingsignatures on by itsaudits.Theseranged from incorrect uncovered numberofprocessing errors of the locationsbecause centres ashigh-risk service has remained at71% to 75% since2009/10. ces, butitsonlinecustomer satisfactionrating online. more licenceplate sticker renewals were done In 2012/13, trans only 30%ofServiceOntario Significant fraud risk still existsrisk still Significant fraud 18 after years In 2012/13, seven noneofServiceOntario’s on did not measure orreport ServiceOntario rated 43% ofits289in-person ServiceOntario hasimproved itswebsiteServiceOntario servi Summaries of Value-for-money Audits - - - - 21

Chapter 1 Chapter 1 22 $82 million went toward of95 operation the spent $142 million ontransfer payments, ofwhich In2012/13,women children. andtheir Ministry the to abused tors, to andservices deliver supports are governedwhich by volunteer boards ofdirec 200communitynot-for-profitmore than agencies, safety lives. andrebuild their whoare victimsofdomestic violencefind children to helpwomen andtheir andservices programs provides anumberof community (Ministry) ofCommunityandSocialServices The Ministry 2013 of Ontario of the OfficeGeneral of the Auditor Annual Report 3.10 VIOLENCE • • • • The Ministry provides transfer paymentsThe Ministry to

polymer documents. provincesother have to already switched Statistics Vital Councilfor Canada.Eightthe andlossasrecommended by forgery theft, ments andanew designto minimizeidentity polymerto (plastic) docu highersecurity onpaperandswitch certificates birth printing provincecosting the about$5 millionannually. were We indeed farmers. could be estimate this rate compared to commercial vehicles— other vehicles—which are registered atareduced registering large commercial agricultural who didnot require them. wereensure they not beingmisusedby people toand controlling accessibleparking permits asdeceased. were to reported ServiceOntario were namesofpeoplewho circulating inthe cardsactive and1,400 licences health driver’s ofmisuse.Aswell, 15,000 risk more than the cards, increasingbeen issuedduplicate health approximately 1,500 had peopleinOntario 13.4 cards issuedinOntario. million health total remained incirculation,of or23% of the 2013, 3.1 million red-and-white cards more secure photo card. AsofAugust health 1, ServiceOntario hadnoplansinplaceto stop ServiceOntario people didnot that verify ServiceOntario hadweak processes for issuing ServiceOntario We 31, asofMarch estimated that 2013, AGAINST WOMEN - - plans and the Council’s recommendations. Council’s plans andthe action implementationfor ofthe co-ordinating the Women’s Directorate (Directorate) isresponsible TheOntario for abusedwomenchildren. andtheir mendations for improving system ofservices the Women’s 45recom with Issues,released areport (Council), created Minister by Responsible the for Council Domestic Advisory Violence in 2009,the Action Violence Sexual Plan(2011).the Aswell, Domestic Action Violence and Plan(2004) the action plansto address violenceagainst women: housing. more permanent to helpwomen secure helplines,andservices crisis includingcommunity-based counselling, services, shelters and$60 milliontoward supportive other following: 1999 to 6.3%in2009. spousalabusedecreased from 7%in experiencing Thepercentage ofwomenin Ontario. whoreported planshowed 2004 release ofthe the somechange prevalence ofdomestic violencebefore andafter implementation ofindividualcommitments. of the dotal status andhave onthe not clearreports offered publicly Directorate by the have beenmainly anec However, have that beenissued reports progress the for abusedwomenchildren. supports andtheir of preventing domestic violenceandimproving ActionViolence Planwas meeting itsobjectives ment to Domestic have 2004 the assessedwhether • By 2013, we would have govern expected the released two last decade,Ontario the During Our more significant observations included the included Our more significantobservations Meanwhile, Statistics onthe Canadadata services there. there. services were referredto agency another andreceived how doesnot track women manyistry ofthese overstates Min unmet demandbecausethe help.However,who soughttheir figure this away 15,000 women, women or56%ofthe turned they emergency that shelters reported gap. Forto closethe example, in2011/12, allocate resources and,inturn, for services it would unmet needto demand identifythe The Ministry doesnot have information The Ministry the - - - - • • •

day cost ofcare ranged from $90to $575. per-from $334,000 to the $624,000, sothat funding for 10-bed emergency shelters ranged vices. In2011/12, Ministry-approved annual costs amongagencies providing similarser result, we inunit found significantvariations Asa to identifiedneedsorpast performance. littleornocorrelation in previous years, with is generally basedonwhatan agency received had paidto problems. the fixanyother of completed agencies themselves the orwhether fundedprojects hadbeen know the whether only 10% deficiencies anditdidnot ofthose hadprovided fundingfor Ministry update), the 31,tion. AsofMarch 2012 latest available (the neededatten safety issuesthat andsecurity than 500 shelter buildingsidentifiedmore were completed for 40%. another for 20%ofagencies andfewer 10 than surveys wererate. Inaddition,nosurveys completed vides limited valuebecauseofitslow response In 2009, an assessment of the condition of conditionof In 2009,anassessmentofthe pro clientsatisfactionsurvey The Ministry’s fundingto transfer-paymentMinistry agencies - - - Summaries of Value-for-money Audits 23

Chapter 1 Chapter 2 24 tion regarding the province’stion regarding financialcondition the providesAnalysis sectionthat additionalinforma aFinancialcontains Statement Discussion and province’s annualreport. are included in the Report, ourIndependent Auditor’s statements, alongwith oromissions. Theconsolidated financial errors is,free ofsignificant misstatement—that material statements are free of the reasonable that assurance statements. Theobjective ofourauditisto obtain and proper records are maintained. assets are safeguarded, aretransactions authorized, procedures, isinplaceto ensuresupporting that aneffective system ofcontrol, with that ensuring sented fairly. Thegovernment isalsoresponsible for amounts basedonestimates andjudgment,ispre many includingthe information, the that ensuring consolidated financial statements involvesthe financial information. volumes ofadditional supplementary and three province’s consolidated financial statements,the including province’s annualreport, comprise the cial Administration Act Minister ofFinance,of the asrequired by the 31ing onMarch direction are prepared underthe PublicAccounts fiscal foryear end each Ontario’s Introduction Chapter 2 Chapter The province’s 2012/13 also annualreport these consolidated financial Our Officeaudits responsibility forThe government’s preparing

(Act). ThePublicAccounts Province Public Accountsofthe Finan - - - - of the government to both the Legislative government toAssembly the of the both fiscalaccountability enhancesthe information such fiscal ment accomplishedyear. inthe Providing 2013, govern includingsomedetails ofwhatthe the 31,and fiscal results year ended March for 180 fiscal year. end ofthe days ofthe three The Lieutenant to Governor the report inCouncilwithin government deliver the circumstances, itsannual financial statements. province’smation presentedconsolidated inthe infor PublicAccounts for the the consistency with andinVolumesprovince’s annualreport 1 and2of following:Accounts consist ofthe public. and the • • • The Act requires except that, inextraordinary the in the information Our Office reviews Public volumes ofthe supplementary The three

recipients. payments to vendors andtransfer-payment audited financial statements; and cial statements, aswell miscellaneous asother province’s consolidated finan included inthe and commissionswhoseactivitiesare boardssignificant provincial corporations, financialinformation; other liabilities,itsloansandinvestments,other and province’s revenue itsdebtsand and expenses, providing a numberofschedules details ofthe Volume ofministry 3—detailed schedules Volume 2—audited financial statements of Volume and 1—statements from allministries - - - respect to the government’s management of the management government’s respect to ofthe the able reflection ofwhathasactually transpired with or deficitasbeingafair, consistent andcompar annualsurplus reported publiccanrely onthe the legislators asPSAB’s and ent standards such sothat independ financial statements inaccordance with continueto prepare its Ontario ally that important this view. hold statements. Ialsofirmly Itiscritic province itsconsolidated financial inpreparing for most appropriate usebystandards are the the cial statements andannualreports. completeness province’s consolidated finan ofthe to improve and continuedefforts clarity in their the (PSAB). Successive governments have beendiligent PublicSector Accountingstandards ofthe Board respects, have consistently the complied with consolidated financial statements, inallmaterial toIt isimportant acknowledgeprovince’s the that and constructive. working relationship hasalways beenprofessional issues,our always onfinancial reporting agree Provincialof the Controller. Whilewe mightnot Office the andparticularly with Finance (Ministry) Accounts of we work Ministry the closely with ber 10, 2013, legislated meeting deadline. the volumes, onSeptemAccounts supplementary Public Financial three Statements, the alongwith 2012/13ince’s andConsolidated AnnualReport 10within timeit resumes sitting. days ofthe lay Assembly itbeforetion publicandthen the Assembly isnot informa if the insession,make the Legislative beforemust lay Assembly the them or, Lieutenant Governordocuments, the inCouncil fiscal year. endofthe of the these Upon receiving Lieutenant 240 Governor days inCouncilwithin volumes must besubmitted to the supplementary Summary My predecessor PSAB haspublicly stated that Public In conductingourannualauditofthe This year, prov government released the the

------financial statements. However,the the future ifin any province’s impact consolidated material onthe timedoingsohasnotcases are not, had butatthis Standards Board (AcSB) standards andinother PSABcases are consistent andAccounting with that insome specific accountingpractices establish introduced legislationonanumberofoccasionsto publicsector. forappropriate the has Ontario consensus onwhataccountingstandards are most regard ongoing challengesandinreaching inthis PSABible andconsistent has financial reporting. by to stakeholders to contribute cred ensure they and consistent andinterpretation understanding individualfinancial statements. their province’s public-sectorto entitiesinpreparing the shouldbeapplied Thissameprinciple public purse. pleased to report that the Independent Auditor’s IndependentAuditor’s the that pleased to report province’s consolidated financial statements. Iam ofthe results ofourexamination annually onthe The chapter.this own. their establish governments will that if PSAB risk isto reduce the standards. Thesestandards must meet userneeds public-sector most appropriate accounting on the point for developinga goodstarting aconsensus set upto review PSAB’s conceptual framework is publicinterest.sector force Thetask entitiesinthe issuesfacedby governmentsreporting andpublic- must work together ifwe are to resolve financial to my Office. ing treatments, concern itcouldbecomeagreater legislated account government introduces further Statements Consolidated 2012/13The Province’s Accounting standards needto facilitate clear These issuesare discussedinmore detail later in Standard-setters, governments andauditors Auditor General Act

Public Accounts of the Province requires we that report Financial

- - 25

Chapter 2 Chapter 2 26 reads asfollows: 31,ended onMarch 2013, It isfree ofreservations. consolidatedthe financial statements year for Legislative province’s to AssemblyReport the onthe 2013 of Ontario of the OfficeGeneral of the Auditor Annual Report ments are misstatement. free from material consolidated financial state the whether audit to reasonable about assurance obtain the requirements andplanperform dards require Icomply ethical that with accepted auditingstandards. Thosestan Canadiangenerallyin accordance with based onmy audit.Iconducted my audit consolidated financial statements on these My responsibility isto anopinion express Responsibility Auditor’s orerror.fraud dueto misstatement,from whether material that aresolidated free financial statements preparation ofcon to enablethe necessary Government determinescontrol is asthe internal accounting standards, andfor such Canadianpublicsectorin accordance with consolidated financial statements of these preparation andfairpresentation for the is responsible The Government ofOntario dated Financial Statements Management’s the Responsibilityfor Consoli information. explanatory of significantaccountingpoliciesand other flowthen endedandasummary the year for change inaccumulated deficit,andcash changements ofoperations, innet debt, 31,March 2013, consolidated state andthe dated statement offinancialpositionasat consoli comprise the which of Ontario, the Provincedated financial statements of I have accompanying audited the consoli of Ontario To Legislative Province Assembly the ofthe Report Independent Auditor’s ------statements fairly province’s fiscal results present the consolidated financial the indicates that tion, which The above any reserva auditopinioniswithout control. Anauditalsoincludesevaluating internal entity’s effectiveness ofthe on the anopinion ofexpressing purpose for the butnot circumstances, inthe appropriate order are to designauditprocedures that consolidated financial statements in of the preparationentity’s andfairpresentation control relevant internal toconsiders the auditor assessments,the risk making those duetoments, whether orerror. fraud In consolidated financial state ment ofthe misstate ofmaterial risks assessment ofthe judgment,includingthe auditor’s on the statements. Theprocedures selected depend consolidated financial and disclosures inthe amounts auditevidence aboutthe obtain An auditinvolves procedures to performing August 14, 2013 Toronto, Ontario [signed] standards. Canadianpublicsector accounting with flowsthen endedinaccordancethe year for in itsaccumulated deficit, anditscash changeoperations, initsnet debt,change 2013 consolidated results ofits andthe 31, asatMarch Provinceof the ofOntario consolidated financialposition respects, the statements present fairly, inallmaterial In myconsolidated financial opinion,these Opinion provide abasisfor my opinion. issufficientandappropriate obtained to auditevidence IhaveI believethe that consolidated financial statements. overall as evaluating the ofthe presentation estimates Government, made by aswell the reasonableness ofaccounting used andthe ofaccountingpolicies appropriateness the

Acting Auditor General Peall, CPA, LPAGary CA, - - - misstated items individually orcollectively exceed of government orrevenue expenses year. for the If greater tions, we at0.5%ofthe threshold set this provincial most other and consistent jurisdic with Thisyear, threshold. materiality asinpast years sion material. we error,then considerthe misstatement oromis the answerdated financial statements?” If is yes, province’s consoli decisions madeby ofthe users that itcouldaffect or omissionsignificantenough error,we question “Isthis askthe misstatement onourprofessional judgment. Essentially,primarily (insignificant)isbased (significant) andimmaterial statements. Anassessmentofwhatismaterial overallin relation to the consolidated financial unrecorded, misstated orimproperly discloseditem the orsignificanceof we materiality considerthe province’s consolidated financial statements. have not beendisclosedproperly notes inthe to the been recorded, have not beenrecorded properly or transactions means significantfinancial have not areservation Anauditopinionwith a reservation. would berequired to issueanauditopinionwith recommended PSAB accountingstandards, we CPA compliance government’s with the Canada’s year.the reflection ofwhathasactually transpired during andprovideany afair orsignificanterrors material consolidated financial statementsince’s donot have We prov the are alsocommunicatingto that users tified Management Canada[CMA].) Accountants andCer Institute Accountants [CICA] ofChartered 1,January 2013, mergerCanadian by the of Accountants ofCanada.(CPA Canadawas created for Professional governments Chartered by the accountingstandards recommended ance with financial statements have beenprepared inaccord province’s consolidated the sonably concludethat basedonourauditwork,means that, we canrea 31,at March 2013. This“clean”auditopinion 2012/13for the fiscal year anditsfinancialposition To assessment,we assist inthis calculate a isneeded, areservation In determining whether If we were to have with significantconcerns ------ces that recognized these enhancements: recognized these ces that Institute onfederal andprovincial practi reporting 2013 C.D.HoweFebruary from the commentary cial Statements, sowe were most pleasedto seea andConsolidated Finan AnnualReport Ontario’s usefulness,readability andtransparency of the Provincialthe Controller over years to enhance the required year. this be required. However, was reservation nosuch would Report our IndependentAuditor’s normally in requiredmake the adjustments, areservation andmanagement threshold, isnot willingtothe rowing to financelarge deficits andinfrastructure to continuedgovernmentwas attributable bor We province’s growing debtburden noted the that debt—total debt,net debtandaccumulated deficit. measureswe different ofgovernment discussedthe debtburden. Inour2 Ontario’s In ourpast two we AnnualReports, commented on 1993/94standards inthe fiscal year. provincesince the moved to adopt PSAB accounting province’s consolidated financial statementsthe have beenableto issue“clean”auditopinionson approved accountingstandards. Accordingly, we power, have complied respects with inallmaterial in governments, politicalparty regardless ofthe past 20years, allOntario inthe it isnotable that Condition Financial The Province’s We have beenworking Officeof the closely with As afinalcomment, that we wish to pointout accounts documents and making an effort accounts documentsandmakinganeffort wayleading the inpresenting clearpublic are andNew Brunswick ments ofOntario The federal govern government andthe spending figures. budgeted andyear-end revenues and to compare deviations from andexplain

Public Accounts of the Province 011 AnnualReport

- - -

- 27

Chapter 2 Chapter 2 28 or exceed itsbudgettargets. deficit province outlookupward ifthe isable tothis meet costs. However, itcouldrevise itdidindicate that province’s aggressive plansto ability ofthe contain debt levels achiev anditsdoubtsregarding the year, next the ing within province’s high citingthe it would lower credit province’s rat long-term the that chance was aone-in-three there projected that it still faceslarge deficits the next overfew It years. tinues to have alarge andwell-diversified economy, con S&Pnoted ratings. whileOntario existing that their Budget, agencies rating allthree confirmed strong, helpingto borrowing costs. contain Investor debthadremained demandfor Ontario borrowing costs. a significantimpact onOntario’s was asyet changes rating have noevidence that had We developments hadnoted despite these there that cost offuture government the borrowing. affecting thus debtsecurities, ofholdingthese risk the offset require they returns credit to to ratings assessthe that investorsfied debtobligations and these use respect to speci with creditworthiness a borrower’s We acredit isanassessmentof rating that explained province’s ofAA(low). rating the had maintained credit-rating agency, DBRS,athird tofrom Aa2. Aa1 creditInvestors rating lowering Ontario’s Service AA- credit anegative rating outlookandMoody’s Standard (S&P)givingOntario’s with andPoor’s government released its2012the Budget, Ontario after province’s credit shortly rating ment ofthe credit-rating agencies assess hadupdated their alarge debtload,including: carrying negativetions, andhighlighted the consequences of jurisdic toratio Canadianandinternational other spending. We net-debt-to-GDP compared Ontario’s 2013 of Ontario of the OfficeGeneral of the Auditor Annual Report • • • Shortly after the release of the 2013 release ofthe the after Shortly Ontario In our

would likely increase borrowing costs. and needed for programs; other potential credit-rating downgrades, which to vulnerability greater interest rate increases; costs crowding outfunding debt-servicing 2012 AnnualReport we noted two that ------deficit was $9.2 billionorsome$5.6less. 2012/13 inits2012 Budget. Ontario Theactual The province projected a$14.8 billion deficit for inour examined indicators, last province’s “financialhealth” the 2013and Fiscal Review tially unchanged inthe 2013forecast inthe Budget Ontario (andsubstan However, decliningbutstill significantdeficits with adeficitlower hasreported than Ontario forecast. consecutive year that fourth the ments marks andConsolidated FinancialAnnual Report State There are several improvement: reasons for this MARCH FINANCIAL • • 31, province’s March The release ofthe 2013, and lower lower borrowing becauseofthe reflecting lower-than-forecast interestrates ment; $0.3 billioninlower interest expenses andgeneral particularly govern health tries, from minis reduced spendingacross allother board $1.3 expenses; school other billion and from reducing retirement and gratuities dayselimination ofbanked for sick teachers, to one-timesavings of$1.3 billion from the due in education-sector primarily expenses forecast. There was a$2.2 billiondecrease lower federal government. transfers from the byrevenue, $0.1 offset partially billion in anda$0.2 billionimprovementprises inother in incomefrom government businessenter There was alsoa$0.4 billionincreasegrowth. due to slower-than-expected economic revenue incometax lion decrease inpersonal includinga$0.5 bil sources oftaxation, other by $1 billioninpooreroffset results from the revisions years. Thiswas for partially prior revenue tax poration related to assessment tax to anunexpected $1.3 billion increase incor Taxation revenue was $0.3 billionhigherdue Expenses were $4.8 billionlower than Revenue forecast. was $0.8 billionhigherthan 31, PERFORMANCE 2013 2010 AnnualReport ), we believe anupdate on Ontario EconomicOutlook Ontario AT , is warranted. , iswarranted.

------Sources of data: 2012/13Sources ofdata: and2013 Statements ConsolidatedFinancial ProvinceofOntario Budget Ontario Revenue andExpenses,2008/09–2017/18Figure 1:Ontario ($billion) budget ordeficitshouldbehow much surplus manner.debt inasustainable continuously monitor andtake actionto manage its lion, orover 20%.Thegovernment willneedto net debtwillhaveOntario’s increased by $52 bil lion, orover 15%. period samethree-year Over the debt willhave increased by anadditional $42 bil year ofadeficit total forecast remaining, Ontario’s fiscal By2015/16,years. three next onemore with projections past sixfiscal the along with overyears, province’s debtlevelsvides details onthe for the fund investments Figure ininfrastructure. 2pro debtand these deficits, replacematuring finance provincethe must still increase itsborrowing to 2017/18, asillustrated inFigure 1. four years before in beingableto balanceitsbooks The government the next isprojecting deficits for 31,at March 2013. rose to $281.1 billion andnet debtto $252.1 billion investments,infrastructure province’s total the debt Note: Numbers may notaddupdue torounding. Note: Numbers Surplus/(Deficit) Reserve Total Expense ondebt Interest expense Program Expense Total Revenue PLAN PERFORMANCE—THE PROJECTED Ultimately, question ofwhatOntario’s the While annualdeficitsare projected to decrease, annualdeficitand becauseofthe Primarily budget reserve. deficit; and$1 billionsaved by not usingthe FINANCIAL 2008/09 103.9 95.3 97.5 (6.4) 8.6 — 2009/10 115.6 2013 BUDGET 106.9 (19.3) 96.3

8.7 — 2010/11 Actual 121.2 (14.0) 107.2 111.7 9.5 — 2011/12

122.8 109.8 (12.9) 112.7 10.1 - - - — 2012/13 122.6 113.4 2012/13 fiscal year. To providethe legislators and the timeanditsfinancial results for position atthat ments provide province’s financial asnapshot ofthe 31,The March 2013, consolidated financial state future generations. level and ofthe debtoncurrent sustainability the impact and financial condition,andconsider required province’s to protect the andpreserve theythe decisions whomust makefinances. Itis province’s state ofthe current the understand government,publicbetter legislatorsthe and ment policy. Thisanalysis ispresented solely to help government shouldincurisoneofgoverndebt the nomic circumstances. Itsfinanceswillalsobemorenomic circumstances. have lessflexibility to changing eco respond to government will andthe become lesssustainable few province’s next Over debtwill years, the the over few next years, itremains the challenging. itisprojected toalthough remain relatively stable global economicdownturn in2008/09,and the condition hasnot improved significantly since ability, andvulnerability. flexibility PSAB-recommended financialindicators: sustain usingseveral financial health government’s the amore completepublic with picture, we assessed 112.3 10.3 INDICATORS ONTARIO’S FINANCIAL (9.2) — Our analysis indicates that Ontario’s financial Ontario’s Our analysis indicates that 2013/14 116.9 127.6 Plan Medium-termOutlook (11.7) 1.0 117. 10.6 1.0 2014/15

129.5 120.5 (10.1) 118.3 11.1 1.2 Public Accounts of the Province 2015/16 124.9 131.0 118.8 12.2 (7.2) CONDITION 1.2 2016/17 Extended Outlook 132.1 130.1 118.8 13.4 (3.5) 1.5

2017/18 132.4 134.4 118.0 - 14.5 0.5 1.5 - - - 29

Chapter 2 Chapter 2 30 assets includecash,accountsreceivable, temporary sions andtransfer payment obligations. Financial includingdebt,accounts payable, pen nal parties, consist ofallamountsagovernment owes to exter total liabilities anditsfinancialassets. Liabilities Net between difference debtisthe agovernment’s annual revenue. of net debtto GDP, ofnet andratio debtto total commitmentsanddebtburden. program ability togovernment’s manage itsfinancialand operates. provides Sustainability insightinto the burdenor tax relative economy it to the inwhich employees debt increasing the andothers—without public anditsfinancialcommitments to creditors, commitmentsto the cial obligations—its service agovernment finan itsexisting which canmaintain asdefined Sustainability the degree to by PSAB is sections. delivery.program actingasaconstraint onfuturecontinue to rise, The province’s debtburden andinterest costs will federal tovulnerable government. decisionsofthe 2012/13Sources ofdata: and2013 Statements ConsolidatedFinancial ProvinceofOntario Budget Ontario Figure 2:Total Debt, 2013 of Ontario of the OfficeGeneral of the Auditor Annual Report b. a. 3. 2. 1. Accumulated deficit Net debt Total debt Ratio of Net Debt to GDP Debt to Ratio of Net Sustainability

2012/13 Statements ConsolidatedFinancial ProvinceofOntario 2013 Budget Ontario assets. tangible capital non-financialassets,suchasits government’s deficit Accumulated debt Net T-bills andU.S. commercialpaper. debt Total There are two key indicators: sustainability ratio We following elaborate onouranalysis inthe is the difference between the government’s total liabilitiesanditsfinancialassets. total government’s betweenthe difference isthe represents the total amount of money the government owes to outsiders and consists of bonds issued in public capital markets, non-public debt, markets,non-publicdebt, ofbondsissuedinpubliccapital andconsists owestooutsiders government amountofmoney the total represents the represents the sum of all past government annual deficits and surpluses. It is derived by taking net debt and deducting the value of the the the valueof taking netdebtanddeducting annualdeficitsandsurpluses.Itisderived government by sumofallpast represents the 2007/08 1 Net Debt Net 156,616 105,617 162,217 a 2008/09 2 and Accumulated Deficit, andAccumulated 169,585 113,238 176,915 a 2009/10 193,589 130,957 212,122 Actual a 2010/11 - - 236,629 144,573 214,511 - 3 2007/08–2015/16 ($million) ment’s financialpositionasitprovidesment’s insightinto ness enterprises. investments andinvestments ingovernment busi ture stimulus spending since that time. Ontario time.Ontario ture stimulus spendingsincethat significantly to fundannualdeficitsandinfrastruc government hasincreased itsborrowingand the provincialthe economy. Tax revenue fell abruptly, 2008globalimpact economicdownturn on ofthe the reflectinghas beentrending upward sincethen, high of32.2%,to 26.2% in2007/08. However, it fell gradually GDP ratio since1999/2000, from a an increasing burden. debtsare becoming ing—it meansagovernment’s wordseconomy isris ratio other isrising—in the outputofan be repaid relative valueofthe to the economy.on the must amountofdebtthat Ifthe debt to burden GDPisanindicatorofdebt ofthe economy. inthe income earned ofnet Theratio by aneconomy. Itisalsoequal sumofall to the produced total valueofallgoodsandservices the andpublicservices. grams devote future financial resources existing proto net-debt abilityto positionreduces agovernment’s to pay down liabilities.Alarge agovernment’s of future provincial revenue willberequired that Essentially,services. the amount net debtreflects ofcontinuingto affordability providethe public a Net measure ofagovern debtisanimportant Figure province’s net-debt-to- 3shows the that Gross domestic product (GDP)isameasure of 2011/12 158 235,582 257,278 ,410 a 2012/13 252, 281,065 6,32 167,13 088 b 2013/14 290,853 179,935 272,810 a 2014/15 Estimate 190,100 290,100 308,100 a 2015/16 303,900 323,800 197,300 - - - - - a 20 25 30 35 40 45 Quebec has a significantly higher ratio than Ontario. than Ontario. Quebec hasasignificantlyratio higher governmenteral are roughly and similarto Ontario, provinces fed Maritime and the whilethe Ontario, a significantly lower net-debt-to-GDPthan ratio in Figure 4.Generally, western provinces the have respective ofnet ratios debtto GDP, isillustrated federal their government,and the alongwith Thenet debtofmost provincesadian jurisdictions. net debtto GDPisto Can compare other itwith net-debt-to-GDP abovethe rises ratio 60%. when tois vulnerable unexpected economicshocks risk and isat fiscalhealth ajurisdiction’s tend that longer-term projections are met. con Many experts years,over andwillimprove three next the only if provincial willbelesssustainable net debtgrowth itto expects government beginfalling.Thus, then a highof40.2%in2015/16. the peak, this After over $303 billionby 2015/16. almost doubledfrom $157 billion in2007/08 to net debtwillhavegrowing debt.Infact,Ontario’s toexpects continueto incurlarge deficitsanda in2005/06. Note: Netdebtincludesbroader-public-sectornetstarting and2013Statements Budget Ontario March31,Sources ofdata: 2013 ConsolidatedFinancial ProvinceofOntario (GDP), 1999/2000–2017/18 (%) Product Domestic Gross Debtto Figure 3:RatioofNet 10 15 0 5 A useful exercise in assessing Ontario’s ratio of of ratio A usefulexercise inassessingOntario’s The net debt-to-GDP isprojected ratio to reach 1999/2000 2000/01 2001/02 2002/03 2003/04 2004/05 2005/06 2006/07 2007/08 2008/09 2009/10 2010/11 2011/12

2012/13 Projected Actual 2013/14 2014/15 2015/16 - 2016/17 -

- 2017/18 eral ofOntario eral and 2013the AuditorGen budgets ofprovincialjurisdictions;andOffice 2013 Statements; Consolidated Financial Budget;budgetupdates Federal 2012/13Sources ofdata: and AnnualReport ProvinceofOntario Canadian Jurisdictions,2012/13 Ratiosof DebtandtheNet-debt-to-GDP Figure 4:Net recession level of27%. net debt-to-GDP reducing pre- the to ratio then the annualdeficit toby 2017/18 eliminatingthe and action.”priate early are notquickly appro ifthey headed off with government to debtburdens pointoutthat canrise province’s debtpicture, innoting only the alarmist United States, 131% inJapan,and81% inFrance.” debt burdens have andthe 73% reached inBritain (2001), Japan(1997) andFrance (1993)…Today, United the (2004), Statesof GDPincludeBritain 35% net current debtwas oncesimilarto Ontario’s recently.” For example, hewrote, “…nationswhose atonetimeand,insomecases,surprisingly tries] “so,however,warned, were coun many of[these news overdominated the past two the years,” he that have dreadful fiscalconditionofcountries the province longwaydictions, andthe is“avery from small compared ofmany to juris international that debtisrelativelymond noted whileOntario’s that PublicServices ofOntario on the Reform ON Federal NS PEI NB MB BC SK AB QC In its2013 Budget, government committed the added:“We donotDrummond meanto be 2012In hisFebruary ofthe report (Net Assets) 252,100 176,575 ($ million) 671,363 Net Debt/ (14,604) Public Accounts of the Province 13,954 15,893 38,136 11,054 5,109 1,971 Commission Net Debt , DonDrum GDP to 35.8 36.9 36.7 33.9 26.8 49.4 17 37.4 (4.7) 6.6 .0 (%) - - - - - 31

Chapter 2 Chapter 2 32 300 2008,2009, 2013Statements; Budgets Ontario March31,Sources ofdata: 2013 Consolidated Financial ProvinceofOntario Annual Revenue, 1999/2000–2015/16 (%) ofTotal DebtasPercentage Figure 5:RatioofNet end offiscal2015.” notedthat either S&Pfurther 250% ofconsolidated revenues operating by the burden ofaround projection ofatax-supported trending materially above [its]base-casescenario growing debtburden be becauseof“Ontario’s increasing(thereby itscost ofborrowing), itwould it were year next rating to downgrade Ontario’s 2013 Budget. Ontario Theagency noted if that government its tabled the ments publishedafter province’s net debthaslessrevenue it. to support net debtto total annualrevenue alsoindicates the to top 240% by 2015/16. Thisincreasing of ratio increased steadily since2007/08 andisexpected cial revenue was increasing. However, has ratio the same,annualprovindebt remained essentially the the province’s net that, while the fact reflecting 200% in1999/2000 to about150% in2007/08, As shown inFigure declinedfrom ratio about 5,this provincialthe debtifallrevenue was devoted to it. itwouldthat take two andahalfyears to eliminate devoted to it.For of250%indicates aratio instance, province’s debtifallrevenueeliminate the couldbe indicator ofhow timewould much beneededto ofnetThe ratio debtto total annualrevenue isan 200 250 2013 of Ontario of the OfficeGeneral of the Auditor Annual Report 10 15 50 Ratio of Net Debt to Total Debt to Ratio of Net Annual Revenue 0 0 0 Of interest are May S&P’s 2013 review com 1999/2000 2000/01 2001/02 2002/03 2003/04 2004/05 2005/06 2006/07 2007/08 2008/09 2009/10 2010/11 2011/12 Projected Actual 2012/13 2013/14 - 2014/15

2015/16 - is ableto bear. economy publiciswillingandthe the limitsthat the measuresity to infuture asitapproaches levy these or government abil government’s fees reduces the Similarly,economic circumstances. increasing taxes future abilitytogovernment’s respond to changing borrowing reduces the cial obligations. Current change its debt or burdentax to meet existing finan agovernment to degree which can isthe Flexibility unplanned debtgrowth. this inabilityto rein inspendingcouldtrigger ment’s govern orthe economicgrowth than-projected from lower-economic orfiscalpressures arising 45%, from $212 billion to over $308 billion. borrowing isexpected to increase by $96 billion,or 2009/10 to 2014/15, evenprovince’s total asthe expected to holdsteady atapproximately 9%from estexpense-to-total-revenue have ratio heldandare actualandprojected inter the decade,both of this rates have lows early beenathistoric sincethe part due to alower interest rate–environment. Because decadeendingin2007/08.in the Thisismainly expense-to-total-revenue decreased ratio steadily orlessersharea greater oftotal revenue. past borrowing takes extent servicing to which the spending. program lesswillbeavailablefrom pastfor borrowing, the ment revenue neededto pay interest costs arising ofgovern can provide. proportion Thehigherthe government that andservices quality ofprograms quantity and candirectlyest expense, the affect total debt,orinter cost ofservicing Increases inthe ment ismanagingitsfinances. ibility indicators to helpassesshow well govern the Flexibility Ratio of Interest Expense to Revenue Expense to Ratio of Interest In the followingIn the section,we two examine flex As Figure 6shows, province’s interest- the The interest-expense-to-revenue illustrates ratio ------ment borrowing decisions mean a growing portion ment borrowing decisions meanagrowing portion Past economiccircumstances. to govern changing government willhavethe lessflexibility to respond toexpense revenue beginningin2015/16 indicates to pay province’s debt. interest outstanding onthe willberequiredvices becauseahigherproportion inusing its revenueflexibility to provide publicser government willhave the rise, considerably less increased significantly. However, ifinterestrates relatively steady even asitstotal borrowing has government to keep itsannualinterest expense atrecordare currently low levels, enablingthe interest-rate Asdiscussedabove, risk. interest rates province’s debt. of revenue collected was the required to service debt. In2007/08, 12 only oneoutofevery dollars its ofrevenuenine dollars collected onservicing toexpects have to spend nearly oneoutofevery by 2017/18This meansthat government the total debtisexpected to bearound $340 billion. by 2015/16 to 11% andfurther by 2017/18, when isexpected ratio to increase to gradually the 10% 18 2008,2009,2013Statements; Budgets Ontario March31,Sources ofdata: 2013 ConsolidatedFinancial ProvinceofOntario 1999/2000–2017/18 (%) Revenue, Expenseto Figure 6:RatioofInterest 12 10 14 16 0 2 4 6 8 The expected increasing ofinterest ratio The province’sitto debtalsoexposes significant latest government’s projections, Based onthe 1999/2000 2000/01 2001/02 2002/03 2003/04 2004/05 2005/06 2006/07 2007/08 2008/09 2009/10 2010/11 2011/12 2012/13 Actual 2013/14 Projected 2014/15 2015/16 2016/17 - - 2017/18 remain in that range.remain inthat GDP hasranged from 13% to 14.6% andisexpected own-source revenueernment’s as apercentage of From 2005/06fiscal year the to 2012/13,the gov infuture flexibility taxesraise to orincrease fees. government may the have isrising, ratio the less sources. userchargesIf orother taxation, through a government revenue economy istaking outofthe and fee revenue—to GDPshows extent to which the ofown-sourceThe ratio tax revenue—primarily future government programs. of revenue willnot beavailable and for current proportion of revenue that the Ontario government Ontario ofrevenue the proportion that downturn. Thisfunding endedin2010/11, andthe 2008globalfunding to economic address the source revenue andfederal–provincial stimulus in 2010/11, largely result ofadrop inown- asthe 2005/06, whenitwas 14.7%, to apeakof 22.2% ment transfers to revenue since rose inOntario As shown inFigure 7, offederal ratio govern the vulnerability: decisions ofothers. the the more itbecomes vulnerable to finances and government hasover lesscontrol the the its reliance onoutsiderevenueThe higherthe sources, levels astransfers from other such ofgovernment. are beyondance onfundingsources that itscontrol, measures provide insightinto reli agovernment’s to creditors, employees Vulnerability andothers. publicandfinancialcommitments ments to the impair commit itsabilityto meet service existing could that tosources risks orisexposed other becomesdependentonoutsiderevenueernment Vulnerability agov to refers degree which to the Vulnerability Revenue GDP to Ratio of Own-source Revenue Total to Government Transfers Ratio of Federal There isonekey indicator for Ontario’s Public Accounts of the Province - - - - - 33

Chapter 2 Chapter 2 34 sion paper, New Controller Zealand’s andAuditor now future. InaMayboth andinthe 2013 discus andfinancialcommitments delivery to meet service generations—in words, other ability a government’s placinganexcessivewithout burden onsuccessive of agovernment to financeitsdebtobligations the capacity fiscalsustainability refers Long-term to ence onfederal transfers to fund programs. revenue andreduce ratio depend willdecrease this spending. Conversely, any increase inown-source dependence onfederal transfers to fundprogram again, ratio indicatinggreater will increase this same,anyremain drop the inown-source revenue projected spendingplans.Even iffederal transfers taxesdebt orraise orfees ifitwishesto its maintain provincecould result inthe having to issuemore unforeseen reductions infuture federal transfers a numberofitsownchallenges, fiscal andany federal rate,government the isfacing remain atthis decreased to 19%. province itto Whilethe expects received government ofCanadahassince from the 2013/14.to beflat-linedat 19% after revenueareassumed tototal transfers government offederal Note: Theratios 25 2008,2009,2013Statements; Budgets Ontario March31,Sources ofdata: 2013 ConsolidatedFinancial ProvinceofOntario Total Revenue, 2005/06–2017/18 (%) Figure 7: to Government Transfers RatioofFederal 20 2013 of Ontario of the OfficeGeneral of the Auditor Annual Report 10 15 0 5 LOOKING

2005/06 2006/07

AHEAD 2007/08 2008/09 2009/10 2010/11 2011/12 2012/13 2013/14 2014/15 Actual Projected 2015/16 2016/17 - 2017/18 - orate further if these government restraint ifthese targets orate further years. Theprovince’s financialconditionwilldeteri 2015/16 level 2016/17 for the and2017/18 fiscal spendingisforecast Program to remain atthe 2013/14, 1.1% in2014/15 and0.4%in2015/16. spendingincreasesto to holdprogram 4.2%in Specifically, economicgrowth. with rises itplans 2017/18 by restraining spendingwhilerevenue acomplex review. such forming of financialconditionisjust afirst step inper between indicators Ourreview them.” ofOntario’s connections ofpublicspending,andthe drivers underlying social,environmental andeconomic requires “anincreasing the focus onunderstanding government isabletothe itselffinancially sustain notedGeneral to whether fully that understand in recent years. spendinghasalready beenrestrained program that risk givencannot beachieved. Thisisasignificant The improvement net-debt-to-GDP inOntario’s to-GDP in2012/13 ratio was actually 37.4%. projected to be41%. net-debt- In fact,Ontario’s forGDP ratio 2012/13 in2010 asreported was its 2010 plan.For net-debt-to- example, the indicators have generally improved relative to Annual Report reviewedGeneral inits statistics these adecade. more than previousfell timein year first from the for the total spending reported spendingandprogram 1%.Inaddition,in2012/13,held to lessthan year-over-year spendingwas inprogram growth year second was inarow the for which relative to its2010 plan.The2012/13 fiscal managing down itsnet-debt-to-GDP trajectory has beensuccessfulincontrolling costs and economy. andstimulate the programs Ontario worldments around the took steps to maintain wakeglobal recession, governIn the of The government by plansto balance itsbooks Since the last time the Office of the Auditor Officeof last timethe Since the MINISTRY , Ontario’s financialcondition , Ontario’s RESPONSE 2010 - - - the WSIB in the government’s financial reporting financial WSIBreporting government’s the inthe WSIB Including trust. the was like operating atrue factors, we other and various questioned whether however, given itssignificant unfundedliability WSIB’s was basedonthe classificationasa“trust”; remained viable.Excluding itsfinancial results might have to provide WSIB fundingto ensure the province the was any that particularly ifthere risk province’s consolidated financial statements,the WSIB’s exclusion ofthe financial the results from mitments to provide worker benefits. being unableto meet andfuture itsexisting com viability, WSIB ofthe ultimate risk includingthe WSIB’s unfundedliabilityposedto the financial the andmagnitudeof growth the that risk cussed the injured workers. Inour estimated financial obligations to pay benefits to WSIB’sbetween valueofthe assets andits the WSIB’s unfundedliability, difference isthe which the growth in significant aboutthe of concerns premiums onemployerthrough payrolls. receives nofundingfrom government; itisfinanced toassistance workers job.TheWSIB injured onthe isto andmedical providepurpose income support Safety andInsurance Act,1997 created by corporation the is astatutory The Workplace Safety Board and Insurance (WSIB) Safety Update essentially unchanged. federal-transfers-to-revenuewhile the is ratio expense-to-revenue have ratios alsoimproved, net-debt-to-revenue ofthe jectories andinterest- avoiding $22.2 billionindebt.Similarly, tra the the pasttargets deficit four and of years ineach trajectory isadirect result ofbettering its We alsourgedgovernment to the reconsider past decade,weOver the have anumber raised and on the Insurance 2009 Annual Report 2009 AnnualReport Workplace (Act). Its primary (Act). Itsprimary Board Workplace

we dis

- - - WSIB with the followingWSIB the key with elements: ticular callingfor anew fundingstrategy for the anumberofrecommendations, inpar contained The May 2012 Arthurs by Professor report Harry workplaceof Ontario’s safety system. andinsurance how financialviability tolong-term best ensure the independent fundingreview to provide adviceon fiscalperformance. ment’s would havethe govern asignificantimpact on lowed in measuring progress against the plan. The lowed plan.The against the progress in measuring accountingpolicies toregulation asto befol the ofthe itsinterpretation ofLabour to clarify Ministry new regulation, and we ofthe interpretation didso. our funding sufficiencyplanisconsistent with theJune 30,2013, whether our Office to confirm a2012with asked TheMinistry sufficiency report. a new fundingpolicyandprovided stakeholders the Minister.ficiency plan to Ithasalsoapproved requirements regulation by issuingasuf ofthe targets. TheWSIB hasto date the complied with measures itwilltake tooutlining the achieve these Ministera planto ofLabourby the June 30,2013, byratios specifieddates: followingensure itmeets the fundingsufficiency 1,Effective January 2013, itrequired WSIB the to government Act. madeanew regulation underthe inJune2012 report, Arthurs the mendations ofthe • • • • • • In September 2010, WSIB the announcedan The WSIB consulted with our Office and the the The WSIB ourOfficeand consulted with The regulation alsorequired WSIB the to submit recom andtoIn response to the ourconcerns

frame andbyframe reasonable measures generate WSIB areasonable time the couldnot within inwhich ping pointbeingdefinedasacrisis (tip a “tippingpoint”of60%fundingratio advice; best actuarial rate basedonthe 90%–110% 20years. within funding ratio sufficient funds to pay workers’ benefits);and moving the WSIBmoving the asquickly asfeasible beyond realistic assumptions, includingadiscount 100% onorbefore December31, 2027. 80% onorbefore December31, 2022;and 60% onorbefore December31, 2017; WSIB toputting the oncourse achieve a Public Accounts of the Province ------35

Chapter 2 Chapter 2 36 Sources of data: WSIB Financial Statements and Fourth Quarter 2012 toStakeholders Quarter Report andFourth Statements WSIBFinancial Sources ofdata: 2010–2012* ($billion) Results andUnfundedLiability, Operating Board andInsurance Figure 8:WorkplaceSafety outcomes andadrop innew claims. improvedbined with recovery andreturn-to-work to higherpremiums andinvestment com returns, $300 milliontocontribute itsinvestment fund,due not only covered costs, butwere current ableto For timesince first the 1997,the premiumsWSIB’s revenue andreduce andclaimscosts. operating to WSIB’s increase result ofthe continuedefforts of almost $1 billionfrom 2011 to 2012 was the environment andinvestment climate. Thedecrease changedthe interest-rateits obligations, reflecting discountrate usedtocant reduction inthe value from 2010 to 2011 from asignifi arose primarily our 2009review. unfunded liabilitysince2010, year following the WSIB’s ofthe results operating and a summary ance, asillustrated inFigure provides 8,which achieved strong perform financialandoperational WSIB’s the requesttimeofwriting. sidering atthe ofLabourwas still con amendment. TheMinistry this suggested assets. with OurOfficeconcurs of the valuation respect to the amendment bemadewith aregulatory that WSIB Ministry hasproposed to the 2013 of Ontario of the OfficeGeneral of the Auditor Annual Report * AsofDecember31. Unfunded Liability theYear Income(Loss)for Comprehensive Legislated obligationsandcommitments expenses andother Administration Loss ofRetirement IncomeFundcontributions Benefit costs Expenses income Net investment Premiums Revenue billion growth in the unfundedliability inthe The $2 billiongrowth 2012 the During calendaryear, WSIB the - - - - commitments to to andprogress date inaddressing over several years. into income beingamortized pension plancurrently WSIB’s lossesinthe net amountofunamortized unfunded liabilitybythe $585 millionto reflect 1,January 2013, willnecessitate anincrease inthe dards (IFRS).Anew standard underIFRSeffective Financial Stanbased onInternational Reporting WSIB itsfinancial results example, the reports to besubjectto uncertainty. considerable For ratios willcontinue fundingsufficiency prescribed 2011, asignificantimprovement. December 31, 2012, from 52.1% asofDecember 31, of assets to liabilities—increased to 56.9%asof ratios andre-evaluateratios ourpositionasnecessary. toward required meeting the fundingsufficiency beingmade will continueto progress monitor the province’s liabilities.However,liability from the we cal year, exclusion ofitsunfunded the andtherefore the 2012/13the for sification of WSIB asatrust fis its unfundedliability, continuedclas we the support 12,438 5,100 4,509 4,714 3,507 1,207 As a result of the government’s and the WSIB’s andthe government’s As aresult ofthe However, WSIB’s the abilityto achieve the The WSIB’spercentage fundingratio—the 2010 (386) 291 227 73 14,222 (1,710) 5,882 5,260 4,172 3,876 2011 228 324 296 70 13,299

4,399 5,520 1,121 1,459 4,061 3,773 2012 328 231 67 - - - Source of data: Pension Benefit Guarantee Fund PensionBenefitGuarantee Source ofdata: Fund Financial Figure 9:PensionPosition, 2007/08–2012/13 BenefitGuarantee ($million) government. The andindependentofthe sustaining PBGF isintended to the beself- that to clarify in $11-million over annualinstalments 30years. isbeingrepaid province that from the in2003/04 existed despite a$330-millioninterest-free loan 31,as ofMarch 2009.These unfundedliabilities $102 31, million asofMarch 2008,and $47 million unfundedliabilitiesof PBGFreported result, the PBGF.had ledto increased claimsonthe Asa economicdownturn in2008 from the primarily insolvencies arising corporate and bankruptcies Figure 9. Inour ofits results isshownand afive-year in summary province’s consolidated financial statements, innotesthe financial positionissummarized to province. accountsofthe However,from the its assets, results liabilitiesandoperating are excluded consolidated financial statements. Thismeansits risk-related fees. fundingbasedonper-memberfinancing, with and covered benefits.ThePBGFisintended to beself- assessments paidby ofpensionplanswith sponsors Act the under conditionsspecifiedin when eligibledefined-benefitplans are terminated pensionbenefits paymentguarantees ofcertain the The Pension BenefitGuarantee Fund (PBGF) Fund ofYear Surplus/(Deficit)–End Fund of Year Surplus/(Deficit)–Beginning Excess/(Deficiency) ofRevenue over Expenses Recoveries expenses Claims andother Revenue Benefit Update In 2009, the government amended the Act government amendedthe In 2009,the the province’s in The PBGFisclassifiedasatrust (Act). The PBGF is funded through annual (Act). ThePBGFisfundedthrough on FundGuarantee 2011 AnnualReport the Pension Pension Benefits we noted that

(102,218) 2007/08 (112,841) 10,623 64,546 75,1

69 — 2008/09 (102,218) (47,351) 123,974 plans were most significant. the of afew Nortel pension large the claims,ofwhich 2009/10 was fully depleted ayear because within $500-millionfundinginfusionin government’s exceeded revenue by $109 million. Inessence,the claims, primarily position becauseannualexpenses, ina$6-millionunfundedliability PBGF was back $500-million cashinfusion,the this withstanding plan windups.However, 31, asofMarch 2011, not financial position and coverthe costs of a number of PBGFto to its help stabilize the $500-million grant limited to itsassets. liabilitiesare PBGF’s the amendments specifythat ment to PBGF. provide orloansto the grants The amendments allow, butdonot require, govern the August 2010 itwouldfollowing: that dothe sustainability,PBGF’s government announcedin the by anestimated 450%. would needto increase itsannualassessmentrates level of$1,000 peremployee, permonth PBGF the maximumcoveragethe current fund benefitsat self-sufficient and continue the long to overterm PBGFto inorder estimated for be the that actuary government to cover expected future claims.The between $680 millionand$1.023 billion from the fundwouldincreased assessments,the require notedstatus inJune2010 absenceof inthe that to andfinancial stability PBGF’s reviewernment the 54,867 69,107 • An independent actuary appointed gov by the An independentactuary 2010,In March government approved the a To PBGFandenhancethe to the risks mitigate the

provided 2010; inMarch build reserves through the $500-million grant $500-milliongrant the through build reserves — 2009/10 150,694 103,343 555,806 406,641 (47,351) (1,529) (109,524) 2010/11 103,343 Public Accounts of the Province 176,671 (6,181) 71 5 67,10 (42) 2011/12 122,318 82,309 40,049 76,128 (6,181) (40) 2012/13 180,037 256,165 201,346 21,309 76,128 - — - - 37

Chapter 2 Chapter 2 38 31,government AsofMarch assistance. 2013, the itioned to cover further requiring itsclaimswithout following: 466/11 1, effective January 2012, didthe which passagewas implemented ofRegulation the with statements to bemateriallythose misstated. dated financial statements was not enough to cause consoli impact PBGFfrom the ofexcluding the statements. However, we the alsoconcludedthat province’s consolidated financial included inthe financial positionandfiscal resultsshouldbe financial statements.a government’s excludedfinanciallyto be independenttrusts from accountingstandard istointent allow ofthe only asthe for accountingpurposes, considered a“trust” financial resources to anorganization, itcannot be government must step to inperiodically provide dependency would continue.Inouropinion,ifthe this likelihoodof government that fundingandthe given to status, retain ahistory its“trust” criteria we PBGFstill didnot meet believedthe the that PBGFonamorefinancial stable footing,place the 2013 of Ontario of the OfficeGeneral of the Auditor Annual Report • • • • • • • • Given these changes, the PBGFis better changes, the Given pos these strategy PBGF The government’s to enhancethe Accordingly, PBGF’s we the concludedthat government hadtaken the steps toAlthough

plans. pensionplancovered;$250 for every and unfunded pensionplans; from $100 to $300; inunfundedpensionplans plan beneficiary from $1 to $5; beneficiaries) and other (active retired members, members beneficiary five and years; plans andbenefitimprovementsthree to from assessments in2012; strengthen pension-funding rules. pension-fundingrules. strengthen for covering eligibilityperiod newextend the futureraise PBGFrevenue by increasing eliminated the exemptioneliminated the for smallpension introduced aminimumannualassessmentof $4 millionassessmentcapforeliminated the maximumannualfee perOntario the raised plan baseannualfee perOntario the raised - - liabilities are limited to Act. itsassets underthe asmentionedearlier,again, although, PBGF’s the once itssustainability downturn couldthreaten the province. economic benefit plansin Another state defined ofmany precarious ofthe and the history remainsPBGF’s given risk considerable the continue to monitor itsaffairs. this time,andwill dated financial statements at province’s consoli PBGFfrom the exclusion ofthe taken andfewer claims.Accordingly, we accept the measures done, hasbeenreduced asaresult ofthe the PBGF,to fundadeficitin asithashistorically province the willhave that The risk ing purposes. for account designationasatrust PBGF’s the with improved financialconditionismore consistent ofover asurplus PBGF reported $250 million.This logical synopsis of these developments:logical synopsisofthese in future hasincreased. Thefollowing isachrono misstatement amaterial of such risk standards, the from PSABcial statements materially departing yet province’s resulted consolidated finan inthe legislated accounting treatments havethese not Public Sector Accounting Board (PSAB). Although followed asthat consultative process such by the anindependent, through than legislation rather precedent to adopt through accountingpractices 2008 AnnualReport public-sector entities. accountingpoliciesforenable itto its prescribe has passedlegislationoramendedregulations to government, several which timesinrecent years Ontario accounting standards. Thisincludesthe applying independently than established rather circumstances accounting treatments incertain adian governments have begunto legislate specific As discussedinour Standards Use While this build-up of reserves is encouraging, isencouraging, build-upofreserves While this We in our practice aboutthis concerns raised of

Legislated Accounting 2012 AnnualReport , warning that itwas atroubling that , warning , someCan - - - -

- • • • • • The was amendedto allow government to the irrespective ofPSAB accountingstandards. government fiscal foryear ofthe that expense actwouldmade underthis berecorded asan solidated financial statements. Any transfers initscon reported from unplanned surpluses vide additionaltransfers to eligiblerecipients regulations allowed government to the pro into revenue by same transfer recipients atthe amountsaretions. Thedeferred to bebrought by transfer recipients contribu asdeferred assets capital toof tangible beaccounted for transfers for acquisitions capital andtransfers deferrals. Standards donot allow (IFRS),which for such adoption Financial ofInternational Reporting plannedCanadian came inanticipationofthe direction to U.S. adoptment’s these rules for govern recognition infuture years; the Hydro OneandOPGto expenses defer current allow rate-regulatedrules as entitiessuch accounting Inc.(OPG).American Generation Power Ontario businessenterprise, ernment same direction to fully another owned gov 2012. Thegovernment has sinceprovided the 1, effective January accounting principles U.S.in accordance with generally accepted to prepare itsfinancial statements enterprise, fully government owned business Ontario Administration Act province’s consolidated financial statements. the whose financial statements are includedin to beusedby any publicornon-publicentity government to specifyaccountingstandards Administration Act cial statements. finan boards their toschool useinpreparing accountingstandards for Ontario prescribe Ontario government regulations nowOntario require In 2011, aregulation underthe 2010/11In the fiscal year,the 2009/10In the fiscal year,the Investing Act,2008, inOntario was amendedto allow the directed Hydro One,a Financial Education Act Financial andrelated ------sincerely hope that this will continue to be the case. willcontinueto bethe this sincerely hope that the past 20 financial statements I years. for ment’s governbeen ableto issue “clean”opinionsonthe Auditor auditopinion.OurOffice has General’s inthe have butto includeareservation nochoice whatitwouldferent than beunderGAAP, we will legislated accountingstandards ismaterially dif under deficitorsurplus reported government’s the ally accepted (GAAP).” accountingprinciples If gener appropriate inaccordance with information PublicAccounts, present fairly inthe reported as consolidated financial statements ofOntario, isrequiredGeneral to the opineon“whether recommended by PSAB. acceptedthose accountingstandards, specifically financial statements inaccordancegenerally with continueto prepare its Ontario that it iscrucial • As the auditor of these statements, the Auditor statements, the auditor ofthese As the To itsfinancialcredibility, maintain we believe

regarding the accountingpoliciesandpracti regarding the to make regulations fullauthority ment with These amendmentsprovided govern the ing the Act(BudgetOntario Measures), 2012 inthe supported was further regulate andrequire treatment. this government to felt area, sothe itprudent this interpreting PSAB standards differently in However,principles. many stakeholders are generallycomplies with accepted accounting underlying andinmost transactions instances the the economic reality of it best reflects accountingaswe this believesupported that related assets.on the We have historically expense recognizerate amortization asthey statements. ces usedto prepare itsconsolidated financial The authority to dictate accountingstandards The authority Financial Administration Act Public Accounts of the Province Strong Actionfor again. , amend ------39

Chapter 2 Chapter 2 40 Source of data: CPA Board Source ofdata: Standards CanadaAuditingandAssurance UnderCanadianAuditing Standards Figure 10: Frameworks FinancialReporting compliance presentation. ora afairpresentation andreflect either purpose, framework now may orspecial begeneral purpose inFigureAs described 10, a financial reporting works preparation offinancial statements. for the provide acceptable frame anumberofdifferent ledgeable offinancial statements. users meaningwas clearto know report’s the ensure that Thisstandard wordingaccounting principles. helped Canadiangenerallyin accordance with accepted financial statements whether were presented fairly indicated reports Canadian standards, most auditor’s Under former report. the independent auditor’s andcontent form ofan address the standards that Canadianmodifications, adopted, appropriate with December 14, 2010. endingonorafter financial statements for periods wereharmonization issuedeffective for auditsof this Canadian auditingstandards reflecting Standardsand Assurance Board (IAASB).New Auditing International standards issuedby the international Canadian auditingstandards with Board (AASB)hasrecently worked to harmonize CPA Auditing Canada’s Standards andAssurance 2013 of Ontario of the OfficeGeneral of the Auditor Annual Report presentation Compliance presentation Fair Auditing Frameworks and Financial Reporting However, new Canadianauditingstandards the AASB recent standards, the Through these • • • • Purpose General

Standards Complies with a non-GAAP accounting framework accounting framework anon-GAAP Complies with ofusers commonneedsofawiderange Meets the (GAAP—full anaccountingframework Complies with ofusers commonneedsofawiderange Meets the (i.e., requirementsoflegislation and/or regulation) PSAB) compliance with Canadian

- - province anditspublic-sector entitiescouldfol were not fairly presented. We alsonoted the that that couldresult infinancial statements cies that for accountingpoli establishing a mechanism itwould that providewarning governments with frameworks, inacceptable reporting expansion this organizations. byestablished industry byestablished law orregulation, orstandards of CPA Canadabutalsoaccountingstandards Accounting(PSAB) Standards or the Board (AcSB) PublicSector Accounting asthe Boardization such standard-settingstandards ofanestablished organ frameworks not only includefinancial reporting financial statements. Acceptable purpose reporting framework asbeingacceptable for general- issues arise. Weissues arise. believe ofgovernment users and ally accepted accountingstandards, anumberof frameworkfinancial from reporting gener departs framework.under that However, legislated ifthe providedreport onfinancial statements prepared work independentauditor’s willnot the affect frame standard-setting that organization, then by standards established anindependent the with byestablished alaw orregulation doesnot conflict reservations. without report anindependentauditor’s standards andstill obtain generallyin accordance with accepted accounting that financial statements general-purpose were not low legislated accountingpoliciesto prepare their In our The standards donot specifyaparticular Generally, framework ifafinancial reporting • • • • • Special Purpose

Complies with a special-purpose framework (i.e., (i.e., aspecial-purposeframework Complies with needsofspecificusers Meets the to Explicit deviationfromanaccountingframework (GAAP aspecial-purposeframework Complies with needsofspecificusers Meets the internal guideline) internal statements offinancial achieve fairpresentation or non-GAAP) 2008 Annual Report 2008 AnnualReport we to readers alerted - - - - - prepared on such abasisare credible,prepared consistent onsuch public-sector organizations. Financial statements financial statements ofall other ments, butthe province’s consolidatedonly financial state the preparation ofnot basisfor the the should form mended by Canadianindependentstandard-setters statements have beenprepared. financial the accountingbasisonwhich the stand public-sector entity financial statements to under notes the to andexamine any reports auditor’s must nowusers carefully wording review the of not require to bespecifically this disclosed.Instead, standards do ing standards, becauseauditreporting entities are not complying Canadian account with ments may not even realize whenpublic-sector consolidated financial statements. have province’s they onthe not affected our report net debtoritsaccumulated deficit. Accordingly, province’s impact deficit,its had amaterial onthe public-sector-entity financial statements have not Canada AcSB Ontario standards for preparing Standards. asallowedreservation, underCanadianAuditing without report anauditor’s provided with them CanadianGAAP.U.S. than rather Theirauditors requires to use ments underlegislationthat them and OPG—alsonow financial state prepare their electricity-sector entities—Hydro One Ontario’s financial statements are “fairly presented.” Two of the that report longer auditor’s astatement inthe legislatedcomply framework. the with There isno statements the indicatingwhether report auditor’s aGAAP frameworkwork than rather andreceive an statements usingalegislative accountingframe boards now school financial prepare their Ontario’s accepted (GAAP). However, accountingprinciples Canadiangenerally was inaccordance with that framework usedareporting entities inOntario aware issues. ofthese public-sector-entity financial statements need to be We believeaccountingstandards recom that However, ofpublic-sector users financial state To from PSAB departures andCPA date, these Until 2010/11 the fiscal year, allpublic-sector ------ship over manage, resources they andthereby the help governments publicly demonstrate steward PSAB standards jurisdictions. are intendedother to against results of actualresults andagainst the public canevaluate expected financialperformance ing standards tothe prepare key financial reports, samesetWhen governments ofaccount usethe and year-end consolidated financial statements. budgets, estimates, printed economicupdates annual use PSAB their standards inpreparing resulting financial statements. ency, credibility and,accordingly, usefulnessofthe utes. Itcouldalsonegatively transpar the affect attrib these accounting standards couldundermine Allowing to preparers to own adopt choose their usefulness. and comparable, enhancingtheir strengthen accountability to accountability taxpayers.strengthen implementation transfer ofcapital accounting in consolidatedboards andother entitiesonthe government providedthe direction to school transfer accountingstandards. Inresponse, regarding PSAB’s direction for government public-sector reporting. in transparency andaccountability supports now more consistent andcomparable andbetter publicsector is by Ontario’s financial reporting government believesing. Asaresult, the that basisofaccount appropriate decisions ontheir agencies andsectors to facilitate entity-level respectiveAuditor andtheir ministries, General, the Officeof the exercise with incollaboration anextensive undertook stakeholdering Ontario standards applicableto public-sector entities, in response to PSAB’s changes to accounting publicsector. Ontario’s throughout In2010, issustained andfiscalaccountability reporting consistent andtransparentensure financial that to accountingstandards isimportant priate ofappro choice the The government that agrees For reason, most Canadiangovernments that At that time, significant uncertainty existed time,significantuncertainty At that MINISTRY RESPONSE Public Accounts of the Province ------41

Chapter 2 Chapter 2 42 PSAB generally follows seven steps: developing orrevising anaccountingstandard, standard-setting process. In objectivity ofthe heard the andconsidered. Thishelpsmaintain views ofallinterested are the parties ensure that ment ofaccountingstandards andattempts to public-sector accounting. source ofguidancefor primary Canada andare the accepted for governments accountingprinciples in public sector. PSAB standards represent generally for accountingstandards for establishing the Sector Accounting Board (PSAB) isresponsible transparent standard-setting process. ThePublic professional body usingacomprehensive, openand are by best established anindependent,recognized To beobjective andcredible, accountingstandards measured anddisclosedinfinancial statements. events andother transactions are to berecognized, Accounting standards specifyhow andwhen 2013 of Ontario of the OfficeGeneral of the Auditor Annual Report Future Accounting dard setters address rate-regulated accounting. regulated entitiesto implement IFRSwhilestan adoption date for mandatory rate- the deferred directedoriginally by PSAB. TheAcSB hassince would have resulted if IFRS had been adopted as province’s energy utilitiesthat by the reporting GAAP andhelpedto avoid inconsistencies in Canadian activities onabasisconsistent with ofrate-regulated economicsubstance the and OPGto adopt U.S. GAAP effectively reflects and transparency inpublicreporting. standard, has significantly enhanced consistency PSAB Thisdirection,practices. together the with province’s accountingpoliciesand the with order consistency to andcomparability preserve • PSAB emphasizes develop dueprocess inthe

The government’s direction toThe government’s Hydro One agenda setting; Standards - - framework. conceptual frame Afinancial reporting ing standards itsconceptual are consistent with have beenraised. and government concerns transfers—where these rate-regulated financial instruments, accounting proposals. of PSAB’s accounting andfinancial reporting have overing Ontario, concerns raised anumber recent years, someCanadian governments, includ challenges. In role willnotleadership bewithout accounting standards. However, this maintaining consistentrespect to applicationofpublic-sector the Canada isgenerally regarded asaworld leaderwith cial statements. preparationoffinan underlie andpresentation the asPSAB. conceptsbodies such Itsets that outthe ing standards areby established standard-setting generallyfoundation onwhich accepted account Theconceptual frameworkand reporting. isthe nature, functionandlimitsoffinancialaccounting the appropriately prescribe of standards that development the cansupport that fundamentals work isacoherent set ofinterrelated objectives and ISSUES THREE • • • • • • PSAB allnew alsostrives to account ensure that In the next section, we next areas— discussthree In the

ment; and bysupported abasisfor conclusionsdocu development andpublicationofastandard by supported anissuesanalysis; draft development andpublicationofanexposure similardocument,and orother principles (optional); similardocument orother of principles procedures after standards are issued. procedures after and review draft, exposure ofresponses to the review statement ofresponses of to the development andpublicationofastatement (optional); forcetask recruitment project planning; SIGNIFICANT

ACCOUNTING

------would ofgovthe economicsubstance not reflect acquired to avoid place.This, initsview, first inthe were derivatives volatility the very the report year would province force the to inappropriately recording papergains andlosseseach argues that resource inflows oroutflows. The government long-term government’s economic impact onthe would isheldandtherefore derivative have noreal any that over other period would each the offset Accordingly, remeasurement gains andlosses mature. them debtsassociatedtives with untilthe intention andabilityto the holditsderiva has both it related to holdingsandthat itslong-term-debt manage andinterest-rate foreign currency risks solely itusesderivatives to view isthat Ontario’s unrealized gains holdings. andlosses onderivative fair-value recognition of remeasurements andthe introduction ofthese the donot support Ontario’s, recorded annually inanew financial statement. any unrealizedat fairvalue,with gains orlosses to includingderivatives, berecordedinstruments, One ofitsmainrequirements financial isfor certain and issimilarto private-sector existing standards. disclosure ofgovernment financialinstruments, recognition, measurement, and presentation the 1,April 2012. Thestandard provides guidanceon beginningonorafter entities for fiscalperiods 2015, andeffective for public-sector most other 1, April beginningonorafter for fiscalperiods on financial statements effective for governments or deficit. annual surplus government’s the changes shouldaffect such its financial statements and,inparticular, whether heldbytracts agovernment shouldbereflected in con fairvalueofderivative changes inthe whether ward Akey contracts. issueis financialinstrument swaps andforeign-exchange ascurrency such for Financial includedebtandderivatives instruments began in2005. financialinstruments reporting PSAB’s project to develop anew standard for Financial Instruments Some Canadiangovernments, including 2011,In March PSAB approved anew standard - - - - Financial project. Performance address issuesidentifiedinitsConcepts Underlying year,later this itmay noting needrevision that to province standard. of the didadopt the consolidated financial statements ofthe part form less, somepublic-sector entitieswhoseresults do 1,April 2015, Neverthe concern. addressed that toers delay new implementing standard until the PSAB’s 2013 decisionto allow adopt allfirst-time provincestandards before the was. However, financial statements were required the to adopt province’s consolidated entities includedinthe government finances. needsfor public’s on transparent information the anddoesnot financingtransactions meeternment electricity generators, electricity transmitters anddistributors. unique nature ofregulated as the entitiessuch wereaccounting practices developed to recognize consolidatedRate-regulated financial statements. regulated government’s assets andliabilitiesinthe ofrecognizing rate- appropriateness about the past fewOver the years, we have concerns raised Rate-regulated AccountingRate-regulated Financial project. Performance results ofitswork onitsConcepts Underlying standards andto take the into accountthe with to addressissues outstanding an opportunity anextension willalsoallowments. Such PSAB associatedassess andprepare forrequire the standards to allowthese governments to fully requested implementation to date extend the for Asaresult, PSABinstruments. hasbeen andfinancial translation on foreign currency negative impacts ofPSAB’s new standards potential the with continue to beconcerned seniorCanadiangovernments andother Ontario PSAB has committed to standard reviewing this The government that was alsoconcerned MINISTRY RESPONSE Public Accounts of the Province - - - 43

Chapter 2 Chapter 2 44 rently permit rate-regulatedrently permit accounting. 1, January 2012.or after IFRSstandards donot cur IFRSfor fiscal accordance beginningon with years financial statements in shouldprepare their prises that allgovernment businessenter AcSB reaffirmed 2012, inJanuary factthat, Canada’s based onthe StandardsReporting (IFRS).Our commentswere werethat converting to Financial International asCanada to such beendingfor jurisdictions ofrate-regulated era the accountingappeared that consolidated financial statements. ment’s govern ofthe assets purposes orliabilities for the assets definition of andliabilitiesmeet the even we though question rate-regulated whether 2013. We have accepted accountingtreatment this 31,accepted asofMarch accountingprinicples) lated liabilities(according to Canadiangenerally rate-regulated assets and$36 millioninrate-regu cant—for example, OPGrecognized $1.9 billion in are signifi numbers regulated accounting.Andthe accounting policiesto remove impact ofrate- the their adjusting financial statements without province’s consolidated to beincludedinthe prises, are definedasgovernmentwhich businessenter Board (AcSB). is now underreview Accounting by Standards the Canadian generally accepted accountingprinciples, rate-regulated entities,whilestill allowed under The useofrate-regulated accountingby certain userate-regulatedHydro One—that accounting. cant provincially owned organizations—OPG and yearcant costs would incurred. inthe beexpensed signifi erally accepted these accountingprinciples, statement offinancialposition. gen Under normal costs are typically set entity’s upasassets onthe deferred costs Such for recovery infuture periods. entities to certain defer for accountingpurposes may allows charge customers, andoften regulated aregulatedBoard, entity that approves prices the regulator,established Energy Ontario asthe such Under rate-regulated accounting,agovernment- 2013 of Ontario of the OfficeGeneral of the Auditor Annual Report In recent Annual Reports weIn recent AnnualReports have commented PSAB standards allow OPGandHydro One, sector includestwo electricity signifi Ontario’s bona fide ------deferral accountsbasedonarecentlydeferral issued Inter anewing to standard onregulatory incorporate 2013, propos AcSB draft issuedanexposure the 1,been extended again to 2015. January InMay qualifying rate-regulatedentities with activitieshad IFRSchangeover mandatory date ofthe fordeferral 1,tional one-year extension, to January 2014. activities. InSeptember 2012, anaddi itgranted over datequalifying for rate-regulated entitieswith 1,to 2013, January IFRSchange mandatory to the 2012 AcSB Canada’s aone-year granted extension, IFRS,inMarch accepted with accountingprinciples to reconcile U.S.ing. Partlyinaneffort generally continuestotherefore allow rate-regulated account change. TheUnited States hasnot adopted IFRSand rate-regulated accounting,asdiscussedabove. We allowaccepted which for accounting principles, U.S.statements inaccordance with generally Hydro OneandOPGto financial both prepare their in 2011 allowing for andsubsequently directing government passedaregulation accounting, the annualdeficitorsurplus. government’s and the year, couldultimately which impact rates electricity regulated entities willrecognize inanythe given costs electricity has significantinfluenceonwhich regulatedregulator entitiesinquestion, andthe it government the controls both position. Sincethe isaware, we this Ministry with donotthe agree rate-regulated provisions from AcSB. Canada’s As referencePSAB standards without to any ofthe rate-regulated assets andliabilitiesmightmeet province’s the alsocontends that The Ministry fiscalpolicydecisions. government’s impact onthe rate-regulatedcontends that accountinghasan couldtake severalities project, which years. completes itscomprehensive rate-regulated activ activities subjectto IASB rate regulation untilthe standard for useby adopters first-time ofIFRSwith proposes aninterim draft Theexposure ure draft. national Accounting Standards Board (IASB)expos At the time of drafting this Annual Report, the the AnnualReport, this timeofdrafting At the However, landscapehascontinuedto the With the uncertainty regarding rate-regulated uncertainty the With ofFinance (Ministry) Ministry Ontario’s ------of government to another, following: includingthe transfersissues relatedfrom onelevel to monetary number ofyears agoto address several accounting PSAB’s Government Transfers project began a regulated accountinginCanada. will have timeregarding rate- decidedatthat the AcSB financial statements, asitisunclearwhat 31,March 2015, andsubsequent consolidated the province’s financial statements on sion ofthese inclu ofthe possibleeffect aboutthe are concerned Transfer Payments resolving this issue. resolving this tolook forward standard-setters appropriately basis until2014. Thegovernment continuesto financial statements ona submit their U.S. GAAP enabledrate-regulatedmission, which utilitiesto Com Securities Ontario Administrators andthe Canadian Securities actions taken by the both province’s decisionwas consistentissue, the with standard-setters’ subsequent ofthis deferral the CanadianGAAP. historical consistent with With rate-regulated assets andliabilitiesonabasis GAAP to allow entitiesto accountfor the their ment directed Hydro OneandOPGto follow U.S. not have govern beenpossible.Inresponse, the to follow IFRS,rate-regulated accountingwould asHydro OneandOPG such business enterprises direction from PSABoriginal for government its consolidated financial the statements. Given PSAB accordance standards inpreparing with by costs reported Hydro OneandOPGin tricity The province accountsfor rate-regulated elec • •

MINISTRY and transferee; fers to transferor berecognized by the both ing provided by onegovernment to another; clarifying the authorization needed for trans authorization the clarifying appropriately accountingfor multi-year fund RESPONSE ------ents shouldaccountfor multi-year transfers. Ifthe address indevelopingstandard was how the recipi 1,April 2012. 2010, effective for fiscal beginningonorafter years standard ongovernment transfers inDecember views, PSABtion ofrespondents’ approved anew several documentsfor commentsandconsidera specifies that actions and communications by the that actionsandcommunications by specifies to take to transfer. keep the The standard also transfer,the the recipient needs orspecifiesactions government by imposing useof stipulations onthe government createsring recipient aliabilityfor the requirementthis doesnot transfer apply whenthe However,recipient hasmet alleligibilitycriteria. andthe as revenue whenithasbeenauthorized recipients shouldrecognize a government transfer is usedto provide publicservices. funds transferred the acquired with orconstructed into revenue overasset capital tangible timeasthe transfers shouldbebrought such heldthat holders However,eligible to receivegrant. the stake other makingit expenditures ent government the incurs should berecognized asrevenue recipi whenthe transfers capital view that stakeholders heldthe boardsAnumberof andhospitals. asschool such province transfers from the tocapital entities other respect to with Asimilarissuearises incial services. revenuerecognize the over years itfundsprov the asrevenue itshould grant orwhether amount ofthe provincethe shouldimmediately full recognize the over several asto question whether years, arises the endofafiscal year the to aprovince to fundservices federal government makes alump-sum transfer near • • One of the most difficultareas PSABOne ofthe had to issuingof discussion,the substantial After The new standard generally recommends that

capital assets.capital are to beusedto tangible acquire orconstruct ent governments; and andrecipi transferring the accounts ofboth timingoftransfer recognition inthe the imposed government affect by atransferring appropriately accounting for transfers that appropriately accountingfor transfers that stipulations to degree which the clarifying Public Accounts of the Province ------45

Chapter 2 Chapter 2 46 with both beingrecorded both asrevenuewith over useful the contributions, capital asdeferred nal contributions accounting for government both transfers andexter new standard, we initially believeitsupports that asset. life ofthe oroverconstructed, service the when received,asset hasbeenacquired whenthe or transfers mightberecognized asrevenuesuch circumstances, assets.capital Dependingonthe of tangible acquisitionto orconstruction fundthe difficult isaccounting for transfersto reach received standard. consider amendingthe more empirical evidence is neededbefore itwill view that matterPSAB andisofthe discussedthe Canadian CouncilofLegislative Auditors in2013. includes arequest signedby ofthe allmembers sistencies andapplication.This ininterpretation transfers standard becauseofincon amending the of over $21 billion. and transfer revenue federal government from the transfer-payment inexcess expenses of$50 billion 2012/13results. Inthe fiscal year, recorded Ontario activity andcanhave impact a great onreported transfers are usually asignificantgovernment because application. Thisisasignificantconcern, pretation, resulting inits insignificantdifferences Its requirements broad andopento inter are very new to standard appears becreating confusion. ability inaccountingfor government transfers, the revenue liabilityissettled over asthe time. transfer andrecognized isdeferred as ment, the deemed to create recipient govern aliabilityfor the matter ofprofessional judgment.Ifatransfer is recognition ofaliabilityis the transfer support aparticular surrounding facts andcircumstances result ofpast andevents. transactions the Whether economic resources to settle it,anditmust bethe tion to avoid must be afuture outflow it,there of there mustPSAB’s benodiscre liabilitydefinition, cancreatefor aliability. aspecificpurpose To meet funds useoftransferred the restrict recipient that 2013 of Ontario of the OfficeGeneral of the Auditor Annual Report While we acknowledge controversy the over this One significantarea where consensus hasbeen Many stakeholders have asked PSAB to consider enhancingconsistency andcompar than Rather ------statements in the future. statements inthe province’s consolidated financial preparation ofthe studying over may last year impact that the the Public Sector Accounting Board (PSAB) hasbeen This sectionoutlinessomeadditionalitems the statements ($5.1 billion in2011/12). 31,province’s March 2013, consolidated financial beingrecorded inthe contributions capital deferred $5.6 billionin we with munications. Assuch, agreed transfer stipulations andrecipient actionsandcom relatedassets capital life basedon tangible ofthe the Taskthe Force paper issued asecondconsultation itskey with proposals. InOctoberagreement 2012, wereRespondents draft ingeneral exposure to the implications. accountingandreporting and their keyofpublic-sector entities on the characteristics in August 2011 to seekinput from stakeholders framework publicsector. for the conceptual existing inthe concepts andprinciples objective isto ofthe appropriateness review the budget-to-actualtort comparisons. TheTask Force’s tend result in volatility results anddis in reported they con revenue definitions,which andexpense byraised several governments regarding current (Task Force) 2011 inApril inresponse to concerns Conceptual Framework the formed Task Force an objective, credible andconsistent manner. PSAB accountingstandards are developedensure that in standard-settingdiscipline into process to the isto instill accounting standards. Itspurpose development the ofconsistent support ciples that interrelated objectives prin andfundamental PSAB’s conceptual existing framework isaset of Board Public PERFORMANCE UNDERLYINGCONCEPTS The Task Force paper consultation issueditsfirst Initiatives Sector Accounting

FINANCIAL

- - - following: the most significantofwhich were the principles, 15 containing a statement ofprinciples proposed Task Force project issued to established leadthis 2013, InApril ernment. JointNot-for-Profit the govmany Ontario organizations fundedby the organizations. Thesestandards are followed by organizations, includinggovernment not-for-profit improve accountingstandards for not-for-profit The AcSB andPSAB recently initiated aproject to PSAB intendsciples that to 2014. issueinMarch astatement ofprin will beconsidered indrafting following:on the public-sector financial statements. Itsoughtinput in financialperformance focused onmeasuring STANDARDS IMPROVEMENTS • • • • • • • Input received two papers from the consultation

accountabilities; and accountabilities; statements canhelpto demonstrate those for; are accountable and whatthey financial reporting; disposing of tangible capital assets. capital disposing oftangible down writing and amortizing, capitalizing, would adopt public-sector standards for enhanced budget and information; indicators, astatement ofnet debtand would ofnet-debt require presentation the governmentand other organizations. This follow samestandards asfor governments the ment not-for-profit organizations would definitionofaliability; the to give anobligation rise meetingcontribution ofthe recognized asrevenue, terms unlessthe performance. government not-for-profit organizations financial statement presentation for govern receivedcontributions would beimmediately financial alternative modelsofmeasuring how provision infinancial the ofinformation who public-sector to, entitiesare accountable objective andcontextthe ofpublic-sector

TO NOT-FOR-PROFIT

- - - ber 2012 following: proposed the issuedinSeptem draft Anexposure performance. financialpositionand entity’s a reporting have transactions related-party that on effect the requirements. disclosures allow Such to users assess measurement anddisclosure their and describes publicsector context ofthe inthe related parties defines at issuinganew accountingstandard that PSAB’s Related-Party Transaction project isaimed public comment. for releaseexpected draft ofanexposure to bethe ber 15, 2013. project is step Thenext inthis TRANSACTIONS RELATED-PARTY • • • • • The task forceThe task isseekingcommentsby Decem

loss would berecognized. gain or amount,the carrying from the differs received, exchange amount orfairvalue.If the paidor consideration amount,the carrying exchange couldbethe amount,which at the entity.reporting may berecognized ordisclosedby the services goodsand Contributed by parties. both would berecognized uted goodsandservices financial statements. ties have orcouldhave onthe effect amaterial andeventstransactions between related par required. similar routine payments would not benot allowances expense arrangements, andother ever, disclosure ofmanagement compensation family how are includedasrelated parties; oftheir andclosemembers ment personnel entity.control ofareporting shared control overwith orsubjectto shared entities undercommoncontrol, andentities control orareentity, controlled by areporting Related parties wouldRelated includeentitiesthat parties Related-party transactions would transactions Related-party berecorded contrib than other transactions Related-party Disclosure would berequired only when ofkeyIndividuals whoare members manage Public Accounts of the Province ------47

Chapter 2 Chapter 2 48 addition, section91 ofthe Treasury year. the Board Ordersissued during In required onany to report SpecialWarrants and Under section12 ofthe 2013 3, 2014. andisseekingcommentsby February onrevenueprinciples for publiccommentinAugust income.Itissuedastatementsales andrental of penalties, royalties, licencefees fees, andother and publicsector,common inthe asfinesand such applicable to abroader range oftypesrevenue fers. PSABneedfor guidance hasrecognized the address taxesstandards andgovernment that trans revenue specific recognition anddisclosure, with accounting standards provide general guidance on public-sector Current ordeficitit reports. surplus asithasadirectthe impactfinancial reporting on Revenue to recognition isfundamental government by Septemberdraft 4,2013. June 2013 significant revisions: with in PSAB issuedare-exposure draft draft, exposure 2013 of Ontario of the OfficeGeneral of the Auditor Annual Report Statutory REVENUE • • • PSAB requested re-exposure commentsto the responses to this inthe raised Due to concerns

• • retroactively orprospectively. amount,unless: carrying ured atthe alsomanagebers entity. another closefamily mem entityortheir reporting the Preparers could apply the proposed standard Preparers couldapply the would transactions Related-party bemeas Entities may berelated whenmanagement of

a recipient’s future economicbenefits or a recipient’s ofoper course normal are inthe they measured at the exchangemeasured amount. atthe would transaction be cases,the In these the transaction. the result of significantly as potential isexpected toservice change ations; or Matters Auditor General Act Legislative Act Assembly

, Iam - -

- - so reported to the Legislature. Ordersfor to Concur so reported the are deemedto Committee bepassedby andare the are not reviewed that ministries estimates ofthose review Legislature. The to onthis the a report estimatesreviews andpresents selected ministry Standing Committee onEstimates (Committee) spendingproposals. The ministry’s basis, each Assembly outlining,onaprogram-by-program Legislative estimates expenditure tables inthe presenting itsbudget, government after Shortly the the Assembly. Officeof mates ofthe between samevote Esti the items inthe within onanyrequires Ireport transfers ofmoney that Interim Act 1,Interim was madeeffective as ofApril 2012. $197 million inlegislativeexpenditures. The office $4.3 billionininvestments,vice expenditures, and to incuruptoernment $114.8 billion inpublicser Assent onJune 20,2012, gov the and authorized Act). Act(Interim TheInterim received Royal 2012-2013 for Appropriation Interim Act,2012 2012/13 fiscal year,the the Legislature passed to itspassage. prior For spending authority the over—so government usually the requires interim relatedsometimes fiscal even the year is after fiscal year—and ofthe start the Assent untilafter 2013, received Royal 23, 2013. AssentonApril fiscal 31, toyear endedMarch the pertained which to beVoted The Appropriations. are expenditures considered individualprogram the estimates. Once the in the detailed typically those programs, spent by ministry Act Supply by approving legala spendingauthority ment with Legislature still needstothe provide govern the maximum oftwo before hours beingvoted on. Legislature for debated a Committee inthe are then estimates on by reported the ofthe rence for each EXPENDITURES LEGISLATIVE The are approved, Ordersfor Concurrence the After Supply Act Supply , which stipulates the amounts that can be canbe amountsthat stipulates the , which APPROVAL does not typically receive Royal

Supply Act Supply OF Supply Act,2013Supply

isapproved,

- - , - - - was repealed and re-enacted within the was the repealed andre-enacted within Legislature. inthe published andtabled have PublicAccounts have beenmadeandthe been when any from ouraudit finaladjustments arising to beclosed books The government the considers fiscal foryear are governmentclosed. the books the year. Theorder may bemadeatany timebefore voted not fiscal fully appropriations spentinthe reduction from other to ofexpenditures beincurred by increase acorresponding isoffset amount ofthe it was made.Theorder may bemadeonly ifthe for which the purpose out insufficient to carry any votedisexpected to be that appropriation amountof toizing expenditures supplementthe allows Treasury the Board to make anorder author Section 1.0.8 ofthe year 31, endedMarch 2013. government. tenant recommendation ofthe Governor onthe by Orders-in-Councilandapproved Lieu by the is insufficient.Special are authorized Warrants appropriation the Legislature orforby which the isnoappropriation there forof expenditures which incurring the ance ofSpecialWarrants authorizing the Legislature isnot insession,section1.0.7If the of lion to $199.6 million. legislative officesfrom $197 ofthe expenditures mil Act,2013,Supply the intended to betemporary, anditwas repealed when Act provided Interim was underthe ing authority received Royal 23, 2013. AssentonApril Thespend 1,from April 2012, to whenthe to allow it sufficient authority expenditures to incur TREASURY SPECIAL WARRANTS Even though the the Even though No SpecialWarrants were fiscal issuedfor the Act The Interim provided government with the Supply Act,2013,Supply Financial Administration Act BOARD also increased total authorized alsoincreased total authorized Financial Administration Act Treasury Board Act,1991 received Royal Assent.The ORDERS

Supply Act,2013,Supply allows issu for the

Financial , -

- - - - a reduction to the amount available under the gov amountavailablea reduction to underthe the by isoffset amountofanappropriation the in which are Treasury appropriations Board Orders mentary from contingencyappropriations funds.Supple between andmakingsupplementary ministries Treasuryof the Board transfers for making program Chair andto the ministries, their within programs Orders to ministers to make transfers between for issuingTreasurydelegated itsauthority Board Treasurynew delegation. Since2006,the Board has tions continueto untilreplaced by beineffect a the employedCouncil orto any publicservant under duties orfunctionsto Executive any memberofthe sion allows Treasury the Board to delegate any ofits repealed actwas retained.5(4) ofthe Thisprovi Administration Act Source of data: TreasuryBoard Source ofdata: 2008/09–2012/13 ($million) Figure 11: Total ValueOrders, Board ofTreasury listing of2012/13 Treasury Board Orders,showing Gazette The Ontario 2012/13 fiscal expectedyear are to bepublishedin Ordersissuedfor the information. explanatory in be printed lative Assembly, Treasury Board Ordersare to of issue. fiscal 31,foryear endedMarch the 2013, by month pastBoard five Ordersissuedfor fiscal the years. centrally controlled contingency fund. ernment’s 12, 10 14 2, 4, 6, 8, , , 000 000 000 000 000 000 000 According Legis Standing to Ordersofthe the Figure 12 Treasury summarizes Board Orders Figure 11 total valueofTreasury the summarizes Public Service of Ontario Act,2006 ofOntario Public Service 0 2008/09 The Ontario Gazette The Ontario 2009/ in December2009,subsection inDecember2013. Adetailed Public Accounts of the Province 10 2 010 /1 12 , together with 011 . Such delega . Such /1 22 01 2/13 - - - - - 49

Chapter 2 Chapter 2 50 Source of data: Treasury Board Source ofdata: Month Relating the2012/13 to FiscalYear by Figure 12: Total ValueOrders Board ofTreasury (Thecomparable amountin2011/12off. was and non-government organizations were written Crown$395.8 million dueto the from individuals fiscal the Public year areAccounts. in to be reported anyThe amountsdeleted accounts during from the subjectofasettlement ordeemeduncollectible. the accounts any Crown are amountsdueto the that anOrder-in-Councilauthorize to delete from the Minister ofFinance,recommendation ofthe may Act Under section5ofthe Estimates. Votemade within 201 2012/13 respect to the with transfer(s) inourAnnualReport. of the Act Assembly samevote, the section91within ofthe the Assembly item Officeof to another mates ofthe transfer Esti ofmoney from oneitemthe ofthe Economy Board ofInternal authorizes When the report. as Exhibit4ofthis isincluded andexpended, amountsauthorized the 2013 of Ontario of the OfficeGeneral of the Auditor Annual Report Total July 2013 April 2013 March 2013 April 2012–February 2013 Month ofIssue UNCOLLECTIBLE ACCOUNTS BOARD TRANSFERS , the Lieutenant Governor, the inCouncil,on the In the 2012/13In the fiscal year, receivables of Accordingly, Figure 13 shows transfers the OF requires we that make specialmention INTERNAL AUTHORIZED Financial Administration ECONOMY 122 BY 36 12 72 2 #

THE Legislative

($million) Authorized 5,088 2,428 1,969 - 270 421 year related following: to the $816.4 2012/13 inthe million.) Thewriteoffs fiscal Source of data: Board of Internal Economy BoardofInternal Source ofdata: of theAssembly, 2012/13 Fiscal Year ($) Figure 13: Relating theOffice AuthorizedTransfers to accountsrequiredthe Order-in-Council approval. financial statements. However, in the actualwriteoff consolidated government’s inthe been expensed ances. Accordingly, hadalready writeoffs most ofthe recorded annually against accountsreceivable bal statements, aprovision for doubtfulaccountsis province’s consolidated financial preparation ofthe by ministry. writeoffs the marizes Item 6 Item 5 Item 2 To: Item 4 Item 3 From: • • • • • • • • Under accountingpoliciesfollowed the inthe Volume 2012/13 2ofthe Public Accounts sum

health tax ($48.9 millionin2011/12); tax health pulp andpapercompanies ($0in2011/12); ($86.3 million in2011/12); Program DisabilitySupport Ontario under the ($155.8 million in2011/12); ($114.1 million in2011/12); Program Studentunder the Support ($382.2 million in2011/12); ables ($29.1 million in2011/12). ties ($0in2011/12); and $13.2 royal million for uncollectibleforestry $15.1 million for uncollectibleemployer million for uncollectibleloansfrom $44.7 $48 million for uncollectiblereceivables tax $60.4 million for uncollectiblecorporate $86.5 million for uncollectiblereceivables $92.1 million for uncollectibleretail salestax million for other tax and non-tax receiv andnon-tax $35.8 million for tax other Sergeant at Arms and atArms Sergeant Services Administrative the Clerk Office of andTechnologyInformation Legislative Services Precinct Properties Services (26,400) (18,200) 12,800 21,900 9,900 - - - - money work professional the inaccordance with results ofpast auditsandrelated follow-up work. related issues ofpublicsensitivityandsafety, andthe perceived sig ororganization’s financialimpact,a program’s its as such criteria, basisofvarious agement onthe year werethis seniorman selected Office’s by the broader publicsectororganizations audited inthe past audityear. VFMauditsconductedfor inthe the andrecommendations conclusions, observations publicisreceiving. the Thischapter contains the level service cost-effectiveness andthe their both ororganization beingauditedprogram to assess delve underlying into ministry the ofthe operations pass compliance issues.Essentially, VFMaudits delivery. auditsalsoencom Where relevant, such effectiveness ofservice onthe measure andreport procedureswhere appropriate were not inplaceto dueregard forwithout economy andefficiencyor whereon any money was spent casesobserved General Act conducted undersubsection12(2) ofthe andactivities.Theseauditsare programs their Crown andCrown-controlledmanage corporations broader publicsector,tions inthe agencies ofthe howexamine well governmentorganiza ministries, Our value-for-money (VFM)auditsare intended to Chapter 3 Chapter We onourvalue-for- andreport plan,perform andactivitiesthe programs The ministry , which requires that the Office report Office report requires the , which that nificance to the Legislativenificance Assembly,to Value-for-money Audits on Reports Auditor - - - and compliance work. conductingthe They entail encompassAccountants), which CanadianInstituteada (formerly ofChartered the Professional Chartered Accountantsby ofCan the standards for engagements assurance established Annual Report. chapter ofthe assectionsofthis are incorporated the VFMaudit reports are finalized, audit report fundingministry.ment ofthe tor, seniormanage discussionsare alsoheldwith broader publicsec caseoforganizations inthe the management responses to ourrecommendations. In andthe report auditee draft tofrom the discussthe senior management meet with senior Office staff isprepared. Then auditreport auditee andadraft year,the significantissuesare discussedwith completednormally audit by ofthat late spring auditfield conclusionofthe At the auditandensure openlinesofcom the auditee of to progress the review the dialogue with anongoing maintain audit,staff the During teria. audit,includingourauditobjectives andcri of the auditee representativeswith focus to discussthe area to into research beaudited the depth andmeet whenappropriate. experts sary, advicefrom external includingobtaining wetests procedures that andother considerneces Once the content VFM andresponses forOnce the each conductin- Before beginning anaudit,ourstaff

valuefor money work, is which munication. - - - - - 51

Chapter 3 Chapter 3 Ministry of Children and Youth Services Section 3.01 Autism Services and Supports for Children

trol and Prevention (CDC) reported that autism Background affected 1 in 88 children in 2008, up from 1 in 150 in 2000. More recently, the CDC reported results from a 2011/12 public survey that show that aut- Autism spectrum disorder (commonly known as ism affects 1 in 50 children aged 6 to 17. In fact, autism) covers a range of neurological develop- statistics released by the CDC and the U.S. Data mental disorders characterized by difficulties with Resource Center for Child and Adolescent Health social interaction and communication, repetitive indicate that autism is being diagnosed in children behaviours and/or a range of cognitive deficits. The more often than juvenile diabetes, cancer and AIDS presence of symptoms and the degree of impairment combined. Using the latest available prevalence vary from individual to individual; some people rates provided by the CDC for 2008 and by the with autism have severe intellectual disabilities National Epidemiologic Database for the Study

Chapter 3 • VFM Section 3.01 while others are high-functioning. This disorder is of Autism in Canada for 2010, we estimated that lifelong and has a significant impact on families and approximately 30,000 to 35,000 children with aut- caregivers. Nonetheless, experts believe that treat- ism were living in Ontario at the time of our audit. ment and support, especially through early inter- Children with autism may access a variety of vention services, can help improve the functional services and supports, such as speech therapy, abilities of affected individuals. occupational therapy and mental health services. The prevalence of autism has been increasing. These programs are funded by various ministries, Whether this is due to a rise in the incidence of including the Ministry of Children and Youth Ser- autism or a rise in the number of people being vices, the Ministry of Education, and the Ministry diagnosed is unclear. At the time of our audit, of Health and Long-Term Care, and are accessible no statistics were available on the prevalence of to all children who qualify. Our audit focused autism in Canada or Ontario as a whole. But a primarily on services and supports funded by the March 2012 report by the National Epidemiologic Ministry of Children and Youth Services (Ministry) Database for the Study of Autism in Canada exclusively to children with autism. indicated that the prevalence rate in southeastern The Ontario Health Insurance Plan does not Ontario was 1 in 77 in 2010, up from 1 in 190 in cover autism services and supports. However, 2003. A similar upward trend has been reported although not legislated to do so, the Ministry has in the United States. The Centers for Disease Con-

52 Autism Services and Supports for Children 53 since the year 2000 funded various services and the amount and value of services provided. Senior supports for eligible children with autism up to ministry management reviewed and agreed to our age 18 and their families. Two significant compon- audit objective and associated audit criteria. ents of what is funded are the Autism Intervention Our audit work was conducted primarily at Program (AIP), which provides intensive behaviour the Ministry’s corporate office, at three of its nine intervention (IBI) services, and applied behaviour regional offices, and at selected service providers in analysis (ABA)-based services. For a comprehen- those regions. We reviewed and analyzed relevant sive list of autism-specific services and supports files, program and financial data, and administra- funded by the Ministry at the time of our audit, see tive policies and procedures. We also interviewed Figure 1. appropriate ministry and agency staff. To gain Ministry-funded autism services and supports insight on how other jurisdictions administer aut- are delivered to children in Ontario through ism services, we reviewed studies and reports from approximately 90 community- or hospital-based elsewhere in Canada, the United States, and select agencies. These agencies are usually not-for-profit Commonwealth countries. We also met with repre- organizations. Some agencies also provide other sentatives from Autism Speaks Canada and Autism services such as mental and family health services, Ontario, and an autism expert in the province to get and hence may receive funding from other govern- their perspectives on autism services in Ontario. In ment ministries and programs. The Ministry’s nine addition, when designing our audit procedures, we regional offices are responsible for overseeing considered comments from parents submitted to program delivery by agencies, and the Ministry’s us directly or published. To determine how schools corporate office is responsible for policy develop- are trying to meet the needs of students with aut- ment and program design. ism, we interviewed superintendents and relevant In the 2012/13 fiscal year, transfer payments for staff responsible for special education in four autism services and supports comprised almost all school boards in the three regions we visited. We program expenditures, and totalled approximately also engaged two independent advisers from other $182 million. jurisdictions who have expert knowledge on autism

Our Office reviewed the AIP in 2004 at the to assist us. Chapter 3 • VFM Section 3.01 request of the Standing Committee on Public Accounts (Committee). Our review and the subsequent hearings of the Committee examined a number of questions and concerns, including Summary cost effectiveness, service hours, and program performance. Autism is becoming more prevalent in Ontario and in other parts of the world. In response to the increased demand for autism services and supports for children, the Ministry of Children and Youth Audit Objective and Scope Services (Ministry) has quadrupled autism funding over the last decade by increasing funding to The objective of our audit was to assess whether the its existing primary service—intensive behaviour Ministry has adequate procedures in place to man- intervention (IBI)—and introducing several new age a system of cost-effective autism services that programs such as applied behavioural analysis are accessible to children up to age 18 with autism (ABA)-based services and respite services. In this and their families, and to monitor that transfer way, the Ministry has been able to provide service payments are controlled and commensurate with to more children with autism and their families. 54 2013 Annual Report of the Office of the Auditor General of Ontario 4 1 1 1 1 14 14 64 100 for Payments and Supports Autism Services Transfer % of Total 7. 8 1.5 2.6 2.5 2.1 25.0 25.0 115.4 181.9 Transfer 2012/13 Payments ($ million) # Served in 2012/13 2,000 children 6,200 children 77,300* educators children 1,250 served in Connections 8,100* children children 47 6,800 parents/ caregivers children 1,000 Various Start Date 2000 2011 2004 for SSP for 2007 Connections (fully implemented in March 2010) 2007/08 2004 2006/07 2006/07 Various Chapter 3 • VFM Section 3.01 Purpose/Description Provides assessment, child and family supports, intervention (IBI), and transition intensive behaviour supports for children diagnosed with an autism spectrum disorder (ASD) toward the severe end of the spectrum. Provides time-limited skill-building services to children with ASD to improve communication, social/ interpersonal, skills, with daily living and behavioural/emotional relevant parent support. assist educators (e.g., school board staff, in 2012/13) education assistantsASD consultants (153 and teachers) in publicly funded school boards to better understand how children with ASD learn and how the principles of ABA can help improve their learning. ASD consultants also provide child-specific consultation and support to the Connections for Students model (a joint initiative involving the Ministry of Education and the Ministry of Children and Youth Services). In Connections, multi-disciplinary transition teams support children who are transitioning from IBI services to publicly funded schools. The transition team supports the child starting six months AIP until six months before leaving after AIP and entering or continuing in school. leaving Provides temporary relief for families caring a child with ASD while providing meaningful opportunities for the children to sustain their skill development and participate in community activities. Services include in-home respite, out-of-home respite and seasonal camps. Provides funding to families for IBI services on a negotiated basis outside the regular system. Provides one-to-one supports, including parent networking opportunities, training, and access to ASD experts Autism Ontario. and resources. Administered by for children with ASD transitioning into adolescence and secondaryTargeted school. The service supportspriorities are crisis intervention, behavioural and/or skill-based training. Includes reimbursement who pursue of tuition costs for AIP employees relevant education programs, funding for parent networking opportunities and resources through Autism Ontario, ABA registry, and local support in the South region through West theChildren’s Centre. Windsor Regional Autism Services and Supports Autism Intervention Program (AIP) Applied behaviour analysis (ABA)- based services School Support Program (SSP) services Respite Intervention services covered outside the regular program Potential program services Transition Other Total not be unique individuals. * These may Figure 1: Autism Services and Supports the Ministry by Services Funded of Children and Youth the Office of Prepared by the Auditor General of Ontario Autism Services and Supports for Children 55

Nevertheless, there are more children with autism • ABA-based services, which constitute the waiting for government-funded services than there only type of provincially funded therapy in are children receiving them. Ontario available to children with mild to IBI is the province’s primary method of therapy, moderate forms of autism, might not be suffi- but it is not being offered to the children for whom cient for those who have a host of behavioural it is likely to make the most difference. Although problems or goals to achieve, because the scientific research shows that children with milder program allows a child to work on only one forms of autism have better outcomes with IBI, the goal at a time; it then requires that the family program is currently available only to those children reapply if it wants the child to receive further assessed as having more severe autism. Research ABA-based services, with the child returning also indicates that children who start IBI before age to the bottom of the wait list after each ABA- 4 do better than those who start after age 4. How- based intervention. ever, due to long wait lists for IBI services, children • It is up to each lead service agency to decide are not typically starting IBI until almost age 7 in how to allocate ministry funding between two Ontario. According to experts, early diagnosis and IBI service delivery options: direct service, treatment of autism might reduce the need for more where the child receives service directly from supports and services later on in life. The Ministry a service provider at no cost; or direct funding, needs to re-evaluate its program design in order to where the family obtains funding from a lead maximize outcomes for all children served. service agency to purchase private services on Although the Ministry formed an expert panel in its own. Wait times for IBI services can differ December 2012 that will provide advice on some of significantly between the two options and the more contentious issues involving IBI (such as among regions depending on how lead service benchmarks for continuation of or discharge from agencies have allocated their funding and this type of therapy), and recently introduced an available capacity. In one region in 2012, the independent review mechanism for when families average wait for IBI services under the direct disagree with service providers’ decisions on IBI funding option was five months longer than the

eligibility or discharge, more work may be needed. average wait under the direct service option. In Chapter 3 • VFM Section 3.01 Some of our other more significant observations another region, the situation was reversed. include the following: • In general, children receiving IBI under the • We estimated that children with autism are direct service option received fewer hours diagnosed in Ontario at a median age of of therapy than they were approved for. For a little over 3 years. This is later than the example, at two lead service agencies we vis- recommended 18-to-24-month screening ited, children who were discharged from IBI in period endorsed by the Canadian Paediatric 2012 had received a median of only 20 hours Society for children with risk factors. As well, of therapy per week, even though they had the median wait time for children with aut- been approved for at least 27 hours of service ism in the three regions we visited to access per week. The agencies told us that this was IBI services was almost four years. Over the because they would “ramp up” to the full level last five years, the number of IBI spots has of approved hours at the start of the service remained relatively constant at 1,400, while period and “ramp down” hours closer to the the number of children waiting for IBI ser- end of the service period, a practice not clearly vices increased by 23%. This means that an explained in the program guidelines. We also increasing number of children are not able to noted that any missed or cancelled appoint- access early intervention. ments by the child or the therapist could not 56 2013 Annual Report of the Office of the Auditor General of Ontario

be made up at a later time. At the time of our that all their teachers who taught children visits, two of the three agencies were not track- with autism had participated in ABA training ing actual hours of IBI services received by sessions. Furthermore, in light of the fact that children under the direct-funding option. many school boards have acquired their own • Of the children discharged from IBI services expertise on teaching children with autism in 2012/13 on a province-wide basis, those with funding from the Ministry of Education, under the direct funding option received on the Ministry of Children and Youth Services average almost one year more of services needs to determine whether the $25 million it than those under the direct service option (35 spends on Autism Spectrum Disorder consult- months versus 25 months). In fact, almost ants for training and consulting with teachers 25% of children under the direct funding under the School Support Program is provid- option received more than four years of ser- ing sufficient value. vices compared to only 5% of children under • The Ministry was not collecting information the direct service option. The Ministry has not that would be useful to help it monitor com- collected data that would indicate whether pliance with program guidelines or the quality children’s outcomes were better under one of services provided. option compared to the other. • The Ministry has not collected information or • Since 2006, the Ministry has reimbursed up set targets that can be used to assess program to 60 individuals for a total of $21 million effectiveness and outcomes, even though it for the ongoing cost of IBI therapy outside identified relevant performance measures to of the regular service system. Per child, this do so almost 15 years ago. represents more than double the amount that a child in the regular service system typically OVERALL MINISTRY RESPONSE receives. Furthermore, some individuals were The Ministry of Children and Youth Services reimbursed for more than the maximum of 40 appreciates the work of the Auditor General and hours a week of service, as well as for expenses welcomes input on how it can further improve

Chapter 3 • VFM Section 3.01 not directly related to their therapy. Expenses autism services in Ontario. included holding fees to retain a spot with a Since Ontario implemented its first autism therapist and the cost of trips and admission program, the Ministry has increased funding to local attractions. Children in the regular for autism services and supports, from an service system are not entitled to these. initial investment of $14 million in 2000/01 to Both the Ministry of Children and Youth • $182 million in 2012/13. New areas of research, Services and the Ministry of Education have approaches to diagnosis, prevalence rates and taken some actions to address the 34 recom- treatments for autism spectrum disorders (ASD) mendations contained in the 2007 document are continually emerging and shifting the aut- entitled “Making a Difference for Students ism service-delivery landscape. As well, children with Autism Spectrum Disorders in Ontario and youth with ASD are not a uniform group; Schools.” However, more work is necessary to their needs vary depending on the severity ensure that ABA methods are being effectively of their ASD, their cognitive functioning and used to educate children with autism. Almost their adaptive behaviours. The government is half of all schools boards reported in 2012 committed to providing responsive services and that they were not always incorporating ABA supports that are based on research evidence for techniques into programs for students with this growing and diverse group of young people. autism. Only 38% of school boards reported Autism Services and Supports for Children 57

The Ministry continues to increase funding for direct treatment for children and youth with Detailed Audit Observations ASD, expand the range of services available, as well as increase support for families and training AUTISM STRATEGY for service providers. These autism-specific ser- Canada does not have a national strategy on aut- vices and supports are just some of the services ism. In March 2007, a Senate committee recom- that children with autism and their families mended that the federal government establish a can access. Some children with ASD may also comprehensive national autism strategy in collab- access other services for children and youth with oration with the provinces and territories. However, special needs, such as rehabilitation services, no such strategy was developed because both mental health services and respite programs. consensus and evidence on autism-related issues In addition, the Ministry has taken the fol- was lacking. Instead, the federal government has lowing steps: chosen to address knowledge gaps by, among other In December 2012, it established the ASD • things, funding research and associated initiatives. Clinical Expert Committee, an expert panel Ontario does not have a provincial autism to provide the Ministry with clinical guid- strategy. However, in May 2013, the provincial legis- ance on up-to-date, evidence-based research lature passed a motion to create a select committee on autism that will help inform the design to work on a comprehensive developmental services and administration of autism programs in strategy for Ontarians. This strategy is to address the Ontario. needs of children and adults with a developmental In August 2013, it began a review of aut- • disability, including autism, and to co-ordinate the ism services with a view to improving early delivery of developmental programs and services identification, access to early diagnosis and across many provincial ministries. In particular, the intervention, efficiency of service delivery, committee is expected to consider the following and families’ experiences with the AIP and types of needs: educational, work related, social ABA-based services.

and recreational, and housing, as well as supports Chapter 3 • VFM Section 3.01 It has been collaborating with partner minis- • for parents such as respite care. The committee tries to streamline access to services—specif- was established in October 2013 and is expected to ically, supporting children transitioning from present a final report in May 2014. IBI to school since 2008/09; and supporting Other provinces, including Saskatchewan, youth transitioning from school to adult Manitoba and Nova Scotia, have released autism developmental services, further education, action plans within the past five years. Most of employment and/or community living start- these plans highlight the need for better access to ing in 2013/14. professionals for more timely diagnosis, so that The Ministry is also planning to re-allocate children with autism may receive interventions at $5 million to the AIP in the 2013/14 fiscal a younger age. In addition, since 2008, many U.S. year to increase IBI spaces and consequently states have implemented autism plans that include decrease wait lists. Most of the funds will be re- partnerships between professionals and children’s allocated from the School Support Program. families, access to financing for services, early and continuous screening for autism, community services organized for easy use, and transition services for youth entering the adult system. 58 2013 Annual Report of the Office of the Auditor General of Ontario

DIAGNOSIS at time of referral to autism services. We calculated the median age at time of referral for all children on Numerous studies indicate that early intensive the wait list for IBI services at the end of February intervention can significantly enhance outcomes 2013 in the three regions we visited and found it to for children with autism. As a result, early diagnosis be 38 months. is key. Currently, there are no biological tests that can detect autism. Autism is usually diagnosed by behavioural evidence such as observing the child ACCESS TO INTERVENTION SERVICES and/or obtaining a history on the child’s develop- The Ontario government funds two types of aut- ment from parents, caregivers or speech-language ism intervention services or therapies—intensive pathologists. In Ontario, only those children who behaviour intervention (IBI) and applied behaviour have been formally diagnosed with autism may analysis (ABA)-based services. According to the apply for provincially funded autism services Ministry, IBI focuses on improving the rate of a and supports. A family physician, psychologist child’s learning and his or her progression across a or developmental pediatrician must provide the broad range of skill areas, while ABA-based services formal diagnosis. Since no data is collected by the focus on mastering specific skills, often one at a Ministry of Children and Youth Services, the Min- time, and learning to apply them in everyday set- istry of Health and Long-Term Care or the Canadian tings. These services are available to children up Paediatric Society on the wait time to get such a to their 18th birthday. Some children qualify for diagnosis, we inquired at each of the three IBI ser- both types of interventions. Figure 2 describes the vice providers we visited. Based on their experience differences between IBI and ABA-based services as with children and families who are referred to their offered in Ontario. intervention services, they said the process to get a Intervention services are delivered by commun- diagnosis could take three to 12 months, depending ity agencies. The Ministry has selected nine lead on where in the province someone lives. service agencies to deliver IBI services and 13 lead The Canadian Paediatric Society endorses service agencies to deliver ABA-based services. screening children for autism spectrum disorders

Chapter 3 • VFM Section 3.01 Lead service agencies may subcontract with other between the ages of 18 and 24 months if a parent service providers to help deliver services in their expresses developmental concerns or a child has region/area. Lead service agencies are responsible risk factors, such as an older sibling with autism or for all aspects of service delivery, including clinical problems with social or communication skills. The decisions regarding eligibility, service intensity and U.S. Centers for Disease Control and Prevention duration, and time of discharge; wait list manage- recommends that children be screened for autism ment; and transition support. The Ministry has at 18 months and again at 24 months. developed program guidelines for both IBI and The Ministry does not have data on the age at ABA-based services. The Ministry’s nine regional which children are first diagnosed, even though offices are responsible for monitoring service agen- one of the objectives of the IBI program when it was cies to ensure they conform to these guidelines. announced in 1999 was to identify children with Families whose children are accepted in the IBI autism by age 3 in order to maximize their oppor- program have a choice between two service deliv- tunities for early learning. However, service provid- ery options. ers maintain data on the age of referral to their Direct service option: The Ministry provides services. Based on the assumption that a child will • funding directly to the lead service agencies, be referred for IBI therapy soon after diagnosis, the which hire therapists for children with autism, age at time of diagnosis should approximate the age and provide ancillary services such as parent Autism Services and Supports for Children 59

training and resource material. There is no hours of service to which a child is entitled cost to the recipient. and funds parents $39 per hour to purchase • Direct funding option: The lead service private IBI services. The lead agency must agency determines the number of approved approve the private IBI provider selected by

Figure 2: Comparison of Intensive Behaviour Intervention (IBI) and Applied Behaviour Analysis (ABA)-based Services Prepared by the Office of the Auditor General of Ontario

IBI Services — Start Date 2000 ABA-based Services — Start Date 2011 Service description IBI is an intensive application of ABA to teach ABA uses methods based on scientific new skills. It involves a step-by-step process principles of learning and behaviour to that teaches language, social interaction, promote positive behaviours and reduce play, fine motor skills and self‑help skills. problematic ones. ABA-based services Each skill is broken down into its simplest provide time-limited skill-building services components and then taught through to children with autism. These services are constant repetition and reinforcement. The intended to improve communication, social/ goal is to create pathways in the child’s brain interpersonal, daily living and behavioural/ to support normal functioning. emotional skills. Parents learn the strategies taught to their children and can incorporate these techniques into daily activities. Treatment delivery mode Primarily one to one. Primarily group-based. Setting Primarily service-provider location or home. Primarily service-provider location or Children usually receive services when other community (e.g., grocery store, public children are in school; some children may transit). attend school part-time. Children receive services after school or on weekends. Intensity and duration 20–40 hours per week, delivered for 2–3 2–4 hours per week, delivered for 2–6 years. months. What happens at the end of Child is discharged. Reapplication is not Child is discharged, but may reapply to

service block permitted. further develop skills or to address new Chapter 3 • VFM Section 3.01 needs. Who provides this service The Ministry contracts with 16 service The Ministry contracts with 13 lead providers (9 lead service agencies and 7 service agencies who partner with over 40 additional agencies in the 2 regions with the subcontractors to deliver ABA-based services largest demand for service). Some service and supports. providers subcontract with other providers to deliver IBI. Who is eligible Children at the severe end of the autism All children with an autism diagnosis. spectrum, as determined by the lead service agencies. Number of children discharged 675 6,500 from service in 2012/13 Number of children receiving 2,000 6,200 services in 2012/13 Number of children waiting for 1,700 8,000 services on March 31, 2013 Age of children in service Median age is 7; 90% are aged 10 and Median age is 8; 90% are aged 14 and under (as of October 2012). under (as of June 2012). Average provincial cost per child $56,000 per year $2,800 per block of service 60 2013 Annual Report of the Office of the Auditor General of Ontario

the parent. If the private service provider Ministry introduced an independent review mech- charges more than $39 per hour, the parent anism where parents can appeal when their child is pays the difference. assessed to be ineligible for service. The IBI lead service agencies administer One-quarter of children who apply for IBI are both service delivery options and determine the declined services because their autism is not con- number of spots available for each option in their sidered severe enough. Research suggests that these region. At the time of our audit, about 60% of IBI children would do better with IBI. For example, recipients had chosen the direct service option. a 2005 study found that treatment outcomes for Although families under the direct funding option IBI were best predicted by pretreatment language may be required to pay out-of-pocket expenses, skills, social responsiveness and the ability to mimic we were told that those who choose this option do others. Similarly, a 2010 study concluded that bet- so because they may not wish to switch from the ter IBI treatment outcomes are linked to, among private provider they started with while waiting other things, children who initially had higher for government-funded services or because it gives adaptive behaviour abilities. Further, the results them more control over scheduling sessions. from a 2006 study commissioned by the Ministry appear to lend support to this research. Although the study was of children with severe autism only, Eligibility for Intervention Services it did find that children in this group who were Although a child might be diagnosed with autism, initially higher functioning made the most progress. ministry guidelines restrict IBI services to children In particular, 57% of the children in the higher- up to the age of 18 whose autism is more severe. functioning group achieved average functioning or By comparison, children with autism in most other had substantial improvement, compared to only 7% provinces are eligible for IBI services regardless of of the lower-functioning children. This highlights severity, but only until they start school. that IBI is potentially more effective when a child is Eligibility assessments are conducted by clinical already higher functioning to begin with. staff and approved by the clinical director at each Based on our discussion with service providers

Chapter 3 • VFM Section 3.01 lead service agency. Clinical staff usually include and a review of their data, we noted the following: therapists with either a community college dip- • The Ministry does not collect data on the loma, university undergraduate degree or graduate length of time between referral and eligibility degree in psychology or a related field. Clinical assessment, so we obtained and analyzed directors are required to have a doctoral degree data from two of the three regional service in psychology and to be registered or eligible for providers we visited. For children who began registration with the College of Psychologists of receiving IBI services in 2012, 75% of them Ontario. Ministry guidelines require that eligibility were assessed within six weeks in one region, be assessed within four to six weeks after an IBI whereas in the other region only 28% were referral is received. assessed within six weeks. We could not use Our analysis of ministry data for the period data from the third region we visited because, 2009 to 2012 showed that IBI service providers contrary to ministry guidelines, children were declined almost 1,900, or 34%, of assessed IBI placed on the wait list before a diagnosis of applicants. In the service providers’ opinion, 74% of autism was confirmed, and as a result eligibil- the declined applicants did not have severe autism, ity assessments were delayed until a firm diag- 24% were not expected to benefit from IBI, and nosis was obtained. The data from this service the remaining 2% did not have autism, contrary to provider did not indicate which children had a the physician’s diagnosis. In December 2012, the confirmed versus provisional diagnosis. Autism Services and Supports for Children 61

• The Ministry does not mandate (a) a common where families with financial means can acquire assessment tool or combination of assessment private services for their children while they wait tools or (b) the intake criteria that clinicians for government-funded services, but other families should use to determine IBI eligibility. The are unable to. agencies we visited use anywhere from four to seven tools to assess eligibility, of which Waiting for IBI Services two tools are common to all three agencies. In the five-year period ending December 2012, Clinicians use their professional judgment the IBI wait list has grown from 1,063 to 1,748. when determining whether a child is eligible The regions of Central East (covering York and for IBI. Research indicates that the choice of Durham regions, Barrie, and Peterborough), assessment tools is not straightforward, given Hamilton–Niagara, and Toronto account for 80% the wide range of ability that children with of the increase in the wait list. autism have. As seen in Figure 3, from 2008 to 2012, more One expert we spoke to told us that on occa- • children were waiting for IBI services than were sion the condition of a child who was assessed receiving services. The number of IBI spots as ineligible for IBI may worsen over time and remained relatively constant at 1,400 during this become more severe. Based on our discussion time, while the number of children waiting for IBI with clinical directors, children are not usually services increased by 23%. re-evaluated if they didn’t meet the IBI eligi- The Ministry does not track how long children bility criteria on the first try, unless the child’s wait for IBI services, but it did advise us that it will development changes. However, neither the start collecting data in 2013/14 to calculate average Ministry nor the lead service agencies had any wait times. We obtained and analyzed wait-time criteria or guidelines to indicate how signifi- data—that is, the length of time from referral to cant a child’s change in development would starting IBI—from the three regions we visited and have to be in order to warrant a re-evaluation. noted that the median wait time for IBI services in 2012, for all three regions combined, was 3.9 years. Wait Information It is up to lead service agencies to decide how Chapter 3 • VFM Section 3.01 to allocate ministry funding between the direct After being formally diagnosed, children with autism generally have to wait to access Ministry- Figure 3: Number of Children Waiting for, and funded autism services. For example, although half Receiving, Intensive Behaviour Intervention Service, the children with autism in the three regions we 2008–2012 visited are diagnosed by just over 3 years of age, Source of data: Ministry of Children and Youth Services more than 75% of children don’t actually start IBI 2,000 until after they turn 6. Similarly, about two-thirds 1,800 of children who start ABA-based therapy are 6 and 1,600 older. Children assessed as eligible are placed on 1,400 the wait list based on the date they are initially 1,200 referred to the program. During the time a child 1,000 is on the wait list, Ministry-funded agencies offer 800 Eligible children waiting for IBI on December 31 some support to families (such as parent education 600 Children receiving IBI service on December 31 and consultation). Children with autism might 400 also be waiting for government-funded speech and 200 0 occupational therapy. This has led to a situation 2008 2009 2010 2011 2012 62 2013 Annual Report of the Office of the Auditor General of Ontario

service option and the direct funding option, and We were informed anecdotally that some chil- wait times for IBI services can differ between the dren with persistent parents were able to access two depending on how lead service agencies have services more quickly than others who had been allocated their funding. In one region in 2012, placed on the wait list before them. To assess the average wait for IBI services under the direct the risk that some children may have received funding option was five months longer than the preferential treatment in accessing services, we average wait under the direct service option. In analyzed wait lists in the three regions visited and another region, the situation was reversed. The compared the order of children’s start dates for agencies involved said it is a challenge to find the IBI services with the order of their referral dates. appropriate mix of spots for each service delivery However, because the documentation maintained option because of the pressure to alter their clinical by the agencies in this area was not clear, we could capacity to meet ever-changing demand. not determine definitively whether any preferen- The last time the Ministry significantly increased tial treatment had been given at the agencies we funding for IBI services in order to reduce the wait visited. In addition, there may be some legitimate list was in 2007/08, when funding was increased reasons for out-of-sequence starts. For instance, by almost 30%, or $21.5 million. This resulted in a there are cases where children are better suited 25% increase in the number of children receiving to the type of delivery mode that becomes avail- services as of March 2008 compared to the same able (group session versus one-on-one), or where time the year before. However, there was no reduc- children are transferred from another area of the tion in the wait list; in fact, the wait list increased province and the original referral date is honoured. by 17% because the increase in service demand continued to surpass the increase in capacity. Waiting for ABA-based Services Since our last review in 2004, the Ministry has ABA-based services were fully implemented in made efforts to increase and retain the number of Ontario in February 2012. Based on province-wide therapists providing IBI services. Between 2004/05 data collected by the Ministry, the wait list for such and 2009/10, the Ministry provided $3.2 million to services almost tripled within one year—from 2,800

Chapter 3 • VFM Section 3.01 the Ontario College Graduate Program in Autism as of March 2012 to 8,000 as of March 2013—as and Behaviour Science to train almost 1,000 new more people became aware of the services. Children IBI therapists; between 2007/08 and 2009/10, the who started services in the 2012/13 fiscal year Ministry provided $1.3 million to train over 400 waited an average of 2.4 months to begin ABA- IBI therapists to deliver services under the direct based therapy. But the average wait time varied funding option model; and between 2005/06 and across regions from three weeks (Eastern region) to 2012/13, the Ministry paid $3 million in tuition over six months (Hamilton–Niagara region). reimbursements to 350 employees delivering services under the direct service option model who were upgrading their credentials. Despite these efforts PROVISION OF INTERVENTION SERVICES to enhance system capacity, service providers in Intensity and Duration of Service the three regions we visited told us that, while they have no problem recruiting IBI therapists, they do Numerous studies have examined the relative have trouble recruiting and retaining qualified sen- effectiveness of IBI intervention at varying degrees ior therapy staff to supervise them, and therefore of intensity. In general, the more intense the ther- have on occasion filled these more-senior clinical apy, the greater the gains in functionality. positions with less-qualified people. According to IBI program guidelines, children may be eligible to receive up to 40 hours of IBI Autism Services and Supports for Children 63 services per week, with the expectation that the that ramping up was common practice to ease the number of hours will generally fall within the range child into therapy, and ramping down was less of 20 to 40 hours per week (in other words, about common but could be appropriate for allowing a four to eight hours per day, five days a week). The child to start transitioning to school. The practice of approved hours are determined by clinicians at the ramping hours up or down is not clearly explained lead service agencies. Each quarter, the Ministry in the program guidelines, other than to state that collects data on the average number of approved the clinical director or supervising psychologist can hours for children receiving services, as well as the modify a child’s hours upon reviewing his or her highest and lowest number of approved hours from progress at regular intervals. each IBI lead service agency. Based on our review of Children might also be receiving fewer hours of ministry data for children receiving IBI services in service than they have been approved for because 2012, we noted the following: of cancelled therapy sessions. The program guide- • Children were approved for an average of 23 lines state that service hours lost, because either hours of therapy per week. the child or the therapist was unable to attend • The average approved amount of therapy the appointment, cannot be made up at a later across regions ranged from 21 to 27 hours time. In our 2004 autism review for the Standing per week, regardless of the service delivery Committee on Public Accounts (Committee), the option. In general, that difference translates service providers we reviewed were providing to an extra day of therapy each week. significantly fewer hours of service on average than • Only one region approved the maximum of the suggested minimum of 20 hours. In November 40 hours per week, and that was under the 2006, the Ministry informed the Committee that direct funding option. Under the direct service all service providers were required to track lost option, none of the regions approved more service hours and that the Ministry would meet than 35 hours of service a week. with lead service agencies to develop a more The Ministry does not collect data on the actual standardized approach to define lost service hours IBI hours provided. In addition, at the time of our across the province. The Ministry further indicated

audit only one of the three service providers we that agencies had been asked to track lost service Chapter 3 • VFM Section 3.01 visited tracked actual hours of therapy for children hours, but the Ministry did not receive that data. receiving services from private providers (direct We followed up with the lead service agencies we funding option), even though they approve the visited, and found that one agency was not aware invoices. Based on our review of actual IBI service of this requirement and had not been tracking lost hours under the direct service option, as recorded service hours. The other two agencies were tracking by two regional agencies, children who were lost hours only for those served under the direct discharged in 2012 received a median of 20 hours service option. In this case, lost service hours due of therapy per week over their entire course of to unavailable staff accounted for 10% of approved treatment, even though they had been approved hours at the one agency and 5% of approved hours for 27 and 30 hours, respectively, at the two agen- at the other. In any case, the Ministry was not mon- cies. The agencies told us that this was because itoring lost service hours or the reasons for them. they would “ramp up” to the full level of approved We also noted that, for the 675 children hours at the start of the service period and “ramp discharged during the 2012/13 fiscal year, on a down” hours closer to the end of the service period. province-wide basis, those under the direct funding In the middle period of service, additional hours option received Ministry-funded IBI services for were not provided to compensate for the ramp-up longer periods than those under the direct service and ramp-down. One expert we consulted told us option, as shown in Figure 4. Significantly more 64 2013 Annual Report of the Office of the Auditor General of Ontario

therapy and referred to other services. The agen- Figure 4: Percentage of Children Receiving Different cies told us anecdotally that the children had more Durations of IBI Services Under Each Service Delivery often been found unsuitable because they met or Option exceeded the skills-set that IBI is meant to teach. Source of data: Ministry of Children and Youth Services However, neither agency systematically tracked Direct Service Direct Funding the reasons children were found to be unsuitable Option (%) Option (%) at time of reassessment or analyzed whether the <1 year 22 19 children had participated in private therapy or >1–2 years 33 19 other types of interventions that could explain their >2–3 years 31 23 change in functionality. >3–4 years 9 16 ABA guidelines indicate that services typically >4 years 5 23 will be provided for two to six months and for two Note: Percentages are based on files for children discharged from IBI in the 2012/13 fiscal year. to four hours per week. Although the Ministry collects data on the length of time children who children under the direct funding option received have been discharged from ABA-based services services for longer than four years as compared to spent receiving them, it does not collect data on the children under the direct service option. On average, number of hours of services they received per week children under the direct funding option received to compare against recommended guidelines. We IBI services for 35 months, whereas children under obtained data from service providers in the three the direct service option received IBI services for 25 regions we visited on children who were discharged months—a difference of almost one year. The Min- from ABA-based services from inception in 2011 to istry had not followed up on these differences. December 2012, and found the data to be unreli- In addition, the average length of time in IBI able for analyzing both the duration and intensity varied across the province for both service delivery of services provided. options, as shown in Figure 5. For example, the average length of service obtained under the direct Discharge Decision

Chapter 3 • VFM Section 3.01 service option was 15 months in the Eastern region compared to 34 months in the Central East region. According to the Ministry, agency staff and The average length of service under the direct fund- stakeholder groups, one of the main complaints ing option ranged from 11 months in the Northern from families in the three regions we visited was region to 49 months in the South West region. disagreement with the discharge decision. The We also noted circumstances where children Ministry informed us that the decision to discharge did not receive IBI therapy once they got to the top a child from IBI therapy is a clinical one made by of the wait list. Two of the lead service agencies the lead service agency in each region, regardless we visited told us that they reassess children for of whether the child receives services provided by suitability once they get close to the top of the wait a lead service agency or private services paid by list because their functionality sometimes changes the Ministry. Between 2006 and 2012, more than during the wait period. Both agencies said that if 3,500 children in Ontario were discharged from a child is determined to be unsuitable for IBI upon IBI services as follows: 70% no longer needed IBI, reassessment, they encourage parents to seek other 8% declined services, 3% moved out of region, and services, such as ABA-based services. In 2012, 19% for other reasons. To understand what “no approximately 20% of the children in one region longer needed IBI” means, we sampled a number and 30% in another region who had got to the top of files in the regions we visited and found that of the wait lists were deemed unsuitable for IBI discharge reasons could include: benefits from IBI Autism Services and Supports for Children 65

Figure 5: Average IBI Duration in Months for Children Discharged During 2012/13, by Region and Service Delivery Option Source of data: Ministry of Children and Youth Services 50

45 Direct service option Direct funding option 40

Eastern Northern Toronto South East Central East Central West North East* South West Chapter 3 • VFM Section 3.01 Hamilton–Niagara *This region does not have any direct funding option clients due to a lack of private providers in the region. have been maximized, IBI has not been effective in were presented to the Ministry in September 2008. changing developmental level, IBI has been effect- In March 2010, the Ministry hired a consultant to ive and child will benefit more from learning in a conduct a pre-implementation review of the bench- natural environment, and child has not made any marks, but the consultant found that there was not significant progress since last assessment. enough information in the clinical files to be able to Over the years, attempts have been made conclude on the benchmarks. Although it has spent to establish consistent provincial discharge or $330,000 to date, the Ministry has not concluded “continuation” criteria. The first set of clinical on discharge criteria. Other jurisdictions do not continuation criteria was developed in 2006 by need discharge criteria given that their services clinical directors in service agencies but was not usually end when children reach a specified age. finalized. Instead, in November 2007, the Ministry In October 2012, the clinical directors of the IBI assembled an expert panel to determine “clinical programs agreed on and approved a common set of practice guidelines.” Subsequently, the Ministry discharge criteria, which are more comprehensive struck another panel to arrive at benchmarks for than previous benchmarks. The three lead service the newly developed guidelines. These benchmarks agencies we visited indicated that they would be 66 2013 Annual Report of the Office of the Auditor General of Ontario

implementing these revised criteria in their regions. ports and services, the Ministry of Children and But at the time of our audit, each was following Youth Services (Ministry) should: different discharge criteria, which included a work with the Ministry of Health and Long- combination of the updated clinical continuation • Term Care and the medical community to criteria from 2006 and the 2008 clinical practice facilitate the identification and diagnosis of benchmarks. However, we also noted that the autism in children before age 3, in accord- region that uses benchmarks indicated it would ance with the original objective of the Min- refer to them on a case-by-case basis, and clinicians istry’s intensive behaviour intervention (IBI) would apply professional judgment in making dis- program; and charge decisions. monitor wait times as well as wait-list data In December 2012, the Ministry formed yet • across the province for both IBI services and another expert panel to, among other things, pro- applied-behaviour-analysis-based services. vide advice on whether benchmarks are appropri- To help improve program transparency and ate for IBI discharge/continuation decisions. At the ensure equity of service in the best interests of time of our audit, the committee was expected to the child, the Ministry should: report back to the Ministry in October 2013. ensure that clear eligibility, continuation Also in December 2012, as mentioned previously, • and discharge criteria for IBI services are the Ministry introduced an independent review developed and are applied consistently, so mechanism, empanelled by a roster of independ- that children with similar needs can access a ent reviewers managed by an external agency, to similar level of services; be used when families disagree with the service ensure that service providers clearly specify, providers’ decision on eligibility or discharge. We • for every child, the reason that the child is were informed that reviewers will use their clinical discharged from the IBI program and report judgment to rule on whether the decision made by this information to the Ministry for analysis; the original IBI service provider was consistent with and the information noted in the child’s file. review the reasons for significant regional

Chapter 3 • VFM Section 3.01 At the time of our audit, the Ministry told us that • differences in the use of the direct service there has been and continues to be disagreement option and the direct funding option, and among the expert community on whether there ensure that decisions on the capacity to pro- should be a consistent set of discharge criteria. How- vide each service are being made objectively. ever, without consistent criteria, there is no assurance that clinicians assessing the same child would MINISTRY RESPONSE reach the same decision on whether the child should continue or be discharged from IBI. Furthermore, As part of a review of autism services initiated in there is a conflict, whether real or perceived, when August 2013, the Ministry is reviewing barriers the lead service agency is responsible for determin- to early identification, diagnosis, assessment ing when services should end, while at the same and treatment with a view to identifying oppor- time being responsible for managing wait lists and tunities for improvement. After focused discus- meeting targets for the number of people served. sions with families, research experts, health and medical professionals, and inter-ministerial RECOMMENDATION 1 partners in education and health, the Ministry will develop a plan to improve early identifica- To help ensure that children with autism and tion and access to diagnosis and assessment. their families have earlier access to autism sup- Autism Services and Supports for Children 67

In 2013/14, the Ministry began collecting will submit its report to the Minister in late fall and monitoring data that allows it to track aver- 2013, at which time the Ministry will review the age wait times for children who began receiving recommendations and consider next steps. services in the reporting period under the The Ministry currently collects some infor- Autism Intervention Program (AIP) by either mation related to discharge from the AIP, and service delivery option, and under ABA-based will consider collecting additional information services. The Ministry will consider collecting related to discharge. and monitoring wait-time data for children on The Ministry is aware that there are different the wait list. levels of demand for the direct service option The AIP guidelines clearly state that clinical and the direct funding option in various parts directors, who oversee the provision of IBI servi- of the province. Beginning in 2013/14, the Min- ces, are responsible for eligibility and discharge istry started to collect distinct wait-time data for decisions. Clinical directors are regulated health each service delivery option to help determine professionals and are responsible for taking into appropriate adjustments to program design. The account the individual goals of each child, using wait-time data that the Ministry has now started their clinical judgment and the most up-to-date to collect will allow the Ministry to measure research to make decisions. agencies’ success at matching their capacity to In 2012, the Ministry established the the demand for each service delivery option in Independent Review Mechanism to allow for their regions. arm’s-length reviews of eligibility and discharge decisions. Independent reviewers, who are regulated health professionals, conduct reviews Appropriateness of Intervention Methods to determine whether the original AIP clinical decisions are consistent with the information Since IBI is available only to children whose aut- in children’s anonymized case file materials ism is found to be toward the severe end of the based on the AIP guidelines, up-to-date IBI spectrum, children with milder forms of autism research and their clinical judgment. As of mid- qualify only for ABA-based services. However, ser- Chapter 3 • VFM Section 3.01 September 2013, independent reviews had been vice providers in the regions we visited told us that completed on 93 cases. In almost 90% of cases, the ABA-based services might not be sufficient for reviewers were in agreement with the original those who have a host of behavioural problems or decisions made by clinical directors in the AIP. goals to achieve. The reason for this is that the ABA As the Auditor General has described, the program allows a child to work on only one goal Ministry has sponsored several attempts to at a time and then requires that the family reapply develop consistent decision-making criteria if it wants the child to receive another ABA-based for the AIP. However, specific decision-making service. The child returns to the wait list in the criteria have not been implemented due to meantime. Experts to whom we spoke indicated the results of an impact analysis on children that these separate blocks of therapy do not work receiving IBI and the emergence of continually for correcting all types of behaviours, because gains evolving research. The Ministry established made might be lost in between blocks. According the Clinical Expert Committee to, among other to one expert, this approach will not work for some things, provide clinical guidance on evidence- behaviour targets that are interrelated and that based research, including advice on clinical prac- should therefore be worked on at the same time. tice guidelines and benchmarks. The Committee Children with multiple behavioural problems likely 68 2013 Annual Report of the Office of the Auditor General of Ontario

require more intense support than is offered under the Ministry is developing a resource kit that would the current ABA model. provide families of children diagnosed with autism Other methods of intervention that have been with information about all stages of their child’s proposed to the Ministry include the following. progress and development. The goal of the resource • In 2008, the Ministry commissioned an expert kit is to increase understanding of autism and the clinical panel to look into non-IBI interven- range of programs and supports available. tions. The review concluded that there is sufficient evidence to recommend at least two RECOMMENDATION 2 focused interventions (Picture Exchange Com- To help ensure that children with autism munication System, which is an aid for people have access to evidence-based interventions with autism and other special needs, and appropriate to their needs, the Ministry should an intervention that focuses on facilitating consider the costs and benefits of offering addi- attention skills) as part of a comprehensive tional types of therapies and interventions not program based on the developmental needs currently provided, and existing interventions at of the child. We asked the Ministry whether various degrees of intensity. these methods had been adopted and were told that clinicians can use their judgment in MINISTRY RESPONSE deciding whether to complement ABA and IBI therapies with these two interventions. The Ministry agrees that children should have • In June 2010, a working group made up of access to evidence-based interventions appro- service providers from across the province rec- priate to their needs. ommended to the Ministry a model for an ABA- As noted, the Ministry established the based continuum of services that included Clinical Expert Committee to confirm that three levels of intervention depending on the Ministry-funded autism programs continue to child’s needs. The model essentially includes be consistent with the latest research findings. the current ABA program, the IBI program and The Committee is examining evidence-based

Chapter 3 • VFM Section 3.01 a “specialized ABA-based service” that would research on autism interventions and will pro- offer six to 12 months of individualized servi- vide the Ministry with advice to inform policy ces for three to 12 hours per month to match design and program development. the child’s needs. However, the Ministry has not adopted this recommendation. In December 2012, the Ministry formed a panel Intervention Services Funded Outside the of experts to, among other things, identify effective Regular Program interventions and treatments besides IBI and ABA- based services. Over the last seven years, the Ministry has reim- At the time of our audit, the Ministry was relying bursed up to 60 individuals a total of $21 million on parent training to provide parents with strategies for the cost of IBI therapy and other expenses, to support their child’s development and manage outside of the regular service system. his or her behaviours at home as they await services, Over 40 of these individuals were still actively or to complement the strategies used by therapists. claiming IBI therapy and other costs as of March 31, Parent workshops and parent training sessions are 2013. These individuals, ranging from 14 to available under both IBI and ABA services; almost 25 years old, have not followed the regular IBI 40% of ABA service hours are provided to parents/ progress review process. As a result, the Ministry caregivers. To further support parents/caregivers, does not know whether their needs have changed Autism Services and Supports for Children 69 or are significantly different from the needs of support and to address existing inequities, the those funded through the regular program. These Ministry of Children and Youth Services should individuals have been receiving services for at least apply the same program guidelines to all those twice as long as children in the regular program. who meet the eligibility criteria. In addition, at the time of our audit over half were 18 years or older and hence would no longer qualify MINISTRY RESPONSE for any services offered by the Ministry. On several occasions, the Ministry had considered options for The Ministry agrees that children with autism transitioning this group to mainstream programs and their families should be treated fairly and (including adult services offered by the Ministry of equitably. The Ministry will consider options for Community and Social Services), but these plans meeting this objective for families who receive were never implemented. funding outside the regular program. Some individuals’ claims were processed by the Ministry and others by the lead service agencies in the regions where they reside. We noted that AUTISM SERVICES AND SUPPORTS IN individuals whose claims were processed by the SCHOOLS Ministry submitted (and were reimbursed for) higher-value claims than those whose claims were Children spend up to six hours a day in school, and processed by the lead service agencies. this will start at younger ages as Ontario fully imple- We reviewed all claims submitted for reimbursements full-day kindergarten by September 2014. ment by a sample of individuals in the 2011/12 According to the Ministry of Education, in 2011/12 and 2012/13 fiscal years to determine if claimants about 16,000 students in publicly funded schools were reimbursed for the same type and level of had been formally identified with an autism spec- services and at the same rates as those funded trum disorder (ASD) by an Identification, Placement under the direct funding option. At the one service and Review Committee. There may be many others agency we visited, they were; but at the Ministry, who have not been formally identified. As previously

they were not. Specifically, we noted that almost noted, most of these students will not have begun Chapter 3 • VFM Section 3.01 half of the individuals we sampled who had their any therapy by the time they enter school. claims processed by the Ministry were consistently Under the Education Act, schools are to provide reimbursed, over many months, for therapy beyond appropriate supports to children with special needs, the maximum allowed 40 hours per week. In addi- including autism, while also attending to the needs tion, the Ministry reimbursed expenses to which of the other children in the classrooms. Special edu- children under the regular government-funded pro- cation staff in school boards we interviewed told us gram are not entitled, such as two months’ worth that most children with autism are placed in regular of “holding fees,” totalling about $6,500, to hold classrooms; some are placed in special education the individual’s time slot with his or her therapist classrooms along with students with other types over the summer months; the purchase of a laptop of exceptionalities; and a very small number with computer; admission to local attractions; and travel significant behavioural problems are placed in seg- costs incurred to fly in therapists for consultation. regated school settings with additional resources. In September 2006, the Minister of Children RECOMMENDATION 3 and Youth Services and the Minister of Education assembled a group of experts to provide advice on To ensure that children with autism and their improving school supports for children with aut- families receive an equitable level of service and ism. The group members were asked to identify 70 2013 Annual Report of the Office of the Auditor General of Ontario

successful education practices in Ontario and other consultants) to provide training and consultation jurisdictions; provide advice based on their back- services to educators (school administrators, teach- ground and expertise; and produce a report with ers and education assistants) to help them under- recommendations to be presented to both ministers. stand how the principles of ABA can be applied to The group’s February 2007 report, “Making a Differ- improve the way that students with autism learn. ence for Students with Autism Spectrum Disorders The Ministry’s program guidelines do not specify in Ontario Schools,” contained 34 recommendations credentials for ASD consultants, other than to state for province-wide implementation. The two min- that they require superior skills (knowledge of istries involved were responsible for implementing autism, ABA principles and behavioural teaching those recommendations that applied to them. The strategies) generally obtained through education ministries provided us with actions they have taken and experience in a relevant field. In April 2012, on each recommendation. Some action has been agencies that deliver the SSP also launched a taken on all recommendations. Notable actions are website to provide school boards with an online highlighted in Figure 6. resource guide on effective educational practices for students with autism. Online resource tools are beneficial from the perspective that teachers and Autism Training for Educators education assistants can access them when needed. The Ministry of Children and Youth Services We noted the following concerns with the introduced the School Support Program (SSP) in School Support Program: 2004 to enhance supports available to publicly • There were significant variances in the funded school boards for students with autism. The activities of ASD consultants across regions program is delivered by the same nine lead service in the 2011/12 fiscal year. For example, the agencies that deliver IBI services. It employs about average number of service hours per consult- 150 autism spectrum disorder consultants (ASD ant, for training, planning, consulting and

Figure 6: Notable Actions Taken on 2007 Report Entitled “Making a Difference for Students with Autism

Chapter 3 • VFM Section 3.01 Spectrum Disorders in Ontario Schools” Prepared by the Office of the Auditor General of Ontario

Ministry of Education Implemented requirement that appropriate ABA teaching strategies be incorporated for students with autism. Provided $37 million from 2006/07 to 2012/13 to school boards and the Geneva Centre for Autism for educator autism training. Provided $45 million from 2008/09 to 2012/13 to school boards to hire professionals with ABA expertise to provide training in ABA teaching strategies and to enhance collaboration between service providers and schools. Hosted ABA Professional Learning Days in March 2012 and May 2013 to promote the sharing of evidence-based resources and effective practices. Ministry of Children and Youth Services Implemented Connections for Students model, which uses transition teams to help children with autism move from IBI services to schools. Funded a variety of support programs to help families care for children with autism, such as respite programs, March Break Reimbursement Fund, and summer camps. Together with the Ministry of Education and the Ministry of Health and Long-Term Care, developed a shared vision for integrated speech and language services for children from birth to Grade 12 to enable seamless access to such services in a more timely and equitable manner. The proposed model is being tested at select sites since 2011. Together with the Ministry of Education and the Ministry of Community and Social Services, is implementing integrated transition planning for young people with developmental disabilities starting in the 2013/14 school year. Autism Services and Supports for Children 71

resource development combined, ranged transition between various activities and settings. from 137 hours to 1,009 hours, and the aver- To support this policy, the Ministry of Educa- age number of educators and support staff tion provided school boards with new funding served by each consultant ranged from 177 ($11.3 million in 2012/13) to hire board-level pro- to 1,321. We noted that of the three agencies fessionals with ABA expertise to provide support, we visited, only one could account for all of including coaching teachers on ABA techniques, its consultants’ time. The other two indicated and to enhance collaboration between service that their ASD consultants worked part-time providers and schools. In addition, since 2006, the on SSP initiatives and spent their remaining Ministry has provided $37 million to school boards time providing ABA-based services. In other and the Geneva Centre for Autism to provide autism words, service providers were using SSP fund- training to educators. The Geneva Centre for Aut- ing to deliver ABA-based services, for which ism is an organization in Ontario that provides clin- they had already been separately funded. The ical intervention services and training programs. Ministry had not analyzed the information to We noted the following about the initiatives identify the causes of such variances among implemented by the Ministry of Education: regions, nor was it aware of the inappropriate • In 2008, Autism Ontario surveyed parents use of SSP funding. of children with autism and found that 45% • The Ministry does not require service provid- reported that ABA methods were never incor- ers to survey all publicly funded school boards porated into programs for their children, and to determine how useful they found the an additional 34% said ABA methods were services of the ASD consultants and whether incorporated only some of the time. The Min- the consultants met the needs of the school istry of Education has surveyed school boards boards. Representatives from three of the four annually on this same issue and has noted a school boards we interviewed told us that they slight improvement in this area. Specifically, don’t use Ministry-provided ASD consultants in 2012, 56% of school boards reported that very much, because they have their own ABA programs for students with autism always

expertise in-house, and as a result the consult- incorporated relevant ABA methods, com- Chapter 3 • VFM Section 3.01 ant added little or no value. pared to 51% in 2009. • One school board told us that it preferred • The Ministry of Education has recommended to pay for a commercial web-based autism to school boards that staff with ABA expertise resource tool for teaching strategies, rather have the following competencies: postgrad- than use the SSP’s online resource tool at no uate studies or equivalent field experience in charge. An expert we spoke to also highly val- autism and behavioural science; experience ued the commercial tool. Prior to the develop- working with children and youth who have ment of the online resource tool, the Ministry special education needs (particularly those had not instructed agencies to review whether with autism); and training in ABA principles existing commercial online resources could from a recognized institution. However, the meet educators’ needs. Ministry of Education did not ensure that The Ministry of Education has also introduced school boards hired such staff with the recom- a number of initiatives in recent years to help mended competencies. educators teach students with autism. The most • Neither the Ministry nor the Ontario College significant was the 2007 implementation of a policy of Teachers (College), the body responsible to incorporate ABA methods into programs for for accrediting Ontario’s teacher education students with autism and provide planning for the programs, can provide specific data on the 72 2013 Annual Report of the Office of the Auditor General of Ontario

amount and content of special education Students (Connections) initiative in 2008/09. By training currently provided by faculties of March 2010, the initiative had been implemented education under existing teacher education province-wide. programs. Starting in September 2015, when The Connections initiative is centred on a multi- all teacher education programs in Ontario are disciplinary, student-specific, school-based transi- expanded to two years, they will include an tion team that includes parents, school board staff, enhanced focus on special education. The Col- a principal or designate, and an ASD consultant lege, with input from the Ministry and others, from the Ministry-funded agency that delivers the will also be developing a guide for faculties School Support Program. This team is established of education with examples and details of approximately six months before the child leaves expected course content. This is an opportun- the IBI program and is intended to provide support ity for the Ministry to help ensure that future until at least six months after the child starts or educators obtain the necessary knowledge to returns to school. help school boards comply with the Ministry’s In 2011/12, about 1,200 children received tran- 2007 policy on incorporating ABA methods sition support services in the Connections initiative, into programs for students with autism. which we calculated represents over 90% of those • According to the Ontario College of Teachers, children who were discharged from IBI within the teachers who complete a qualification course applicable period (from October 1, 2010 to Sep- about teaching students with communication tember 30, 2012). The service agencies we visited needs and autism are exposed to ABA meth- estimated that their ASD consultants spend 25% to ods. But as of May 2013, only 500 of Ontario’s 55% of their time on Connections matters. 234,000 teachers had completed this course. We reviewed a sample of files for children At the time of our audit, the Ministry of discharged from IBI between April 2011 and Febru- Education told us that over 16,000 educators ary 2013, and determined that, for the most part, have been trained by school boards or the children’s strengths, needs and issues related to the Geneva Centre for Autism to use ABA teaching transition process were discussed in monthly transi-

Chapter 3 • VFM Section 3.01 strategies in the classroom. Overall, however, tion meetings in the presence of an ASD consultant, according to the Ministry of Education’s 2012 the child’s parent and teacher. However, in 20% of survey, 62% of school boards reported that cases, there was no evidence that ASD consultants not all their teachers who work with children transferred instructional strategies involving ABA to with autism have had formal training in school staff. ABA strategies. At the four school boards we visited, this lack of formal training was some- RECOMMENDATION 4 what mitigated by the fact that they had their To better ensure that children with autism own ASD resource teams with whom teachers receive cost-effective supports while in school, could consult. the Ministry of Children and Youth Services, in conjunction with the Ministry of Education, Transitioning from Community-based should: Intervention to Schools • review the need for the use of autism spectrum disorder (ASD) consultants at many To help children leaving the IBI program to start school boards that already employ people to school or return to school full-time, the Ministry of provide similar services, and ensure that all Children and Youth Services along with the Min- ASD consultants are effectively utilized; istry of Education introduced the Connections for Autism Services and Supports for Children 73

• define minimum training requirements to MINISTRY OF EDUCATION RESPONSE assist existing and future educators to use The Ministry of Education recognizes the applied behaviour analysis (ABA) principles importance of training educators who work in the classrooms, and monitor uptake of or may work with students with ASD to use these education programs; and ABA principles in the classroom. The Ministry assess the usefulness of various online • recently established a provincial ABA Expertise and other resource tools available to assist Working Group to define training requirements teachers with effective educational practices to assist educators in incorporating and using for students with autism, and facilitate cost- ABA principles in the classroom. It also plans effective access to the best tools available. to conduct regional consultations in spring 2014. The Ministry plans to communicate these MINISTRY OF CHILDREN AND YOUTH training requirements at the third annual ABA SERVICES RESPONSE Professional Learning Day in April 2014. The School Support Program (SSP) was The monitoring of the uptake of ABA train- designed so that its ASD consultants work ing is conducted at the school board level. closely with school boards to customize their However, the Ministry will annually monitor services based on local needs and, as a result, how training requirements are implemented by delivery of the program may vary across the school boards starting in 2015. province. When the program was first intro- The Ministry will communicate training duced in 2004, there were few autism-specific requirements to assist educators in incorporat- or ABA supports available in schools. Since ing and using ABA principles in the classroom to then, school boards have developed increased the Ontario College of Teachers and faculties of expertise and capacity to support students with education as an example of an effective special ASD. In the context of this increased school education instructional strategy. board capacity, as well as the cumulative posi- The Ministry recognizes that in recent years

tive impact of the SSP in building capacity a wealth of research and resource materials Chapter 3 • VFM Section 3.01 among educators, the Ministry has taken some has become available on how best to support initial steps to review the SSP and is planning to students with ASD. The ABA Expertise Working move $3.6 million in 2013/14 and $4.5 million Group is expected to identify resources that in 2014/15 from the SSP to the AIP to relieve have proven to be effective in improving the some of the wait-list pressures for IBI services. outcomes for students with ASD by spring 2014. The Ministry will direct service providers Such resources will be disseminated via an to prioritize SSP services that are child-specific online forum for professionals with ABA exper- (for instance, the Connections for Students tise that the Ministry plans to launch in spring initiative) over other types of SSP services 2014. The Ministry will continue to facilitate provided to school boards (for instance, board- educators’ access to the best tools on how to wide training or resource development). The support students with ASD. Ministry will continue to work with the Ministry of Education to assess how to use the program’s remaining resources to best meet the needs of Transition Services for Older Children children with ASD. Changes, such as moving from elementary to secondary school, entering adolescence or completing 74 2013 Annual Report of the Office of the Auditor General of Ontario

secondary school, can be challenging for children have the means to at least track students with aut- with autism and the people responsible for their ism who go on to college or university. care. In 2011 the Ministry of Children and Youth Since 2006, the Ministry has provided annual Services and the Ministry of Community and funding totalling $1.5 million to approximately 40 Social Services introduced a transition planning agencies to help children with autism transition framework for young people with developmental into adolescence and high school. These agencies disabilities and indicated that planning is to begin provide services such as developing interpersonal early. In December 2012, the Ministry’s regional and coping skills; coaching youth with employ- offices implemented protocols to formalize transi- ment, volunteer or recreational activities; crisis tion planning responsibilities between organiza- intervention; behavioural supports; and family tions funded by either the Ministry of Children and counselling and support groups to give parents the Youth Services or the Ministry of Community and skills to help their children transition. This fund- Social Services, and to help establish expectations ing is also used for purposes other than transition for a more systematic, co-ordinated and transpar- planning, such as enhancing respite services and ent approach to transition planning for youth with training parents or caregivers on the disorder. developmental disabilities. In January 2013, the In the 2012/13 fiscal year, the program served initiative was expanded to include the Ministry of approximately 1,000 youths and their families. Education with the intent to help support smooth Based on our discussions with service providers transitions from secondary school to adult develop- that deliver transition programs in the three regions mental services, further education, employment, we visited, access to these programs varied from participation in life activities and community living. referrals through schools to youth hand-picked by The inclusion of the Ministry of Education in the the agency. Wait time for such services could range transition planning process required revising the from 4 months to 3 years. recently implemented protocols. The protocols took To help children transitioning within the school effect at the start of the 2013/14 school year, after system (for example, moving from one grade to the our audit fieldwork had been completed. The agen-

Chapter 3 • VFM Section 3.01 next or changing schools), boards we visited have cies are expected to implement transition planning autism resource teams, ABA experts, and special for youth as part of their existing program funding. education resource teachers to support teachers who We noted the following concerns with the transi- have students with autism. These supports include tion planning process: providing advice to teachers in developing behav- • It is unclear whether community agencies that iour safety plans and individual education plans, serve youth or adults with autism are required responding to crisis situations, and providing link- to participate in transition planning. The ages to post-secondary schools and work experience. transition planning protocols are designed When it comes to transitioning youth to the for youth and adults with developmental adult system, representatives from school boards disabilities and are not specific to youth and stakeholders told us there is a shortage of with autism. Neither the protocols nor the adult services, so some parents stop working to framework define developmental disabilities. stay home with their adult child. In addition, The Ministry recognizes that the meaning of school boards did not generally collect data on developmental disabilities currently differs what becomes of youth with autism after they under the Services and Supports to Promote the leave school. By 2014, all Ontario students will Social Inclusion of Persons with Developmental have a unique identifier that will follow them to Disabilities Act, 2008 and the Child and Family post-secondary education. The government will Services Act, and there is no definition under Autism Services and Supports for Children 75

the Education Act. In September 2013, an RECOMMENDATION 5 implementation guide developed by the ministries of Children and Youth Services, Educa- To help ensure that appropriate services and tion, and Community and Social Services supports are available to persons with autism was provided to school boards to advise them as they prepare to leave the children and youth that students with exceptionalities besides system, the Ministry of Children and Youth Ser- developmental disabilities—for instance, vices, in conjunction with the Ministry of Com- autism—could be considered eligible for tran- munity and Social Services and the Ministry of sition planning. However, it also stated that Education, should develop processes to assess community agencies were “expected to con- whether individuals with autism made suc- tinue to use their current practice regarding cessful transitions, including surveys to gauge the definitions of developmental disabilities.” satisfaction for those who made the transitions • The Ministry is unable to tell us how many and their families. youth with autism will be addressed by this framework. Anticipating that there would be MINISTRY RESPONSE a large volume of youth and adults affected by The Ministry of Children and Youth Services, the this transition planning initiative, the Ministry Ministry of Community and Social Services, and has prioritized its implementation to first serve the Ministry of Education have worked collab- adults still residing in children’s residential ser- oratively to establish processes that support inte- vices, then young people aged 14 to 17 who are grated transition planning. Through integrated in children’s residential services, and finally transition planning processes, young people with young people 14 and over who are not in chil- developmental disabilities, including autism, dren’s residential services. The Ministry does will have a single integrated transition plan that not have data on the number of youth with will inform educational planning, help the young autism in each of the three priority groups. person transition from secondary school and People we consulted suggest that most youth child-centred services to adulthood, and help

with autism are in the last priority group. Chapter 3 • VFM Section 3.01 prepare parents or guardians and other family We reviewed the outcomes listed in the frame- • members for these transitions. work and noted that they were mainly focused As part of the implementation of integrated on aspects of the transition plan and did not transition planning, the three ministries are define what would constitute a successful developing a plan to evaluate the effectiveness transition. Further, the Ministry had not other- of the integrated transition planning protocols. wise established a process to assess whether individuals made a successful transition—for example, through satisfaction surveys. As mentioned previously, the Legislature AUTISM FUNDING recently created a select committee to work on a Between 2003/04 and 2012/13, the Ministry comprehensive developmental services strategy quadrupled annual autism funding from $43 mil- for Ontario that will help co-ordinate the delivery lion to $182 million, primarily through an almost of developmental programs and services across three-fold increase in funding for IBI services and ministries, with a particular focus on needs related through the introduction of measures such as the to education, work and housing. School Support Program and applied behaviour analysis-based services, as illustrated in Figure 7. 76 2013 Annual Report of the Office of the Auditor General of Ontario

Figure 7: Autism Services and Supports Expenditures, 2003/04–2012/13 ($ million) Source of data: Ministry of Children and Youth Services 200 Other autism services and supports Intervention services covered outside the regular program 180 Applied behaviour analysis-based services School Support Program Autism Intervention Program 160

140

120

100

0 2003/04 2004/05 2005/06 2006/07 2007/082008/09 2009/10 2010/112011/122012/13

Chapter 3 • VFM Section 3.01 As previously mentioned, there has been a wait operational efficiency at agencies, fewer hours of list for autism therapies since program inception. therapy being offered to children, and/or children Based on the prevalence rates of autism in South- being discharged from therapy sooner. eastern Ontario from 2003 to 2010 (the only prevalence rates available in the province), the change in Reasonableness of Funding Allocation total program funding surpassed the change in the prevalence rates for each year from 2004 to 2007, In order to assess whether resources were being dis- yet this still did not significantly reduce the wait list tributed equitably across the province, the Ministry for IBI therapy. From 2008 to 2010, the change in would need to compare funding distribution to the prevalence rates surpassed the change in total fund- demand for services across the regions. However, it ing by an average of 8% a year. had not done so by the time of our audit. Demand Over the five-year period ending in 2012/13, for services is represented by children being served transfer payments to service providers for IBI and those waiting for service. We compared eight services increased by 20%, while the number of IBI of the regions on this basis (we omitted one region spots remained virtually unchanged at 1,400. Dur- because it places children on the wait list before ing this time, the number of children who received diagnosis, contrary to policy) and noted that in two IBI services increased by 14%, which could have regions, their share of total funding was not in pro- been due to a number of factors, such as improved portion to the demand for services in those regions. Autism Services and Supports for Children 77

We analyzed unit costs for two of the most are likely to receive more clinical supervision, significant autism services and supports in the and therapists’ costs are incurred for cancelled 2012/13 fiscal year, and noted a wide variation in appointments; and cost per service provided across regions, as illus- • providers under the direct funding option trated in Figure 8. Significant variances were noted might be charging parents more than in the prior year as well, but had not been followed the capped rate, with parents paying the up by the Ministry. difference. The Ministry informed us that it took no action on the consultant’s findings because it felt there Comparison Between Service Delivery were too many variables across regions and the Options report was not conclusive enough to lead to any In November 2004, the Ministry told the public kind of costing benchmark. However, the Ministry accounts committee that it would examine the did not attempt to do a more meaningful analysis cost variances for IBI services between the direct of the reasons for the cost variances under the two funding model and the direct service model. In service delivery options. In particular, the Ministry 2008, the Ministry hired a consultant to conduct has not required agencies to track and submit a costing analysis of the IBI program. Among the supervision and direct therapy hours for either consultant’s findings was that for the 2005/06 fis- service delivery option. cal year, the average cost per hour under the direct Due to a lack of information on actual IBI therapy service option was $55 and ranged from $47 to hours, we analyzed the average cost per child for one $87 across the nine regions, whereas the average year of therapy under both service delivery options cost per hour under the direct funding option was in the three regions visited and noted that it costs $37 and ranged from $27 to $44 across the nine 66% more for the government to deliver services regions. According to the consultant’s report, the under the direct services option than it does under lead service agencies’ reasons for higher unit costs the direct funding option, even after we allocated per hour under the direct service option included overhead costs—costs for administration, wait-list

the following: management, and clinical supervision—between the Chapter 3 • VFM Section 3.01 • the direct service option gives clients access two service delivery options. This analysis does not to a wider range of clinical services and include any amounts that parents would pay out of covers all IBI-related costs (such as travel, pocket under the direct funding option. materials, equipment, assessments, progress The direct funding option rate of $39 per hour, reviews, parent meetings), while parents set in 2006/07, is meant to capture all aspects of themselves pay these costs under a direct required services including direct therapy, super- funding option arrangement; vision, travel and materials. The rate has not been • providers under the direct service option have reviewed since. higher staff costs because their therapists are However, rates obtained by families often more likely to be unionized, their therapists depended on their negotiating skills; we noted

Figure 8: Unit Costs of Selected Autism Services, 2012/13 Source of data: Ministry of Children and Youth Services

Provincial Regional Autism Service/Support Basis of Comparison Average ($) Variation ($) IBI Cost per child per year of service 56,000 50,800–67,000 ABA Cost per hour of service to children, youth and parents 140 70–340 78 2013 Annual Report of the Office of the Auditor General of Ontario

instances where the same private provider charged OVERSIGHT OF SERVICE PROVIDERS clients different amounts for the same service. For The Ministry collects service-level and financial example, for the same one-month time period, one data from its service providers on a quarterly basis provider charged a client its established fees for for each service and/or support offered. Service all hours of service provided, which exceeded the data tracked includes, among other things, the client’s allowance by $480 for the month; the same number of children receiving IBI services, number service provider charged another client for fewer of children discharged from IBI services, aggregate hours of services than what the client actually number of days that children who started IBI received, just so the client could be fully reimbursed waited for service, number of hours of ABA-based by their direct funding option allowance, resulting services received by children, and number of in the latter client saving $460. consultation hours provided by ASD consultants under the School Support Program. Targets are set RECOMMENDATION 6 for each of these areas. Regional offices follow up To ensure that all regions use autism funding with service providers when actual levels of service cost-effectively to meet local demands, the Min- provided differ from targets by more than 10%. istry of Children and Youth Services should: Several years ago the Ministry also began • ensure that all lead service agencies place collecting monthly data from service providers children on the wait list for IBI services only for IBI services, respite care and the Connections after determining their eligibility, and review initiative. Some of the monthly data requested is whether its funding allocation is aligned similar to that submitted quarterly (for example, with service demand; the number of children who ended IBI services in • periodically compare and analyze agency the period) and some is different (the number of costs for similar programs across the province, children waiting for IBI services, and a breakdown and investigate significant variances; and of children in IBI by service delivery option). • review the reasonableness of the hourly rate Unlike the quarterly information, no targets are set under the direct funding option, which was for these data elements. Chapter 3 • VFM Section 3.01 set in 2006. We had the following concerns with the data collection and analysis: MINISTRY RESPONSE • Some regional offices we visited did not verify data that is submitted by the service The Ministry will direct all AIP agencies to providers. As a result, some data forwarded to review their practices for placing children on the Ministry was not accurate. For example, wait lists and ensure that their practices are con- in one region, the lead IBI service agency sistent with the AIP guidelines. The Ministry will reached its target for number of children direct agencies whose practices are not in line served in IBI by, contrary to policy, including with the guidelines to bring their practices into children still waiting for direct services but compliance by December 2013. The Ministry will whose families were receiving consultation also consider reviewing how funding allocations services from a senior therapist. In another are aligned with service demand for the AIP. example, an ABA partner agency submitted The Ministry will consider comparing and the wrong quarter’s data on the number of analyzing agency costs for similar programs children waiting for service and the number across the province. of days they waited for services. The data was The Ministry will review the hourly rate for understated by 330 children and 36,000 days the direct funding option in the AIP. in total. Autism Services and Supports for Children 79

Some information reported to the Ministry • ability, transparency and relevance of the data. was non-verifiable; for instance, for the As a result of this review, the Ministry adjusted School Support Program, service providers the amount and type of data being collected. we visited tracked participation at training For example, the Ministry is now collecting and sessions via a head count rather than a sign-in analyzing information relative to the number list. In addition, because participants were not of service hours that children and youth receive individually identified, the Ministry could not and the length of time they wait to receive ser- determine how many unique individuals were vice. The Ministry is also tracking the number of served and how many hours of training or children and youth who receive their eligibility consultation services each received. assessment for the AIP within the four to six We also noted that the Ministry did not collect weeks prescribed by the AIP guidelines. The information that would help it monitor compliance new data collected should help the Ministry’s with program guidelines and evaluate program efforts toward continuous quality improvement. effectiveness. For example, the following informa- The Ministry will consider collecting the tion would be useful to monitor and evaluate the additional information suggested by the Aud- IBI program: itor General. • wait time for each child on the wait list to determine the individual’s length of wait for services; • percentage of families on the wait list that EFFECTIVENESS OF AUTISM SERVICES received support services; AND SUPPORTS number of IBI service hours actually delivered • Because the prevalence of autism is increasing and to each child per week to determine whether government’s financial resources are limited, it is the agency actually provides between 20 and imperative that the Ministry evaluate the effective- 40 hours of service each week; ness of its autism services and supports periodically lost hours of service by cause; and • to ensure that children with autism receive the

change in a child’s functionality from the time Chapter 3 • VFM Section 3.01 • most appropriate and effective services that meet he or she starts intervention until the time of their needs. discharge.

RECOMMENDATION 7 Performance Indicators

To better monitor whether service agencies are Similar to other provinces we researched, the meeting key program guidelines and providing Ministry does not publish any outcome measures quality services, the Ministry of Children and to assess its autism services and supports. The Youth Services should review the type of data Ministry has only one performance measure—the that agencies are required to submit, and ensure number of children receiving IBI at year-end. How- key information is received and analyzed, and ever, this is not useful in assessing the effectiveness periodically verified through site visits. of the Autism Intervention Program (AIP). In the 1999 Cabinet submission for the AIP, the MINISTRY RESPONSE Ministry proposed a number of relevant long-term performance measures that would help track the Every year, the Ministry reviews its data require- success of the program. These included: identify- ments to improve data collection. In 2013, the ing children with autism by age 3; significantly Ministry focused its review on the quality, reli- 80 2013 Annual Report of the Office of the Auditor General of Ontario

improving functioning for two-thirds of children In 2006, the Ministry commissioned an external receiving three years of intensive therapy, and consultant to evaluate the outcomes of children who successfully integrating half of these children into received Ministry-funded IBI services. Specifically, regular classrooms; avoiding future health, social the goal of the study was to determine whether chil- service and education costs; and ensuring that 80% dren showed significant improvement and to iden- of parents are satisfied with services. However, we tify factors that predict greater improvement. The found that the Ministry has not collected informa- consultant reviewed the case files of over 300 chil- tion to measure the achievement of any of these dren who received IBI services at any time between objectives. Furthermore, in November 2004, during 2000 and 2006, and among other things compared a public accounts committee hearing following their assessments at time of entry and exit from our 2004 special report on IBI services under the the program. The study found that 75% of children AIP, the Ministry stated that it would develop more showed measurable progress or improvement, and a outcome-based performance measures but it has subset (11%) of them achieved average functioning. yet to do so. Improvements were seen in the severity of their With regard to ABA-based services, the Ministry autism, their cognitive level, and their adaptive expects all service providers to collect information behaviour (that is, communication, daily living, pertaining to child outcomes, parent/caregiver socialization, and motor skills). Improvements were outcomes, parent/caregiver satisfaction with ser- noted with all groups of children regardless of their vice delivery, and system outcomes. Similarly, with initial level of functionality, but those who were regard to the School Support Program, the Ministry initially higher functioning had the best outcomes expects service providers to provide annual reports or made the most progress. Children who started outlining achievement of key outcomes. However, IBI before age 4 did better than those who started in both cases the Ministry does not specify any after age 4. Children who received two years or performance measures. For the ABA program, we more of IBI did better than those who received a noted the only outcome data that the Ministry has shorter duration of IBI. The consultant concluded asked service providers to submit was on the num- that the initial level of a child’s functionality was a

Chapter 3 • VFM Section 3.01 ber of children who met their goals upon comple- better predictor of improvement, although it didn’t tion of ABA-based services, which in 2012/13 was account for all the variability, followed by the child’s 88%. Although this would be a good performance age at the start of therapy and then the duration indicator, no target was set and no other objective of therapy. While the study had its limitations, the performance outcome data was collected, such as experts we consulted said these findings were valid that which could be obtained from parent satisfac- and consistent with other research. Despite the tion surveys, for instance. In the case of the School results of this evaluation, no modifications were Support Program, no service quality or outcome- made to the program, such as letting children with based information was collected. milder forms of autism access IBI. Although the IBI program has been implemented in Ontario since the year 2000, no study has fol- Program Evaluations lowed the cohort of children who received or were In 1999, the Ministry indicated that it would evalu- denied IBI services in that time to help assess the ate the program to demonstrate that it is making a program’s long-term impact. In addition, no study difference to families of children with autism. The has been done to determine whether children’s Ministry further noted that it would modify the pro- outcomes differ by service delivery option. Without gram based on evaluation data in order to increase such studies, the Ministry has not been able to the likelihood of meeting its long-term objectives. assess whether the program is effective as designed. Autism Services and Supports for Children 81

The lack of a long-term effectiveness study (that is, a MINISTRY RESPONSE study looking at the long-term outcomes of children with autism who acquired intervention services at a The Ministry agrees that it is important to assess younger age) is not unique to Ontario. Having said the effectiveness of its services and supports for that, we noted that a national study, funded by the children and youth with autism and to adjust the Canadian Institutes of Health Research and others, programs if necessary so that they are as effect- is following groups of children with autism from ive, cost-efficient and accessible as possible. diagnosis until age 11. The study was announced The Ministry strategically reviews the aut- in 2004 and will continue until 2014, and includes ism data that is collected to ensure it addresses children from one part of Ontario. The initial five areas—effectiveness, efficiency, access- findings of this study speak to the importance of ibility, equity, and client profile. The Ministry developing ASD intervention services that are deliv- will continue to evaluate its data with a view to ered as early as possible and are diverse, flexible, developing a broader autism services evalua- and sensitive enough to meet the needs of children tion plan, including performance indicators with ASD who have very different clinical profiles and targets. and follow different developmental pathways. The Ministry has developed an ABA-based services evaluation plan with key performance RECOMMENDATION 8 indicators to assess program effectiveness, efficiency and accessibility, and families’ experien- To help ensure that services and supports for ces with ABA-based services and supports. This children with autism are meeting their needs, the plan also includes the use of evidence-based Ministry of Children and Youth Services should: tools to assess client outcomes and client satis- develop performance measures and targets • faction with ABA-based services. The evaluation for each of its autism services and supports began in fall 2013. Based on this work, the Min- to assess their effectiveness in improving istry will consider options for measuring family children’s outcomes; experience in the AIP. conduct periodic program evaluations,

• The Ministry will continue to closely monitor Chapter 3 • VFM Section 3.01 including parent satisfaction surveys, and external research on the effectiveness of its consider conducting a long-term effective- programs, including research being conducted ness study of children who received IBI by the Canadian Institutes of Health Research services and children who were denied IBI on the effectiveness of the AIP. services; and • modify services and supports as required. Chapter 3 Ministry of Health and Long-Term Care Section 3.02 Health Human Resources

care and education systems, and better manage the Background supply of health human resources. Total expenditures for the strategy grew from $448 million in the 2006/07 fiscal year Health human resources—physicians, nurses to $738.5 million in the 2012/13 fiscal year, an and other health-care providers—are crucial to increase of about 65%. These amounts included the delivery of health services. They represent $431 million for physician and nursing initiatives in the single greatest asset, as well as cost, to the 2006/07 and $728 million for them in 2012/13, as health-care system. Acting to address concerns over well as ministry operating expenses of $17 million provincial physician and nursing shortages, long in 2006/07 and $10.5 million in 2012/13, as shown wait times and an increasing number of patients in Figure 1. without family doctors, the Ministry of Health and

Chapter 3 • VFM Section 3.02 Long-Term Care and the Ministry of Training, Col- leges and Universities jointly developed a strategy called HealthForceOntario in the 2005/06 fiscal Audit Objective and Scope year. As part of the strategy, the Ministry of Health and Long-Term Care established the HealthForce The objective of our audit was to assess whether the Ontario Marketing and Recruitment Agency Ministry of Health and Long-Term Care, in conjunc- (Agency) in 2007. The Agency’s activities focus on tion with the Agency, had adequate systems and recruitment and retention of health professionals. procedures in place to: The strategy’s goal is to ensure that Ontarians identify and assess the appropriateness of have access to the right number, mix and distribu- • the mix, supply and distribution of qualified tion of qualified health-care providers, now and in health-care professionals to help meet the cur- the future. Responsibility for its implementation rent and future needs of Ontarians across the lies with the Health Human Resources Strategy province; Division of the Ministry of Health and Long-Term ensure that strategy initiatives were delivered Care (Ministry), but its Assistant Deputy Minister • in accordance with established regulatory reports to the Deputy Ministers at both ministries. requirements, applicable directives and poli- This is meant to establish a link between the health cies, and agreements; and

82 Health Human Resources 83

to representatives of other jurisdictions—Manitoba, Figure 1: Health Human Resources Strategy Division Alberta and British Columbia—to gain an under- Expenditures, 2006/07–2012/13 ($ million) standing of how health human resource planning is Source of data: Ministry of Health and Long-Term Care done in those provinces. 800 Operating expenses 700 Physician and nursing initiatives 600 Summary 500

400 Over the last six years, the Ministry of Health and 300 Long-Term Care (Ministry) has spent $3.5 billion 200 through its HealthForceOntario strategy to address

100 the shortages of physicians, nurses and other health professionals across Ontario. In 2012/13 the Min- 0 7 8 9 0 1 2 3 istry directed $738.5 million toward this strategy: /0 /1 /1 07 10 011 12/1 2006/0 20 2008/0 2009/1 20 2 20 $505 million for physician initiatives, $151 million for nursing initiatives, $72 million for other health • measure and report regularly on the progress human resource initiatives and the remaining of the strategy’s objectives. $10.5 million for operating expenses. The Ministry and Agency senior management Overall, Ontario has seen an 18% increase in reviewed and agreed to our audit objective and physicians from 2005 to 2012 and a 10% increase criteria. in nurses from 2006 to 2012. While the initiatives Our audit focused on physician and nurse human increased enrolment, created more postgraduate resources. In conducting our audit, we reviewed training positions and attracted more doctors and relevant legislation, administrative policies and pro- nurses from other jurisdictions, Ontario has not met cedures, and interviewed staff at the Ministries of its goal of having the right number, mix and distri-

Health and Long-Term Care and Training, Colleges bution of physicians in place across the province to Chapter 3 • VFM Section 3.02 and Universities. We visited three Local Health Inte- meet the population’s future health-care needs. gration Network (LHIN) offices and three hospitals Specifically, we noted the following: in the North West and South West regions, and we • The province spends an average of about contacted two hospitals in the Greater Toronto Area $780,000 (including $375,000 for resident region to interview staff and obtain relevant docu- salaries and benefits) to educate one special- ments. We also obtained information related to vari- ist who completes a four-year undergraduate ous nursing initiatives from Greater Toronto Area degree and up to five years of postgraduate hospitals. To gain an overall understanding and residency training. For a specialist who perspective of the health human resources area, we enters Ontario at the postgraduate level from spoke with a number of external stakeholders such outside the province, this cost is $225,000. as the College of Family Physicians, the Ontario However, many specialists trained in Ontario Hospital Association, the Registered Nurses Associa- do not stay and practise here. Retention tion of Ontario, the Registered Practical Nurses statistics show that, on average, 33% of Association of Ontario, the Professional Association Ontario-funded surgical specialist graduates of Residents of Ontario (formerly the Professional left the province each year between 2005 Association of Internes and Residents of Ontario), and 2011. The lack of full-time employment and the Ontario Medical Association. We also spoke opportunities for graduating residents of 84 2013 Annual Report of the Office of the Auditor General of Ontario

certain surgical specialties may lead to more • Although the physician forecasting model physicians deciding to leave the province, built in partnership with the Ontario Medical despite long wait times for these services. For Association was a positive step in determining example, wait-time data for the three-month physician workforce requirements, it is ham- period from June to August 2013 showed pered by the limited reliability and availability waits of 326 days for forefoot surgery and of data. These limitations make planning the 263 days for cervical disc surgery. optimal number, mix and distribution of phys- • The Agency provides temporary physician icians with appropriate funding, training and or “locum” coverage in eligible communities deployment difficult. As well, a simulation across the province to support access to care. model being developed by the Ministry to help However, vacancy-based locum programs plan for future nursing education positions meant as short-term measures continued to and to help formulate nursing policies aimed be used for long periods of time. At the time at recruitment and retention determines only of our audit there were about 200 specialist what the supply of nurses will be without vacancies in Northern Ontario, and of those considering how many nurses will be needed hospitals using locum services, one-third that to meet the population’s needs. had been using the Emergency Department Coverage Demonstration Project before Janu- OVERALL MINISTRY RESPONSE ary 2008 had been continuously using its The Ministry of Health and Long-Term Care locum services from as early as 2007, and one (Ministry) and the HealthForceOntario Market- hospital had been using them since 2006. ing and Recruitment Agency acknowledge and Over the four fiscal years from 2008/09 to • thank the Auditor General for the timely audit 2011/12, $309 million was dedicated to hir- and the recommendations in this report. ing 9,000 new nurses. Our review showed In a Canadian first, the province launched that while the system was unable to hire that the HealthForceOntario strategy in May 2006. many nurses in the four years, it had increased This was an innovative response to existing

Chapter 3 • VFM Section 3.02 the number of nurses by more than 7,300 and critical shortages in health human resources, the Ministry was on track to achieve its goal and it aimed to ensure that existing gaps would within five years. not worsen. At the end of 2011, 66.7% of nurses were • The strategy has led to a significant improve- working full-time in Ontario, which was just ment in the health human resource capacity of slightly under the Ministry’s goal of 70% of Ontario. Shortages of health providers, includ- nurses working on a full-time basis. However, ing physicians and nurses, are no longer the the Ministry needed to improve its oversight primary barrier to access or cause of wait times. and assessment of the effectiveness of its nurs- The strategy has mitigated the shortages and ing programs and initiatives. For example, improved the province’s ability to plan, train funding for the Nursing Graduate Guarantee and support its health workforce, with some key Program is provided for up to six months with results since May 2006 including: the expectation that organizations will offer more than 35,000 new regulated providers, permanent full-time employment for partici- • including an 18% increase in physician sup- pating new graduate nurses. However, only ply and a 10% increase in nurse supply; about one-quarter of program participants in expanded first-year undergraduate enrol- 2010/11 and one-third in 2011/12 actually • ment in medical schools (up by 22%) and obtained permanent full-time positions. first-year postgraduate trainees (up by 60%); Health Human Resources 85

professionals across Ontario. While the province 15,644 more nurses working full-time, a • was able to increase the number of physicians, some 23% improvement; Ontario communities face shortages of health-care 25 nurse practitioner-led clinics providing • providers, especially physicians. Primary-care care to over 36,000 patients; physicians, also known as family physicians, are more than 15,100 employment opportun- • not always available in small, rural or remote com- ities for new Ontario nursing graduates; munities. In Northern Ontario, general specialists new health-care provider roles including • (for example, in the areas of general surgery, inter- physician assistants, clinical specialist radia- nal medicine and psychiatry) also remain in high tion therapists and five new nursing roles; demand despite a significantly improved provincial creation of evidence capacity to inform • physician supply. planning; Although the significant amount of funds that legislative and regulatory changes increas- • the Ministry has expended over the last six years ing the quality and safety of patient care, has increased the supply of physicians in the prov- expanding scopes of practice and regulating ince, shortages remain in certain specialties and new health professions; and geographical areas even as physicians in those spe- establishment of the HealthForceOntario • cialties are unable to obtain full-time employment. Marketing and Recruitment Agency. Ontario is now able to focus health human resource activities on health-system transforma- Increased Supply of Physicians in Ontario tion rather than responding to critical shortages Medical education is funded jointly by the Min- of providers. The Ministry’s work continues istry of Training, Colleges and Universities and to evolve to address today’s challenges. The the Ministry of Health and Long-Term Care. The Ministry is renewing the HealthForceOntario Ministry of Training, Colleges and Universities strategy so that it: funds universities for undergraduate positions, builds on the successes of previous • while the Ministry of Health and Long-Term Care accomplishments;

funds most aspects of postgraduate training. The Chapter 3 • VFM Section 3.02 aligns with the goals of Ontario’s Action Plan • majority of the $485 million the Ministry of Health for Health Care; and and Long-Term Care spent on physician initiatives advances evidence-informed planning and • in the 2011/12 fiscal year was in two areas: total decision-making. payments of $315 million to medical schools and The recommendations in this audit will hospitals for the salaries and benefits of residents inform the strategy renewal. who provide clinical services across Ontario; and $107 million paid to medical schools to support academic activities such as teaching, educational infrastructure and related administrative costs for Detailed Audit Observations clinical education of medical learners. From 2005 to 2012, the Ministry of Health and PHYSICIANS Long-Term Care worked with the Ministry of Train- ing, Colleges and Universities to increase enrol- Over the last six years, the Ministry of Health and ment in physician training programs. First-year Long-Term Care (Ministry) has spent $3.5 billion undergraduate enrolment in medical schools went through its HealthForceOntario strategy to address up by 22% and first-year postgraduate trainees by the shortages of physicians, nurses and other health 60%. In family medicine, the number of first-year 86 2013 Annual Report of the Office of the Auditor General of Ontario

postgraduate trainees went up by 67% and spe- north was twice as high. Access has been a long- cialists by 56%. In addition, the number of inter- standing issue in many rural, remote and northern national medical graduates who entered residency communities in Ontario with chronic physician training went up by 48%. As seen in Figure 2, shortages. Geographic isolation, long travel dis- between 2005 and 2012, the number of physicians tances, low population densities and inclement increased by 18%, or about 4,100. At the same weather conditions are just some of the challenges time, the number of family doctors per 100,000 to providing health care in these areas. people went from 84.9 to 91, and specialists from A 2011 Canadian Institute for Health Informa- 92.9 to 104.3. The total number of doctors per tion report showed that 95% of physicians in 100,000 people went from 177.8 to 195.3. Accord- Ontario practised in urban areas while the remaining to the Canadian Institute for Health Informa- ing 5% practised in rural areas. This number falls tion, the number of specialists per 100,000 people short of urban-rural population distribution in in Ontario in 2011 was in line with the Canadian Ontario: according to Statistics Canada’s 2011 average, while the number of family physicians per census, 86% of the population lived in urban areas 100,000 people in Ontario was about 10% below while 14% lived in rural areas. To help assess the average. the accessibility of health care in rural areas, the Ministry uses the Rurality Index of Ontario (RIO), developed by the Ontario Medical Association. The Sub-optimal Distribution of Physicians in RIO incorporates data on population and physicians Ontario practising in rural and northern areas, including Despite the overall increase in primary health-care large urban centres in the north. The RIO indicates providers and specialists, access to health care is that in 2011, 8.1% of physicians in Ontario prac- still a problem for some Ontarians. According to the tised in these areas, which contained 11.6% of the Ministry, based on data collected between October province’s population. 2012 and March 2013, 6% of Ontarians lacked a Although the Ministry acknowledged that family physician. Although more recent regional physician distribution across Ontario was still not

Chapter 3 • VFM Section 3.02 data is not available, in 2010 the percentage in the optimal, it cited factors that could account for the

Figure 2: Increase in the Number of Physicians and Physician Trainees in Ontario, 2005–2012 Source of data: Ministry of Health and Long-Term Care

2005 2012 Increase (%) First-year undergraduate enrolment 797 972* 22 Medical school graduates 663 875 32 First-year postgraduate trainees 757 1,213 60 Family medicine — first-year postgraduate trainees 305 508 67 Specialty — first-year postgraduate trainees 452 705 56 International medical graduates 171 253 48 Family physicians 10,641 12,296 16 Specialists 11,636 14,086 21 Total physicians 22,277 26,382 18 Family medicine physicians per 100,000 population 84.9 91.0 7 Specialists per 100,000 population 92.9 104.3 12 Total physicians per 100,000 population 177.8 195.3 10

* Latest data available for undergraduate enrolment is from 2011. Health Human Resources 87 way physicians are distributed. For example, some training programs may attract trainees from juris- highly specialized health-care services are delivered dictions where these programs are not offered, and in tertiary care units, which means patients in some some may complete residency training in Ontario communities must travel to large urban centres to and return to their home province afterward. receive specialized care. However, others may leave Ontario because they have difficulty finding stable employment after graduation. Medical Specialties Facing Employment The Royal College of Physicians and Surgeons of Problems Canada (College) and the National Specialties Soci- On average, the province invests about $780,000 eties conducted a mini-study from July to Novem- (including $375,000 for resident salaries and ber 2010 that found physician unemployment and benefits) to educate one specialist for a four-year underemployment were common in the following undergraduate degree and up to five years of post- areas: cardiac surgery, nephrology, neurosurgery, graduate residency training. For a specialist who plastic surgery, public health and preventative enters Ontario from outside the province at the medicine, otolaryngology (ear, nose and throat postgraduate training level, this cost is $225,000. specialists) and radiation oncology. In light of these In 2011, the province spent a total of $438 million results, the College expanded its research in April on specialist education—a 63% increase since 2011 to conduct a multi-phase national study of 2005, when the amount spent was $269 million. medical specialist employment in Canada. The final However, many specialists trained in Ontario report was released in October 2013. The report do not stay and practise here. Figure 3 shows indicated that the specialties affected included that, on average, about 33% of surgical specialist orthopaedic surgery, urology, gastroenterology, graduates (including neurosurgeons and cardiac, hematology, critical care, general surgery, ophthal- orthopaedic, paediatric and general surgeons) who mology, neurosurgery, nuclear medicine, otolaryn- were funded by the Ministry left Ontario each year gology and radiation oncology. In our discussions between 2005 and 2011. with medical associations, we repeatedly heard that

Not every graduating specialist who leaves graduating specialists face employment difficulties Chapter 3 • VFM Section 3.02 Ontario does so because of employment difficul- in various surgical specialties, including many of ties. The size and breadth of Ontario postgraduate the above.

Figure 3: Surgical Specialists Leaving Ontario, 2005–2011 Source of data: Ministry of Health and Long-Term Care retention data extracted from Canadian Post–M.D. Education Registry (CAPER) Annual Census of Post-M.D. Trainees

Average for 2003 2004 2005 2006 2007 2008 2009 2003–09 Surgical specialists graduating in Ontario 120 122 120 125 111 114 147 123 Average for 2005 2006 2007 2008 2009 2010 2011 2005–11 Surgical specialists practising in Ontario two years 79 79 79 87 73 70 108 82 after graduation Surgical specialist graduates leaving Ontario (41) (43) (41) (38) (38) (44) (39) (41) % of surgical specialist graduates leaving Ontario 34 35 34 30 34 39 27 33 Surgical specialists coming into Ontario to practise 22 8 12 19 18 14 9 15 Net number of specialists leaving Ontario 19 35 29 19 20 30 30 26 88 2013 Annual Report of the Office of the Auditor General of Ontario

The College’s 2011–12 study found that about choosing specialties over general practice because 20% of new specialists and subspecialists in Ontario of the perception that specialists have more pres- (compared to 16% in Canada) could not find a job tige and higher earning potential. According to the after completing their residency training periods of Future of Medical Education in Canada Postgrad- two to five years following medical school. It also uate Project funded by Health Canada, a 50/50 noted that employability was impacted by personal balance of generalists and specialists is needed to factors and preferences such as jobs not being based provide optimal care to patients. In 2011, Ontario in new graduates’ preferred locations; hospital had about 1,700 more specialists than general- budgetary restrictions; and delayed retirements. ists. The Ministry has worked with the faculties of Such factors could result in physicians choosing to medicine to increase the number of family medi- prolong their studies or make do with contract and cine residency positions by 119% from 2003/04 temporary work, losing skills, leaving Ontario or to 2011/12. Since 2012, the Ministry and medical having to work in non-surgical practice. At the same schools moved to implement a more structured, time, there are specialist vacancies in some areas of annual planning cycle to better support decision- the province. For example, in the north, almost all making and fine-tuning of the size and composition (99%) of the $13 million spent on temporary special- of the postgraduate training system. ist coverage in 2011/12 was for covering specialist The Ministry acknowledged that graduating vacancies while recruitment was being pursued. residents faced a number of employment concerns The Ontario Medical Association also collected and that unemployment and underemployment employment data in a 2011 survey. It described were concentrated in specific specialties, particu- some of the barriers new graduates face in finding larly those requiring hospital resources. However, positions in anaesthesiology, cardiac and thoracic we found that it had not collected data from surgery, general surgery, neurosurgery, orthopaedic hospitals nor analyzed existing data to identify the surgery and vascular surgery: causes or to develop solutions. For example, the • Many cardiac surgeons were working as Ministry had not examined how hospital funding surgical assistants because they could not find might affect areas such as operating-room capacity,

Chapter 3 • VFM Section 3.02 jobs in their specialty; 34% of those graduates and how this in turn might impact employment in who were working as cardiac surgeons still some surgical specialties. It also had not collected considered themselves underemployed. data on factors such as the volume of individual • More general surgeons were choosing to do physicians’ surgical bookings, the allocation of fellowships in surgical subspecialties, which surgical bookings among physicians by level of decreased the number of general surgeons in experience, or the available operating room cap- the health-care system. acity across the system. The Ministry indicated that • There was competition among orthopaedic it continues to work with stakeholder partners such surgeons for operating room time; older as the Ontario Medical Association and the Ontario surgeons were reluctant to relinquish operat- Hospital Association to better understand how ing room time to enable new physicians to profession-specific challenges, including hospital practise. operating practices, affect physician employment • Senior vascular surgeons were working past and underemployment. the normal retirement age, which meant they The Ministry told us that once the final results of were holding on to operating room time and the College’s national study are available with juris- hospital clinic resources. dictional results, it will help inform it of the current Naturally, individual job preferences also affect status of and the multiple factors that contribute to employment patterns. More students may be Health Human Resources 89 physician unemployment and underemployment as neurosurgery, orthopaedic surgery, psychiatry, across Canada. paediatrics, obstetrics/gynaecology, geriatrics and emergency medicine, but had difficulty recruiting physicians to meet their needs. They indicated Lengthy Wait Times for Specialist Services that graduating physicians often prefer to work Ministry data that we examined for the three- in large urban centres rather than rural, remote month period from June to August 2013 showed and northern areas. Practising in non-urban areas long waits for certain surgical services, as shown presents challenges that may be quite different in Figure 4. We found that some of the procedures from those encountered during physicians’ medical with long wait times were in the same surgical training or posed by practising in an urban centre. specialties in which graduating residents faced There are differences in the level of back-up, the unemployment and underemployment. For extent of on-call work and the types of illnesses that example, for some orthopaedic surgeries, waits need treating. There may also be fewer social and following a specialist’s assessment were as long as cultural activities available and limited employment 326 days (forefoot) and 263 days (cervical disc). opportunities for physicians’ partners. Patients often wait months just to see a specialist after the family physician’s referral. Physician Initiatives Our discussions with hospitals we visited and the Ontario Medical Association suggested that A number of studies have suggested that one factor long wait times could be related to factors such affecting a physician’s practice location decision as hospital funding. Reduced capacity caused by is where he or she receives a significant portion budget constraints could mean long wait times of postgraduate medical training. For this reason, for some services; if funding is constrained then rural, remote and northern communities may have operating room hours and/or the resources who more trouble attracting physicians than urban cen- staff them could be cut, resulting in unemployment tres that are close to medical faculties and teaching and underemployment among the specialists who hospitals. Figure 5 shows a number of programs

provide these services. and initiatives the Ministry has funded to help Chapter 3 • VFM Section 3.02 The hospitals we spoke to in rural areas said those communities recruit and retain physicians. they needed resources in various specialties, such The Ministry also manages a Return of Service Program that requires international medical gradu- Figure 4: Provincial Wait Times* in Surgical ates and participants in certain other physician Specialties with High Unemployment/ postgraduate training programs to practise in Underemployment, June–August 2013 eligible communities in Ontario, generally for a Source of data: Ministry of Health and Long-Term Care period of five years. (The program is covered in a subsequent section of this report.) The Ministry, Type of Service/Procedure Wait Time (days) through separate divisions, also funds a number of Neurosurgery (overall) 134 related initiatives, such as the Northern and Rural Orthopaedic surgery (overall) 192 Recruitment and Retention Initiative, which offers Cervical disc 263 financial incentives to physicians who establish a Forefoot 326 Hip replacement 186 full-time practice in an eligible northern, rural or Knee replacement 220 remote community. Lumbar disc 251 At the time of our audit, the effectiveness of these initiatives had yet to be evaluated. Some of * Wait time is calculated as the number of days from when 9 out of 10 patients see their specialist to when they undergo surgery. the initiatives had only recently been implemented. 90 2013 Annual Report of the Office of the Auditor General of Ontario

Figure 5: Selected Ministry Initiatives for Physician Recruitment and Retention in Rural and Remote Communities Source of data: Ministry of Health and Long-Term Care

Funding Received in 2011/12 Initiative Description ($ million) Northern Ontario School of Medicine Rural-distributed, community-based medical school that seeks to recruit students coming from Northern Ontario or rural, remote, aboriginal or 12.7 francophone backgrounds (started 2005) Distributed Medical Education Organizations co-ordinate clinical teaching placements in small urban and rural communities for undergraduate and postgraduate learners 11.7 (started 1995) Northern and Rural Recruitment and Financial incentives to physicians who establish a full-time practice in an 3.4 Retention Initiative eligible community (started 2010) Hospital Academic and Operating Funding to cover academic and operating costs for hospitals affiliated Costs for hospitals affiliated with the with the Medical Education Campuses (started 2008) 3.5 medical education campuses

The 2010 evaluation of the Return of Service Physician Locum Programs Program found that it was not meeting the needs The Agency provides support for temporary phys- of most northern and remote communities. In ician or “locum” coverage in eligible communities response, the Ministry implemented changes to across the province. Locum support targets two that program, including expanding the eligible geo- specific types of need: graphic boundaries and providing other targeted • Respite coverage is an ongoing retention sup- funding for certain geographic areas that were port to physicians who work in northern and underserviced. rural communities. Because there are fewer The Northern Ontario School of Medicine, physicians in these communities, there are

Chapter 3 • VFM Section 3.02 opened in September 2005, has not been operating limited options for local replacements. Respite long enough for a meaningful evaluation, but a locum coverage provides these rural and five-year tracking study from 2010/11 to 2014/15 northern physicians with back-up when they is under way to determine the extent to which the are temporarily away on leave, continuing school’s undergraduate and postgraduate programs medical education or vacation. improved the supply and distribution of physicians • Vacancy-based coverage is intended as a in northern and rural communities. short-term solution to provide access to care In addition, as noted in Figure 5, the Ministry in areas where there are physician vacancies funds the Distributed Medical Education Program, while long-term recruitment is pursued. in which organizations co-ordinate clinical teaching As well as providing access to physician care in placements in small urban and rural communities communities with temporary physician absences or for undergraduates and postgraduates. The Min- vacancies, these programs are also meant to sup- istry informed us that it has been working with port the retention of rural and northern physicians. medical schools since autumn 2012 on an approach In the 2011/12 fiscal year, the Ministry spent a total to evaluate this initiative. of about $22 million on three physician locum programs administered by the Agency. The programs include the Northern Specialist Locum Programs, the Emergency Department Health Human Resources 91

Coverage Demonstration Project and the Rural arium—about four times the average amount paid Family Medicine Locum Program. for respite coverage. In addition to fees for service, We focused our audit work on the Northern physicians receive eligible travel and accommoda- Specialist Locum Programs and the Emergency tion expense reimbursement for respite coverage, Department Coverage Demonstration Project which amounts to $241 per day on average. In some because these programs specifically targeted phys- Northern Ontario communities, physician short- ician vacancies while permanent recruitment was ages and recruitment challenges might have con- pursued. tributed to the extended use of physician locums to support ongoing access to care for patients. Our Costly Long-term Use of Northern Specialist Locum review showed that using locums has become a Programs service delivery model. Almost all (99%) of the Northern communities can access up to 26 locum $13 million spent on locum coverage in 2011/12 specialty services. The Northern Specialist Locum was for covering specialist vacancies. Programs incurred $13 million in expenditures in We looked at locum programs in other Canadian 2011/12 to provide temporary physician specialty jurisdictions and found that they generally provide coverage through two sub-programs that provide only respite coverage. The Ministry indicated that short-term coverage for specialist physician vacan- Ontario is unique in that it provides large-scale cies and ongoing respite coverage to support hospital-based services to five Northern Urban retention. Referral Centres and has a medical school based According to the Agency, the latest available in the north. The mass of critical services in the data at the time of our audit indicated that about north combined with physician vacancies and 30% of specialist positions in Northern Ontario recruitment challenges in some communities and/ were vacant. This translates to a total of about 200 or specialties creates a need for locum support that specialist vacancies, or 40,000 work days that need may not exist in other jurisdictions. The Agency’s coverage. Data that we examined for the years long-term goal is to transition to a predominantly from 2009 to 2011 showed that the specialties respite program in Ontario and eliminate use of the requiring the greatest number of locum days in

locum program as a service delivery model. The Chapter 3 • VFM Section 3.02 Northern Ontario were internal medicine, diagnos- Agency indicated that it is working to implement tic imaging, general surgery and psychiatry. Over new eligibility criteria, with full implementation by the past five fiscal years, from 2008/09 to 2012/13, 2014/15. four large northern cities—Sault Ste. Marie, We found extensive reliance on locum programs Thunder Bay, Timmins and Sudbury—received to deliver needed health-care services to some more than 80% of specialist locum coverage days. rural, remote and northern communities. For According to the Ministry, the four larger northern example, at a number of the hospitals we visited city hospitals have the highest usage because they we reviewed the on-call locum usage for a single act as critical referral centres to the smaller rural month in 2012 and found that locum coverage was northern communities where low population and as high as 94% for internal medicine at one hospital other factors would not support specialist practice. and 72% for diagnostic imaging at another hospital. Also, they provide teaching and research to the Although hospitals with specialist vacancies Northern Ontario School of Medicine. receiving coverage by the Northern Specialist In addition to fees for services or claims for Locum Programs are required to post the positions work sessionals for daily clinical work, payments to on the Agency’s HealthForceOntario jobs website, physicians for vacancy coverage averaged $1,017 they are not required to report on their progress per day for travel, accommodation and honor- in recruiting for and filling their vacancies. The 92 2013 Annual Report of the Office of the Auditor General of Ontario

Agency informed us that it was in the process of 20 hospitals in meeting the emergency needs of developing new criteria for locum coverage eligibil- communities at any one time, the amount that ity. In a phased approach over the next two fiscal has been spent is not significant overall, averaging years starting in 2013/14, the Agency will require about $200,000 per hospital. hospitals to regularly complete a form, which was being piloted at the time of our audit, to inform it Problems With Retaining Physicians in Northern of recruitment efforts that have been made to fill and Rural Areas vacancies by specialty area. The Return of Service Program is intended to pro- Emergency Department Coverage Demonstration vide greater access to physicians in smaller urban Project communities and underserviced northern and rural The Ministry and the Ontario Medical Association communities by allowing them to recruit from a designed the Emergency Department Coverage pool of physicians. The Ministry funds postgraduate Demonstration Project to be a measure of last resort training and assessment for international medical for hospitals to prevent unplanned emergency graduates and other physicians seeking to qualify to department closures due to physician unavailabil- practise in Ontario in return for a commitment from ity that otherwise would result in patients being them to provide services for a period of time (usu- unable to access critical emergency services in their ally five years) in an Ontario community (except own community. the cities of Ottawa and Toronto and adjacent About 50 of 164 emergency departments across municipalities). As of August 2013, the Ministry Ontario have used the project’s locum services since had active return of service agreements with 550 it started in 2006. According to the Agency, about physicians who had completed their training. 20 hospitals use this resource at any one time, on According to the Ministry, before 2010 under the average. The Agency’s total expenditures on the Underserviced Area Program, physicians could only project for the 2011/12 fiscal year were approxi- complete their return of service commitment in a mately $4 million. community designated as underserviced. Origin- We found at the time of our audit that of those ally, the Underserviced Area Program was focused Chapter 3 • VFM Section 3.02 hospitals using the locum services since before on northern and rural communities. However, over January 2008, one-third had been continuously time more communities outside of the north were using the services from as early as 2007. One designated as underserviced to attract return of hospital had been using them from 2006 up to the service physicians, with the result that designated time of our audit. These hospitals received a total of communities close to major population centres in about 9,000 hours of locum coverage in 2011/12. the south outnumbered those in Northern Ontario, We reviewed a sample of the monthly emergency which made it even more difficult for northern department schedules between local and locum and most rural communities to compete for phys- doctors at one of the hospitals we visited and found ician resources. From 2010 onward the Ministry that more than half of the emergency department attempted to improve access to care by expanding shifts had been covered by locum physicians. eligible practice areas to include all but the cities of These findings imply that some hospitals Ottawa and Toronto and adjacent municipalities. facing physician shortages and recruitment chal- These areas were excluded so they would not be lenges need long-term use of the locum support able to out-compete smaller urban, rural, northern to maintain access to services while permanent and remote communities for physician resources. recruitment is pursued. However, given that In our discussions with staff at the rural and approximately $4 million was able to cover about northern hospitals we visited, we heard repeatedly Health Human Resources 93 that they were in need of more physicians. They Some of the unsuccessful candidates may go on told us that it was challenging to keep return of to consider a field of practice outside their original service physicians on after their five-year commit- profession, such as physiotherapy, dietetics and ment was up. The Ministry does not keep track of chiropractics. In the 2012/13 fiscal year, the Agency international medical graduate physicians who stay helped 27 international medical graduates who had on and practise in eligible communities after their been unsuccessful in obtaining residencies to tran- return of service commitments have been met, but sition to other careers in the health sector. information we obtained from one of the hospitals Although its current primary mandate does not we visited showed that only one of three physicians include helping international medical graduates stayed on after completing their service commit- find employment in alternative fields, the Access ments. The other hospitals we visited did not keep Centre informed us that it plans to further develop information on retention. services to assist internationally educated health The Ministry has not evaluated the Return of professionals who are unable to practise in their Service Program to assess its effectiveness since it field to transition to an alternative health career was redesigned in 2010. There are currently no per- consistent with their education and experience. At formance measures or metrics to measure the pro- the time of our audit, the first module of an Alterna- gram’s success. The Ministry has only a quarterly tive Career Toolkit was available on the Access update on the number of international medical Centre’s website. graduate physicians in the program, broken down by LHIN and specialty at a snapshot date. RECOMMENDATION 1

To better meet the health-care needs of Ontar- Alternative Health Careers for International ians, the Ministry of Health and Long-Term Care, Medical Graduates in conjunction with the HealthForceOntario Marketing and Recruitment Agency, should: The Access Centre is a unit within the Agency to compare the existing mix and distribution help physicians and all other regulated health • of physicians across the province to patient

professions who want to practise in Ontario with Chapter 3 • VFM Section 3.02 needs and consider what measures it can the licensing, certification and regulatory process. take to reduce any service gaps; Many international medical graduates use the assess how various factors, including hospi- Access Centre for assistance with navigating the • tal funding and capacity and health-delivery system and competing for Canadian Resident models, affect patients’ access to needed Matching Service (CaRMS) residency positions. services and physician employment, and CaRMS is a national not-for-profit organization develop cost-effective solutions where con- that provides an electronic matching service for cerns are identified; postgraduate medical training in Canada. It enables continue to work with medical schools and applicants to decide where they wish to train in • associations to encourage more medical Canada and enables programs to indicate which students to select fields of study and geo- applicants they wish to enrol in postgraduate med- graphic areas in which to practise that are in ical training. In the 2011/12 fiscal year, approxi- demand; and mately 2,100 international medical graduates were assess the effectiveness of its various phys- registered with the Agency’s Access Centre, but only • ician initiatives in meeting the health-care 173 obtained residencies in Canada; of those, 156 needs of underserved areas. obtained residencies in Ontario. 94 2013 Annual Report of the Office of the Auditor General of Ontario

practitioners (NPs)—in Ontario to improve access MINISTRY RESPONSE to care. RNs usually obtain their education through The Ministry welcomes this recommendation a four-year university degree. RPNs usually obtain as it is consistent with and supports work that it education through a college program. A nurse prac- has undertaken. Working with the Agency and titioner is a registered nurse who has acquired the other key health-system partners, the Ministry’s knowledge base, decision-making skills and clinical HealthForceOntario strategy strives to continu- competencies for a practice that extends beyond ally advance evidence-informed health human that of an RN. resource planning that is responsive to the From 2006 to 2012, the number of nurses health-care needs of the people of Ontario. in Ontario increased by 10%, from 138,583 to The Ministry will continue working with 153,073. The number of nurses employed per partners to better understand these factors 100,000 people in Ontario increased by 3.5%, from and their impact on access to health care and 1,106 in 2006 to 1,145 in 2012. Figure 6 provides a health human resource requirements, including detailed breakdown of the increase. physicians. We found that the Agency played a small role in The Ministry is actively engaged with the nursing sector. Agency regional advisers who medical schools and professional associations operate from various LHINs across Ontario focused to identify and promote a stable supply of phys- mainly on physicians, and very few Agency advisers icians across the continuum of specialties with worked at the Access Centre to counsel internation- a focus on those that are required for harder-to- ally trained nurses. service patient populations or geographic areas. Although the Nursing Strategy has successfully The Ministry is committed to ongoing assess- increased the number of nurses in Ontario, we ment of the effectiveness of initiatives launched found that improvements were generally needed in to address specific needs, and looks forward ministry oversight and assessment of the effective- to the outcome of such evaluations to inform ness of its nursing programs and initiatives. future planning.

Chapter 3 • VFM Section 3.02 Nursing Initiatives NURSES According to data from the College of Nurses of Ontario, 66.7% of members employed in nursing A comprehensive strategy on nursing was an in this province said they worked full-time in 2012. integral part of the HealthForceOntario strategy The rest were categorized as part-time or casual. launched by the Ministry in May 2006. The Nursing Figure 7 breaks down the working status for nurses Strategy is a collection of programs and initiatives in Ontario from 2008 to 2012, and Figure 8 shows intended to achieve the right number and mix of employment by practice sector for full-time, part- nurses in Ontario now and in the future. It addresses time and casual employment, where the nurse does issues such as work environments, full-time employ- not have a set number of hours and is called in to ment, and recruitment and retention of nurses. work as needed. At the time the Nursing Strategy was launched, The Ministry has implemented several nursing the province was experiencing a shortage of nurses, initiatives aimed at stabilizing the workforce and and further shortages were anticipated with increasing full-time opportunities and retention of imminent retirements. The Ministry’s focus was on nurses across Ontario. The Health Human Resour- increasing the number of nurses—registered nurses ces Strategy Division spent $151 million on nurs- (RNs), registered practical nurses (RPNs) and nurse ing initiatives in 2012/13. This amount does not Health Human Resources 95

Figure 6: Number of Nurses in Ontario, 2006–2012* Source of data: College of Nurses of Ontario

Type of Nurse 2006 2012 Increase Increase (%) Registered nurse 108,185 112,194 4,009 4 Registered practical nurse 29,706 38,859 9,153 31 Nurse practitioner 692 2,020 1,328 192 Total 138,583 153,073 14,490 10

* The data provides a “point-in-time” snapshot of the available labour supply of nurses (at the end of the prior year or beginning of the stated year).

Figure 7: Overall Working Status of Nurses in Ontario, 2008–2012* Source of data: College of Nurses of Ontario

2008 2009 2010 2011 2012 Working Status # % # % # % # % # % Full-time 75,649 62.9 78,694 63.9 80,356 63.8 83,972 66.4 85,010 66.7 Part-time 34,820 29.0 34,371 27. 9 34,939 27. 8 32,316 25.6 32,712 25.6 Casual 9,796 8.1 10,026 8.2 10,549 8.4 10,117 8.0 9,889 7. 7 Total 120,265 100.0 123,091 100.0 125,844 100.0 126,405 100.0 127,611 100.0

* The data provides a “point-in-time” snapshot of the working status of nurses (at the end of the prior year or beginning of the stated year).

Figure 8: Employed Nurses1 by Practice Sector, 20102 Source of data: College of Nurses of Ontario

Registered Registered Nurses Practical Nurses Total

Working Status # % # % # % Chapter 3 • VFM Section 3.02 Full-time Hospital 41,030 44 7,544 25 48,574 40 Long-term care 4,851 5 6,690 22 11,541 9 Community 10,947 12 2,636 9 13,583 11 Other & not specified 4,656 5 771 2 5,427 4 Part-time Hospital 15,576 16 4,669 15 20,245 16 Long-term care 2,201 2 3,597 12 5,798 5 Community 4,757 5 1,409 4 6,166 5 Other & not specified 2,052 2 468 1 2,520 2 Casual Hospital 4,352 5 1,168 4 5,520 4 Long-term care 552 1 804 3 1,356 1 Community 2,134 2 506 2 2,640 2 Other & not specified 808 1 180 1 988 1 Total 93,916 100 30,442 100 124,358 100

1. Not including nurse practitioners. 2. Most recent year that data was available. 96 2013 Annual Report of the Office of the Auditor General of Ontario

include funds spent by other program areas within Specific data was not available to determine the the Ministry for nursing initiatives. number of full-time, part-time or casual positions We examine a few of the Ministry’s more signifi- that the funded organizations had created through cant initiatives in the following sections. the 9,000 Nurses Commitment.

Meeting the 9,000 Nurses Commitment Inadequate Assessment of the Nursing Graduate In 2007, the government committed to hiring 9,000 Guarantee Program’s Effectiveness more nurses over a four-year period. It also com- Announced in February 2007, the Nursing Graduate mitted to a goal of 70% of nurses working full-time. Guarantee Program’s objective was to support new However, in fall 2008, the province indicated that it Ontario nursing graduates (both RNs and RPNs) would take longer to achieve the goal of 9,000 new in finding full-time employment immediately upon hires. From 2008/09 to 2011/12, $309 million was graduation. Some of the program’s other objectives dedicated to this initiative. Our review of the initia- include facilitating recruitment in all nursing sec- tive showed that: tors; transforming employer practices to make more • From year-end 2007 (reported as 2008 by the full-time nursing positions available; and increasing College of Nurses of Ontario), the number the total supply of nurses by providing full-time of nurses in the province increased by more employment to nurses who may have otherwise than 7,300 over four years. At the time of our sought work in other jurisdictions or professions. audit, CNO nursing data was not available for The program provides funding for temporary 2013 (which would represent 2012 figures), full-time, above-staffing-complement positions for but it appeared likely that the goal of hiring 26 weeks with the expectation that these bridg- 9,000 new nurses would be achieved by the ing positions will lead to permanent full-time end of 2012. employment. Employers must commit to funding • The province was slightly under its goal of an additional six-week full-time position for the having 70% of nurses working on a full-time new graduate nurse if he or she is not bridged to basis. As of the year ended 2011, reported by permanent full-time employment by the end of Chapter 3 • VFM Section 3.02 CNO as 2012 numbers, 66.7% of nurses were the 26 weeks. The program is open to employers working full-time. From the year ended 2007 in all health sectors (hospitals, long-term-care (reported as 2008) to the year ended 2011 homes and community care organizations); all (reported as 2012), there were almost 9,400 Ontario-educated new graduate nurses are eligible more nurses working full-time, representing a to take part as long as they register on the Agency- 12% increase. administered new graduate guarantee job website We reviewed five Nursing Secretariat programs and accept a job offer within six months of complet- that reported creating 1,316 nursing positions. Two ing their studies. In 2011, there were approximately of the programs we reviewed had been in place 1,200 potential employers. In the 2011/12 fiscal since 2008/09 and the other three had been imple- year, about $66 million of ministry funding was mented in 2011/12. The Ministry indicated that provided to about 210 participating health-care 1,125 of the 1,316 new positions (85.4%) had been organizations (representing an 18% employer par- filled by March 2013. We also noted that the 1,316 ticipation rate), which employed 2,235 new gradu- nursing positions created were not all full-time, but ate nurses under the program. included part-time and casual positions. The Nurs- According to the Ministry’s January 2011 ing Secretariat did not have detailed information Guidelines for Participation in the Nursing Gradu- to determine the type of employment obtained by ate Guarantee for New Graduate Nurses, program some of the positions funded through the Division. funding is provided with the expectation that the Health Human Resources 97 bridging positions offered by organizations will 24% of its graduates had transitioned to full- lead to permanent full-time jobs. In the transfer- time employment and 59% to part-time. payment agreements that they enter into with the • In 2007/08, another organization reported Ministry, participating organizations commit to that 85% of its graduates had transitioned making their “best effort” to place a new graduate to full-time employment and 2% went to in a permanent full-time position after he or she has part-time. By 2011/12, the same organization worked with the employer for at least 12 weeks, but reported that 19% of its graduates had tran- the Ministry does not assess program data against sitioned to full-time employment and 47% to established program targets to determine how well part-time. organizations are doing with bridging participants • In 2011/12, an organization reported that to permanent full-time employment. We conducted 40% of its nursing graduates had voluntarily our own analysis of 2010/11 and 2011/12 program left the program early. Sixty percent of them data and found that only about one-quarter of left because they found employment else- participants had been transitioned to permanent where, while no explanations were given for full-time employment after the six-month period the others who left. in 2010/11 and one-third in 2011/12; about an The Ministry has allocated funds ranging from additional one-third of participants had been tran- about $86 million to almost $100 million per year sitioned to part-time employment in both 2010/11 for the program since it began in 2007/08. We and 2011/12. found that for each of the past five fiscal years We also found that the Ministry had not (from 2007/08 to 2011/12), program expenditures reviewed employment trends at participating were less than the amount of funds allocated. As health-care organizations to determine whether shown in Figure 9, other than for 2009/10, the employers were making their best efforts to transi- amounts of unspent funds varied considerably, tion program participants to full-time permanent ranging from $17.2 million (20%) to $33.6 million employment. We reviewed a sample of health-care (37%) of total funds allocated for the year; they organizations that received funding from the totalled $105.7 million over the five-year period.

program in 2007/08 to 2011/12 to look for employ- Figure 9 also shows how declining program Chapter 3 • VFM Section 3.02 ment patterns and found the following: expenditures are related to a one-third drop in • In 2011/12, one organization reported that program participation by eligible graduate nurses. 15% of its graduates had transitioned to Participation was as high as 62% in 2007/08 but full-time employment while 65% went to part- declined to 35% in 2011/12. When we asked time. In 2007/08, the same employer reported the Ministry about the reasons for this decline,

Figure 9: Nursing Graduate Guarantee Program Funding and Participation Source of data: Ministry of Health and Long-Term Care

Allocated Actual Unused # of New Nurse % of New Nurse Amount Expenditure Portion # of New Nurse Graduates Graduates Fiscal Year ($ million) ($ million) ($ million) Graduates Participating Participating 2007/08 88.9 71.7 17. 2 4,300 2,660 62 2008/09 94.2 72.3 21.9 4,902 2,825 58 2009/10 85.8 85.5 0.3 5,139 2,598 51 2010/11 87. 5 54.8 32.7 5,555 1,804 32 2011/12 99.6 66.0 33.6 6,386 2,237 35 Total 456.0 350.3 105.7 98 2013 Annual Report of the Office of the Auditor General of Ontario

it indicated that not every nursing graduate will the people of Ontario. Patients at these clinics see secure a job through this program. Some move a nurse practitioner (NP) as their primary health into positions in the same organization where they care provider, consulting with a physician only completed their clinical placement, but outside the when needed. The first clinic opened in Sudbury in program, while others may decide to continue their 2007 and served as a pilot project for the initiative. studies or take time off. Nevertheless, the signifi- In November 2007, the Ministry announced that cant decline in graduate participation in this pro- it would establish 25 more NP-led clinics across gram raises questions that need to be considered in the province. In 2011/12, the 26 clinics received evaluating the program’s effectiveness. $29 million in ministry funding for development An external party has evaluated the Nursing and implementation. Graduate Guarantee Program annually since Each clinic can have up to four NPs who operate 2007/08 using ministry data supplemented by a in collaboration with an interprofessional team survey of participants and employers. The 2011/12 (such as RNs, dietitians and social workers) to evaluation resulted in nine recommendations that provide increased access to primary health care. To included, for example, continuing to promote the achieve this, the clinic may, for example, focus on participation of long-term-care and community providing: employers in the program; supporting the partici- • family health care for people previously with- pation of northern, rural and small organizations out a primary care provider; in the program; and examining the differences in • chronic disease detection and management, employment status, retention and transition into such as obesity programs, smoking cessation the nursing profession across sectors. The Min- and cancer screening; and istry indicated that it was working to address the • faster access to care through house calls and recommendations. same-day or next-day appointments and However, the evaluator has not assessed the extended hours. overall effectiveness of the program. For example, We looked at information from a sample of clin- it has not reported on the total percentage of nurse ics to determine whether the majority of patients

Chapter 3 • VFM Section 3.02 graduates who have transitioned to permanent full- had indeed previously been without primary care time employment through the program. providers. We found that only two of the five clinics The Ministry has set performance targets for we contacted had taken steps to document whether the number of temporary full-time positions to be their patients had family physicians. Two of the bridged—the number of participants—but has not other clinics informed us that patients were asked set an outcome-based performance target— the if they had a family physician. The remaining clinic number of participants to achieve permanent full- did not begin collecting this information until April time positions—for the Nursing Graduate Guaran- 2013, when it began having patients complete an tee Program. In the program’s first year (2007/08), intake form that included a question about family the target was 3,000 positions to be bridged; by physicians. 2011/12 the target had decreased to 2,500 pos- At the time of our audit, the Ministry‘s overall itions. The Ministry met its targets in only one of target was to have a total of 40,000 registered the five years—2008/09. patients (who do not have regular access to a family health-care provider) at all 25 clinics. As of January 2013, the 24 NP-led clinics that were open (one NP- Inadequate Assessment of Nurse Practitioner– led clinic is targeted to open late in the 2013/14 fis- led Clinics cal year) reported having about 33,000 registered The purpose of nurse practitioner–led clinics is to patients, or 82% of the program target. Given that provide increased access to primary health care to Health Human Resources 99 the clinics are a new model of primary health-care monitor the nurse practitioner-led clinics delivery and that many of them had been open less • more closely to ensure that they are meeting than two years, not all clinics are at full capacity, as program requirements and achieving their it takes time to establish a patient roster. patient targets and program objectives. Clinic budgets setting out operational and one- time costs are approved annually by the Ministry. MINISTRY RESPONSE Clinics are required to submit quarterly and annual financial and performance information and statis- The Ministry acknowledges the recommenda- tics to the Ministry for review as well as an audited tion and comments regarding the Nursing statement of revenues and expenditures for the Graduate Guarantee (NGG) program and the year. Our review of ministry documentation from a nurse practitioner–led clinics. sample of clinics indicated that the Ministry gener- Recognizing the importance of monitoring ally followed up with clinics on matters during and evaluating the NGG program, work has their development phase. Ministry staff informed been under way since early 2012 to enhance its us that they reviewed clinic operating costs, com- online management. A new tool was launched paring actual operating expenditures to those set in April 2013 that allows the Ministry to collect out in the clinic’s budget, approving funding levels and analyze information that was previously and discussing any variances with the appropriate unavailable, including monitoring aspects of clinic staff. NGG participation rates by both nursing gradu- The Ministry’s performance measure for ates and employers, as well as employment this program is the establishment of 25 nurse outcomes. practitioner–led clinics, which has been met. The annual program evaluation will con- However, this measure does not assess whether the tinue and will be enhanced to examine the clinics are effectively meeting program goals. At variations in participation rates. the time of our audit, the Ministry had evaluated The Ministry will also implement a targeted only one clinic (Sudbury) for its effectiveness in communication strategy to promote increased

meeting program objectives. uptake of NGG participants across health-care Chapter 3 • VFM Section 3.02 sectors, with particular attention to the com- RECOMMENDATION 2 munity—for example, in home care, long-term care, primary care and public health sectors. To provide an appropriate level of nursing ser- The recommendation about nurse vices and thereby improve access to care across practitioner–led clinics is timely as the clinics the health sector, the Ministry of Health and transition from startup to full operations. Long-Term Care should: The Ministry will review how it can apply monitor nursing employment trends and • greater oversight to these clinics and ensure assess the outcome of its nursing initiatives accountability for outcomes including in transitioning graduating nurses to perma- achievement of patient targets and program nent full-time employment; objectives. The Ministry will continue to assess the reasons for declining participa- • take timely, appropriate action when non- tion rates of nurse graduates in its Nursing compliance with agreements is identified. Graduate Guarantee Program, and take steps to improve program effectiveness, including encouraging participation in the program across sectors; and 100 2013 Annual Report of the Office of the Auditor General of Ontario

Untimely Recovery of Unspent Funds recovering from the clinics. The Ministry also informed us that it is working to complete the Organizations that receive transfer payment fund- reconciliations that remained for 2009/10, ing from the Ministry are required to submit annual 2010/11 and 2011/12. financial statements. Subsequent to year-end, the In October 2012, the Ministry of Finance’s Ministry reviews year-end financial statements to internal audit department issued a report on assess whether it is owed any surplus funds. Any oversight and monitoring of transfer payment payables to the Ministry are recovered from the recipients that made observations similar to ours. transfer payment recipients. It recommended establishing outcome-based For the nursing initiatives we examined in our performance measures and guidelines for review audit, we found that some related transfer payment and analysis of financial reporting to increase agreements did not set out time requirements for consistency and enhance efficiency. The Ministry submitting or completing the financial reconcilia- informed us that the focus of the strategy to date tion. For some organizations that received fund- has been to establish capacity in the province by ing through the 9,000 Nurses Commitment, the increasing the supply of providers, which relates information provided was lumped into broader to an output-based measure. It has plans to estab- programs instead of being broken down by initia- lish outcome-based performance measures and tive. For example: guidelines where appropriate. • Our review of a sample of the organizations that received 9,000 Nurses Commitment RECOMMENDATION 3 funding found 36 programs for which many of the year-end reconciliations had not been To improve financial oversight of funded completed on a timely basis. organizations and recover unspent funds, the • At the completion of our audit, the Ministry Ministry of Health and Long-Term Care should had completed reconciliations up to only perform timely reviews of relevant financial 2009/10 for the Nursing Graduate Guarantee statements. Program. Pending completion of the other Chapter 3 • VFM Section 3.02 years, the total amount of recoveries identi- MINISTRY RESPONSE fied to date was at least $7.3 million. The The Ministry is committed to enhancing finan- Ministry was still in the process of recovering cial management systems to ensure optimal the funds. use of financial resources. In 2013/14, the We looked at a sample of five nurse • Ministry is implementing processes that will practitioner–led clinics and identified total ensure emphasis is placed on in-year review operating funds of about $1.3 million owed to and analysis of financial reports submitted by the Ministry for the 2011/12 fiscal year. This funding recipients to support timely recovery or amount represented about 30% of the total payment adjustments. funding provided to these clinics. In addition, For the HealthForceOntario strategy, the the audited statements of two of the clinics Ministry has made significant progress over the showed about $360,000 owing to the Ministry past two fiscal years through dedicated efforts in for the 2010/11 fiscal year. Shortly after we the area of reconciliations. Its goal is to have all completed our audit fieldwork, the Ministry reconciliations current by March 31, 2014. indicated that it had identified approximately $3.4 million in recoveries related to 2009/10 and 2010/11, which it was in the process of Health Human Resources 101

HEALTH HUMAN RESOURCE able quantifiable data on physician productivity. FORECASTING MODELS Physician productivity is an important component, and even small improvements in productivity can Forecasting models are recognized as one import- have a significant impact on the number of human ant component of evidence-based health human resources required in the system. resource planning. Good information and proper During development, the consultant who built health human resources planning are essential if the model defined productivity as the number of the Ministry and health system stakeholders are to patients seen in the physician’s practice for a given work together to determine an appropriate number period of time, noting several factors that affect and mix of health professionals to meet the health physician productivity, including: needs of Ontarians across the province. • information and communication technology (electronic health records, telemedicine); Better Physician Forecasting Data Needed • health system change (new or different primary health-care models, such as Family Some Canadian jurisdictions are engaged in Health Teams); physician forecasting and modelling. For example, non-physician clinicians (other health-care Alberta and Nova Scotia have developed needs- • providers working with physicians, such as based physician forecasting models similar to NPs); and Ontario’s to plan their physician supply require- funding and compensation models. ments. This type of planning generally involves • Even for these four factors, specific quantifiable estimating the health services required to meet the data for only one category of non-physician clin- needs of the population and then translating them icians—NPs—was available and incorporated into into the number and type of physicians required to the model. deliver those services. Also, while physician human resource forecast- Historically, physician human resources plan- ing reflects factors such as workforce demographics ning in Ontario has been supply- or utilization- and changing population health needs, it is also sig- based; however, this method does not provide a

nificantly affected by broader economic, social and Chapter 3 • VFM Section 3.02 complete picture because it does not consider the health-system trends, as well as health technology population’s health needs. In 2009, the Ministry advancements. Many of these other factors—the partnered with the Ontario Medical Association availability of diagnostic and laboratory equipment, and used a tendering process to select an external operating-room time and space to perform surger- party to develop a new, needs-based model. The ies, physician preferences for certain specialties and new model works by examining the population’s practice locations, and employment opportunities health needs and translating them into needs for across the province—can affect access to and physician services, then comparing these needs to delivery of health-care services, but they are not the supply of physician services currently available. easily incorporated into the model. Although the Service gaps are quantified and converted into the needs-based simulation model does make it pos- number of physicians required to meet the needs. sible to test “what-if” simulations that may help to Ontario’s physician forecasting model is a assess the impact of some of these factors, it cannot positive step in determining physician workforce incorporate all of them. requirements. However, the model is hampered by For these reasons, the results obtained from the limited reliability and availability of data. We Ontario’s physician forecasting model can be found that Ontario’s model does not account for considered only one of many tools and pieces of some important variables because of a lack of avail- evidence available to support policy formulation. 102 2013 Annual Report of the Office of the Auditor General of Ontario

The model’s limitations make it difficult to use in conduct a simulation model analysis that would planning for the optimal number, mix and distri- accurately estimate the requirements for this nurs- bution of physicians with appropriate funding, ing group. training and deployment across the province. During the course of our audit, the Ministry was in the process of working with the Ministry of Training, Colleges and Universities to develop Lack of Forecasting of Demand for Nursing a supply-based nursing simulation model for RNs Services and RPNs, with results expected to be available by In 2008, the Ministry also engaged an external late 2013. consultant to develop a needs-based nursing The Ministry advised us that the purpose of model that would be able to project the gap the supply-based simulation model is to help between the need for and the actual supply of the government plan properly for future nursing RNs, RPNs and NPs in Ontario for each year over a education positions and formulate nursing recruit- 10-year period. Separate simulation models to test ment and retention policies. The initial model various health human resources policy scenarios is to provide projections for the future supply of were also developed for both RNs and RPNs. nurses (RNs and RPNs) at the regional level and Although the model cost about $435,000 to will enable planners to test the potential impact of develop, it was initially not used to specifically policy changes on the supply, such as changes to inform any nursing policies because of concerns enrolment numbers and percentages of full-time about the accuracy of its predictions. nurses, the introduction of incentives for working In 2012, as part of its ongoing evidence in rural and remote areas, shifting distribution development work, the Ministry found that the of nurses by employment sector, recruitment model had understated data for first-year enrol- and retention rates and other attrition factors. ment of RNs in 2007 and incorrectly assumed However, a supply-based model cannot assess that all RNs provided the same rate of direct whether the supply is appropriate because it does patient care regardless of their years of experi- not take into account the population’s need for

Chapter 3 • VFM Section 3.02 ence; in practice, younger RNs just entering the nurses. In addition, the new supply-based model profession may be providing different amounts will not include NPs because some historical data of direct patient care than RNs near the end of specific to them is not available from the College their careers. The model had also applied an of Nurses of Ontario. The Ministry informed us estimated percentage of total RNs providing direct that the initial focus is on the future supply of patient care based on previous outdated data. nurses, which is important work that needs to be The Ministry corrected the first-year enrolment completed to support future needs-based model- data and updated the direct patient care data and ling considerations. completed two simulations. However, assump- To get better information to inform future tions regarding attrition, retirement and workload policy work, the Ministry entered into an agree- were not updated, and no other data reviews were ment with a large hospital in late 2012 to create a conducted. one-time snapshot of the current supply, distribu- The model has other limitations. For example, tion and predicted shortfall or surplus of RNs, it forecasts the gap between supply and need at RPNs and NPs working in selected primary health- the provincial level but not at a regional level. care organizations, long-term-care homes and At the time of model development, data on the acute-care hospitals across the province for the patterns of service delivery for NPs was not avail- next three months. Each surveyed health organ- able, and therefore the consultant was unable to ization is to report on current staffing, vacancies, Health Human Resources 103 details about leaves of absence and predictions of MINISTRY RESPONSE short-term staffing changes. There will also be a comprehensive analysis of the overall staffing situa- The Ministry agrees with the audit findings that tion in each area. This data will be used to identify its physician forecasting model is a positive step which organizations and geographical areas are in determining physician workforce require- having difficulty in recruiting and retaining nurses ments. Similarly, the Ministry concurs with the in Ontario. audit observations regarding forecasting models as one of many types of tools that are required RECOMMENDATION 4 to support health human resource planning. As such, the Ministry is actively engaged To provide reasonable and reliable forecasts with the health sector to improve evidence for of the requirements for physicians and nurses decision-making. Over the summer of 2013 and to better ensure effective health human the Ministry has been meeting with the field resources planning, the Ministry of Health and regarding the current health human resource Long-Term Care should: environment, including how we continue to conduct assessments of employment trends, • evolve and develop evidence. This work will the supply and projected needs for health contribute important information to inform services, and the associated health workforce future HealthForceOntario work. requirements to best meet those needs cost- The HealthForceOntario strategy will effectively; and continue to provide innovative health human for physicians and nurses, further refine its • resource solutions to meet patient-care needs. forecasting models and their capabilities Evidence for decision-making will continue to to assess the impact of various factors on be a key aspect of the strategy and the Ministry service-provider productivity. will seek to enhance and expand tools, including forecasting models, to improve planning. Chapter 3 • VFM Section 3.02 Chapter 3 Ministry of Education Section 3.03 Healthy Schools Strategy

Ontario government set a goal to reduce childhood Background obesity by 20% in five years. The Ministry has the primary responsibility for developing and supporting the implementation While academic success is a major priority for the of policies and programs related to students in Ministry of Education (Ministry), so too is student publicly funded schools. In recent years, we have health and well-being. Because healthy children audited the Ministry’s four key strategies designed are better prepared to learn, and schools can help to contribute to student achievement: Literacy and students lead healthier lives, the Ministry has estab- Numeracy Strategy (2009); Safe Schools Strategy lished the Healthy Schools Strategy to help support (2010); Student Success Strategy (2011); and now student learning and growth through proper nutri- the Healthy Schools Strategy. tion and daily physical activity. To achieve better Ontario’s 72 publicly funded school boards are

Chapter 3 • VFM Section 3.03 student health, the Ministry relies on the support responsible for implementing Ministry policies and of other government organizations, such as the programs. These boards operate 4,900 elemen- Ministry of Health and Long-Term Care, which often tary and secondary schools with an enrolment of takes a lead role in child and youth health-related approximately 2 million students. The Ministry matters, and the Ministry of Children and Youth advised us that it spent approximately $4 mil- Services, which sponsors programs to provide meals lion annually over the three fiscal years 2009/10 to students in many Ontario schools. The Healthy to 2011/12 on activities related to the Healthy Schools Strategy also supports the efforts of parents, Schools Strategy. who play the primary role in child development. The number of overweight children and youth in Canada has increased dramatically over the past 30 years. Nearly one in three students is over- Audit Objective and Scope weight. Almost 12% are considered obese—almost twice as many as in the late 1970s. In addition, The objective of this audit was to assess whether Statistics Canada says just 7% of Canadian children the Ministry of Education and selected school get the recommended 60 minutes of physical activ- boards had adequate procedures in place to: ity daily. The increasing rate of overweight children is a significant public concern, and in 2012 the

104 Healthy Schools Strategy 105

• implement policies and initiatives designed to help improve health and academic achieve- Summary ment for Ontario’s students through better eating habits and increased physical activity; and The Ministry of Education has recognized that ensure the identification of good practices, • healthy students are better prepared to learn and oversight of schools, and the measurement has taken several steps to help students increase and reporting of results. their physical activity and eat healthier foods. For Senior management at the Ministry and selected example, the Ministry has developed policies for school boards reviewed and agreed to our audit the nutritional requirements of food and beverages objective and associated criteria. sold in schools, and revised the school curriculum to Our audit work was conducted at the Ministry’s require that all elementary students get 20 minutes Healthy Schools and Student Well-Being Unit, of daily physical activity. However, the Ministry and which holds primary responsibility for the Healthy school boards need to make greater efforts to ensure Schools Strategy, as well as at three school boards compliance with their requirements and they need and at selected elementary and secondary schools to work more effectively with other organizations within these boards. The school boards we visited and stakeholders, including parents, to share best were the York Catholic District School Board, the practices and achieve common goals. Our more Hamilton-Wentworth District School Board and the significant concerns include the following: Trillium Lakelands District School Board. Neither the Ministry nor the school boards We also spoke with representatives from a num- • visited had an effective monitoring strategy in ber of other ministries and organizations, including place to ensure that food and beverages sold the Ministry of Health and Long-Term Care; Public in schools comply with the nutrition standards Health Ontario, an arm’s length government agency in the Ministry’s School Food and Beverage dedicated to protecting and promoting the health Policy. To illustrate, none of the three school of all Ontarians; the Healthy Kids Panel, a body boards we visited had reviewed the food and of experts that provided recommendations to the beverages sold in their cafeterias to ensure

Minister of Health and Long-Term Care on how to Chapter 3 • VFM Section 3.03 that the items met nutrition standards. reduce childhood obesity and improve children’s Furthermore, a cafeteria vendor at one school health; the Ministry of Children and Youth Services, board did not have sufficient nutrition infor- which provides oversight and funding for school- mation to show compliance, and based on the based student nutrition programs; local public nutrition information that was provided, we health units that have a mandate to work with identified a number of instances where the school boards and schools on topics such as healthy products did not comply. eating and physical activity; and the Ontario Society Both the Ministry and school boards visited of Nutrition Professionals in Public Health, which • had limited data to assess whether the represents dietitians working in public health. School Food and Beverage Policy contributed In conducting our audit work, we reviewed to better student eating behaviours. After relevant legislation, policies and procedures. We introducing healthier food choices, second- also met with appropriate staff from the Ministry ary school cafeteria sales at the three boards and the school boards and schools visited, includ- visited decreased between 25% and 45%, and ing supervisory officers, principals and teachers. vending machine revenues dropped between We also researched policies and practices in other 70% and 85%. The secondary school princi- jurisdictions and consulted with experts with know- pals to whom we spoke indicated that many ledge of healthy eating and physical activity in the school environment. 106 2013 Annual Report of the Office of the Auditor General of Ontario

students now prefer to eat at nearby fast food introduced in 2005. The teachers we spoke to outlets instead of choosing the healthier foods confirmed the Panel’s observations. offered in the school cafeteria. • The Ministry’s only requirement for physical • Information we received was not always com- activity at the secondary school level is the plete in terms of nutritional detail or listing completion of one credit course in health and all food items available for sale. Nevertheless, physical education during a student’s four our review of a sample of menu items at one years of high school. A 2011 survey by the school board identified a significant number Centre for Addiction and Mental Health of that did not meet the nutrition criteria in the students in grades 7 to 12 indicated that just School Food and Beverage Policy, including 20% of students reported that they partici- some that deviated from it significantly. For pated in 60 minutes of daily physical activity example, we noted a soup that contained as recommended by the Canadian Physical twice the amount of fat allowed, a side dish Activity Guidelines. Some other jurisdictions that exceeded the allowable limit of sodium have substantially greater physical activity by more than 40%, and a dessert that had just requirements for secondary students; for one quarter of the required amount of fibre. example, Manitoba students must obtain • Many of the board and school staff we spoke four high school health and physical educa- to noted that children’s eating habits can be tion credits in order to graduate, and British more effectively influenced while children Columbia expects high school students to are still in elementary school. However, most participate in at least 150 minutes of physical elementary schools do not have cafeterias or activity per week. otherwise give students the opportunity to • The Ministry and school boards need to make healthy food choices. Therefore, effect- better integrate their activities with other ive communication is critical to establishing ministries and organizations, and leverage healthy eating habits at an early age. Greater their resources and expertise. For example, efforts are needed by boards and schools to in the 2011/12 school year, student nutrition

Chapter 3 • VFM Section 3.03 identify and share good practices and the programs funded by the Ministry of Children materials they have developed. and Youth Services provided almost 700,000 • There is no formal monitoring strategy at children and youth with meals and snacks at either the Ministry, the school boards or little or no cost that were subject to different the schools visited to ensure that students nutrition standards than those of the Ministry in grades 1 to 8 get 20 minutes of daily of Education. In another example, in 2011, in physical activity during instruction time as preparation for the School Food and Beverage the Ministry’s curriculum requires. Two of Policy’s implementation, one board arranged the three boards we visited surveyed school for the local public health unit to make site representatives, and more than half of those visits to its schools’ cafeterias and found that who responded said that students at their a significant number of items did not meet the schools did not get the required 20 minutes Ministry’s nutrition requirements. Although of daily physical activity. As well, a recent the health unit offered to undertake a subse- report by the Healthy Kids Panel said teachers quent review, the school board did not com- find it difficult to integrate the policy and still mit to a follow-up visit. achieve other learning goals, and that the policy did not appear to have had a significant impact on students’ activity levels since it was Healthy Schools Strategy 107

OVERALL MINISTRY RESPONSE capacity through evidence-based strategies, and align our collective efforts. The Ministry of Education appreciates the Finally, as the Ministry embarks on its exten- work of the Auditor General in highlighting the sive consultations on the next phase of Ontario’s contribution healthy schools make in supporting education strategy, a key conversation will be student learning and growth. The Ministry about student well-being and the role of schools acknowledges the recommendations that sug- in supporting it. As we continue to define the gest more needs to be done to help ensure that role of student well-being in the education provincial policies related to healthy eating and system, any potential future ministry activity physical activity are being implemented effect- related to healthy eating and physical activity ively across the province. will need to be examined within this context. Although parents play the primary role in child development, through the education system, the Ministry is uniquely positioned to work with other ministries, such as the Ministry of Children and Youth Services (MCYS), the Min- Detailed Audit Observations istry of Health and Long-Term Care (MOHLTC) and the Ministry of Tourism, Culture and HEALTHY EATING Sport, to contribute to healthy child and youth development. For instance, the Ministry sup- The number of overweight and obese children in ports Ontario’s Healthy Kids Strategy, a three- Canada has increased dramatically over the past pronged approach to combat childhood obesity: 30 years. Although the magnitude of the increase start all kids on the path to health, change the varies depending on the definition of overweight food environment, and create healthy com- and obese, the trend is upward, and the increasing munities. In this regard, the Ministry works with rate of overweight children is a significant public MOHLTC and MCYS through a ministers’ work- concern. This concern led to amendments to the

ing group and steering committee to help imple- Education Act in 2008 that gave the Minister of Edu- Chapter 3 • VFM Section 3.03 ment elements of the Healthy Schools Strategy cation the power to establish policies with respect relevant to the Ministry’s mandate. to nutrition standards for food and beverages pro- Within schools, the Ontario curriculum vided on school premises. continues to provide a foundation for healthy In 2010, the Ministry introduced the School eating and physical activity and encourages the Food and Beverage Policy, which sets nutrition development, commitment and capacity to lead standards for food and beverages sold in publicly healthy, active lives. The Ministry has developed funded elementary and secondary schools. Ministry policies and programs specifically designed to efforts have subsequently focused on providing complement the curriculum and create health- supports to implement this policy. School boards ier learning environments for students. The were required to ensure that all food and beverages report acknowledges the Ministry’s efforts in sold on school premises complied with the policy by this regard. In the interest of Ontario’s students, September 1, 2011. the Ministry remains committed to continuing The Ministry expects its policy to improve to work with all of its partners at the provincial, the overall nutritional quality of food and bever- regional and local levels to examine imple- ages offered for sale in schools. This policy is also mentation of healthy schools policies, explore intended to complement what is taught through the opportunities to further promote and build curriculum and contribute to overall government 108 2013 Annual Report of the Office of the Auditor General of Ontario

efforts to encourage healthier food choices and The School Food and Beverage Policy says improve student eating behaviours. As well, it is school boards are responsible for monitoring intended to complement what is already taught in its implementation. However, we observed that health and physical education classes and contribute centralized school board efforts to ensure compli- to overall government efforts to increase the percent- ance were either not in place or were limited. At all age of school-aged children and youth who have three school boards we visited, the responsibility to healthy weights and decrease the rates of chronic ensure compliance with the policy had been dele- weight-related health issues such as heart disease gated in whole or in part to the school principals. and type 2 diabetes. The Healthy Kids Panel recently As well, a recent survey of the perceptions of diet- reported that obesity alone cost Ontario about itians from public health units identified challenges $4.5 billion in 2009, including $1.6 billion in direct that may warrant school board and/or ministry health-care costs. attention, including concerns regarding efforts to monitor for compliance, and inconsistencies in the policy’s implementation from school to school. Monitoring Implementation of the School Since the implementation of the School Food Food and Beverage Policy and Beverage Policy, none of the boards visited Ministry efforts to ensure that school boards had reviewed the food and beverages sold in their comply with the School Food and Beverage Policy school cafeterias to ensure that the items complied were mainly limited to obtaining letters from the with the policy. This was of particular concern at boards indicating they would be in compliance by one school board we visited where its cafeteria September 1, 2011. The Ministry advised us that vendors either did not have sufficient nutrition all but one of the 72 school boards had provided information to show compliance with the policy, such a letter. The Ministry also initiated an annual or the nutrition information provided identified a support and monitoring plan in the 2012/13 school number of instances where items did not comply. year to gain insight into the implementation of In lieu of direct monitoring or assessments, one ministry policies and programs relating to healthy of the school boards we visited required school

Chapter 3 • VFM Section 3.03 schools, including the School Food and Beverage principals to obtain letters from vendors indicating Policy. As part of this process, the Ministry gave compliance with the School Food and Beverage web-based seminars and distributed bulletins on Policy. Principals were also expected to complete related research and promising practices from the a form concluding on whether the food and bev- field. The Ministry also conducted site visits to 12 erages sold were compliant. However, we were school boards and a number of schools in each of informed that in the 2012/13 school year the board these boards. Although the Ministry did not specif- did not collect the forms completed by principals, ically assess compliance with the School Food and and the principals at the schools visited indicated Beverage Policy, it did identify strategies deemed that they did not maintain documentation to sup- important to the successful implementation of the port their conclusions. At another board, we were policy, including working with community partners told that school principals were required to obtain and engaging parents to promote healthy eating. letters from vendors indicating that they would The Ministry also identified school board chal- comply with the Ministry’s policy. The principals lenges to the implementation of the policy, such as were not expected to confirm that the items for a decline in cafeteria revenues and concerns about sale did in fact comply. The third school board did parental support for the policy. The Ministry told not require principals to obtain or complete any us it plans to share the information gathered from documentation, and the principals at the schools these visits with all school boards. we visited in this board did not formally monitor compliance with the policy. Healthy Schools Strategy 109

None of the elementary schools in the boards • In some circumstances, where a serving we visited had cafeterias or vending machines. contains more than one major ingredient, At two of the boards visited, food and beverages the School Food and Beverage Policy can were seldom sold in elementary schools. How- be applied to individual product ingredients ever, we were told that elementary schools in the rather than the meal as a whole. If whole meal third board regularly offered food and beverages information is not available, each individual for sale. While the schools visited at this board ingredient must comply with the policy’s nutri- had obtained letters from most of their vendors tion criteria. At two school boards we were indicating they would comply with the policy, the supplied with information on the ingredients schools had generally not formally assessed the of entrees that had been classified as healthy nutrition information for the food sold in such options. Upon reviewing this information, we programs to ensure that it complied. found examples where entrees were compliant because each ingredient met the nutrition criteria, whereas if the criteria for the meal Compliance with the School Food and as a whole were used, the entree would have Beverage Policy been reclassified as a less healthy option or not Virtually all the secondary schools at the three permitted for sale in a school cafeteria because school boards visited had a cafeteria for students to the sodium or fat exceeded the policy’s limits. purchase food and beverages, and most had vend- • Compliance with the policy’s nutrition ing machines. External vendors operated the vast standards can depend on portion size. Con- majority of cafeterias and vending machines. Since sequently, a student can purchase two servings none of the boards we visited maintained nutrition that, although individually compliant with the information for the food and beverages sold in policy, would not comply if they were deemed their cafeterias, we obtained nutrition information a single meal. For example, at one elementary directly from these external vendors. school we visited, pizza was the most popular Information we received from the vendors was hot lunch offered for sale to students, and each

not always complete in terms of nutritional detail piece complied with the policy. However, if Chapter 3 • VFM Section 3.03 or listing all items available for sale. Nevertheless, two pieces were purchased, the meal would our review of a sample of menu items at one school exceed the criteria for fat and sodium. We board identified a significant number that did not found that approximately 20% of students pur- meet the nutrition criteria in the School Food and chased more than one piece. We also observed Beverage Policy, including some that deviated from that other pizza brands could have been it significantly. For example, we noted a soup that ordered by the school that would have been contained twice the amount of fat allowed, a side compliant even if two pieces were consumed. dish that exceeded the allowable limit of sodium • We reviewed school board revenue informa- by more than 40%, and a dessert that had just one tion for the cafeterias that had sufficient quarter of the required amount of fibre. comparable data and found that sales Although our review of the information provided decreased by 25% to 45% at the three school by vendors at the other two boards revealed only boards visited following implementation of minor compliance exceptions, we noted that compli- the School Food and Beverage Policy. Vend- ance with the policy presents challenges and might ing machine revenues at these school boards not always result in offering students the healthiest also dropped by about 70% to 85%. Most options or students making the healthiest choices: school board administrators indicated that the substantial decline in sales suggested 110 2013 Annual Report of the Office of the Auditor General of Ontario

that the policy’s introduction was not as suc- All three school boards we visited indicated that cessful as had been hoped. The secondary they had provided additional training on the policy school principals generally shared this view to all their principals in 2010; two of the boards and noted that many students now choose to said they also trained all their vice-principals. We eat lunch at nearby fast food outlets instead observed that all three boards had used a train-the- of eating the food available in the school trainer approach whereby they centrally provided cafeteria. A survey of dietitians from public training to teachers from individual schools who health units also highlighted these concerns were expected to take that information back to the as challenges to the implementation of the rest of the school’s teachers and staff. Based on policy. As well, cafeteria vendors from all the information available, we found that the three three school boards identified concerns to boards trained between 5% and 15% of their teach- varying degrees about their ability to meet ers. However, none of the boards had collected the nutrition requirements while providing information to determine how many teachers were food at a reasonable price and quantity that subsequently trained by either these teachers or was also appealing to students. their principals. At one school board, most teachers we spoke to indicated that they had not received training on Training on the School Food and Beverage the policy. At another board, less than half of the Policy teachers to whom we spoke said they had received To reinforce the benefits of healthy eating and sup- training although some instruction had been given port the implementation of the School Food and in staff meetings. At the third school board, we Beverage Policy, which took effect in September were told that all teachers were trained during 2011, the Ministry developed and distributed sev- staff meetings, and almost all teachers to whom we eral resources to school boards. These included a spoke at this school board said they had received resource guide for school board and school admin- such training. istrators and a quick reference guide for those Overall, although both the Ministry and boards

Chapter 3 • VFM Section 3.03 responsible for purchasing food and beverages. indicated that it would be beneficial for teachers to Resource guides were also produced for elementary be trained on the requirements of the School Food and secondary school teachers that included infor- and Beverage Policy and how to integrate healthy mation about the policy’s nutrition standards and eating concepts into classroom instruction, including strategies to make connections to healthy eating in in subjects that do not cover curricular components classroom instruction. on healthy eating, no procedures were in place to The Ministry also provided three rounds of ensure that current and new teachers and school training between January 2010 and March 2012 administrators received such training. In addition, for principals, teachers and public health staff. For many of the teachers to whom we spoke said teach- each round of training the Ministry invited each ers do not generally have the training to integrate school board to send seven to eight participants; healthy eating concepts into subjects that do not in total, more than 1,200 school board staff were already include a curricular component on healthy trained. The Ministry told us that the intent was for eating. They also noted that, if the intent is for teach- those who attended the sessions to subsequently ers to include healthy eating concepts in classroom train other teachers and principals in their school instruction, healthy eating concepts should be added boards. Accordingly, the Ministry provided approxi- to the curriculum. The school board representa- mately $2.4 million in funding to school boards for tives to whom we spoke generally shared this view. such training. Furthermore, a recent survey of the perceptions of Healthy Schools Strategy 111 dietitians from public health units cited a lack of buy- The Ministry notes that the school environment in and knowledge of the policy among school staff as has a significant impact on student attitudes, prefer- one of the barriers to its successful implementation. ences and behaviours. The Ministry’s policy groups food and beverages into three categories—“sell most,” “sell less” and “not permitted for sale.” Food Measuring the School Food and Beverage and beverages meeting the “sell-most” nutrition cri- Policy’s Effectiveness teria are described as the healthiest and must com- The Ministry issued the School Food and Beverage prise at least 80% of available options. However, Policy to improve the overall nutritional quality while many “sell-most” items are healthy options, of food and beverages offered for sale in schools. they may not all be the healthiest options to encour- The policy was also intended to help reinforce the age better eating behaviours. To illustrate, we noted instruction provided to students on healthy eating examples in the “sell-most” category included hot through courses such as health and physical educa- dogs, pizza, muffins and cookies that meet nutrition tion. The Ministry’s intent is to provide students requirements. The Ministry informed us that part with the opportunity to put into practice what they of the intent is to teach students that it is possible are taught in the classroom by ensuring that food to eat healthier versions of food not traditionally and beverages sold in schools are healthy. However, considered healthy, and it is better to offer healthier the Ministry does not have information systems in versions of the foods students prefer than have stu- place to gather data that would provide insight into dents eat unhealthy options elsewhere. the degree to which the policy has been success- The School Food and Beverage Policy applies fully implemented. The Ministry has also not yet only to items sold at publicly funded schools. Since established how it plans to measure the success of food and beverages were not generally sold in ele- the policy or assess whether it has contributed to mentary schools at two of the three boards visited, healthier student eating behaviours. the opportunity for the policy to affect students at As part of its Healthy Schools Strategy, the Min- the elementary level was limited at those boards. istry encouraged schools to participate in its Healthy Only one of the three school boards visited gave

Schools Recognition Program, a voluntary program students regular opportunities to purchase lunch Chapter 3 • VFM Section 3.03 in which schools pledge to undertake at least one supplied by a vendor. Thus, the impact of the policy healthy activity. From the time the program began at elementary school often is limited to food and in 2006 to its temporary suspension in the 2012/13 beverages sold for fundraising and special events. school year, the Ministry noted that more than Most school board representatives to whom we 2,300 schools had pledged to undertake more than spoke said they could more effectively influence 11,600 healthy activities. The Ministry did not have the eating habits of elementary students than sec- aggregate information on how many such activities ondary students. As well, a number of elementary related to healthy eating or physical activity and school teachers and administrators indicated that could not say how many students these activities more in-school opportunities could help their stu- reached or whether they were effective in increasing dents develop healthier eating habits. physical activity or encouraging healthier eating. Two of the three school boards visited had This was similarly the case at the school boards we not attempted to measure their success in imple- visited, where we observed that participation in this menting the policy or to determine how well they program ranged from about 2% to 35% of schools had contributed to healthier eating behaviours. in the 2011/12 school year. Also, the boards were Nevertheless, our discussions with school and generally unaware of the reach or effectiveness of board staff at these two boards revealed that most school-based programs that might be in place. felt the food and beverages sold at their schools 112 2013 Annual Report of the Office of the Auditor General of Ontario

were healthier since the policy’s implementation. activities, and periodically measure progress The third school board had put in place a good in achieving these objectives. practice to review its policy annually. As part of its most recent review, this board surveyed almost MINISTRY RESPONSE 200 stakeholders, including parents, students and school board staff. Most respondents said that the The Ministry agrees that in order to contribute board’s policy encouraged students to make health- to improving student eating behaviours, strat- ier choices, that information about healthy eating egies need to be in place to effectively imple- had been provided to parents, and that foods with ment the School Food and Beverage Policy. poor nutrition had been removed from the school. The School Food and Beverage Policy is one However, only half of the survey respondents indi- initiative intended to contribute to improved cated that students were now eating the more nutri- child and youth eating behaviours. It represents, tious food available at school. As well, the majority for many, a significant shift in the type of food of school staff we spoke to at this board said they and beverages schools can choose to sell. The had not seen a significant change in student eating Ministry remains committed to its policy that habits since the policy’s implementation. Overall, if food and beverages are offered for sale in the results suggested that some progress had been schools they meet nutrition standards. made in encouraging healthier eating by students As such, the Ministry will continue to engage but significant work remained to be done. with school boards, educators, students, parents and our partners in the health and food services RECOMMENDATION 1 sectors to promote effective evidence-based strategies that contribute to the promotion of healthy To help ensure that offering healthier food eating behaviours among Ontario’s students. choices in schools contributes to improved The Ministry will do the following: student eating behaviours and their goals of review and improve our support and mon- improving student health and academic achieve- • itoring plan in an effort to capture and share ment, the Ministry of Education (Ministry) and

Chapter 3 • VFM Section 3.03 implementation challenges, effective practi- school boards should: ces and supports; develop consistent and effective strategies • work with school boards to establish a risk- to monitor compliance with the Ministry’s • based approach to monitoring compliance School Food and Beverage Policy, especially and reporting on implementation of the ensuring that all items sold in schools com- School Food and Beverage Policy; ply with the policy’s nutrition standards; provide implementation supports to those capture additional data on the benefits of • • within the education sector, based on identi- and challenges to implementing the School fied needs; and Food and Beverage Policy in order to assess continue to encourage and foster the the policy’s impact and identify areas on • development of local partnerships between which to focus future efforts; the education and health sectors to support ensure that school administrators and teach- • implementation and contribute to improved ers receive sufficient training and supports student health and academic success. on how to implement the policy and promote The Ministry is also committed to developing healthy eating concepts in the classroom; and measurable objectives and related performance develop measurable objectives and related • indicators for its healthy eating activities, as part performance indicators for healthy eating of a broader effort to establish objectives and indicators for its Healthy Schools Strategy. Healthy Schools Strategy 113

PHYSICAL ACTIVITY RESPONSE OF SCHOOL BOARDS Research indicates that increased daily physical All three school boards agreed with this recom- activity may help improve a student’s academic mendation and all three were supportive of achievement. In general, physical inactivity is the need to ensure that items sold in schools increasingly becoming a national concern as several comply with the Ministry’s School Food and sources suggest that Canadian children and youth Beverage Policy. One school board noted that do not engage in the minimum 60 minutes of daily more strategies to monitor the implementation moderate-to-vigorous physical activity recom- of the policy would be helpful. Another school mended by the Canadian Physical Activity Guide- board indicated that it would be beneficial if lines. For instance, Statistics Canada says just 7% of an accreditation process was established for Canadian children aged six to 19 participated in at businesses that provide food services in Ontario least 60 minutes of moderate-to-vigorous physical education settings. Such a process, which would activity at least six days a week. rely on professionals with expertise in nutrition, To address concerns about the health and would strengthen compliance and allow educa- physical fitness of students and to help improve tors to focus their resources on education. academic achievement, in October 2005 the All three school boards were also support- Ministry issued a policy on daily physical activity. ive of capturing additional data to assess the The policy requires school boards to provide all ele- policy’s impact and developing measurable mentary students (grades 1 to 8) with a minimum objectives and performance indicators for of 20 minutes of sustained moderate-to-vigorous activities intended to improve healthy eating. physical activity each school day during instruc- However, one board commented that any meas- tional time. This policy noted that procedures urable objectives that are established must be must be developed to ensure the highest level of manageable within the context of the education safety during such activities. To support the policy’s setting, and another board cautioned that it will implementation, the Ministry included these always be a challenge to assess the impact on requirements in the elementary school curriculum.

student eating habits because most meals and Chapter 3 • VFM Section 3.03 Schools were required to implement the policy fully snacks are consumed at home and many factors by the end of the 2005/06 school year. impact what a student eats at home, including financial considerations. The school boards were also supportive of Implementation of Daily Physical Activity in the need to ensure that school administrators Elementary School and teachers have sufficient training on how The Ministry took a number of measures to sup- to implement the School Food and Beverage port the implementation of daily physical activity Policy and promote healthy eating concepts in in elementary school. These included developing the classroom. One board noted that provid- daily physical activity resource guides for school ing training to teachers and principals to link boards, principals and teachers that also address healthy eating to the curriculum has the most safety; creating an e-learning module to provide positive potential to impact students’ under- guidance on how to implement daily physical standing of the impact that healthy eating can activity, including ideas for such activities in the have on their quality of life. Another school classroom; and funding school boards for purposes board commented that it would be beneficial if such as professional development for teachers and healthy eating and nutrition was a mandated principals, the purchase of school athletic equip- component of teacher pre-service programs. ment and other resources. 114 2013 Annual Report of the Office of the Auditor General of Ontario

As previously noted, in the 2012/13 school year, to the survey indicated that daily physical activity the Ministry visited 12 school boards and a number was not provided at their schools to the extent of schools in these boards to see how well the the policy required. At this board, the administra- Healthy Schools Strategy was being implemented. tion at an elementary school we visited said it did Although the Ministry did not specifically measure not require teachers to provide students with the the degree to which daily physical activity had opportunity for daily physical activity, choosing been implemented, it identified promising prac- instead to focus efforts on literacy. At the third tices and challenges. According to the Ministry, board, while no survey had been done, about some good practices to increase physical activity half of the teachers to whom we spoke told us included committed leadership by school staff students did not get 20 minutes of daily physical and encouraging intramural sports. The most activity. They cited reasons such as a lack of space frequently identified challenges were limited time, to exercise in the classroom and a lack of time to space and facilities for physical activities. The schedule such activities around other curriculum Ministry plans to share this and other information requirements. gathered with all school boards. The Daily Physical Activity Policy says school Secondary School Physical Activity boards are responsible for monitoring its imple- Requirements mentation to ensure that all elementary students receive 20 minutes of physical activity during The only requirement for physical activity at the instruction time each day. However, we found that secondary school level is the completion of one none of the boards or schools we visited had a for- credit course in health and physical education mal process in place to monitor whether students during a student’s four years of high school. In took part in the required physical activity. A study many cases this requirement can be completed in at a school board we did not visit noted that less just half a school year. According to the Ministry, than half of the students took part in physical activ- secondary school students who graduated in 2012 ity every school day and not a single child engaged earned an average of just over two health and

Chapter 3 • VFM Section 3.03 in sustained moderate-to-vigorous physical activity physical education credits. However, 37% of these for 20 minutes or more. In addition, a 2013 report students completed only one health and physical by the Healthy Kids Panel said that according to education credit. parents, teachers and students, the Daily Physical In addition to physical education classes, all Activity Policy has not been implemented consist- three school boards provided opportunities for ently and is not having the desired impact. The students to participate in intramural sports, com- report also noted that teachers find it difficult to petitive team sports and other activities such as integrate daily physical activity into the school day dance. However, many students do not participate and still achieve all other learning expectations. and these boards could not provide an overall At two of the school boards we visited, recent indication of the participation time in such activ- surveys suggested that the policy was not imple- ities. Nevertheless, a 2011 survey by the Centre for mented as required. At one board, 63% of the Addiction and Mental Health found that only two in school principals who responded to the survey said 10 Ontario high school students surveyed reported students did not get the required 20 minutes of that they participated in 60 minutes of physical daily physical activity for reasons that included a activity daily as recommended by the Canadian lack of time and space, as well as giving priority to Physical Activity Guidelines. other academic areas. At the other board, at least Most of the staff to whom we spoke at the school half of the school representatives who responded boards and secondary schools we visited were of Healthy Schools Strategy 115 the opinion that more physical activity should be on how to implement daily physical activity in the required for secondary students. The Ministry told classroom. Also, the three boards said training was us that it too saw value in requiring additional provided to 15%, 30% and 45% of total elementary physical education for secondary students but teachers, respectively. The boards expected these noted a number of challenges in accommodating teachers to disseminate this information at their additional requirements, including all the other cur- schools. However, the boards did not know how ricular commitments. Recent reports by the Healthy many of the other teachers were provided with Kids Panel as well as Public Health Ontario and training, and none of the schools we visited main- Cancer Care Ontario recommend making physical tained records of how many of their teachers had education compulsory in every year of high school. received training on daily physical activity. Only In other provinces, more physical activity is one of the three school boards indicated that it required of students. For example, British Columbia offered new teachers training that included at least expects high school students to participate in at some instruction on daily physical activity, but we least 150 minutes of moderate-to-vigorous physical were informed that few new teachers participated activity per week; in Manitoba, secondary school in such training. students must obtain a minimum of four health and physical education credits. Ensuring Safe Physical Activities

The Ministry’s Daily Physical Activity Policy for ele- Daily Physical Activity Policy Training mentary school children stipulates that procedures To support the implementation of its elementary must be developed to ensure the highest level of school policy on daily physical activity, the Min- safety during physical activity sessions. Although istry provided school boards with approximately safety information is included in the health and $15 million between the fiscal years 2005/06 and physical education curriculum, the Ministry 2008/09 for purposes that included professional encourages boards and schools to use the Ontario development for teachers and principals. How- Physical Education Safety Guidelines put out by the

ever, the recent Ministry school board visits found Ontario Physical and Health Education Associa- Chapter 3 • VFM Section 3.03 that many boards identified a need for training on tion (Ophea), a not-for-profit organization. These the Daily Physical Activity Policy. guidelines outline safe practices for teachers and The three boards we visited did not maintain other personnel involved in physical activities for records on how many principals and vice-principals students in order to minimize the risk of accidents had received training on the policy. However, one or injuries. The guidelines address topics such as board informed us that it trained all principals and equipment, clothing and footwear, supervision and vice-principals while a second board said it trained the facilities where activities take place. all principals but not specifically vice-principals Although the Ministry had not provided training when the policy was introduced in the 2005/06 on these safety guidelines, it partnered with Ophea school year. The third board was unable to tell us to develop and distribute resource guides for school how many of its principals and vice-principals were boards, principals and teachers on daily physical trained. None of the three boards had a process in activity that included safety considerations. In place to train administrators appointed subsequent addition, the Ministry entered into a contract with to the 2005/06 training. Ophea in 2012 to provide its safety guidelines on a The Ministry, as well as those to whom we spoke publicly accessible website. at all three school boards visited, indicated that all All three of the school boards visited said they elementary teachers would benefit from training required or strongly encouraged their schools to 116 2013 Annual Report of the Office of the Auditor General of Ontario

use Ophea’s safety guidelines, but they had not success of its Daily Physical Activity Policy. We also provided training specifically on the guidelines to noted that the Ministry and school boards do not principals and vice-principals. Two of the three have information systems in place to gather data boards said they had provided specific training on that would show such progress. Consequently, the guidelines to some teachers. more than seven years after the policy was issued, The elementary schools we visited at the three the Ministry has not formally measured its success. boards could not determine how many teachers had Furthermore, a recent report by an expert panel to received training on the safety guidelines. While the Minister of Health and Long-Term Care identi- almost all of the elementary teachers we inter- fied that the policy does not appear to have had a viewed at two of the school boards were aware of significant impact on student activity levels. the guidelines, about half the elementary teachers In contrast to Ontario, California requires we interviewed at the third board were not. At all students in grades 5, 7 and 9 to have an annual three boards, none of the elementary teachers we physical fitness test. The results of these tests are interviewed said they had received training on the to be used to help students plan personal fitness guidelines, although the majority thought at least programs, assist teachers in developing physical some training would be beneficial. education programs, and provide parents with an understanding of their child’s fitness level and needs. As well, these results are used to monitor Measuring the Effectiveness of Physical changes in the physical fitness of California’s Activity Requirements students in general, and the aggregated results are The Ministry expects its Daily Physical Activity reported publicly. Policy to not only increase students’ physical activity but also contribute to decreased sedentary RECOMMENDATION 2 behaviour and improved student achievement. To help safely increase physical activity as well The Canadian Physical Activity Guidelines recom- as contribute to reduced sedentary behaviour mend 60 minutes of physical activity daily, well in and improved academic achievement, the Min-

Chapter 3 • VFM Section 3.03 excess of the Ministry’s elementary school policy istry of Education (Ministry) and school boards of 20 minutes per day during instruction time. should: Although students spend only part of their day at assess options, including practices in other school, the Ministry did not have a rationale for • jurisdictions, for providing sufficient physical why it set its requirement at just 20 minutes a day activity to both elementary and secondary for elementary students. Other provinces, such as school students; British Columbia and Alberta, expect elementary ensure that elementary school administra- students to get 30 minutes of physical activity each • tors and teachers receive sufficient training school day, which can be achieved during instruc- on good practices and on how to effectively tional time and non-instructional time, such as incorporate daily physical activity into the recess. In addition, the Healthy Kids Panel recently school day; recommended increasing the minimum amount of familiarize teachers with physical activity daily physical activity in elementary schools from • safety guidelines; and 20 to 30 minutes. set specific goals and targets for increasing We noted that the Ministry specified outcomes • physical activity in schools, and periodically for elementary students in very general terms and monitor, measure and publicly report on the had not established specific targets or measurable progress made. goals against which to measure and report on the Healthy Schools Strategy 117

MINISTRY RESPONSE RESPONSE OF SCHOOL BOARDS

The Ministry agrees that the education system All three school boards agreed with this recom- is uniquely positioned to make a significant mendation. One school board commented that contribution to increasing awareness of the continued work to ensure that daily physical importance of physical activity and in helping to activity is provided to students and monitored is lay the foundation for increased physical activ- needed and the board will work collaboratively ity among Ontario’s students. with the Ministry and others to explore best Recognizing the many benefits of physical practices to provide opportunities for and mon- activity, the Ministry is committed to full imple- itoring of daily physical activity and to promote mentation of the Daily Physical Activity Policy the importance of physical activity. and exploring options to increase physical Another school board indicated that it activity opportunities in secondary schools. Any strongly encouraged the recommendation to future decisions on revising physical activity assess options to increase secondary school requirements for elementary or secondary stu- students’ participation in physical activity and dents will need to be informed through research that increasing teacher training in policy imple- and outcome-based evidence. mentation and communicating safety guidelines The Ministry will do the following: to all teachers will increase teacher confidence • work on research on implementation of the to undertake physical activity in the classroom policy, including examining the extent to setting. However, this board cautioned that which it is being implemented in Ontario’s goals and targets for increasing physical activity elementary schools and identifying factors in schools should be well researched so that sug- associated with implementation; gested strategies will be evidence-based. • review and improve our support and mon- The remaining school board noted that it itoring plan in an effort to identify and share would be beneficial if daily physical activity was implementation challenges and effective a mandated component of teacher pre-service

practices of the policy and communicate programs and that it would be beneficial if the Chapter 3 • VFM Section 3.03 strategies to increase physical activity oppor- Ministry would issue an updated online train- tunities in secondary schools; ing module that informs educators of the key • work with school boards to establish a risk- learning points related to daily physical activity based approach to monitoring compliance and and safety guidelines to ensure consistent and reporting on implementation of the policy; current messaging is communicated throughout • update, as necessary, and further promote the province. The board also noted that creating existing ministry resources designed to assist these modules at the provincial level would be with implementation of the policy; both a cost-saving and time-saving measure. • work with our partners to develop initiatives to help increase interest and motivate Ontario’s students to lead active and healthy COMMUNICATION WITH PARENTS lives; and • continue to work with the Ontario Physical Parents play an important role in developing chil- and Health Education Association (Ophea) dren’s eating habits and in helping children learn to promote and provide free and open access to be active and stay active throughout their lives. to the Ontario Physical Education Safety The Healthy Kids Panel suggested developing a Guidelines. comprehensive social marketing program aimed 118 2013 Annual Report of the Office of the Auditor General of Ontario

at children, youth and parents to reinforce the RECOMMENDATION 3 importance of issues such as healthy eating and active living. To help encourage healthier eating and The Ministry said it does not generally engage increased physical activity among students, the in direct communication with parents but does Ministry of Education (Ministry) and school provide information on its website about healthy boards should further explore opportunities to eating and physical activity. Other ministry com- improve communication with parents and assess munication efforts include the distribution of a the effectiveness of such efforts. guide for parents concerning healthy schools that provides suggestions for physical activity and MINISTRY RESPONSE healthy eating. We noted that all three school The Ministry agrees that parents, as primary boards we visited also provided information to par- caregivers, are a critical audience to target infor- ents on their websites. Such information included mation to on the importance of healthy eating notification of events involving physical activity, a and physical activity to overall child and youth parent handbook, school board nutrition policies, development. information related to healthy eating and videos The Ministry will work with school boards focusing on healthy eating. Other school board and parents to identify appropriate and effective communications included providing parents with means to communicate information to parents information on the School Food and Beverage across the province on ministry requirements Policy and providing schools with materials to be and activities related to healthy eating and safe distributed to parents. physical activity. The Ministry will also work At the schools visited, we observed varying with other ministries and partners to support degrees of communication with parents. Such connections with parents and students through efforts included distributing information through the education system. newsletters, websites, parent meetings and direct conversations. However, many of the teachers to RESPONSE OF SCHOOL BOARDS

Chapter 3 • VFM Section 3.03 whom we spoke indicated that more outreach to parents to promote healthy eating was necessary All three school boards agreed with the recom- to help improve the eating habits of students. As mendation and were supportive of exploring well, representatives from the public health units to opportunities to improve communication with whom we spoke were generally of the opinion that parents. One board indicated that increased par- more communication with parents about healthy ental awareness of healthy habits should be very eating and physical activity was needed. beneficial to students and noted that while the We also noted that neither the Ministry nor the development and implementation of practices school boards visited had evaluated how effectively to monitor communication strategies could take they communicated with parents about healthy considerable time and effort, gains in student eating and physical activity. Nevertheless, one of health will justify the effort needed to encour- the school boards did gather some insight into age best practices in this area. Another board the effectiveness of its communication efforts by commented that parental involvement is a surveying stakeholders, including parents. For priority since parents have the primary respon- example, more than half the parents would like sibility for their children’s health and well-being additional information about the board’s nutrition and that school boards have a responsibility policy but did not want information on healthy to work with other stakeholders to promote nutrition and how to put it into practice at home. student health and well-being. The remaining Healthy Schools Strategy 119

of health-related initiatives. At the school boards board noted that it would be beneficial if the visited, we were advised that a number of schools government launched a communication strategy worked with public health units to promote student regarding healthy nutrition and physical activity health activities through school teams or commit- with messaging that reinforces adults as role tees of various school stakeholders. Such activities models to health and well-being and that such a included presentations on healthy eating, cooking strategy could include the distribution of infor- lessons to help students establish better eating mation to families through school boards via habits, and a campaign that encouraged students newsletters and websites. to put down their electronic devices and engage in physical activity. Gathering additional information on the success of such initiatives can help school CO-OPERATION WITH OTHER boards identify successful activities worth imple- MINISTRIES AND ORGANIZATIONS menting in other schools. In one specific example, just prior to the 2011 The goals of the Ministry’s policies on daily physical implementation deadline for the School Food and activity and food and beverages are to contribute to Beverage Policy, one board in co-operation with better student health and academic achievement. dietitians from public health conducted visits to Other ministries and organizations have comple- schools to gauge how well the board’s cafeteria mentary goals or activities, including the Ministry of vendor had begun to implement the policy. The Health and Long-Term Care, public health units and dietitians identified a significant number of items the Ministry of Children and Youth Services (MCYS). that did not meet nutrition requirements and The Ministry, school boards and schools often co- offered to undertake a subsequent review. However, operate with these and other ministries, organiza- the school board did not commit to any follow-up tions and stakeholder groups to develop policies and visits. In general, dietitians from public health units help implement healthy school initiatives. could be a resource to provide school boards and Although the school boards and schools we principals with the expertise needed to assess the visited work with many other organizations, the

nutrition of items sold to students to reinforce the Chapter 3 • VFM Section 3.03 activities of these organizations did not formally value of healthier eating habits. factor into the school boards’ strategies to promote MCYS provides partial funding for student nutri- healthier eating or increased physical activity tion programs in many of Ontario’s schools. The goal among students. As well, the Ministry and the of these programs is to support the healthy growth school boards we visited generally did not have and development of students by providing them information about the contribution of other organ- with generally free meals and snacks so that they are izations to the development of healthier eating ready to learn. According to MCYS, when children habits by students or increasing their physical activ- and youth arrive at school hungry, their capacity to ity at the school level. Nevertheless, we observed learn is diminished. Many of the schools at the three programs and supports in place at the schools we boards we visited had student nutrition programs. visited that could help the Ministry and school According to MCYS, almost 700,000 students prov- boards achieve their goals concerning better stu- ince-wide participated in such nutrition programs in dent health and academic achievement. the 2011/12 school year. These nutrition programs For example, Ontario’s local boards of health, can promote healthy eating by students, particularly through the public health units they govern, have at the elementary level where there may be fewer a mandate to work with school boards and schools opportunities for the Ministry’s School Food and to influence the development and implementation Beverage Policy to affect student behaviours. 120 2013 Annual Report of the Office of the Auditor General of Ontario

The Ministry of Education’s School Food and government, the broader public sector and not- Beverage Policy does not apply to food and bever- for-profit organizations. ages that are provided to students free of charge, and The Ministry will continue to work with the providers of these products and other nutrition other ministries and partners to help encour- program providers were not required to follow age healthier eating and physical activity. For MCYS nutrition guidelines. Although all three school instance, the Ministry will do the following: boards that we visited recommended or required support implementation of Ontario’s Healthy that their schools follow MCYS nutrition guidelines, • Kids Strategy; neither the school boards nor the schools we visited continue to consult with education- and monitored these programs for compliance with the • school-based health organizations on mat- guidelines. In addition, at the three school boards ters related to healthy schools; visited, only half of the organizations that work examine ways to build strong, collaborative with MCYS to provide nutrition programs visited • and sustainable partnerships between school schools to monitor compliance with these nutrition boards and public health units; programs. A recent report by the Healthy Kids Panel highlight the important role that public recommended developing a single standard for food • health and other community agencies and and beverages served or sold in schools, as other regional networks can play in supporting provinces have done. The school nutrition policies in implementation of ministry policies through provinces such as Nova Scotia and New Brunswick its support and monitoring plan; and apply equally to food sold to students and food pro- continue to meet with representatives from vided to them free of charge. • other Canadian jurisdictions to discuss and share practices intended to improve the over- RECOMMENDATION 4 all health of young people. The Ministry of Education (Ministry) and school boards should work more effectively with other RESPONSE OF SCHOOL BOARDS relevant organizations with similar goals to bet-

Chapter 3 • VFM Section 3.03 All three school boards agreed with the rec- ter integrate and leverage their activities to help ommendation. One board commented that encourage healthier eating and physical activity a more co-ordinated effort among all those among students. invested in children’s health would definitely be helpful and that better sharing of informa- MINISTRY RESPONSE tion among those involved in children’s health The Ministry agrees that given the numerous could greatly increase the effectiveness of each other ministries, levels of government, not-for- group’s efforts. Another board commented that profit organizations and other groups involved effective partnerships, consulting, co-planning in activities designed to encourage healthy and co-funding with other relevant organiza- eating and physical activity among children tions with similar goals can all help encourage and youth, it is important to mobilize collective healthier eating and physical activity among knowledge, resources and efforts. students. The remaining board commented that Throughout the development and imple- following significant work in the past few years mentation phases of its policies related to to establish working relationships between the healthy eating and physical activity, the Ministry board and community organizations, the board has relied on input and expertise from within planned to expand its collaborative work to focus on overall student well-being. Chapter 3 Ministry of Health and Long-Term Care Section 3.04 Land Ambulance Services

830 ambulances and an additional 300 emergency Background response vehicles (which have a paramedic but cannot transport patients). The Ministry of Health and Long-Term Care RESPONSIBILITIES (Ministry) oversees ambulance services based on The provision of land ambulance services in requirements set out in the Act. The Ministry’s Ontario is governed by the Ambulance Act (Act). responsibilities include setting patient-care and Under the Act, the Minister of Health and Long- ambulance equipment standards, monitoring and Term Care must ensure “the existence throughout ensuring compliance with those standards, and, Ontario of a balanced and integrated system of through service reviews, certifying ambulance ambulance services and communication services service providers to operate in Ontario. The Min- used in dispatching ambulances.” The Act further istry’s land ambulance functions employ about 560 states that every municipality will “be responsible full-time equivalent staff, most of whom work at Chapter 3 • VFM Section 3.04 for ensuring the proper provision of land ambu- Ministry-run dispatch centres. lance services in the municipality in accordance with the needs of persons in the municipality.” DISPATCH CENTRES AND BASE Accordingly, 42 municipalities and eight other HOSPITALS designated delivery agents, primarily in remote areas (collectively referred to in this report as Twenty-two dispatch centres are responsible for municipalities) are responsible for providing land dispatching Ontario’s land ambulances. Of these, ambulance services in Ontario. Most municipalities 11 are run by the Ministry, six by hospitals, four by provide the services directly, although about 15% municipalities and one by a private operator. Seven have chosen to contract with a third-party provider. base hospitals (each of which comprises a group Two types of paramedics generally provide land of doctors working out of an established hospital) ambulance services—primary care paramedics provide medical oversight to paramedics—includ- (who perform basic and some advanced life sup- ing any required advice on pre-hospital patient port procedures) and advanced care paramedics care, as well as continuing education. Since 2008, (who perform basic and all advanced life support the number of calls requesting an ambulance, the procedures). In total, municipalities have about number of ambulances dispatched and the number

121 122 2013 Annual Report of the Office of the Auditor General of Ontario

of patients transported have been gradually increas- load nurse program, in which hospital nurses take ing, as shown in Figure 1. In 2012, about 1.3 million responsibility for ambulance patients in order to ambulances were dispatched and about 970,000 reduce ambulance delays at busy hospitals. patients were transported in Ontario, an increase of There is a glossary of terms at the end of this about 15% for both since 2008. report.

FUNDING

Over the last few years, the Ministry has funded Audit Objective and Scope about 50% of each municipality’s prior-year costs for municipal land ambulance services, plus an Our audit objective was to assess whether the Min- increase for inflation. The Ministry funds 100% istry has procedures in place to ensure that munici- of the cost of land ambulance services for the 10 pal land ambulance services are meeting Ontarians’ First Nations ambulance services and for certain transportation health-care needs in a cost-effective other (primarily remote) areas. The Ministry also manner and are in compliance with ministry and funds 100% of the Ministry-approved costs of legislative requirements. Senior management at the ambulance dispatch centres and base hospitals. Ministry reviewed and agreed to our objective and For the 2011/12 fiscal year, total land ambulance associated audit criteria. costs were an estimated $1.1 billion, which includes Our audit work was primarily conducted at the $627 million of ministry funding (as shown in Ministry’s Emergency Health Services Branch. We Figure 2) and $477 million of municipal funding. also visited three municipal ambulance services— Ministry funding includes $12 million for the off- Toronto Emergency Medical Services (run by the City of Toronto), the Superior North Emergency Figure 1: Number of Calls Received, Ambulances Medical Service (run by the City of Thunder Bay), Dispatched to Patients,1, 2 and Patients Transported, 2008–2012 Figure 2: Estimated Ministry and Municipal Source of data: Ministry of Health and Long-Term Care Chapter 3 • VFM Section 3.04 Expenditures on Land Ambulance Services, 2011/12 1,400,000 ($ million) Source of data: Ministry of Health and Long-Term Care 1,200,000 Dispatch [Ministry-funded] ($103) 1,000,000 Ministry administration 800,000 [Ministry-funded] ($21)

600,000 Number of calls received Base hospitals 400,000 Number of ambulances [Ministry-funded] dispatched to patients ($15) 200,000 Number of patients transported Other 0 [Ministry-funded] 2008 2009 2010 2011 2012 ($15) 1. Although not included in the number of ambulances dispatched to patients, dispatch workload also involves dispatching emergency response vehicles (which occurred 116,000 times in 2012) and repositioning ambulances— Municipal land for example, to be closer to the next anticipated call (which happened ambulance services 620,000 times in 2012). [municipally funded] 2. While an ambulance is generally dispatched for each call received, in ($477) some cases (such as when there is a highway traffic accident), many more calls are received than ambulances dispatched. In other cases (such as Municipal land ambulance services when the closest ambulance is dispatched, as well as when the closest [Ministry-funded, including ambulance with an advanced care paramedic is dispatched), more ambulances are dispatched than calls received. offload nurse program] ($473) Land Ambulance Services 123 and Essex-Windsor Emergency Medical Services services and thereby increasing the grant from (run by the County of Essex)—to examine certain $241 million to $401 million to meet that com- accounting records relating to ministry grants paid mitment. Since 2008/09, annual increases have to municipalities, as well as to obtain their perspec- averaged 6%. Overall, while the cost to fund land tive on the delivery of land ambulance services ambulance services almost doubled, the number in Ontario. In addition, we visited two dispatch of patients transported in that same time frame centres—one operated by the Ministry and one increased by only 18%. The Ministry does not know operated by a municipality—and a base hospital whether the additional funding has resulted in to obtain further information on their policies and better value for money in terms of service levels practices regarding land ambulance services. We and patient outcomes. Ministry data indicated also spoke with representatives from the Ontario that since 2005 there has been some improvement Association of Paramedic Chiefs (a not-for-profit in ambulance response times, but in the 2012 organization consisting of senior management from calendar year still only about 60% of the 50 munici- 46 municipalities and nine contracted ambulance palities responded to 90% of their emergency calls service providers that provides advice to the Min- within 15 minutes, as shown in Figure 5. Further- istry regarding emergency medical services) and more, there is no patient-centred measure of the from the Ontario Hospital Association to obtain time from receipt of an ambulance call to the time their perspectives on land ambulance services, an ambulance arrives at a patient’s location. as well as with representatives from the Ontario The Ministry’s funding formula provides more Stroke Network and the Cardiac Care Network of funding to land ambulance services that spend Ontario to learn more about ambulance transporta- more, regardless of the level of service they actually tion of stroke and cardiac patients, respectively. provide. The Ministry indicated that varying service In conducting our audit, we reviewed relevant levels were expected and that it had not compared documents and administrative policies and the funding provided to each municipality with the procedures; analyzed information; interviewed ambulance services provided. Further, the Ministry appropriate staff from the Ministry, municipal land had not determined the reasons for variations in

ambulance services, base hospitals, and dispatch funding, which could result from differences in the Chapter 3 • VFM Section 3.04 centres; and reviewed relevant research from distances ambulances travel for patients (urban Ontario and other jurisdictions. In addition, we versus rural), efficiency of ambulance operations, asked the Ministry to run a number of computer or municipal priorities and tax-based funding. reports to assist in our analysis of dispatch proto- Data from the Ontario Municipal Benchmark- cols. We also engaged two independent consulting Initiative representing some municipalities ants, each of whom has expert knowledge of land indicated that the 2012 total cost per hour of land ambulance services, to advise us. ambulance services among 13 reporting municipalities ranged from a low of $156 to a high of $247, with significant cost variations even among urban municipalities. Summary The Ministry has not tracked or reviewed any patient outcome information, such as the survival Ministry funding to municipalities almost doubled rates for people with cardiac arrest or stroke who between the 2004/05 and 2011/12 fiscal years, were transported to hospital, either overall or by with the largest increases between 2004/05 and ambulance service. This type of information could 2008/09 reflecting the government’s commitment be used to improve ambulance services. There have to provide 50% of the cost of land ambulance been some improvements to parts of the quality 124 2013 Annual Report of the Office of the Auditor General of Ontario

assurance processes since our audit in 2005, such when the ambulance is dispatched to pick as more timely service review follow-ups. However, up a patient. Instead, they will report on more work is needed to ensure that dispatch staff how quickly patients are reached based on are consistently evaluated and that their workloads paramedics’ assessment of each patient when remain reasonable in order to prevent errors in the the ambulance arrives at the scene. The dispatch process. Ministry had not analyzed the inherent dif- In addition, we noted the following other areas ficulties in using this retrospective approach where action is required. to measure ambulance response times. We found no other jurisdiction that used a similar approach. Other jurisdictions generally meas- Ambulance Service Response Times ure response time based on a call’s assessed Although the Ministry has recently set more • urgency at the time of dispatch. meaningful response-time measures for the most time-sensitive patients (such as those who are choking or experiencing cardiac Dispatch arrest), it has not set standard response-time • In 2012, 20 of the 22 dispatch centres tracked targets for other urgent patients, such as their time to dispatch emergency calls. None stroke patients or most heart attack patients. of them dispatched 90% of emergency calls Each municipality sets its own response-time within two minutes, as required by ministry targets for transporting these patients, and policy. However, all dispatched 90% of these the targets vary significantly based on fac- calls within three and a half minutes. Even tors such as geographic distances and the though dispatch is legislatively a ministry amount of tax-based funding available to responsibility and half of the dispatch centres municipalities. are Ministry-run, starting in 2013, each dis- • The Ministry needs to ensure that response patch centre can choose what percentage of times are reported by municipalities in a high-priority calls it needs to dispatch within consistent and comparable manner, factoring two minutes. We noted that the chosen per- Chapter 3 • VFM Section 3.04 in geographic differences, so that users can centages ranged from a low of 70% to a high meaningfully compare their municipality’s of 90%, which may affect response times for performance with others. In 2006, the Stand- urgent patients. ing Committee on Public Accounts (PAC) • Dispatch protocols are generally designed to recommended that response-time targets be over-prioritize calls when there is uncertainty similar for similar communities. Although about a patient’s condition. Only about recommended by the Ministry’s Response 25% of patients actually require an urgent Time Standard Working Group in 2006, defin- response, but about two-thirds of calls are pri- itions to distinguish target response times for oritized at the most urgent code, requiring the different geographic areas have not yet been fastest response. The municipalities we spoke developed. with indicated that over-prioritizing this many • The Ministry expects to start publicly calls can leave few or no ambulances available reporting municipal land ambulance response to respond to new calls that are truly urgent, times in 2014. Under the Ministry’s new thereby causing delays. The two dispatch approach for measuring ambulance response centres that use a different type of dispatch times, municipalities will no longer report system experienced less over-prioritization. times based on the call’s assessed urgency Land Ambulance Services 125

• The Ministry has not assessed whether the • Ministry policy requires that all patients be current number of dispatch centres is optimal transported by an ambulance responding to for performance. Centralized dispatch may a call unless the patient signs a form refusing help to contain costs and ensure that the clos- transport. The Ministry has not assessed using est ambulance responds to a call. emergency department diversion strategies • Only one dispatch centre is able to provide to reduce ambulance wait times at emergency callers with the locations of publicly access- departments and free them up to respond ible automated external defibrillators (AEDs), to new calls more quickly. Strategies similar which can significantly improve survival to those used successfully in other jurisdic- rates for cardiac arrest patients if available tions include referring low-risk patients to within minutes. The other dispatch centres Telehealth Ontario to obtain a nurse’s advice are not able to tell callers whether there is an or having paramedics treat low-risk patients AED nearby. It may therefore take the caller at the scene without transporting them. additional time to locate an AED and could Notwithstanding this, we noted that in 2012 increase the risk to the patient. over 25% (or about 350,000) of ambulances • Non-ambulance emergency response vehicles, dispatched did not transport a patient. The which cannot transport patients and which Ministry has not assessed the underlying require that an ambulance also be dispatched, reasons to determine, for example, how many account for about 25% of the municipal of these situations arose from patient refusals, ambulance fleet, yet such vehicles responded calls cancelled before arrival of an ambulance to only 10% of calls. These vehicles are about or paramedics having successfully treated 50% Ministry-funded, and the municipalities patients at the scene. we visited indicated that they were often used for administrative purposes rather than being Patient Transfer at Hospital deployed for ambulance calls. A portion of The Ministry started funding an offload nurse provincial funding could potentially be better • program in 2008 as a temporary measure to

directed to serving callers. Chapter 3 • VFM Section 3.04 reduce the time ambulances spend waiting at hospitals for patients to be accepted. It Patient Transport to Hospital has not evaluated this program’s ongoing • The Ministry has no provincial protocol to effectiveness or analyzed whether there are enable consistent identification of certain more cost-effective ways to reduce offload heart attack patients (called “STEMI” patients, delays. Between the 2008/09 and 2012/13 which stands for ST-segment elevation fiscal years, ministry funding for this program myocardial infarction—a type of heart attack totalled $40 million. We found that since this resulting from a blocked artery). Outcomes program was implemented, ambulance wait- for STEMI patients can be greatly improved ing time has actually increased at 20% of the if they are transported in time to specialized hospitals funded. care centres. A June 2013 survey conducted • Ministry data indicated that offload wait times by the Cardiac Care Network of Ontario indi- of more than 30 minutes occurred for about cated that not all ambulances had both the 80% of the ambulances transporting the most appropriate ECG equipment and paramedics urgent patients, but the Ministry generally did trained to read the test results to identify not know whether this was due to the hospital STEMI patients and thereby help ensure not accepting the patient or other reasons, timely treatment with better outcomes. such as time spent cleaning and restocking the 126 2013 Annual Report of the Office of the Auditor General of Ontario

ambulance. Hospitals in only one municipal- response time (that is, the time from call receipt ity in the province inform that municipality’s until a dispatcher advises an ambulance service to dispatch centre when a patient is accepted. send an ambulance) and the ambulance response Therefore this is the only municipality able to time (that is, the time from when the dispatcher determine the time it takes hospitals to accept notifies the ambulance service until the ambulance a patient once an ambulance arrives. arrives at the patient’s location). This approach • Paramedics orally brief emergency depart- enables the Ministry, which has legislative control ment staff about the patient. Patient records over dispatch, to monitor dispatch response times, generally cannot be electronically shared and the municipalities, which control ambulance because the electronic patient-care records service provision, to monitor ambulance response introduced by most municipal land ambulance times. services over the last few years are not com- Most 911 requests for land ambulances are patible with hospital systems. In some cases, transferred to the local dispatch centre nearest patient-care records are not received by emer- the caller. Twenty of the 22 dispatch centres pri- gency departments until days later and some oritize calls using a dispatch protocol, which was test results are not received at all, which could developed by the Ministry with input from phys- affect time-sensitive treatment decisions. icians, called the Dispatch Priority Card Index II (DPCI II), as described in Figure 3 column A. The other two dispatch centres prioritize calls with Quality Assurance Over Patient Care the internationally used Medical Priority Dispatch The Ministry has assigned responsibility for • System (MPDS) codes, as described in Figure 3 col- oversight of the vast majority of paramedic umn B. DPCI II Code 4, as well as MPDS Codes Echo patient-care activities (referred to as “basic life and Delta, are all considered emergencies, and support activities” and including management ambulances are sent out to such calls generally with of chest pain, childbirth and hip fractures) lights and sirens. Upon arrival at the patient’s loca- to municipal land ambulance services. Base- tion, paramedics assess how urgently the patient hospital physicians, who are responsible for

Chapter 3 • VFM Section 3.04 requires care using the same scale used in emer- reviewing paramedics’ performance of more gency departments: the Canadian Triage and Acuity complex or risky medical procedures, told us Scale (CTAS), as described in Figure 3 column C. that municipal land ambulance services may Until December 2012, ministry policy required not have the expertise to provide proper med- both dispatch and ambulance response times ical oversight of basic life support activities to be tracked for all emergency calls. In 2012, performed by paramedics. almost 710,000 (60%) of ambulances dispatched to patients were for calls classified as emergencies (that is, DPCI Code 4 or MPDS Codes Echo Detailed Audit Observations or Delta). Changes to a regulation under the Ambulance Act that took effect in January 2013 require tracking of specific dispatch and ambulance RESPONSE PRIORITIZATION AND TIME response-time measures for only those patients Many jurisdictions measure overall ambulance whose conditions are classified as CTAS 1 by the response time—that is, from when a dispatch cen- paramedics who arrive on the scene. The Ministry tre receives a call to when the ambulance arrives indicated that this new requirement was based on at the patient’s location. In Ontario, two separate recommendations made in 2006 by the Response response-time measures are used: the dispatch Time Standard Working Group, which consisted of Land Ambulance Services 127

Description Resuscitation Examples: cardiac* and/or respiratory arrest; major trauma (severe injury or burns); unconsciousness Emergent Examples: chest pain with cardiac features; stroke; serious infections Urgent Examples: moderate trauma (fractures, dislocations); moderate asthma Less urgent Examples: constipation with mild pain; chronic back pain; earache Non-urgent Examples: minor trauma (sprains, minor lacerations); request for medication or dressing change; sore throat 1 2 3 4 5 and Acuity Scale (CTAS) C. Canadian Triage paramedics Used by when ambulance reaches patient’s location CTAS Level Chapter 3 • VFM Section 3.04 Description Life-threatening calls, usually identified early in call; excludes calls with scene safety concerns, such as gunshot to chest Examples: cardiac arrest*; choking; severe asthma Other life-threatening calls Examples: major trauma (severe injury); unconsciousness; gunshot to chest Calls involving, for example, chest pain with cardiac features or stroke, as well conditions by advanced- benefiting from assessment care paramedic, such as a person requiring glucose; irregularintravenous heart rhythms requiring an ECG test Calls involving, for example, moderate trauma (fractures, dislocations) or calls where patient condition is unknown (e.g., caller unable to answer questions) Less-serious calls from the community Examples: constipation with mild pain; chronic back pain; minor trauma (minor lacerations, sprain) Calls primarily from long-term-care homes; patient condition less serious No ambulance transportation required; caller connected with appropriate assistance (e.g., Telehealth) B. Medical Priority Dispatch System (MPDS) 2 of Ontario’sUsed by prioritize 22 dispatch centres to calls received Code Echo Delta Charlie Bravo Alpha Alpha 1, 2, and 3 Omega life- or limb- threatening; unscheduled inter-facility transfer or transfers between health-care all other calls

Description Urgent/Emergency: person requires immediate attention Examples: cardiac arrest*; stroke; major or moderate trauma (severe injuries or fractures); major and moderate asthma Prompt: Scheduled: facilities (e.g., for diagnostic tests or treatment); not detrimental to patient safety delay Deferrable: low-risk lift assists (e.g., person has fallen and is not unable to get up, but has no injuries); delay detrimental to patient safety 4 3 2 1 A. Dispatch Priority Card (DPCI) II Index 20 of Ontario’sUsed by prioritize 22 dispatch centres to calls received Code * In cardiac arrest, the heart stops beating. In a heart attack, part of the heart a blocked artery. muscle is damaged due to lack of oxygen caused by Figure 3: Three Ways of Prioritizing Patient Conditions When an Ambulance Is Needed, from of Prioritizing Patient Most toFigure 3: Three Ways Least Urgent Source of data: Emergency Ministry Medical Services, and Canadian Association of Emergency Physicians Care, Toronto of Health and Long-Term 128 2013 Annual Report of the Office of the Auditor General of Ontario

ministry, municipal and physician representatives patient, to be experiencing either sudden cardiac and reported to the Land Ambulance Committee, arrest or any other CTAS 1 condition. That is, which was co-chaired by the Ministry and the after the paramedic reaches the patient, he or she Association of Municipalities of Ontario (AMO). assesses whether or not the call should have been Ministry policy still requires those dispatch centres dispatched within two minutes. As a result, the that use DPCI II to track their response times for all determination of which calls were required to be emergency calls. dispatched within the two-minute standard occurs Studies have indicated that a one- or two- only after the paramedic reaches the patient, rather minute delay in the arrival of emergency medical than at the time of dispatch. personnel can significantly affect the survival of Even though dispatch is legislatively a ministry CTAS 1 patients. In 2012, fewer than 12,000 calls, responsibility, and half of Ontario’s 22 dispatch or less than 1% of total calls, involved CTAS 1 centres are Ministry-run, each dispatch centre can patients. Therefore, under the new regulation, choose its own target for the percentage of calls to tracking of specific response times is required only be dispatched within two minutes. These targets for this relatively small number of ambulance do not require ministry approval. For the 2013 cal- calls. Figure 4 shows both the pre- and post-2013 endar year, we noted that the targeted compliance response-time standards. rate ranged from a low at two dispatch centres of 70% of emergency calls dispatched within two minutes to a high at seven dispatch centres of 90%. Measuring Dispatch Response Times

In 2012, 20 of the 22 dispatch centres tracked the Measuring Municipal Ambulance Service time it took them to respond to emergency calls. Response Times None of them dispatched 90% of emergency calls within two minutes as required by ministry policy. As noted earlier, ambulance response times are However, all dispatched 90% of these calls within measured separately from dispatch response three and a half minutes. times. Ministry data indicated that since 2005,

Chapter 3 • VFM Section 3.04 As required by changes to a regulation under the there has been some improvement in ambulance Ambulance Act, each dispatch centre was to estab- response times, but in the 2012 calendar year, lish by October 2012 a target for the percentage still only about 60% of the 50 municipalities of calls to be dispatched within two minutes when responded to 90% of their emergency calls within those calls involve a patient who is determined by 15 minutes, as shown in Figure 5. the paramedic, when the ambulance reaches the

Figure 4: Response-time Standards, Pre- and Post-January 2013 Source of data: Ministry of Health and Long-Term Care and Ambulance Act

Dispatch Ambulance Service Pre-2013 90% of Code 4 calls within 2 minutes, 90% of Code 4 calls within the actual time it took to respond to per ministry policy 90% of Code 4 calls in 1996 Commencing Dispatch centres measure whether Municipality measures the percentage of cardiac arrest patients January 2013 90% of Code 4 calls are dispatched reached with an automated external defibrillator within 6 minutes within 2 minutes, per ministry policy. and the percentage of CTAS 1 patients reached by paramedics within 8 minutes, per legislation. Dispatch centres measure percentage Each municipality determines its own response-time standard for of CTAS 1 calls to be dispatched in CTAS 2 to CTAS 5 calls, and also sets a target for the percentage of 2 minutes, per legislation. calls that it aims to reach within this time standard, per legislation. Land Ambulance Services 129

the scene within eight minutes of the ambu- Figure 5: Percentage of Municipalities Responding to lance service being notified by the dispatcher. 90% of Code 4 Calls* within 15 Minutes, 2005–2012 For patients at other CTAS levels (as determined Source of data: Ministry of Health and Long-Term Care • by the paramedic when the ambulance reaches 60 the patient)—how often an ambulance 50 reaches the patient, after being notified by the dispatcher, within a time established by the 40 municipality. For these patients, the munici- 30 pality sets both the response-time targets to be met and the target rate of compliance, which 20 can reflect, among other things, differences

10 in geographic distances and funding available to municipalities through taxes. As shown in 0 Figure 6, this approach has resulted in a wide 2005 2006 2007 2008 2009 2010 2011 2012 variety of response-time targets and target * Includes calls from MPDS dispatch centres, based on Ministry’s determination of which MPDS calls are equivalent to DPCI II Code 4. compliance rates for the same CTAS code: one municipality may aim to have an ambulance In our Annual Reports for 2000 and 2005, reach 60% of CTAS 2 patients within 10 we recommended that the Ministry, together minutes, whereas another might aim to reach with municipalities, review the response-time 90% within 10 minutes. requirement then in use for reasonableness and Municipalities submit their chosen targets in consistency. As well, in 2006, the Standing Commit- their annual response-time performance plans to tee on Public Accounts (PAC) recommended that the Ministry, but may change the plans at any time the Ministry report to PAC on how it would update by notifying the Ministry. The targets do not require response-time targets. Ministry approval, including those for CTAS 2 Until January 2013, the legislated standards patients who require an emergency response. required land ambulance services to respond to

The Ministry allows municipalities to use their Chapter 3 • VFM Section 3.04 90% of emergency calls within the actual times own definition of urban versus rural areas in their it took to respond to 90% of Code 4 calls in 1996. response-time performance plans. Although many The legislation was subsequently changed based Ontario municipalities include rural and urban on recommendations the Response Time Standard areas, only three such municipalities have provided Working Group made in 2006. separate targeted compliance rates for these areas. Legislation that came into effect as of January Our review of the municipally established targeted 2013 requires each municipality to establish a tar- compliance rates for cardiac arrest and CTAS 1 get rate of compliance for the following response- patients indicated that they varied significantly, as time measures: shown in Figure 7. The lower targets were gener- For all patients with sudden cardiac arrest— • ally set by municipalities in rural areas, due to the how often any individual equipped with a longer anticipated time to reach a patient. defibrillator reaches the scene within six min- As with dispatch, the determination of which utes of the ambulance service being notified calls are required to be responded to within the by the dispatcher. established time frames occurs when the paramedic For all CTAS 1 patients (as determined by the • reaches the patient, rather than at the time the paramedic when the ambulance reaches the ambulance service is notified. patient)—how often an ambulance reaches 130 2013 Annual Report of the Office of the Auditor General of Ontario

Figure 6: Municipalities’ Chosen Response-time Compliance Rate Targets for CTAS 2 to CTAS 5 Patients, 2013 Source of data: Ministry of Health and Long-Term Care

Chosen Compliance Rate Chosen Ambulance # of Municipalities (% of time chosen response-time Response-time Choosing Targets target will be met) Target* (minutes) in this Range Lowest Highest CTAS 2 8–15 43 50 90 16–30 7 50 90 >30 0 — — CTAS 3 8–15 41 50 90 16–30 9 50 90 >30 0 — — CTAS 4 8–15 20 50 90 16–30 29 55 100 >30 1 50 50 CTAS 5 8–15 18 50 95 16–30 29 55 100 >30 3 50 75

* Chart includes the urban response-time target for the three municipalities that set both urban and rural targets.

We noted that other jurisdictions measure suburban areas and a 30-minute standard for rural ambulance response times based on urgency of areas. the call at the time of dispatch, an approach that The 2006 report from the Response Time enables the ambulance’s response to be adjusted Standard Working Group recommended three geo- based on the information available at that time. graphic levels for reporting response times: urban, We could find no other jurisdictions that evaluate rural/light suburban, and remote. In 2006, PAC Chapter 3 • VFM Section 3.04 the timeliness of ambulance response based on the also recommended that response times be similar assessment made by the paramedics after reaching for similar communities considering, for example, the patient. The Ministry has not evaluated the local factors such as urban population densities and practical difficulties inherent in this retrospective road infrastructure. As well, both a 2010 Ministry- approach. Subsequent to our fieldwork, the Min- commissioned report and a 2011 report released by istry indicated that it has begun discussions with the Ontario Association of Paramedic Chiefs recom- the municipalities on how best to track and report mended that municipalities establish definitions for response times using the retrospective approach. urban, rural, and remote service areas. However, as of May 2013, no such standardized definitions for measuring response times had been adopted by Differentiating Rural and Urban Response municipalities or the Ministry. Times

Generally, rural areas will have longer ambulance Transporting Patients to Specialized Care response times than urban areas, because longer distances must typically be travelled to reach patients. Adopting Transportation Protocols We noted that in Nova Scotia, response times are To obtain the best outcomes, patients with certain measured using a nine-minute response-time conditions, such as stroke and a certain type of standard for urban areas, a 15-minute standard for Land Ambulance Services 131

Figure 7: Targeted Response-time Compliance Rates by 50 Municipalities for Two Emergency Standards, 2013 Source of data: Ambulance Act and municipal response-time performance plans

% of Time Standard Will be Met Lowest Highest Standards Target Rate (%) Target Rate (%) Defibrillator arrives within 6 minutes of dispatch for patients in sudden cardiac arrest 1, 2 15 75 Ambulance arrives within 8 minutes of dispatch for CTAS 1 patients, including sudden 9 85 cardiac arrest patients 3

1. The legislation specifies only the arrival of a first responder—not necessarily a land ambulance—with a defibrillator within six minutes. Therefore, some municipalities set two targets: one for the first responder and a second for the ambulance. For municipalities with two targets, the first-responder target is shown. 2. Data excludes four municipalities that had not set a target for a defibrillator arriving within six minutes. 3. Data excludes one municipality that had not set a target for arriving in eight minutes for CTAS 1 patients. heart attack (referred to as STEMI, which stands for Ontario, does not provide electronic prompts to ST-segment elevation myocardial infarction—the ensure that such municipally adopted protocols are technical term describing one type of heart attack considered when appropriate, and therefore there resulting from a blocked artery, so called because is risk that dispatchers will not direct the ambu- of the way it looks on an electrocardiogram or ECG lance to the appropriate hospital. test), should be transported within specific time frames to specialized hospitals that have the neces- Transporting Patients to STEMI Centres sary medical equipment and professionals trained For STEMI, the recommended maximum time for to provide the required care. The same level of care transporting the person to a hospital is generally is not readily available at other hospitals. Munici- 60 minutes from when the ambulance reaches the palities can choose whether or not to participate in patient, plus an additional 30 minutes after reach- a protocol to transport patients with these condi- ing the hospital for the patient to receive treatment. tions to a specialized hospital, which may be within The Ministry indicated that the Branch’s Medical or outside their municipality. The Ontario Stroke

Advisory Committee is working toward a standard- Chapter 3 • VFM Section 3.04 Network indicated that all municipal ambulance ized STEMI protocol. At the time of our audit, the service providers have adopted the Ministry’s 2011 Ministry had not confirmed an implementation Ontario Acute Stroke Protocol, which includes date. instructions on transporting certain stroke patients As part of their responsibility to assess the to stroke centres. Although the Ministry receives quality of care provided by ambulance services, some information on which municipalities have base hospitals have performed some monitoring adopted a STEMI protocol, it does not assess why of compliance with specialized care protocols. For other municipalities choose not to adopt one. example, a base hospital in the Greater Toronto For the municipalities that have adopted proto- Area found that during one month in 2012, 53% cols, the Ministry does not assess whether patients of STEMI patients did not receive treatment within are transported to specialized care centres when 90 minutes from when the ambulance reached appropriate or within the recommended time frame the patient. However, there is no provincial policy for the patient’s condition. The municipalities to ensure consistent and appropriate paramedic we visited indicated that they review this as part STEMI care. A June 2013 Cardiac Care Network of of their quality assurance reviews of paramedic Ontario report indicated that STEMI patients were performance. Furthermore, the Ministry’s DPCI II often not being transported to appropriate special- dispatch system, used by most dispatch centres in ized care centres, partly because of the lack of a 132 2013 Annual Report of the Office of the Auditor General of Ontario

provincial protocol. For example, three of the 35 Also, starting with the 2013 calendar year, ambulance services surveyed by the Cardiac Care municipalities are responsible for calculating and Network indicated that not all of their ambulances reporting to the Ministry their compliance with had the appropriate ECG equipment, and two ambulance response-time standards using either others said that none of their ambulances did. This information in the Ministry’s dispatch system, infor- ECG equipment is necessary for identifying STEMI mation in their own electronic patient records, or a patients; a timely ECG test also ensures timely combination of both. The municipalities we spoke treatment with better patient outcomes. Further, to indicated a number of concerns about using the three ambulance services with appropriate ECG Ministry’s dispatch data, including the following: equipment indicated that none of their paramed- • Although preliminary data is available after ics had been trained in how to interpret the ECG 48 hours, the Ministry takes about four tests to identify STEMI patients. The Cardiac Care months to finalize its data, making it difficult Network further noted that because of the lack of for municipalities to perform timely reviews of a co-ordinated system for identifying and treating their ongoing performance. STEMI patients, it could not estimate what propor- • The Ministry does not have standardized tion were reaching specialized care centres. reports to assist municipalities in determin- We noted that in Nova Scotia, paramedics are ing their compliance with the new legislative trained to administer drugs when treating STEMI standards. patients who are too far from specialized care cen- The Ministry informed us that it would be tres. This practice is not done in Ontario, although developing standardized reports and expected to one base hospital told us that this practice would have them available for municipalities to use by likely benefit patients in remote locations. No esti- fall 2013. mate has been made of the number of patients in The Ministry plans to post, in spring 2014, the Ontario who cannot reasonably be transported to a achieved 2013 response-time compliance rates as specialized care centre due to their remote location. reported to it by the 22 dispatch centres and the 50 municipal land ambulance services. However,

Chapter 3 • VFM Section 3.04 the Ministry has no plans to publicly report the Public Reporting percentage of calls dispatched as emergencies In our 2005 Annual Report, we recommended that to which dispatch centres responded within two the Ministry and municipalities “jointly establish minutes. Although the Ministry has most of the pertinent performance measures such as response data it needs to confirm the accuracy of municipally times and report publicly and regularly on these reported information, the Ministry indicated that land ambulance service performance measures.” it had no plans to do so because this is a municipal The 2006 PAC report on land ambulance services responsibility. also recommended that the Ministry report publicly We also noted that the Ministry obtained data on response times. on each ambulance call received, each ambulance Before 2013, no response-time information was dispatched and each patient transported. At our publicly reported by the Ministry, and only some request, the Ministry accumulated this data by municipalities publicly reported such information year, as shown in Figure 1. However, the Ministry on their websites. Beginning in 2013, the Ministry does not regularly review this fundamental data for posted on its website the response-time perform- trends, nor is the data publicly reported. ance plans for each of the 22 dispatch centres and We further noted that other jurisdictions report the 50 municipal land ambulance services. publicly on ambulance performance, including response-time reporting by rural versus urban Land Ambulance Services 133 areas. As well, the United Kingdom’s ambulance within which 90% of patients are offloaded. services publicly report on a range of performance According to this reported information, between measures, such as the percentage of patients trans- the 2008/09 and 2011/12 fiscal years, the offload ported to a stroke centre within an hour of a 999 times for CTAS 3 to CTAS 5 patients actually call (similar to a 911 call in Ontario), and cardiac increased at 20% of the hospitals participating in arrest patient survival rates from transport until the program. At our request, the Ministry ran a discharge from hospital. report on the number of patients with offload times greater than 30 minutes during the 2012 calendar year. This report indicated that about 80% of the Evaluating the Offload Nurse Program ambulances with urgent patients evaluated by Ambulance response times can be delayed if ambu- paramedics as CTAS 1 waited over 30 minutes, as lance crews are detained while offloading a patient did about 75% of ambulances with CTAS 2 patients. at a hospital and are therefore not available to Furthermore, at some hospitals, more than 90% of respond to another call. Upon reaching a hospital, these urgent patients waited with paramedics for the ambulance crew must update hospital staff more than 30 minutes before being admitted to the on the patient’s condition and wait for the patient emergency department. to be triaged and admitted before departing for Despite expressing concerns regarding ongoing the next call. However, offload delays (which the delays at hospitals, all three municipalities we vis- Ministry generally defines as time spent at hospital ited indicated that the offload nurse program had exceeding 30 minutes) occur, especially in busy been helpful in reducing the time spent by ambu- urban-area hospitals’ emergency departments. By lances at hospitals and that without the program, contrast, several other jurisdictions—including delays would be significantly longer and/or occur Nova Scotia, the United Kingdom and Western more frequently. However, as also noted in our Australia—have target offload times of less than 30 2010 audit of hospital emergency departments, the minutes from the ambulance’s arrival at the hospi- municipalities indicated that hiring offload nurses tal until it is ready to leave. did not represent a full solution. One municipality

The Ministry introduced the offload nurse thought that some offload funding would be better Chapter 3 • VFM Section 3.04 program in 2008, initially as an interim initiative spent on improving the patient flow process, which to reduce ambulance offload delays. Under this they believe contributes to delays at hospitals. The program, the Ministry provides funding for nurses Ministry had not evaluated the current program to to assist with offloading patients. Because the determine whether there are more cost-effective Ministry expects urgent (CTAS 1 and 2) patients to ways to reduce offload delays. receive immediate care from hospitals, the offload Through the National Ambulatory Care nurses care for the less-urgent (CTAS 3 to CTAS 5) Reporting System (NACRS), hospitals also report patients. Between the 2008/09 and 2012/13 fiscal on ambulance patient offload times, using the time years, ministry funding for the offload nurse pro- from when the ambulance arrives at the hospital gram totalled $40 million. (which the hospital estimates based on the time it initially registers or triages the patient, to determine his or her urgency) to the time the patient is Tracking and Monitoring Offload Times accepted by the hospital. The Ministry has never Offload time for each patient is tracked from the compared the offload times reported by municipal- time when the ambulance arrives at the hospital ities to the NACRS offload times. Our review of this to when it is ready to depart, as reported by the data for one hospital indicated that the reported paramedics to the dispatch centres. Municipalities times varied significantly. report to the Ministry twice a year on the time 134 2013 Annual Report of the Office of the Auditor General of Ontario

One dispatch centre had implemented software ambulances are equipped with the appro- that allowed hospitals to electronically inform the priate type of electrocardiogram (ECG) centre when the ambulance patient was accepted. machines, that paramedics are appropriately Ministry staff noted that even after a hospital trained to interpret the ECG test results, and accepts a patient, the ambulance may not be able that paramedics are directed to conduct such to leave immediately—for example, due to the tests for all potential STEMI patients—and ambulance requiring cleaning or restocking. Hav- implement electronic prompts throughout ing accurate information on when the hospital the dispatch system for transporting these accepted the patient and comparing this informa- patients to specialized care centres; and tion to when the ambulance actually left would consistently account for the time spent by enable municipalities and the Ministry to better • an ambulance at a hospital until the patient monitor the extent to which delays occur because is accepted, based on patient urgency and the hospital is unable to accept the patient or any additional time the ambulance spends at whether there are other reasons for an ambulance hospital until it is free to return to service. remaining at a hospital after the patient is accepted. To ensure that Ontarians have access to No other dispatch centres had this software. relevant information on the performance of dispatch centres and municipal land ambulance RECOMMENDATION 1 services, the Ministry, in conjunction with To better ensure that patients receive timely and municipal land ambulance services and base high-quality ambulance services, the Ministry of hospitals, should: Health and Long-Term Care should: • establish other key measures (including out- • establish consistent provincial dispatch come measures) of land ambulance perform- centre targets for the percentage of calls ance (for example, total ambulance response to be responded to within the legislated time from call receipt to arrival at the patient response-time measures for patients experi- location, and the survival rates of patients encing sudden cardiac arrest and other with certain conditions such as stroke and Chapter 3 • VFM Section 3.04 patients whose conditions are assessed as cardiac arrest); and fitting into the highest priority according • publicly report on these indicators and on to the Canadian Triage and Acuity Scale response times for each municipality in (CTAS)—that is, all CTAS 1 patients—and a consistent and comparable format (for establish response-time targets and compli- example, separately by urban and rural ance targets for CTAS 2 (second-highest areas, as well as by patient urgency levels). priority) calls, since such calls also involve time-sensitive emergencies; MINISTRY RESPONSE monitor ambulance response times for all • The Ministry will review existing dispatch calls dispatched as emergencies in addition response targets in consultation with provincial to the legislated evaluation of response dispatch centres and municipal land ambulance times based on the paramedics’ determina- service providers to determine the extent to tion of the patient’s condition after reaching which these targets can be more consistent the scene; while also recognizing differences in local com- finalize a provincial protocol for ST-segment • munity demographics, geography and resour- elevation myocardial infarction (STEMI) ces. As part of this review, the Ministry also heart attacks—such as ensuring that all Land Ambulance Services 135

DISPATCH plans to expand dispatch response-time measures to include calls determined to be CTAS 2. Call takers at dispatch centres must quickly obtain The Ministry will monitor and report on critical patient details from callers in order to cor- response times dispatched as emergencies, in rectly prioritize requests for ambulances. Many addition to the current legislated measure- municipalities, including those we visited, have said ment of response times, to further enhance the that in order to have the best chance of responding monitoring and oversight of ambulance service quickly to the most urgent calls, they need to delivery. control dispatch. However, the Ministry indicated The Ministry will request its Medical Advis- that centralized dispatch ensures that the closest ory Committee to consult with the Ontario Base ambulance responds to a call and also helps contain Hospital Group, municipal land ambulance costs. We noted that British Columbia has three service providers, the Cardiac Care Network dispatch centres and that Alberta plans to consoli- of Ontario, and dispatch centres to review date its 14 dispatch centres into three by the end and determine the most appropriate medical of 2013. The Ministry has not assessed whether or approach to ensure that effective, evidence- not the current 22 dispatch centres are the optimal based emergency responses, including elec- number for Ontario. tronic prompts through dispatch centres, are Twenty of Ontario’s 22 dispatch centres use provided for STEMI patients. a computerized dispatch protocol to prioritize The Ministry will initiate a program requests for ambulances. (The Ministry indicated evaluation of the Offload Nurse Program in the that the other two dispatch centres, which are in 2013/14 fiscal year to review program design, remote areas, will implement computerized pro- performance metrics, offload-time measure- cesses by fall 2013.) ment definitions and funding methodology As noted earlier, 20 of the dispatch centres use to ensure the program’s alignment to broader the Dispatch Priority Card Index (DPCI) II, which emergency room strategies and initiatives. This was developed by the Ministry with input from review will be informed by a recent analysis of physicians. This protocol was implemented in emergency room offload processes conducted by 2009 to replace DPCI I, about which we identified Chapter 3 • VFM Section 3.04 a municipality. concerns in our 2005 Annual Report. As detailed The Ministry will request that the Ontario earlier in Figure 3 column A, DPCI II has four Base Hospital Group, in consultation with muni- priority codes: Code 4 (for the most urgent calls), cipal land ambulance service providers, develop Code 3 (for calls requiring a prompt response), evidenced-based medical key performance indi- Code 2 (for scheduled calls) and Code 1 (for defer- cators for the provision of ambulance services. rable calls). In 2012, the 20 dispatch centres using The Ministry will work with municipalities to DPCI II ranked 93% of calls as either Code 4 or publicly report on these performance indica- Code 3, with 3% ranked as Code 2 and 4% ranked tors. This information will augment the existing as Code 1. public reporting of dispatch response-time plans At Ontario’s other two dispatch centres, the that were posted on the Ministry’s website in municipalities that run them use the Medical 2013 and the public reporting of response-time Priority Dispatch System (MPDS), an internation- results planned for early 2014. ally used dispatch protocol. As detailed earlier in Figure 3 column B, MPDS ranks patients under one of five codes—in order from most to least urgent, Echo, Delta, Charlie, Bravo and Alpha—with the 136 2013 Annual Report of the Office of the Auditor General of Ontario

response for each code (for example, whether or the 25% of patients whose conditions were actually not the ambulance is authorized to use lights and urgent enough to warrant an ambulance being sirens) determined by the dispatch centre. In 2012, dispatched as Code 4 indicates a high degree of the two dispatch centres using MPDS ranked about over-prioritization. As a result, ambulances may not 40% of their calls as Echo and Delta, with 17% be available to respond to truly urgent calls. ranked as Charlie, 26% as Bravo and 17% as Alpha. In contrast, in 2012, the two municipalities that use MPDS ranked about 40% of total calls as Echo or Delta (that is, corresponding to a lights- Dispatch Priority and Responsibility and-sirens response); 2% to 3% of total calls were Because it is difficult to quickly and conclusively ranked as Echo. identify over the telephone all patients with In response to a 2006 request from PAC, the urgent conditions, dispatch protocols are generally Ministry indicated that it would evaluate MPDS designed to over-prioritize calls—that is, they err as part of a pilot project involving municipally run on the side of treating the call as more rather than dispatch centres. The resulting 2010 report indi- less urgent—when there is uncertainty about the cated, among other things, that fewer calls were urgency of the patient’s condition. However, if a dispatched as emergencies under MPDS than under protocol over-prioritizes too many calls, the avail- DPCI II, which could lead to more efficient resource ability of ambulances to respond to truly urgent management for ambulance services. calls may be compromised. Although the Ministry may ask the base-hospital The municipalities we visited confirmed that group to review medical evidence to ensure that having to respond to numerous Code 4 calls at DPCI II reflects current best practices, no medical once can deplete their ambulance fleets, leaving review, other than for stroke, has been conducted few or no ambulances to respond to new calls. The by the base-hospital group since 2009, when Ministry does not routinely track such instances. In DPCI II was implemented. We noted, for example, 2010, one municipality used data from its ambu- that although MPDS (which is updated more lance call reports (the medical records used by frequently to reflect new medical studies on best

Chapter 3 • VFM Section 3.04 paramedics to document each call) and found more practices in emergency medical services) includes than 1,000 instances when all in-service ambu- pre-arrival instructions (provided by call takers to lances were already being used to respond to calls. callers after an ambulance has been dispatched but During 75% of these instances, responses to new before the ambulance arrives) to give aspirin to requests for an ambulance had to be delayed. patients who are experiencing heart attack symp- Between 2006 and 2012, 67% of total calls were toms, DPCI II has not been updated to give such dispatched as Code 4 (the highest priority level). instructions. A Ministry-commissioned 2011 study stated that DPCI II was good at identifying the most urgent Dispatch Ambulance Selection patients, but with high rates of over-prioritization. Based on study data, we noted that about two- Each municipality is responsible for creating an thirds of the calls ranked by DPCI II as Code 4 ambulance deployment plan. Among other things, were over-prioritized. Further, our analysis of these plans set out the location where ambulances 2012 patient CTAS data indicated that only 1% of wait for new calls, how many ambulances and non- patients assessed by paramedics at the scene were ambulance emergency response vehicles (that is, categorized as CTAS 1 and 24% were CTAS 2, for vehicles generally staffed with one paramedic and a combined total of 25%. The significant variance equipped to treat but not transport patients) are between the 67% of calls dispatched as Code 4 and available at any given time, and which calls can be Land Ambulance Services 137 deferred. As a result, the availability of ambulances We further noted that the two dispatch centres for dispatch varies among municipalities. that use MPDS have resource allocation software Electronic systems assist dispatchers in selecting that considers not only which ambulance is clos- the most appropriate ambulance. For emergency est but also which one would be most appropriate calls—primarily Code 4 calls under DPCI II and to use in order to maintain emergency coverage Echo or Delta calls under MPDS—such systems across the entire geographic area involved. None identify the ambulance closest to the patient by of the other dispatch centres had such software, considering available ambulances as well as those and therefore the dispatch centres may not always that were previously assigned to lower-priority select the most appropriate ambulance to meet calls. Dispatchers may also use their judgment to patients’ needs. select which ambulance to send based on an electronic map that shows each ambulance’s location Defibrillator Locations within a geographic area. Our discussions with dispatch staff at a dispatch centre that handles a high For patients experiencing cardiac arrest, the timely volume of calls indicated that they often do not use use of an automated external defibrillator (AED) the electronic system’s recommendation—because, can significantly improve survival rates. Research for example, it selects the ambulance that is the indicates that delays of even a few minutes in start- shortest distance away “as the crow flies,” rather ing defibrillation after cardiac arrest can result in than the shortest distance based on available roads poor patient outcomes, including death. Accord- and speed limits. Further, electronic alternatives at ingly, for all patients with sudden cardiac arrest, the dispatch centre are too time-consuming to use. the legislated response-time measure is how often Staff also indicated that it is difficult for dispatchers any individual with an AED—whether that person in higher-volume dispatch centres to select the most is a paramedic, a police officer, a firefighter, or a appropriate vehicle using their judgment due to the bystander—is able to reach the patient within six multiple demands for ambulance services. minutes of when the ambulance service is notified. An ambulance may be asked to respond to a call Our 2005 Annual Report recommended that

outside its municipality—for example, if it is the the Ministry assess the costs and benefits of a fully Chapter 3 • VFM Section 3.04 closest ambulance to answer an emergency call or if co-ordinated emergency response system that it is returning home after transporting a patient to includes the strategic placement of AEDs in public a hospital in another municipality. Due to the use of places. In June 2011, the then-premier announced different dispatch systems, Toronto vehicles cannot the Ontario Defibrillator Access Initiative, which be viewed on any DPCI II dispatch centre’s elec- involves providing funding for the placement of tronic maps at the same time as other ambulances, AEDs in publicly accessible places such as sports nor can non-Toronto ambulances be viewed on and recreation facilities and schools, as well as cre- Toronto’s screens. Although DPCI II dispatch cen- ating an Ontario-wide AED registry. tres in areas surrounding Toronto have a separate The Ministry indicated that a web-based regis- screen that shows Toronto vehicles, dispatch centre try listing AEDs funded by the Ministry and by staff indicated that this screen is rarely used given municipalities in public, other municipal, and First the time-sensitive nature of dispatching. Further, Nations locations is expected to be implemented in vehicles can be viewed by only one DPCI II dispatch late 2013. Privately installed AEDs (such as those centre at a time. Therefore, dispatch centres are located in casinos or shopping centres) may also be generally not aware of the location of ambulances included in this registry. positioned outside their borders even though these One municipality that tracks the locations of may be closest to the patient. AEDs at municipal facilities as well as a few other 138 2013 Annual Report of the Office of the Auditor General of Ontario

public locations indicated that it has customized its RECOMMENDATION 2 MPDS dispatch system, so that dispatch staff can tell callers if a publicly accessible AED is nearby. To ensure the most efficient use of land ambu- However, although none of the other dispatch cen- lance services, the Ministry of Health and Long- tres have similar information available, the Ministry Term Care should: indicated that dispatchers will ask callers if they • assess the effectiveness of the two proto- are aware of a nearby AED. People calling these cols used in Ontario to prioritize calls and dispatch centres are expected to determine whether dispatch ambulances, including comparing an AED is available, which may take additional time the dispatch priority determined by the and therefore increase the risk to the patient. One protocols with the paramedics’ evaluation municipality we visited indicated that it had asked upon reaching the patient, and adjusting the the Ministry to incorporate AED locations in the protocols where needed to reduce excessive dispatch protocol, but had been turned down. over-prioritization of patients; We noted that the province of Manitoba imple- • consider updating software that assists mented legislation in 2012 requiring AEDs to be dispatchers in choosing the best ambulance installed in high-traffic public places such as gyms, to dispatch so that it identifies both the arenas, community centres, golf courses, schools ambulance with the shortest actual travel and airports by January 31, 2014. This law also time and the most appropriate one in order requires AEDs to be registered so that 911 dispatch- to maintain emergency coverage across ers can direct callers to locate them in situations the entire geographic area involved, as two involving cardiac arrest. municipalities have already done; and • work with dispatch centres to best match staffing with call volumes, with a view Dispatch Staffing to reducing the number of staff handling In our 2005 Annual Report, we indicated that we significantly more calls than the Ministry’s would follow up on dispatch staff turnover rates at target, and thereby helping to reduce the

Chapter 3 • VFM Section 3.04 the time of our next audit. potential for delays and errors. At our request, during our current audit the To better enable patients experiencing Ministry conducted an analysis of dispatcher turn- cardiac arrest to receive treatment as soon as over rates for the 2012 calendar year. This analysis possible, the Ministry should incorporate infor- indicated that the turnover rate for both full-time mation on the locations of publicly available and part-time staff had improved since the time of automated external defibrillators (AEDs) into our last audit, with a significant improvement in dispatch protocols. the turnover rate for full-time staff. However, we noted that many dispatch staff MINISTRY RESPONSE handled significantly more calls than the min- The Ministry has engaged a provincial base istry target of 4,200 calls per year for a full-time hospital to conduct a comprehensive review to dispatcher. According to the Ministry, handling assess the two medical dispatch protocols used significantly more calls than the target may result in Ontario. This review includes a comparison in delays or errors in call-taking and/or dispatch- of key elements of the two protocols, which are ing, both of which can negatively affect patients. designed to be highly responsive and ensure (See also the “Ministry Oversight of Dispatch Staff” that patients receive the most appropriate section later in this report.) Overall, 13% of staff ambulance response. As part of the review, the handled more than 5,000 calls each in 2012, well in Ministry will also consider the results of a 2011 excess of the ministry target of 4,200. Land Ambulance Services 139

transporting patients by ambulance are used. In evaluation report, conducted by a provincial 2011, the United Kingdom’s National Audit Office base hospital, which indicated that enhanced estimated a minimum savings of £100 million prioritization is a necessary property of medical (about $150 million) annually resulting from vari- dispatch protocols. ous emergency room diversion strategies, such as The Ministry will consider updating its providing telephone medical advice and providing software to continue to improve the provincial treatment at the scene. ambulance dispatch system. In doing so, the In Ontario, once an ambulance is dispatched, Ministry will continue to consult with working ministry policy—which was developed based on groups to add enhanced functionalities to sup- advice from its Medical Advisory Committee—gen- port dispatch decision-making that have been, erally requires ambulance paramedics to transport and continue to be, implemented at Ministry patients to a hospital, unless the patient signs a early adoption sites. Information technology form indicating that he or she refused transport. work currently under way includes enhance- Therefore, even if paramedics successfully treat ments to existing tools used by dispatchers to a patient’s condition at the scene, they still trans- select the most appropriate ambulance and port the patient. The Ministry indicated that this maintain emergency coverage. Successes from approach is taken because of concerns that para- the initial implementations will inform decisions medics may miss a potential risk to the patient. for appropriate province-wide distribution. The Ministry is currently reviewing dispatch staffing levels and call volumes to determine Telephone Medical Advice optimal staffing levels at each of its dispatch Telehealth Ontario is a 24-hour, seven-day-a-week centres to ensure effective service delivery. Ministry-funded service that provides telephone The Ministry is currently developing a web- medical advice. Telehealth’s nurses assist callers based AED registry that will list AEDs in public in determining whether their medical condition and other municipal and First Nations locations. can be treated at home, and if so, advise callers on AED locations will be provided to the Ministry

self-treatment. For more serious conditions, callers Chapter 3 • VFM Section 3.04 on a voluntary basis by municipalities and First are advised to see their family physician or go to the Nations, and published on the ministry website. emergency department, as appropriate. The Ministry will request that its Medical Only one of Ontario’s 22 land ambulance dis- Advisory Committee review the medical efficacy patch centres uses a dispatch protocol that identi- of incorporating the location of AEDs from this fies patients who can be referred to Telehealth. This registry into dispatch protocols. The Ministry dispatch centre offers low-risk patients a choice will incorporate AED information into dispatch between calling Telehealth and having an ambu- protocols if the Medical Advisory Committee lance dispatched. When appropriate, it will also supports this initiative. refer patients to other services such as the Ontario Poison Centre. This dispatch centre estimates that in a typical month, more than 200 calls (or about ALTERNATIVES TO TRANSPORTING 1% of its call volume) are referred to Telehealth, PATIENTS TO EMERGENCY ROOM and that an ambulance is subsequently dispatched for about 15% of these calls. In Ontario, ambulances generally transport The Ministry has not assessed this dispatch patients to hospital emergency rooms. In con- centre’s policy of referring low-risk patients to Tele- trast, in the United Kingdom, alternatives to health or other programs such as the Ontario Poison 140 2013 Annual Report of the Office of the Auditor General of Ontario

Centre. Such an analysis could include determining for example, how many of these situations arose whether or not patient outcomes indicate that the due to patient refusals, calls cancelled before practice is safe, and whether it could be appropri- arrival of an ambulance, or paramedics having ate for broader use across Ontario. The Ministry successfully treated patients at the scene. Although informed us that it does not currently support base hospitals review a sample of calls where no ambulance diversion strategies such as referring patient is transported to ensure that appropriate low-risk callers to Telehealth due to concerns that patient care was provided, they do not identify the dispatchers may identify patients as low risk when number of patients who were successfully treated they are actually higher risk. We noted that the by paramedics at the scene. United Kingdom publicly reports on referrals to medical telephone advice. For example, in January Treating Patients at the Scene: Emergency 2013, 12 ambulance services reported data indicat- Response Vehicles ing that 6% of callers received telephone advice; in 87% of these instances, the issue was fully resolved A non-ambulance emergency response vehicle by phone. (ERV) cannot transport patients, but is staffed with a paramedic who can provide treatment at the scene. We noted that other jurisdictions, such Treating Patients at the Scene: as Australia and the United Kingdom, use these Paramedic Care vehicles to treat patients at the scene. One muni- Paramedics in some jurisdictions can treat certain cipality we visited had expressed interest in doing types of patients at the scene, resulting in the this for patients when medically appropriate. patient not requiring ambulance transport. For In Ontario, ERVs are generally dispatched only example, in Nova Scotia and Alberta, paramedics in conjunction with an ambulance, because all treat diabetic patients who are experiencing hypo- patients are expected to be transported. The Min- glycemia (low blood sugar) and provide them with istry indicated that the ERV enables patients to be instructions on caring for themselves, instead of assessed and treated earlier, while waiting for an

Chapter 3 • VFM Section 3.04 transporting them to hospital. Further, in Calgary, ambulance. The Ministry also indicated that it is up Alberta, policies on treating patients at the scene to each municipality to decide whether or not to use resulted in fewer patients being transported to hos- ERVs. pital in 2012. In the United Kingdom, 12 ambulance Although the Ministry funds about half the cost services reported data indicating that in January of ERVs, it has not evaluated the extent of their 2013 about 30% of patients were treated by para- use or their cost-effectiveness. At our request, the medics at the scene. They further reported that Ministry produced a report on municipalities’ use of only 6% of these patients subsequently requested these vehicles. This report indicated that ERVs were an ambulance in the next 24 hours. These and dispatched for only 10% of calls in 2012, despite other jurisdictions have established medical poli- making up 26% of the municipal land ambulance cies on when and how patients are to be treated at services’ total combined fleet. By contrast, this the scene to assist paramedics in providing patient type of vehicle represents only 18% of the fleet in treatment in accordance with best practices. New South Wales, Australia, where patients can be Notwithstanding the ministry policy generally treated on the scene and avoid transport. We fur- requiring ambulances to transport a patient, we ther noted that some Ontario ambulance services noted that in 2012, over 25% (or about 350,000) of used their ERVs infrequently to respond to calls. ambulances dispatched did not transport a patient. For example, although ERVs constituted about The Ministry has not assessed the underlying rea- 37% of the total active fleet in one municipality, it sons for not transporting patients—to determine, responded to about 1% of calls with these vehicles. Land Ambulance Services 141

Furthermore, although municipalities’ ambulance the most effective emergency room diversion deployment plans indicated that many of the ERVs strategies for Ontario to ensure that patients get were staffed with advanced-care paramedics, the care they need at the right time and in the some were staffed with ambulance service chiefs right place. or assistant chiefs, whose primary duties do not In partnership with municipal land ambu- include responding to calls. At the three munici- lance service providers, who are responsible palities we visited, various vehicles were used as for determining the appropriate composition ERVs, including SUVs and pickup trucks. The cost of their ambulance fleets, the Ministry will of these vehicles, fully equipped (including about conduct an evaluation of the use of emergency $30,000 for a defibrillator), ranged from $53,000 response vehicles to identify best practices for to $117,000. The municipalities we visited indicated their utilization. that their ERVs were often used for administrative purposes, including supervision, training and real- time quality assurance. As well, the vehicles are fully equipped so that they can respond to a patient QUALITY ASSURANCE call if needed. We noted that in other provinces that In order to ensure consistent quality in ambulance more regularly treat patients at the scene, less than services, ongoing processes are needed to identify 5% of their ambulance fleets consist of ERVs. and resolve issues, particularly those that may negatively affect patients. To be most effective, RECOMMENDATION 3 such processes should follow the continuum of care To ensure that patients receive necessary care from the time the call is received until the patient that meets their needs and that patients are is released from the hospital. Various methods are not unnecessarily transported to an emergency used to gain assurance regarding the quality of department, the Ministry of Health and Long- these services, as shown in Figure 8. Term Care should consider introducing emer- Every three years, the Ministry conducts service gency room diversion policies, similar to those reviews of dispatch centre, land ambulance, and used in other jurisdictions, that meet patients’ base-hospital services. Such reviews aim primarily Chapter 3 • VFM Section 3.04 care needs by, for example, providing referrals to assess whether legislative requirements are met to Telehealth for telephone medical advice, and and ministry policies are followed—including, for treating at the scene. example, compliance with the Ministry’s patient-care The Ministry, in conjunction with the standards. Since our last audit, the Ministry has municipal land ambulance services, should also improved the timeliness of the follow-up on these evaluate the cost-effectiveness of non-ambulance reviews, and most have concluded after one visit that emergency response vehicles, including how the service is complying with required standards. many are needed and how best to use them Although the Ministry has improved its service to meet patient needs. The evaluation should review, inspection and complaint processes since include a study of practices in other jurisdictions the time of our 2005 audit, we noted further sug- with better utilization. gestions to enhance these processes and shared them with the Ministry. MINISTRY RESPONSE Because service reviews occur only every three years and complaint investigations occur only if a The Ministry will request that the Ontario Base complaint is received, ambulance services require Hospital Group, in consultation with municipal other ongoing quality assurance processes to land ambulance service providers, determine 142 2013 Annual Report of the Office of the Auditor General of Ontario

Figure 8: Selected Quality Assurance Processes and Who Conducts Them Source of data: Ministry of Health and Long-Term Care

Responsible Entity Ministry Municipalities Base Hospitals Quality Assurance on • Conducts quarterly reviews of call taking and • No role with respect • No role with respect to Dispatch Centres ambulance dispatching in accordance with to dispatch centres’ dispatch centres’ quality the Ministry’s standardized process. quality assurance. assurance. • Conducts service reviews1 every three years to ensure compliance with legislation and ministry policies. • Conducts random inspections2 of limited scope that generally look at security of call records maintained and cleanliness of the dispatch centre. • Conducts investigations of dispatch-related complaints. Quality Assurance on • Conducts service reviews1 every three years • Conduct processes • Conduct processes Ambulance Service to ensure compliance with legislation and determined by each determined by each ministry policies, and certifies those passing municipality to base hospital to ensure to provide services in Ontario. (Results ensure paramedics that paramedics comply provided to ambulance services.) comply with the with the Ministry’s • Conducts random inspections2 of limited Ministry’s Basic Life Advanced Life Support scope that generally look at equipment Support Patient Patient Care Standards. stock levels, ambulance/ambulance station Care Standards. (Results provided to cleanliness, and maintenance of vehicles. Ministry.) (Results provided to ambulance services.) • May conduct patient • Conducts investigations of ambulance outcome reviews service-related complaints. (Results provided (at base hospital to ambulance services.) discretion). Quality Assurance on • Conducts service reviews1 every three years • No role with respect • No role with respect to Chapter 3 • VFM Section 3.04 Base Hospitals to ensure compliance with legislation and to base hospital base hospital quality ministry policies. (Results provided to base quality assurance. assurance. hospitals.)

1. Service reviews are conducted by a Ministry-led team and take several days. 2. Inspections are conducted by one person and take a couple of hours.

promptly identify and resolve service problems, outcomes, either overall or by ambulance service. particularly those that may affect the quality of The Ontario Municipal Benchmarking Initiative care given to patients. Ministry inspections are (OMBI) collects data from participating munici- sometimes performed more frequently, but their palities on a number of service areas. Among other scope is limited. topics, the OMBI collects data on the survival rate of cardiac arrest patients during ambulance trips to hospital. The 2011 results—the most recent avail- Monitoring Patient Outcomes able at the time of our audit—indicate significant The quality of ambulance services, including variations among the 11 reporting municipalities, response times and paramedic care provided, can with survival rates for such patients ranging have a significant effect on the prognosis of cer- between 11% and 32%. No explanation was pro- tain transported patients. However, the Ministry vided for the variance. generally does not obtain information on patient Land Ambulance Services 143

Municipalities indicated that they have had revised in 2012. We noted the following regarding access to only certain patient-care information the revised quality assurance process: from dispatch centres, and this information can- • A specially trained quality programs officer is not readily be compared to that maintained by the to conduct quarterly reviews of 48 call-taking municipality. For example, municipalities indicated and 48 dispatching activities (a total of 96 that they cannot obtain electronic information calls), giving each a numeric score, and to for- from dispatch centres on every patient over a given ward summary information and details speci- period of time who had no vital signs at the scene— fying areas where dispatch staff performed information that would allow the municipalities to poorly to the Ministry’s senior management. perform quality assurance reviews to ensure that • Dispatch supervisors are to conduct quarterly appropriate patient care was provided. They also live (real-time) reviews of three call-taking indicated that they cannot assess whether patients and three dispatching activities for each survive beyond the ambulance trip—for example, dispatch staff person, with a smaller number to the point of hospital discharge. All three munici- of peer reviews to be conducted by dispatch palities we visited indicated that such information supervisors on the performance of other dis- could help their land ambulance services identify patch supervisors at the same dispatch centre. ways to improve ambulance services. However, the Although the Ministry obtains the number Ministry indicated that municipalities should be of live and peer reviews conducted, ministry able to access this information both from dispatch management who oversee the dispatch centre centres and from hospitals and was not sure why do not get the results of these reviews unless they had been unable to do so. We noted that a specific dispatcher has recurring problems. ambulance services in the United Kingdom measure Therefore, the Ministry cannot identify any how many cardiac arrest patients transported by systemic issues from these reviews. Quality ambulance survive until discharge from hospital. programs officers are not required to ensure Base hospitals have access to the patient rec- that these reviews are done. ords maintained by each ambulance service on Implementing this quality assurance process

every person transported. The Ministry gives base is a positive step in the Ministry’s management of Chapter 3 • VFM Section 3.04 hospitals discretion on whether or not to review dispatch centres. Our review of the results for two patient outcomes. We noted that reviews of patient recent quarters indicated that dispatch staff com- outcomes are rare, in part because base hospitals plied with policies over 90% of the time for most are, under the Personal Health Information Protec- requirements. However, in the most recent of these tion Act, 2004, generally unable to access patient quarters, dispatch centre staff were only about 60% information maintained by hospitals in their region compliant in documenting both the reason for any regarding patients transported. As well, the Ministry deviations from the recommended priority code has no province-wide data by patient condition (for and the reason for not providing any recommended example, number of stroke patients transported). pre-arrival instructions to callers. (For example, for patients experiencing cardiac arrest, DPCI II requires dispatchers to suggest that callers perform Ministry Oversight of Dispatch Staff CPR—cardiopulmonary resuscitation—and then At the time of 2005 audit, the Ministry told us that instruct callers on how to perform it.) We also it was piloting a standardized quality assurance noted that the Ministry’s analyses of dispatching process for dispatch centres. This process was performance did not include certain systemic implemented in 2006 at the 20 centres that use the issues that would highlight the need for additional DPCI II dispatch protocol and was subsequently training, such as insufficient medical knowledge 144 2013 Annual Report of the Office of the Auditor General of Ontario

and/or understanding of the dispatch protocol, or monitor the quality of more advanced life support insufficient computer skills to effectively use the procedures performed by paramedics.) It is up dispatch tool. Subsequent to our fieldwork, the to each municipal service provider to determine Ministry indicated that it had not analyzed systemic the type and frequency of monitoring. All three issues because this process has been in place for just ambulance service providers we visited indicated 17 months. that they randomly selected ambulance call reports We noted that about half of the dispatch centres for review. The frequency of such reviews varied employed an independent quality programs officer among the three service providers: one reviewed to conduct these reviews, whereas the other centres about 15 (of about 120) ambulance call reports a used other staff. For example, two dispatch centres day, another reviewed slightly more than that for that did not have a quality programs officer told its urban areas and all of its rural calls, and the us that their call reviews were done by a training third performed reviews of selected call reports for officer and an operations manager, respectively, each paramedic only once a year for the purposes both of whom share some responsibility for the of annual performance reviews. None of the service performance of dispatch staff and therefore may providers performed issue-specific reviews-—for not be objective. example, to review all childbirth-related calls in a When dispatch centres conduct live reviews, the six-month period. staff being reviewed are generally given advance Senior management, such as deputy chiefs, notice. In our view, live reviews conducted without performed the reviews at two of the three service advance notice would be more likely to reflect the providers we visited, whereas superintendents staff person’s typical performance. generally performed the reviews at the third. In addition, we found that two of the six dispatch However, one service provider indicated that it centres we reviewed were not providing timely sometimes asked its base hospital to provide feed- feedback to staff on their performance. One of these back on whether its paramedics were following dispatch centres had not completed any individual basic life support standards. Two base hospitals dispatch staff audits during half of the six months we contacted indicated that, when requested, they

Chapter 3 • VFM Section 3.04 we reviewed. At the other, in most cases there reviewed paramedics’ compliance with basic life was no evidence that feedback on areas requiring support standards for some land ambulance service improvement was provided after the reviews. providers; two base hospitals also noted that, in their view, such municipal providers do not have the expertise to provide proper medical oversight. Municipal Oversight of Paramedics’ One indicated that base hospitals should review Performance paramedics’ treatment of higher-risk conditions, Most ambulance patients require paramedics to such as childbirth and fractures. perform only basic life support procedures, such The Ministry has not asked municipal land as those needed when assessing and managing ambulance services to report to the Ministry on the chest pain, hip fractures, labour and childbirth, and results of their basic life support reviews. The Min- allergic reactions. The Ministry’s Basic Life Support istry indicated that it reviews a sample of ambulance Patient Care Standards document explains when call reports to test the municipalities’ quality assur- and how to perform these procedures. ance process. However, the results of this review Municipal land ambulance service providers are were not documented. As a result, the Ministry is responsible for monitoring paramedics’ compliance not aware of whether a sufficient number of reviews with the Ministry’s basic life support standards. are being conducted or whether there are systemic (As discussed in the next section, base hospitals issues that should be addressed province-wide. Land Ambulance Services 145

Base-hospital Oversight of Paramedics’ as 2% of all ambulance call reports noted the per- Performance formance of any procedure that required an ACP. Research indicates that advanced life support The Ministry has agreements with the seven procedures—and in particular those specified by base hospitals—consolidated from 21 in 2009 in law as generally performable only by ACPs (such response to recommendations in our 2005 Annual as inserting a breathing tube)—are typically more Report—to, among other things, monitor the risky for patients than basic life support procedures. appropriateness and quality of the patient care that It is therefore all the more important for ACPs to paramedics provide. Each base hospital is assigned maintain their abilities through practice. However, a different region of the province. with so few opportunities to perform advanced life Most land ambulance paramedics in Ontario support procedures, ambulance services run the are either primary-care paramedics (PCPs) or risk of their ACPs’ proficiency diminishing. This is advanced-care paramedics (ACPs). These prac- especially the case in municipalities with a high titioner levels reflect which medical procedures proportion of ACPs. Two of the base hospitals we each is able to perform according to legislation and talked to indicated that they were concerned about ministry policy. More specifically, PCPs can, for ACPs’ proficiency dropping due to lack of practice example, check patients’ airways and breathing, and an insufficient amount of ongoing training. administer certain medications such as aspirin, and use an external defibrillator on a patient. ACPs can perform the same medical procedures as PCPs, but Transferring Patient Information to Hospitals can also perform others, such as intubating patients Once a patient arrives at the hospital, paramedics (inserting a breathing tube) and treating seizures. need to ensure that information about the patient’s Municipalities are responsible for deciding how condition and the care provided so far is com- many PCPs and ACPs to hire. We noted that in nine municated as efficiently and accurately as possible. municipalities, more than 30% of paramedics were However, almost none of the ambulance services ACPs, whereas in 35, less than 10% were ACPs. are able to electronically download their ambulance Overall, about 20% of Ontario’s approximately call reports to the admitting hospital. Instead, after 7,000 paramedics are ACPs. orally reporting the relevant information to hospital Chapter 3 • VFM Section 3.04 staff, ambulance crews either provide a paper call report before leaving for the next call or send in the Monitoring Paramedic Provision of Advanced call report within the next day or two. Staff from Life Support Procedures three busy emergency departments across Ontario The Ministry does not track how often ambulance told us they were generally satisfied with the patient calls require advanced life support procedures (any information paramedics provided to them orally. of which can be performed by ACPs, but only a few But they also confirmed that receiving a copy of the of which—such as administering glucose—can be call report a day or two later is not useful for making routinely performed by PCPs) or how often an ACP time-sensitive patient treatment decisions. is needed. One base hospital indicated that about One particular type of information that is key 85% of its ambulance call reports (the medical to providing appropriate patient care, but that record used by paramedics to document each call) may not always be passed on to hospital staff, described only basic life support acts. Our review is the results of any paramedic-performed elec- of data from an ambulance service provider from trocardiogram (ECG) test. A three-month study another region of Ontario indicated that 70% of its conducted by one base hospital in 2011 found that calls required just basic life support and that as few in 13% of cases where a paramedic had done an 146 2013 Annual Report of the Office of the Auditor General of Ontario

ECG test that showed heart rhythm abnormalities, formats, which limits their ability to analyze it on a later ECG test performed at the hospital did an overall basis (for example, for all patients with not. In two-thirds of these cases, the paramedic- a certain condition). As well, base hospitals had to performed ECG test indicated that the patient may manually enter the paper-based information. At the have had a heart attack. The Emergency Health time of our audit, base hospitals were planning to Services Branch’s Medical Advisory Committee contract for a common database to house data from has expressed concerns about such information ambulance call reports for all base hospitals. not being provided to hospitals, and emergency department staff we spoke with indicated that they RECOMMENDATION 4 would have no way of knowing if this information To promote better-quality land ambulance was not provided to them. dispatch services and patient care by paramed- At the time of our audit, the Medical Advisory ics, the Ministry—working in conjunction with Committee was evaluating whether to recommend municipalities where applicable—should: that paramedics be required to submit an ambu- require independent unannounced reviews lance call report to the hospital before leaving, but • of calls received by dispatch centres to no solution had been proposed for ensuring that ensure that they are being appropriately paramedics provide all available test results to handled by all dispatch staff, including hospital staff. timely feedback to staff to prevent recurring Because base hospitals do not have the resources problems, and obtain summary information to periodically accompany paramedics in order to on these reviews in order to identify any assess the care they provide first-hand, most of the systemic issues; patient-care reviews conducted by base hospitals consider establishing guidelines on the focus on ambulance call reports. The agreements • desired proportion of advanced-care para- with the Ministry require base-hospital staff to medics (ACPs) and ensure that ACPs receive review the reports only from those calls in which sufficient ongoing experience to retain their a paramedic performs an advanced life support proficiency;

Chapter 3 • VFM Section 3.04 procedure, such as using an external defibrillator ask base hospitals to periodically review or intravenously administering specific drugs. Our • paramedics’ basic life support skills, since review of 2011/12 information from three base these skills are used on every ambulance call; hospitals showed that paramedics in those regions ensure that paramedics provide patient complied with standards over 90% of the time when • information documents (including all avail- performing advanced life support procedures. able test results) to emergency departments In 2006, the Ministry provided municipalities in time for the information to be useful for with a list of items that must be included in elec- making patient-care decisions; and tronic ambulance call reports (called e-PCRs). Even ensure that processes are in place to enable though by the time of our audit, most municipalities • municipal land ambulance services to read- had transitioned to e-PCRs—about 15% of muni- ily access dispatch information required for cipalities, including Peel Region, continue to use patient-care trend analyses and to periodic- paper call reports—the Ministry did not centrally ally analyze hospital outcomes for ambu- co-ordinate the acquisition of these patient-care lance patients. technologies, with the result that many different brands of software are now used even within a single base-hospital region. Therefore, when base hospitals download the data, it is in different Land Ambulance Services 147

MINISTRY RESPONSE information-sharing protocols in order to ensure the timely and appropriate exchange of The Ministry will review this recommenda- patient information to further improve patient tion as part of the continuous improvement outcomes. of the current Quality Assurance Program for ambulance dispatch to ensure that dispatch staff are provided with timely feedback and that MINISTRY FUNDING TO MUNICIPALITIES corrective action is taken to address individual and systemic issues. The Quality Assurance The Ambulance Act states that municipalities are Program is now providing comprehensive mon- responsible for funding land ambulance services itoring, evaluation and reporting of dispatcher and gives the Ministry discretion on whether or not performance and compliance with ministry to fund municipalities for these services. At the time policies, practices, standards and procedures to of our 2005 audit, the Ministry generally funded accurately assess dispatching and deployment 50% of Ministry-defined eligible costs, which decisions on the individual, dispatch centre, and resulted in the Ministry funding less than 40% of the system levels. land ambulance costs incurred by some municipal- Municipal governments are responsible for ities. However, in general, municipalities that spent making decisions on the composition of their more received more ministry funding, regardless paramedic workforces, based on the needs iden- of the number of calls for ambulances received, the tified by each municipality and the resources service levels provided, the population size served, available in each municipality. The Ministry will or the geographical area covered. At that time, direct provincial base hospitals, in consulta- the Ministry informed us that varying ambulance tion with municipal land ambulance service services levels were expected because of the varying providers, to review the existing paramedic resources of municipalities (due to, for example, education and training programs to ensure that differences in municipal tax bases). As a result, we all paramedics receive appropriate training and recommended that the Ministry develop a process to

ongoing experience to maintain and improve better achieve the existence throughout Ontario of a Chapter 3 • VFM Section 3.04 their proficiency. balanced land ambulance system. Further, the PAC The Ministry will request that the Ontario recommended in 2006 that the Ministry re-examine Base Hospital Group and municipal land its funding model, including incentives and disin- ambulance service providers evaluate practices centives aimed at promoting efficiencies in the use currently used to review paramedics’ basic life of the health-care system’s resources, specifically support skills. related to land ambulance services. The Ministry has initiated discussions with Between 2005 and 2009, the Ministry adjusted its Medical Advisory Committee and the Ontario its funding formula three times. Although some Association of Paramedic Chiefs to review municipalities received larger increases than existing patient documentation standards and others, these revisions, along with increases to develop recommendations that will ensure the compensate for inflation, resulted in the combined timely provision of patient information docu- funding to municipalities (including funding for ments to emergency departments. the offload nurse program that ranged from $4 mil- The Ministry will work with municipal lion in 2008/09 to $12 million in 2011/12) almost land ambulance service providers and the doubling between the 2004/05 and 2011/12 fiscal Ontario Base Hospital Group to standardize years, as shown in Figure 9. The number of patients transported increased by 18% over the same period. 148 2013 Annual Report of the Office of the Auditor General of Ontario

By 2009, the Ministry was funding 50% of all sal- the funding is based on prior-year expenditures, the ary increases (previously, only a maximum percent- Ministry does not fund the first year of municipal age increase was funded) and 50% of all municipal service enhancements, such as additional paramed- overhead costs allocated to land ambulance services ics or a new ambulance base: funding begins only (previously, only a maximum overhead allocation the year after a municipality has introduced these was funded). Since 2009, ministry funding to muni- services. Therefore, less-affluent municipalities may cipalities has increased about 6% per year. However, delay introducing such enhancements. at the time of our current audit, municipalities that The Ministry does not review whether the costs spent more still received higher ministry funding, to provide certain service levels are comparable regardless of service levels and other factors. In among similar municipalities with similar targeted this regard, the Ministry had not analyzed—for service levels. Further, neither the Ministry nor the example, through a review of municipalities’ municipalities know whether the additional min- ambulance deployment plans—whether similar istry funding has resulted in better value for money ambulance coverage is provided for similar popula- in terms of service levels and patient outcomes. tion sizes or similar geographic areas. The Ontario Municipalities we visited indicated that the Municipal Benchmarking Initiative reported that Ministry’s funding rules lead to uncertainty about in 2012, the total cost per hour of land ambulance how much funding will be received each year. services for the 13 reporting municipalities ranged This situation hinders municipal planning for from a low of $156 to a high of $247, and averaged ambulance services, especially when the Ministry’s $189. The cost varied significantly even among funding notification is often not received by muni- urban municipalities. cipalities until partway through the funding year. By 2012, the Ministry was funding approximately For example, the Ministry notified municipalities in 50% of each municipality’s estimated prior-year June 2012, or halfway through the year, what their expenditures plus a Ministry-established percent- funding would be for 2012. Further, municipalities age increase for inflation. (For example, funding do not always know which costs the Ministry will for 2012 was based on each municipality’s 2011 fund. For example, municipalities did not know

Chapter 3 • VFM Section 3.04 revised and approved budgets, plus 1.5%.) Because until August 2012 whether the Ministry would pay for any costs associated with the offload nurse pro- Figure 9: Ministry Funding to Municipal Land gram during the 2012/13 fiscal year. Ambulance Services and Patients Transported, 2005–2012 RECOMMENDATION 5 Source of data: Ministry of Health and Long-Term Care To ensure a balanced land ambulance system 500 1,800 throughout Ontario, the Ministry should: 450 1,600 • determine—for example, through a review of 400 1,400 municipalities’ ambulance deployment plans 350 ted (000) 1,200 and service costs—why there are differences 300 1,000 anspor Tr in ambulance service levels and costs for simi- 250 800 200 lar populations and geographic areas; and 150 600 • develop processes, such as incentives, to nding to Municipalities ($ million) 100 Ministry funding to municipalities 400 promote efficient ambulance service deliv- Fu

ry 50 Number of patients transported 200 ery—including minimum service levels or 0 0 Number of Patients

Minist benchmarks—especially where differences 07 008 010 11 12 2005 2006 20 2 2009 2 20 20 exist. Land Ambulance Services 149

The Ministry should also clearly communi- lance services in accordance with legislated cate planned funding levels to municipalities in responsibilities under the Ambulance Act. time to support municipal planning processes. The Ministry will ensure that funding rules are communicated clearly and on a timely basis MINISTRY RESPONSE to municipalities. Ministry funding is based on a municipality’s Council-approved revised budget The Ministry will consult with municipal from the previous year, with an incremental land ambulance service providers to identify adjustment to account for increased costs. The potential areas to review, such as differences in Ministry’s Land Ambulance Services Grant ambulance service levels and costs for similar reflects municipally budgeted expenditures, populations and geographic areas, to determine and the Ministry remains committed to the best practices in ambulance service delivery. 50/50 cost-sharing framework, which provides The Ministry will provide the results of this municipalities with the necessary assurances for consultation to municipalities to assist them in system and budget planning. planning and delivering municipal land ambu-

Glossary

advanced care paramedic (ACP)—A paramedic who is trained and certified to perform advanced life support procedures as well as basic life support procedures. advanced life support procedures—More complex medical procedures, all of which can be performed by advanced care paramedics and some of which can be performed by primary care paramedics. ambulance call report—A report, in either paper or electronic (called an e-PCR) form, that must be completed for all patients seen by ambulance paramedics. It is required to include, among other things, the patient’s name and condition, as well as

details of the care provided by the paramedics. Chapter 3 • VFM Section 3.04 ambulance response time—The time from when the dispatcher notifies the ambulance crew until the time the ambulance arrives at the scene. Association of Municipalities of Ontario (AMO)—An organization that represents and provides support for its over 400 municipal members in Ontario. automated external defibrillator (AED)—A portable electronic device that can analyze a patient’s heart rhythm and deliver an electric shock to a patient with life-threatening irregular heartbeat in order to re-establish a normal rhythm. base hospitals—Seven hospitals in the province with agreements with the Ministry of Health and Long-Term Care to, among other things, monitor the appropriateness and quality of the advanced life support procedures that land ambulance paramedics perform. Each base hospital is assigned a different region of the province. basic life support procedures—Less complex medical procedures performed by all paramedics, such as assessing and managing chest pain and allergic reactions. call takers—The staff at the dispatch centre who obtain information from each caller about the patient and determine the call’s priority. Canadian Triage and Acuity Scale (CTAS)—The method used by triage nurses in hospital emergency rooms, and by paramedics on arrival at the patient’s location and when departing the scene with the patient, to assess how urgently a patient requires care. See Figure 3 for descriptions of the various levels. 150 2013 Annual Report of the Office of the Auditor General of Ontario

cardiac arrest—The sudden cessation of a person’s heartbeat. Cardiac Care Network of Ontario—A non-profit organization funded by the Ministry of Health and Long-Term Care that helps co-ordinate and evaluate cardiovascular care in Ontario. cardiopulmonary resuscitation (CPR)—A series of life-saving procedures that improve the chance of survival for people who experience cardiac arrest. CPR includes chest compressions to assist with blood circulation to the heart and brain and may also involve checking to ensure that the person’s airways are open and administering breaths to improve oxygen flow. deployment plan—A plan developed by each municipality that is used by dispatch centres to assign ambulances and non-ambulance emergency response vehicles to calls, as well as to reposition them (for example, to be close to the next anticipated call). designated delivery agents—District Social Services Administration Boards, created by the province, in northern districts to deliver community services, including land ambulance services. dispatch centres—Call centres that receive requests for ambulances, primarily from 911 call centres or hospitals. Dispatch centres are responsible for prioritizing calls and notifying land ambulance crews to go to the patient. Dispatch Priority Card Index (DPCI) II—The dispatch system, developed by the Ministry of Health and Long-Term Care with input from physicians, used by 20 of Ontario’s 22 dispatch centres to prioritize patients. See Figure 3 for descriptions of the various priority codes. dispatch response time—The time from call receipt until a dispatcher advises an ambulance crew to go to the patient location. dispatchers—Staff at a dispatch centre who assign calls to ambulance crews and direct the movement of ambulances to respond to new calls. ECG (electrocardiogram)—A diagnostic test that checks the functioning of a patient’s heart by measuring and recording its electrical activity. emergencies—911 calls prioritized by DPCI II as Code 4, and by MPDS as Codes Echo and Delta. Ambulances are generally sent to these calls with lights and sirens. Refer to Figure 3 for a description of the various priority codes. Emergency Health Services Branch—The branch within the Ministry of Health and Long-Term Care that oversees the land ambulance program, including dispatch operations. It sets patient-care and ambulance equipment standards, monitors and Chapter 3 • VFM Section 3.04 ensures compliance with those standards, and, through service reviews, certifies ambulance service providers to operate in Ontario. Emergency Response Vehicles (ERVs)—Vehicles, such as SUVs and pickup trucks, generally staffed with one paramedic and equipped to treat but not transport patients. ERVs can also be used for administrative purposes, such as supervision and training. e-PCR—See ambulance call report. heart attack—A condition in which a person’s heart continues to beat but blood flow is blocked. intravenous—A drug or other liquid solution injected into a patient’s vein. intravenous glucose—A sugar liquid solution that is injected directly into a patient’s vein, often used to restore blood sugar levels in patients. Land Ambulance Committee—A committee co-chaired by the Ministry of Health and Long-Term Care (Ministry) and the Association of Municipalities of Ontario that includes representatives from various municipal ambulance services. It considers municipal concerns related to the delivery of land ambulance services in Ontario and provides advice on these issues to the Ministry. Medical Advisory Committee—A group consisting primarily of senior staff from base hospitals that advises the Ministry of Health and Long-Term Care on medical issues related to the delivery of emergency medical services and pre-hospital care. Land Ambulance Services 151

Medical Priority Dispatch System (MPDS)—The dispatch system used by two of Ontario’s 22 dispatch centres to prioritize patient conditions when a call is received. See Figure 3 for a description of the various priority codes. offload—The process of transferring a patient from the ambulance to the hospital. offload nurse—A nurse hired by a hospital exclusively for receiving lower-risk patients who arrive by ambulance. Offload Nurse Program—A program introduced by the Ministry of Health and Long-Term Care in 2008 to reduce ambulance offload delays by providing funding for offload nurses. Ontario Association of Paramedic Chiefs—A not-for-profit organization, consisting of senior management from municipal land ambulance services and nine contracted ambulance service providers, that provides advice to the Ministry of Health and Long-Term Care regarding emergency medical services in Ontario. Ontario Hospital Association (OHA)—An organization that advocates on behalf of its members, which comprise about 150 hospitals. Among other things, it strives to deliver high-quality products and services; to advance and influence health system policy in Ontario; and to promote innovation and performance improvement in hospitals. Ontario Municipal Benchmarking Initiative (OMBI)—A partnership of about 15 Ontario municipalities that collect data on more than 850 measures across 37 municipal service areas, including land ambulance services, to allow comparison of performance between municipalities. over-prioritizing—Prioritizing a call at a more urgent priority when there is uncertainty about a patient’s condition. primary care paramedic (PCP)—A paramedic who is trained to perform basic life support procedures, as well as some advanced life support procedures. respiratory arrest—Cessation of breathing due to the failure of the lungs to function properly. Response Time Standard Working Group—A subgroup of the Land Ambulance Committee tasked with reviewing the 1996 response-time standards and providing advice on a replacement standard. Standing Committee on Public Accounts—An all-party committee empowered to review and report to the Legislative Assembly on its observations, opinions and recommendations on the Report of the Auditor General and the Public Accounts. STEMI (ST-segment elevation myocardial infarction)—A specific type of heart attack resulting from a blocked artery, so called because of the way it looks on an electrocardiogram (ECG) test. Chapter 3 • VFM Section 3.04 Chapter 3 Ontario Power Generation Section 3.05 Ontario Power Generation Human Resources

for about 64% of its total operations, maintenance Background and administration (OM&A) costs. About 90% of OPG’s employees are represented by two unions: the Power Workers’ Union (PWU) and the Society Ontario Power Generation (OPG), a corporation of Energy Professionals (Society). As Figure 1 also wholly owned by the province of Ontario, was shows, staffing levels at OPG have dropped by 13% established in April 1999 as one of the five successor over the past 10 years (from about 12,800 employ- companies to Ontario Hydro. Most of OPG’s revenue ees in 2003 to about 11,100 in 2012). This came is regulated by the Ontario Energy Board, which mainly from a reduction in non-regular (temporary regulates Ontario’s natural gas and electricity sec- and contract) staff; regular staffing levels have tors in the public interest. To the extent that OPG’s remained relatively stable at around 11,000. revenues exceed its expenses, any excess, if suf-

Chapter 3 • VFM Section 3.05 ficient, goes toward paying down the stranded debt Figure 1: Staffing Levels* and Labour Costs at OPG, that remained when Ontario Hydro was split up. 2003–2012 OPG has a generating capacity of more than Source of data: Ontario Power Generation 19,000 megawatts, making it one of the largest Staffing levels power generators in North America. It produces Labour costs about 60% of the province’s power at its three 1,800 13,000 12,000 1,600 nuclear stations, five thermal stations, and 65 hydro- 11,000 electric stations. However, the amount of power that 1,400 10,000 f

1,200 9,000 af OPG produces has decreased by 23% over the last 8,000 decade (from 109 terawatt hours in 2003 to 84 tera- 1,000 7, 000 watt hours in 2012), with the reduction in demand 800 6,000 5,000 Number of St for electricity, closure of coal plants and more Cost ($ million) 600 4,000 private-sector involvement in new power generation. 400 3,000 2,000 200 OPG has been facing considerable challenges 1,000 in recent years in trying to improve its operational 0 0 4 07 10 11 12 efficiency and reduce its operating costs, especially 2003 200 2005 2006 20 2008 2009 20 20 20 labour costs. As Figure 1 shows, OPG’s labour costs * These numbers represent year-end staffing levels. They include regular staff and non-regular (temporary and contract) staff but exclude nuclear security in 2012 were about $1.7 billion, which accounted staff for reasons of confidentiality.

152 Ontario Power Generation Human Resources 153

Audit Objective and Scope Summary

The objective of our audit was to assess whether Over the last decade, the amount of electricity Ontario Power Generation (OPG) has adequate OPG generates has been declining, mainly because procedures and systems to: of reduced demand, coal plant closures and more • ensure that its human resources are acquired private-sector involvement in new power genera- and managed with due regard for economy tion. Despite the declining demand, electricity and efficiency, and in accordance with prices have been rising in Ontario. Given that OPG applicable policies, legislative requirements, still generates about 60% of Ontario’s electricity, contractual agreements and sound business its operating costs have a significant impact on the practices; and cost of electricity, as well as on OPG’s profitability, • measure and report on its results in this regard. which in turn affects how quickly the legacy debt of This objective along with our audit criteria the former Ontario Hydro can be paid off. were agreed to by senior management at OPG. About two-thirds of OPG’s operating costs are In conducting our audit, we reviewed applicable human resources-related. It is therefore critical that policies, files and studies; analyzed data; and inter- OPG’s human resources expenditures be effectively viewed appropriate staff at OPG, the Ministry of managed. OPG’s operational efficiency has been the Energy and the Ontario Energy Board. OPG had not subject of many internal and external reviews and conducted an employee engagement survey since studies. Most of these reviews have identified con- 2009, so we conducted an anonymous survey of cerns over high staffing and compensation levels. more than 800 non-unionized staff with a response Recognizing these concerns, OPG initiated a rate of more than 80%. The objective of the survey Business Transformation project in 2010. Its target was to identify common employee concerns about is to reduce staffing levels by 2,000 employees OPG’s human resources practices. We did not through attrition by 2015. Between January 2011 survey unionized staff as OPG was in collective and the end of our audit fieldwork in April 2013, bargaining with one of the unions at the time of our OPG had reduced its staff by about 1,200 employ- Chapter 3 • VFM Section 3.05 audit work. ees. Although OPG projects that it will meet its Most of our audit work took place at OPG’s target by the end of 2015, with the number of staff corporate office in Toronto, but we also visited it needs to operate expected to drop by almost 50% power stations and regional offices at Pickering, by 2025, we believe it will continue to face signifi- Darlington, Kipling, Niagara Falls, Whitby and cant challenges in making necessary adjustments. Ajax. As part of our cross-jurisdictional study of OPG has started to make some progress in government-owned utility organizations in North reducing its overall staffing levels and labour costs. America, we visited the Tennessee Valley Authority However, we found several areas where its human (TVA), whose organizational structure and oper- resource management practices need further ations are similar to those of OPG. improvement if it is to achieve its Business Trans- We reviewed the work of internal audit in the formation objectives. In addition to high staffing Ministry of Energy and OPG in planning our audit. and compensation levels, the areas that particu- We also engaged an independent consultant with larly concerned us were recruitment practices, expertise in human resources in the energy sector. performance management, succession planning, outsourcing arrangements, overtime usage, absenteeism and staff training. The respondents to our 154 2013 Annual Report of the Office of the Auditor General of Ontario

anonymous survey of over 800 OPG staff echoed • About 700 pairs or groups of OPG employ- many of our concerns. Some of our key audit find- ees reside at the same address and are ings were as follows: likely related. In some cases, OPG had no • OPG’s overall staffing levels have gone down documentation to show whether family by 8.5% (from about 12,100 in 2005 to 11,100 members of existing staff had been hired in 2012), but the size of its executive and through the normal recruitment process. senior management group (directors, vice In other cases, family members were given presidents and above) has increased by 58% jobs although they had not appeared on (from 152 in 2005 to 238 in 2012). Many any interview shortlists following the pre- respondents to our survey questioned the screening processes. rationale of reducing overall staffing levels • All OPG employees are required to obtain while creating a “top-heavy” organization. a security clearance and renew it every five • OPG rehired some of its former employees, years. However, more than 50% of the OPG mainly for the purpose of identifying, groom- staff in our sample, including senior staff ing and training successors. Almost all were with access to confidential nuclear infor- rehired shortly after leaving OPG. Some mation, either had never obtained security continued to receive significant amounts in clearances or were working with expired allowances and Annual Incentive Plan (AIP) clearances. awards, and some had already drawn their • We found a number of cases between 2005 pensions in single lump-sum payments upon and 2012 where the annual base salaries of leaving. Many respondents to our survey non-unionized staff exceeded the maximum felt that this was an indication of knowledge set out in the base salary schedule by more transfer and succession planning at OPG not than $100,000, and in one case in 2005 and keeping pace with attrition and retirement. 2006 by more than $200,000. OPG told us • OPG has reduced staffing levels at its nuclear that before 2010 it had treated the maximum facilities since 2011. Even after cuts, one of as a guideline rather than a limit, and had

Chapter 3 • VFM Section 3.05 the most overstaffed areas in 2013—facility approved and implemented salary increases maintenance, janitorial and custodial servi- before the 2010 pay freeze legislation. ces—was still 170% (or 187 staff) above the • OPG gives Annual Incentive Plan (AIP) awards industry benchmark based on data from other to all non-unionized employees. The awards nuclear operators in North America. Some can range from $1,600 to about $1.3 million, operational functions continue to be under- depending on the employee’s job band, base staffed while their associated support func- salary level and the score achieved on a scale tions continue to be significantly overstaffed. of “0” (lowest, with no award) through “4” For example, in 2013 the staffing level for (highest). Therefore, a senior executive in job nuclear plant operations was 8% (or 51 staff) band A, B or C, for example, would receive an below the benchmark, while support staff for award of 45% to 100% of his or her base salary this area was 82% (or 143 staff) above the for a score of “2,” and 55% to 150% for a score benchmark. of “3” or “4.” On average, we found that from • Although OPG has adequate policies and pro- 2010 to 2012, 67% of executive and senior cedures in place to govern its recruitment and management staff received high scores (“3” or security clearance processes, we identified “4”) while only 24% of staff in lower job bands areas of non-compliance: achieved them. Many respondents to our survey felt that there was a lack of transparency in Ontario Power Generation Human Resources 155

scoring and that it has been in favour of staff in about 700 IT staff to the vendor. In 2009, OPG senior positions. We also found in our review decided to end the contract early and renew a number of cases with limited documentation it with the same vendor without competition to support the score achieved. for a term of six years and four months at • OPG engaged a consultant to conduct a $635 million. In awarding a contract of this compensation benchmarking study in 2012, size on a single-source basis, OPG has not which found that base salary, cash compensa- taken advantage of the benefits of open com- tion and pension benefits for a significant petition, which can help demonstrate fairness proportion of staff were excessive compared and accountability, ensure value for money, to market data. Our analysis showed that total eliminate the risks associated with over- earnings were significantly higher at OPG reliance on a single supplier, and minimize than total earnings for comparable positions the perception of conflict of interest. in the Ontario Public Service (OPS), and • OPG’s total overtime costs were about many of OPG’s senior executives earn more $148 million in 2012. Although they have than most deputy ministers. declined somewhat in recent years, the number • OPG has contributed disproportionately more of OPG employees earning more than $50,000 to its pension plan than its employees have. in overtime pay has doubled since 2003, from Since 2005, the employer–employee contribu- about 260 to 520 in 2012. Planned outages tion ratio at OPG has been around 4:1 to 5:1, have resulted in high overtime pay, especially significantly higher than the 1:1 ratio at OPS. for inspection and maintenance (I&M) techni- OPG is also solely responsible for financing its cians. During outages, I&M technicians who pension deficit, which was about $555 million are regular day-workers are placed on different in its latest actuarial valuation. schedules and their normal base hours are • OPG provides numerous employee benefits, shown as unpaid leaves while the hours they such as relocation benefits and meal and work are considered overtime and paid at a travel allowances, some of which we found rate of 1.5 or 2 times their base pay. In 2012,

questionable. For example, an employee who the average overtime pay earned by OPG’s 180 Chapter 3 • VFM Section 3.05 transferred to another office received over I&M technicians was more than $66,000 each. $392,000 in housing and moving allowances The perception of many respondents to our and related reimbursements from OPG, on survey was that poor planning and scheduling top of the proceeds of $354,000 from the sale led to unnecessary overtime. of his old residence. Another employee who • OPG monitors its nuclear training on a regular moved further away from his new work loca- basis, but it needs to act on previously identi- tion received over $80,000 in 2011 and 2012. fied ways to improve the quality of its training • OPG incurred losses on 95 of the 98 purchase programs, and review the nature and timing guarantees it offered to employees whose of its mandatory training for staff in its hydro/ properties had not sold within a 90-day listing thermal unit. period, resulting in a total loss of about $2 million between January 2006 and April 2013. OVERALL ONTARIO POWER • OPG has been outsourcing its IT services to GENERATION RESPONSE the same private-sector vendor since 2001, Ontario Power Generation (OPG) is commit- when it conducted a competitive process and ted to continuous improvement. We regularly signed a 10-year, $1‑billion contract with the benchmark against the performance of our vendor. Under this contract, OPG transferred 156 2013 Annual Report of the Office of the Auditor General of Ontario

peers and invite scrutiny to help us further diligence and further tighten controls in some improve. OPG welcomes the Auditor Gen- areas of our company and our culture. OPG is eral’s audit as an opportunity to strengthen committed to taking actions that will strengthen our policies and implement recommended and further ensure that its human resources improvements. practices are managed with due regard for To enable OPG to continue to be the lowest- economy and efficiency, and in accordance with cost generator of electricity for Ontarians, a applicable legal requirements. OPG has a Code multi-year Business Transformation initiative of Business Conduct policy and will follow up on was launched in 2010, with the specific object- any exceptions identified in the report. OPG will ives of reducing labour costs and creating a report to the Office of the Auditor General the sustainable cost structure by implementing over actions taken to address the report’s recommen- 120 key improvement initiatives. OPG continues dations, as we did with respect to the Auditor to moderate consumer electricity prices, as it General’s 2006 audit of OPG’s Acquisition of currently produces 60% of Ontario’s electricity Goods and Services. at an average price that is 45% below the aver- OPG will continue to pursue its Business age price received by all other electricity gener- Transformation initiatives to deliver value to its ators in Ontario. shareholder and Ontario ratepayers. Our Business Transformation successes to date include: • headcount reductions of 1,350 from January 2011 to August 2013 (a further reduction of Detailed Audit Findings 150 since April 2013), with a target of 2,000 over the 2011–15 period; STAFFING LEVELS AND RECRUITMENT • a forecast productivity (production/headcount) improvement of 11% over 2011–15; The Ontario Energy Board (OEB), which regulates and the power produced by OPG’s nuclear and major Chapter 3 • VFM Section 3.05 • a significant decrease in the overall manage- hydro stations, raised concerns about overstaffing ment compensation, and employee business at OPG in its March 2011 decision on OPG’s rate travel and expenses, since 2008. application, stating that “although collective agree- A review of OPG’s cost-saving opportunities ments may make it difficult to eliminate positions conducted by a consulting firm concluded that quickly, it is not reasonable to ratepayers to bear “OPG has employed a systematic and structured these additional costs in face of strong evidence approach to developing a company-wide trans- that the positions are in excess of reasonable formation plan.” requirements.” While OPG has started to reduce its The Auditor General conducted an staffing levels, given its projected decreases in the employee survey and noted that the major- amount of energy it will produce, it will face signifi- ity of the responses were favourable with cant challenges in further reducing its staffing lev- some exceptions, recognizing that the survey els in the coming years. We also found several areas was conducted during a period of significant for improvement in OPG’s recruitment practices. reorganization when employees were experiencing uncertainty and stress. Business Transformation We acknowledge that the findings of the Auditor General demonstrate a need to improve With the reduction of electricity demand, closure of coal plants and more private-sector involvement Ontario Power Generation Human Resources 157 in new power generation, the amount of electricity reduction target by the end of 2015. Beyond 2015, generated by OPG has been decreasing steadily. The OPG plans to make further organizational changes decline has been sharpest over the past four years, and assess whether it needs to reduce staffing levels dropping 22%, or from 108 terawatt hours in 2008 by a further 500 employees as part of its 2016 busi- to 84 terawatt hours in 2012. Over the same period ness planning. of time, the number of staff at OPG has decreased To avoid having to offer staff costly severance by 13%, from about 12,800 employees in 2008 to packages, the reductions are to take place through about 11,100 in 2012 (see Figure 2). attrition (gradually reducing staff through retire- OPG’s projections show that the amount of elec- ment or resignation) and redeployment (relocating tricity it needs to produce will continue to decrease staff to areas where they are required) rather than (see Figure 3). Therefore, the number of staff layoffs. OPG informed us that it decided not to needed to operate, maintain and support its busi- lay off staff en masse because a large number of ness activities is expected to drop significantly from staff are eligible to retire between 2011 and 2015 2013 to 2025—by close to 50%. As a result, OPG and because layoffs would pose difficulties in a will need only about 5,400–7,000 staff by 2025. In unionized environment. For example, the collective response to these projections, OPG has initiated a agreements in place not only give first refusal for Business Transformation project that is expected voluntary job termination by seniority, they also to reduce its staffing levels through organizational provide a displacement right that allows a senior restructuring over a five-year period (2011–15) and staff member to take over the job of a junior staff save about $700 million. OPG’s target is to reduce member instead of being laid off. If unionized staff the number of its staff by 2,000, going from 11,640 exercised those rights, OPG would bear severance in January 2011 to 9,640 by December 2015. costs for junior staff as well as relocation and At the end of our audit fieldwork in April 2013, retraining costs for senior staff. In addition, with OPG had about 10,400 staff—a reduction of about many people eligible to retire, staff might stay to 1,200 since January 2011. OPG projected that at its take advantage of severance packages equivalent to current rate of reducing staff it would meet its staff a maximum of 24 months’ salary in the event of a

layoff announcement. This would curtail the rate of Chapter 3 • VFM Section 3.05 Figure 2: Electricity Generation and Staffing Levels* at staff leaving through attrition. OPG, 2003–2012 OPG told us that to achieve its staff reduc- Source of data: Ontario Power Generation tion target and sustain its operations with fewer Staffing levels staff, it has introduced 120 initiatives to improve 14,000 Electricity generation 120 efficiency and eliminate unnecessary work. OPG

12,000 100 also informed us that there is no direct correlation between specific initiatives and attrition—the pos-

10,000 Wh ) f 80 itions vacated will not match up exactly to the areas af 8,000 tion (T in which work has been eliminated. 60 ra 6,000 Although OPG informed us that staff who leave 40 Number of St 4,000 through attrition do not receive packages, we noted that its staff reduction in recent years has still cost 20 2,000 Electricity Gene a significant amount. There has been a fourfold 0 0 increase in total severance and termination costs 4 07 11 12 005 010 2003 200 2 2006 20 2008 2009 2 20 20 (from about $4 million in 2009 to about $17 million * These numbers represent year-end staffing levels. They include regular staff in 2012). The two key components of these costs and non-regular (temporary and contract) staff but exclude nuclear security staff for reasons of confidentiality. are retirement bonuses (equivalent to one month 158 2013 Annual Report of the Office of the Auditor General of Ontario

Figure 3: Projected Electricity Generation* and OPG Staffing Levels, 2013–2025 Source of data: Ontario Power Generation 12,000 100 Base scenario: staffing levels Low scenario: staffing levels Base scenario: electricity generation 90 Low scenario: electricity generation 10,000 80

70 8,000 Wh )

f 60 af tion (T ra

6,000 50 Number of St

40 Electricity Gene

4,000 30

20 2,000

0 0 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025

* Projections were prepared by OPG at the end of 2010. Both scenarios assume that all coal production will cease by 2014, that the Darlington refurbishment will begin in 2016 and that hydroelectric projects will proceed as planned. Variations between the scenarios relate to the timing of the nuclear new build, the length of time the Pickering nuclear facility will remain in operation, and the number of thermal units being converted to biomass or gas. Chapter 3 • VFM Section 3.05 of base pay for unionized staff and three months • Business Transformation came too late—it of base pay for non-unionized staff) and severance should have started much sooner for the pay, which employees negotiate with management financial health of OPG. along with input from the legal department. In • It has been under way for two years but lim- addition, under the Pension Benefits Act, employees ited practical changes have been made. can choose to receive their pensions in one lump • It has put too much focus on staff reduction sum as long as they are eligible for early retirement and not paid enough attention to developing a or they resign before age 55. Our review noted that succession plan, deploying the right people to some employees who received lump-sum payouts the right places and reducing workloads. were rehired by OPG shortly after they retired • The collective agreements and the “culture of or resigned (see the section on Rehiring Former entitlement” among staff have restricted OPG Employees as Temporary or Contract Staff). from making many changes through Business Respondents to our employee engagement Transformation. survey generally felt the intention of Business • There was no consultation to obtain input Transformation was valid but raised some concerns from all staff before Business Transformation about its execution, for example: was rolled out, and there has been a lack of Ontario Power Generation Human Resources 159

meaningful, informative and effective com- 12,100 in 2005 and 11,100 in 2012, a reduction of munication to employees about Business 8.5% since 2005. However, the number of execu- Transformation since rollout. tives and members of senior management dropped • “Working in silos” has led to a lack of initially from 173 in 2003 to 152 in 2005 but went engagement, commitment and buy-in from up again to 238 by 2012, an increase of 58% since OPG employees in response to Business 2005. Specifically: Transformation. • The number of executives (vice presidents and above) dropped from 70 in 2003 to 54 in 2005 but increased to 94 by 2012—an increase of Staffing Levels for Executives and Senior 74% since 2005. Management • The number of senior management staff In the rate application it submitted to the OEB in (directors and equivalent) decreased from 103 2007, OPG indicated that it had made changes since in 2003 to 98 in 2005 but increased to 144 by 2004 “to signal a return to a more public-sector 2012—an increase of 47% since 2005. employment situation.” One of these changes was • The most obvious jump occurred in 2012, reducing the number of executives at OPG. How- during Business Transformation. Nine vice ever, we noted that this has not been the case in presidents and 21 directors left OPG that year, recent years. but 17 employees were promoted to VPs and Despite the overall reduction OPG has recently 50 to directors, indicating that many of the made to its staffing levels, the size of its executive promotions were for newly created positions and senior management group (directors, vice rather than to fill vacant positions. OPG presidents and above) has moved in the opposite informed us that the new positions were part direction. Figure 4 shows the overall number of of Business Transformation and for nuclear staff has decreased from about 12,800 in 2003 to refurbishment. We also found that the number of vice pres- Figure 4: Number of Staff* vs. Number of Executives idents and directors with no specific titles or job

and Senior Management Staff at OPG, 2003–2012 descriptions has increased considerably, from 12 Chapter 3 • VFM Section 3.05 Source of data: Ontario Power Generation in 2005 to 40 in 2012. OPG explained that some employees were not assigned specific titles or Directors and equivalent f portfolios because they were working on special

af Vice presidents and above Total number of staff at OPG 250 14,000 projects without job descriptions, or their job descriptions were still being written. 12,000 200 Many of the respondents to our survey ques- 10,000 tioned the rationality of reducing overall staffing f 150 8,000 af levels while creating a “top-heavy” organization. They felt that the only visible change brought about 6,000 100 by Business Transformation was numerous promo- 4,000 Number of St tions to expand the size of the executive and senior 50 2,000 management group. They also felt that promotions had been made hastily with no transparent selec- 0 0 Number of Executive and Senior Management St tion process and had been communicated poorly, 07 10 011 12 2003 2004 2005 2006 20 2008 2009 20 2 20 creating ill feeling and mistrust among employees. * These numbers represent year-end staffing levels. They include regular and non-regular (temporary and contract) staff but exclude nuclear security staff for reasons of confidentiality. 160 2013 Annual Report of the Office of the Auditor General of Ontario

Benchmarking of Staffing Levels at Nuclear that it has since adjusted its staff reduction target Facilities to address the imbalances. In the second report, issued on the last day of our audit fieldwork in April OPG has been under increasing scrutiny from the 2013, the consultant found that OPG’s nuclear OEB to demonstrate that its operations are in line staffing level was 8% (or 430 employees) above with those of other nuclear stations across Canada the benchmark, with 23 overstaffed areas and 16 and in the United States. In its March 2011 deci- understaffed areas. sion, the OEB directed OPG to submit in its next Figure 5 shows selected functional areas identi- rate application a study comparing staffing levels at fied as over- or understaffed in the two studies. its nuclear facilities with industry benchmark data Both benchmarking studies found that the over- from other nuclear operators in North America. staffed areas related mainly to support functions OPG engaged a consultant who produced two (for example, general maintenance, administra- reports for OPG’s management to measure and tive support and human resources) while the report on whether OPG’s nuclear staffing level was understaffed areas related mainly to operational in line with comparable organizations. The first, functions (for example, maintenance/construc- issued in February 2012, noted that OPG’s nuclear tion, plant operations, engineering, emergency staffing level was 17% (or 866 employees) higher planning and safety). We noted that several oper- than the benchmark in 2011, with 23 overstaffed ational functions were understaffed while their areas and 14 understaffed areas. OPG informed us

Figure 5: Selected Areas Identified as Overstaffed/Understaffed at OPG by Nuclear Benchmarking Studies Source of data: Ontario Power Generation

280 2011 240 2013 rk

200 2 Chapter 3 • VFM Section 3.05 160

120

f at OPG Over/(Under) Benchma 40 af

0 Nuclear Safety Review Emergency Planning Decontamination of Radioactive Wast e Plant Operation s Maintenance/Construction Engineering–Technical Engineering–Plant Radiation Protection/HP Applied l 1

(40) Number of St Facilities Schedulin g (80) Budget/Finance Human Resources (120) Administration/Clerica Plant Operations Support Maintenance/Construction Suppor t

1. “Facilities” refers to general maintenance and custodial services, such as cleaning and changing light bulbs. 2. “HP” is an acronym for health physics, the physics of radiation protection. Ontario Power Generation Human Resources 161 associated support functions were overstaffed. • Four of the employees were offered jobs For example, in 2013, Maintenance/Construction although their names had never appeared was 6% (or 55 staff) under the benchmark, but on interview shortlists following the pre- Maintenance/Construction Support was 78% (or screening process. 194 staff) above it. Similarly, Plant Operations • Another four employees had no documents was 8% (or 51 staff) below the benchmark while in their files to show whether they had been Plant Operations Support was 82% (or 143 staff) hired under the normal recruitment process. over the benchmark in 2013. A similar pattern was • Two other employees had been hired as tem- shown in 2011. porary staff based on referrals without going One of the most overstaffed areas, Facilities through the normal recruitment process and (general maintenance, janitorial and custodial were later offered permanent jobs on the basis services), has improved only slightly. It went from of their temporary work experience. being 173% (or 199 staff) above the benchmark in 2011 to 170% (or 187 staff) above it in 2013. Security Clearance Requirement Other key understaffed areas have shown limited All employees are required to obtain security clear- or no improvement. For example, staffing levels in ances before commencing work with OPG and must the Engineering–Technical and Engineering–Plant renew them every five years. There are three types areas remained almost unchanged in 2013, still of security clearance: about 30% below the benchmark. 1. Standard: A Criminal Record Name Check (CRNC) must be completed for staff from Recruitment Practices and Requirements hydro/thermal and corporate support units, as well as contractors working in nuclear units Although we found that OPG had adequate policies for a specific timeframe but with no access to and procedures in place to govern its recruitment protected areas or nuclear information. practices, it did not always follow them. We found 2. Site Access: In addition to a CRNC, a Can- non-compliance in several areas. adian Security Intelligence Service check and verification of employment and education Chapter 3 • VFM Section 3.05 Hiring Process must be completed for staff from nuclear units We identified about 700 pairs or groups of OPG as well as for some other employees with employees (about 1,400 staff, or more than 10% of access to nuclear information. OPG employees) who resided at the same address, 3. Level II (Secret): All the checks in a site indicating that they were most likely family mem- access clearance plus a financial credit check bers. OPG has no policy prohibiting the hiring of must be completed for staff with access to family members so long as proper recruitment information classified as “secret” by the fed- practices are followed: family members of the eral government. prospective employee cannot be involved in the We reviewed security clearances initiated by hiring decision and family members should not be OPG during a five-year period, from January 2008 in reporting relationships with one another. We to December 2012, and noted the following: reviewed the personnel files for a sample of 20 • Aside from the Chair and the CEO, none of pairs or groups and found that it was not evident the members of OPG’s Board of Directors had whether proper recruitment processes had been obtained security clearances even though followed for half the employees in the sample. they had access to confidential information. Specifically: OPG indicated that it was in the process of obtaining security clearances for them. 162 2013 Annual Report of the Office of the Auditor General of Ontario

There were numerous examples of employees • There are currently a number of interim pos- who had started working at OPG before their itions relating to Business Transformation, pro- security clearances were issued. ject work and other new initiatives. By August In a sample of 50 employees who were on • 2013, there were 218 senior management pos- OPG’s payroll but not on its security clearance itions compared to 238 at the end of 2012. This record, 13 had never obtained security clear- number is forecast to continue to decline. ances. OPG informed us that this was because OPG has conducted extensive benchmarking hydro/thermal and corporate support staff of its nuclear and other operations. Based on hired before May 2003 were exempt from this benchmarking, we are executing several security clearance. One of these employees initiatives that are designed to address oppor- had held various senior positions in nuclear tunities for efficiencies, cost reductions and staff finance, nuclear reporting and nuclear waste imbalances in nuclear operations. In 2012, the management, and had access to sensitive Ministry of Energy engaged a consulting firm to information. The remaining 37 employees assess OPG’s existing benchmark studies, and in our sample had joined OPG after May to identify organization and structural oppor- 2003, but more than half of them had never tunities for cost savings. The report validated obtained security clearances or were working OPG’s Business Transformation initiative and with expired clearances. its objectives. We will continue to identify and implement other improvement initiatives. RECOMMENDATION 1 As recommended by the Auditor General, To ensure that staffing levels are reasonable and OPG will review and monitor compliance with that it has the right people in the right positions its recruitment and security clearance processes. to meet its business needs, Ontario Power Gen- We will also conduct an internal audit of our eration should: hiring practices. • evaluate and align the size of its executive and senior management group with its over- Chapter 3 • VFM Section 3.05 all staffing levels; COMPENSATION • address the imbalances between overstaffed and understaffed areas in its nuclear oper- OPG’s labour costs account for most of its total operations; and ating costs. This proportion has increased from 55% • review and monitor compliance with its in 2003 to 64% in 2012. In its March 2011 decision, recruitment and security clearance processes. the OEB also noted the significance of OPG’s labour costs compared to its total operating costs and that ONTARIO POWER GENERATION its compensation levels were a concern in light of RESPONSE the overall poor performance of its nuclear business, in terms of operations and costs, compared to its In 2010, Ontario Power Generation (OPG) peers. Therefore, the OEB disallowed $145 million launched a multi-year Business Transforma- in compensation costs, stating in its decision that tion initiative to reduce labour costs, create a the staffing levels and amount of compensation at sustainable cost structure and allow OPG to con- OPG were both too high. OPG appealed the OEB’s tinue to moderate consumer electricity prices. ruling. In June 2013, the Ontario Court of Appeal The number of executive and senior manage- found that the OEB had based its decision on infor- ment positions, as well as overall staffing levels, mation that had not been available to OPG when it is addressed through Business Transformation. Ontario Power Generation Human Resources 163 was in collective bargaining, concluding that OPG extent. Since 2010, the average total earnings for could not unilaterally reduce staffing levels and non-unionized staff has increased 3%, from about compensation rates that had already been set by col- $134,000 in 2010 to about $138,000 in 2012. lective agreements. We found a number of reasons for the increase in average total earnings for OPG’s staff over the last 10 years. Under collective bargaining, wage Compensation Levels increases for unionized staff have been between 2% Unionized and Non-unionized Staff and 3% per year since 2003. This trend continued At the time of our audit, OPG had about 11,100 through to 2012 because unionized staff were not employees. Approximately 90% of them are union- subject to the 2010 pay freeze legislation, making ized: 58% are skilled trades, such as electricians wage increases possible under their collective and technicians, represented by the Power Work- agreements so long as the increase could be offset ers’ Union (PWU); and 32% are professionals, by cost savings elsewhere. Specifically, with OPG’s such as engineers and scientists, represented by reduction in staffing levels in recent years, the sav- the Society of Energy Professionals (Society). ings gained from paying salaries to fewer staff were The extent of unionization at OPG has generally more than enough to raise wages for existing staff. remained constant over the years. As in any union- This enabled PWU to negotiate wage increases of ized environment, changes to compensation can 2.75% in 2012, in 2013 and in 2014, and the Society be made only through collective bargaining, griev- to reach wage increases of 0.75% in 2013, 1.75% ances or arbitration. in 2014 and 1.75% in 2015 through an arbitration In response to the ballooning provincial deficit, process. OPG indicated that these settlements were the government passed the Public Sector Compensa- favourable in comparison with previous settlements tion Restraint to Protect Public Services Act in March and with settlements reached by other organiza- 2010 to freeze compensation growth for non- tions in the electricity sector. unionized employees in the Ontario Public Service Non-unionized staff also received salary (OPS) and Broader Public Sector (BPS). Although adjustments that were exempt from the pay freeze

the legislation did not apply to unionized staff, the legislation. One such adjustment was incentive Chapter 3 • VFM Section 3.05 2010 Ontario Budget contained a policy statement awards. For example, the 50 highest earners at with clear expectations that new collective agree- OPG saw their earnings increase by an average of ments would provide no net increase in compensa- about 11% in 2011 from the previous year. Another tion for at least two years. adjustment was pay increases resulting from pro- OPG’s payroll data showed that the average motions; as we have already noted in this report, total earnings increased by 7% since the 2010 pay many OPG employees were promoted to executive freeze legislation, from about $102,000 in 2010 to and senior management levels in 2012. A third about $109,000 in 2012 (see Figure 6). Specifically, adjustment was made to temporarily mitigate wage the average total earnings for unionized staff went compression, where non-unionized supervisors up by 6% (from about $118,000 in 2010 to about earn less than their unionized subordinates. For $125,000 in 2012) for Society staff, and by 7% example, 680 Society staff earned more than their (from about $99,000 in 2010 to about $106,000 in non-unionized supervisors in 2012, so an adjust- 2012) for PWU staff. Meanwhile, the average total ment was made to raise the salaries of 220 non- earnings for non-unionized staff dropped slightly unionized supervisors 3% above their highest-paid between 2008 and 2010, even before the 2010 pay unionized subordinates. freeze legislation, because OPG limited base pay We also found in our review of OPG payroll data increases and reduced incentive awards to some from 2005 to 2012 a number of non-unionized 164 2013 Annual Report of the Office of the Auditor General of Ontario

Figure 6: Average Total Earnings* for OPG Staff, 2003–2012 ($) Source of data: Ontario Power Generation Non-union staff 150,000 Union staff (the Society of Energy Professionals) Union staff (the Power Workers’ Union) 135,000 OPG staff overall

120,000

105,000

90,000

75,000

60,000

45,000

30,000

15,000

0 20032004 20052006200720082009 2010 2011 2012

Chapter 3 • VFM Section 3.05 * Average total earnings include base salary, overtime, incentives and bonuses as well as various types of allowances.

staff whose annual base salaries exceeded the max- 2006. Essentially, if an employee’s old base salary imum amount set out in the base salary schedule exceeded the maximum set out in the new schedule, by more than $100,000, and in one case in 2005 he or she was “green circled” to maintain the old and 2006 by more than $200,000. OPG told us level while still receiving annual wage increases. that before 2010 it had treated the maximum as a guideline rather than a limit, and had approved and Sunshine List implemented salary increases before the 2010 pay OPG is required by the Public Sector Salary Dis- freeze legislation. OPG also informed us that since closure Act, 1996 to disclose annually the names, 2010, no salary increases had been provided to the positions, salaries and total taxable benefits of any employees whose base salaries already exceeded employees who made $100,000 or more in a calen- the maximum. dar year. (This disclosure is popularly known as the We found similar instances for about 1,200 “Sunshine List.”) unionized staff who had received more than the The number of OPG staff on the Sunshine List maximum set out by the base salary schedule in has grown steadily since the organization was 2012. OPG explained that this was because of created in 1999, albeit at a slower pace after the the implementation of new base salary sched- 2010 pay freeze legislation. Over the last 10 years, ules for PWU staff in 2002 and Society staff in Ontario Power Generation Human Resources 165 the number has doubled, from 3,980 employees in Figure 7: OPG’s Total Cash Compensation Above/ 2003 to 7,960 in 2012, representing about 62% of Below Canadian Market Median, 2012 (%) the employees on OPG’s payroll; the corresponding Source of data: Ontario Power Generation increases in total salaries and taxable benefits paid OPG vs. Group 1 to those on the list were $513 million for 2003 and (power generation and electric utilities) $1.11 billion for 2012. The number of OPG top- OPG vs. Group 2 earners (people who earned $200,000 or more) on 30 (nuclear power generation and electric utilities) OPG vs. Group 3 (general industry) the Sunshine List has increased at an even faster 25 rate—in 2012 it was almost four times higher (448 employees) than it was in 2003 (117 employees). 20

15 Compensation and Pension Benchmarking 10 OPG vs. Similar Organizations 5 In its March 2011 decision, the OEB noted that OPG’s compensation benchmarking analysis has 0 not been comprehensive. It directed OPG to file a Non-unionized Staff Unionized Staff Unionized Staff (5) (Management) (Society) (PWU) full, independent compensation study with its next [10% of OPG Staff] [32% of OPG Staff] [58% of OPG Staff] application and recommended that the study cover “a significant proportion of OPG’s positions” and OPG vs. Ontario Public Service that the benchmark should generally be set at the In January 2007, the government established an median (50th percentile). Agency Review Panel to review specific issues at OPG engaged a consulting firm to conduct OPG and the other four provincial electricity-sector a compensation benchmarking study in 2012. institutions (Hydro One, the Independent Electri- The study compared base salary levels and total city System Operator, the Ontario Power Authority cash compensation for about 50% of staff at and the Ontario Energy Board). Commenting on

OPG with similar organizations, including Bruce the organizations OPG chose to use as comparators Chapter 3 • VFM Section 3.05 Power and utility companies in other Canadian for its compensation benchmarking, the Panel said jurisdictions. The study looked at three groups of there appeared to be “a bias in favour of utility/ positions (Power Generation & Electric Utilities, energy organizations in the private sector. To the Nuclear Power Generation & Electric Utilities and extent public-sector organizations are used as com- General Industry) and found that compensation parators, it is almost exclusively Canadian utilities for a significant proportion of OPG’s staff was (for example, Hydro-Quebec, BC Hydro and Atomic well above the market median (see Figure 7). Energy of Canada), and there is only very limited The study also found that OPG’s annual pension use of a broader public-sector group (for example, and benefits (health, dental and life insurance as Ontario Public Service, provincial and federal well as disability benefits) were higher than the Crown corporations or agencies and regulators).” market average, depending on base salary level. Given that the Province of Ontario is OPG’s For example, the annual pension and benefits of sole shareholder, we compared total earnings and an OPG employee earning a base salary of $60,000 pensions at OPG with those in the Ontario Public would be about 19% ($2,400/year) higher than the Service (OPS) for perspective. For total earnings, market average; for an employee with a base salary we selected 16 typical positions below the execu- of $220,000, they would be about 38% ($13,000/ tive levels at OPG in areas such as administration, year) higher than the market average. finance and human resources to benchmark against 166 2013 Annual Report of the Office of the Auditor General of Ontario

comparable positions in the OPS. For 13 of the 16 of its employees every year. Since 2005, the positions, the average total earnings at OPG were employer–employee contribution ratio at OPG higher than the maximum total earnings in the OPS has been around 4:1 to 5:1, significantly higher (see Figure 8). As for the executive levels, the total than the 1:1 ratio at OPS. For example, employ- earnings for most OPG senior vice presidents sig- ees contributed $70 million to the pension fund nificantly exceeded those for most deputy ministers in 2012 while OPG put in $370 million. in the OPS. • Executives, who contribute only 7% of their Pensions are a very significant part of total earnings up to a maximum of $17,254 annu- compensation at OPG. This is especially the case ally while OPG contributes 18.1%, are eligible for executives, whose pensionable earnings can for particularly generous pensions. For be greatly increased when bonuses or awards example, the top five executives at OPG will are added to their base salaries. Unlike the OPS, be eligible to receive annual pensions ranging which has a 50–50 split between employer and from $180,000 to $760,000 when they reach employees for making pension contributions and age 65. funding pension shortfalls, OPG has unequal cost- • OPG also bears the responsibility of financing and responsibility-sharing between employer and any pension funding shortfalls. The most employees. We noted in particular: recent actuarial valuation, as at January 1, • OPG’s contributions to the pension plan have 2011, showed OPG’s pension fund in a deficit been disproportionately larger than those position, with a shortfall of $555 million. This

Figure 8: Comparison of Average Total Earnings at OPG vs. Maximum Total Earnings at Ontario Public Service (OPS) ($) Sources of data: Ontario Power Generation, Ministry of Government Services 250,000

OPG 200,000 OPS Chapter 3 • VFM Section 3.05

150,000

100,000

50,000

0 t t Secretary Finance Cler k Business Analys t Financial Analys Section Manager, Finance Controlle r Finance Specialist Administration Clerk Director, Accounting Administration Assistan Manager, Compensation Human Resources Advisor Manager, Human Resources Team Leader, Compensation Office Support Representative Human Resource Admin. Support Senior Advisor, Workforce Planning Ontario Power Generation Human Resources 167

was more than twice its projected shortfall and compensation are closely linked. Providing of $239 million as at January 1, 2008. The the right incentives can help keep people account- next actuarial valuation will be prepared as at able.” However, the Committee found that there January 1, 2014. was “not a strong enough link between achieve- • In July 2013, Dominion Bond Rating Service ment and rewards” at OPG. We found that this was (DBRS), a Canadian-owned and globally still the case. recognized ratings agency, released its Under OPG’s Annual Incentive Plan (AIP), annual pension study reviewing 461 pension non-unionized employees are scored on their job plan funds in Canada, the U.S., Japan and performance on a scale of “0” (the lowest, with no Europe. The report highlighted the 20 Can- award) to “4” (the highest), and receive an annual adian funds with the largest pension deficits. cash award for meeting key financial and oper- OPG was at the top of the list with a deficit ational objectives. As Figure 9 shows, awards can of $3.3 billion. This amount, derived from range from 4% of base pay (starting at $1,600) the accounting valuation used for preparing to 150% of base pay (as high as $1.3 million) OPG’s financial statements, was different depending on an employee’s position, base salary from the $555-million deficit amount from level and AIP score. Therefore, a senior executive the most recent actuarial valuation, which is in job bands A, B or C, for example, would receive the valuation used for funding purposes. an award of 45% to 100% of his or her base salary for a score of “2,” and 55% to 150% for a score of “3” or “4.” Compensation and Staff Performance Figure 10 shows that the distribution of high Non-unionized Staff AIP scores (“3” or “4”) has been skewed toward In 2004, the OPG Review Committee established by executives and senior management staff (directors, the Ontario government noted that “accountability vice presidents and above). On average, 67% of

Figure 9: Annual Incentive Plan (AIP) Award Structure* Source of data: Ontario Power Generation Chapter 3 • VFM Section 3.05

AIP Score and Base Salary Range ($) Associated % Award Band Position Group Positions (Example) Min Mid Max 1 2 3 4 A Chief Executive Officer 580,000 720,000 860,000 50 100 125 150 B Senior Executive Executive Vice Presidents 315,000 390,000 465,000 22.5 45 55 67. 5 C Senior Vice Presidents 265,000 330,000 395,000 22.5 45 55 67. 5 D Chief Information Officer 195,000 260,000 325,000 12.5 25 30 37. 5 Executive E Vice Presidents 160,000 200,000 240,000 12.5 25 30 37. 5 F Directors 120,000 150,000 180,000 10 20 25 30 G Management Managers 95,000 130,000 160,000 7. 5 15 20 22.5 H Section or First Line Managers 85,000 110,000 140,000 7. 5 15 20 22.5 I Analyst 65,000 85,000 105,000 5 10 12.5 15 Professional J Service Co-ordinator 55,000 70,000 90,000 4 8 10 12 K Administrative Assistant 45,000 55,000 65,000 4 8 10 12 Administrative L Secretary 40,000 50,000 60,000 4 8 10 12

* Award amounts are calculated by multiplying the base salary by the percentage that corresponds with the AIP score. Both base salary ranges and AIP structure have remained unchanged since January 2008. There is no award for an AIP score of “0.” 168 2013 Annual Report of the Office of the Auditor General of Ontario

been to the benefit of senior management staff, and Figure 10: Distribution of Annual Incentive Program that scores were based on factors other than job (AIP) Scores by Job Bands, 2010–2012 performance and productivity. Source of data: Ontario Power Generation Scores 0 and 1 80 Score 2 Scores 3 and 4 Unionized Staff We found that performance evaluations of union-

f 60 ized employees have not been done adequately and af consistently. For example, the collective agreement for PWU staff stipulates that progression through ge of St 40 steps in salary ranges will be time-based subject to en ta rc satisfactory performance and successful completion Pe 20 of training, and that progression is to be withheld for six months if performance is not satisfactory. The usual method of determining whether staff 0 Executive and Below Executive and performance has been satisfactory is a performance Senior Management Senior Management evaluation, but in our review of a sample of 15 PWU (Bands A–F) (Bands G–L) staff, we found that only two out of a possible 30 evaluations for 2010 and 2011 had been completed. executive and senior management staff received OPG informed us that it does not have a require- high AIP scores from 2010 to 2012. Only 24% of ment to prepare and document formal performance staff in lower job bands received high scores during evaluations for PWU staff. the same period; the majority of them achieved a The majority of respondents to our survey score of “2.” felt that OPG did not have timely, effective and Some executives had incomplete or no perform- appropriate performance management in place ance evaluation documentation to support their for its unionized staff. They felt that collective high AIP scores. OPG explained that AIP scores are agreements, grievances, arbitrations and automatic

Chapter 3 • VFM Section 3.05 reviewed and validated in calibration meetings, progression had created a perception that “nothing but acknowledged that many performance evalua- can be done” and a tendency to avoid dealing with tions were verbal and not documented in writing. poor performance. We noted one case where an employee received a At the time of our audit, there were 960 union- severance payment of $450,000 when terminated ized employees in managerial and supervisory for ineffective performance and inappropriate roles. In 2004, the government’s OPG Review behaviour. This employee had received a total of Committee also noted that “many staff members $760,000 in AIP awards in the previous four years. that OPG considers to be managerial belong to OPG informed us that the employee’s behaviour a bargaining unit, which may be an obstacle to had become an issue only in the last few months of accountability and effective pursuit of company his employment and was not related to his perform- goals. We strongly encourage all parties to make ance before then. every effort to put in place a more rational arrange- The majority of respondents to our survey ment.” OPG informed us that two-thirds of its indicated that they felt AIP was unfair and said they unionized staff with managerial or supervisory did not feel it encouraged them to be as productive roles are represented by the Society, and a clause in as possible. In particular, respondents cited a lack their collective agreement allows them to perform of transparency in AIP scoring, which they felt had those functions. Ontario Power Generation Human Resources 169

The majority of respondents to our survey also • An employee transferring to another office indicated that they felt unionized staff performing sold his former residence for about $354,000 managerial or supervisory functions had a nega- and purchased a new property for $1.35 mil- tive impact on accountability and performance lion. Payroll data showed that he had received management. They cited conflicts of interest more than $244,000 for housing assistance and reluctance amongst unionized managers or and moving expenses. However, when we supervisors to carry out performance reviews or added up the other expenses his file showed deal with performance problems of their unionized that he had claimed, we found the total subordinates. amount that he received was actually over $392,000. Another employee chose to rent an apartment Other Employee Benefits • instead of buying a property in his new loca- In addition to base salary and incentive awards, tion. Payroll data showed that he had received OPG grants its employees various other types of $75,000 for rental assistance and moving benefits. Some were for significant amounts, which expenses. However, with the other benefits his we found questionable in some cases. file showed that he received, the actual total was $140,000. A third employee, when transferring to Housing and Moving Allowances • another office, sold his old residence for When regular OPG employees change their work $380,000 and bought a new property for location, they are eligible for housing and mov- $830,000. Payroll data showed that he had ing allowances and relocation benefits that cover received about $43,000 for housing assistance various expenses. These include legal fees and and moving expenses. With the other benefits disbursements related to the sale and purchase his file showed that he received, the actual of properties; real estate brokerage fees; upkeep total was $79,000. costs on former residences that have not yet sold; OPG’s policy is that employees must move a interim living expenses before moving into a new

minimum of 40 kilometres closer to their new work Chapter 3 • VFM Section 3.05 residence; packing and shipping of household location to qualify for housing and moving allow- goods; temporary storage; house-hunting trips; ances. However, OPG informed us that staff who home-inspection fees; and incidental out-of-pocket moved fewer than 40 kilometres closer could qual- expenses. OPG indicated that all relocation benefits ify if a move caused hardship. In one example of are subject to Canada Revenue Agency taxation this, an employee who transferred from the Toronto requirements and employees are cautioned to office to Pickering received over $80,000; however, retain receipts in case they are audited. not only had he moved only 10 kilometres, but he Payroll data from 2009 to 2012 showed that moved further away from his new work location OPG spent on average about $1.4 million each (the move was within the same city as his old resi- year on housing and moving allowances. When we dence, which was not Toronto or Pickering). reviewed the files documenting the costs of moving OPG also provides a purchase guarantee in the individual employees, we found employees who event that a transferring employee’s property is had not only received housing and moving allow- not sold within a 90-day listing period. It incurred ances granted by OPG through payroll but also losses for 95 of the 98 properties it purchased received further benefits by claiming various other and resold on behalf of its employees from Janu- expenses. OPG was unable to locate the supporting ary 2006 to April 2013, for a total loss of about documents for some of these claims. For example: $2 million. 170 2013 Annual Report of the Office of the Auditor General of Ontario

Travel and Miscellaneous Allowances make its Annual Incentive Plan (AIP) more Payroll data for 2009 to 2012 shows that OPG • effective by creating a stronger link between incurred about $2.8 million each year on average awards and staff performance based on for travel and miscellaneous allowances. Staff can documented annual evaluations; and request these allowances for a number of reasons, review salary levels and employee benefits, some of which we found questionable. For example: • including pensions, to ensure that they are OPG assigned three employees to work on a • reasonable in comparison to other similar and rotational job and provided a $15,000/year broader-public-sector organizations and that allowance to one of them because she was they are paid out in accordance with policy, unable to drive and needed to take a taxi to adequately justified and clearly documented. work. However, we noted that OPG had also paid $15,000 each to the other two employees, who did drive to work. ONTARIO POWER GENERATION RESPONSE • OPG offered $1,500 per month for one year to an employee who had accepted a position Ontario Power Generation (OPG) recognizes the in a new location, because he had to drive importance of strongly linking individual incen- further to work until he could move into his tive awards with performance. Annual Incentive new home. His letter of employment stated Plan (AIP) awards are based on individual, busi- that the allowance was “to offset some of the ness unit and corporate performance. As recom- hardships that he and his family may experi- mended by the Auditor General, OPG will assess ence with this move.” His file also noted that options to further reinforce this linkage. he could “live for free until the construction OPG’s management compensation is currently of his new home was completed.” Although at the 50th percentile (i.e., median) relative to payroll data showed that he received about the benchmark based on data from Canadian $17,000 in housing and moving allowances, organizations in both general and specific indus- the amount of total benefits he actually tries in sectors such as power generation/utilities,

Chapter 3 • VFM Section 3.05 received was close to $115,000 when other mining, petroleum/natural gas, and nuclear expenses such as groceries, meals out, car research, development and engineering. We rental and a car damage claim were included. have reduced total management compensation • Payroll data from 2009 to 2012 also showed since 2008. Compensation for OPG’s executives, that OPG spent about $1.4 million on average including vice presidents, continues to be frozen. each year on “miscellaneous” allowances, OPG has also reached collective agreements mainly for annual, non-pensionable “execu- with its unions that reflect government direction tive allowances” of various amounts ($30,000, regarding compensation constraints. $24,000, $20,000 and $12,000) depending on There are controls in place to ensure the executive’s income and length of service. employee salaries, benefits and pensions are in accordance with OPG policy, Canada Revenue RECOMMENDATION 2 Agency taxation requirements, and other legislation. As with any pension plan, retiring To ensure that employees receive appropriate employees are entitled by law to elect to receive and reasonable compensation in a fair and the commuted value of their pension in a single transparent manner, Ontario Power Generation lump-sum payment. As recommended by the should: Auditor General, OPG will continue to monitor Ontario Power Generation Human Resources 171

pensions in single lump-sum payments upon leav- and amend controls as needed to ensure coming. We noted in particular: pensation is justified and clearly documented. An employee who chose to receive his pension We acknowledge that OPG pension and • in a lump sum was rehired by OPG shortly benefits are higher than market average. As after he retired and continued to work at a result, in 2011, we completed a review of OPG for about six years. His total earnings pension and benefit plans to reduce costs and in his sixth year as a temporary employee improve sustainability. OPG also participated in were $331,000, which included an executive a 2012 pension reform committee established allowance of $12,000 and an AIP award of by the government, and will be participating in $98,200—double his annual amount as a the electricity sector working group, consisting regular employee. of employer and employee representatives, as Another employee who chose to draw his pen- announced in the 2013 Ontario Budget. • sion in a significant lump sum returned to work at OPG a month after his retirement. His total earnings that year as a temporary employee USE OF NON-REGULAR STAFF AND working three days a week were $328,000, CONTRACT RESOURCES which included an AIP award of $147,000 for his performance before retirement. Apart from regular employees, OPG’s other human Shortly after leaving OPG, two nuclear resources include non-regular staff (temporary • employees who chose to receive their pen- and contract), outsourced information technology sions in lump-sum payments were rehired as (IT) workers, and contractors from private-sector contract employees. vendors. Of particular concern to us were OPG’s We also found that selection processes and deci- practice of rehiring former employees, the IT sions to rehire former employees were not always outsourcing arrangement, and management of transparent: nuclear contractors. • All the temporary staff in our sample had been

selected and rehired by executive or senior Chapter 3 • VFM Section 3.05 Rehiring Former Employees as Temporary management staff without job postings or or Contract Staff competitions. OPG explained that these were unnecessary because only former employees There were approximately 1,700 temporary staff would have been suitable for the positions. and contract staff working for OPG in 2012. We Most of their original contracts were extended noted that about 120 of them had formerly been beyond 12 months with only a one- or two- regular employees. In our review of a sample of page document attached indicating the con- temporary and contract staff who were former tract length and terms but without specifying employees we found that most had been rehired why the contract needed to be extended. mainly for the purpose of identifying, grooming For the contract staff in our sample, justi- and training successors or meeting core business • fications for extending contracts beyond needs, suggesting that knowledge transfer and 12 months had been documented, but no succession planning at OPG has not kept pace with evaluations were kept on file. OPG explained attrition and retirement. We also found that almost that these were unnecessary because contract all of them had been rehired shortly after leaving employees who did not perform satisfactorily OPG. Some of them continued to receive significant could have their contracts terminated with- amounts in allowances and Annual Incentive Plan out any significant notice period or penalty (AIP) awards, and some had already drawn their payment. 172 2013 Annual Report of the Office of the Auditor General of Ontario

Many of the respondents to our survey expressed Although OPG did not go through an open- concerns similar to ours. They felt that rehiring competition process, its management did prepare a former employees on an ongoing basis was an “single-source justification” form, which indicated indication of poor succession planning. They also that renewing the contract would avoid transition felt that better processes should have been put into costs of $25 million and save $105 million from place to capture the knowledge and experience of 2009 to 2015, and identified labour relations as a retiring staff; to identify and train their successors factor that would make switching to a new vendor with sufficient lead time for the transition; and to unfavourable. OPG informed us that if it stopped avoid “double-dipping” by former employees who using the current vendor, it would have an obliga- had withdrawn their pensions in lump sums upon tion to reimburse the vendor for severance costs leaving OPG only to return and earn a salary again. associated with about 270 staff who are former In response to the above concerns, OPG indi- OPG employees. We note, however, that OPG is still cated that it was necessary to hire former employ- responsible for the severance costs whenever these ees and to pay them at higher rates because it was staff leave the vendor’s employ (for example, by difficult to find people with the right skills to fill the being laid off or retiring)—staying with the current positions right away, and that it could not influence vendor simply means the severance payout will not employees who wished to draw their pensions in be immediate. single lump sums before returning to work at OPG OPG’s management submitted its proposal to because this was a personal choice. renegotiate and renew the contract with the current vendor to its Board on October 1, 2009, and received approval on the same day. However, only Outsourcing of Information Technology after it received this approval did OPG start looking Services for consultants to validate and endorse the pro- OPG has been outsourcing its information technol- posal. Two consultants were engaged on October 6, ogy (IT) function to the same private-sector vendor 2009, and issued their final reports within a week. since February 2001, after it conducted a competi- There are good reasons for public-sector organ-

Chapter 3 • VFM Section 3.05 tive process and signed a 10-year (February 1, izations to use open competition rather than non- 2001–January 31, 2011), $1-billion contract with competitive approaches. Through open competition, the vendor. They formed a joint venture (owner- organizations can determine a fair market price for ship: 51% vendor and 49% OPG) for delivering IT the goods and services they require when a variety services to OPG, and 684 OPG employees (about of suppliers submit competitive bids, and this also 400 unionized) were transferred to the joint ven- helps demonstrate accountability and ensure value ture. A little over a year later, in March 2002, OPG for money. In addition, competition eliminates risks accepted the vendor’s offer of purchasing OPG’s associated with over-reliance on a single supplier share of joint venture ownership. and minimizes the perception of conflict of interest. In March 2007, OPG reviewed its existing By single-sourcing its IT services, OPG did not take outsourcing arrangement and decided to end the full advantage of these benefits. contract early in October 2009 and then renew it with the same vendor without competition for a Time Reporting of Nuclear Contractors term of six years and four months (October 1, 2009– January 31, 2016) at $635 million. Including the OPG uses Oncore, a web-based time management durations of the original and renewed contracts, the system, to track the hours and costs of nuclear total contract length is 15 years. contractors. It uses a three-step process to do this: Ontario Power Generation Human Resources 173

1) Each vendor has “contractor time entry super- recommendation noted, “[T]his system has visors” who input contractors’ paper timesheets the capability to generate Job Clock reports into Oncore; 2) OPG “contract administrators” that can be used by contract administra- verify and approve the timesheets in Oncore; 3) tors to reconcile time entered into Oncore OPG “contract owners” give final approval on the prior to approval.” However, we found that timesheets, which are then consolidated into an contract administrators often did not do so. invoice to be automatically paid by OPG. We reviewed about 2,600 hours reported by Oncore processed the hours reported by about contractors at sites where Job Clock was in 1,200 contractors in 2011 and 2,200 in 2012, with place and found that about half of them were associated labour costs of about $56 million in not supported by Job Clock reports. 2011 and $88 million in 2012. Overtime pay has • Overtime hours reported in Oncore were accounted for a significant percentage of the labour often not supported with documentation costs for contractors supplied by several large showing requests and approvals. OPG contract vendors, ranging from 19% to 43%. OPG indicated administrators told us that they either could that overtime was often a result of outages and not locate the documents or had approved emergent (unplanned or unscheduled) work. the overtime verbally. OPG also informed us We selected a sample of contractors and it had no standard method for documenting reviewed their hours in Oncore for one week in approval of overtime. 2012. The cost of labour for each contractor was high, ranging from about $8,000 to $12,000 per RECOMMENDATION 3 week. We noted that the hours in Oncore had not To ensure that its non-regular and contract always been reconciled with supporting docu- resources are used cost-efficiently, Ontario ments, which could lead to inaccurate time inputs Power Generation should: and overpayment to vendors. In 2010, OPG’s improve its succession planning, knowledge Internal Audit department identified a similar issue, • retention and knowledge transfer processes which it ranked as high risk and flagged for “prompt to minimize the need to rehire retired

management attention.” However, we found that Chapter 3 • VFM Section 3.05 employees for extended periods; OPG has not fully addressed this issue: conduct an open competitive process for out- In 2010, Internal Audit recommended “more • • sourcing its information technology services detailed information in the contract logbooks, before the current contract expires; and including the start and end times of work manage and monitor closely the hours activities, the contractor supervisors’ names • reported by the contractors to avoid the risk and titles, the applicable work orders and the of overpayment. contractor workers’ names. This information should be reconciled to the time submitted in Oncore.” We noted that the logbooks often ONTARIO POWER GENERATION did not contain these details. OPG informed RESPONSE us that the recommendation was never imple- Ontario Power Generation’s (OPG) contracting mented and it had no standard practice for practices are consistent with nuclear industry logging contractor activities. practices, which address both the need for • In 2011, in response to a 2010 Internal Audit specialized skills and demographic imbal- recommendation, OPG implemented a sys- ances of its workforce. Using the short-term tem called “Job Clock” to track contractor services of existing trained and skilled workers attendance and time spent on site. The also mitigates the need to hire a permanent 174 2013 Annual Report of the Office of the Auditor General of Ontario

staff claimed overtime in each of these years, earn- workforce during periods of transition or peak ing on average about $15,000 each in overtime work, resulting in substantial cost savings. As pay. The nuclear unit accounts for about 80% of recommended by the Auditor General, OPG OPG’s annual overtime costs; about half of these will review its practices related to rehiring were related to planned outages at nuclear facili- retired employees. ties, particularly Pickering. OPG conducted a competitive process when OPG’s overtime cost percentage (overtime costs we outsourced our information technology ser- divided by base salary) dropped from 16.2% in vices in 2001. Through an assessment of alterna- 2008 to 13% in 2011, but was slightly higher than tives initiated in 2007, and through third-party the averages (14.3% in 2008 and 12.1% in 2011) validation, we concluded that renewal under a of large utility companies in the U.S. According to significantly restructured contract would provide OPG, planned outages have been the main driver the most significant value to both OPG and rate- of its overtime costs because its outage periods payers. We plan to assess all potential options are generally much longer than those of its U.S. before the current contract expires, including an counterparts due to technical differences and dif- open competitive process that is consistent with ferent inspection requirements. the recommendation of the Auditor General. Although OPG’s overtime costs have been OPG concurs with the Auditor General on decreasing in recent years, its number of high the importance of accurate contractor payments overtime earners has increased significantly. Over and will investigate alternatives to manage and the last 10 years, the number of OPG employees monitor contractor hours. In 2012, we enhanced who earned more than $50,000 in overtime pay controls by implementing new contracting has doubled, from about 260 in 2003 to 520 in strategies and will be assessing further control 2012. The number of staff who earned more than opportunities with regard to time-tracking tools $100,000 in overtime pay has also grown consider- and the time-approval process. ably—in 2003 there was only one such employee, but by 2012 there were 33. Chapter 3 • VFM Section 3.05 OVERTIME Management of Overtime In its March 2011 decision, the OEB expressed OPG informed us that all overtime must be pre- concerns about the “extensive use of overtime, approved by a supervisor, who has the discretion to particularly in the nuclear division” at OPG and do so as long as his or her overtime budget has not said that it expected “OPG to demonstrate that it been exceeded. We looked at a sample of employees has optimized the mix of potential staffing resour- with high overtime pay and noted that 20% of ces.” In our review of staffing records, we found them had no supporting documents for overtime that management of overtime at OPG still required pre-approvals. We also noted that about one-third significant improvement. of the departments covered in our sample had exceeded their overtime budgets every year since Ten-year Overtime Trend 2009. In addition, each department used different methods of pre-approving overtime—some depart- Prior to the OEB’s decision, OPG’s overtime ments required paper overtime request forms to be costs rose steadily from $133 million in 2003 submitted and approved before any overtime hours to $169 million in 2010, and then dropped to could be worked, but in most departments verbal $148 million in 2012. About three-quarters of OPG approvals were sufficient. Ontario Power Generation Human Resources 175

We performed an analysis of overtime pay and staff sometimes treated overtime as an avenue to noted that OPG could improve its deployment of increase their pay. staff, especially for inspection and maintenance (I&M) technicians, who conduct regular inspec- RECOMMENDATION 4 tions and work on outages at nuclear stations. To ensure that overtime hours and costs are In our review of payroll data, we noted that I&M minimized and monitored, Ontario Power Gen- technicians consistently earned high overtime each eration should: year. For example, in 2012 the average overtime decrease overtime costs for outages by plan- pay for OPG’s 180 I&M technicians was more than • ning outages and arranging staff schedules $66,000 each, representing more than half of their in a more cost-beneficial way; and annual base salaries. review other ways to minimize overtime. OPG acknowledged that planned outages have • resulted in high overtime pay, especially for I&M technicians who are regular daytime employees ONTARIO POWER GENERATION but who are placed on schedules different from RESPONSE their normal hours during outages. Every hour they Nuclear outages are extremely complex projects work that is not one of their normal working hours that are planned and resourced two years in is considered overtime—even if they work none of advance. The scope of work may be affected by their normal hours. Their compensation for those emerging issues, unforeseen equipment condi- hours is one-and-a-half to twice their basic pay, tions and changes in regulatory requirements. depending on the days and times they worked. For The majority of overtime costs are associated example, we noted that the highest overtime earner with activities relating to these outages. Ontario at OPG in 2012 received $211,000 in overtime pay, Power Generation (OPG) continuously balances but his annual base salary had been reduced from the use of overtime versus contractors and $135,000 to $58,000 because when he was put considers the related amount of lost generation on an outage schedule he no longer followed his and revenue caused by extending the duration

normal schedule. His normal base hours therefore of the outage. Our overtime cost percentage is Chapter 3 • VFM Section 3.05 showed up as unpaid leaves and all the hours he comparable to large utility companies in the worked outside his normal schedule were paid at United States. the overtime rate. OPG will conduct a cost-benefit analysis to The collective agreement stipulates that OPG explore various ways, including scheduling and is responsible for preparing and administering hiring staff and/or contractors, to minimize outage schedules. According to OPG, there were overtime cost. about four or five planned outages each year at Pickering and it developed outage plans two years in advance to calculate the number of months each ABSENTEEISM year in which I&M technicians would be required to provide 24/7 coverage. Sick Leave Trend Many of the respondents to our survey felt that OPG’s sick leave plans are relatively generous com- the most common contributor to inappropriate and pared to those of the Ontario Public Service (see inefficient uses of overtime was poor planning and Figure 11). In particular, unionized staff who began scheduling. They also felt that outages could have working for OPG before 2001 are entitled not only been planned better by moving around shift sched- to carry over unused sick days from one year to the ules instead of using overtime, and that unionized 176 2013 Annual Report of the Office of the Auditor General of Ontario

next, but also to restore their used sick days every more than both the private (8.2 days) and utility five years. For example, an employee who took four (7.3 days) sectors. sick days in Year 1 will receive these four sick day credits back after five years of service in addition to Management of Sick Leave the normal number of sick leave credits he or she is entitled to for the year. As of December 31, 2012, We noted that some of OPG’s key sick leave man- about 5,200 employees—or almost half of OPG’s agement programs were not being used as effect- staff—were still under the old plan. On average, ively as they could be. While we noted no abuses of each of them has restored and accumulated 162 sick leave credits in our sample testing, a significant sick leave credits with full pay and 191 sick leave accumulation of sick leave credits is possible, lead- credits with 75% pay. Unused credits are not paid ing to a higher risk of abuse if these programs are out on termination or retirement. not used effectively. The average number of sick days taken per OPG The Short-Term Absence Management Pro- employee, including both short-term absences and gram is in place to identify the medical reasons major medical absences, has gone up 14% (from for an employee’s absence pattern. Supervisors 9.2 days in 2003 to 10.5 days in 2012). Direct costs are expected to regularly examine their staff’s associated with sick days have grown significantly, attendance records; if an employee’s sick leave by 41% (from $29 million in 2003 to $41 million usage is above the business unit’s standard, they in 2012). OPG informed us that sick days and are to meet with the employee to discuss the right their associated costs have gone up because of the course of action and document the outcomes. 12-hour shift arrangement that is followed by most We reviewed the files of a sample of employees of OPG’s nuclear staff—if a 12-hour shift worker whose sick leaves were above the business unit misses a shift because of illness, it is counted as 1.5 average from 2009 to 2012 and found no docu- sick days. Compared to other sectors, the average ments indicating whether their supervisors had number of sick days taken per employee at OPG met with them and what the outcomes had been. was fewer than the public sector’s 12.9 days but OPG explained that it had no formal requirements Chapter 3 • VFM Section 3.05 Figure 11: Sick Leave Plans at OPG vs. Ontario Public Service (OPS) Sources of data: Ontario Power Generation, Ministry of Government Services

OPG Unionized Staff Old Plan New Plan Non-unionized OPS (Staff hired before 2001) (Staff hired in or after 2001) Staff Annual entitlement (100% pay) 6 days 8 days 8 days 130 days Annual entitlement (75%) 6 months 15 days 6 months No Accumulation of unused sick days No Indefinitely with no limit1 Indefinitely with no limit1 No (100% pay) Accumulation of unused sick days Indefinitely with No No No (75% pay) a limit of 200 days1 Restoration of used sick days No Yes2 No Yes3

1. Unused sick day credits are not paid out on termination or retirement. 2. After five years of service, sick day credits used in the first year are restored. From the sixth through fourteenth years, sick day credits used in the five previous years are restored. On the fifteenth year, sick day credits used before the second-last year of service are restored. After that, sick day credits used in the second-last year are restored annually. Unused sick day credits are not paid out on termination or retirement. 3. After one month back to work, the number of sick day credits will increase back to 130 days. Ontario Power Generation Human Resources 177 for this documentation to be retained as official records. After we completed our audit fieldwork, ONTARIO POWER GENERATION OPG informed us that it was implementing a new RESPONSE program with more stringent requirements. Ontario Power Generation (OPG) is committed OPG’s Disability Management Program is in to having a healthy and productive workforce place to ensure that employees are fit to do their while minimizing sick leave costs. The aver- job after longer periods of sick leave (four or more age number of days lost through short-term consecutive days for PWU staff and five or more absences in 2012 was approximately five for Society and non-unionized staff). Supervisors days per employee, excluding major medical are expected to notify OPG’s staff nurse about the absences. As recommended by the Auditor absences and employees must submit a Medical General, OPG will review its sick leave plans Absence Report completed by a physician within and assess the costs and benefits of any changes 14 days of their first day off sick. We reviewed the that are required through collective bargaining. files of a sample of employees with longer sick leave OPG will continue the Business Transforma- absences since 2010 and noted that 55% of the tion efforts already under way to minimize the employees in our sample should have filed Medical costs associated with sick leave by proactively Absence Reports, but almost half of them had not supporting employees in improving and done so on at least one occasion. OPG informed us maintaining their health, while implementing that the requirement might be waived for recurrent processes and tools such as the automated absences caused by chronic disease. employee absence calendar to assist managers OPG has an automated employee absence cal- in effectively managing sick leave issues. endar to help managers identify unusual sick leave patterns. However, more than half of the respondents to our survey said they were not aware of the STAFF TRAINING calendar or did not use it, and another quarter of them said they used the calendar only infrequently In 2012, OPG centralized its staff training into a sin-

(annually or quarterly). OPG informed us that some gle business unit called Learning and Development Chapter 3 • VFM Section 3.05 managers used the calendar more frequently than (L&D). Before then, staff training had been man- others, depending on the types of absences and the aged separately by each functional area: nuclear, size of the department or group. hydro/thermal and corporate support. At the time of our audit, OPG had about 290 L&D employees RECOMMENDATION 5 and its training costs for 2012 were $127 million. About half of this amount was for developing train- To minimize the cost of sick leaves and avoid ing materials, delivering courses, paying trainers, potential misuses or abuses of sick leave entitle- managing training records, administering tests, and ments, Ontario Power Generation should: maintaining training simulators and equipment; review its sick leave plan for staff who joined • the other half was for paying workers’ salaries while prior to 2001; and they attended training. • monitor the results of sick leave management programs to identify and manage unusual sick leave patterns. Nuclear Training

OPG provides training to about 7,000 nuclear staff at two learning centres, Pickering and Darling- ton. OPG’s Nuclear Oversight and Performance 178 2013 Annual Report of the Office of the Auditor General of Ontario

Improvement Department oversees the training which OPG believed was preventing it from along with two external organizations, the Can- attracting good candidates. adian Nuclear Safety Commission (CNSC) and the • The completion rate for the ANO training pro- World Association of Nuclear Operators (WANO), gram at OPG has been around 56%, which was who both routinely send out inspection teams to below both its own workforce planning goal review OPG’s nuclear training programs. Both (70%) and Pilgrim’s completion rate (75% ). internal and external reviews help OPG’s manage- We noted some additional areas to address in ment identify areas for improvement and report on our review of OPG’s nuclear training: whether OPG’s nuclear training programs adhere • Only one of OPG’s 19 NLO trainers was a to applicable standards and requirements. Supervisory Nuclear Operator, considered The majority of OPG’s nuclear staff are nuclear by OPG to be the ideal position for an NLO operators who fall into two main categories: non- trainer. Two other trainers had worked as licensed operators (NLOs) and authorized nuclear nuclear operators for only one year. operators (ANOs). NLO candidates must undergo • An ANO can go through additional training a 24-month training period. To become an ANO, a to become a Control Room Shift Supervisor candidate must be a fully qualified NLO for at least (CRSS). The completion rates for CRSS one year and then complete a 36-month training training programs in 2012 at Darlington and period. At the time of our audit, OPG had about Pickering were 0% and 57%, lower than the 950 NLOs and 160 ANOs. The minimum education industry completion rate of 60–65%. OPG required to become a nuclear operator in Ontario is informed us that the length of the CRSS train- completion of Grade 12 with university-preparation ing program (32 months) has contributed to course credits in math, physics and chemistry. low completion rates. Accordingly, the training that OPG provides is necessary to ensure that nuclear operators are suf- Hydro/Thermal Training ficiently prepared for the job. In 2012, the average annual earnings at OPG for NLOs and ANOs were OPG delivers training to about 2,000 hydro/ther-

Chapter 3 • VFM Section 3.05 $112,000 and $207,000, respectively. mal staff at the Etobicoke learning centre and at To identify best practices and opportunities hydro and thermal stations across Ontario. Unlike for improvement, OPG benchmarked its NLO and the nuclear sector, there is no regulatory oversight ANO training programs against those at the Pilgrim of hydro/thermal training, and OPG’s training in Nuclear Station in Massachusetts (Pilgrim) in Sep- this area has never been evaluated by itself or third tember 2012. OPG informed us that it has prepared parties. We identified the following issues related to improvement plans to address the following issues staff training requirements and course attendance identified in the benchmarking study: in our review of hydro/thermal training: • OPG’s NLO training program was not well- • In 2012, 30% of the courses OPG requires structured, class sizes were larger and training had not been completed. OPG informed us material was not as comprehensive. that even if a training course was recorded as • OPG’s NLO trainers had varying levels of required in the database, supervisors might qualifications, experience and ability. not send their staff to training if they felt there • OPG’s NLO trainees generally lacked hands- was no immediate need for them to learn a on experience in any industry and lacked specific skill set. discipline. • In June 2010, OPG’s Hydro/Thermal Training • OPG’s ANO training program was lengthy Decision Making Committee raised a concern (32 months versus 16 months at Pilgrim), about last-minute cancellations of scheduled Ontario Power Generation Human Resources 179

courses and recommended that plant man- review the nature and timing of its manda- agers should try to reduce them to optimize • tory training requirements as well as its the use of training resources. This was still an delivery methods for hydro/thermal staff issue at the time of our audit. In 2012, about to ensure they are meeting business needs 4,500 of 21,000 scheduled courses for trainees cost-effectively. had been cancelled. No reasons were given for about 1,400 of the cancellations; the remaining had been cancelled for reasons such as ONTARIO POWER GENERATION employee no-show, illness, or pre-approved RESPONSE vacation day, among others. We also noted Ontario Power Generation’s (OPG) nuclear similar course cancellation patterns for 2011. training programs are extensively benchmarked against industry best practices and are routinely RECOMMENDATION 6 audited by the Canadian Nuclear Safety Commis- sion and the World Association of Nuclear Oper- To ensure that its employees are adequately ators. OPG is in the process of implementing trained for their jobs, Ontario Power Generation enhancements to its nuclear training programs should: where there are opportunities for improvement continue to review and monitor the • while continuing to build on identified strengths. adequacy, quality and completion rates of its As recommended by the Auditor General, OPG nuclear training programs in order to iden- will continue with its review of the nature, tim- tify areas for improvement, and address the ing and delivery methods of mandatory training areas that have already been identified; and requirements for hydro/thermal staff. Chapter 3 • VFM Section 3.05 Chapter 3 Ministry of Education Section 3.06 Private Schools

the Ontario secondary school diploma (OSSD). Background The Ministry conducts program inspections at only those registered private schools that offer credits toward an OSSD. The programs offered at non- The purpose of education, as stated in the Educa- credit-granting schools are not inspected by the tion Act (Act), is to provide students with the oppor- Ministry. The number of credit-granting and non- tunity to realize their potential and develop into credit-granting private schools in Ontario is shown highly skilled, knowledgeable, caring citizens who in Figure 1. contribute to society. The Act states that every child Parents choose to send their children to private who attains the age of six years shall attend a public schools for a variety of reasons, such as the school school unless that child is receiving satisfactory offers an educational approach that may better instruction at home or elsewhere. Private schools suit their children, the school reinforces the reli-

Chapter 3 • VFM Section 3.06 are considered one of the alternatives to public gious practices of the home, or they believe that education and are defined in the Act as institutions private schools achieve better academic results. that provide instruction between 9:00 a.m. and The actual academic results of standardized test- 4:00 p.m. on any school day for five or more school- ing suggest that the quality of education provided age pupils in any of the subjects of the elementary by participating private schools varies from well or secondary school courses of study. below average to excellent. Not only do Ontario All private schools are to be registered with private schools range in quality from well below the Ministry of Education (Ministry). During the average to some of the best schools in Canada; 2012/13 school year, there were over 1,000 regis- they also vary significantly in size from sometimes tered private elementary and secondary schools in less than the minimum five students to enroll- Ontario that informed the Ministry that they had ments of well over 1,000 students. enrolled approximately 110,000 students. These Private school tuition fees generally range schools are considered to be independent organiza- from $5,000 to $20,000 but can be significantly tions, are not required to follow policies developed more. The Ministry does not provide any funding for publicly funded schools (those schools in but, given that publicly funded education exceeds either English or French district or Catholic school $10,000 per student per year, private schools in boards), and are not required to follow the Ontario effect either save the taxpayers over $1 billion curriculum unless the school offers credits toward annually or enable the Ministry to allocate this

180 Private Schools 181

Figure 1: Registered Private Schools, 2012/13 School Year Source of data: Ministry of Education

OSSD Credit-granting Non-credit- Schools1 granting Schools2 Total Elementary 0 517 517 Combined elementary and secondary 169 73 242 Secondary 239 15 254 Totals 408 605 1,013

1. The Ministry performs OSSD program inspections at credit-granting schools. 2. No ministry inspections are performed at non-credit-granting schools. amount to other education priorities. Accordingly, Our audit work was conducted at the Ministry’s a strong private school system can benefit the head office and at selected regional offices with the taxpayers as well as both public and private school responsibility for overseeing approximately 80% of students. private schools. We reviewed and analyzed ministry Although the Ministry focuses on the delivery files, administrative directives, and policies and of publicly funded education, section 16 of the procedures, and interviewed ministry staff. We Act provides direction to the Ministry and to pri- also met with staff at the regional offices, including vate schools regarding their legislated roles and education officers responsible for validating and responsibilities with respect to establishing private inspecting private schools, but we did not include schools, ministry inspections of credit-granting audit visits to private schools in the scope of our schools, data collection and student testing. Min- audit because they are not provincial grant recipi- istry functions are performed by three full-time and ents. We researched private school oversight practi- one part-time head office staff assisted by 24 educa- ces in other jurisdictions and solicited the opinions tion officers (inspectors) located in six regional of universities, the Ontario Universities’ Application

offices who devote about 20% of their time to pri- Centre, Ontario College Application Service and Chapter 3 • VFM Section 3.06 vate schools. Based on this allocation of personnel, several private school associations. the Ministry has the equivalent of about eight staff devoted to overseeing private schools in Ontario. Summary

Audit Objective and Scope Ontario has one of the least regulated private school sectors in Canada. Consequently, on its web- The objectives of our audit were to assess whether site, the Ministry cautions parents to exercise due the Ministry had adequate procedures in place to: diligence before entering into a contract to educate • assess the effectiveness of private schools in their children at a private school. The Ministry providing satisfactory instruction; and provides very little oversight to ensure that private • ensure compliance with the Education Act and school students are receiving satisfactory instruc- related ministry policies. tion. In fact, although private school results vary Senior ministry management reviewed and greatly, we found that public school students on agreed to our audit objectives and associated audit average performed significantly better on standard- criteria. ized tests than private school students. In addition, 182 2013 Annual Report of the Office of the Auditor General of Ontario

although the Ministry inspects the standard of accredited by the Ministry. Parents, students instruction at the 408 private schools that offer and the public could be misled into thinking high school diploma credits, at 100 of these schools that the Ministry ensures some level of educa- it noted significant concerns, many of which related tion quality at these schools. We also found to credit integrity, meaning whether a student several examples of entities advertising what actually earns the credits granted toward his or her appeared to be private school services with- grade 12 diploma. For the 605 elementary and non- out being registered with the Ministry. The credit-granting secondary schools, education offi- Ministry does not have procedures in place to cers perform a brief visit to new schools, but there proactively identify unregistered schools that is no process in place to ever visit these schools are operating illegally. again. In addition to academic concerns, this poses • Ministry data indicates that 235 private the risk that some private schools may be operating schools ceased operations over the last five unlicensed child-care centres. According to ministry school years (2007/08–2011/12), often as information, there may be more than 15,000 chil- a result of declining enrolment or financial dren in private schools below compulsory school problems. Private schools are not required to age, with as many as 3,000 below the age for junior demonstrate that they are financially viable kindergarten. operations. Closures during the school year Our other significant observations include the could put students at risk academically and following: their parents at risk financially. In addition, • All private schools are required to submit closed schools must forward student records a notice that they intend to operate in the to the Ministry to ensure that essential infor- coming year. For new schools the Ministry mation on students is preserved, but fewer conducts a brief validation visit to check the than half the schools we sampled had done so. information submitted and confirm that the • In Ontario, anyone who cares for more than school meets the legal definition of a private five children under the age of 10 must be school. During these visits the Ministry does licensed under the Day Nurseries Act. The Min-

Chapter 3 • VFM Section 3.06 not evaluate the curriculum for either quality istry allows private schools registered before or content; does not check for any health and June 1993 to operate child-care facilities with- safety issues, or have a process in place to out a licence. In contrast to licensed daycare, inform other oversight agencies of any con- there is no limit to the number of children of cerns observed; and, in contrast to its practice any age that private school staff can oversee, with public schools, does not ensure that there are no fire safety requirements, and criminal record checks are performed on pri- private school staff are not required to possess vate school operators, teachers or staff. Except any child-care qualifications. The Ministry for this one-time visit, the Ministry provides inspects licensed child-care facilities annually. almost no oversight of private elementary However, after their first month of operations, schools or secondary schools that do not offer the Ministry may never visit private elemen- high school credits. tary schools again. • Given the limitations of the validation pro- • The Ministry inspects the standard of instruc- cess, private schools are not permitted to tion in the 408 private schools that offer state that the Ministry has approved their credits toward the OSSD. About 100 of these academic program. However, we identified schools are inspected more frequently than several cases where private schools were the others because of issues that may indicate advertising that their programs had been credit integrity concerns. For example, it Private Schools 183

was brought to the Ministry’s attention that penalties is not economical, as legal costs to some private schools were issuing students find someone guilty outweigh the fines that higher grades than earned or giving credit for would be collected. For example, a private courses that students had not attended. The school that does not submit the required sta- Ministry has developed additional proced- tistical information, on conviction, is liable to ures to investigate such practices, but many a fine of not more than $200. inspectors informed us that they did not have • The Education Quality and Accountability sufficient time to perform these supplement- Office (EQAO) helps to ensure satisfactory ary procedures. instruction by testing all students in the public • Approximately 250 private schools had still school system at grades 3, 6 and 9. Some pri- not submitted the required information on vate schools participate in EQAO testing, and their students for the 2011/12 school year all private school students pursuing an OSSD by June 2013, a full year after the school must write the EQAO’s Ontario Secondary year had ended. For data that is submitted, School Literacy Test (OSSLT). We reviewed the Ministry has no process in place to verify EQAO test results and noted that a greater its accuracy and relies on the good faith of percentage of public school students achieved private school administrators. For the public the provincial standard than private school school system, the Ministry analyzes such students. In addition, in 2012, 82% of public data to determine if students are receiving school students passed the OSSLT on the first satisfactory instruction and progressing aca- attempt, compared to 73% of private school demically. However, the Ministry has not done students. The results for a sample of these any such analysis for private school students. private schools varied considerably, from an • The Ministry has exclusive authority to grant overall school pass rate of 19% to 100%. The the OSSD. To help prevent diploma fraud and Ministry does not analyze such EQAO results ensure control over blank diplomas pre-signed to determine if students in private schools are by the Minister of Education, the Ministry rec- receiving satisfactory instruction.

onciles public schools’ requests for diplomas Chapter 3 • VFM Section 3.06 to grade 12 student enrolments. However, OVERALL MINISTRY RESPONSE this procedure has not been applied to private We agree with the recommendations of the schools. In fact, the Ministry provided thou- Auditor General and have given thorough con- sands of diplomas to private schools without sideration to their implementation. To support identifying for whom these diplomas were a foundation of a fair, productive and socially intended. For example, for the 2011/12 school cohesive society, the Ministry of Education’s year, 30 private schools were issued a total of three priority goals are high levels of student 1,500 more diplomas than their grade 12 stu- achievement, reduced gaps in student achieve- dent populations, and 50 other private schools ment and increased public confidence in public were issued 2,300 diplomas even though they education. had not submitted any student enrolment data The Ministry acknowledges the right of by June 2013. parents and students to choose a source of The Ministry informed us that it has not • education outside the publicly funded system, sought prosecution for any offence under the whether for religious, cultural or other reasons. Education Act against any private schools or This audit report states that approximately individuals associated with these schools. 110,000 students attend private schools in It stated that enforcing compliance through 184 2013 Annual Report of the Office of the Auditor General of Ontario

of the quality of instruction and evaluation of Ontario, representing about 5% of the 2 million student achievement; a principal in charge of the children attending publicly funded schools. The school; a common school-wide evaluation policy; a Ministry’s resources are deliberately focused common procedure for reporting to parents; a com- on ensuring that high-quality, publicly funded mon school-wide attendance policy; and a central education is available to every Ontario student. office to maintain student records. However, the Ministry will take appropriate steps to expand initiatives to provide information regarding consumer awareness about the Validating New Private Schools private school sector. When the Ministry receives a notice of intention to Private schools in Ontario operate outside operate a new private school, an education officer the publicly funded education system as conducts an unannounced validation visit within independent businesses or non-profit organiza- the first month of the school’s operation. This visit tions, and unlike those in many other provinces, is to verify that the information contained in the they receive no public funding or financial notice of intention form is correct, that the school assistance. The Education Act does not provide meets the legal definition of a private school and the Ministry with oversight and monitoring that the Ministry’s general requirements for a responsibilities with respect to the day-to-day private school are in place. If a private school meets operations of private schools. However, the these validation requirements, the school principal Ministry inspects private secondary schools that is so informed and the education officer will recom- wish to offer credits toward the Ontario Second- mend the school for registration. The school’s name ary School Diploma. will then be added on the Ministry’s website to the list of private schools currently operating in Ontario. Over the past three years, the Ministry has received notices from 275 prospective private Detailed Audit Observations schools and has registered 190. The Ministry has

Chapter 3 • VFM Section 3.06 not tracked the reasons for which the remaining 85 were not registered. We reviewed a sample of min- ESTABLISHING AND MAINTAINING istry records for these schools and found that either STATUS AS A PRIVATE SCHOOL they could not meet the enrolment requirement of The Education Act (Act) requires all private schools five students or they notified the Ministry during to submit a notice of intention to operate to the the validation visit that they did not plan to operate Ministry of Education by the first of September each in the current school year and were no longer seek- year. The Ministry prescribes the form and content ing authority to register. of this notice and requests information such as the We reviewed a sample of validation reports school’s name and contact information, its principal for schools that were registered and discussed the and its owners, its hours of operation, its projected process with several education officers. We noted and actual enrolment, any religious affiliations, any that, although education officers complete a stan- memberships in private school associations, and dard validation report template that typically asks whether the school intends to offer Ontario second- questions requiring “yes” or “no” responses, they ary school diploma credits in the coming school year. generally do not retain supporting documentation The Ministry has also established seven general from their visits or record their procedures. We requirements for private schools: control over the found that, to ensure a prospective school meets the content of the program or courses of study; control statutory definition of a private school, education Private Schools 185 officers count the number of students present and courses offered in private schools. In Prince Edward verify their ages against registration information to Island, private school programs of study must be ensure that the school has five students of compul- approved by the minister, and no private school can sory school age. To confirm that schools are open change its program of study without the minister’s during the legislated hours of operation, education prior written approval. officers review timetables and conduct their valida- Other provinces in Canada also require their tion visits between 9:00 a.m. and 4:00 p.m. education ministries to specifically review health The validation process also requires educa- and safety conditions. For example, British Colum- tion officers to determine if there is evidence that bia private schools must maintain adequate educa- the general requirements for a private school are tional facilities, and Manitoba inspectors annually present. However, there are no criteria or specific ensure that the space chosen for a private school procedures in place to guide them in their assess- is suitable for teaching and learning, has passed a ment of these requirements. We were informed building and fire inspection, and meets all health that education officers simply search for evidence regulations. In Prince Edward Island, a private that the general requirements exist, but they do not school must provide the education department with evaluate how effectively these requirements have evidence that it meets the health, fire and safety been implemented. For example, one of the general standards established by the province. There are no requirements is “control of content of the program similar legislative requirements for private schools or courses of study.” The Ministry has defined this in Ontario, although some other provinces base requirement for secondary schools offering high funding to private schools on adherence to prov- school credits as the delivery of the Ontario cur- incial guidelines for curriculum, teacher qualifica- riculum but, for non-credit-granting schools, the tions, health and safety, and other requirements. Ministry has defined “control of content of the pro- In Ontario, education officers noted that during gram or courses of study” as a full-day day-school validation visits they sometimes identify health and program. To verify that non-credit-granting schools safety concerns such as inadequate washroom facili- have met this requirement, some education officers ties, a lack of fire exits or classrooms that appear

inquire into what programs are being taught, while too small for the number of students being taught. Chapter 3 • VFM Section 3.06 others review the school’s textbooks. Education However, these schools are still recommended for officers noted that a validation visit takes from as registration and are allowed to operate, since the little as 30 minutes to half a working day, and all education officers stated that they do not have the the officers we interviewed stated that, at private authority to deny registration of a private school elementary schools and secondary schools that do based on health and safety issues. Furthermore, not offer diploma credits, they do not evaluate the although education officers may inform private curriculum for either quality or content. school principals of any major health and safety con- In contrast to Ontario, to varying degrees, many cerns observed, there is no formal process in place private schools in other Canadian provinces are to document these concerns or to inform oversight required to follow an approved curriculum. All Que- agencies such as public health, the fire department bec private schools must adhere to the curriculum or children’s aid societies. In addition, at publicly established by the province’s ministry of education. funded schools in Ontario, for the purpose of ensur- In Manitoba, private schools do not need to follow ing students’ safety, all teachers, staff and service the provincial curriculum, but they must deliver the providers who come in contact with students must same standard of education that is provided in a undergo a criminal background check. Education public school. In Newfoundland and Labrador, the officers are not required to ensure that a criminal minister of education prescribes or approves the background check has been performed on private 186 2013 Annual Report of the Office of the Auditor General of Ontario

school operators, principals, teachers or staff as does not verify that the general requirements for a there is no legislative requirement for private private school are present, and non-credit-granting schools to perform such checks. schools are not required to provide any information on how they continue to meet these requirements. Prior to September 2012, the notice of intention Notice of Intention to Operate Existing form required private schools to submit only their Private Schools projected enrolment for the coming school year. The Ministry issues the same notice of intention This self-reported information was not sufficient form for new and existing schools and requests to ensure that private schools met the statutory the same information, including the number of definition of having at least five students, because students enrolled and the hours of operation. actual enrolment may in fact have been less. In Although we found that all the schools we sampled September 2012, the Ministry began to request had a valid notice of intention form on file, the that the notice of intention form include the actual Ministry does not perform a validation visit or enrolment for the previous year in addition to the otherwise confirm that the information submitted projected enrolment. by existing private schools is correct. This informa- The Ministry also requires private schools to tion is self-reported by the school and the Ministry submit student information to be input into its accepts it if the school declares, for example, that Ontario School Information System (OnSIS). We it has at least the five students required to meet the reviewed data collected through OnSIS and found definition of a private school. In fact, as long as a that several schools reported actual enrolment of private school continues to submit notice of inten- fewer than five students, with one school reporting tion forms, its name will continue to be published fewer than five students for six consecutive years. on the Ministry’s website listing of private schools We selected a sample of these schools and found currently operating in Ontario. that the enrolment reported in OnSIS did not cor- The Ministry performs inspection visits to the respond to enrolment reported on the notice of 408 secondary schools that offer high school credits intention form for any of the samples selected. For

Chapter 3 • VFM Section 3.06 and may observe, for example, that the required example, one school reported having an actual stu- minimum of five students are enrolled. However, dent enrolment of 20 students on its notice of inten- education officers do not specifically verify the tion form but reported only one student through information recorded on the notice of intention OnSIS. Education officers do not have access to forms during inspections. According to ministry OnSIS information and therefore cannot identify records, 85% of the 605 elementary and non- and follow up on such discrepancies. The Ministry credit-granting secondary schools began operations stated that schools with fewer than five students before 2010, with the majority of these schools are allowed to operate but are given notice that established more than 10 years ago. We selected their enrolment has declined below the minimum a sample of these schools and confirmed that they required by the Act and that they are in jeopardy of had not been visited by the Ministry since their losing their status as private schools. However, the establishment. Education officers confirmed that Ministry could not provide us with a list of which they are not required to, and have not, revalidated schools had been so informed or evidence that it any private schools, as the Ministry does not have has revoked the registration of any schools that did a policy to revalidate the information submitted on not meet the legislated minimum of five students. the notice of intention forms. Furthermore, subsequent to the first year of operations, the Ministry Private Schools 187

Private Schools with More Than One Private School Closures Location The Ministry has indicated that a school could lose The notice of intention requires a private school its status as a private school if it does not meet the to provide data for only its primary location. As definition of a private school, submit a notice of a result, several private schools are operating intention form by the stated deadline or submit additional locations that are undocumented by the required statistical information. Ministry data the Ministry. In 2010, a one-time request was sent indicates that 235 schools have ceased operations to private schools requesting information on any and lost their status as private schools over the last locations in addition to the schools’ main sites. five school years (2007/08–2011/12). The Ministry Through this exercise, 117 private schools reported does not track the reasons for which private schools that they were operating at 180 additional loca- close, but we found that in the majority of cases tions. The Ministry had no previous knowledge sampled, schools closed before the Ministry had of the existence of many of them. Eighty-seven discovered that they were no longer in operation. of these locations were offering diploma credits. Many schools were deemed closed by the Ministry Since this information was self-reported by private because they had not submitted their annual notice schools without any validation or verification by of intention form by the required deadline. Other the Ministry, there is the potential that even more schools were identified as being closed when an private schools are operating at additional locations education officer attempted to perform an inspec- without the Ministry’s knowledge. tion but discovered that the school was no longer One private school that was authorized to grant operating. Only in very few cases did private credits was operating a second location that had schools notify the Ministry of their intent to cease not been previously inspected by the Ministry. operations. In most cases where information was When the Ministry was made aware of this location available in the Ministry’s files, the schools had an inspection was conducted. The Ministry identi- indicated that they had ceased operations due to fied several compliance issues at the second loca- declining enrolment and financial problems. tion, including assessment and evaluation practices Financial information is not required to be that were not based on ministry policy. These issues submitted with the notice of information form, and Chapter 3 • VFM Section 3.06 were not identified as concerns at the school’s main private schools are not required to demonstrate that site. As a result, the Ministry denied the second they will be financially viable operations. Schools location the authority to grant credits. that close during the year could put students at risk Although some private schools have subse- academically and their parents at risk financially. quently submitted additional notices of intention In contrast, students in Ontario’s private career col- for each location, at the time of our audit the Min- leges are protected financially and academically, as istry had not taken any formal action with regard every college is required by legislation to prove it is to the additional locations identified in 2010, had financially viable and is required to deposit money not implemented an action plan to validate all into a fund to help its students find alternative proof these previously unknown locations, and had grams in the event the college ceases operations. not inspected the 87 locations that offered credits Private schools that cease operations must for- toward a diploma. The Ministry informed us that it ward student records to the Ministry to ensure that will begin requesting that private schools list addi- essential information on their students is preserved. tional locations on the 2013/14 notice of intention Although the Ministry sends letters to private forms. schools that have closed informing them of this obligation, it does not perform any additional follow-up 188 2013 Annual Report of the Office of the Auditor General of Ontario

if the closed school does not forward student files or public. We found several more examples of entities does not respond to the letter. We selected a sample advertising what appeared to be private school of schools that were identified as being closed and services. These entities were once known to the determined that fewer than half of these schools Ministry, as they were either formerly registered had forwarded student records to the Ministry. private schools that were no longer submitting notices of intention to operate, or prospective private schools that had not completed the validation Program Advertising and Unregistered process. The Ministry does not have procedures in Private Schools place to proactively identify private schools that are Private schools are not permitted to claim that the operating illegally. Ministry has approved or accredited their academic program. However, the Ministry does not have RECOMMENDATION 1 a process in place to ensure that private schools To help ensure that private school students are complying with these advertising guidelines. receive satisfactory instruction in a safe and Through Internet searches, we identified several healthy environment and to ensure compliance cases where private schools listed on the Ministry’s with ministry policy and legislation, the Min- website were advertising that their programs had istry of Education (Ministry) should: been accredited by the Ministry. Parents, students enhance the notice of intention and valida- and the public could be misled into thinking that • tion processes to require private schools to the Ministry has evaluated and approved the cur- demonstrate that their students are receiving riculum of private schools whose names are listed. satisfactory instruction; This is especially a concern for elementary schools notify the appropriate authorities of any and secondary schools that do not offer credits • health and safety concerns observed during toward the high school diploma, as the Ministry onsite school visits; does not evaluate or approve any aspect of the cur- revalidate private schools annually or on a riculum being taught at these schools. The Ministry • cyclical basis to ensure that information pro-

Chapter 3 • VFM Section 3.06 recognizes that the annual submission of a notice vided is correct and to revoke the authority of intention form provides little accountability and to operate for those schools that do not meet assurance that a school is providing quality educa- the definition of and general requirements of tion, and notes on its website that private schools a private school; operate independently and that their inclusion on provide education officers with access to the the site is not an endorsement of the schools. The • Ontario School Information System to, for site, in a buyer beware fashion, cautions parents to example, reconcile and validate enrolment; perform due diligence before entering into any con- identify all private school locations and tract with a private school. However, the Ministry • verify that all locations comply with ministry does not have a link on its website to easily access policy and legislation; this caution or any other information related to ensure that closed schools forward all stu- private schools. • dent records to the Ministry as required; and Over the last three years, the Ministry has develop a process to proactively identify issued nine cease and desist letters to organizations • schools that are not complying with the for false advertising or for claiming to be private advertising guidelines or are operating schools when they were not registered with the illegally without being registered. Ministry. All of these cases were identified through complaints made to the Ministry by members of the Private Schools 189

MINISTRY RESPONSE PRIVATE SCHOOLS WITH DAY NURSERIES The Ministry agrees that students should receive In Ontario, any facility that cares for more than five instruction in a safe and healthy environment, children under the age of 10 who are not of com- and will assess options to ensure that private mon parentage must be licensed by the Ministry schools with credit-granting authority document under the Day Nurseries Act. The Day Nurseries Act compliance with provincial health and safety also states that a day nursery is not a private school legislation, and options to empower education as defined in theEducation Act, which defines a pri- officers to notify the appropriate authorities of vate school as an institution at which instruction is any suspected violations of health and safety laws. provided to five or more pupils who are of compul- The Ministry will continue to expand sory school age (5 years, 8 months to age 18). The and refine the collection and analysis of data Ministry requires newer private schools that enroll received from private schools through the five or more children under junior kindergarten age annual Notice of Intention to Operate a Private (3 years, 8 months) to be licensed under the Day School form, and the use of this information Nurseries Act. However, pursuant to a policy issued during the validation and inspection processes. by the Ministry of Education and the Ministry of The form has been updated for the current Community and Social Services, private schools 2013/2014 school year, requiring private offering services to children under junior kinder- schools to provide information regarding any garten age at the time the policy took effect (June educational programs being operated at “cam- 1993) would be allowed to continue offering those pus” locations other than the main school site. services without a licence. The Ministry could not This is part of the Ministry’s policy to eliminate provide us with an accurate number of preschool the operation of campus locations by private children in private schools. schools with credit-granting authority, effective The Day Nurseries Act requires licensed daycares September 2014. In addition, the Ministry has to comply with a comprehensive list of standards directed education officers to inspect any addi- for the health, safety and developmental needs of

tional locations of private schools with credit- Chapter 3 • VFM Section 3.06 children. For example, daycare staff can oversee granting authority identified through this form. only a limited number of children, with more staff The Ministry is working to develop a profile required for younger children; the local fire chief for each private school with credit-granting must approve fire evacuation plans; and super- authority in Ontario, as a tool to provide educa- visory staff must hold a diploma in early childhood tion officers conducting inspections with data education. In contrast, there is no limit to the num- collected through the Ontario School Informa- ber of children of any age that private school staff tion System. can oversee, there are no fire safety requirements, The Ministry maintains a list on its public and private school staff are not required to possess website of all private schools currently operat- any qualifications. Such a disparity in requirements ing in Ontario, including information regarding may give private schools a significant economic a school’s credit-granting authority. The advantage over licensed child-care providers while Ministry will review the public website content exposing preschool children to greater risk. with a view to increasing consumer awareness Based on ministry information, there are more and will continue to take appropriate steps to than 15,000 children in private schools below expand initiatives to provide information to compulsory school age, with at least 3,000 of parents and students regarding the choice to these children below the junior kindergarten age pursue private education. 190 2013 Annual Report of the Office of the Auditor General of Ontario

of three years and eight months. In addition, there MINISTRY RESPONSE are over 350 private schools that could claim to be exempt from compliance with the Day Nurseries Act. As part of its work on modernizing child care, the However, the Ministry is not aware of all the private Ministry is planning a policy change to direct that schools that operate child-care facilities without all private schools serving children under junior a licence and, except during validation visits in kindergarten age must be licensed under the their first year of operations, the Ministry does not Day Nurseries Act. The Ministry also intends to visit private elementary schools. In contrast, in introduce new legislation that, if passed, would accordance with the Day Nurseries Act, the Ministry replace the Day Nurseries Act. The proposed bill inspects all licensed child-care facilities annually. would clearly outline which programs require a The Ministry could face significant liability if any- child-care licence and which are exempt, includ- thing untoward happens to a private school child ing provisions supporting the direction that who should have been afforded the protections of private schools serving children under junior kin- the Day Nurseries Act. dergarten age require a child-care licence. This The following example illustrates a school that would be supported by communications to all operated a daycare centre for at least five years private schools in Ontario and a transition period without any oversight from the Ministry. In 2012, for operators to become licensed. the Ministry received a complaint from a municipal The Ministry will continue assessing options health department regarding child-care practices at to identify private schools offering services a private school. Upon investigation, the Ministry that should be licensed under the Day Nurseries learned that the owner of a school that was in Act, and withdrawing the policy exemption existence before 1993 had been submitting notice from the Day Nurseries Act for private schools of intention forms to the Ministry even though the offering services to children under junior school was no longer in operation. In exchange kindergarten age. The Ministry will also con- for payment, this owner continued to submit the tinue its practice of investigating complaints old private school’s information on behalf of an regarding private schools offering services that

Chapter 3 • VFM Section 3.06 unlicensed daycare operation that was at a different may need a licence under the Day Nurseries Act, location, with different owners. This was done to and identifying such schools from information take advantage of private school exemptions from received through the annual Notice of Intention the Day Nurseries Act. The Ministry informed the to Operate a Private School form. daycare’s owners that they would require a day nursery licence if they continued to operate.

RECOMMENDATION 2 DIPLOMA PROGRAM INSPECTIONS Subsection 16(7) of the Education Act (Act), To reduce health and safety risks to preschool Inspection on Request, states that the Ministry children and ensure compliance with legisla- may inspect the standard of instruction in a pri- tion, the Ministry of Education (Ministry) vate school in the subjects leading to the Ontario should identify all private schools that oper- secondary school diploma (OSSD). These OSSD ate child-care facilities and ensure that these program inspections are limited to a review of the schools are licensed under the Day Nurseries Act school’s operations related to the delivery of high and inspected as required by legislation. school credits and are only to be undertaken at the request of the private school. The Ministry charges a fee to perform program inspections, and private Private Schools 191 schools will not be given the authority to grant Program Inspection Selection Process diploma credits without a ministry inspection. The The Ministry inspects private schools that offer purpose of inspections is to determine whether credits toward the high school diploma on a cyclical the standard of instruction in courses leading to basis, generally once every two years. However, if the OSSD is being delivered in compliance with an inspection determines that a school’s operations ministry requirements, including the provincial are significantly non-compliant with ministry curriculum. policies, potentially affecting the integrity of the Subsection 16(6) of the Act gives the Ministry diploma credits being issued, an education officer the authority, at its discretion, to inspect all private may recommend more frequent inspections. We schools, but the Ministry does not have a general reviewed a sample of inspection reports of such inspection process in place and generally does not schools and noted that there was limited rationale inspect any private schools under this provision of on file as to why the schools were recommended for the Act. Although the Ministry will conduct pro- more frequent visits. Nevertheless, approximately gram inspections at schools that offer high school 100 schools were identified to be inspected again credits, elementary schools are not inspected by the following year rather than in two years’ time, the Ministry for any aspect of their operations, and and five schools were recommended to be inspected neither are secondary schools that do not offer twice annually. In other words, significant concerns diploma credits. In contrast, an annual inspection were identified in over 25% of the schools offering process for all private schools is in place in several high school credits. other provinces, such as Manitoba, Saskatchewan Given the number of schools with concerns and British Columbia. These inspections include noted by education officers, in September 2012 not only an assessment of program delivery but also the Ministry established risk management criteria a review of the facilities to ensure the health and to identify priority schools to be inspected earlier safety of students. in the year or more frequently: these are schools There are 408 private schools in Ontario that have significant unresolved issues related to authorized to offer credits toward a high school credit integrity, are in their first year of operation

diploma, a number that has grown by more than Chapter 3 • VFM Section 3.06 or are considered “at-risk” schools outside the 25% over the past five years. In the 2012/13 inspection cycle. Criteria related to credit integrity school year, the Ministry performed about 260 include failing to provide 110 instructional hours program inspections. This inspection process is per course; granting equivalent credits to students based on evidence gathered through the review of from non-recognized out-of-country institutions; pre-inspection materials such as course calendars having a principal with a limited understanding and school timetables; on-site discussion with of curriculum requirements; and confirmed credit the principal and other school staff; observations integrity complaints. Regional office teams were to during classroom visits; reviews of students’ work; collectively review each inspector’s school inspec- examination of school policies and procedures; tion priorities to ensure that the risk management and an examination of school records. To assess criteria were applied consistently. In the 2011/12 if private schools are in compliance with ministry school year, each region was asked to identify up to policies, education officers are required to com- 12 priority schools. That year, the regions identified plete an inspection template that contains stan- 27 priority schools, and 66 priority schools were dard questions to be answered for each inspection identified in 2012/13. and complete an inspection report detailing their We reviewed the process used to identify prior- findings on the school’s compliance with ministry ity schools at the three regions we visited and requirements. 192 2013 Annual Report of the Office of the Auditor General of Ontario

determined that although all the regions stated that number of classrooms and performed additional they used the risk management criteria to deter- procedures to check whether curriculum expecta- mine priority schools, only one of these regions had tions were being delivered, we noted that there was documented its assessments. The other two regions no documented evidence to support the number of stated that they held informal meetings and could classrooms visited or the duration of the visits in not provide us with any evidence to support their approximately two-thirds of the samples reviewed. discussions or the conclusions reached. Further- In general, education officers did not sufficiently more, none of the three regions we visited had document their inspection activities and decision- performed a region-wide analysis to assess the risk making. As a result, we could not assess how at all schools or rank schools against one another, well education officers were complying with the therefore potentially not identifying all high-risk Ministry’s requirements. These officers informed schools for more frequent inspections. us that although most of them had been principals, vice-principals or teachers and therefore brought their educational expertise to the inspection pro- Inspecting Existing Credit-granting Schools cess, and that they attended regular professional We reviewed a sample of inspection files to assess the development meetings, they would welcome formal quality of the inspections undertaken by education training in inspection and investigative practice. officers. Many of the files we selected did not include We also noted that private school teachers are supporting documentation or note the activities per- not required to be certified by the Ontario College formed in order for us to assess whether education of Teachers. Furthermore, private school owners, officers were following consistent inspection pro- principals and teachers are not required to meet cedures based on the risks identified. For example, any minimum qualifications or demonstrate that the Ontario Student Record (OSR) is an ongoing they have obtained the OSSD or similar qualifica- record of a student’s educational progress. Educa- tions, either individually or collectively. Therefore, tion officers are expected to examine a minimum individuals providing instruction toward a high of 10 OSRs and complete a checklist for each OSR school diploma may not have obtained this certifi-

Chapter 3 • VFM Section 3.06 examined to determine if private schools have poli- cate themselves. Some education officers noted that cies and procedures for the establishment, mainten- it is difficult to discuss ministry policies with respect ance, use, retention, transfer and disposal of student to the delivery of the Ontario curriculum with indi- records. Although education officers in the regions viduals who do not have teaching qualifications, we visited stated that they review 10 OSRs during an as there is a significant knowledge gap. Our review inspection, in the majority of the inspection files we of practices in other jurisdictions found that many selected we were unable to confirm this as there was provinces (Quebec, Nova Scotia, Newfoundland no record of this review and the OSR checklists were and Labrador, Prince Edward Island, Saskatch- not on file. ewan) require teachers who deliver provincial cur- Another procedure performed by education riculums at private schools to be certified teachers. officers is to visit five to seven classrooms for approximately 20 to 30 minutes each to review Inspecting Specific Risk Areas daily lesson plans and ensure that submitted course outlines correspond with the instruction observed. The ministry guideline Inspection Requirements for The classroom visits also require education officers Private Schools Granting Secondary School Credits to sample student assessments and classroom work, (IRPS) sets out the policies that govern the inspec- as well as check student attendance. Although edu- tion of private schools and summarizes the Ontario cation officers stated that they visited the suggested curriculum and ministry policies related to the Private Schools 193 delivery of credits. To assess compliance, education additional procedures that were not part of officers complete an inspection template based on the inspection template. IRPS. However, we identified several areas where • The private school inspection process does the Ministry’s inspection template did not suf- not include procedures for education officers ficiently address important risk areas such as ensur- to confirm that students have met diploma ing that credits granted are earned, that sufficient requirements. Some of the education officers credits are accumulated for a diploma and that we interviewed stated that they would per- online school programs meet ministry standards. form additional procedures, not contained Even though the Ministry had developed or com- in the inspection template, to ensure that municated additional procedures to address these students who had been issued diplomas risk areas, in many of the inspections we sampled had met the Ministry’s requirements. These there was insufficient evidence that inspectors had procedures included verifying that students implemented these procedures satisfactorily. Some had completed the 18 compulsory and 30 total of these risk areas are as follows: credits; passed the Ontario Secondary School • The public, school boards and other stake- Literacy Test; and performed 40 hours of com- holders have raised concerns over some munity service. The officers who performed private schools either giving students higher additional procedures would do so to varying grades than they had earned or giving degrees of thoroughness, from doing a quick students credit for courses that they did not scan to examining one to 12 student records. attend or complete any course work in. The However, there was very little supporting Ministry receives such complaints either evidence to assess how thoroughly education directly from stakeholders or from the gen- officers were performing these additional eral public through its website. It responds procedures. One education officer identified to general complaints within 15 business a case where a private school student was days and specifically tracks and investigates awarded a diploma but had not completed the complaints related to credit integrity. Over minimum 30 credits.

the past three years, the Ministry has received • Although 24 online private schools are Chapter 3 • VFM Section 3.06 approximately 140 complaints related to authorized to grant credits toward a high credit integrity, and on occasion has revoked school diploma, the inspection process credit-granting authority for schools that does not ensure that the practices at online grant credits inappropriately. Although the schools meet the standards identified in Ministry has discussed procedures that educa- the Ministry’s curriculum and policies. For tion officers could perform during an inspec- example, education officers have difficulty tion to proactively uncover these activities, verifying that students have completed the the inspection template does not include required 110 hours for full-credit courses, specific procedures to help identify schools because some student activities are per- that violate ministry policy in this regard. formed offline. Accordingly, in September Some education officers said they perform 2012 the Ministry created a checklist to assist supplementary procedures like comparing education officers in performing supple- students’ work to their grades, but we saw mentary procedures when inspecting online very little supporting evidence or documenta- schools. These procedures include reviewing tion to verify that such procedures were being student learning logs that document both performed. Other education officers noted online and offline activities in order to ensure that there was not enough time to perform that the required hours have been completed. 194 2013 Annual Report of the Office of the Auditor General of Ontario

Although some of the education officers time frame, rather than proceeding directly to the stated that they use the checklist when removal of credit-granting authority. inspecting online schools, we did not see any completed checklists in the inspection files Following Up on Non-compliance with we reviewed. Ministry Policy

During the inspection process, an education officer Inspecting New Credit-granting Schools may determine that a private school with credit- Over the last three years, approximately 85 new granting authority is not complying with ministry schools have been authorized by the Ministry to policies to an extent that could affect the integrity grant credits toward the high school diploma. The of the credits issued by the school. These concerns Ministry permits new schools that have enrolled are to be communicated to the private school’s prin- at least five students and been validated to deliver cipal, who is supposed to prepare an action plan to instruction toward diploma courses before they are address any significant non-compliance. A follow- given authority to grant these credits to students. up inspection is then to be performed to ensure that The schools are officially authorized to grant credits any issues identified have been corrected. when they have successfully passed an inspection. We reviewed a sample of inspection reports We reviewed a sample of inspection reports of and noted that the majority had some concerns new schools and found that there were cases of in relation to compliance with ministry policies, significant non-compliance but the schools were with about one-third of the reports identifying still authorized by the Ministry to issue credits. non-compliance at a level that could potentially These compliance issues included situations where affect the integrity of credits issued by the school. curriculum expectations were not always evident Among these concerns were the following: there in the classroom; it was not evident that student was a lack of evidence that the required 110 hours achievement was based on curriculum expecta- of instruction were being scheduled and delivered; tions; and there was no documentation to indicate the expectations set out in the curriculum were

Chapter 3 • VFM Section 3.06 that the mandatory 110 credit hours had been not evident in the classroom; no Prior Learning scheduled. The education officers made follow-up Assessment and Recognition (PLAR) procedures visits to these new schools to assess whether the were in place to ensure that the knowledge students schools had resolved the concerns identified. In one obtained outside Ontario meets the expectations of case we sampled, almost all the compliance issues the provincial curriculum; and the required PLAR identified during the inspection continued to exist forms were not on file to support awarding equiva- at the time of the follow-up visit. However, this lent credits. school was still permitted to grant diploma credits. We noted that education officers at times made We were informed that over the past five follow-up visits after an inspection to verify that years only one new school has been denied the concerns they had noted had been rectified. credit-granting authority. One education officer Through discussions with education officers, we stated that officers are expected to work with learned that compliance issues corrected by the time new schools to help move them into compliance. of the follow-up visit are generally not documented Another inspector told us that officers have to give on the inspection report. Furthermore, since the new private schools the opportunity to improve Ministry does not usually obtain action plans from before making the decision to deny credit-granting private schools indicating how non-compliance will authority. The Ministry noted that its practice is be rectified, there is limited evidence that many to provide schools with an opportunity to address of the compliance issues identified ever existed. some non-compliance issues within a specified Consequently, inspectors would have to rely on their Private Schools 195 memories to assess recurring compliance issues, and noted that there is limited management oversight a record of these issues would not be available for over the follow-up of compliance issues identified management review or for future inspectors. during an inspection. Where no follow-up visits were scheduled, we found that the Ministry generally did not provide Public School Students Taking Diploma deadlines to private schools to resolve compliance Courses at Private Schools issues. Where concerns were noted, the Ministry provided schools with the opportunity to correct In 2009 concerns were expressed in the education them by the next inspection cycle, which could sector and among the public that some students occur up to two years after the non-compliance had who were registered primarily at publicly funded been identified. In our sample, we noted a private schools were taking courses at private schools to school that was granting PLAR equivalencies obtain higher marks in order to gain an advantage toward the high school diploma that was not fol- in university admissions and scholarship applica- lowing the proper assessment procedures to justify tions. In response to this concern, the Ministry awarding the credits. The Ministry identifies viola- requested that public schools flag student tran- tion of PLAR procedures as a significant risk factor scripts with a “P” notation on courses taken by their that affects credit integrity. However, the private students at private schools. We reviewed the “P” school was given two years until its next inspection notation information reported by public schools for to demonstrate that it had corrected this problem. 2010/11 and 2011/12 and noted that for each of Since 2004, the Ministry has revoked the credit- these academic years, approximately 6,000 courses granting authority of 23 schools. The Ministry has were taken by public school students at private developed procedural guidelines for revocation of schools, with two-thirds of these courses being at credit-granting authority, but there are no guiding the grade 12 level. principles to assist education officers in determin- We contacted several universities, the Ontario ing the degree of non-compliance that would lead Universities’ Application Centre and the Ontario to credit-granting authority being denied. As a College Application Service. Many of those we

result, the recommendation to revoke credit-grant- spoke to stated that the “P” notation is not well Chapter 3 • VFM Section 3.06 ing authority is based on the judgment of individual understood and that they accept credits issued by education officers, which could lead to inconsisten- private schools at face value, since these schools cies. Some education officers we interviewed stated pass ministry inspections and are given credit- that minimum compliance standards need to be granting authority. Consequently, it is assumed that defined and more detailed policies and procedures courses are being delivered properly. put in place to assist in determining when to revoke credit-granting authority. Program Inspection Revenue Although regional managers review inspection reports for completeness, we noted that there To oversee the private school sector, the Ministry is limited review to ensure that inspections are spends approximately $225,000 for head office staff adequate and that the recommendation that a and approximately $575,000 for education officers. private school be authorized to grant credits toward Private schools must pay a fee to cover the cost of the diploma is appropriate. For example, since edu- ministry inspections. Prior to 2009, inspection fees cation officers generally do not retain supporting were based on the number of students enrolled documentation from inspections or document their at each private school and ranged from $800 for procedures, managers are unable to assess the schools with fewer than 100 students to $1,100 procedures performed or the recommendations to for schools with over 400 students. The Ministry award credit-granting authority. Furthermore, we now charges a flat fee per inspection. This fee has 196 2013 Annual Report of the Office of the Auditor General of Ontario

been steadily increasing, from $1,100 in 2009/10 inspection process, and will continue to review to $2,450 in 2012/13, and the Ministry plans to and assess policies and procedures in this area. increase the fee to $4,050 over the next few years The Ministry has implemented a risk-based to recover the full cost of each year’s inspections by procedure to identify priority schools requiring 2014/15. For the 2011/12 school year, the Ministry early inspection, and continues to determine the charged private schools approximately $425,000 frequency of inspections through the inspection for inspections and, with the exception of a few process. The Ministry is reviewing the documen- schools that closed, the vast majority of private tation, tracking and follow-up aspects of the schools paid their inspection fees on a timely basis. inspection process to look for ways to improve the effectiveness of its monitoring activities. RECOMMENDATION 3 The Ministry will evaluate options regarding To ensure that adequate policies and proced- private schools that are persistently non-compli- ures are in place to verify that credit-granting ant with legislative and policy requirements. private schools are awarding course credits and diplomas in compliance with ministry policies, including the provincial grade 9 to 12 cur- REQUIRED DATA SUBMISSIONS AND riculum, the Ministry of Education (Ministry) REPORTING should: • use its established criteria to assess the risk Section 16 of the Education Act states that private of non-compliance so that it can rank all schools are to provide statistical information credit-granting schools and devise an inspec- regarding student enrolment, staff, courses of tion frequency schedule according to the study and other information as and when required risks identified; by the Ministry. Elementary schools and second- • document procedures undertaken, sig- ary schools that do not offer diploma credits are nificant non-compliance observed and required to submit only their aggregate student conclusions reached during inspections, and enrolment for the year, while secondary schools Chapter 3 • VFM Section 3.06 retain all documentation for management that offer diploma credits must provide specific oversight and subsequent review; information, including credits taken and grades • consider a conditional rating for new private achieved, for each student registered. This informa- schools that are not yet fully compliant; tion is to be submitted three times a year through • review whether the “P” notation on public the Ministry’s Ontario School Information System school student transcripts is influencing (OnSIS), a web-based application that integrates post-secondary admission decisions as school, student, educator and course data. intended; and Data collection for private schools was to be • establish effective procedures to identify, fully implemented in the 2006/07 academic year. track and take timely corrective action However, the Ministry is having significant dif- against private schools that are repeatedly ficulty in obtaining all the required information non-compliant with ministry policies. from all private schools in a timely manner. As of June 2013, the Ministry still had not received the MINISTRY RESPONSE required data from approximately 10% of schools for the 2010/11 academic year and 25% of schools The Ministry agrees that the protection of credit for the 2011/12 school year. All student-specific integrity in the granting of credits and diplo- data for the 2011/12 academic year should have mas is a critical function of the private school been finalized by September 2012. However, by Private Schools 197

June 2013, one year after the school year ended, system in general. Private secondary schools that approximately 100 secondary schools offering offer high school credits must assign an OEN to diploma credits and 150 elementary and secondary every student pursuing a diploma, but the 605 schools that do not offer credits still had not sub- private elementary schools and secondary schools mitted any information to the Ministry. that do not offer diploma credits are not required For the information submitted, the Ministry to assign OENs to their students. does not have a process in place to assess its To ensure that students of compulsory school age accuracy and relies on the good faith of private are being educated, OnSIS requires every student school administrators. Also, since the education who has been assigned an OEN to be accounted for officers are not given access to data collected somewhere in the education system. Without this through OnSIS, they cannot assess the informa- identifying number, students at private elementary tion reported about students in private schools, schools and secondary schools that do not offer such as courses taken, grades received and credits credits are not accounted for. The Ministry does not granted. Additionally, since the Ministry does not have student-specific information to verify that all revalidate, inspect or visit elementary schools and children in the province who are not in the public secondary schools that do not offer diploma credits, system are being educated in institutions such as pri- the annual student enrolment reported by these vate schools, and therefore cannot demonstrate that schools is also not verified. Therefore, the Ministry all children are in compliance with the legislated accepts the student enrolment numbers submitted requirement for compulsory school attendance. by private schools and publicly reports this infor- Providing all children in both the public and the mation without ensuring its accuracy. private education system with an Ontario Education Ontario’s public schools submit similar informa- Number would help ensure that all students of com- tion to that requested from private schools, for pulsory school age are receiving an education. which the Ministry has implemented a rigorous verification process. The Ministry uses data collected RECOMMENDATION 4 through OnSIS to make informed policy decisions To help ensure that sufficient information is

for the public schools based on graduation rates, Chapter 3 • VFM Section 3.06 submitted to enable effective oversight of the course pass rates and student credit accumula- private school sector and to ensure compliance tion. This information is used to help ensure that with legislation and related policies, the Min- students in the public school sector are progressing istry of Education (Ministry) should: and receiving satisfactory instruction. However, the consider various options to encourage pri- Ministry has not done any such analysis of the data • vate schools to submit the required informa- received from private schools. In order for data to be tion on a timely basis; useful for analysis, it must be complete and accur- implement procedures to periodically verify ate, and it must be submitted on a timely basis. With • the accuracy of the data submitted by private private school information, the Ministry is facing schools; significant challenges in all three of these areas. analyze data received to highlight potential OnSIS requires that all students be assigned • concerns and to determine if private school an Ontario Education Number (OEN), which is students are progressing appropriately; and a unique identification number that enables the consider assigning Ontario Education Num- recording of student-specific information as well • bers to all private school students to help as each student’s progress through the educational verify compulsory school attendance. system. The number also facilitates the collection and analysis of data about Ontario’s education 198 2013 Annual Report of the Office of the Auditor General of Ontario

MINISTRY RESPONSE ISSUING BLANK DIPLOMAS AND CERTIFICATES The Ministry agrees that the collection of The Ministry has exclusive power in Ontario over timely and accurate information is required diplomas and certificates that are granted to pupils for effective oversight and monitoring, as well and the conditions under which they are granted. as for evidence-based decision-making and The OSSD is awarded to students who have dem- policy development, and will continue working onstrated that they have successfully completed to improve processes for data collection and the Ministry’s diploma requirements. The Ministry analysis. The Ministry will continue to provide has authorized 408 private secondary schools to resource materials, help-desk support and train- issue credits toward high school diplomas. Blank ing to assist private schools in completing their diplomas, pre-signed by the Minister of Education, required submissions. The Ministry will extend are the same for both public and private school stu- the data quality assurance processes in place dents. The school types the student’s name on the for publicly funded schools to the data collected blank diploma, and the diploma is dated and signed from private schools. This five-pillar approach by the school principal. includes consistency, completeness, accuracy, To help prevent diploma fraud and ensure con- precision and timeliness. trol over the number of blank diplomas provided, The Ministry will use the private school any public school request in excess of 10% above profile under development and conduct trend the previous year’s grade 12 student enrolment analysis to track achievement for students is rejected. However, this procedure has not been attending private schools and their progress applied to private schools. In the 2011/12 school through the education system, including year, private schools requested a total of about comparisons to other private-school and public- 16,000 blank diplomas from the Ministry. The school peers across the province. Ministry has not been able to demonstrate adequate The Notice of Intention to Operate a Private oversight over the diploma distribution process School form has been updated for the cur- and, as a result, has issued thousands of diplomas

Chapter 3 • VFM Section 3.06 rent 2013/2014 school year to require private without identifying for whom these diplomas were schools to declare whether or not they have pro- intended. We noted that other jurisdictions have vided the statistical information required by the additional control measures such as dual or mul- Education Act, noting that failure to do so may tiple signatures and embossed or official seals, and result in a fine upon conviction and the revoca- in Alberta each diploma is uniquely numbered. tion of the ministry-issued school identification Private schools submit requests to the Ministry number required to operate. each year identifying the number of diplomas The Ministry will also consider options needed for their graduating class. However, the regarding the issuance of Ontario Education Ministry has not been comparing the number of Numbers to all private school students. This graduating students to the number of diplomas number is currently issued to all students in requested. We compared the student enrolment publicly funded schools and private schools reported in OnSIS to the number of diplomas with credit-granting authority, and to students requested and issued to private schools for the past in private schools that do not grant credits but three academic years. We noted, for example, that choose to issue Ontario Education Numbers. in 2011/12, 30 private schools were issued a total The Ministry will inform those private schools of 1,500 diplomas in excess of their entire grade 12 not currently issuing Ontario Education Num- student populations. bers of the process to apply for access to the online Ontario Education Number application. Private Schools 199

We also noted that the Ministry is issuing adequate controls are in place over their distribu- diplomas to private schools that are not submit- tion, the Ministry of Education (Ministry) should: ting enrolment figures. The Ministry informed the reconcile the number of diplomas and certifi- schools that the grade 12 enrolment reported in a • cates requested to the number of graduating private school’s October OnSIS submission would students reported at each private school, and be used to assess the reasonableness of diploma investigate any unreasonable discrepancies; requests for that academic year, as is done with and public schools. However, over 175 credit-granting distribute diplomas and certificates to only private schools had not submitted their 2011/12 • those private schools that submit student- student enrolment information by the end of the specific data for graduating students. school year, but were still issued the diplomas they requested. Additionally, we noted that at the com- MINISTRY RESPONSE pletion of our audit in June 2013, over 50 of these schools had still not submitted the required data. The Ministry agrees that proper restrictions In total, these 50 schools had received over 2,300 are required for ministry documents certifying diplomas from the Ministry without having to dem- student achievement and will continue with onstrate that they had any graduating students. two recent policy initiatives to tighten control. We also reviewed the Ministry’s distribution The first policy, already in effect, is to reject of blank Ontario scholar certificates. An Ontario and investigate orders for diplomas and cer- scholar certificate is intended to be awarded to tificates from private schools with more than high-achieving students who obtain at least an 5% above their reported grade 12 enrolment. 80% average. The blank certificates are signed and The second policy, which will begin in the sealed by the Minister of Education. We observed 2014/2015 school year, is to not send diplomas that 50 schools requested a total of 3,350 Ontario and certificates automatically to private schools scholar certificates and an equal number of OSSDs, with credit-granting authority if they have not suggesting that all of their graduates would achieve submitted the required statistical data. Instead,

an 80% average. the Ministry will investigate and determine Chapter 3 • VFM Section 3.06 Education officers inspect private schools that the appropriate follow-up action, which may offer credits at least once every two years. We noted include an adjustment or even denial of the that during their inspections the officers do not school’s request. reconcile diplomas or certificates requested to the The Ministry is also developing a private number of graduating students. The Ministry has school profile document to provide education recognized that private schools have been receiv- officers with current, school-specific informa- ing more diplomas than required. As a result, in tion from OnSIS, including a comparison of the October 2012 the Ministry requested that private number of graduates with the number of diplo- schools return unused or damaged diplomas. At the mas and certificates ordered. completion of our audit, about 700 diplomas had been returned.

RECOMMENDATION 5 POLICY AND LEGISLATIVE ENFORCEMENT To help ensure that Ontario secondary school Section 16 of the Education Act (Act) outlines a diplomas and Ontario scholar certificates are number of requirements for private schools and issued only when they are earned and that stipulates penalties for non-compliance with these 200 2013 Annual Report of the Office of the Auditor General of Ontario

requirements. This section was passed in 1962 and RECOMMENDATION 6 has not changed significantly since that time. In the 1970s, penalty amounts were marginally increased. To better ensure compliance with the Education Currently, the penalties outlined in the Act are a fine Act and policies related to private schools, the of $50 a day for every person managing a private Ministry of Education (Ministry) should con- school without a notice of intention; as much as sider a legislative framework that would provide $200 for the person in charge of a school who has more flexible and cost-effective enforcement not provided statistical information to the Ministry tools that are commensurate with the nature within 60 days of the request; and up to $500 for and extent of non-compliance. every person who knowingly makes a false statement on a notice of intention form or information MINISTRY RESPONSE return. However, according to the Act, an individual The Ministry will continue to take appropriate or school must be convicted of these offences before steps to expand initiatives to provide information any fines can be imposed. The Ministry has stated to parents and students regarding consumer that as a result of this requirement, enforcement is awareness in the private school sector. Regarding not fiscally responsible, as legal costs of pursuing a issues of enforcement, the assessment of options conviction far outweigh the fines that might be col- will be commensurate with the Ministry’s def- lected. As a result, the Ministry informed us that it inition of its role in this sector, and will in turn has not sought prosecution for any offence commit- recognize the differences between the role taken ted by any private schools or individuals associated by Ontario’s Ministry of Education and that of with these schools. education ministries in other provinces. In contrast to private school fines, penalties for non-compliance by private career colleges in Ontario can be significant. ThePrivate Career Col- leges Act outlines that the purpose of penalties is TESTING OF PRIVATE SCHOOL STUDENTS to encourage compliance with that act and with The Education Act requires all children of compul-

Chapter 3 • VFM Section 3.06 orders to restrain from contravening the act, and sory school age to attend a public elementary or to prevent a person from deriving any economic secondary school on every school day unless they benefit as a result of a contravention of the act. We are receiving satisfactory instruction elsewhere. reviewed the penalty structure for private career The Ministry inspects private schools that offer colleges in Ontario and noted that some penalties high school credits but does not have any process in do not require successful prosecution to impose. For place to ensure that satisfactory instruction is being example, the Superintendent of Private Career Col- provided to students attending elementary private leges can levy an administrative penalty of $1,000 schools or secondary private schools that do not on a private career college for non-compliance with- offer diploma credits. In fact, we compared ministry out going to court, and can quadruple this penalty oversight to that in other Canadian provinces and if the college repeatedly offends within three years. found that Ontario has one of the least regulated In addition to administrative penalties, private private school sectors in Canada. career college fines can be substantial. For example, The Education Quality and Accountability whereas Ontario private schools can be fined $500 Office (EQAO) helps to ensure satisfactory instruc- for submitting false information, the same offence tion by testing all students at various grades in at an Ontario private career college can result in a the publicly funded school system. The EQAO fine of up to $50,000 and one year in jail for an indi- administers standardized tests to measure student vidual and $250,000 for a corporation. Private Schools 201 achievement against curriculum expectations in the provincial average to excellent, with pass rates grades 3 and 6 for reading, writing and mathemat- ranging from 19% to 100%. ics; grade 9 for mathematics; and grade 10 for the The purpose of education is to provide students Ontario Secondary School Literacy Test (OSSLT). with the opportunity to realize their potential and Some private school students participate in the develop into highly skilled, knowledgeable citizens grade 3, 6 and 9 EQAO assessments, although who contribute to society. The Ministry does not they are not required to do so. However, an OSSD have processes in place, such as an analysis of EQAO requirement for both public and private school test results, to assess whether private school students students is the successful completion of the OSSLT. are acquiring these skills and knowledge. Further- EQAO test results for both public and private more, since EQAO testing is not mandatory, such schools participating in the OSSLT are publicly analysis cannot be undertaken for the private school reported, but only for schools with a minimum sector as a whole. We noted that such testing, while number of students, in order to ensure student often paid for by the province, is mandatory for pri- confidentiality. vate schools in several other Canadian jurisdictions, All private schools can participate in the grade 3, such as British Columbia, Alberta, Saskatchewan, 6 and 9 EQAO tests but must pay for their students Manitoba, and Newfoundland and Labrador. to take these assessments. Only private schools that offer high school credits leading to the OSSD are RECOMMENDATION 7 eligible to participate in the OSSLT, and there is no To help ensure that private school students charge to take this test. Participation in EQAO tests receive satisfactory instruction and are provided can be seen as a proactive measure by some private with the opportunity to realize their potential schools to be more accountable, as these assess- and develop into highly skilled, knowledgeable ments can be used by both the schools and parents to citizens, the Ministry of Education (Ministry) periodically assess the progress of their students in should: relation to their public school peers. In the 2011/12 consider options to increase private school school year, 112 private schools participated in the • participation in standardized testing; and

grade 3 and 6 assessments, and 18 participated in Chapter 3 • VFM Section 3.06 analyze test results for private school stu- the grade 9 assessment. All private schools that are • dents and follow up on any outcomes that approved to offer high school credits that have eli- suggest these students are not receiving a gible students participate in OSSLT testing. quality education. We reviewed the EQAO grade 3, 6 and 9 assessments for 2010, 2011 and 2012 of participating MINISTRY RESPONSE private schools and noted that although individual private school results varied significantly, a greater The Ministry will assess options to require percentage of public school students achieved the private schools with credit-granting author- provincial standard than private school students. ity to participate in the grade 9 assessment As well, for the same three years, among students of mathematics conducted by the Education writing the OSSLT for the first time, public school Quality and Accountability Office. The Ministry students outperformed private school students. will explore options to develop data collection In 2012, 82% of public school students passed processes and will analyze private school pass the OSSLT compared to 73% of private school rates for the Ontario Secondary School Literacy students. We reviewed a sample of private school Test annually to identify issues related to private OSSLT results and found that the outcomes for school student achievement and to determine these schools varied considerably, from well below appropriate responses. Chapter 3 Ministry of Natural Resources Section 3.07 Provincial Parks

At the time of our audit, Ontario’s provincial Background parks were divided among six zones for the purposes of operation and management (Figure 1). Most provincial parks in southern Ontario operate Ontario has 334 provincial parks covering over from the second weekend in May until mid-October. 8.2 million hectares, an area roughly the size Most parks in Northern Ontario open on the of New Brunswick and Prince Edward Island Victoria Day weekend and close just after Labour combined. The Provincial Parks and Conservation Day. There are, however, about 20 provincial parks Reserves Act, 2006 (Act) governs the development, scattered throughout the province that operate operation and management of these provincial year-round. parks as well as Ontario’s conservation reserves. The Ministry had approximately 235 full-time The purpose of the Act is to permanently protect staff involved in the operation and management

Chapter 3 • VFM Section 3.07 a system of provincial parks and conservation of provincial parks at the time of our audit. This reserves that contain significant elements of staff was distributed between the Ministry’s head Ontario’s natural and cultural heritage and provide office in Peterborough, the park zone offices and opportunities for ecologically sustainable recrea- the operating parks. In addition to full-time staff, tion. The Ministry of Natural Resources (Ministry) the Ministry uses approximately 600 seasonal staff is responsible for establishing, operating and man- and 1,600 students at operating parks during peak aging provincial parks in accordance with the Act. season each year. About a third of the province’s parks are operat- The Act gives the Minister of Natural Resources ing parks; these provide recreational opportunities the authority to set fees for the use of provincial such as day-use areas and overnight and interior parks or any facilities or services offered within camping. Operating parks have staff on site and the parks. To help meet park operating expenses, contain visitor centres, museums, park stores, and the Ministry charges such fees in the 114 operat- other services and facilities. In the 2012/13 fis- ing parks. The fees depend on the activities (for cal year, operating parks attracted over 9 million example, skiing, hiking, swimming, boating, visitors. Non-operating parks, while still accessible wildlife viewing) and amenities available. Fees are to the public, have no staff on site and offer only not charged in most non-operating parks. In the limited facilities. 2012/13 fiscal year, provincial parks generated

202 Provincial Parks 203

Figure 1: Provincial Parks by Park Zone (as of March 2013) and Operating Status Source of data: Ministry of Natural Resources

Total Operating and Non-operating Operating Non-operating Area Area Area % of Total Park Zone (Zone Office) # (hectares) # (hectares) # (hectares) Park Area Northwest (Thunder Bay) 77 1,868,489 18 1,864,419 95 3,732,908 45.3 Northeast (Sudbury) 75 2,883,243 36 670,819 111 3,554,062 43.1 Algonquin (Whitney) 2 2,040 1 772,300 3 774,340 9.4 Central (Huntsville) 32 57, 519 20 29,332 52 86,851 1.0 Southeast (Kingston) 13 7, 576 21 63,558 34 71,134 0.9 Southwest (London) 21 9,733 18 10,888 39 20,621 0.3 Total 220 4,828,600 114 3,411,316 334 8,239,916 100.0

Note: On April 1, 2013, the Ministry eliminated the Central zone. Following this, a number of parks were reallocated among the five remaining zones. about $69 million in revenues. Operating expenses, we interviewed staff and reviewed pertinent docu- including head office expenses, totalled about ments. We also visited six provincial parks that $80 million. Historically, revenues generated by were located in these three zones. user fees paid by visitors have covered over 80% We engaged an ecologist with expertise in the of the parks’ operating costs, with the province field of environmental management to review making up the difference. Expenditures related ministry policies and a sample of management to the planning and protection of the park system directions for specific parks, and to provide us with (for example, costs associated with park research an opinion on whether the policies and directions and monitoring) are funded solely by the province. meet the requirements of the Act and adequately The province also funds park infrastructure such protect these parks. as washroom and shower facilities, visitor centres, We met with the Chair of the Ontario Parks water and sewage systems, and other capital Board, established in 1997 as an advisory commit- Chapter 3 • VFM Section 3.07 requirements. tee to the Minister of Natural Resources, and staff at the Office of the Environmental Commissioner of Ontario to obtain their perspectives on the province’s park system. We also researched park Audit Objective and Scope operations and management practices in other jurisdictions and met with officials at Alberta Parks The objective of our audit was to assess whether and Parks Canada to identify best practices that the Ministry had adequate systems, policies and may be applicable in Ontario. procedures in place to manage provincial parks cost-effectively and in compliance with legislation and ministry policies, and to reliably measure and report on its performance. Senior management at Summary the Ministry reviewed and agreed to our objective and associated criteria. Over the last 10 years, provincial parks have Our audit work was conducted at the Ministry’s grown in both number and size. The Provincial head office and three of the six zone offices where Parks and Conservation Reserves Act, 2006 (Act), 204 2013 Annual Report of the Office of the Auditor General of Ontario

which governs the management of provincial organisms), nonbiological components (such parks, expanded the requirements for ensuring as geology and water) and processes (such that the natural values within the parks are pro- as reproduction and population growth) in tected. The growth of the provincial park system, individual parks, and the pressures that affect combined with the expanded responsibilities them. In this regard, we noted that one ecolo- contained in the Act, has challenged the Ministry’s gist aided by a seasonal assistant ecologist and ability within its funded resources to meet its a few park biologists may be responsible for legislated mandate to protect Ontario’s park sys- conducting research and monitoring activities tem and provide opportunities for ecologically sus- in anywhere from 20 to 50 provincial parks. In tainable recreation. Currently, the Ministry risks comparison, Parks Canada informed us that falling further behind in meeting its mandate, each park in the federal system has a science specifically: team composed of at least one park ecologist • The Act states that maintaining ecological supported by a team of technicians, the size integrity is the first priority in the manage- of which depends on the size of the park and ment of provincial parks. It requires each the ecological issues being addressed. Parks park to have in place a management direc- Canada further supports these science teams tion that provides policies for the protec- with a team of senior ecosystem specialists, tion, development and management of the although it too has a backlog of work. significant resources and values within the • Activities such as hunting and fishing are park. At the time of our audit, the Ministry regulated in provincial parks, and the Act had reviewed just over half of the 334 direc- specifically prohibits activities such as tions in place and had concluded that 104 commercial timber harvesting (except in needed to be either amended or rewritten to Algonquin Park) and mining. However, due reflect the intent of the new Act. Only half to constrained resources, significant portions of these amendments and rewrites had been of the operating parks (which provide a range completed or were in progress. The ecologist of recreational activities), as well as the 220

Chapter 3 • VFM Section 3.07 we retained for this audit reviewed a sample non-operating parks that cover about half the of directions that the Ministry had deemed to area of Ontario’s provincial park system, are be consistent with the intent of the Act, and subject to little or no enforcement. Park staff concluded that none contained a clear state- advised us that they are aware of violations ment that ecological integrity was the first regularly taking place, such as illegal hunting, priority in managing the park. In fact, every boundary encroachments by adjacent land- management direction reviewed noted sig- owners, waste dumping, and the cutting and nificant damage to environmental conditions, removal of trees and plants. The province’s but none put forward meaningful strategies or Environmental Commissioner was critical of had been updated to address them. the Ministry recently when he said in a news • The Ministry’s 2011 survey of park planners, release accompanying his 2012/13 Annual ecologists, biologists and park superintend- Report: “It appears that the Ministry of Natural ents confirmed that the Ministry lacked the Resources is walking away from many parts of baseline scientific data on the provincial park its job to safeguard wildlife and natural resour- system that it requires to meet the rigorous ces. Important legal safeguards for provincial standards of the Act. The survey revealed gaps parks, species at risk, hunting, and Crown in information with respect to the native bio- lands have been significantly weakened.” logical components (plants, animals and other Provincial Parks 205

• A key objective of the Act is for provincial parks 43 of the most visited parks. However, results to provide ecologically sustainable outdoor of the most recent visitor survey conducted recreation opportunities. On average, over by the Ministry in 2011 indicated that the each of the last 10 years, more than 9 million programs are underutilized and generally fail visits have been made to the 114 operating to meet visitors’ expectations. parks within the province. With respect to the • The Act requires the Minister to publicly operation and management of these parks, we report, at least once every 10 years, on the noted the following: state of the provincial park and conserva- • The Ministry’s minimum operating stan- tion reserve systems. The Ministry released dards covering aspects of park operations its first State of Ontario’s Protected Areas such as waste management, sanitation, and Report in 2011. We noted that similar reports cleaning and maintenance of facilities and in other jurisdictions more fully reported on grounds were established over 20 years items such as the status of park management ago. Visits have since increased by over plans and the results of actions taken to meet 40%. When day-use visitors and campers objectives in the plans; threats to the parks were asked about how parks could be and their impact; relationships with Aborig- improved, better general maintenance and inal communities in planning and managing amenities were at the top of the list. parks; and the condition of capital assets. • The Ministry’s current backlog of desired Furthermore, the Ministry has established capital asset expenditures within the performance measures for only two of the provincial park system is significant. For Act’s four objectives, and lacks benchmarks instance, we estimated that assets such as to evaluate its performance in maintaining buildings, roads, bridges, drinking-water ecological integrity and monitoring ecological systems and septic systems listed as being change in the parks. in “poor” or “defective” condition require Currently, there are nearly 600 private cot- over $590 million to replace. Since our last tage properties held under lease in two provincial

audit of the provincial parks in 2002, the parks. The term of the current leases is expected Chapter 3 • VFM Section 3.07 backlog has increased by approximately to end in 2017, and is under review. We noted that $170 million. Without additional invest- these lease payments are significantly below fair ments, it will continue to grow. market value and should generate approximately • Although parks in southern and central $6.7 million more in revenue than the Ministry Ontario often operate at capacity and have currently receives. In addition, the fees charged by significantly more visitors than parks in the Ministry for providing services such as garbage other regions, the Ministry has not fully collection and snow removal are also well below the explored the possibility of increasing fees Ministry’s actual costs. in the more popular parks in the south and lowering fees in less visited parks, mainly OVERALL MINISTRY RESPONSE in the north, to increase visits and improve The Ministry of Natural Resources appreciates cost recovery. the Auditor General’s recognition of the growth Another key objective of the Act is to provide • of the parks system and the expanded respon- opportunities for park visitors to increase their sibilities under the Provincial Parks and Conserv- knowledge of Ontario’s natural and cultural ation Reserves Act, 2006 (Act), and agrees that heritage. The Ministry arranges Natural Herit- the sustainability of the parks system continues age Education (NHE) programs for visitors in 206 2013 Annual Report of the Office of the Auditor General of Ontario

Provincial Parks and Conservation Reserves Act, 2006 to be a fundamental priority. The Ministry is (Act), which laid out new requirements to ensure supportive of the recommendations made in this that the parks are adequately protected. The Act report and offers the following as context. lists four objectives for provincial parks: Protecting Ontario’s parks system while to permanently protect ecosystems, biodivers- providing opportunities for ecologically sus- • ity and significant elements of Ontario’s tainable recreation are dual priorities for the natural and cultural heritage, and to manage Ministry. Beginning with the enactment of the these areas to ensure that ecological integrity Act in 2006, the Ministry has moved to a parks is maintained; system model that emphasizes biodiversity to provide opportunities for ecologically and ecological integrity in managing and plan- • sustainable outdoor recreation and encourage ning parks. As the largest provider of outdoor associated economic benefits; recreation in the province, the Ministry has to provide opportunities for the residents made significant investments in parks facilities • of Ontario to increase their knowledge of over the last 10 years, including investments in Ontario’s natural and cultural heritage; and drinking-water systems, roads and other built to facilitate scientific research to support mon- infrastructure. • itoring of ecological change. Since 2005, the Ministry has followed The growth of the park system, combined with the National Quality Institute’s Progressive the Ministry’s expanded responsibilities under the Excellence Program, resulting in a number of revised legislation, has challenged the Ministry in initiatives designed to ensure the quality of the meeting its mandate with respect to the manage- natural and cultural resources found in parks ment and operation of the park system. Currently, and protected areas across the province. the Ministry risks falling further behind in meeting The Ministry published its first State of its mandate. We discuss this more fully below. Ontario’s Protected Areas Report (SOPAR) in 2011. SOPAR established benchmarks to measure future progress made by the provincial PARK PROTECTION Chapter 3 • VFM Section 3.07 parks and conservation reserves programs and Ecological Integrity is intended to keep Ontarians up to date on provincial parks and conservation reserves. A key objective of the Act is to permanently protect The Ministry will continue to evaluate significant elements of Ontario’s natural and existing policies, processes and tools to ensure cultural heritage by establishing and managing they remain applicable and relevant to its parks provincial parks. To this end, the Act makes the program. maintenance of ecological integrity its first priority. The Ministry considers ecological integrity within the park system to be maintained if native biological components (plants, animals and other Detailed Audit Observations organisms), nonbiological components (such as geology and water) and processes (such as reproduction and population growth) remain Over the last 10 years, provincial parks have grown intact. According to the Act, the Ministry is also to in both number and size. In 2002, Ontario had 277 consider restoring the parks’ ecological integrity provincial parks covering about 7.1 million hec- where necessary. In this regard, the Act and its tares. It now has 334 parks covering over 8 million accompanying Ontario Protected Areas Planning hectares. In addition, the government passed the Provincial Parks 207

Manual require the preparation of a management time. At the time of our audit, only 52 of the 104 direction for each provincial park that provides amendments and rewrites were in progress. The policies for the protection, development and man- remaining 75 management directions were deemed agement of the significant resources and values by the Ministry to be consistent with the intent of the within the park. Act and required at most administrative changes. In June 2012, the Act was amended to require The ecologist we retained reviewed a sample the Ministry to examine management directions of directions that the Ministry had either updated that have been in place 20 years (previously 10 or deemed to be consistent with the intent of the years) to determine if the directions need to be Act, to confirm whether these directions did indeed amended or replaced. adequately consider the protection and restoration As seen in Figure 2, at the time of our audit, with of the parks’ ecological integrity. In addition, the the exception of five provincial parks established in ecologist reviewed a management direction from 2011, all the parks had management directions in 1985 for one of the flagship parks in the system, place. However, over 40% of the directions had not which, at the time of our audit, the Ministry had been amended for 20 years or longer. just completed reviewing for compliance with the At the time of our audit, the Ministry had Act’s current direction on ecological integrity. reviewed 179 management directions to determine The ecologist concluded that none of the if these reflect the overall intent of the Act, and spe- directions reviewed contained a clear statement cifically whether the directions speak to the assess- that ecological integrity was the first priority in ment, maintenance and restoration (where needed) managing the park it pertained to. The ecologist of ecological integrity. The Ministry concluded that also found that the directions did not call for an 26 management directions need to be amended and assessment of the ecological condition of the parks 78 need to be completely replaced. Our discussions and therefore could not be considered to meet the with zone and park staff indicated that it takes, on intent of the Act. In fact, every management direc- average, five to 10 years to complete a management tion reviewed noted significant damage to environ- direction from the initial information-gathering mental conditions at the park it covered; however,

phase to the final approval, with the review and none put forward any meaningful strategies to Chapter 3 • VFM Section 3.07 approval process taking up about two-thirds of this address them, specifically:

Figure 2: Age and Status of Current Management Directions Source of data: Ministry of Natural Resources

Total Approved Management Directions Reviewed Since Enactment of Management Directions Provincial Parks and Conservation Reserves Act, 2006 Total # of Outcome of Review Amendment or # of Management No Significant Amendment or Replacement Management % of Directions Changes Replacement Currently in Age (Years) Directions All Parks Reviewed Required Required Progress <5 12 4 0 0 0 0 5–9.9 87 26 15 9 6 3 10–19.9 90 27 51 19 32 22 20–29.9 131 39 106 45 61 24 >30 9 3 7 2 5 3 No management directions 5 1 0 0 0 0 Total 334 100 179 75 104 52 208 2013 Annual Report of the Office of the Auditor General of Ontario

• A 2012 ministry review of a management At the time of our audit, the Ministry had just direction from 1989 concluded that only an completed reviewing this direction. The ecolo- administrative update was required to make gist, consistent with the Ministry’s assess- the direction compliant with the Act. How- ment, concluded that this was an outdated ever, the direction made few references to plan that did not contain the current direction the natural features within the park, despite of maintaining or restoring ecological integ- the availability of a considerable amount of rity, and that it needed to be replaced. information on them, mostly collected by universities and the federal government. The Research and Monitoring park has many endangered species, including snakes, birds and plants, but the direction did The ecologist that we retained advised us that the not contain strategies for protecting them. In maintenance and restoration of ecological integrity fact, the ecologist noted that the species that is a relatively new standard for protected area man- were at risk were mentioned only in passing. agement and its adoption into legislation makes • In its 2010 review of another direction, Ontario a global leader in this area. The fact that which dated back to 1986, the Ministry again it has a more rigorous scientific basis than older concluded that the direction complied with management standards places significant respon- the Act and needed only an administrative sibilities on the Ministry, requiring it to have the update. However, the ecologist noted that it capability to develop the following: contained only an anecdotal assessment of • detailed inventories of significant values the park’s ecological condition and no plans to within provincial parks to assess their monitor natural changes. The direction cited condition; red and white pine trees as the only significant • an ecological monitoring system within the natural value in the park and noted that many parks with defined indicators that track how had died from the impact of recreational users an ecosystem is changing; of the park. Nevertheless, the direction did • scientifically based thresholds that define

Chapter 3 • VFM Section 3.07 not contain a strategy to address this problem. when an indicator is acceptable or when a • A management direction recently approved critical condition is reached; for one park but awaiting release at the time • the ability to define, conduct and assess eco- of our audit did list ecological integrity as a logical restoration projects; and priority and aimed to protect the park’s rare • a data management and reporting system to features such as sensitive sand dunes and capture all required information. rare aquatic habitats. The direction acknow- The Ministry’s 2011 survey of park planners, ledged that recreational use had significantly ecologists, biologists and park superintendents impaired the park’s main natural features. indicated that the Ministry lacked baseline scien- However, it provided no consideration to tific data on the provincial park system. The survey restoring these values or even establishing a results revealed gaps in information with respect program to monitor the impact of continued to native biological and nonbiological components recreational use. and processes that exist in individual parks and the • The overall goal of the 1985 direction for one pressures that affect them. Our discussions with of the flagship parks in the system focused on ministry staff during our visits to zone offices and recreation. It made little provision for nature parks confirmed this lack of research data. conservation and had no plans to monitor and Each park zone has only one full-time ecolo- assess the natural conditions within the park. gist on staff. This ecologist, aided by a seasonal Provincial Parks 209 assistant ecologist and a few park biologists, is encourage research and monitoring in provincial responsible for conducting research and monitoring parks. At the time of our audit, the Ministry had not activities in all the parks within the zone. There- addressed the Board’s recommendations. fore, this one ecologist may be responsible for 20 to 50 provincial parks. As a comparison, Parks Canada RECOMMENDATION 1 informed us that each park in the federal system is assigned a science team composed of at least one To help ensure that the maintenance and park ecologist supported by a team of technicians; restoration (when necessary) of ecological the size of the team depends on the size of the park integrity is the first priority in the planning and and its ecological issues. Parks Canada further management of Ontario’s provincial park sys- supports these science teams with another team of tem, as established by the Provincial Parks and senior ecosystem scientists from the national office Conservation Reserves Act, 2006, the Ministry of who specialize in areas such as species conserva- Natural Resources (Ministry) should: tion, environmental assessment and ecological • develop an overall strategy that includes restoration. However, according to the November partnering with the outside research com- 2013 report issued by the interim Commissioner munity to ensure that sufficient baseline of the Environment and Sustainable Development, scientific data exists on native biological and Parks Canada is still experiencing a backlog of work nonbiological components and processes even with these resources. within the province’s park system, and the Universities and environmental groups also pressures that affect these; and apply to the Ministry to conduct research in the • develop a plan to adequately monitor province’s parks. Before the Ministry grants permis- changes in ecosystems within the province’s sion to these third parties, they must agree to share parks, conduct ecological restoration when any data collected. However, ecologists in the zones the need to do so has been determined, and that we visited informed us that time constraints assess the results of such restoration. often keep them from reviewing this data. Research

requests are also often unrelated to the Ministry’s MINISTRY RESPONSE Chapter 3 • VFM Section 3.07 needs. In contrast, Alberta Parks informed us The Ministry agrees with the Auditor General’s that, to gain additional research capacity, it tries recommendation that an overall strategy should to leverage outside research efforts by identifying be developed to obtain the necessary baseline knowledge gaps within its park system and setting information on biodiversity (biological and research priorities that it then communicates to nonbiological components, as well as ecological potential researchers. Alberta Parks also attempts to processes), as well as the pressures upon those provide partial funding to entice outside research- values. In 2010, the Ministry conducted a ers to conduct research it deems worthwhile. research needs survey of protected area staff and In 2009, the Ontario Parks Board, responsible managers to determine their priorities, and to for providing advice to the Minister on aspects of develop products that can be used to help focus planning, managing and developing the provincial the research of our partners. In addition, the park system, put forward a number of recommen- Ministry participates in a research consortium dations regarding research in Ontario’s provincial of academic institutions and other government parks. One was to hire a full-time manager to bodies known as Centre for Applied Science in review ministry policies surrounding research and Ontario Protected Areas. The centre’s mandate existing zone research strategies. The Board also is to facilitate and transfer applied scientific highlighted the need for new funding models to 210 2013 Annual Report of the Office of the Auditor General of Ontario

100 full-time park superintendents and assistant research that enhances policy, program develop- superintendents, in addition to their other respon- ment and on-the-ground management of sibilities, are also designated park wardens. Ontario’s protected areas. Based on our discussions with park staff and The Ministry will review approaches to mon- our analysis of enforcement activities in the six itoring and reporting on pressures and changes parks we visited, we noted the following: to ecosystems within parks. Broader landscape- In the parks we visited, the area patrolled scale monitoring of ecosystem change will occur • by enforcement staff varied significantly, as maps, databases and ecosystem classifica- ranging from five square kilometres to tions are updated. 3,900 square kilometres and averaging about The Ministry has recently partnered with 700 square kilometres. other Canadian protected area jurisdictions Due to constrained resources, enforcement under the auspices of the Canadian Parks Coun- • at operating parks is focused mainly on areas cil to develop a set of principles and guidelines known to have heavy human traffic. These for ecological restoration in protected areas. areas represent only a small portion of these These guidelines can be applied where needed parks. Therefore, significant portions of the and where resources permit. Currently, restora- operating parks, as well as all areas within tion and resource management activities occur the 220 non-operating parks that cover about annually in the province’s parks based on park 4.8 million hectares, or over half the area of and zone level priorities and within available Ontario’s provincial park system, are subject resources. The Ministry will develop a more to little or no enforcement presence. strategic approach to resource management Limited enforcement in provincial parks planning, including ecological restoration. increases the risk that violations of the Act will go undetected. Although the Ministry has not assessed the full impact of this risk, park staff Enforcement advised us that violations take place regularly in

Chapter 3 • VFM Section 3.07 provincial parks as a result of a lack of enforce- The Act specifically states that provincial parks are ment. These violations include illegal hunting, dedicated to the people of Ontario and visitors for boundary encroachments by adjacent landowners, their inspiration, education, health, recreational waste dumping, and the cutting and removal of enjoyment and other benefits, and that the parks trees and plants. are to be managed to leave them unimpaired for We raised similar concerns with respect to the future generations. To this end, activities such as lack of enforcement in our 2002 Annual Report. hunting and fishing are regulated in provincial In response, the Ministry made a commitment parks, and the Act specifically prohibits activities to undertake a review of park enforcement and such as commercial timber harvesting (except to develop a strategy for enforcement in non- in Algonquin Park) and mining. Park wardens, operating parks based on the level of risk. While who have the same authority as members of the we found that the Ministry did in fact undertake Ontario Provincial Police within a provincial a review and has developed a risk-based strategy park, are responsible for enforcing legislation for enforcement in non-operating parks, it has in provincial parks. In 2012, 360 seasonal park been unable to execute this strategy, as it lacks the wardens on two- to six-month contracts were pri- additional enforcement resources to address the marily responsible for carrying out enforcement identified risks. activities in operating parks. The approximately Provincial Parks 211

able outdoor recreation and encourage associated RECOMMENDATION 2 economic benefits. On average, each year over the To help ensure that provincial park resources last 10 years more than 9 million visits have been are adequately protected, the Ministry of made to the 114 operating parks that provide rec- Natural Resources (Ministry) should update its reational opportunities such as day-use areas and review of its risk-based enforcement strategy for overnight and interior camping. Figure 3 shows the parks and examine cost-effective strategies for number of visits in 2012/13 by provincial park zone. addressing the identified risks. Park superintendents manage the 114 operating parks, supported by full-time, seasonal, student MINISTRY RESPONSE and volunteer staff who perform various functions such as managing park revenues and expenditures, The Ministry agrees with the Auditor General’s maintaining park infrastructure, ensuring the safety recommendation to review the risk-based of visitors, delivering natural heritage education enforcement strategy and examine cost-effective programs and maintaining park facilities. In the strategies for addressing risks. Since 2002, the 2012/13 fiscal year, the operating parks gener- Ministry has allocated additional resources to ated about $69 million in revenues. As Figure 4 support custodial management needs in non- indicates, camping and day-use services offered by operating parks, which included new funding parks, and the parks’ merchandise and sales conces- for additional staff dedicated to monitoring and sions generated over 90% of these revenues. enforcement. As a result, staff have visited over In 1996, the government established a business 150 non-operating parks to complete assess- model that required operating parks to use rev- ments. Ontario Parks also receives assistance enues from park fees to fund their direct operating from conservation officers to help address non- costs, in order to enhance financial self-sufficiency. compliant activities in non-operating parks. On average, over the last five years more than 80% The Ministry has recently provided additional of park operating expenditures has been recovered funds to implement a resource stewardship through park fees. The government directly funds program to support monitoring activities in non-

capital repairs and activities related to park plan- Chapter 3 • VFM Section 3.07 operating parks and in particular land manage- ning, such as research and monitoring. ment activities. This funding includes additional As shown in Figure 5, provincial parks located human resources to address concerns regarding in the southern and central parts of Ontario, where non-compliant activities occurring in those parks. the population is larger, are able to generate rev- The Ministry will regularly review the risk- enues greater than their operating costs. This helps based enforcement strategy for both operating and non-operating parks and update the strat- Figure 3: Operating Park Visits by Park Zone, 2012/13 egy as new or changing regulatory requirements Source of data: Ministry of Natural Resources are introduced. # of Operating # of Park Zone Parks Visits PARK OPERATIONS Central 20 3,036,813 Southwest 18 2,061,244 Visits and Revenues Southeast 21 1,901,968 As noted earlier, one of the key objectives of the Algonquin 1 828,372 Act in establishing and managing provincial parks Northeast 36 749,663 is to provide opportunities for ecologically sustain- Northwest 18 615,478 Total 114 9,193,538 212 2013 Annual Report of the Office of the Auditor General of Ontario

to subsidize parks located in the north where visits erated by these seven parks over the last four years tend to be fewer and a smaller percentage of the on average recovered less than half of their operating operating costs is recovered. costs, and capital repairs of approximately $2.5 mil- In September 2012, the Ministry announced lion were expected to be needed. We therefore that it was changing the designation of 10 parks concluded that the Ministry, from its perspective, (all but one of them located in Northern Ontario) had valid financial reasons for changing the status of from operating to non-operating, citing these parks’ these parks from operating to non-operating. low visiting rates and inability to recover much of their operating costs through the limited revenues Park Fees they generate. In changing the status of these 10 parks, the Ministry expected to save approximately While we acknowledge that recovering park operat- $1.6 million in annual operating costs and $4.4 mil- ing expenses is a worthwhile goal, we note that lion in capital repairs. In January 2013, the Ministry park fees in Ontario are already among the highest retracted this decision for three Northern Ontario of any province in Canada, as indicated in Figure 6. parks, stating that it would work with the affected As seen earlier, parks located in the southern municipalities to continue operating the parks with and central parts of Ontario, where the population the goal of increasing their revenue and visiting is greater, are generally more popular and have rates. We reviewed statistics supporting the decision significantly more visits than parks located in the to keep the remaining seven parks closed and noted northern parts of the province. Fees for day use and that these parks combined had averaged only about overnight camping differ according to the location 70,000 visitors annually over the last four years, or and popularity of a park in addition to the activities less than 1% of the total number of annual visitors to (for example, skiing, hiking, swimming, boating, all provincial parks combined. In addition, fees gen- wildlife viewing) and amenities that the park has to offer. The Ministry has not fully explored how Figure 4: Park Revenues by Source, 2012/13 ($ 000) further varying provincial park fees based on popu- Source of data: Ministry of Natural Resources larity (increasing fees in parks that are currently

Chapter 3 • VFM Section 3.07 Total Revenue: $69,310 operating at or near capacity and lowering fees in the less visited parks, mainly in the north) could Equipment rental4, $860 (1%) affect visits and revenues, and hence cost recovery. Fines and penalties, $1,370 (2%) Other 3, $1,836 (3%) Figure 5: Cost Recovery by Park Zone, 2012/13 Source of data: Ministry of Natural Resources Land leases 2, $2,462 (3%)

Sales1, Operating Cost $8,046 (12%) Revenue Costs Recovery Park Zone ($ million) ($ million) (%) Day use, Southwest 18,052 14,993 120 $7,733 (11%) Central 15,851 13,560 117 Southeast 14,328 12,896 111 Camping, $47, 003 (68%) Algonquin 10,485 10,071 104 Northeast 6,276 9,638 65 1. Sales revenues include revenues from concessions, merchandise sales, vending, and sales of firewood and camping supplies. Northwest 3,972 6,960 57 2. Land lease revenues are from private cottage leases in Algonquin and Subtotal 68,964 68,118 Rondeau Provincial Parks. 3. “Other” includes revenues from donations, trailer storage, Parks Guide Head Office 346 12,600 advertising, etc. 4. Equipment rental is rental of canoes, boats, skis, picnic shelters, Total 69,310 80,718 86 barbecues, etc. Provincial Parks 213

Figure 6: Comparison of Ontario’s Camping and Day-use Fees with Fees in Other Provinces ($) Prepared by the Office of the Auditor General

ON BC AB MB SK Camping 31.36–48.31 11.00–30.00 5.00–23.00 11.55–28.35 13.00–26.00 Day use — vehicles 10.75–20.00 Free Free 5.00 7.00

Note: Fees include all applicable taxes. Fees for camping and day use vary according to the facilities and services provided, and the popularity of the park.

to waste management, the Ministry’s standards RECOMMENDATION 3 currently require central trash containers and day- To help increase overall visits to provincial use containers to be emptied twice a week during parks, draw more visitors to underused parks periods of high and moderate use, once a week dur- and increase its revenue from the provincial ing periods of low use, and as required during the park system, the Ministry of Natural Resources off-season. Similarly, with respect to maintenance (Ministry)should assess the impact on visits and of facilities and grounds, the Ministry’s operating revenues that would result from reducing fees in standards require litter to be picked up twice a the less visited parks and increasing fees in the week in public areas during high-use periods and more popular parks that are currently operating once a week during moderate-use periods. at or near capacity. While we found that the parks we visited met the Ministry’s minimum operating standards, we MINISTRY RESPONSE noted that the standards were established over 20 years ago. Visits have since increased by over The Ministry acknowledges the Auditor Gener- 40% and, therefore, the standards may no longer al’s recommendation and will assess the current be appropriate. There is evidence that the current park fee structure as well as research the fee operating standards do not meet the expectations structures of other jurisdictions to consider their of many visitors. applicability within our program. A differential

The 2011 visitor survey conducted by the Min- Chapter 3 • VFM Section 3.07 fee system is already in place that results in istry found that only 57% of day-use visitors were lower fees in Northern Ontario than in Southern satisfied with the cleanliness of the washroom and Ontario. The Ministry implemented reduced shower facilities. The rating was higher among fees in 2007/08 with limited success. Ontario overnight campers, at 70%. Similarly, only 57% Parks undertakes an annual review of its fees to of day-use visitors were satisfied with the level of determine which fees may require adjustment enforcement of park rules. Again, the rating among and measures customer reaction to fees through overnight campers was higher, at 77%. Overall, regular consumer research. when day-use visitors and campers were asked how parks could be improved, better general maintenance and amenities were at the top of the list. Operating Standards RECOMMENDATION 4 The Ministry has established minimum operating standards covering, among other things, security In light of the significant increase in visits to and enforcement, waste management, sanita- provincial parks since the Ministry of Nat- tion, and cleaning and maintenance of buildings, ural Resources (Ministry) last set minimum facilities and grounds. For example, with respect operating standards for, among other things, 214 2013 Annual Report of the Office of the Auditor General of Ontario

update the system as required. The Ministry relies security and enforcement, waste management, on the completeness and accuracy of the informa- sanitation, and cleaning and maintenance of tion in this system to make key management deci- buildings, facilities and grounds, the Ministry sions, including how to allocate capital funding should review and update its standards. In among parks. However, based on our discussions addition, the Ministry should continue to con- with staff in the zones and parks that we visited and duct visitor surveys and monitor the results to our review of the parks’ capital asset listings, we ensure that visitor expectations are met. found the following: Park staff did not verify the existence and con- MINISTRY RESPONSE • dition of assets listed in the Ministry’s system The Ministry agrees with the recommendation as required in ministry procedures. In most and is in the process of revising the minimum cases, the asset condition listed was the same operating standards. As a result of the level of as the state of the asset when it was initially use, many parks currently exceed these min- acquired and entered into the system. The imum standards; for example, some washrooms Ministry’s asset management system was also are cleaned three times per day rather than not updated regularly to reflect new or deleted twice as stated in the standards. Some parks assets. have enforcement coverage for 12, 14 or even 24 • The value of the assets in the Ministry’s asset hours a day compared to the minimum standard management system had been significantly of eight hours during the peak season. misstated. As a result of our inquiries, the The Ministry conducts park user surveys that Ministry significantly reduced the value of the have been a successful measure of customer assets in its asset management system after it feedback for over 30 years. We will continue discovered numerous recording errors. The to conduct the survey program on its current errors were mainly a result of the inaccurate three-year cycle. recording of pooled assets. We also noted a significant current backlog of

Chapter 3 • VFM Section 3.07 required capital asset expenditures in the Ministry’s asset listings. Specifically, over one-third of the Capital Asset Management buildings and structures in the provincial park Capital assets within the province’s parks include system were listed as being at, near the end of or buildings (for example, visitor centres, roofed beyond their service life. In its asset listings, the accommodations, comfort stations, offices, main- Ministry estimated the total cost to replace these tenance buildings), machinery and equipment, buildings and structures to exceed $300 million. drinking water systems, campsites, roads and trails, Other assets, such as roads, bridges and septic and bridges. In the 2011/12 fiscal year, the Ministry systems, that were listed as being in “poor” or publicly reported the replacement value of the “defective” condition in the Ministry’s listings capital assets in Ontario’s provincial parks to be had an estimated replacement cost that exceeded $1.2 billion. $280 million. Figure 7 lists some of these assets Each individual park is responsible for main- that, based on the assets’ age, the Ministry has taining up-to-date information on its own assets determined to be in “poor” or “defective” condition. within the Ministry’s asset management system. In Also, at the time of our audit, 25 of the 181 fact, ministry procedures require each park to verify drinking water systems in individual parks were on the existence and condition of each asset listed in a “boil water” advisory. Eighteen of these advisories its asset management system every two years and have been in place for nine years. The Ministry of Provincial Parks 215

Figure 7: Park Assets Considered “Poor” or “Defective” by the Ministry Based on Their Age Source of data: Ministry of Natural Resources

Assets in “Poor” or Estimated Cost Total # “Defective” Condition to Replace Asset Category/Type of Assets # % of Total ($ million) Small Machinery and Equipment 2,358 1,282 54 32.0 Sanitation Facilities Sewage lagoons 14 14 100 11.2 Septic systems 938 598 64 29.9 Infrastructure Bridges 53 48 91 36.0 Footbridges 130 77 59 2.3 Roads 2,000 km 1,400 km 70 84.0 Chain and wire fencing 98 km 85.4 km 87 6.2

Health’s local Public Health Units had completed MINISTRY RESPONSE risk assessments and issued reports for 110 of the Ministry’s 181 drinking water systems; after The health and safety of park staff and visitors assessing these reports, the Ministry projected that are of paramount importance to the Ministry. 42 drinking water systems required improvement The Ministry continues to ensure that any or replacement. The cost of the improvements infrastructure deficiencies that may pose a and replacements was estimated to be about threat to health and safety are corrected and $11 million. will continue its ongoing efforts to restore the Since our last audit of provincial parks in 2002, parks’ infrastructure with available resources. the backlog of required capital asset expenditures The Ministry has invested over $100 million to has increased by approximately $170 million. In improve more than 50 drinking water systems the 2012/13 fiscal year, the Ministry spent only in Ontario parks since 2001 and has commit- Chapter 3 • VFM Section 3.07 $13 million on capital assets, and over the next five ted additional capital funds beginning with years the Ministry’s spending on capital assets is the 2013/14 fiscal year to continue to address expected to average only about $15 million annu- high-priority projects, such as drinking water ally. At this rate of spending, as existing assets con- systems, and to increase park sustainability. tinue to age, the Ministry’s backlog of capital asset The Ministry accepts the Auditor General’s expenditures will continue to grow. finding regarding the asset management system and will undertake the development of a system RECOMMENDATION 5 that contains complete and accurate information on the condition and value of capital To ensure that park infrastructure is in a satis- assets in each park. The Ministry is currently factory state, the Ministry of Natural Resources developing an updated asset management plan (Ministry) should take action to correct infra- for Ontario parks and is working collaboratively structure deficiencies already identified. The with program areas to implement processes that Ministry should also ensure that its asset man- support the plan. agement system contains accurate, complete Maintaining an accurate and up-to-date asset and up-to-date information on the condition management system is a concern for many park and value of the parks’ capital assets. 216 2013 Annual Report of the Office of the Auditor General of Ontario

the plan in one of the remaining two zones had not programs across Canada. The Ministry is part been reviewed in 20 years. In addition, of the 43 of a broader Asset Management Working Group operating parks with interpretive NHE programs, involving federal, provincial and territorial park only about half had an updated NHE operating plan jurisdictions to develop a comprehensive picture in place. of the state of Canada’s park assets. The group The most recent visitor survey conducted by the will also complete a jurisdictional scan to see Ministry in 2011 indicated that educational pro- what types of software-based asset management grams are underutilized, for example: systems are in place. • only 8% of day visitors and 18% of overnight campers surveyed said that they had taken part in educational programs; and NATURAL HERITAGE EDUCATION • 35% of day visitors and 18% of overnight campers surveyed said they did not know the As noted earlier, a key objective of the Act in programs were available. establishing and managing parks is to provide In 2011, the Ministry also conducted a stra- opportunities for residents of Ontario and visitors tegic review of its NHE programs and found the to increase their knowledge of Ontario’s natural following: and cultural heritage. Natural Heritage Education There has been very little change in the types (NHE) is offered by the Ministry in 43 of the most- • of interpretive programs offered over the last visited operating parks. NHE is designed to educate few decades. As a result, in some locations visitors on the natural and cultural heritage of the attendance in these programs has declined. parks and their surrounding areas. The Ministry Many parks with NHE programs did not have uses predominantly seasonal staff and students • a comprehensive NHE plan, and many existing to present interpretive programs in these parks plans were very outdated. that include guided walks, children’s programs, The NHE program collects quantitative data, evening programs, night hikes and special-event • such as the number of people attending an weekends. An additional 64 parks provide self-use

Chapter 3 • VFM Section 3.07 interpretive program, but very little qualita- NHE activities in which education is carried out tive data about the success and outcomes of through signs, park tabloids and trail guides, but the interpretive program. The trend toward with no park staff to provide interpretive programs. having more students present NHE programs In the 2012/13 fiscal year, the Ministry spent has also negatively affected the quality of the approximately $2.5 million on NHE programs and programs being delivered to the public. estimated that approximately 2.8 million visitors Smaller parks do not get the direction or participated in an NHE program that year. • attention needed from senior zone personnel Ministry policies require an NHE plan to be pre- to develop and present effective programs for pared for each park zone. These zone plans are to the public. be reviewed and updated every 10 years, or as new At the time of our audit, the Ministry was in the parks are established in the zones. In addition, indi- process of implementing some changes to address vidual NHE operating plans that provide direction concerns raised about its NHE program from its for the NHE programs are to be prepared for each of strategic review and visitor survey. the 43 parks that provide staff-led interpretive programs. These park operating plans are to be evaluated and updated annually. At the time of our audit, four of the six zones did not have an NHE plan, and Provincial Parks 217

There is no requirement to limit the report to these RECOMMENDATION 6 broad areas, however. To ensure that Natural Heritage Education The first State of Ontario’s Protected Areas (NHE) programs meet visitor expectations and Report (SOPAR) was released by the Ministry in program objectives, the Ministry of Natural 2011. We reviewed the SOPAR and noted that it Resources (Ministry) should develop or update meets the minimum reporting requirements of the NHE plans in all zones and parks that offer NHE Act. However, when we compared the SOPAR with programs. The Ministry should ensure that the similar reports in other jurisdictions, we noted the plans address the concerns that were noted in following: its 2011 strategic review of NHE programs. • The SOPAR provides an overview of the Min- istry’s management planning process for pro- MINISTRY RESPONSE tected areas, but does not provide the status of management plans for individual parks. In The Ministry agrees with the Auditor General’s comparison, Parks Canada and Parks Victoria findings regarding the NHE program. Ontario in Australia both reported on the status of Parks has the largest interpretive program in park management plans for all established Canada. In 2011 the Ministry completed a stra- parks within their jurisdictions, including the tegic review of the program; recommendations number of parks with completed plans and the included reviewing traditional interpretive pro- age of existing plans. In 2008, Parks Canada grams, developing new methods for delivering started preparing a state of the park report for effective interpretation, and demonstrating a each park in the federal system. These reports stronger link between the NHE program and highlight actions taken at individual parks Ontario Parks objectives. The Ministry will con- and the results of those actions relative to key tinue to implement these recommendations. objectives in their management plans. The Ministry agrees with the Auditor The SOPAR provides only a general discussion General’s finding that NHE plans should be • of threats such as climate change, water and developed for all zones and parks offering the

air pollution, invasive species and fire to the Chapter 3 • VFM Section 3.07 program. Updated NHE plan guidelines and park system as a whole. It does not speak to document templates to facilitate these plans will specific threats and their impact on key values be prepared and distributed to the zones. in individual parks. There is also no assessment in the SOPAR of the extent to which ecological integrity is being maintained in REPORTING individual parks and in the park system, nor is there an assessment of areas in parks where The Act requires the Minister to publicly report on ecological integrity needs to be restored. Parks the state of the provincial park and conservation Canada informed us that, in comparison, reserve system at least every 10 years. The report it established indicators that track changes should assess the extent to which the objectives of in ecosystems within individual parks and the provincial parks and conservation reserves set thresholds that define when an indicator is out in the Act are being achieved. It should also acceptable or signifies a critical condition. detail the number and size of the provincial parks Parks Canada reports discuss the current trend and conservation reserves, their ecological health in these indicators. and any known threats to their ecological integrity, While reporting on the status of relation- and the socio-economic benefits they provide. • ships with Aboriginal people is not a specific 218 2013 Annual Report of the Office of the Auditor General of Ontario

requirement of the Act, we noted that other density and lack of available land preclude estab- jurisdictions tended to report on their rela- lishing such large parks. In addition, the Ministry tionships with these communities in planning reported in the SOPAR that it has been 65% suc- and managing their parks. For instance, cessful in achieving its plan to establish natural Parks Canada reported on recent actions it environment parks throughout the province. How- had taken with respect to Aboriginal com- ever, our analysis suggests that the Ministry has munities in five areas: building meaningful been only 48% successful. relationships, creating economic partnerships, Similarly, for the Act’s objective of providing the increasing Aboriginal interpretation pro- population with opportunities for ecologically sus- grams, enhancing employment opportunities tainable outdoor recreation, the Ministry has set a and commemorating Aboriginal themes. target of 1.3 day visits and 0.5 camping days per year Similarly, BC Parks reported on the number of by every individual living within a two- to three- collaborative management agreements with hour travel radius of a provincial park. However, the First Nations in British Columbia’s protected Ministry does not track its success in meeting these areas; New South Wales in Australia reported targets. In addition, Ontario’s population has grown on the state of Aboriginal cultural heritage by over 60% since these targets were established in in protected areas and the park system’s 1978. The Ministry has not assessed whether the role in protecting and promoting Aboriginal parks have the capacity to accommodate this num- objectives, places and features of value. The ber of visits in an ecologically sustainable manner, SOPAR is silent on the state of the relation- given the province’s population growth. ships between the Ministry and Aboriginal The Ministry also has not established any bench- communities, even though they are significant marks to evaluate its performance in meeting the stakeholders in Ontario’s provincial park Act’s requirements to maintain ecological integrity system. in provincial parks, to provide residents with oppor- • Unlike the SOPAR, some jurisdictions also tunities to increase their knowledge of Ontario’s reported on the condition of capital assets natural and cultural heritage, and to facilitate sci-

Chapter 3 • VFM Section 3.07 such as buildings, dams and bridges. entific research to support monitoring of ecological The Ministry has established performance change in the parks. measures for only two of the four objectives of the Act noted earlier. To gauge its performance in RECOMMENDATION 7 relation to the objective of permanently protecting The Ministry of Natural Resources (Ministry) ecosystems, the Ministry has established six classes should compare its State of Ontario’s Protected of provincial parks in Ontario, with each class hav- Areas Report (SOPAR) with similar reports in ing specific purposes and permitted uses. Specific other jurisdictions to identify and emulate best targets have been set for the number, size and practices in reporting. The Ministry should also distribution of some classes of parks throughout set appropriate benchmarks and collect the the province. For example, the Ministry’s target is information it needs to assess its performance to establish wilderness-class parks of not less than against all four legislated objectives for the 50,000 hectares and averaging at least 100,000 effective management of Ontario’s parks, and hectares in each of 14 predetermined sectors across present the results in future reports. the province. The Ministry has reported that it has been only 57% successful in meeting this target, which may not be a realistic one, especially in the southern part of the province where population Provincial Parks 219

parks, but at the time of our audit it had not yet MINISTRY RESPONSE received the results. Infrequent ministry inspections The Ministry acknowledges this recommen- of these cottages indicate that some leaseholders dation and will build on the comparisons have encroached on public park lands outside the completed, and other best practices identified boundaries of their leased areas. to date, when developing the next SOPAR, The lease payments for the cottage properties as it did during the development of its first typically range from $1,500 to $2,000 per year. In SOPAR in 2011. The Ministry will consider the addition to the annual lease payments, each cot- development of benchmarks as appropriate tage owner pays an annual fee that typically ranges indicators through the process of completing from $204 to $421 for services such as garbage the next SOPAR. As mandated by the Provincial removal. Further, the majority of owners do not pay Parks and Conservation Reserves Act, 2006, the municipal property taxes. In September 2012, the role of the SOPAR is to report on the system Ministry contracted a consulting firm to assess the of protected areas, rather than individually on net economic value of these leases. The consultant Ontario’s over 600 provincial parks and con- concluded that the private leaseholders were enjoy- servation reserves. ing a benefit that was not available to other Ontar- ians, specifically: • Revenue from the lease payments is significantly below fair market value. The consultant OTHER estimated that at fair market value, the lease Privately Leased Lands payments should generate approximately $6.7 million more in revenue than the Ministry Currently, there are nearly 600 private cottage currently receives from the lease payments. properties held under lease in two provincial parks. The fee charged for services is also well below These lease agreements were initially entered into • the Ministry’s actual cost of providing these in the late 1800s and the early 1900s. In 1954, the services. The Ministry collects approximately government enacted the Provincial Parks Act, which

$182,000 annually in service fees, but incurs Chapter 3 • VFM Section 3.07 stipulated that no new leases were to be approved about $474,000 in actual costs. and existing leases were to be phased out as their terms expired. Nevertheless, the government RECOMMENDATION 8 continued to renew the leases. The term of the current leases is expected to end in 2017 and is under The Ministry of Natural Resources (Ministry) review. The existing leases permit leaseholders to should, once its study is complete, act to miti- sell the cottages on the leased land, with the leases gate any negative environmental and economic then automatically transferring to the new owners. impacts posed by private cottages in the two Over the last 10 years, there have been 10 such provincial parks identified. If the decision is sales ranging from $60,000 to $500,000. made to renew these leases in 2017, the Ministry While the current Act does not allow the Min- should ensure that the lease payments are ister to enter into new leases, it does allow the increased to at least fair market value and that Minister to extend existing leases, providing that the fees charged for services to the cottagers the extensions are consistent with all the require- recover the Ministry’s cost of providing the ments of the Act. In this regard, the Ministry had services. commissioned a study on the environmental and economic impact the cottages have had on the two 220 2013 Annual Report of the Office of the Auditor General of Ontario

MINISTRY RESPONSE Reservation and Registration Services At the time of our previous audit in 2002, reserva- The Ministry appreciates the Auditor General’s tion and registration services were provided by a recommendation on the private cottages in private contractor. When the Ministry’s 10‑year two provincial parks. The Ministry has recently agreement with the contractor ended in 2009, a completed economic and environmental studies request for proposals was issued for a new reserva- regarding the private cottages on leased land in tion system. A new 10‑year, $25‑million contract the two provincial parks, and is examining the was awarded to a new contractor that was the low- results to consider the financial arrangements est bidder, effective November 2009. between the cottagers and the Crown as well The Ministry, however, claimed that this new as the environmental impacts posed by the cot- contractor was unable to provide contract deliv- tages and their use. erables with respect to hardware and software Should the government decide to renew the development, and that it did not meet service‑level leases in 2017, it will consider an updated fee requirements for the call centre and for Internet structure that will move toward ensuring that connectivity. Accordingly, the Ministry terminated the province receives a fair rate of return for use its agreement with this contractor effective Octo- of the land and recovering its costs of providing ber 31, 2010, and the Deputy Minister approved services to the cottagers. If the government the awarding of the contract to the second-ranked decides to renew the leases in 2017, the Ministry bidder in the 2009 request for proposals, which will also develop lease conditions intended to was the contractor whose 10-year agreement had address environmental impacts. In the mean- expired. As a result of the termination, the new time, the Ministry will continue to monitor and contractor filed a Statement of Claim against the enforce the current lease conditions to help Ministry and the original contractor in September address ongoing environmental impacts. 2011. The new contractor is claiming substantial damages against the Ministry for breach of contract. At the time of our audit, the lawsuit was Chapter 3 • VFM Section 3.07 ongoing. The reservation and registration system put in place by the replacement contractor was working well at the time of our audit. Chapter 3 Ministry of Health and Long-Term Care Section 3.08 Rehabilitation Services at Hospitals

• are eligible for Ontario Works or the Ontario Background Disability Support Program. Publicly funded rehabilitation for eligible persons includes services provided at: DESCRIPTION OF REHABILITATION • hospitals—both inpatient and outpatient clin- Rehabilitation services in Ontario generally provide ics for registered patients; support to people after certain types of surgery and • patients’ homes; to people with injuries, chronic conditions and dis- • until August 2013, 90 privately owned physio- abilities, to help them regain, maintain or improve therapy clinics that had Ontario Health Insur- their health and to carry out their daily activities. ance Plan (OHIP) billing privileges; and Rehabilitation services can include, among other • after August 2013, at privately owned or hos- things, physiotherapy, occupational therapy, pital-based physiotherapy clinics with which speech-language pathology, social work and nurs- the Ministry contracts to provide services. Chapter 3 • VFM Section 3.08 ing. (For definitions of “rehabilitation” and other Individuals not eligible for publicly funded terms, see the Glossary at the end of this report.) rehabilitation can access private-pay services from community rehabilitation providers and certain hospital-based outpatient programs. These patients ELIGIBILITY FOR REHABILITATION pay for the services themselves if they are not The Ministry of Health and Long-Term Care covered by a private insurance provider or the (Ministry) funds rehabilitation services for eligible Workplace Safety and Insurance Board. Ontarians. This includes all hospital rehabilitation inpatients and hospital-registered outpatients. In TYPES AND EXTENT OF INPATIENT terms of community-based services, the Ministry REHABILITATION funds physiotherapy only for patients who: • are 19 and under or 65 years of age and over; The Ministry funds inpatient rehabilitation services or in 61 hospitals through the province’s 14 Local • have spent at least one night in hospital prior Health Integration Networks (LHINs), which are to rehabilitation; or accountable to the Ministry. Inpatient rehabilita- • require physiotherapy at home or reside in a tion in Ontario can be shorter-term in nature, with long-term-care home; or frequent rehabilitation sessions (known as regular

221 222 2013 Annual Report of the Office of the Auditor General of Ontario

rehabilitation) or longer-term in nature (known as 30% increase in this population is expected. An restorative or slow-paced rehabilitation) for people even greater increase is anticipated after 2021, unable to participate in frequent sessions. The 61 when baby boomers —those born between 1946 hospitals have almost 2,500 regular rehabilitation and 1964—will start to turn 75. As a result, the beds to which more than 30,000 patients were demand for rehabilitation services is expected admitted in the 2012/13 fiscal year. As Figure 1 to increase significantly. Rehabilitation can help shows, in 2012/13, orthopedic conditions (includ- people who are aging or living with various health ing hip and knee replacements) and stroke were conditions maintain the functioning they have. the most common reasons for admission to regular rehabilitation inpatient programs. The Ministry did not have information available on the total public funding spent on rehabilitation services. Province- Audit Objective and Scope wide information was not available on the number of restorative rehabilitation beds and associated The objective of our audit was to assess whether admissions. As well, the Ministry did not have infor- selected hospitals have effective processes in place mation on the total number of patients attending to ensure that patients have access to rehabilitation or how often they visited hospital-run outpatient programs, including services and equipment, based programs. on their needs, and that oversight practices are in place to monitor the cost-effectiveness of these programs. Senior ministry and hospital manage- FUTURE NEED FOR REHABILITATION ment reviewed and generally agreed to our audit In the 2012/13 fiscal year, about half of inpatients objective and associated audit criteria. admitted to hospital for regular rehabilitation were Our audit focused on rehabilitation services over 75 years of age. Between 2012 and 2021, a provided by hospitals because hospitals provide a large portion of publicly funded rehabilitation Figure 1: Percentage of Regular1 Rehabilitation services. We conducted our audit work at three

Chapter 3 • VFM Section 3.08 Inpatient Admissions by Condition, 2012/13 different hospitals across the province that provide Source of data: Ministry of Health and Long-Term Care rehabilitation services: Hamilton Health Sciences Stroke (16%) Corporation, with 129 regular and 44 restorative Brain Injury rehabilitation beds; The Ottawa Hospital, with (5%) 80 regular rehabilitation beds; and Providence Neurological (2%) Healthcare, a Toronto hospital that provides only Other 2 (26%) Spinal Cord rehabilitation services, with 87 regular and 140 Injury (3%) restorative rehabilitation beds. The three hospitals offer rehabilitation for a variety of more common patient conditions, which can include joint replacement surgery and stroke. The Hamilton Health Sciences Corporation and The Ottawa Hospital also Major Multiple Amputation Trauma (2%) (4%) offer specialized rehabilitation, such as programs for patients with spinal cord injuries and acquired Lung Disease Orthopedic (3%) (34%) brain injuries. We did not audit privately owned physiotherapy Cardiac (5%) clinics that are publicly funded or home-based 1. Figure excludes restorative rehabilitation beds because province-wide information was not available. rehabilitation services provided by Community 2. Includes various conditions, such as arthritis, burns, infections and spina Care Access Centres. bifida. Rehabilitation Services at Hospitals 223

The scope of our audit included the review patient care. As a result, a patient deemed eligible and analysis of relevant files and administrative for services at one hospital might not be eligible for policies and procedures, as well as interviews with similar services at another. appropriate hospital and ministry staff. We also Although there are minimal waits for most reviewed relevant research, including best practices people determined by hospitals to be eligible for for rehabilitation in other jurisdictions. In addi- regular inpatient rehabilitation, there is a lack of tion, we held discussions with senior management information on those who are rejected. The one at each of the Local Health Integration Networks hospital we visited that tracked this information associated with the three hospitals audited. We also rejected almost 40% of patients referred for regu- obtained the perspective of the Ontario Hospital lar—that is, shorter-term—rehabilitation and over Association, which represents Ontario hospitals; 20% of applicants referred for restorative—that the GTA (Greater Toronto Area) Rehab Network, is, longer-term—rehabilitation. Hospitals have which represents hospitals and community-based closed many outpatient programs over the last 10 organizations involved in planning for and provid- years. Wait times for outpatient programs range ing rehabilitation services; and the Ontario Physio- from immediate access, to a few days, to a couple therapy Association, which represents Ontario’s of years. registered physiotherapists. As well, we engaged The Ministry has recently begun several initia- the services of two independent experts in rehabili- tives aimed at improving the rehabilitation system, tation services to advise us. which may help to address some of our recommendations. This includes expanding the role for the Rehabilitative Care Alliance, a group tasked with building on the Rehabilitation and Complex Summary Continuing Care Expert Panel’s framework for rehabilitative care planning. There is a need for a provincially co-ordinated Some of our more significant observations are as rehabilitation system. Ontario’s population is aging, follows:

so there will be an even greater need for rehabilita- Chapter 3 • VFM Section 3.08 tion services in the future. This is especially true Ministry Co-ordination of Rehabilitation System given that two of the main conditions requiring There is a wide variation in the supply of rehabilitation services—stroke and orthopedic con- • regular rehabilitation inpatient beds across the ditions, such as knee and hip fractures—are more province—a situation that may require people prevalent in older people. Rehabilitation services to travel outside their LHIN for rehabilitation across the province have evolved over many years services. The number of beds ranges from 57 such that there are now significant variations in the per 100,000 people in the Toronto Central availability and type of services provided, which LHIN to only six per 100,000 people in the can impact patient access to services. Central West LHIN, with a provincial average The lack of a co-ordinated system has led to of 18 beds per 100,000. Further, according to individual hospitals—some with input from their a 2011 joint report by the Orthopaedic Expert Local Health Integration Network (LHIN)—gener- Panel, the Ontario Physiotherapy Association ally determining which inpatient and/or outpatient and others, the availability of outpatient pro- rehabilitation services they will offer, if any. As grams was inconsistent across LHINs and there such, each hospital establishes its own policies and was little information on the demand for servi- procedures for determining patient eligibility for its ces, service capacity and service accessibility. services, prioritizing eligible patients and providing 224 2013 Annual Report of the Office of the Auditor General of Ontario

• It is difficult for the Ministry or the LHINs to as rehabilitation hospitals, were occupied by determine system capacity because there is ALC patients waiting for post-discharge care, a lack of system-wide information available such as home-care services or accommodation for decision-making on restorative inpatient in a long-term-care home, making these beds rehabilitation and outpatient rehabilitation. unavailable for other patients requiring acute Further, the Ministry had limited information care or rehabilitation. on the actual use of complex continuing care • There is no listing, such as on a website, that beds in hospitals. Hospitals may use these patients and their families can access of all beds for a wide range of purposes, including publicly funded rehabilitation services avail- restorative rehabilitation. Unlike regular able in the province, by LHIN or otherwise. inpatient rehabilitation, there is no system- The GTA (Greater Toronto Area) Rehab wide information available to the Ministry or Network has made a good start, listing by hos- the LHINs on the extent to which restorative pital and by Community Care Access Centre inpatients or outpatients improve as a result of (CCAC) the rehabilitation services offered the therapy received. Therefore, the effective- across the GTA. ness of inpatient restorative or hospital-based outpatient rehabilitation services provided is Hospital Services not tracked overall. All three hospitals we visited were managing vari- Approximately one-third of patients admitted • ous processes well for determining patient access to to inpatient rehabilitation at the two hospitals their rehabilitation programs, and all had a range of we visited with stroke programs had been oversight practices in place. However, all had areas assessed by an acute hospital as having mild for improvement. functional impairment. This suggests that they With the exception of stroke, for most condi- might have been better served in outpatient • tions requiring rehabilitation, there are few programs if these less costly services were best practice standards in Ontario for such available. Further, the Ontario Stroke Network matters as when therapy should start, how

Chapter 3 • VFM Section 3.08 reported in 2012 that implementation of best often it should occur and what type of treat- practices related to stroke, such as serving ment should be provided. Not unexpectedly, people with mild functional impairment in the hospitals we visited varied in their practi- an outpatient setting, would have a positive ces and, therefore, patient care varied. impact on patient outcomes while resulting in Hospitals generally met ministry requirements savings of about $20 million per year. • to discharge total joint replacement—that Patients no longer requiring hospital care • is, total hip and knee replacement—patients may occupy beds needed by other patients. from acute-care hospitals in 4.4 days, with at A report by the Ontario Hospital Association least 90% of them returning home and a max- indicated that as of March 2013, about 2,300 imum of 10% sent to inpatient rehabilitation. alternate-level-of care (ALC) patients who However, patients might experience waits for were ready to be discharged were waiting in associated outpatient rehabilitation. acute-care hospital beds for post-discharge At the three hospitals visited, the median care arrangements. Of these, 25% were • time to determine outpatient eligibility waiting for a regular rehabilitation bed or ranged from the same day, to five days, to a complex continuing care (which includes 19 days from the date of referral. This could restorative rehabilitation) bed. In addition, impact when patients start their outpatient 13% of beds in post-acute-care facilities, such rehabilitation. Rehabilitation Services at Hospitals 225

Two of the three hospitals visited did not offer • Ontario is investing $156 million a year outpatient rehabilitation services during even- • to support access to physiotherapy, and to ings or weekends. Patients who work during enhance exercise and fall prevention services the day may not be able to attend. to more than 200,000 additional seniors and At the hospitals we visited, there was gener- • eligible patients. ally no replacement coverage for therapists The LHINs have established and retain who were absent due to illness or vacation, so • oversight of the Rehabilitative Care Alliance at times there were fewer therapists available (Alliance). The Alliance will provide a sup- for the same number of patients. Further, port system for improving access, efficiency, although therapists determine the extent effectiveness, quality, integration, value of therapy each patient is to receive and are and equity in the delivery of rehabilitative responsible for providing this level of therapy, services across the care continuum. Its man- we were unable to determine how much ther- date includes endorsing or, where absent, apy patients actually received. This is because, developing best practice guidelines to although the hospitals and the therapists’ pro- enhance outcomes and increase community fessional colleges required some documenta- capacity. tion of therapy, none required documentation The LHINs have also recently undertaken an of all sessions each patient attended. • Integrated Orthopaedic Capacity Planning exercise to identify opportunities for opti- OVERALL MINISTRY RESPONSE mizing orthopaedic capacity across settings, The Ministry of Health and Long-Term Care including rehabilitation services in hospitals (Ministry) welcomes the advice and recom- and outpatient clinics. mendations contained in the value-for-money • Under Health System Funding Reform, audit of hospital-based rehabilitation services. the Ministry and LHINs are implementing The audit acknowledges the processes already an Integrated Quality-Based Procedure in place with respect to patient access and Scorecard under which providers—including oversight practices. A number of initiatives are hospitals providing rehabilitation services— Chapter 3 • VFM Section 3.08 also being implemented collaboratively by the will report on indicators of effectiveness, Ministry, Local Health Integration Networks appropriateness, integration, access and (LHINs) and Community Care Access Centres value, including for rehabilitation services. to further strengthen the rehabilitation system, To this end, Health Quality Ontario has con- with a goal of ensuring that patients receive vened a Hip/Knee Expert Panel to develop timely care in the most appropriate setting. For additional best practices on targets for total example: joint replacement procedures. • In the 2012 Ontario Budget, the government increased investments in home care and community services by an average of 4% annually for the next three years to ensure Detailed Audit Observations that there is capacity to care for people outside the hospital setting. Ensuring that INITIATIVES patients receive the right care in the right place is essential for high-quality service and In recent years, the Ministry of Health and Long- for managing health-care costs. Term Care (Ministry) has supported a number of initiatives that it indicated are intended to improve, 226 2013 Annual Report of the Office of the Auditor General of Ontario

among other things, the rehabilitation system, years, funding based on three components. including the following: Thirty percent will be based on historical • Resource Matching and Referral System. This global funding; 40% on the Health Based Allo- system helps match hospital patients to the cation Model, which considers past service earliest available bed in the most appropriate levels, demographics and population health setting, including both regular (shorter-term) information; and 30% on the Quality-based and restorative (longer-term) rehabilitation Procedures model based on best practices. beds. At the time of our audit, two LHINs were • Rehabilitative Care Alliance. (This replaced piloting the system, and the remaining LHINs the Rehabilitation and Complex Continuing were expected to begin implementing it by Care Expert Panel, which was a sub-committee March 2014. of the Ministry’s Emergency Room/Alterna- • Wait Time Strategy. As part of this strategy, tive Level of Care Expert Panel.) Established in the 2011/12 fiscal year, the Ministry set in October 2012, the Alliance is to take a targets for acute-care hospitals to discharge system-wide view of rehabilitation in Ontario. patients who have undergone hip or knee It reports to the LHINs and works with the surgery within an average of 4.4 days, with at Ministry, CCACs and experts on various pro- least 90% of people returning home—that is, jects. Issues the Alliance is focusing on include with a maximum of 10% referred to inpatient system accessibility and quality. In this regard, rehabilitation. In the absence of best practices, it is also assisting in defining best practices in the Ministry based the 4.4 days on perform- rehabilitation that are expected to help stan- ance data from Ontario’s optimally per- dardize the definitions of regular and restora- forming hospitals. It based the 90% “returning tive rehabilitation to better track services and home” indicator on a 2005 study by the costs. Ontario Health Technology Advisory Commit- • Funding for Community Rehabilitation. The tee. This study concluded that there was no Ministry indicated that OHIP payments to advantage for total joint replacement patients private physiotherapy clinics were one of

Chapter 3 • VFM Section 3.08 to receive inpatient physiotherapy rather than the fastest-growing expenditures in the community- or home-based physiotherapy. A health-care system, more than doubling from related study by the Ministry also noted that $87 million in 2007/08 to $185 million in having patients receive rehabilitation services 2012/13. Starting in August 2013, the Min- outside a hospital setting is generally more istry changed the way it funds some eligible cost-effective than having them as inpatients. community-based (also known as outpatient) The Ministry indicated that its Orthopaedic services. This includes ceasing OHIP billing Expert Panel is now developing new targets, privileges for 90 privately owned physio- which the Ministry plans to link to funding therapy clinics and instead contracting with in the future. The Ministry expected that privately owned clinics and other providers this will help move patients out of acute care (such as hospitals and family health teams) to more quickly and ensure that acute-care and provide community-based physiotherapy. As rehabilitation beds are available for patients well, through the LHINs, the Ministry started who need them the most. funding long-term-care homes to directly • Health System Funding Reform. Commencing acquire physiotherapy services for their resi- in the 2012/13 fiscal year, the Ministry plans dents, and made the CCACs responsible for to move away from historical global funding co-ordinating all in-home rehabilitation. The for hospitals, and toward, over the next few Ministry noted that the new arrangements Rehabilitation Services at Hospitals 227

were aimed at serving more people in more Current Co-ordination of Services and areas of the province more cost-effectively. Capacity

At the time of our audit, we noted that the LHIN SYSTEM CO-ORDINATION AND CAPACITY associated with one hospital we visited was co- ordinating access to restorative rehabilitation Stakeholders Call for Co-ordinated System across the LHIN and that it planned to do the same Many times over the years, stakeholders have called with regular rehabilitation in the future. The LHIN for better provincial co-ordination of rehabilitation associated with another hospital we visited was programs in order to, among other things, improve involved in developing new rehabilitation programs patient flow from acute-care hospitals to rehabilita- and changing existing ones within its boundaries. tion and ensure that patients receive the rehabilita- The third LHIN was looking primarily at patient tion they need when required. For example: flow from acute-care hospital beds to rehabilita- • A 2000 report by the Provincial Rehabilitation tion beds. Some LHINs have formed rehabilitation Reference group, including representatives networks consisting of hospitals and community- from rehabilitation hospitals and the Ministry, based organizations involved in the planning and identified the need for a policy framework provision of rehabilitation services. These networks aimed at creating a more accessible, equitable look at system-wide issues and cost-effective and and integrated rehabilitation system. efficient strategies for the integration of rehabilita- • In 2006, an Ontario Hospital Association tion services to improve patient access to care. The report, Optimizing the Role of Complex GTA (Greater Toronto Area) Rehab Network, for Continuing Care and Rehabilitation in the example, has focused on promoting best practices Transformation of the Health Care Delivery and knowledge exchange and on developing meas- System, recommended that the Ministry ures for service planning and performance improve- and the LHINs work with post-acute-care ment. Each of the three hospitals we visited belongs hospitals, such as those offering rehabilita- to a local rehabilitation network. tion and mental health services, to develop a However, with the exception of a few prov- systemic approach to managing and planning incially co-ordinated specialty rehabilitation Chapter 3 • VFM Section 3.08 rehabilitation services at the local, regional programs—such as those for spinal cord injuries and provincial levels. and acquired brain injuries—each hospital gener- • A June 2010 round-table discussion between ally determines (some with LHIN input) which the Ministry, the Ontario Hospital Association, inpatient and/or outpatient rehabilitation services and the LHINs recommended a “single prov- it will offer, if any at all. As a result, since services ince-wide vision and conceptual framework to vary, each hospital generally establishes its own guide the future development of new service policies and procedures for admitting rehabilitation delivery models.” The conceptual framework patients, determining patient eligibility, prioritizing was to include determining access to rehabili- patients for services, managing patient wait lists tation at a regional level, conducting earlier and providing patient care. assessments and treatment of rehabilitation This approach to service delivery has resulted in patients, increasing access to and intensity of differences in the types and levels of inpatient and rehabilitation services for complex patients in outpatient services provided by hospitals across hospital, and requiring the use of best-practice the province. As a result, a patient might be eligible guidelines for rehabilitation. for services at one hospital but not eligible for the same services at another hospital. We also noted a 228 2013 Annual Report of the Office of the Auditor General of Ontario

wide variation in the supply of regular rehabilitation through the Canadian Institute for Health Informa- inpatient beds across the province, ranging from 57 tion’s National Rehabilitation Reporting System. beds per 100,000 people in the Toronto Central LHIN This included the number of beds and number to only six per 100,000 people in the Central West of admissions. However, the Ministry does not LHIN, as shown in Figure 2. The provincial average have access to similar information on restorative was 18 beds per 100,000 people. The Ministry indi- rehabilitation, such as the number of restorative cated that the location of rehabilitation beds across rehabilitation beds and associated admissions. Each the province was set before the LHIN boundaries hospital’s accountability agreement with its LHIN were developed, and therefore some patients may contains performance targets. The main rehabilita- receive rehabilitation outside their LHIN. tion targets relate to the number of regular rehabilitation inpatients each hospital is expected to serve and the total number of days restorative patients Information Available on Inpatient Services stay in hospital. Without complete information, it is Since 2002, the Ministry has required all hospitals difficult for the Ministry or the LHINs to determine to submit data on their regular rehabilitation beds system capacity and utilization.

Figure 2: Number of Regular* Rehabilitation Beds per 100,000 People as of September 2010, by Local Health Integration Network Source of data: Toronto Central LHIN Commissioned Report 60

40 Chapter 3 • VFM Section 3.08

0 n a Central Ontario Champlai North East North West South East Central EastCentral West Erie St. Clair South West Toronto Central Mississauga Halton Waterloo Wellington North Simcoe Muskok

Hamilton Niagara Haldimand Brant

* Figure excludes restorative rehabilitation beds because province-wide information was not available. Rehabilitation Services at Hospitals 229

Information Available on Outpatient Rehab Network had on its website a user-friendly Services “Rehabilitation Finder” that helps people find rehabilitation programs provided by hospitals and With respect to outpatient services, according to CCACs in their area, including program descrip- various stakeholder reports—including a 2011 tions, eligibility information and how to apply. We report by the GTA Rehab Network and a 2011 also noted that two other LHINs in the province had joint report by the Orthopaedic Expert Panel, on their websites some information about publicly the Ontario Physiotherapy Association and other funded rehabilitation services available in their area. partners—there has been a reduction in publicly funded outpatient services. This includes the closure of many hospital-based outpatient clinics Impact of Aging Population starting more than 10 years ago. In fact, according As the population ages, the need for rehabilitation to the 2011 joint report, 50% of Ontario hospital services is expected to increase, which will also sites responding to a survey said they had reduced increase the importance of a well co-ordinated outpatient rehabilitation services over the past system. Rehabilitation programs can help seniors two years; 16% indicated that even more reduc- in a number of ways: they help seniors return home tions were planned for the following year. This after a hospital stay instead of requiring a long- report also noted that the availability of outpatient term-care home, decrease their visits to emergency programs was inconsistent across the LHINs and departments and their hospital readmission rates, that there is little information on the demand for and maintain their mobility in long-term-care services, service capacity and service accessibility. homes. According to a 2010 report from the Can- The 2011 report by the GTA Rehab Network, while adian Orthopedic Care Strategy Group, musculo- confirming the lack of information on outpatient skeletal disease, such as knee and hip fractures, rehabilitation services, did note that demand for affected 11 million Canadians over the age of 12 publicly funded outpatient rehabilitation services in 2007 and is predicted to increase with the aging appears to exceed supply. baby boomer population to 15 million in 2031. We noted that, although the Ministry has

This anticipated increase in cases is expected to put Chapter 3 • VFM Section 3.08 information on outpatient rehabilitation visits to pressure on the demand for rehabilitation because hospital physicians and nurses, it does not have orthopedic conditions are the most common reason information on the number of rehabilitation visits for rehabilitation. Similar trends can be expected to hospital physiotherapists or occupational ther- for patients suffering from stroke, the second-most- apists—the clinicians whom outpatients primarily common reason for inpatient rehabilitation, given deal with. Nor does it have information on the the aging population and that most strokes occur in unique number of patients (individuals generally people over 65. make multiple visits). The LHINs overseeing the hospitals we audited also did not have this informa- RECOMMENDATION 1 tion. Further, none of the hospitals we audited had determined their outpatient service capacity—that To better ensure that Ontarians requiring is, the maximum number of patients they could rehabilitation have equitable access to services, serve given their currently available outpatient the Ministry of Health and Long-term Care resources, such as the number of therapists and (Ministry) should work with the Local Health rooms or equipment available for therapy. Integration Networks to: The Ministry also did not have information on • establish a province-wide co-ordinated sys- the types of hospital-based and other outpatient tem for rehabilitation, including both regular rehabilitation services available. However, the GTA (shorter-term) and restorative (longer-term) 230 2013 Annual Report of the Office of the Auditor General of Ontario

inpatient services and all community-based assess‑and-restore approach to provide outpatient services; and clarity for patients, families and referring • provide the public with detailed information professionals regarding the focus and clinical on programs available, eligibility and how to components of rehabilitative care programs. apply, such as through a public website. The Ministry’s physiotherapy reforms In order to have good information for cur- include the expansion of provincial capacity to rent and future decision-making, the Ministry deliver physiotherapy in publicly funded com- should establish, in conjunction with its share- munity physiotherapy clinics. Under the trans- holders, what information should be collected fer-payment agreements, physiotherapy clinics on restorative inpatient and outpatient services are required to report on patient volumes and and how best to collect the data. outcome measures such as average pain/mobility scores when patients begin treatment against MINISTRY RESPONSE average pain/mobility scores when patients complete their course of care. Community Care The Ministry supports this recommendation and Access Centres (CCACs) are also receiving fund- will continue to explore options regarding LHIN- ing to provide increased one-on-one in-home led provincial co-ordination of the rehabilitation physiotherapy services. These changes will system, including rehabilitation best practices result in services being available in more places and associated data-reporting requirements. across the province. They also recognize that Leading this work will be the Rehabilitation “rehabilitation” is a care continuum that extends Care Alliance (Alliance). With the Ministry’s beyond the hospital into the community. participation and support, the Alliance is The Ministry appreciates the Auditor investigating and developing recommendations General’s recommendation regarding the avail- that will help guide provincial standards for ability of public information on rehabilitation rehabilitative care programs and services across programs and services and will review possible the care continuum. The additional deliverables enhancements to web-based communication

Chapter 3 • VFM Section 3.08 of this expert body will include: materials. At present, if an individual needs descriptions of level of care across the • in-home physiotherapy or would like a list of rehabilitative care continuum; where clinic-based services are available, he eligibility (including restorative and/or • or she can contact the local CCAC by visiting rehabilitative potential) and discharge thehealthline.ca or www.310CCAC.ca or criteria for each level of care across the by calling 310-CCAC (2222) (no area code rehabilitative care continuum; required). Additionally, information on the tools for determining eligibility; • August 2013 changes to publicly funded physio- standardized patient outcomes and/or • therapy services can be found on the Ministry’s performance measures criteria for each website, including a list of frequently asked level of care across the rehabilitative care questions, clinic locations and other resources. continuum; Working through the LHINs and other tools to support optimal management of • provider groups, the Ministry will explore data transition points; collection requirements that are meaningful standardized definitions that describe • and useful in terms of informing the delivery of rehabilitative care resources across the rehabilitation services. care continuum, including a system-wide Rehabilitation Services at Hospitals 231

INPATIENT SERVICES per day. The other hospital required patients to be able to participate in therapy for 30 minutes to an Referral Process hour three times per day. In another example, for People are generally referred by a physician or a its amputee rehabilitation program, one hospital registered nurse for inpatient rehabilitation pro- required patients with single limb amputations grams. At one hospital we visited, referrals were to have a prosthesis that fits adequately, while also accepted from any member of the interdisci- another required the patient to be able to tolerate plinary team caring for the patient being referred. 60 minutes or more of therapy five days per week, Over 90% of patients are already hospitalized for an and a third hospital had various requirements, acute condition, such as a stroke or fractured hip, including the patient’s being able to sit for two when they are referred for inpatient rehabilitation. hours and having a discharge destination within The hospitals we visited varied in how they the hospital’s LHIN. received patient referrals. One hospital we visited The actual process for determining eligibility received most of its patient referrals via electronic also varied between hospitals we visited. At one systems, including a Resource Matching and Refer- hospital, patient-flow co-ordinators—physiother- ral system. Physicians and nurses unable to access apists or occupational therapists—made the admis- these systems referred patients by fax. However, sion determination. At another hospital, eligibility even though most of the information was elec- was generally determined by a physiatrist—a tronically received, this hospital still had to manu- medical doctor specializing in physical medicine ally re-enter all patient information into its own and rehabilitation. At the third hospital, a nurse information system—an inefficient process that determined eligibility in consultation with a physia- increases the risk of data entry errors. At the other trist. At one hospital, it took a median of four days two hospitals, most patients were referred inter- between April and December 2012 to determine nally for rehabilitation after, for example, surgery patient eligibility. The other two hospitals deter- or stroke care. One hospital received notification mined patient eligibility within a day. of internal referrals electronically, while the other Although a ministry report indicates that,

received these referrals by phone or fax. However, province-wide, 55% of patients considered ready Chapter 3 • VFM Section 3.08 in both cases, patient information was electronic- for regular inpatient rehabilitation were admitted ally accessible on the hospitals’ systems and there- within one day in the 2012/13 fiscal year, certain fore did not have to be re-entered. As a result, only rehabilitation programs do have wait lists. For patient information related to external referrals, example, at the two hospitals we visited that had which were generally received by fax, had to be an acquired-brain-injury program, the wait time at manually entered in these two hospitals’ systems. both was a median of 21 days. If a space is not immediately available in a particular rehabilitation program, individuals are Eligibility and Wait Times added to the hospital’s wait list. Neither the prov- Each hospital generally has its own eligibility ince nor the LHINs have established a standardized criteria for accepting or declining patients referred prioritization policy for hospitals to follow, so each to it for rehabilitation. The hospitals we visited var- hospital decides how to prioritize its own patients. ied in how they determined eligibility for similar One of the hospitals we visited generally did not programs. For example, one of the two hospitals have wait lists. Of the two with wait lists, one pri- offering an orthopedic rehabilitation program oritized individuals based on who had been waiting required that patients be able to participate in the longest. The other considered length of wait therapy five days per week, for at least one hour plus factors such as the patient’s medical condition. 232 2013 Annual Report of the Office of the Auditor General of Ontario

These two hospitals prioritized internally referred completed, on average, in nine days. This hospital patients over those waiting elsewhere if the hospital noted that it had reduced the time to five days by needed to free up acute-care hospital beds for other June 2013. The other two hospitals did not track patients. this information. One hospital we visited tracked the number of The CIHI also collects assessment information patients who were declined and the reason they on patients in restorative rehabilitation programs were declined. This hospital told us that it declined using the Continuing Care Reporting System. 39% of applicants referred for regular rehabilita- Patients are to be given a Resident Assessment tion and 22% of applicants referred for restorative Instrument—Minimum Data Set assessment, which rehabilitation during the first nine months of the measures a patient’s needs and strengths with 2012/13 fiscal year. The most common reason regard to cognition, communication, behaviour, for declining applicants was that they had not toileting and other criteria. established rehabilitation goals, such as being able As well, both regular and restorative rehabili- to walk up stairs or dress oneself. At this hospital, tation patients receive additional assessments, acute care therapists would generally determine conducted by each type of therapist, in order to any initial goals as part of the referral process. develop an individualized plan of care based on Another hospital generally accepted all patients their needs. It is important that these assessments referred, declining few applicants overall. The be completed promptly so that therapy can begin third did not track the overall number of patients as soon as possible after admission. We noted that declined service or the reasons they were declined. the time frames for assessment varied at the three hospitals we audited. At one hospital, therapists were allowed 48 hours from admission to complete Assessment and Extent of Therapy Provided their assessments; another allowed seven days, and Assessment of Therapy Needs the third allowed 14 days. Our review of a sample Once a patient has been admitted to a rehabilitation of files indicated that two of the hospitals gener- facility, he or she is assessed by an inter-profes- ally completed assessments within their required

Chapter 3 • VFM Section 3.08 sional team that generally includes a physiother- time frames. However, at the third hospital, 16% apist, an occupational therapist and a nurse. All of the assessments were not completed within the patients referred for regular rehabilitation are required seven days. assessed using a standardized tool called the Functional Independence Measure (FIM), which Extent of Patient Therapy measures the level of a patient’s disability. The FIM With the exception of stroke (discussed in the assessment also indicates how much assistance Stroke section later in this report), there are few is required to carry out various activities of daily best-practice standards in Ontario for the amount, living, such as eating, washing, dressing and toilet- type and frequency of inpatient therapy that ing. According to the Canadian Institute for Health patients should receive for specific conditions. At Information (CIHI), the FIM assessment is to be the hospitals we visited, the amount and type of completed within 72 hours of admission. (The CIHI therapy that each patient is to receive is based on maintains the National Rehabilitation Reporting the professional judgment of his or her therapists System containing patient data collected from and on the resources available. participating adult inpatient rehabilitation facilities A 2010 report resulting from a round-table and programs across Canada.) One hospital we vis- discussion between the Ministry, the LHINs and the ited tracked this information and indicated that in Ontario Hospital Association noted that providing the 2012/13 fiscal year, the FIM assessments were more therapy is less expensive than having patients Rehabilitation Services at Hospitals 233 spend more time in the hospital. In this regard, The number of patients seen by each therapist— a 2012 study by a Toronto rehabilitation hospital that is, patient caseload—varied at each of the compared the results of its programs providing hospitals we visited. Some therapists were seeing rehabilitation seven days per week with those patients with different needs from more than one providing rehabilitation five days per week. It noted program, and others worked in both inpatient and that patients in its seven-days-per-week program outpatient programs. Therefore, it was difficult to got similar results and were able to go home one compare among hospitals. However, at all three day earlier than those in the five-days-per-week hospitals, we noted that there was generally no program. However, the report concluded that it coverage for therapists who were sick or on vaca- was too early to evaluate the cost-effectiveness of tion, so at times there were fewer therapists avail- the seven-days-per-week program. We noted that able for the same number of patients. One hospital one of the hospitals we visited did not provide any indicated that it had piloted providing coverage for inpatient rehabilitation services on weekends, one therapists who were away during peak vacation of the other two offered some therapy on Saturdays periods and was evaluating the impact. for one unit, and the third offered some therapy on weekends for two of its many programs. One of Impact of Patient Therapy these hospitals indicated that weekend therapy was Before discharging a patient, hospitals complete not offered on most units because weekends were another FIM assessment of him or her, which is a time for patients to rest, recover and practice new compared to the results of the initial FIM assess- skills. A common complaint noted in patient satis- ment to determine the extent of the patient’s faction surveys at one of the hospitals was the lack improvement. Patients in regular rehabilitation of therapy available on weekends. beds at all three hospitals had improved FIM assess- It was difficult to determine how much therapy ment scores when discharged. The FIM improve- was actually provided to each patient at the three ment is the result of the rehabilitation received hospitals we visited. Although all three of the hospi- combined with the natural healing process and the tals, as well as the therapists’ professional colleges, passage of time. Further, the percentage of regular

require some documentation of therapy, none Chapter 3 • VFM Section 3.08 rehabilitation inpatients returning home ranged required documentation of all sessions each patient from 85% to 87% at the hospitals we visited. FIM attended. None of the hospitals was documenting assessments are not required for patients in restora- all rehabilitation provided to each patient. Two hos- tive beds, so the extent to which they improve after pitals did track the specific days on which therapy rehabilitation is generally not known. However, was provided to each patient, but not the actual one of the hospitals we visited was conducting amount of therapy provided per day. Although the these assessments on its restorative rehabilitation hospitals required the therapists to document elec- patients, and noted a significant improvement in tronically how they spend their time each day on patient functionality. various tasks, such as time spent with patients, this information was collected at the therapist level only and was not being used to determine how much Co-payment for Restorative Rehabilitation therapy each patient received. In the United States, Regular rehabilitation generally takes place in for Medicare-eligible rehabilitation inpatients, beds that have been designated by the Ministry as therapists are required to record face-to-face inter- rehabilitation beds, and restorative rehabilitation actions with patients in 15-minute increments, and takes place in beds designated as complex continu- managers must ensure that patients receive three ing care (CCC) beds. Historically, CCC beds were hours of therapy each day. occupied on a permanent basis by, for example, 234 2013 Annual Report of the Office of the Auditor General of Ontario

patients who could not be managed at long-term- Ontario Hospital Association and the provincial care homes. However, these beds are now generally Rehabilitative Care Alliance have both recognized used for other purposes, including restorative that rehabilitation beds can be a valuable resource rehabilitation and palliative care. With the current for the health-care sector, by helping to keep ALC wide range in the services provided for patients patients out of acute-care hospitals, relieving pres- in CCC beds, the Ministry has limited information sure on emergency departments and allowing for on the actual use of these beds. As well, two of the an efficient flow of patients through the system. three LHINs associated with the hospitals we visited However, ALC patients may be difficult to place did not have this information. if they have a complex medical condition. The Under the Health Insurance Act, hospitals may rehabilitation hospitals we visited said that costs for charge a co-payment fee to their long-term CCC their ALC patients were usually only marginally less patients who have effectively become permanent than for other patients because ALC patients still residents of the hospital or who are awaiting required some therapy to ensure that their condi- discharge to a long-term-care facility, but not to tion does not decline. those returning to the community. The co-payment The Ministry’s Rehabilitation and Complex Con- charge is intended to eliminate any financial incen- tinuing Care Expert Panel (Expert Panel), which tive for patients to stay in hospital, where a patient comprised rehabilitation experts and stakeholders would normally pay nothing, rather than move to a from across Ontario, issued a report in June 2011 long-term-care home, where payment is normally providing advice and guidance to the Ministry’s required. The hospital co-payment charge is usually Emergency Room/Alternative Level of Care the same as the basic rate charged in long-term- Expert Panel. This report focused on how best to care homes, and, similar to this charge, can be reduce ALC lengths of stay throughout the system reduced for people with low incomes. One of the by properly utilizing the regular and restorative two hospitals we visited that had CCC beds charged rehabilitation resources for stroke, hip- and knee- a co-payment fee only to the approximately 20% of replacement, and hip-fracture patients. The Expert its CCC patients who were not expected to return Panel made 30 recommendations, grouped on the

Chapter 3 • VFM Section 3.08 home. However, the other charged the co-payment basis of urgency. The more time-sensitive recom- to all of its CCC patients, including the restorative mendations included introducing best practices, rehabilitation patients, regardless of whether they aligning financial incentives with best practices, were expected to return home. and enhancing the role for hospital-based outpatient rehabilitation. In mid-2013, the Rehabili- tative Care Alliance, which replaced the Expert Alternate-level-of-care Patients Panel, began refining the 30 recommendations for Alternate-level-of care (ALC) patients are patients implementation. who are ready to be discharged but need to wait in A report by the Ontario Hospital Association hospital for post-discharge care, such as home-care indicated that about 2,300 ALC patients occupied services or placement in a long-term-care home. acute-care beds in the province as of March 2013. Some ALC patients are waiting in an acute-care Of these, 16% were waiting for a regular rehabili- hospital bed for placement in a rehabilitation bed. tation bed and 9% for a CCC bed (CCC beds The potential risks of staying in an acute-care include restorative rehabilitation beds). Province- hospital longer than medically necessary include wide in the 2012/13 fiscal year, 7% of patients in hospital-acquired infections, such as C. difficile, an acute-care bed waited there over a week for a and a decline in physical and mental well-being regular rehabilitation bed, as shown in Figure 3. due to the lack of physical activity. Further, the This percentage varied across the LHINs, from a Rehabilitation Services at Hospitals 235 low of 1% in the Central and Central East LHINs to Central LHIN. About 5% of regular rehabilitation a high of 35% in the North Simcoe Muskoka LHIN. beds and 14% of CCC beds at the audited hospitals Despite the higher percentage of people waiting, were occupied by ALC patients as of March 31, the North Simcoe Muskoka LHIN had a similar 2013. Most of these patients were waiting for a number of beds per 100,000 people as the Central long-term-care home, supervised/assisted living or East LHIN, as shown in Figure 2. Further, the home-care services. Champlain LHIN was experiencing longer-than- Turnaround time—the time to clean a room average waits despite having 20 beds per 100,000 and admit a new patient—for rehabilitation beds people, the second-most of all LHINs. is important because a patient in the emergency Other people are waiting in rehabilitation beds department awaiting an acute-care bed could have for post-discharge care. The Ontario Hospital Asso- a lengthy wait while a patient in the acute-care bed ciation report indicated that, while waiting for care is waiting to be moved to a rehabilitation bed. None elsewhere, ALC patients occupied 13% of beds in of the hospitals we visited tracked the time it took post-acute-care facilities, such as regular rehabili- to fill a vacated rehabilitation bed. However, they tation, CCC and mental-health institutions. This all indicated that it normally took less than a day percentage varied significantly across the province, because discharge dates are estimated beforehand, from fewer than 1% of post-acute-care beds in the allowing for the admission of a new patient to be Mississauga Halton LHIN to 20% in the Toronto planned for the same or next day.

Figure 3: Patients Waiting Over One Week in an Acute-care Hospital Bed for Rehabilitation in 2012/13, by Local Health Integration Network (%) Source of data: Ministry of Health and Long-Term Care

30 Chapter 3 • VFM Section 3.08

0 n a Central Ontario Champlai North East North West South East Central EastCentral West Erie St. Clair South West Toronto Central Mississauga Halton Waterloo Wellington North Simcoe Muskok

Hamilton Niagara Haldimand Brant 236 2013 Annual Report of the Office of the Auditor General of Ontario

RECOMMENDATION 2 this information, one hospital indicated that it would not be beneficial to track the amount To better ensure that inpatient rehabilitation of therapy actually received by each patient, meets patients’ needs as efficiently and equit- because outcome measures—such as the ably as possible, hospitals should: patient’s discharge destination and the change implement systems for accepting patient • in the each inpatient’s Functional Independence referrals and uploading associated patient Measure (FIM) score—are more meaningful. data electronically; Another hospital suggested that information in conjunction with the Ministry of Health • on the amount of therapy provided to patients and Long-term Care (Ministry) and the be tracked in conjunction with the Ministry of Local Health Integration Networks (LHINs), Health and Long-Term Care, to help ensure that develop standardized practices regarding the information is consistently collected across patient eligibility for similar programs, pri- the province. oritization of patients based on patient need, One of the two hospitals with restorative and the frequency and duration of therapy; rehabilitation patients was already following track and monitor information on the • the practice of charging a co-payment only to amount of therapy actually provided to patients who were not expected to return home. patients, the number of patients declined The other hospital thought that the Ministry and the associated reasons, and the time should clarify the intent of the legislation, to it takes to fill a bed after a patient is dis- prevent having it interpreted differently by hos- charged; and pitals across the province. • consistent with the Health Insurance Act, charge a co-payment only to restorative MINISTRY RESPONSE rehabilitation patients who are not expected to return home. Although this recommendation was directed toward the hospitals, the Ministry will also

Chapter 3 • VFM Section 3.08 RESPONSE FROM HOSPITALS review the tracking and monitoring recommendation and explore opportunities to refine The hospitals we visited all agreed with having standardized practices. This work will be systems in place to accept patient referrals and undertaken in consideration of the work being upload patient data electronically. One of the conducted by the LHIN-led Rehabilitative Care hospitals commented on the need for funding to Alliance (Alliance). The Alliance is uniquely implement such a system. positioned to propose tools that can be applied Two of the hospitals generally agreed with across the province to assist health-care provid- implementing standardized practices in the ers in consistently determining patient eligibil- recommended areas, and one indicated that ity, and to create tools that support the optimal this should also be done in conjunction with the management of transition points. Rehabilitative Care Alliance as well as clinician- The Ministry will also provide a clarification led condition-specific networks. The third hospi- on co-payment requirements, which will be tal suggested developing best practices in these issued through the LHINs to appropriate health- areas instead, because standardized practices service providers. may reduce the hospital’s flexibility. Although the hospitals generally agreed on the importance of tracking and monitoring Rehabilitation Services at Hospitals 237

OUTPATIENT SERVICES the median time to determine outpatient eligibility from the date of referral ranged from the same day Outpatient rehabilitation services are commonly to five days to 19 days. used by patients with milder functional impair- ments, including after discharge from an acute- care or rehabilitation hospital. They are usually Waiting for Outpatient Services provided at hospital-based or other clinics or at the After being deemed eligible, an applicant might patient’s home, including retirement homes and not receive rehabilitation services right away if the long-term-care homes, with a goal of improving location has a wait list. Two of the hospitals we patient functionality and, therefore, quality of life. visited had wait lists; the third did not have any However, other than for stroke programs, there are patients waiting. Only one of the two hospitals with few best practice standards in Ontario for when wait lists tracked wait times. At this hospital, the therapy should start, how much therapy should be overall wait time from referral to rehabilitation was provided, what type of therapy should be provided, a median of 33 days. For one outpatient clinic loca- the length of therapy sessions and the number of tion at the other hospital, our file review noted a weeks therapy should be provided. median wait time of five days. This hospital told us that, at its other outpatient clinic location, patients Determining Eligibility for Outpatient had a wait time of about two years or more for Services some programs, such as for ongoing back and neck problems. Most of the outpatient programs at the three hos- There is no provincial or LHIN-wide policy for pitals we visited required patients to have a referral prioritizing patients on wait lists: each hospital from a hospital physician. Two of the hospitals also follows its own procedures. The policy at two of the accepted referrals from community physicians, such hospitals we visited was to prioritize on the basis of as family physicians, for some of their programs. who had been waiting the longest. The third hospi- When a referral is received at a hospital outpatient tal told us that its policy was to also consider factors program, the application is reviewed by hospital

such as the patient’s medical issues and risk of Chapter 3 • VFM Section 3.08 staff—such as a triage nurse, a therapist or a group falling, although the rationale to support decisions of therapists—to determine eligibility according was not required to be documented. One of the hos- to the hospital’s criteria. There are no standard- pitals prioritized internally referred patients over ized provincial or LHIN-wide eligibility criteria for referrals from the community. Similarly, the 2011 admission to outpatient programs in Ontario. At GTA Rehabilitation Network report noted that 70% the three hospitals we visited, we noted that the of orthopedic and stroke programs—the programs eligibility criteria varied for similar programs. For with the most patients—prioritized internal refer- example, one hospital’s outpatient stroke program rals over external ones, meaning that externally required external applicants to have a FIM score referred patients might wait longer. indicating only a mild functional impairment, which is consistent with the Expert Panel’s sug- gestion. At another hospital, however, there was Attending Outpatient Services no requirement for a specific FIM score. We also Once reaching the top of the wait list, patients can found that there was no standardized tool used by face challenges in attending outpatient services. the hospitals we visited to document the hospital’s These challenges include a lack of transportation to decision on whether to accept or reject the patient. and from the outpatient facility, and few or no even- At the three hospitals we visited, we found that ing or weekend services for clients not able to attend 238 2013 Annual Report of the Office of the Auditor General of Ontario

programs on weekdays. Two of the three hospitals Health Integration Networks (LHINs), we visited did not offer outpatient rehabilitation develop standardized practices for com- services during evenings or on weekends. The third mon patient conditions, such as total joint hospital offered some services at one of its two clin- replacements, regarding when to begin ics until 7 p.m. from Monday to Thursday. outpatient therapy, as well as the type and All three hospitals had information on the duration of therapy. number of outpatients served and the total number Further, hospitals should collect informa- of times patients saw rehabilitation staff. How- tion to better ensure that available outpatient ever, only one tracked information, at one of its resources are utilized efficiently and effectively, two clinics, on whether each therapist was fully such as information on the number of appoint- booked, how many appointments were cancelled ment cancellations and patient no-shows, and by patients, and the extent of patient no-shows. on the change in patient functionality between This information was not summarized on an overall when outpatients start and when they complete basis, but we noted that, from February 2012 to outpatient rehabilitation. January 2013, cancellations per therapist ranged from 1% to 13% of appointments, and no-shows RESPONSE FROM HOSPITALS ranged from none to 6%. Although two of the hospitals agreed with prioritizing patients for outpatient services based on Determining Impact of Outpatient Services need, the third hospital indicated that this rec- Whereas regular rehabilitation inpatients are ommendation would be difficult to implement assessed by FIM scoring at the beginning and end because patient need is not currently defined. of treatment to determine their functional improve- All three of the hospitals agreed with assessment, rehabilitation outpatients are not similarly ing the need for, and the costs and benefits of, assessed using a standardized measure. Therefore, providing evening and weekend outpatient there is little information on whether outpatient services.

Chapter 3 • VFM Section 3.08 programs are effective. The Ministry indicated The three hospitals generally agreed with that the Rehabilitative Care Alliance is developing developing standardized outpatient practices a standardized data set for Ministry-funded out- for common patient conditions. One hospital patient programs. indicated that this should also be done in conjunction with clinician-led condition-specific RECOMMENDATION 3 networks. Another hospital expected the Rehabilitative Care Alliance to conduct work in To better ensure that patients have timely access this area. to required outpatient services, hospitals should: Although the hospitals agreed with the prioritize eligible patients based on need, • importance of collecting most of this outpatient rather than on other factors such as whether information, two of the hospitals expressed they were referred from the hospital’s concerns regarding monitoring the change in inpatient program or externally; outpatient functionality. Both of these hospitals assess the need for, and the costs and • used various measures for monitoring this benefits of, providing evening and weekend change, but one of these hospitals cautioned services; and that it may be difficult to find one measure to in conjunction with the Ministry of Health • capture this change. The other hospital thought and Long-Term Care (Ministry) and Local that no such indicator currently existed and Rehabilitation Services at Hospitals 239

stroke survivors are usually left with some degree that it was more important to monitor whether of disability. outpatients achieved their goals. The Ontario Stroke Network (OSN), created in 2008, receives funding from the Ministry to MINISTRY RESPONSE provide leadership and co-ordination for Ontario’s Although this recommendation was directed 11 Regional Stroke Networks, including stroke pre- toward the hospitals, the Ministry is also com- vention clinics and Ontario Stroke centres, which mitted to improving quality. One example of its are hospitals specializing in stroke treatment. All efforts in this regard is the provincial assess- have a goal of decreasing the incidence of stroke; and-restore policy for frail older adults that ensuring that Ontarians have access to appropri- is currently under development. In addition, ate, quality care in a timely way; and improving the Rehabilitative Care Alliance (Alliance) is care and outcomes. actively engaged in establishing a rehabilitative In 2011, the OSN established the Stroke care approach for frail senior/medically com- Reference Group, which recommended a series plex populations to support “operationalization” of stroke-rehabilitation and patient-flow best of priority elements of the “Essential Elements practices, including those shown in Figure 4. The of Assess and Restore Framework.” As part recommendations were accepted in November 2011 of the work plan, the Alliance is developing a by the Rehabilitation and Complex Continuing Care standard process for identifying and supporting Expert Panel. In January 2013, the Stroke Clinical timely navigation and entry of high-risk older Advisory Expert Panel at Health Quality Ontario—a adults with restorative potential to the most provincial government agency that, among other appropriate level of rehabilitative care. things, evaluates the effectiveness of new health- Further, the Ministry will work with the care technologies and services—made similar rec- LHINs, using an evidence-based approach, to ommendations with respect to the timely transfer assess the demand for and benefits of providing of patients and greater intensity of therapy. evening and weekend services. The OSN’s 2012 report, The Impact of Moving

to Stroke Best Practices, estimated that savings in Chapter 3 • VFM Section 3.08 the acute-care and inpatient rehabilitation sectors COMMON CONDITIONS REQUIRING arising from full implementation of these best REHABILITATION practices could reach $20 million per year. This report also indicated that incorporation of these As part of our audit, we focused particularly on two best practices would have a positive impact on specific conditions requiring rehabilitation—stroke patient outcomes. Hospitals can decide whether and total joint replacement, including hip and knee to follow all, some or none of these best practices. replacements—because they account for the largest We noted that both of the hospitals we visited that number of admissions to inpatient rehabilitation had stroke programs were implementing some of services, at 15% and 18%, respectively. these best practices. We noted the following with respect to the Stroke Stroke Reference Group’s recommendations.

Stroke is the leading cause of adult disability in Canada. A stroke can affect various basic functions Timely Transfer such as speech, sight, memory and the ability to According to the Ontario Stroke Evaluation Report walk. According to the Ministry, over 90,000 Ontar- 2013 prepared by the Ontario Stroke Network, ians currently live with the effects of stroke, and the Canadian Stroke Network and the Institute 240 2013 Annual Report of the Office of the Auditor General of Ontario

Figure 4: Selected Best Practices for Stroke Rehabilitation and Patient Flow Source of data: Ontario Stroke Network

Timely transfer Timely transfer of appropriate patients from acute-care facilities to rehabilitation: Ischemic stroke patients should be transferred to rehabilitation within five days of their stroke on average and hemorrhagic stroke patients within seven days on average. (Ischemic strokes, accounting for 80% of all cases, are caused by an interruption of blood flow to the brain. Hemorrhagic strokes, accounting for the remaining 20%, occur when blood vessels in the brain rupture.) Greater-intensity Provision of greater-intensity therapy in inpatient rehabilitation: Stroke patients should receive three therapy hours of therapy a day—one hour each of physiotherapy, occupational therapy and speech language pathology—seven days per week. Timely outpatient Timely access to outpatient (either hospital- or community-based) rehabilitation for appropriate (hospital- or patients: This includes two to three outpatient visits or visits by CCAC health professionals per week community-based) per required discipline for eight to 12 weeks. rehabilitation Equitable access Equitable access to all necessary rehabilitation for all rehabilitation candidates.

for Clinical Evaluative Sciences, in the 2011/12 the functional recovery of patients is not greater fiscal year, province-wide, it took a median of 10 with more rehabilitation per day than with the days from the time of a patient’s stroke for him or standard amount of rehabilitation. However, the her to be transferred to rehabilitation. One of the review recognized that there was some discrepancy two hospitals we visited that had a stroke program between these results and the opinions of some reported a median of 16 days in 2011/12, while experts in the field of stroke rehabilitation. For the other’s median time was 13 days. Both hospi- this reason, Health Quality Ontario planned to tals told us that timing of transfers was affected undertake a full analysis of this topic. The OSN has by the acute-care hospital’s reluctance to transfer noted that increasing therapy intensity may shorten patients earlier because they were considered to the patient’s length of stay in hospital, and thereby be medically unstable. decrease costs.

Chapter 3 • VFM Section 3.08 In March 2013, Health Quality Ontario released Similar to other types of rehabilitation, at the its review of the available research on the optimal hospitals we visited, the amount and type of stroke time to access rehabilitation after a stroke. The therapy that each patient receives is based on the report concluded that, until better evidence is avail- professional judgment of his or her therapists. able, rehabilitation ought to be initiated as soon as Neither of the hospitals we visited that had a stroke the patient is ready for it. However, the report noted program tracked how much therapy each patient that 19% of stroke patients remained in an acute- received. However, one hospital had begun to track care hospital longer than necessary while waiting the total hours of therapy provided to all stroke for access to an inpatient rehabilitation bed. patients—though not the hours per patient. It told us that it was not yet meeting its goal to provide three hours of therapy per patient per day. The other Greater-intensity Therapy hospital had no such goal. A 2010 report by the GTA Although there is expert consensus recommending Rehab Network included the results of a province- that stroke inpatients receive three hours of wide survey of stroke programs. We noted that rehabilitation per day, the research currently only three of the 12 regular rehabilitation stroke available on the intensity of stroke rehabilitation programs and three of the five restorative rehabilita- is mixed. In fact, Health Quality Ontario’s March tion stroke programs that responded to the survey 2013 review of related research concluded that provided the recommended amount of therapy. Rehabilitation Services at Hospitals 241

One Ontario stroke expert noted in 2008 that having difficulty living independently. The CCACs leaving the amount of therapy each patient is to provided, on average, only about four sessions of receive and the delivery of that therapy to the rehabilitation for each patient over an eight-week therapist’s discretion appears to result in less period, as compared to the two to three visits per direct patient-therapy time and tends to produce week per type of therapy over an eight- to 12-week less-than-optimal outcomes. As mentioned earlier, period recommended by the Expert Panel. in the United States, for Medicare-eligible rehabili- Neither of the two hospitals we visited that had tation inpatients, therapists are required to record stroke programs monitored whether it was provid- face-to-face interactions with stroke patients in ing two to three visits per week by each type of 15-minute increments, and managers ensure that therapist—such as physiotherapist, occupational patients receive three hours of therapy per day. therapist and speech language pathologist—for On the basis of 2011/12 data in the Ontario Stroke eight to 12 weeks. Evaluation Report and 2012 US eRehab data, we We noted the existence of a successful program noted that even though the Medicare-eligible in Calgary called the Early Supported Discharge rehabilitation inpatients’ increase in functionality Program, which was implemented as part of the was similar to that of Ontario stroke inpatients, Calgary Stroke Program in 2011. The goal of the their length of stay in hospital was only about half program is to discharge patients with mild or mod- that of the Ontario patients. (The U.S. patients erate strokes directly to the patient’s home, with generally had a lower functionality when they the same rehabilitation therapy at home—starting started inpatient rehabilitation compared to the within one or two days of discharge—as they would average for Ontario stroke patients, which might have otherwise received in hospital. The program influence their rate of increased functionality over estimated savings of about $1.8 million annu- that time period.) ally for about 160 patients. In Ontario, one LHIN proposed in May 2013 to pilot a new Community Stroke Rehabilitation Model that will provide early Timeliness of Outpatient (Hospital- or supported discharge from hospital. It will focus on Community-based) Rehabilitation

transitioning patients to their homes, which could Chapter 3 • VFM Section 3.08 We found that there is a general lack of informa- reduce the length of acute-care hospital stays after tion available about access province-wide to stroke a stroke. outpatient and/or community-based rehabilitation. According to the Canadian Best Practice Recom- mendations for Stroke Care, the suggested best Equitable Access practice for outpatient rehabilitation for stroke is According to the OSN report The Impact of Moving to start any needed rehabilitation within 48 hours to Stroke Best Practices in Ontario, data suggests of discharge from an acute care hospital or within that many patients are unable to access the 72 hours of discharge from inpatient rehabilitation. rehabilitation services they need. The best avail- One of the two hospitals we visited that had a stroke able estimates suggest that approximately 40% program reported that it took an average of 31 days of stroke patients are candidates for inpatient from referral until the patient started his or her rehabilitation when discharged from acute care, outpatient rehabilitation. The other hospital did not yet less than 25% were discharged to inpatient have a wait list for its outpatient stroke program. rehabilitation in the 2010/11 fiscal year. The Ontario Stroke Evaluation Report 2013 found Further, although the Stroke Reference Group that the extent of services provided through the estimated that all patients discharged from an CCACs was low and likely inadequate to help those inpatient rehabilitation program would require 242 2013 Annual Report of the Office of the Auditor General of Ontario

outpatient rehabilitation, the Ontario Stroke the extent of therapy, accepted by the Ministry’s Evaluation Report 2013 states that approximately Rehabilitation and Complex Continuing Care 33% of these patients were sent home without Expert Panel. outpatient services in 2011/12. The OSN reports also noted that “perhaps the MINISTRY RESPONSE most troubling finding in this report was the extent to which patients with very high levels of function The Ministry agrees with the recommendation are admitted to, or remain in, inpatient rehabilita- and will explore opportunities, where appropri- tion in Ontario.” The Ontario Stroke Evaluation ate, to examine best practices for patient flow. Report 2013 noted that approximately 19% of all The Ministry is an active partner of the Rehabili- inpatient rehabilitation admissions are patients with tative Care Alliance (Alliance)—a group that is mild functional impairment from their stroke, who, endorsed and funded by the 14 LHINs, and that according to the Expert Panel and other research, is tasked with building on the Rehabilitation and can generally be cared for in an outpatient setting. Complex Continuing Care Expert Panel’s Concep- The report suggested that the reason these patients tual Framework for rehabilitative care planning. were admitted to inpatient rehabilitation might be In addition, through Health System Funding the low number of outpatient and community-based Reform, quality-based procedures for stroke and rehabilitation resources. The Expert Panel recom- total joint replacement have been defined as mended that patients with an initial FIM score of 80 part of best practices for the continuum of care, or more (indicating mild functional impairment) go including the rehabilitation phase. directly from acute care to outpatient rehabilitation, rather than to an inpatient rehabilitation program. However, at the two hospitals we visited that had Total Joint Replacement stroke programs, we noted that approximately one- third of patients admitted to inpatient rehabilitation Total joint replacements—that is, total hip and had been assessed by the acute-care hospital as knee replacements—are among the most com-

Chapter 3 • VFM Section 3.08 having mild functional impairment, suggesting that monly performed surgical procedures in Ontario. they might have been better served as outpatients. In the 2010/11 fiscal year, more than 17,000 hip- One hospital told us that this was because of a replacement and almost 22,000 knee-replacement shortage of available outpatient services, as well as surgeries were performed in the province. Following because certain patients with dementia are better surgery, physiotherapy rehabilitation or exercise served as inpatients. programs are a standard treatment to maximize a person’s functionality and independence. They RECOMMENDATION 4 generally consist of various exercises, including transfer training—such as getting on and off a To better ensure that stroke patients receive chair, or in and out of a car—walking training and rehabilitation services that address their needs instruction in activities of daily living. As with most and that rehabilitation resources are used other types of rehabilitation, there are no commonly efficiently, the Ministry of Health and Long- accepted best practices province-wide; therapists term Care (Ministry) should work with the treat patients on the basis of their professional Local Health Integration Networks (LHINs) to judgment. As the Ministry expressed it in a 2012 implement, at least on a pilot basis, the stroke- report: for total joint replacement, “practice varia- rehabilitation and patient-flow best practices, tion in community rehabilitation is widespread with including those relating to timely access and limited evidence-based standards for determining a successful community rehabilitation episode.” Rehabilitation Services at Hospitals 243

We noted that the number of regular rehabilita- the number of inpatient rehabilitation cases, the tion inpatient admissions for total joint replacement percentage of patients discharged home, and the has decreased from about 9,700 in 2007/08 to average change in regular inpatients’ functional 3,900 in 2012/13. In addition, as of December 31, score from admission to discharge. This variation in 2012, acute-care hospitals across the province were performance measures limits the ability of hospitals, generally meeting the Ministry’s 4.4-day target for the LHINs and the Ministry to compare performance discharging patients after hip and knee surgery, and thereby identify better rehabilitation practices. and over three-quarters had met the target of at Each hospital also had performance measures least 90% of these patients returning home. One and processes in place related to patient safety, of the hospitals we visited indicated that it closed including incident reports and the number of six rehabilitation beds as a result of more patients patient falls. However, although all the hospitals we being discharged home for rehabilitation instead of visited required incidents to be followed up on, the to inpatient rehabilitation. The other two hospitals hospitals had different interpretations of incidents had not closed rehabilitation beds; rather, these and reporting requirements. One of the hospitals beds were available for patients with other condi- appeared to take incident reporting quite seriously: tions who needed them. it identified more than 800 falls and a total of almost One of the hospitals we visited had established 1,500 incidents in the course of the year. At one hos- a new outpatient program to help address the pital we visited, 35% of the incidents sampled either expected increase in outpatients. The other two were not reviewed within a week as required at that hospitals we visited had wait lists for their associ- hospital, or the review date was not documented, so ated outpatient programs. it was not possible to determine how long it took to complete the review. Another hospital had no time requirement for reviewing incidents, leaving the PERFORMANCE MONITORING time frame up to the rehabilitation manager’s profes- All three hospitals we visited monitored their sional judgment. At this hospital, we found that from performance and maintained oversight of their ser- April 2011 to September 2012, management usually

vices through two committees that reported to their took a median of eight days for review. At the third Chapter 3 • VFM Section 3.08 boards of directors. Their medical advisory com- hospital, the time for management to review an inci- mittees, composed of medical staff, have the goal of dent was required and documented only for medi- ensuring the quality of care provided by physicians. cation incidents. We noted that most medication Their quality of care committees, composed of incidents sampled at this hospital were not reviewed several members of their boards of directors and by senior management within a maximum of six senior hospital staff, monitor the quality of patient days, as required by this hospital’s policies. Subse- care, resolve issues and make recommendations to quent to our fieldwork, this hospital implemented an improve the quality of care. electronic system for tracking incidents, which the As well, all three hospitals had established per- hospital indicated has addressed this issue. formance measures for their rehabilitation servi- Another important factor in performance mon- ces and had systems in place to monitor and report itoring is determining the level of patient satisfac- on this information to senior management and tion. Doing so can help hospitals identify areas that their boards of directors. At two of the hospitals, need improvement. The Excellent Care for All Act, this performance information was also available 2010 (Act), requires that this be done annually. on the intranet. Each of the hospitals we visited had processes in The performance measures tracked at each place to survey inpatient satisfaction, and two also hospital varied, and included information such as conducted surveys of outpatients. Survey results 244 2013 Annual Report of the Office of the Auditor General of Ontario

were generally positive. One hospital also contacted each other. One of the hospitals was already caregivers to determine how well they were manag- comparing certain performance information ing after the patient returned home. However, none with selected hospitals in Ontario and other of the three hospitals surveyed patients’ caregivers provinces. Another hospital indicated that who had contact with the hospital in order to hospitals within its Local Health Integration determine their satisfaction in connection with the Network (LHIN) are now comparing some per- services provided to the patient, which is also a formance information. requirement of the Act. Although all three hospitals generally agreed with surveying caregivers, as required RECOMMENDATION 5 under the Act as well as outpatients, one com- In order to enhance the performance of hospi- mented that this was not a priority. tals providing rehabilitation services, hospitals should: MINISTRY RESPONSE in conjunction with the Ministry of Health • Although this recommendation was directed and Long-term Care (Ministry), develop toward hospitals, the Ministry supports the standardized performance measures that Rehabilitative Care Alliance in developing a will provide hospitals with useful and com- standardized rehabilitative care evaluation parative information, such that they can framework and set of tools, which will include benchmark their performance against other a list of indicators that can be used by organiza- hospitals and better identify areas, if any, tions to evaluate rehabilitative care system requiring improvement; and performance. This undertaking will incorporate survey patient caregivers, as required under • standardized patient outcome and/or perform- the Excellent Care for All Act, 2010 (Act), and ance measure criteria for each level of care conduct outpatient satisfaction surveys. across the rehabilitative care continuum. As well, the Ministry and the LHINs will RESPONSE FROM HOSPITALS

Chapter 3 • VFM Section 3.08 work together to ensure that appropriate All three of the hospitals agreed with developing accountability processes are followed with standardized performance measures that can regard to compliance with the Act. be used to benchmark Ontario hospitals against

Glossary

alternate level of care (ALC)—ALC patients are ready to be discharged but are waiting in hospital for post-discharge care. This can include waiting in an acute-care hospital for a rehabilitation bed, and waiting in a rehabilitation bed for home-care services or placement in a long-term-care home. Canadian Institute for Health Information (CIHI)—CIHI develops and maintains comprehensive and integrated health information, including information collected from the National Rehabilitation Reporting System for rehabilitation hospitals. Community Care Access Centres (CCACs)—Among other things, CCACs co-ordinate services for seniors, people with disabilities and people who need health-care services to help them live independently in the community. They also co- ordinate long-term-care home placement and may determine eligibility for certain complex continuing care and rehabilitation beds. There are 14 CCACs across the province, one for each Local Health Integration Network. Rehabilitation Services at Hospitals 245

complex continuing care (CCC)—CCC is hospital-based care that includes continuing, medically complex and specialized services, such as restorative rehabilitation. Functional Independence Measure (FIM)—The FIM measures the level of a patient’s physical and cognitive disabilities, and also indicates how much assistance is required to carry out various activities of daily living, such as eating, washing, dressing and toileting. GTA (Greater Toronto Area) Rehab Network—The GTA Rehab Network’s membership consists of publicly funded hospitals and community-based organizations from across the GTA that are involved in the planning and provision of rehabilitation services. One area of focus is promoting best practices and knowledge exchange. Health Quality Ontario (HQO)—HQO is a provincial agency that evaluates the effectiveness of new health-care technologies and services, reports to the public on the quality of the health-care system, supports quality improvement activities and makes evidence-based recommendations on health-care funding. Local Health Integration Networks (LHINs)—LHINs are responsible for prioritizing and planning health services and for funding certain health-service providers, including hospitals and CCACs. There are 14 LHINs, representing 14 different geographic areas of Ontario; each LHIN is accountable to the Ministry of Health and Long-Term Care. Each hospital and CCAC is directly accountable to its LHIN, rather than to the Ministry, for most matters. National Rehabilitation Reporting System (NRS)—The NRS collects data from participating adult inpatient rehabilitation facilities and programs across Canada, including specialized facilities, hospital rehabilitation units and hospital rehabilitation programs. Ontario Disability Support Program (ODSP)—ODSP, also known as social assistance, provides income and employment assistance to people with disabilities who are in need. This may be longer-term in nature. Financial assistance is provided to help pay for living expenses, such as food and housing. Employment assistance is provided to help people who can work prepare for, find and keep a job. Ontario Hospital Association (OHA)—The OHA advocates on behalf of its members, including about 150 hospitals. Among other things, it strives to deliver high-quality products and services, to advance and influence health-system policy in Ontario, and to promote innovation and performance improvement of hospitals. Ontario Stroke Network (OSN)—The OSN, created in 2008, receives funding from the Ministry of Health and Long-Term Care to provide leadership and co-ordination for Ontario’s 11 Regional Stroke Networks, whose membership includes stroke prevention clinics and Ontario stroke centres. All have a goal of decreasing the incidence of stroke and ensuring that Chapter 3 • VFM Section 3.08 Ontarians have access to quality care. Ontario Works—Also known as social assistance, Ontario Works provides financial and employment assistance for people who are in temporary need. Financial assistance is provided to help pay for living expenses, such as food and housing. Employment assistance is provided to help people prepare for and find a job. physiatrist—A medical doctor specializing in physical medicine and rehabilitation. regular rehabilitation—Inpatient rehabilitation that is shorter term, with frequent rehabilitation sessions. It is also known as high tolerance short duration rehabilitation. rehabilitation—While definitions of rehabilitation vary, the Rehabilitative Care Alliance is working on establishing a provincial definition. According to the GTA Rehab Network, “Rehabilitation helps individuals to improve their function, mobility, independence and quality of life. It helps individuals live fully regardless of impairment. It helps people who are aging or living with various health conditions to maintain the functioning they have.” Rehabilitative Care Alliance (Alliance)—Taking a system-wide view of rehabilitation in Ontario, the Alliance reports to the LHINs and works with the Ministry of Health and Long-Term Care, the CCACs and experts on various projects, such as improving system accessibility and defining best practices. Established in October 2012, the Alliance replaced the Rehabilitation and Complex Continuing Care Expert Panel, a sub-committee of the Ministry’s Emergency Room/Alternate Level of Care Expert Panel. 246 2013 Annual Report of the Office of the Auditor General of Ontario

Rehabilitation and Complex Continuing Care Expert Panel—This Expert Panel comprised rehabilitation experts and stakeholders from across Ontario. Formed to re-think the delivery of rehabilitation and complex care across the continuum, it provided advice and guidance to the Ministry’s Emergency Room/Alternate Level of Care Expert Panel on how best to reduce ALC lengths of stay throughout the system. The Rehabilitative Care Alliance replaced this Expert Panel. Resident Assessment Instrument—Minimum Data Set (RAI-MDS)—A standardized common assessment instrument used to assess and monitor the care needs of restorative rehabilitation patients in areas such as cognition, communication, behaviour and toileting. Resource Matching and Referral System—A system developed to help match hospital patients to the earliest available bed in the most appropriate setting, including both regular and restorative rehabilitation beds, as well as beds in long-term-care homes. restorative rehabilitation—Inpatient rehabilitation that is longer term in nature for people unable to participate in frequent sessions. It is also known as slow-paced rehabilitation or low tolerance long duration rehabilitation. Stroke Reference Group—Established by the Ontario Stroke Network, the Stroke Reference Group consists of rehabilitation experts and stakeholders from across the province. Chapter 3 • VFM Section 3.08 Chapter 3 Ministry of Government Services Section 3.09 ServiceOntario

renewal and registration, transferred from the Background Ministry of Health and Long-Term Care (Health) in 2008. Other products and services provided by ServiceOntario is a distinct and separate part of the ServiceOntario include: Ministry of Government Services (Ministry) that, outdoors cards and fishing and hunting since 2006, has had a mandate to provide central- • licences for the Ministry of Natural Resources; ized service delivery to individuals and businesses intake services on behalf of some ministries, seeking government information and to process rou- • such as payments to the Minister of Finance; tine transactions such as registrations and licensing. and It is one of the largest and most diverse government operating contact centres for various minis- customer service operations of its kind in North • tries, including Labour and Finance.

America. It administers several programs involving: Chapter 3 • VFM Section 3.09 Service-level agreements with the ministries vital events, such as birth, marriage and death • set out the roles and responsibilities transferred certificates; to ServiceOntario and those that remain with the business services, including company • transferring ministry. ServiceOntario provides its registration; services under a legislative framework involving personal property security registration and • more than 30 statutes. services, such as liens on vehicles; and ServiceOntario handles transactions primar- land registration, searches and title services. • ily through two delivery channels: Internet or ServiceOntario delivers these services in-house, online access; and in-person service centres, which except for an arrangement with Teranet Inc. (Tera- include 82 sites operated by ServiceOntario itself net), which has been under contract since 1991 as and 207 privately operated service provider sites. the exclusive provider of Ontario’s Electronic Land In addition, it provides information and referral Registration System. services through its website and through seven ServiceOntario also provides for 14 other ServiceOntario-operated telephone contact centres ministries high-volume, routine transactions, in Toronto, Oshawa, Thunder Bay and Kingston. most significantly driver licensing renewals and Mail is also used to a lesser extent to receive appli- vehicle registration, transferred from the Ministry cations and deliver products such as licences and of Transportation (MTO) in 2007; and health-card

247 248 2013 Annual Report of the Office of the Auditor General of Ontario

permits. As well, for several years ServiceOntario and guided by its internal strategic planning self-service kiosks were available at 71 locations, process. These changes have included developing typically in malls. ServiceOntario discontinued the ServiceOntario brand name as a recognized kiosks in 2012, primarily due to security concerns. customer-centred gateway for government service; In the 2012/13 fiscal year, ServiceOntario han- improving and streamlining back-office operations dled more than 35 million transactions, as shown and technology; integrating services for the public in Figure 1. In-person service centres accounted and businesses; making more services available for 70% of the transactions, and 30% were done online; improving service levels and timeliness of over the Internet. ServiceOntario also handled services, including offering money-back guarantees about 12 million requests for information and and premium options for certain services; and referrals—55% of these were made online, 38% seeking out cost efficiencies in service delivery. In through the telephone contact centres and 7% at its addition, since 2011, the government has directed in-person service centres. ServiceOntario to explore opportunities for alterna- In 2012/13, ServiceOntario, which has a staff of tive service delivery, including greater private- approximately 2,000, collected $2.9 billion in rev- sector involvement and capital investment. enues, including $1.1 billion under MTO’s driver’s licence and vehicle registration programs and $1.5 billion under the land transfer tax program. ServiceOntario’s expenditures totalled $289 mil- Audit Objective and Scope lion, 55% of which was spent by its Customer Care Division on operating costs for its in-person service Our audit objective was to assess whether Service- centres and telephone contact centres, and on com- Ontario had adequate systems and procedures in missions for its private operators. place to: Changes made by ServiceOntario over the years • provide the public with one-stop access to gov- have been driven by government direction, often ernment information and routine transactional as announced in the province’s annual budget, services in a timely manner with due regard

Chapter 3 • VFM Section 3.09 for economy and efficiency and in compliance Figure 1: Number and Type of Transactions Handled, with legislation and program policy; and 2012/13 (million) • measure and report on the effectiveness of Source of data: ServiceOntario service delivery. Total Transactions: 35.0 million Senior management at ServiceOntario reviewed and agreed to our audit objective and criteria. Driver and vehicle (20.2) Our audit work included interviews with Service- Other (0.8) Ontario management and staff, as well as reviews Vital statistics (1.0) and analysis of relevant files, registration and Business (2.1) licensing databases, and policies and procedures at ServiceOntario’s head office, in-person service Real property —Land (3.9) centres, contact centres and back-office operations across the province. We visited 14 ServiceOntario Personal property in-person service centres, including both publicly (2.6) and privately run sites; three telephone contact centres; Teranet; and the service provider that Health card (4.4) manufactures and distributes photo identity cards, including driver’s licences and health cards. ServiceOntario 249

We interviewed several stakeholders who are encourage people to switch to doing business major users of registration programs operated by online instead of in person. For instance, ServiceOntario. We met with senior personnel from we estimated that ServiceOntario’s operat- Health and MTO to solicit their views on their part- ing costs would decrease by approximately ner relationship with ServiceOntario. We conducted $2.9 million annually if 50% more licence research into similar programs in other provinces plate sticker renewals were done online. and foreign jurisdictions. We also engaged an • ServiceOntario has made improvements to independent expert on public service delivery. its website services, but its online customers’ satisfaction rating has remained at 71% to 75% since 2009/10. • ServiceOntario rated 43% of its 289 in-person Summary service centres as high-risk locations because of the number of processing errors uncovered Notwithstanding its substantial accomplishments by its audits. These ranged from incorrect in centralizing services, ServiceOntario needs to financial charges to missing signatures on improve in several key areas. It needs to continue health-card applications to renewing the to strengthen its systems and procedures in wrong licence plate number or transferring a order to reduce service delivery costs, effectively vehicle to a name other than the one on the monitor service levels and customer satisfaction, application. and reduce its risks in issuing and managing • In the fourth quarter of 2012/13, 98% of licences, certifications, registrations and permits. clients surveyed at in-person service centres In particular, ServiceOntario’s Audit Oversight Unit reported they were either satisfied or very had identified, and was working on addressing, an satisfied with the services they received. error rate for processing transactions that was too However, site managers are notified of the high at many of its in-person service centres. surveying days in advance, and counter staff We noted no significant backlogs or delays are aware that clients could be questioned,

with most services provided by ServiceOntario, which could skew the results on those days, Chapter 3 • VFM Section 3.09 and ServiceOntario is generally meeting certain making the survey of questionable value. service-level targets, which were for the most part • ServiceOntario did not measure or report on at reasonable levels compared to other provinces. the customer wait at peak times or at specific Ontario was the first jurisdiction in North America service centres, which often far exceeded its to offer money-back guarantees on the prompt target time of 15 minutes. processing and delivery of some services, including • In 2012/13, none of ServiceOntario’s seven birth and marriage certificates and personalized telephone contact centres met its service licence plates. ServiceOntario fulfills its goals on standards for answering calls. The range of these transactions virtually 100% of the time. success in answering calls within targeted However, if ServiceOntario is to further improve times was 51% to 77%, compared to its goal the delivery of cost-effective services to Ontarians, of 80%. This may be reflected in survey action is needed in the following areas: results that found the customer satisfac- • In the 2012/13 fiscal year, only 30% of tion level was 64% in the fourth quarter of ServiceOntario transactions were done online, 2012/13, down from the 70% maintained for well short of ServiceOntario’s forecast of 55% several quarters previously. Clients’ satisfac- to 60%. Further savings could be achieved if tion level for timeliness of service was only ServiceOntario had an effective strategy to 52%, down from 65%. 250 2013 Annual Report of the Office of the Auditor General of Ontario

• ServiceOntario was still charging fees over even though they process both types of trans- and above what it costs to run certain regis- actions. As well, approximately 166,000 active tration programs. (A 1998 Supreme Court heath cards, including 144,000 of the red-and- of Canada decision concluded that user fees white cards, were listed in the database as not could be repayable if the amounts charged having current addresses for the cardholders. were excessive and did not have a reasonable This means there was no way to determine relationship to the cost of the services pro- whether cardholders were residents of Ontario vided.) As well, user fees did not cover the full and thus eligible for coverage. cost of certain other programs as required by • ServiceOntario had weak processes for issuing government policies and guidelines. and controlling accessible parking permits to • ServiceOntario had no plans in place to stop ensure they were not being misused by people printing birth certificates on paper and switch who did not require them. to higher-security polymer (plastic) docu- • ServiceOntario staff did not verify that people ments and a new design to minimize identity registering large commercial farm vehicles— theft, forgery and loss, as recommended by who are charged a reduced rate compared the Vital Statistics Council for Canada. Eight to individuals registering other commercial other provinces have already switched to vehicles—were indeed farmers. An applicant polymer documents. merely had to tick a box on a form identifying • Significant fraud risk still exists 18 years after that he or she was a farmer. We estimated that the government announced its plan to reduce this weakness could be costing the province costs by replacing the red-and-white health about $5 million annually in lost commercial card, which has no expiry date, with the more vehicle registration fees. secure photo health card. As of August 1, • ServiceOntario did not obtain independent 2013, 3.1 million red-and-white cards assurance that the performance reports on the remained in circulation, or 23% of the total of province’s land registry system operated by 13.4 million health cards issued in Ontario. Teranet were complete and accurate, and that

Chapter 3 • VFM Section 3.09 The conversion rate has declined by about disaster recovery plans and security measures 45% since ServiceOntario assumed respon- were validated routinely. sibility from Health in 2008. Full conversion is not expected until 2018. OVERALL SERVICEONTARIO • We estimated that as of March 31, 2013, RESPONSE approximately 1,500 people in Ontario had ServiceOntario appreciates the work of the been issued duplicate health cards, increasing Auditor General and her staff, and the valuable the risk of misuse. As well, more than 15,000 observations and recommendations provided active health cards (including 6,000 red-and- as a result of this audit. We recognize that our white cards) and 1,400 driver’s licences were transformational agenda is not yet complete. Pro- circulating in the names of people who were moting greater adoption of electronic services is reported to ServiceOntario as deceased. a foundational component of our ability to drive We also estimated that as many as 800,000 • service delivery change within government. We people with red-and-white health cards remain committed to championing and promot- had old addresses attached to those cards ing the benefits of the online channel to our cli- compared to their driver’s licence records. ents and ministry partners at every opportunity. ServiceOntario did not cross-reference basic information such as addresses in databases ServiceOntario 251

primarily due to security concerns. However, our We will continue to operate in a cost-effective audit found that ServiceOntario fell short of these manner and leverage existing funds wisely, targets. In 2012/13, 70% of all transactions were recognizing that some recommendations, such still done at in-person service centres and only 30% as examining the benefits and cost savings from were done online. In fact, in-person transactions introducing a smart card, may require additional increased from 68% in 2011/12, mainly because investment. the ServiceOntario kiosks were shut down. The As well, ServiceOntario will continuously majority of kiosk users switched to visiting in- improve oversight of the service delivery net- person service centres rather than completing their work. We will explore with ministry partners transactions online. and consult with the Office of the Information ServiceOntario offers a number of driver and and Privacy Commissioner to find acceptable vehicle transactions online—most recently allowing ways for additional information-sharing, qualified motorists to renew licences through its including short-term opportunities related to website. People who want to register changes of name changes. address, renew licence plate stickers, order per- All of these efforts are consistent with sonalized plates, order vehicle records or request ServiceOntario’s three key strategic priorities: used-vehicle information packages also may do so to provide customer service excellence, to find online. (With licence plate sticker renewals, people cost savings and to protect the integrity of the can complete the information and payment parts programs we deliver. of the transactions online, and the stickers are then mailed to them within five business days.) How- ever, it’s clear that most people prefer to visit in- person service centres, where they receive personal Detailed Audit Observations assistance with these transactions. Of 20 million driver and vehicle transactions in 2012/13, approximately 900,000 (less than 5%) were completed SERVICE DELIVERY COSTS

online. Of approximately 6.6 million licence plate Chapter 3 • VFM Section 3.09 Use of Internet sticker renewals in 2012/13, almost 90% were done at in-person service centres. To reduce costs, ServiceOntario is attempting to It would save the government a significant get Ontarians to complete as many eligible driver, amount of money if people could be persuaded to vehicle and health-card transactions as possible switch to online transactions. For example, Service- online, rather than by visiting service centres in Ontario calculates that the direct cost to the govern- person. However, for the most common Service- ment of a licence plate sticker renewal transaction Ontario transactions—issuing and renewal of a online is $2.91, compared to $3.84 at a privately driver’s licence or health card, vehicle registration run in-person service centre and an average cost and licence plate sticker renewal—people still most of $8.70 at a location operated by ServiceOntario. often go to service centres in person. We estimate that if 50% more of these transactions In its 2008 strategic plan, ServiceOntario fore- were completed online, the government would save cast that 55% to 60% of all its transactions would approximately $2.9 million annually. In addition, if be completed over the Internet by 2012. It wanted more transactions were processed online, over time to reduce the number of transactions at in-person the cost per Internet transaction would decrease service centres to 30%, with the remainder handled due to economies of scale. at kiosks, which were subsequently closed in 2012 252 2013 Annual Report of the Office of the Auditor General of Ontario

ServiceOntario’s success in having more people incorporation of a business online, and $360 if done use the Internet has occurred primarily where, as by mail or at an in-person service centre. with land registration and personal property security transactions, it has made the service available In-person Service Centres only online. Approximately 60% of ServiceOntario’s 10.2 million website transactions occur for services In addition to trying to redirect transactions to the that are available only online. In contrast, only Internet, ServiceOntario developed a retail oper- 15% of people who had a choice opted for online ations optimization plan to streamline over-the- transactions. counter procedures and find cost savings by closing We noted that ServiceOntario has not extensively some of the in-person service centres it operates studied why Ontarians prefer to use in-person ser- or by altering operating hours and improving staff vice centres instead of its online option. One reason productivity. might be that people prefer to have their documents, Of the 289 in-person service centres, Service- such as a registration, permit or licence, handed to Ontario operates 82; the other 207 are owned by them when they complete the transaction, rather private operators who are paid a commission for than wait for delivery by mail at a later date. For each transaction they process. In 2012/13, the instance, we noted that some other provinces and in-person service centres processed almost 25 mil- several American states no longer require that an lion transactions, with 80% handled by the private annual validation sticker be attached to licence operators. There is a significant difference in cost plates. The vehicle owner must still renew the plate per transaction between the sites ServiceOntario registration annually and pay the fee, but this can runs itself and those run by private operators. The easily be done online. The fact that there is no sticker Ministry calculated that the average cost of trans- eliminates the part of the transaction that may be actions at its publicly run sites was $9.92, compared discouraging people from using the online renewal to the overall average commission of $3.30 per method, particularly if they wait until the last transaction paid to operators of privately run sites. minute—their birthday—to renew. In addition, the operating costs of each publicly

Chapter 3 • VFM Section 3.09 Another way to persuade people to do their run service centre varied significantly, with the transactions online would be to offer discounts on average cost of transactions at individual sites the website, or, conversely, charge higher fees for across the province ranging from $5 to $21. While in-person services. As noted, it costs less to process we expected that rural and northern publicly run transactions online, but these savings are not passed sites would have higher operating costs, we also on to clients. ServiceOntario has no clear strategy found that many publicly run sites in large cities on setting fees, either for programs it fully adminis- had relatively very high costs. ters or for those it administers with other ministries, A number of factors contribute to the higher to encourage greater Internet usage. During the transaction costs at publicly operated in-person 2013 Ontario Budget process it proposed to the service centres. Publicly run in-person service cen- Minister of Finance that it raise fees for in-person tres generally are more costly to operate because transactions, but such increases were not approved. they often have larger premises to maintain and Currently, only ServiceOntario’s business registra- greater overhead costs, including higher wages tion fees are structured this way. A premium ranging paid to more full-time staff. ServiceOntario pays a from 13% to 33% for some business transactions set commission rate to privately operated centres, had been set prior to ServiceOntario’s establishment which are typically small businesses that keep in 2006. For example, it costs $300 to register the their overhead costs, including wages to staff, at levels that enable their owners to make a profit. ServiceOntario 253

ServiceOntario limits services offered at privately most efficiently meet its service-level standards by operated in-person service centres to primarily employing a mix of 70% full-time staff and 30% high-volume health-card and driver and vehicle part-time staff at each contact centre. This was transactions, whereas publicly operated centres designed to allow each centre the flexibility to have offer several more relatively low-volume services, more staff answering phones at peak call-volume including issuing fishing and wildlife licences and times. However, we found that ServiceOntario was receiving landlord/tenant board filings. still working on this and had made some progress In 2012/13, ServiceOntario closed six public through staff attrition, although six of the seven call in-person service centres, of which four were in centres had not yet met the 70/30 split. One centre southern Ontario, one in the east and one in the had less than 10% part-time staff. north. The decisions were based on having other nearby in-person service centres handle more trans- RECOMMENDATION 1 actions. This saved $2.5 million in 2012/13 and was To help further reduce service delivery costs, expected to save $4.2 million in 2013/14. Service- ServiceOntario should: Ontario advised us that no final decisions had been better identify the reasons people opt for in- made about closing any more offices in 2013/14. • person service rather than use the Internet, Many rural and northern ServiceOntario-run and examine possible changes it could make, in-person service centres handle fewer transactions. including to its pricing strategy, to promote Most are open five days a week for seven hours greater use of online transactions; and a day, just like high-volume locations. To reduce examine ways to expedite reducing operat- operating costs, ServiceOntario determined in • ing costs at its publicly run in-person service 2012 that it should reduce operating hours for 23 centres to bring them closer to the already- rural and northern centres and open them only 2 lower cost of commissions paid at the pri- to 3.5 days per week, depending on the location. vately run in-person service centres. As of June 2013, service hours had been reduced at five of these locations through the attrition of full- SERVICEONTARIO RESPONSE time staff, some of whom were then replaced with Chapter 3 • VFM Section 3.09 part-time workers. ServiceOntario has said it has We support the Auditor General’s observation no plans to lay off any staff to accommodate such that the online channel represents a tremendous changes. Further savings will be achieved more opportunity for government services in Ontario. slowly through attrition. Once reduced operating During June to August 2013, ServiceOntario hours are in effect for all low-volume locations, conducted research to better understand cus- ServiceOntario expects that further savings will be tomer behaviour with respect to the use of our $1.5 million annually. channels. The findings will result in a refresh of ServiceOntario’s action plan by 2014 and will address possible promotional opportunities Telephone Contact Centres designed to encourage higher usage of the The cost of running ServiceOntario’s seven call online channel. centres in the 2012/13 fiscal year was $38 million. ServiceOntario will continue its efforts to Most of this was spent on about 350 staff, who promote the online channel, including: provided callers with information and referrals • continuing to expand our electronic suite of but generally did not handle transactions. Service- services; Ontario had a plan in place to address staffing. As of 2011, ServiceOntario calculated that it could 254 2013 Annual Report of the Office of the Auditor General of Ontario

Wait Times • encouraging customers to use our online channel through various marketing efforts; ServiceOntario does not publicly report its wait- and time standards or actual wait times for the 82 in- • exploring different approaches to acceler- person service centres it operates. Internally it has ate the shift online, potentially including a a target of a 15-minute average wait time. This falls differential fee structure or mandatory use within the Ontario Public Service Common Service of electronic services. Standards, which require a wait time in a queue to In the last 15 months, ServiceOntario has be less than 20 minutes unless otherwise communi- realized savings by reducing its public in-person cated. We found that ServiceOntario had calculated footprint and hours of service in some commun- the average wait times at its in-person service cen- ities to more closely match operating hours with tres over the past four fiscal years as follows: demand for services. We will continue to assess • 13.6 minutes in 2009/10; community needs and explore options to further • 13.3 minutes in 2010/11; reduce service delivery costs, while respecting • 9.5 minutes in 2011/12; and our obligations as an employer. • 9.1 minutes in 2012/13. This data is for only the service centres run by ServiceOntario. It started collecting wait-time data for the 207 in-person service centres run by private SERVICE LEVELS operators just last year. Service standards are public commitments to a level The averaged, long-term data for in-person of service that customers can expect under normal service centres run by ServiceOntario does not circumstances; they typically address timeliness, measure the wait customers can expect at peak accuracy and accessibility of a government service. times or at specific locations. We reviewed Service- Service standards are meant to be monitored and Ontario reports on publicly run sites and noted revised over time so that the government can that many larger urban sites had peak-time waits improve its responsiveness to the public and oper- far greater than 15 minutes. Many had several Chapter 3 • VFM Section 3.09 ate more efficiently. days during the month in which the average wait ServiceOntario has developed service standards time for the day exceeded the standard. It was not for transactions involving programs it administers uncommon for wait times during peak hours to be directly, and for the in-person services it provides 45 minutes, with some customers waiting more for transactions administered on behalf of other than two hours for service. However, when Service- ministries, such as driver and vehicle transactions Ontario averages these numbers over full days and (Ministry of Transportation) and health cards (Min- over a month, the wait-time calculation usually falls istry of Health and Long-Term Care). within the 15-minute standard. ServiceOntario offers a money-back guarantee Some of ServiceOntario’s privately run infor the prompt processing and delivery of a birth or person service centres also experienced long marriage certificate, or personalized licence plates. wait times. Nineteen of those centres exceeded a Ontario was the first jurisdiction in North America 15-minute average wait time in 2012, and there was to offer money-back guarantees on public services, no reporting on peak times. and ServiceOntario meets its standards on these ServiceOntario has also established service transactions virtually 100% of the time. levels for its seven telephone contact centres. The However, we found there was room for improve- targets for the time in which 80% of calls should be ment in monitoring and reporting on wait times answered are as follows: and levels of client satisfaction. ServiceOntario 255

• general inquiries: 30 seconds; In the fourth quarter of 2012/13, 98% of cus- • driver and vehicle questions: two minutes; tomers surveyed were satisfied or very satisfied • 24/7 health line: one minute; with the service centres. Typically, customers who • health information: two minutes; and had to wait more than five minutes for service were • business information line: 30 seconds. less satisfied than those who were served faster. We These service levels adhere to Ontario Public asked ServiceOntario whether it might be better to Service Common Service Standards, which use the “mystery shopper” technique to assess how require that calls received through a call centre be counter staff handled customers and transactions; answered within two minutes, unless otherwise however, we were advised it would do so only communicated. In 2012/13, however, none of the under extraordinary circumstances. The same num- seven contact centres answered 80% of the calls ber of clients were surveyed both at publicly run within the target times. The range of success was and at privately run sites, even though privately run only 51% to 77%. sites account for 70% of in-person service centres. ServiceOntario determines how many staff For Internet transactions, since 2008, customers each contact centre should have by calculating how have been asked to complete a short online survey many people are needed to reach the expected ser- at the end of the transaction; about 50,000 surveys vice level. However, we noted that one contact cen- are completed every quarter. While we expected tre had fewer staff than the recommended number that customer satisfaction would have grown with for the period we reviewed, and had poor service the improvements that ServiceOntario has made to levels as a result. Another contact centre had more its Internet services, the overall satisfaction rating than the recommended number, and its service was has remained at 71% to 75% since 2009/10. relatively much better. For the telephone contact centres, Service- Ontario began measuring customer satisfaction in 2008. Each quarter, an independent survey com- Client Satisfaction pany questioned a sample of about 500 people who ServiceOntario measures client satisfaction for its recently used the service. In the fourth quarter of

in-person service centres, Internet transactions and 2012/13, the contact centre satisfaction level was Chapter 3 • VFM Section 3.09 telephone contact centres. 64%, down from the 70% that had been maintained For in-person service centres, it employs an for several quarters previously. For the specific independent survey company to poll 250 clients at question about timeliness of service, the satisfac- publicly run sites and 250 at privately run sites each tion level was only 52%, down from 65%. These quarter to assess their overall satisfaction with the numbers indicate that ServiceOntario contact cen- services they received. tre service requires substantial improvement. Survey sites were chosen randomly, but regional ServiceOntario also set a target of having call- representation was considered. Site managers were centre staff spend only 30% to 35% of their time notified in advance of the survey, and on the day of on administration rather than than handling calls. the survey, the counter staff were fully aware that From 2011 to 2013, the actual time spent at each call clients could be questioned by the survey company. centre on non-phone duties, which include admin- Normally, clients were surveyed in the service cen- istration and customer follow-up work, was 35% to tre lobby in front of counter staff. Thus, managers nearly 50%, which could have had a negative impact and counter staff would be highly motivated to on customer service. However, ServiceOntario did provide their best customer service on survey day, not have a system that would allow it to better ana- making the survey results of questionable value. lyze non-phone duties, and was working on capturing this information at the time of our audit. 256 2013 Annual Report of the Office of the Auditor General of Ontario

ServiceOntario also gathered data on turn-away consider a method of surveying clients that is rates for such things as health-card transactions • not done with full knowledge of counter staff and driver and vehicle transactions. Counter staff at in-person service centres, who may then may turn away customers for a number of reasons: be highly motivated to provide their best for instance, when they do not meet identifica- service only on survey day; and tion requirements, or when the computer system devise a method for counter staff to report is down. However, while the number of people • on why customers are turned away for turned away was recorded, the reasons for turning such services as health-card and driver and them away were not. In addition, turn-away rates vehicle transactions, and use this data to were gathered only for publicly operated in-person improve customer service where required. service centres; privately operated sites were not required to collect this information. SERVICEONTARIO RESPONSE For health-card transactions, the customer turn- away target rate was not to exceed 12.8% (clients As noted by the Auditor, ServiceOntario has are typically turned away because they do not bring already achieved a 33% reduction in average the identity or citizenship documents needed to wait times at our publicly operated offices since complete a transaction). However, over the past two 2009/10. We will continue to evaluate our years, 15% to 17% of clients were turned away. In wait-time data collection methodology against the 2012/13 fiscal year, some service centres turned industry best practices to reflect a typical away only 2% of customers, and others as many customer experience. Capturing all wait-time as 28%. Since turn-aways are at the discretion of data requires additional investment in smart the counter staff, ServiceOntario should confirm queuing systems, which is feasible only in the that its policies are applied consistently and should largest offices. We will re-evaluate technologies investigate the specific reasons that people are as they evolve to determine feasibility in all turned away in order to develop effective strategies ServiceOntario centres. to reduce such instances. For in-person centres experiencing load chal-

Chapter 3 • VFM Section 3.09 lenges, an expert task force has been in place RECOMMENDATION 2 since May 2013. The task force is responsible for developing practical wait-time improvement To ensure that ServiceOntario has appropriate strategies, and as a result of its efforts, we are management information that would allow it to already observing progressive improvements at further improve its service and increase client these centres. satisfaction, it should: ServiceOntario notes that our contact collect data and report on peak-hour wait • centres experienced a temporary dip in perform- times at both the in-person service centres it ance as they transitioned to our new technology runs itself and those run by private operators, platform. While these types of transitional as well as examine and address the reasons impacts are typical of large-scale technology for long wait times at many of the large, and process transformations, they do not urban in-person service centres; reflect ServiceOntario’s commitment to service examine why none of the seven telephone • excellence. contact centres met the service levels estab- Accordingly, a number of corrective meas- lished for answering calls from the public, ures were initiated, and we are pleased to note and take steps to improve client satisfaction continuous improvements in our service-level ratings for these services as well as for online performance since the results of the 2012/13 transactions; fiscal year: ServiceOntario 257

have a reasonable relationship to the cost of the Six of 18 lines of business have now sur- • services provided. passed their 80% service-level target. In 2011, we reported that the Ministry had Fifteen of 18 lines of business provide a less- • identified potential strategies to address this risk, than-two-minute Average Speed of Answer. including possibly reducing the fees over time, and The most recent second-quarter customer • that it was working with the Ministry of Finance to satisfaction survey results have returned to present a strategy to the Treasury Board of Cabinet pre-transition levels of 70% “Very Satisfied.” for consideration. However, no timetable was In 2014, ServiceOntario will review its cus- provided for completing this. As part of our current tomer satisfaction survey program with experts audit, we again followed up on this matter and in the field to ensure our methodologies address noted that no further progress had been made. the Auditor’s concerns. ServiceOntario had direct responsibility for Through frontline staff focus groups that about $104 million in fees collected in the 2012/13 convened in April, May and September 2013, fiscal year for programs that it fully administers, ServiceOntario has identified the most com- including services for land, personal property, busi- mon reasons for turn-aways. The groups most nesses and vital events. Any proposals for fee chan- affected are youth, new immigrants and people ges would normally be made by ServiceOntario to for whom English is a second language. the Ministry of Finance and require government We have developed a plan to reduce these approval. The other $2.8 billion in fees and taxes turn-aways. The plan will be implemented by ServiceOntario collected are flow-through rev- the end of this fiscal year (March 31, 2014). It enues since they were collected on behalf of other includes: ministries’ programs, such as for driver and vehicle a multilingual handout for agents to distrib- • transactions, land transfer tax, and fish and wildlife ute to help customers understand what docu- transactions. Responsibility for proposing fee chan- ments are required when they return; and ges for flow-through revenues is with these other stakeholder outreach to ensure that youth • ministries. There are no revenues for health-card and new immigrant communities under-

services, as fees are prohibited under the federal Chapter 3 • VFM Section 3.09 stand what documentation is required prior Canada Health Act. to their first visit. Government policies and guidelines require ministries to regularly review services and rates, and when it is reasonable and practical to do so, the cost USER FEES of providing services to the public should be borne by those who benefit from the service. Service- Our 2009 Annual Report section on government Ontario did not have robust processes to ensure this user fees noted that the Ministry of Government was the case, and it had not established a strategy Services was at risk of a constitutional challenge for restructuring its fees to meet these requirements. over its collection of non-tax revenues for certain No fees have been changed since 2006 for programs registration services because the fee revenues that ServiceOntario fully administers. As Figure 2 exceeded the cost of providing the services by indicates, there still are significant differences in approximately $60 million. In 1998, the Supreme revenues and costs for its registration programs. Court of Canada ruled that user fees could be con- ServiceOntario was working to lower its operat- sidered unlawful and therefore may be repayable ing costs, including by restructuring for greater if they were determined by a court to be a tax that efficiency, upgrading technology, improving man- was not established by enacted legislation or if the agement information and reporting, and promoting fee amounts charged were excessive and did not 258 2013 Annual Report of the Office of the Auditor General of Ontario

greater use of its lower-cost Internet services. How- ISSUING AND MANAGING LICENCES, ever, these efforts had not led to any fee reviews CERTIFICATIONS, REGISTRATIONS AND and thus any operating savings that were achieved PERMITS would not result in adjustment to fees. Birth Certificates Figure 2: Comparison of Fees and Costs for The Office of the Registrar General (Office) is a Registration Programs, 2012/13 ($ million) branch of ServiceOntario responsible for regis- Source of data: ServiceOntario tering births, deaths, marriages, adoptions and Vital Personal name changes in the province. ServiceOntario, Program Statistics Business Property through the Office, provides certificates and certi- Fees collected 23.5 37. 9 40.8 fied copies of registrations to the public. Each year, Direct and approximately 300,000 events are registered and 25.8 18.9 6.9 indirect costs 580,000 certificates and certified copies are issued. Net operating (2.3) 19.0 33.9 Our 2004/05 audit of the Office found sig- profit (loss) nificant backlogs and processing delays for birth certificates. However, as a result of a new system in 2007 and other organizational changes, the turn- RECOMMENDATION 3 around time for processing registrations and issuing To ensure that registration-related fees are set at certificates has improved significantly, and these levels that would recover the costs of providing times are reasonable in comparison to service levels services when it is reasonable and practical to reported by other provinces. However, we noted do so and also to meet the legal requirement two areas that need improvement: that fees not be set at excessive amounts, • The Vital Statistics Act requires guarantors for ServiceOntario should conduct a full costing applications for birth certificates for anyone and revenue analysis, and develop a strategy over the age of 9. Applications with guarantors with time frames for restructuring its fees. accounted for 43% of all applications for birth

Chapter 3 • VFM Section 3.09 certificates or requests for certified copies of SERVICEONTARIO RESPONSE birth registrations received annually. Policy of the Office of the Registrar General states that We agree with the Auditor General’s recommen- guarantors must be audited on a sample basis. dation that registration-related fees should meet We found that very few guarantor audits were legal requirements. done. In 2012, only 151 guarantor audits were There are two streams of user fee revenue: completed among the over 150,000 applica- services that ServiceOntario manages directly tions for people over the age of 9. and services that are offered on behalf of other Ontario is one of the last provinces to still ministries. • print birth certificates on paper. The Vital Sta- ServiceOntario is continuing to refine the tistics Council for Canada has recommended cost of each transaction it manages directly and that all provinces update from paper to poly- will develop a costing analysis and a strategy mer (plastic) birth certificates with security for restructuring its fees for registration-related features designed to minimize identity theft, services in 2014. forgery and loss. From 2007 to 2010, eight provinces adopted the more secure polymer birth certificates, but ServiceOntario has no plan to do so. ServiceOntario 259

Health-card Registrations In 2010, ServiceOntario expanded the number of locations that could issue health cards to 289 ServiceOntario annually issues about 305,000 from 27. This improved customer access, but it health cards to new eligible registrants, including also increased the risks pertaining to processing 137,000 to newborns and 168,000 to newcom- health-card transactions, since many of these loca- ers, and renews about 1.4 million for existing tions were small offices with limited management cardholders using procedures agreed on with the oversight. In 2012/13, ServiceOntario found that Ministry of Health and Long-Term Care. Since 130 of the 289 in-person service centres had high- health cards provide cardholders with essentially risk error rates greater than 15% with respect to the free medical services anywhere in Canada, Service- health-card application process. Ontario must ensure that cards are provided only to individuals who are legally eligible to receive the services. People applying for OHIP coverage and an Conversion to New Health Cards accompanying health card are required to submit In 1995, the Ministry of Health and Long-Term Care original documents that provide: (Health) introduced a photo health card to eventu- proof of citizenship or OHIP-eligible immigra- • ally replace the red-and-white, non-photo cards tion status; that were then carried by all Ontarians eligible to proof that they live in Ontario; and • receive OHIP benefits. Health originally planned to support of their identity, including name and • have all eligible Ontarians carrying the new photo signature. card by 2000. However, the government did not However, once an applicant shows the required make conversion mandatory, and many red-and- documents at the ServiceOntario counter and the white cardholders chose not to replace their cards. information is recorded, all source documents The program to convert to the more secure card are returned to the applicant. In most cases, the offered many opportunities to Health. It provided information is authenticated electronically with a chance to verify that each person who was issued the source organization, either ServiceOntario’s a new card indeed met the requirements for OHIP Registrar General or Citizenship and Immigration

eligibility. The red-and-white card has no photo and Chapter 3 • VFM Section 3.09 Canada. In other cases where the documents used no information other than the cardholder’s name— can’t be authenticated, no copies are made of what no date of birth or address, for example—so it is of proof was shown, so there is no audit trail available little value in confirming a cardholder’s identity for to make sure counter staff processed transactions eligibility. And unlike the new photo card, which according to policy requirements. This is of par- requires periodic renewal, the red-and-white card ticular concern for higher-risk transactions, such as does not expire. applications by newcomers to the province whose After 18 years, as of August 1, 2013, there were documents cannot be electronically authenticated. still 3.1 million red-and-white cards—23% of the ServiceOntario’s internal auditors mentioned this total of 13.4 million health cards issued—in circula- problem in a November 2011 report; however, tion in Ontario. As we reported in our 2006 Annual no changes have been made. In addition, we Report audit of OHIP, from 2002/03 to 2004/05, noted there was no requirement for a supervisor the number of red-and-white cards taken out of to double-check counter staff work, such as con- circulation was about 400,000 annually. But the firming that a new registrant has provided a proper reduction rate declined by about 45% on average identity document. Nor is a supervisor required to annually over the last five fiscal years since Service- authorize higher-risk transactions, as is the practice Ontario assumed responsibility from Health for the at banks, for example. conversion, as shown in Figure 3. 260 2013 Annual Report of the Office of the Auditor General of Ontario

Figure 3: Number of Red-and-white Cards Removed from Circulation, 1995–2012 Sources of data: Ministry of Health and Long-Term Care (Health) and ServiceOntario 900,000

800,000

2008—ServiceOntario assumed 700,000 responsibility from Health for health-card conversion project

600,000

500,000

400,000

300,000

200,000

100,000

0 1995 1996 199719981999 200020012002 20032004 2005 2006 2007 2008 2009 2010 2011 2012

Chapter 3 • VFM Section 3.09 Red-and-white cards fall out of circulation when We estimated that 25% of the addresses of the cardholders die, move out of the province or holders of red-and-white cards were outdated as country, lose the card and must get a replacement, of 2012/13. Many of these cardholders would have or voluntarily request an updated card. To encour- come to ServiceOntario for driver’s licence and age the voluntary exchange of old cards for new vehicle transactions, but ServiceOntario did not use ones, ServiceOntario mails out notices requesting the address information from these transactions to that red-and-white cardholders replace their update the addresses assigned to health cards. cards. Due to budget constraints, ServiceOntario In our 2006 OHIP audit, we noted that Health has sent only about 36,000 requests in each of the did little monitoring of individual health-card last two years. As well, we were told by owners of usage. In 2005, a consulting firm hired by Health privately run in-person service centres that they are estimated the value of consumer fraud in Ontario’s aggressively promoting voluntary card conversion health-card system at $11 million to $22 million to people coming in to renew their driver’s licence annually. Health had not updated that estimate at or plate stickers. The centre receives an additional the time of this audit. commission for a health-card replacement trans- In its 2013 budget, the provincial government action. In contrast, management at publicly run announced that it would invest $15 million over service centres told us they were not instructed to three years, starting in 2013/14, to accelerate the promote health-card conversions. conversion of the remaining red-and-white health ServiceOntario 261 cards to the more secure photo cards. The full example—have moved to some form of smart-card conversion is expected to be completed by 2018. system, combining at least two government cards. In the 2013/14 fiscal year, the plan was to remove 500,000 old cards from circulation. Commercial Farm Vehicles Starting in late 2013, counter staff were to ask customers to verify their health-card addresses As of March 31, 2013, Ontario had almost 1.5 mil- when they came to ServiceOntario locations for any lion registered commercial vehicles that weighed other transactions. more than 3,000 kilograms, and 78,100 registered farm vehicles in the same weight categories. The province allows farmers to pay reduced annual Smart Card registration fees for licence plate stickers for com- The Ontario government has, over the last 15 years, mercial farm vehicles compared to what would launched initiatives that explored the possibility of otherwise be paid for commercial vehicles. The replacing a number of government cards—driver’s annual fee for a farm vehicle registration is $43 licences, health cards and birth certificates, for to $2,802 less than the fee paid for a commercial example—with a single, secure identity card. This vehicle registration, depending on weight. For has been commonly referred to as a smart card. example, an operator of a commercial vehicle with Microchip technology and other evolving security a gross weight of 25,000 kilograms would pay an measures have made the prospect for such a card annual fee of $1,331. If the vehicle were registered more feasible. If a smart card was implemented, as a farm vehicle, however, the owner would pay an the public likely would want reassurance that the annual fee of only $322. personal information stored on this kind of univer- ServiceOntario staff do not verify that the sal card remains private and is used only for the owner of a vehicle is indeed a farmer. An applicant purposes for which it is intended. merely has to tick a box on a form identifying In 2012, Ontario passed legislation that estab- that he or she is a farmer. We found that from lished the authority for developing such a card. 2003/04 to 2012/13, the number of commercial

Advantages for consumers would include having farm vehicles registered with MTO increased by Chapter 3 • VFM Section 3.09 to carry and renew only one card. For the govern- 56%, while the number of commercial vehicles ment, the advantages would include streamlining registered increased by only 13% overall. More- card production processes with reduced production over, Statistics Canada’s farm activity indicators for and transaction costs. For example, we estimate Ontario declined from 2001 to 2011. We estimated that the annual savings in card production costs that weaknesses in ServiceOntario’s verification alone from combining the health cards and driver’s procedures could be costing the province about licences of 9 million people into a single ID card $5 million annually in lost commercial vehicle would be about $3.4 million, although significant registration fees, assuming that the number of upfront investment in card design and data transfer farm vehicles did not actually increase more than would be required. Such a card could also allow the rate for other commercial vehicles. government to work toward giving each Ontarian only one identity number, which would reduce the Accessible Parking Permits need for individuals to have multiple IDs across government databases and would help to integrate In our 2005 Annual Report section on Disabled government services. Person Parking Permits, we identified that MTO did Other jurisdictions—British Columbia, the not adequately review applications for accessible state of Queensland in Australia, and Germany, for parking permits. In response, MTO held discussions 262 2013 Annual Report of the Office of the Auditor General of Ontario

with the medical community, reassessed its criteria Since our 2005 audit of the driver and vehicle for medical conditions that qualified a person for a private issuing network, which included accessible permit, and started using death records to identify parking permits, the number of permits seized by deceased permit holders more quickly; however, law enforcement agencies had decreased. In 2005, no substantial changes were made to improve the about 1,600 permits were seized, compared with verification of the application forms. 710 in 2012. However, enforcement was difficult Since our last audit, we noted that the number because parking enforcement officers did not have of active accessible parking permits had increased access to ServiceOntario’s database to see if permits from 540,000 to 615,000 as of January 2013. are legitimate. ServiceOntario is now responsible for issuing Once a month, ServiceOntario matched a list accessible parking permits and still had weak proof names of people who died, provided internally cesses for reviewing and verifying applications. by its Office of the Registrar General branch, to its To obtain an accessible parking permit, an list of accessible parking permit holders. An exact applicant’s health condition must be certified by a match automatically rendered the permit inactive. regulated health-care practitioner. Either a tempor- However, ServiceOntario did not require that the ary or a permanent permit is issued, depending permit be sent back, and misuse of a technically on the applicant’s health condition. A temporary inactive permit was difficult to catch. As well, only permit is valid for up to five years, and the applicant exact matches were inactivated. In our examination needs to reapply upon the permit’s expiry. A perma- of a sample of renewal notices, we noted a few had nent permit is issued for a five-year period, and an been sent to people who were deceased, including applicant who renews the permit does not need to one who had been dead for four years. obtain recertification of his or her health condition. Our testing found that the permits lacked Permits allow parking in designated accessible effective security features and could be copied eas- parking spaces, and, depending on the jurisdiction, ily. As well, blank permits were kept at the desks can also be used to get free parking at meters and of employees and were not numbered serially in in pay-and-display spaces, and to park in some no- advance, which means there were no controls over

Chapter 3 • VFM Section 3.09 parking zones. The advantages create an incentive the number of permits that could be printed. for misuse of the permits and for counterfeiting. Other jurisdictions have improved their pro- At the time of our audit, ServiceOntario was cesses for issuing accessible parking permits. In following MTO’s earlier policy for accessible park- British Columbia and Quebec, permit holders must ing permits by randomly verifying the professional carry an accompanying permit card or certificate registration numbers of health-care practitioners that enforcement officials can ask to see. In New before mailing out a permit. These professional York City, a city health department physician must registration numbers are publicly available on the recertify disabilities. In Australia, permit stickers Internet, so verification of the numbers provides are placed on vehicles, with the name, date of birth no assurance that the practitioner supported and and picture of the permit holder, and enforcement signed the application. Temporary, three-month officers carry scanners to detect fake permits. permits were issued right at ServiceOntario Subsequent to our discussions during our audit counters, where counter staff simply made sure the fieldwork, ServiceOntario began work to improve application had been filled out. There was no veri- its accessible parking permit policy and procedures. fication of the information, and front-counter staff It began implementing a policy to require appli- could not determine whether the applicant had had cants to provide documents to verify their identity. a permit seized by enforcement officials or had had As well, temporary permits were no longer to be an application for a permanent permit rejected. handed over to people submitting an application on behalf of someone else. ServiceOntario 263

RECOMMENDATION 4 the integrity of our data and authentication processes. ServiceOntario had considered the use of To improve service and security surrounding polymer composition materials in birth certifi- the issuing and management of licences, certificates but did not implement this option due to cates, registrations and permits that it adminis- cost. We agree to re-examine in 2014 the feasibil- ters, ServiceOntario should: ity of using polymer stock and will analyze the ensure that it completes enough guarantor • experiences of other Canadian jurisdictions. audits for birth certificate applications, and ServiceOntario has available staff support consider updating its birth certificate iden- for the delivery of higher-risk transactions, tity document to the newer polymer com- as well as transactions requiring policy inter- position and design standard to minimize pretation/adjudication. This support includes identity theft, forgery and loss; on-site supervisors and subject matter experts, reassess the processes in use and supervisory • telephone hotline specialists, and reviews con- oversight over counter staff at in-person ducted by our Eligibility Unit. In addition, the service centres to better ensure policies Ministry of Health and Long-Term Care provides and procedures are followed for processing policy support and clarification of more complex higher-risk transactions and verifying that OHIP-eligibility issues to ServiceOntario staff customers provide proper documents when as required. ServiceOntario agrees to explore registering for health cards; other cost-effective and operationally feasible complete its long-delayed conversion from • approaches to high-risk transaction oversight the old red-and-white health cards so that to further enhance the integrity of delivery and all Ontarians are carrying the more secure maintain customer service standards. photo health cards that reduce the risk of With government support and funding that fraudulent medical claims; was confirmed in the 2013 Ontario Budget, we examine the benefits and cost savings from • will begin an accelerated, mandatory conver- creating a smart card that would combine sion of red-and-white health cards to the more

more than one government ID card, and set Chapter 3 • VFM Section 3.09 secure photo health card in winter 2013/14. timelines to achieve them; This conversion will be completed in the improve verification requirements for appli- • 2018/19 fiscal year. In June 2013, we started to cations to make sure that vehicles registered more than double the number of red-and-white as farm vehicles, and thus subject to a much health-card conversion letters sent weekly and lower annual registration fee than other are actively marketing a “keep your address up commercial vehicles, are indeed used for to date” campaign to customers. farm purposes; and ServiceOntario recognizes the potential value improve processes for issuing accessible • of an integrated card for multiple government parking permits, and introduce changes that programs and has begun a review of possible would make it easier to identify abusers. options. ServiceOntario will work closely with its ministry partners to determine the feasibility SERVICEONTARIO RESPONSE and value of the card, and assess the legislative Noting that Ontario is the only jurisdiction that authority required for potential options. Privacy- performs guarantor audits for birth certificate friendly design, cost effectiveness, and the applications, ServiceOntario will conduct an potential for use across a variety of government analysis of the effectiveness of guarantor audits programs are key themes being explored prior to and associated policies as a means of ensuring any commitment to implement. 264 2013 Annual Report of the Office of the Auditor General of Ontario

identification or licences to ineligible people, or ServiceOntario has consulted with the Min- improper or duplicate registrations on its data- istry of Transportation (MTO) on the licensing base. As well, it needs to ensure the accuracy and of commercial farm vehicles. MTO is developing integrity of its registration and licensing services policy options to address the Auditor General’s and databases. We identified a number of areas concern and will be consulting with stakeholders where controls could be improved. on possible options. Once a policy direction has been confirmed, MTO will work to determine an implementation and communications plan. Audit Oversight ServiceOntario agrees with the need to ServiceOntario has implemented a robust audit enhance the integrity of its administration of program of its 289 in-person service centres to the Accessible Parking Permit (APP) program. identify locations with high error rates in process- It is currently addressing the Auditor General’s ing transactions. We were concerned that 43% of its concerns by: centres are rated as high risk because of the number ensuring consistency between accessible • of processing errors the audits uncovered. parking permits and driver’s licences in ServiceOntario’s Audit Oversight Unit (Unit) how a person’s name is recorded in order to conducts both full on-site audits and more limited improve ServiceOntario’s ability to prevent off-site audits of a sample of transactions. Both the fraudulent use of permits and strengthen privately and publicly run in-person service centres oversight of the issuance of renewal permits; are audited to ensure that there is appropriate and assessing the security of the permit, and • accurate documentation for all transactions; that evaluating new and effective design ele- transactions have been processed correctly and all ments, including serial numbers to control commissions calculated accurately; and, for full and measure permit production and distri- audits, that valuable stock (such as licence plates bution; and and renewal stickers) is properly secured and collaborating with municipalities that • accounted for. enforce APP-related laws to identify appro-

Chapter 3 • VFM Section 3.09 The Unit has increased the number of audits it priate mechanisms for tracking permit seiz- conducts. In the 15 months up to March 31, 2013, ures and enforcement. 88% of all in-person service centres had an on-site In addition, ServiceOntario will explore audit and 99% of centres had at least one off-site opportunities to collaborate with MTO to audit of transaction records. By comparison, in the incorporate the APP program into the Medical 2011 calendar year, 45% of centres had on-site aud- Reporting Modernization Project, enabling regu- its and 57% had off-site audits. (The Unit changed lated health practitioners to facilitate the direct its reporting period from a calendar year in 2011 submission of approved APP applications and to a fiscal-year period in 2013, which included a the immediate production of temporary permits. one-time three-month difference. It was unable to provide us with identical periods for comparison.) The Unit considers an in-person service centre QUALITY CONTROL OVER PROCESSING to be high risk when the audit results in an error TRANSACTIONS rate higher than 15%, calculated by the number of errors divided by the number of transactions sam- ServiceOntario needs to ensure that transactions pled. The audit results do not include minor errors; are processed correctly and securely because of the auditors instead focus on more significant the substantial risks involved, such as issuing errors, including missing signatures on health-card ServiceOntario 265 and driver’s licence applications, incorrect identity volumes and amount of revenue generated; and document type recorded on the application, health- whether the centre has changed staff, management card renewals without citizenship information or ownership. being on file, a vehicle transferred to a name other than the one indicated on the application, wrong Database Integrity licence plate number renewed, and incorrect cash, cheque or credit-card adjustments or transactions. ServiceOntario’s procedures and IT system controls In the 2011 calendar year, the Unit found are designed to mitigate the risk of issuing dupli- that 23% of locations audited had error rates cate health cards, driver’s licences or birth certifi- higher than 15%; in the 15-month period ending cates that could allow people to obtain services or March 31, 2013, this percentage increased to 43%. privileges for which they are not eligible. As well, Many of the locations’ error rates far exceeded the if a deceased person’s identity card is not cancelled 15% threshold, with some reaching 50% to 60%. promptly, it could be used inappropriately. When Sixteen of the 125 high-risk locations identified in issuing or renewing a health card or renewing a the 15-month period ending March 31, 2013, were driver’s licence, ServiceOntario staff perform a also identified as high risk in the calendar year limited search of the databases of Health or MTO 2011. There were no significant differences in the using name, birthdate and sex to see whether any error rates between privately and publicly operated existing health cards or driver’s licences are issued in-person service centres. in the same name. However, ServiceOntario has For the 2013/14 fiscal year, the Unit plans to not established procedures for its counter staff to focus on in-person service centres that were con- cross-reference the information in those databases sidered high risk from the previous year’s audits, to further verify the applicant’s identity even particularly those with error rates higher than 30%. though the same counter staff can process both The Unit also intends to improve its interactions, types of transactions. such as holding more regular status meetings, with Based on our analysis of the databases as of high-risk sites to monitor their progress, and take March 31, 2013, and in some cases data going back

other action—including legal action—as needed. over the previous five years, we found a number of Chapter 3 • VFM Section 3.09 We were advised that errors identified in the control weaknesses that affected data integrity that audits are discussed with management of the in- we shared with ServiceOntario. The following are person service centres. However, the Unit did not among the more significant: compile regular reports that summarized the types • We estimated that approximately 1,500 people and frequency of errors found, including whether in Ontario had been issued duplicate health the errors were financial or clerical, or whether they cards; 580 of these individuals held two of were more serious and affected the security and the old red-and-white cards, which have no integrity of registration and licensing databases. expiry date, and no photo or other identifying Such reports would help identify areas in which information on them except a name, and thus staff need training and would identify errors that carry a significant risk for fraudulent use. In result from problems in processes and IT systems. comparison, MTO has virtually eliminated the Besides past error rates, ServiceOntario’s issuance of duplicate driver’s licences since audits should consider other risk factors related it uses electronic photo comparison to detect to operating in-person service centres—for duplicates before they are authorized. No simi- instance, whether the site handles more complex lar technology is used by ServiceOntario or transactions, such as a relatively higher number of Health for health cards. Furthermore, Service- health-card registrations to newcomers; transaction Ontario counter staff have previous electronic 266 2013 Annual Report of the Office of the Auditor General of Ontario

photos of driver’s licence cardholders available for the cardholders attached to them; this on their system, but do not have photos avail- means that neither Health nor ServiceOntario able for health-card renewals. In addition, we can locate these cardholders or verify their found a few cases where the application pro- Ontario residency, a key requirement for cess allowed newborns to receive two separate eligibility for health services. Furthermore, birth registrations. we compared address information for hold- • To make sure that the health cards and drivers’ ers of the red-and-white health cards with licences of people who have died are cancelled their addresses in MTO’s database for driver’s promptly, MTO and Health receive a monthly licences, which must be renewed every five notification of deaths from ServiceOntario. years, and found that as many as 800,000 For deaths that Health’s automated matching of them had a more current address in the system fail to match, the exceptions list is MTO database. However, ServiceOntario provided to ServiceOntario, which manually staff had no established procedure to access checks the list against the health-card data- or use MTO addresses to update addresses base. We compared the death records from in the health-card database, even though the ServiceOntario’s Registrar General database same counter staff can process both types of to Health’s health-card and MTO’s driver’s transactions. licence databases and estimated that there • Many people who legally changed their names were more than 15,000 active health cards with ServiceOntario’s Office of the Registrar (including 6,000 red-and-white cards) and General did not inform Health or MTO of this, 1,400 driver’s licences in the names of people even when they renewed their health card who have died that the systems and processes or driver’s licence with ServiceOntario. The failed to cancel. When a health card or driver’s Registrar General does not share name change licence is not cancelled promptly, there is information with the MTO and Health, an increased risk of it being misused; in the although it does inform the Ontario Provincial case of a health card, fees could continue to Police, who then inform the Canadian Police

Chapter 3 • VFM Section 3.09 be paid to the deceased person’s health-care Information Centre (CPIC) operated by the provider until the card is terminated. Health RCMP for updating criminal records. We officials advised us that in some cases there reviewed the data of the 50,000 people over may be legitimate medical claims for services the last five years who had legally changed performed on deceased persons, and that their names and found that an estimated there was a need to positively ensure that only 2,400 had not updated the name on their cards for people who are verifiably deceased health card and 800 had not changed the persons are cancelled. However, they agreed name on their driver’s licence. At the time that to minimize risk, health cards should be these people had their new legal name regis- cancelled promptly upon receiving notification tered, they would have received a new birth of a death. Health and ServiceOntario advised certificate from ServiceOntario with that new us that they are committed to reviewing their name. Thus, there is a risk that people have related policies and procedures. two different identification documents, which • Approximately 166,000 active health cards, could result in their inappropriately receiving including 144,000 of the red-and-white cards duplicate government services, for example. that have no expiry dates, were listed in the database as not having current addresses ServiceOntario 267

RECOMMENDATION 5 guard an individual’s privacy and to improve data integrity while meeting the government’s To ensure that transactions are processed in statutory obligations under the Freedom of accordance with legislation and established Information and Protection of Privacy Act and procedures, and to reduce the risk of fraud and the Personal Health Information Protection Act. misuse of government-issued identity docu- To this end, ServiceOntario will prepare options ments, ServiceOntario should: for providing electronic change-of-name regularly identify from its audit activities the • notifications to the Ministry of Transportation types and frequency of errors found that can (MTO) and the Ministry of Health and Long- be used to target staff training and changes Term Care (Health). to its systems and procedures needed to Maintaining the integrity of records is a high reduce the high transaction error rate at priority for ServiceOntario and all of its partner many of its service centres; ministries. Equally, ensuring that records do not recommend to its partner ministries the need • get incorrectly changed and that mismatches for further automated and other processing are avoided is of critical concern. Service- controls to improve the security and integrity Ontario will seek to build on previous efforts of registration and licensing databases; with MTO and Health and explore additional improve its systems for cancelling identity • improvements in data-matching processes for documents for people who have died; and death records. At the same time, ServiceOntario co-ordinate with the Ministry of Health and • will continue to reconcile addresses between a Long-Term Care, the Ministry of Transporta- driver’s licence and health card whenever client tion and the Office of the Information and consent is received. Privacy Commissioner to introduce measures More significant changes such as a such as limited sharing of current addresses centralized and consolidated approach to among databases in order to mitigate the authentication and verification of some eligi- risks posed by erroneous and duplicate iden- bility requirements necessitate a longer time

tity documents. Chapter 3 • VFM Section 3.09 frame, investment and may require changes to ServiceOntario’s existing scope of authority. SERVICEONTARIO RESPONSE ServiceOntario will consult with the Office of Since 2010, ServiceOntario has expanded its the Information and Privacy Commissioner of Quality Assurance audit program to include Ontario and work closely with its partners to health-card registration, as well as new risk develop a proposal that considers expansion of and intervention frameworks. It encompasses existing information-sharing agreements. service delivery through both publicly and privately operated centres. We agree that taking steps to further realize the business value TERANET IT PERFORMANCE of Quality Assurance audit data in supporting MONITORING process and system improvements will be beneficial, and we have already begun to take As part of its licensing agreement, Teranet is appropriate action. required to adhere to industry standard methodol- ServiceOntario continues to explore ways ogy to ensure effective controls are in place for the to further integrate products and the delivery key information technology processes involved in of services to improve customer service, to safe- providing electronic land registration services. To 268 2013 Annual Report of the Office of the Auditor General of Ontario

demonstrate that it is meeting this requirement, RECOMMENDATION 6 Teranet provides ServiceOntario with quarterly IT performance reports on measures including To better ensure the ongoing reliability and accessibility, availability, system response time, availability of Ontario’s electronic land regis- server performance, network performance, secur- tration system, ServiceOntario should obtain ity, application functionality and data integrity, and independent assurance that Teranet’s perform- system and data backup. Committees comprising ance reports, and its disaster recovery plans representatives from ServiceOntario and Teranet and security measures, meet industry-accepted meet regularly to monitor Teranet’s performance standards and are validated routinely. Service- and whether established targets have been met. Ontario should also periodically test its copy of We noted that ServiceOntario relies on infor- the land registration source code software. mation provided by Teranet for its monitoring activities, and reports are not independently SERVICEONTARIO RESPONSE verified either by ServiceOntario or by internal or As part of our ongoing commitment to service external auditors. ServiceOntario does not obtain improvement, ServiceOntario and Teranet have independent assurance that performance reports agreed to apply a comprehensive assessment from Teranet are complete and accurate, and that framework that is consistent with what the disaster recovery plans and security measures are Chartered Professional Accountants of Canada validated routinely. (formerly the Canadian Institute of Chartered Teranet provides ServiceOntario each quarter Accountants) recommends regarding reporting with a copy of the source code software that would on controls for a service organization. This allow the Ministry to use or recreate the electronic new framework will be applied to reporting land registration system in the event Teranet was as of March 2014. ServiceOntario will explore unable or unwilling to fulfill its obligations under alternative cost-effective ways to obtain addi- the agreement. We verified that ServiceOntario was tional third-party assurance of disaster recovery receiving the source code regularly; however, it had plans and security measures standards.

Chapter 3 • VFM Section 3.09 not tested the software to ensure it could use the The licence agreement with Teranet does program without further support and co-operation include a master transition plan to execute an from Teranet. orderly transition of the electronic land registration system from Teranet to another third-party or government operator. ServiceOntario will investigate cost-effective means to verify its copy of the source code software it receives from Teranet for the land registration. Chapter 3 Ministry of Community and Social Services Section 3.10 Violence Against Women

• assist women experiencing violence and Background their children by enhancing self-esteem and supporting them to access resources to live independently; and In Ontario, a wide range of ministries, sectors, pro- enhance the co-ordination of VAW services at fessionals and community members are involved in • the community level. providing services and supports to women and their The Ministry provides transfer payments to children who are fleeing violence. These include more than 200 not-for-profit agencies within local shelters and counselling services, child welfare communities to deliver supports and services to workers, police, health-care professionals, the jus- abused women and their children. These agencies tice sector and social assistance and housing. In the are governed by volunteer boards of directors. 2010/11 fiscal year (the latest year for which data The Ministry is responsible for prioritizing and

was available), the province estimated that it spent Chapter 3 • VFM Section 3.10 co-ordinating local service delivery, as well as for a total of $220 million across all ministries dealing allocating public funds in response to priorities with the issue of violence against women (VAW). identified by VAW agencies and the local commun- Two-thirds of these costs were for VAW programs ity. The Ministry’s head office establishes program and services that were administered by the Ministry policies and procedures, and its nine regional of Community and Social Services. offices oversee funding and program delivery for The Ministry of Community and Social Ser- the agencies in their respective jurisdictions. vices (Ministry) provides community programs In the 2012/13 fiscal year, the Ministry spent and services aimed at helping women and their $142 million in transfer payments. Of that amount, children who are victims of domestic violence find approximately $82 million went toward the oper- safety and rebuild their lives free of violence. The ation of 95 shelters. The remaining $60 million was programs also serve adult survivors of childhood for other supportive services, including community- sexual abuse. The objectives of the Ministry’s VAW based counselling, telephone-based counselling programs and services are to: (crisis help lines) and connecting women with sup- increase the safety of women who are experi- • ports to help them secure more permanent housing. encing violence and their children by provid- Figure 1 shows a breakdown of transfer-payment ing safe shelter, safety plan development and funding for VAW programs and services. crisis counselling;

269 270 2013 Annual Report of the Office of the Auditor General of Ontario

abuse. Senior management at both the Ministry of Figure 1: Ministry Funding Allocation to Violence Community and Social Services and the Ontario Against Women Programs and Services, 2012/13 Women’s Directorate reviewed and agreed to our Source of data: Ministry of Community and Social Services audit objective and associated audit criteria. Emergency Shelter Services (59%) This audit focused on VAW programs and servi- Other (5%) ces administered by the Ministry of Community and Provincial and Regional Social Services, and on the co-ordination efforts of Crisis Line Counselling (2%) the Ontario Women’s Directorate. Child Witness In conducting our audit, we reviewed relevant Program (4%) documents; analyzed information; interviewed appropriate ministry, directorate and agency staff; Transitional and Housing Support and reviewed relevant research from Ontario and Program (10%) other jurisdictions. Our audit work was conducted primarily at the Ministry’s head office, at three of the Ministry’s nine regional offices, and at the Counselling Services Ontario Women’s Directorate. We also visited six (20%) women’s shelters to gain a better understanding of the services provided and to review selected pro- In the last decade, the province released two cedures, and met with the chairs of four Domestic multi-ministry action plans to deal with the issue Violence Community Coordinating Committees. of violence against women: the Domestic Violence We followed up with the Ministries of the Attorney Action Plan (2004) and the Sexual Violence Action General and Municipal Affairs and Housing on Plan (2011). As well, in 2009 the Domestic Violence select issues. We also followed up on the status of Advisory Council (Council), created by the Minister all action plans released by the government over Responsible for Women’s Issues, released a report the last decade that were relevant to the issue of with 45 recommendations for improving the system violence against women and on the Domestic Vio-

Chapter 3 • VFM Section 3.10 of services for abused women and their children. lence Advisory Council’s 45 recommendations. This The Ontario Women’s Directorate, a government audit excluded programs and services for victims office reporting to the Minister Responsible for of rape or sexual assault, which are funded by the Women’s Issues, is responsible for co-ordinating the Ministry of the Attorney General. implementation of the action plans and the Coun- The internal audit team for the Ontario Women’s cil’s recommendations across the government. Directorate conducted a risk assessment on the grants process in 2008 and reviewed a sample of grant files in 2011. We reviewed its reports and considered its work and any relevant issues identified Audit Objectives and Scope when planning our audit work.

The objective of our audit was to assess whether the Ministry of Community and Social Services and the Ontario Women’s Directorate had adequate Summary mechanisms in place to meet the needs of abused women and their children cost-effectively, and to Effectiveness of the Multi-ministry Domestic measure and report on the effectiveness of services Action Plan and initiatives aimed at curtailing violence against During the last decade, Ontario developed action women and at helping victims of this type of plans to address violence against women: the Violence Against Women 271

Domestic Violence Action Plan (released in 2004) where they did receive services. The Ministry and the Sexual Violence Action Plan (released in also has no information on how many of 2011). Nine years after the release of the Domestic these women were turned away because the Violence Action Plan, we would have expected the shelter was full and how many were turned government to have assessed whether the plan away because they had not been abused and was meeting its objectives of preventing domestic were therefore ineligible for VAW services. violence and improving supports for abused women Emergency shelter directors said that if their and their children. However, the progress reports shelter is full, they refer abused women to publicly issued to date by the Ontario Women’s Dir- other emergency shelters first, followed by ectorate have been mainly anecdotal, with no clear homeless shelters, because homeless shelters indication of each commitment’s implementation do not have appropriate supports for abused status or of what outcomes have been achieved. In women and their children. And, as noted, this regard, Statistics Canada data on the preva- neither the shelter directors nor the Ministry lence of domestic violence before and after the knew whether the women who were referred 2004 plan showed some change in Ontario: the elsewhere ultimately received services. percentage of women who reported experiencing • Despite the recommendations made in our spousal abuse decreased, from 7% in 2004 to 6.3% 1994 and 2001 audits of VAW programs and in 2009 (latest available information). Moreover, services, the Ministry still has not developed the Ontario rate for self-reported spousal abuse in any standards, service directives or guidelines 2009 was in line with the national rate. for services provided under VAW funding, such as minimum staffing levels, admission criteria and exit criteria for emergency shelters. VAW Programs and Services Administered by The Ministry’s monitoring efforts are not Ministry of Community and Social Services • sufficient to identify possible service gaps, For programs and services funded by the Ministry inefficiencies in service delivery and inequi- of Community and Social Services (Ministry) to ties across agencies and regions. For instance, assist women and children who are fleeing domestic

although agencies that deliver the same type Chapter 3 • VFM Section 3.10 violence, we found that the Ministry did not know of service are required to report on the same whether services were sufficient to meet the needs of types of data, the Ministry does not compare abused women and their children and did not have the results of one agency to another. Instead, sufficient information to properly assess the effect- the Ministry’s analysis is limited to totalling iveness of the VAW programs and services offered. reported results by region and for the prov- Our more significant observations included the ince overall, but only for select types of data. following: The types of data that are not analyzed, but The Ministry does not have the information • that could provide useful insight, include the needed to identify the unmet demand for VAW number of women not served by service type, services and in turn allocate resources to close and the proportion of women served who the gap. For example, in the 2011/12 fiscal found housing. year, emergency shelters reported that they Ministry funding to transfer-payment agen- initially turned away almost 15,000 women • cies is generally historically based, with little in total, or 56% of women who sought help or no correlation to identified needs or past from them. However, this figure overstates performance. As a result, we found significant unmet demand because the Ministry does variations in actual unit costs among agencies not track how many of the women who were providing similar services. To illustrate, in turned away were referred to another agency 272 2013 Annual Report of the Office of the Auditor General of Ontario

2011/12, Ministry-approved annual funding the true status of safety and security issues at for 10-bed emergency shelters ranged from VAW shelters until it performs another Build- $334,000 to $624,000. Consequently, the per- ing Condition Assessment of VAW shelters, day cost of care at emergency shelters ranged which is expected to occur by March 2019. from $90 to $575. • For approximately 20 years, Statistics Canada • The Ministry’s client satisfaction survey, has been surveying all residential facilities conducted to assess users’ perceptions of VAW providing services to abused women and services, provides limited value because of their children across Canada and collecting a low response rate and the limited number information on both the services provided and of agencies represented. For example, the the clientele. This survey, currently called the response rate in 2011/12 could have been as Transition Home Survey, collects information low as 4% if women followed the Ministry’s that the Ministry would find useful in helping requirement of completing a separate survey it assess its programs’ effectiveness, such as for emergency shelter services, counselling the number of repeat users, the number of services and services from the Transitional women turned away from shelters and the and Housing Support Program. In addi- reasons for their being turned away, and what tion, no surveys were completed for 20% of service gaps and other challenges are faced agencies, and fewer than 10 surveys were by shelters and residents. Since Statistics Can- completed for an additional 40% of agen- ada publicly reports only select information cies. The survey results’ usefulness was also that may or may not have been included in a limited because responses to the survey were previous report, the Ministry would be well consolidated irrespective of the nature of the served to request more detailed survey results service being provided. Consequently, it is not in order to best identify where improvements known to which specific service the survey are needed in Ontario and how it compares to responses pertained. Most agencies we visited other jurisdictions. conducted their own satisfaction surveys or

Chapter 3 • VFM Section 3.10 exit interviews with clients, but in general OVERALL MINISTRY RESPONSE they did not compile or analyze the responses The violence against women (VAW) program to identify areas for improvement. provides a system of support services that are In 2009, a Building Condition Assessment • designed to meet the diverse needs of women of VAW shelters (which included a security and children at the local level, including assessment) identified more than 500 safety emergency shelter, counselling, child witness and security items that required attention program, transitional and housing support, across all VAW shelters. As of March 31, and provincial crisis line services. Programs 2012, which was the latest available status are delivered by non-profit, volunteer boards of update, the Ministry had provided funding for directors that are accountable to the Ministry for only 10% of the identified safety and secur- the effective use of public funds. ity deficiencies, but did not know whether The Ministry appreciates the findings and the funded projects had been completed or recommendations of the Auditor General that whether the agencies themselves had paid to build on improvements under way: fix any of the remaining 90% of the identified In 2010, the Ministry developed a resource deficiencies. The Ministry does not perform • guide to assist shelter agencies with the site inspections. Therefore, it might not know Violence Against Women 273

development of their policies and procedures. It is intended to help agencies provide high- Detailed Audit Observations quality services. • In the 2012/13 fiscal year, the Ministry PROVINCIAL INITIATIVES updated the Transfer Payment Governance Over the last decade, the provincial government has and Accountability Framework in order to released two action plans to help prevent violence support the implementation of new data against women and improve supports for those reporting requirements and has implemented affected: standardized expenditure categories in order the Domestic Violence Action Plan (2004), and to provide better analysis of agency costs. • the Sexual Violence Action Plan (2011). Reporting requirements for transfer payment • • The two action plans were developed by the gov- agencies were changed for 2012/13 and ernment after consultation with survivors/victims, 2013/14 to improve the Ministry’s ability front-line service providers and other experts in the to collect accurate information on factors health, education and justice sectors, as well as in that affect program costs and to compare the community. Both plans outlined commitments costs between agencies providing similar that were initially expected to be implemented over programs. These changes will provide more a four-year period. consistent, meaningful and reliable data to In 2007 the government also posted on its measure the performance of programs and website the Strategic Framework to End Violence support the program planning needs of the Against Aboriginal Women, which was developed Ministry and agencies. by aboriginal organizations after consultation The Ministry is also reviewing the Transfer • with aboriginal community leaders. Although the Payment Risk Assessment Methodology and Domestic Violence Action Plan is intended for all Tools to improve its effectiveness. women, the aboriginal community believed that a The Ministry is developing an Asset Manage- separate strategy was required because aboriginal ment Framework to better support capital fund-

women suffer higher levels of abuse than non- Chapter 3 • VFM Section 3.10 ing decision-making and will complete Building aboriginal women. To illustrate, in 2009, the most Condition Assessments of all Ministry-funded recent year for which data was released, Statistics sites over the next five fiscal years. Canada reported that on a national level aboriginal The Ministry agrees that the co-ordination of women were almost three times more likely to services could be strengthened through building experience domestic violence than non-aboriginal on existing forums and established relation- women, and more than 40% more likely to suffer ships. The Ministry will develop a strategic plan injury from that abuse. The government has not to identify priorities for areas such as regional made a commitment to implement the recom- planning activities, provincial reporting and mendations of the strategic framework, but it has enhanced service system co-ordination across endorsed the framework’s overall objectives and sectors. To improve client evaluation, the Min- approach as a useful tool for planning and estab- istry is assessing ways to capture the impact lishing government priorities. of VAW programs on women escaping abuse Figure 2 summarizes the objectives, areas of who may not be willing or able to recount their focus and commitments for both action plans and experiences. the strategic framework. 274 2013 Annual Report of the Office of the Auditor General of Ontario

Responsibility for implementing the action the Directorate publicly released a progress plans rests with the Ministerial Steering Committee report. Similar to the 2012 publicly released on Violence Against Women (Committee), which progress report on the Domestic Violence comprises 13 ministers and is chaired by the Minister Action Plan, the report is mainly anecdotal Responsible for Women’s Issues. The Committee and contains no clear listing of commitments is supported by the Ontario Women’s Directorate or their status. (Directorate), which is responsible for co-ordinating • Action taken to address concerns raised in the action plans’ implementation across the minis- the Strategic Framework to End Violence tries. With respect to the strategic framework, the Against Aboriginal Women has been slower Committee established a Joint Working Group on than expected. The Ministerial Steering Com- Violence Against Aboriginal Women to identify prior- mittee established the Joint Working Group ities and opportunities for support, development and on Violence Against Aboriginal Women in fall implementation of policies, programs and services 2010, three years after the framework was that prevent and reduce violence against aboriginal developed. In May 2012, the working group women and their families. developed a work plan, and in September We requested a status update from the Director- 2012, it released its first progress report, ate on the various commitments and recommen- covering initiatives undertaken between 2010 dations under the action plans and the strategic and 2012. We reviewed the work plan and the framework and noted the following: progress report and noted that work was in • The Directorate maintained an internal track- progress for all actions in the work plan. ing report for the Domestic Violence Action The establishment of the action plans, including Plan that outlined commitments, implementa- the collaborative process used and the govern- tion status and achievements by ministry for ment’s recognition of the strategic framework for each focus area. This internal tracking report aboriginal women, are steps in the right direction was last updated in 2008, even though the with regard to helping reduce violence against latest progress report on the action plan was women and attempting to provide a more accessible

Chapter 3 • VFM Section 3.10 released in 2012. The Directorate informed and responsive system for those who experience us that after 2008, it had ongoing verbal com- abuse. However, neither the overall plans nor the munication with the ministries to update the individual commitments within these plans had status and achievement for each commitment. any specified measurable outcomes against which According to the 2008 tracking report, 75% of to assess their effectiveness in preventing violence commitments were completed, 20% were in or improving services for those affected. The status progress and 5% were outstanding. The status updates generally reported achievements in the of commitments in 2012 was unclear because form of activities, such as an increase in number of the publicly released progress report is mainly shelter beds, number of women served or number anecdotal and does not include a clear listing of safety plans completed. To illustrate: of commitments. • The Domestic Violence Action Plan included • The Directorate has been tracking the stage a commitment to increase funding to of completion for each commitment in the community-based counselling services in Sexual Violence Action Plan. At the time of order to address wait lists and gaps in service our audit fieldwork, the last update prepared for specific populations. The status update by the Directorate was as of January 2013; reported an increase in the number of people this assessment indicated that 60% of com- counselled, but made no determination on mitments were completed and 40% were whether the wait lists and gaps in service were in progress. Subsequent to our fieldwork, addressed. Moreover, the Ministry did not Violence Against Women 275 Commitment ($) $113M new ($87M funding and $26M reallocation of existing funding) $18M new ($11M funding and $7M reallocation of existing funding) None # of Ministries with Commitments 7 ministries and the Ontario Women’s Directorate 3 ministries and the Ontario Women’s Directorate None directly Chapter 3 • VFM Section 3.10 violence centres related to aboriginal women and violence against all aboriginal women in Ontario aboriginal women well as government capacity to end violence against aboriginal women violence Identify women and children at risk intervene earlier Change attitudes to prevent violence from happening in the first place Provide better community-based supports for victims Improve access and equity to services Strengthen the justice system response Provide better access to French-language services Prevent sexual violence front-line service providers Train sexual assault centres and domesticImprove services provided by Help survivors the navigate system of supports Improve supports for aboriginal women Improve access to services for francophone women Improve access to interpreters to victims of human trafficking Respond Strengthen the criminal justice response Reform legislation and hold offenders accountable Community collaboration Undertake comprehensive research and data collection on issues Legal reform and legislative change Creation of a comprehensive policy to target and address violence Creation of a sustained policy and program infrastructure Public education campaign to raise awareness of violence against as Build and sustain aboriginal community and organizational capacity, Support and build community leadership that works toward ending Ensure accountability for a broad commitment to the strategy

Areas of Focus • • • • • • • • • • • • • • • • • The following are not ministry the strategies, been proposed by but have authors of the framework: • • • • • • • • Objective/Pupose bring a collaborative To approach focused on preventing domestic violence and improving supports for abused women and their children take a co-ordinated and To collaborative approach to prevent sexual violence and improve education, justice and service supports for women experienced sexual who have violence the have government work To collaboratively with aboriginal organizations and communities to develop a “continuum of care” to address issues related to violence against aboriginal women Initiative Domestic Violence Action Plan (2004) Sexual Violence Action Plan (2011) Strategic Framework to End Violence Against Aboriginal (2007) Women Figure 2: Initiatives Developed to Address Violence Against Women the Office of Prepared by the Auditor General of Ontario 276 2013 Annual Report of the Office of the Auditor General of Ontario

have information on what the wait lists and of our audit fieldwork, action was still in progress gaps in service were. on three-quarters of the recommendations. • Under the focus area of strengthening the criminal justice response, the Sexual Violence Change in Prevalence of Violence Against Action Plan included a commitment by the Women Ministry of the Attorney General to provide enhanced training to justice personnel in Because the progress reports on the Domestic order to improve their understanding of the Violence Action Plan and the Sexual Violence impact of sexual assault on victims and in turn Action Plan are silent on whether there has been improve the criminal court system’s response any change in the prevalence of violence against to sexual assault. The status update noted women since the plans were created, we reviewed that a two-day training program was held for the latest available data from Statistics Canada’s Crown attorneys, police and co-ordinators General Social Survey to assess their impact, if any. of Domestic Violence Treatment Centres. As shown in Figure 3, the percentage of women in However, there was no assessment of how this Ontario who reported having experienced spousal action improved the criminal court system’s violence within the previous five years decreased response to sexual assault. by 0.7 percentage points, from 7% in 2004 to 6.3% The Directorate told us that individual ministries in 2009. Moreover, self-reported spousal abuse has were responsible for setting their own targets and been declining across the country, and in 2009, tracking their progress. However, our audit work Ontario’s rate was in line with the national rate. As at the Ministry of Community and Social Services a result, it is not clear whether this co-ordinated indicated that the Ministry had not done this for its effort by the province has made a difference in the own commitments. prevalence of domestic violence. At the time of our audit, no statistics were available to determine whether the prevalence of sexual Domestic Violence Advisory Council violence against women in Ontario has changed as

Chapter 3 • VFM Section 3.10 In addition to the action plans and the strategic a result of the Sexual Violence Action Plan. framework, in 2007 the Minister Responsible for Women’s Issues established the Domestic Violence RECOMMENDATION 1 Advisory Council (Council), which comprises To assess whether the province’s Domestic primarily stakeholders and researchers, to provide Violence Action Plan and Sexual Violence Action advice on how to improve the system of services Plan have reduced domestic and sexual violence to better meet the diverse needs of abused women and improved supports for women who have and their children without incurring any additional experienced violence and their children, the costs for the government. In May 2009, the Council Ontario Women’s Directorate should ensure that released a report that contained 45 recommenda- the commitments contained within the action tions in the following priority areas: government plans have measurable goals or targets attached leadership, access to and equity in delivering VAW to them and that progress is regularly assessed programs and services, education and training for and reported. professionals and the public, child welfare, legal response to violence, and threat assessment and risk management to identify those who are most ONTARIO WOMEN’S DIRECTORATE dangerous to women. Most of the recommenda- RESPONSE tions were directed toward the Ministry of the The Ontario Women’s Directorate acknow- Attorney General and the Directorate. At the time ledges the recommendations made by the Violence Against Women 277

Figure 3: Percentage Rate of Self-reported Spousal Violence, by Province Source of data: Statistics Canada 14 1993 1999 2004 12 2009

0 AB SK MB PEIONNSBCNBQCNLCanada

SHELTER/HOUSING AND COUNSELLING Chapter 3 • VFM Section 3.10 Auditor General of Ontario, and welcomes her SERVICES FOR VICTIMS OF DOMESTIC input on how it can further improve its tracking ABUSE and reporting on initiatives aimed at improving supports for victims and preventing domestic Overview of Service Delivery violence and sexual violence in Ontario. Women who are experiencing domestic abuse (that Domestic violence and sexual violence affect is, abuse by a partner, a significant other or a male women and girls across Ontario. While progress family member) and who wish to leave a violent is being made, there is much more to be done. domestic situation can access community-based The Ontario Women’s Directorate appreci- emergency shelters and crisis support services for ates that measurable goals and targets are themselves and their children. Although women necessary for assessing progress. In response may be referred by health professionals or social to the recommendations, the Ontario Women’s workers, abused women can also access VAW Directorate will work with ministries to services directly. Emergency shelters, which are determine ways to refine existing goals and intended only for women who have experienced targets, and improve the assessment and public domestic abuse and their children, provide safe reporting on progress. temporary lodging and crisis counselling. Staff at 278 2013 Annual Report of the Office of the Auditor General of Ontario

the shelters develop personalized safely plans to under VAW funding. We recommended that the help women stay safe, provide esteem building and Ministry develop such standards and regularly offer crisis counselling. monitor agencies’ performance against them. In Many emergency shelters provide additional 2010, the Ministry created the Resource Guide to services for which they are funded separately—for VAW Shelter Policy and Procedure Development to example, counselling services and transitional and assist VAW shelters in providing consistent high- housing support services, which aim to help connect quality services to women and their children who the shelters’ clients with community supports for access Ministry-funded shelters across the province. the purpose of finding and maintaining housing in This document is intended to be a resource for VAW an effort to live independently and away from their shelters in developing and/or refining their current abuser. Many emergency shelters also provide ser- policies and procedures. It is not intended to be vices to children who have witnessed abuse at home a directive or a set of standards that the Ministry and whose mothers are receiving supports through expects all VAW shelters to meet. The guide encour- the VAW program, to help them heal from the harm- ages service providers to have policies in several ful effects of witnessing violence and thus avoid areas (including governance, admission criteria, the need for more intensive supports in the future. staffing and physical security). However, the guide If shelters do not provide these services in-house, does not indicate what the standards or guidelines they can refer abused women to other agencies should be. Its application is therefore unlikely to for service. Emergency shelters also provide crisis result in consistent province-wide service quality. phone-counselling assistance to abused women who For the three regional offices we visited, we saw are still at home by informing them of their rights no evidence that the Ministry reviewed whether and options and of services available to help them agencies had in fact put policies in place as outlined manage their situations. In contrast, women who in its guide. The Ministry’s position is that each have become homeless for any reason other than agency’s board of directors is responsible for the domestic abuse are typically served in homeless day-to-day management of the shelters, including shelters, which are administered by municipalities. admission criteria, and operating policies, including

Chapter 3 • VFM Section 3.10 Homeless shelters, which may house both men and staffing levels. During our visit to select agencies, we women in the same facility, do not offer the sup- noted that these agencies had established policies ports and services available in emergency shelters. in many areas. However, none of the agencies we In the 2012/13 fiscal year, shelters in Ontario visited had policies on staffing levels represented by that provided temporary accommodation and minimum staff-to-bed ratios, or policies aimed at security to women and children fleeing violence ensuring that women who were referred elsewhere had a total capacity of approximately 2,000 beds. for services due to overcapacity actually received In 2011/12 (the latest year for which occupancy services. Moreover, we noted a number of inad- and length-of-stay information was available at equate and differing policies across the agencies we the time of our audit), the shelters’ average annual visited, a situation that could lead to inconsistent occupancy rate was 82%, and the average length of access to services and that did not permit useful stay at a shelter was one month. comparisons of service-level data. For example: • Admission Criteria: One shelter told us that it accepted women who were homeless but Standards for Service Quality not abused, whereas others indicated that In our 1994 and 2001 audits, we noted that the they accepted only women who were fleeing Ministry had not developed any standards, service domestic violence, as intended by the Min- directives or guidelines for services to be provided istry. Based on the Transition Home Survey Violence Against Women 279

conducted by Statistics Canada to collect data admissions, minimum staffing levels and on residential services for abused women, periodic Canadian Police Information Centre more than 20% of women who sought shelter checks for shelter staff; and in Ontario on April 15, 2010, did so for rea- regularly monitor agencies’ performance sons other than abuse. One shelter accepted • against standards and take appropriate cor- only women with children, whereas most rective action if necessary. shelters accepted abused women with or without children. In the latter case, some agencies MINISTRY RESPONSE would not house a woman without children in the same room as a woman with children, but The Ministry balances the need for VAW shelters others would. The age at which male children to be accountable to the Ministry with the need could not be admitted also varied, ranging for VAW shelters to be reasonably autonomous from 15 to 18. and flexible in carrying out their day-to-day • Exit Criteria: Although most agencies had responsibilities. This includes allowing shelters discharge policies stating that specific to develop their policies and procedures for behaviours—such as violence, drug use or responding to the unique needs of their com- weapons possession—would result in immedi- munities. Each shelter has an independent ate discharge from a shelter, we noted that board of directors that is responsible for the others stated only that women were allowed day-to-day management of the shelter, including to remain in the shelter until housing or other setting admission criteria and policies for resi- alternatives were found in the community. dential operations such as staffing levels. Only one agency had established a length-of- The annual service contract between the Min- stay policy. istry and each agency sets out the requirement • Staff Screening: All agencies we visited for agencies to evaluate the quality of service required that employees undergo a back- delivery according to the objectives set out in the ground check through the Canadian Police service contract. The Ministry will strengthen

Information Centre (CPIC), including Vulner- its monitoring of agencies’ performance against Chapter 3 • VFM Section 3.10 able Sector screening, before being hired. agency-established standards and take appropri- However, only 69% of the employee files we ate action if necessary. reviewed included a CPIC check. As well, The Ministry will also require Canadian three-quarters of the agencies we visited did Police Information Centre checks for shelter not require employees to get an updated CPIC staff every three years. check after being hired. As a result, some CPIC checks on file were more than a decade old.

RECOMMENDATION 2 MONITORING SERVICE DELIVERY AND EXPENDITURES To help ensure that the services provided by The Ministry enters into an annual service contract transfer-payment agencies to abused women with each of its VAW transfer-payment agencies and their children are of an acceptable and that, among other things, outlines the services to be reasonably consistent quality standard, the Min- provided, the amount of annual funding, and the istry of Community and Social Services should: service-level targets to be achieved. Agencies are establish acceptable quality standards for • required to submit quarterly reports that compare shelter services, particularly with regard to actual expenditures and service-level data against 280 2013 Annual Report of the Office of the Auditor General of Ontario

targeted amounts, and provide explanations for under the Transitional and Housing Support significant variances. Program reported finding housing for more Ministry monitoring activities do not include site women than it reported actually serving. inspections of VAW agencies. One-third of the agen- • We noted that the Ministry’s analysis of cies we visited told us that regional ministry staff service-level data was not sufficient to identify had not come to their facility in more than a year. possible service gaps, inefficiencies in service Where regional ministry staff did visit an agency, delivery, and inequities across agencies and it was to help provide clarification on changes to regions. Although agencies that deliver the service-level data requirements or to attend a board same service are required to report on the meeting. Most VAW agencies we visited told us that same types of data, the Ministry does not within the previous year a ministry representative compare the results reported by one agency had attended at least one of their board meetings, to those reported by other agencies. For the which don’t necessarily occur at the shelter. 2011/12 fiscal year, the Ministry’s analysis We reviewed the Ministry’s monitoring activities consisted of totalling reported results by and analyzed the data submitted by agencies for the region and for the province overall—and only 2011/12 fiscal year, and noted the following: for selected types of data. Although the Min- • Although all agencies had submitted quar- istry analyzed the number of service providers terly reports for 2011/12, almost 20% of the by service type, the number of women served quarterly reports we sampled did not contain by service type, and occupancy rates and all required variance explanations. In addi- average length of stay for emergency shelters, tion, we noted that where explanations for it did not analyze the number of women not significant variances were provided, they served (because of either ineligibility or lack often provided little insight into the cause, of capacity) by service type, and the propor- and that the Ministry’s regional staff did little tion of women served who found housing. or no follow-up. • We also found that some of the information • The Ministry does not have adequate pro- collected had limited usefulness. For example,

Chapter 3 • VFM Section 3.10 cedures in place to verify the accuracy or until 2011/12, each emergency shelter was reasonableness of the data received from asked to report the number of women it did agencies—a situation that could lead the Min- not serve (that is, turned away) but was not istry to make decisions based on unreliable asked to report the cause. Therefore, it is data. For instance, we saw no evidence that not known how many women were turned the Ministry spot-checks supporting informa- away because they were ineligible for service tion. We compared a sample of the 2011/12 and how many were eligible and were ultim- data contained in the Ministry’s information ately served by another shelter. Starting in system with records maintained at the agen- 2012/13, each shelter was asked to report the cies we visited and found that 42% of the number of women referred to more appropri- data sampled did not agree with the agencies’ ate services (that is, how many were turned internal records. Moreover, agency staff were away because they were deemed ineligible not able to reconcile the figures. The Ministry for emergency shelter services or because, also does not analyze the service-level data even though eligible, they required services in for reasonableness. As a result, we found that French) and the number of women referred 36% of shelters reported preparing more elsewhere due to service capacity (that safety plans than the number of clients they is, eligible women who were turned away served, and one agency providing services because that shelter was full). However, the Violence Against Women 281

Ministry still does not know whether eligible In addition, in the 2013/14 fiscal year women who were turned away from one the Ministry implemented standardized emergency shelter were ultimately served by expenditure categories that are in line with the another. In another example, occupancy rates Ministry’s Chart of Accounts. The standardiza- at emergency shelters are calculated based tion of accounts allows for cost comparisons on the number of beds occupied. However, a between agencies. This information is currently shelter may be considered full even though uploaded into the Ministry’s centralized infor- all beds are not filled—as would be the case, mation system, which will allow Ministry staff for example, where a woman and her child to develop reports. or children occupy a room with more beds As well, for a number of years, ministry staff than the number of family members. Given have had the ability to generate business reports the existence of such scenarios, shelters may that allow them to identify missing data, data frequently reach capacity, yet occupancy rates anomalies and significant variances reported by would seldom reflect that. agencies. The Ministry will assess the feasibility of developing further procedures, including RECOMMENDATION 3 periodic spot checks at VAW agencies, to ensure To better ensure that the quarterly reporting that data reported by agencies is accurate, con- process for transfer-payment agencies providing sistent and reasonable. services to abused women and their children As improvements are made, the Ministry furnishes sufficient information to enable cost- will continue to provide staff training to ensure effective monitoring of expenditures and service a consistent approach to contract management delivery, the Ministry of Community and Social and analysis of quarterly reporting information. Services should: • require transfer-payment agencies to submit only data that is useful for analyzing service MONITORING QUALITY OF SERVICES costs and gaps in services; and

PROVIDED Chapter 3 • VFM Section 3.10 • develop procedures, such as periodic spot checks of submitted data, to ensure that data Satisfaction Surveys reported by transfer-payment agencies is Since June 2010, the Ministry has been conducting accurate, consistent and reasonable. a client satisfaction survey to assess client perceptions of VAW services. The survey is voluntary and MINISTRY RESPONSE intended for women who have accessed emergency In the 2012/13 fiscal year, the Ministry made shelters (11,600 women in 2011/12), counselling changes to the type of data transfer-payment services (48,000 women in 2011/12) and/or the agencies are required to submit for monitor- services of the Transitional and Housing Support ing purposes. These changes are expected to Program (22,000 in 2011/12). (Since abused enhance the Ministry’s ability to examine value women can access more than one VAW service, for money and explain significant variances. these numbers should not be added to yield the The Ministry will monitor whether the new data total number of women receiving VAW services.) requirements are useful in analyzing service The survey aims to assess whether the programs costs and gaps in service, and will make changes have increased women’s safety, well-being and as necessary. sense of empowerment, as well as increased accessibility and responsiveness of VAW services. Women 282 2013 Annual Report of the Office of the Auditor General of Ontario

can complete the survey anonymously, either online agency, even though the computer system could or in paper format at the various agency locations. generate such reports if requested. As a result, the The Ministry pays agencies an annual total of Ministry could not assess satisfaction with the ser- $430,000 to administer this survey. vices for each agency. The 2011/12 survey results indicated that almost 90% of women who responded to the Program Evaluations survey felt safer, more confident and better able to make decisions and set goals for themselves Contracts require agencies to outline how they will after receiving VAW services. Regarding access to evaluate each VAW program or service they offer. services, 54% of respondents said that the agency However, we found that the evaluation methods had helped them find a safe place to live and 72% varied widely among agencies. For instance, some of respondents said that the agency had helped agencies listed the Ministry’s reporting require- them find other services in their community. When ments as the only method of evaluation (for asked about waiting times, 53% of respondents said example, reporting against service-level targets or that they were able to get help immediately from having clients complete the Ministry’s client satis- the agency, an additional 25% were served within faction survey), whereas others listed more com- a week, and the remaining 22% said that they had prehensive evaluation methods (such as internal waited more than one week. program reviews, focus groups with staff or former Although the survey is available in approxi- clients, and exit surveys). Under the terms of the mately 15 languages, the response rate has been contracts, agencies are not required to submit their low. In both the 2010/11 and 2011/12 fiscal years, program evaluations to the Ministry as evidence only 3,200 surveys were completed. Assuming that that their programs and services have been evalu- a woman completes a separate survey for each VAW ated. In addition, the Ministry has no other proced- service she has accessed in the year, as stipulated ures to ensure that programs and services are being under the Ministry’s service agreements with agen- evaluated as stipulated in the contracts. cies, the client satisfaction survey participation rate For the agencies we visited, where the program

Chapter 3 • VFM Section 3.10 may be as low as 4%. Furthermore, respondents evaluation methods were other than or in addition did not answer every question on the survey, with to the regular reporting requirements, only half the number of responses per question often being provided us with evidence that they completed the only half the total number of survey respondents. In evaluations as stated in their contracts. No agency addition, most responses came from women served had voluntarily submitted any program evaluations by a limited number of agencies. For example, in it had completed, and there was no evidence that the 2011/12, no surveys were completed for 20% of Ministry had requested the evaluations for review agencies, and fewer than 10 surveys were com- in an attempt to assess quality of service, determine pleted for an additional 40% of agencies. These areas for improvement, or highlight best practices agencies were nevertheless paid almost $260,000 that could be shared with other service providers. in total to administer this survey. Therefore, the Most agencies we visited indicated that they survey results may not be representative of the conduct their own satisfaction surveys or exit inter- women served or of the agencies providing VAW views with clients, but two-thirds did not compile services. and analyze the responses to assess satisfaction Because many agencies provide multiple VAW with services and identify areas for improvement. services, it is not always clear to which specific For the two agencies that did compile and analyze service the survey responses pertained. The Min- survey and interview responses, one of the agencies istry also told us that it does not review results by informed us that it provided a summary to its board Violence Against Women 283 of directors, and the other said that it did not do ment as of March 31, 2012, the latest available this and had not been asked to do so by its board. during our audit, and noted that the Ministry had provided funding for only 10% of the identified safety and security issues, regardless of priority Risk Assessments level, but did not know whether the projects had Ontario’s Transfer Payment Accountability direc- been completed, as illustrated in Figure 4. Urgent- tive requires ministries to establish risk criteria for priority items that had not been funded by the assessing the ability of service providers to meet Ministry and could still be outstanding included service-delivery objectives. To this end, the Ministry fire alarm systems and emergency power systems. has developed a risk-assessment questionnaire to be The Ministry informed us that the decision to fund completed by service providers. The Ministry uses a project was based not on its assigned priority level the agencies’ self-assessments to determine their from this assessment, but rather on what the agen- overall risk level. All agencies were assessed in the cies put forward through the annual infrastructure 2011/12 fiscal year, and almost all assessed them- survey. In addition, the Ministry informed us that selves as low risk, yet three-quarters of all agencies it is aware of only those capital projects it funds did not meet their service-level targets as set out directly, and that more safety and security deficien- in their contracts for more than 50% of the pro- cies may have been addressed if they were funded grams and services the agencies were contracted directly by the agencies through their operating to provide. In addition, for a sample of agencies in funds or other sources. Because the Ministry does the three regions visited, we noted that although not perform site inspections, it might not know the all agencies with identified risks or problems had status of safety and security issues at VAW shelters developed action plans for mitigating those risks, until it performs another Building Condition the Ministry had requested verification of corrective Assessment of VAW shelters, which is expected to action taken in only one-third of these cases. occur by March 2019.

RECOMMENDATION 4 Security Assessments To ensure that the services being provided to Chapter 3 • VFM Section 3.10 Providing a safe and secure environment for women abused women and their children are meet- in emergency shelters is paramount in helping them ing their needs and are delivered in a safe and overcome the trauma associated with the violence secure environment, the Ministry of Community they and their children have experienced. and Social Services should: In 2009, the Ministry completed a Building consider ways to increase the response rate Condition Assessment of VAW shelters, which • on the client satisfaction survey, and analyze included looking at facility security measures (such results by the nature of the service being as site surveillance cameras, motion sensor lighting, provided; enclosures for outdoor areas, security windows, require agencies to periodically submit their interior security systems, entrance supervision and • program evaluations for ministry review, door locks). This assessment identified more than and subsequently ensure that areas requiring 500 safety and security items that required atten- attention are corrected and best practices are tion across all VAW shelters. The estimated cost to shared with other service providers; and upgrade or install these security measures totalled implement a plan for correcting significant $10.3 million. Each item that was identified was • safety and security deficiencies identified labelled either low, medium or urgent priority. We in the Ministry’s 2009 Building Condition reviewed the status update on the security assess- Assessment. 284 2013 Annual Report of the Office of the Auditor General of Ontario

Figure 4: Status of Ministry Funding of 2009 Recommended Safety and Security Installations/Upgrades for Emergency Shelters, as of March 31, 2012 Source of data: Ministry of Community and Social Services

Safety and Security Funded by Not Funded % Not Funded Priority Level Deficiencies Ministry by Ministry by Ministry Low 307 18 289 94 Medium 66 9 57 86 Urgent 133 9 124 93 Total 506 36 470 93

MINISTRY RESPONSE MEETING DEMAND FOR SERVICES

The Ministry is currently assessing ways to The Ministry lacks the information that would capture the impact of violence against women allow it to identify the unmet demand for services (VAW) programs on women escaping abuse and in turn to allocate the appropriate resources who may not be willing or able to recount to close the service gap. The two crucial pieces of their experiences. The Ministry will work with information the Ministry needs are: agencies with the expertise on the dynamics of • How many abused women who were eligible violence against women to help make improve- for services did agencies turn away because ments to client-based outcome tools such as the they did not have the space or resources to client satisfaction survey. serve them? The Ministry acknowledges the need to have • Of those women, how many were referred to agencies periodically submit their program other VAW agencies and received the neces- evaluations for ministry review. The Ministry sary help? will ensure that areas requiring attention are The Ministry only tracks the number of women

Chapter 3 • VFM Section 3.10 corrected and, where possible, share best practi- who, whether eligible or not, sought services and ces among agencies. did not initially receive them. In the 2011/12 fiscal The Ministry is developing an Asset Manage- year, that number for emergency shelter services ment Framework to better support decisionwas almost 15,000, or 56% of women who sought making regarding the use of the Ministry’s these services, and the number for VAW counselling limited capital funding. As the basis for the services was more than 3,000, or 6% of women framework, the Ministry will be procuring servi- who sought these services. But because the Min- ces to complete Building Condition Assessments istry does not track which of those women who of all Ministry-funded sites, including VAW were eligible were then referred to other VAW agen- sites, over the next five fiscal years. The Ministry cies and served, it does not know for any given year is determining which sites will be assessed at how many eligible women who sought help were which times, but all VAW sites will be assessed not served. over the life of the project. All of the emergency shelter directors we spoke to said that they try to refer abused women to other emergency shelters first, followed by municipally operated homeless shelters. However, none followed up with the shelter to which they referred each woman to determine whether she had arrived Violence Against Women 285 and received help. To maintain confidentiality, MINISTRY RESPONSE emergency shelter staff told us that they call around to locate a shelter with room, give that shelter’s As part of the improvements made to the address to the woman, and leave it up to the woman Ministry’s reporting requirements in 2012/13, whether or not she goes there. We contacted the the Ministry implemented the requirement for largest municipalities in three regions to determine agencies to track and report both the number of how many abused women were placed in their women who are referred elsewhere for services homeless shelters. Only one of the three municipal- and the number of women waiting for service at ities we contacted maintained information on VAW any point in time during the reporting period. clients served in its homeless shelters. In this region, The Ministry acknowledges the importance of almost 900 VAW clients were accommodated in collecting agency wait lists and will assess the homeless shelters (which are less suitable because feasibility of requiring VAW agencies to collect they do not provide the appropriate supports for this information. abused women and their children). The collection of data must consider the The Ministry does not track information on safety and well-being of abused women and wait-list length and wait times for VAW services. children. The Ministry does not require agen- Only one of the shelters we visited kept a wait list for cies to track whether women have received counselling services. In this case, the wait for family services when they are referred to other agen- counselling was three months. We also reviewed cies for shelters, counselling, and transitional documentation regarding service needs from agen- and housing supports. In the 2008/09 fiscal cies we did not visit and noted a five-month wait year, the Ministry removed the requirement for for long-term counselling at one agency and an service providers delivering the Transitional 18-month wait for individual trauma therapy at and Housing Support Program to track and another agency. report the number of women who had found In 2011/12, cumulative data reported by all and maintained housing for six months. Service agencies indicated that only one-third of women providers informed the Ministry that results

who sought services from the Transitional and would be unreliable and attempting to contact Chapter 3 • VFM Section 3.10 Housing Support Program found housing. Agen- these women could place them at an increased cies told us that social housing is harder to find in risk of violence. metropolitan areas, as was evident from the fact In light of the Auditor’s recommendation, that the length of stay at shelters in metropolitan the Ministry will assess the feasibility of requir- areas was higher than in smaller communities. ing VAW agencies to develop protocols and procedures to determine whether women received RECOMMENDATION 5 services from other service providers.

To better ensure that the service needs of abused women and their children are met, the Ministry of Community and Social Services should: FUNDING require agencies to maintain wait-list infor- • Over the past five years, Ministry funding to mation for their services; and transfer-payment agencies for VAW programs and review the feasibility of implementing a • services increased by 16%, from $122 million in the system to determine whether women who are 2007/08 fiscal year to $142 million in 2011/12, as eligible for VAW services but must be referred shown in Figure 5. This $20 million increase was a elsewhere by an agency, because of capacity direct result of the Domestic Violence Action Plan, issues, actually receive the needed services. 286 2013 Annual Report of the Office of the Auditor General of Ontario

which included increases in base funding for all Figure 5: VAW Transfer Payments, 2007/08–2012/13 VAW programs and services, as well as additional ($ million) funding for existing and new shelter beds and for Source of data: Ministry of Community and Social Services the expansion of francophone services. 145 Although transfer-payment agencies are required to submit an annual budget to secure fund- 140 ing for the following year, we noted that agency 135 funding is generally historically based, with little or no correlation to identified needs or past perform- 130

ance. In particular, we identified the following 125 with respect to the Ministry’s funding of transfer- payment agencies: 120 • The approved budget remained the same 115 for three consecutive years, from 2009/10 110 through 2011/12, for 84% of agencies that 2007/08 2008/09 2009/10 2010/112011/12 2012/13 operated emergency shelters and 93% of An agency’s performance had little impact on agencies that provided counselling services. • the funding it received the following year. For There was little correlation between service- • example, 30% of emergency shelters missed level targets and the amount of annual their service targets for the number of women funding the Ministry approved. For example, to be served by at least 10% in 2010/11, but 50% of emergency shelters that reduced their received the same amount of funding or more service target for the number of women to the following year. be served by at least 10% for 2011/12 were For an individual agency, a permanent change approved either for the same amount of • in its annual funding level occurs mainly when funding or more than in the previous year. there is a change in the programs or services Conversely, 56% of emergency shelters that it provides. During the last decade, most other

Chapter 3 • VFM Section 3.10 increased their service level targets for the changes to an agency’s annual funding level number of women to be served by at least have occurred when there were across-the- 10% for 2011/12 were approved for the same board funding increases. To illustrate, there amount of funding or less than in the previous were annual funding increases to the base year. Fixed costs incurred by shelters prevent budgets of all VAW programs and services immediate changes in funding level. However, from 2004/05 through 2009/10, and annual we noted that agencies with similar targets increases to all agencies’ salaries and wages for the same service received different levels from 2006/07 through 2008/09. As well, of funding. For example, in 2011/12, funding in 2008/09, annual funding was increased for 10-bed shelters ranged from $334,000 to to ensure that each shelter received at least $624,000, even though the agencies at the $30,000 per bed. high and low ends of the range were located in the same region. Funding for a target of 1,200 direct hours of transitional and hous- Reasonableness of Funding Allocation ing support services ranged from $67,000 to At the time of our audit, the Ministry had not done $141,000 per agency. The Ministry has not an analysis to determine whether resources were looked at the variances to determine whether properly allocated across the province to meet they are reasonable. Violence Against Women 287 demand. Ideally, this analysis should be based on RECOMMENDATION 6 the number of women who could not be served by any VAW agency during the year, by region, but To ensure that funding provided to transfer- as previously indicated, the Ministry’s figures are payment agencies is commensurate with the overstated in this area. Therefore, in order to assess value of services provided to abused women whether resources are being distributed equitably, and their children and is properly allocated we compared the distribution of total VAW funding to meet the demand for these services across and shelter beds across the province with the distri- the province, the Ministry of Community and bution of females. Our analysis indicated that the Social Services should periodically compare Central West region (which includes Peel, Dufferin, and analyze agency costs for similar services Wellington, Waterloo and Halton) had about an 8% across the province, investigate significant vari- discrepancy between its share of total VAW funding ances that seem unjustified, and ensure that and the percentage of Ontario’s female population funding is based on the trend in actual service living there (adjusted to reflect the ages of women levels provided. in shelters). The Ministry recognizes that the province has growth regions and that it needs to find a MINISTRY RESPONSE way to shift capacity to meet the demand. In the 2012/13 fiscal year, the Ministry made We also analyzed various unit costs for the three changes to the type of data transfer-payment most significant VAW programs and services in the agencies are required to submit for monitor- 2011/12 fiscal year, and noted a wide variation ing purposes. These changes are expected to in unit costs among agencies for similar services, improve the Ministry’s ability to compare costs as illustrated in Figure 6. We did not note a large between agencies providing similar programs. variance between rural and urban areas, but we did Further changes were made to the financial note that the Eastern region had some of the higher data reporting requirements for the 2013/14 average unit costs, followed by the Northern region. fiscal year. These changes will allow more At the time of out audit, the Ministry had not fol- accurate information on program cost factors

lowed up on these variances. Chapter 3 • VFM Section 3.10 and variances. The Ministry will develop tools and pro- Figure 6: Key Service Costs in VAW Programming, cedures for Ministry staff to use in conducting 2011/12 analyses of agency costs and variances to ensure Source of data: Ministry of Community and Social Services that funding is based on trends. Provincial Variation Among Median ($) Agencies ($) Shelter Costs Costs per person 4,400 1,500–17,500 CO-ORDINATION Annual cost per 39,500 26,400–63,200 The Ministry is responsible for co-ordinating available bed regional service delivery through its nine regional Cost per person per 140 90–575 day of residential care offices. The Ministry requires each regional office Counselling Services to conduct strategic service planning. Regional Cost per person 630 20–3,500 strategic service planning involves bringing together Cost of hour of service 84 20–520 Ministry-funded VAW agencies to discuss service Transitional Housing Support Program issues, best practices, emerging issues and regional Cost per person 730 90–6,800 priorities. For the three regions we visited, we found 288 2013 Annual Report of the Office of the Auditor General of Ontario

that the degree of regional service planning varied. two regions, where committees had provided such a For example, one of the regional offices involved all report, the descriptions of what they did were often agencies in its service-planning activities. Another too general and did not contain quantifiable targets regional office involved only agencies provid- or outcomes. There was no evidence that the Min- ing shelter services, and the third regional office istry had followed up with these committees. involved only agencies from a certain part of the We met with some committee chairs in the region. As a result of these planning activities, one regions we visited and inquired about their activ- region developed a strategic plan but told us that ities. Although most said that they meet monthly due to a lack of funding, it had not taken action to to identify gaps in service, only one reported that address the needs identified. Another region shifted it conducted this activity in its annual report to some annual funding from one agency to another to the Ministry. The most common activity among better meet demand for counselling services. the committees was promoting public awareness. Additional co-ordination activities are also We were also informed that some committees conducted through the Domestic Violence Com- were securing funding from other organizations munity Coordinating Committees, collaboration to conduct research. For example, one committee agreements between VAW service providers and examined policies that inadvertently put women Children’s Aid Societies, and referral agreements who experience violence at increased risk. The between the Transitional and Housing Support resulting report proposed a framework for assessing Program service providers and social housing ser- the determinants of women’s safety and provided a vice providers. map of potential areas of focus for service delivery and policy design. However, the Ministry does not collect or review the research materials generated Domestic Violence Community by the Domestic Violence Community Coordinating Coordinating Committees Committees, even though doing so could inform The Ministry provides an average of $30,000 in decision-making about services and ways to address annual funding to each of the 48 Domestic Violence service gaps and inequities.

Chapter 3 • VFM Section 3.10 Community Coordinating Committees in order to The Ministry informed us that the first two-day strengthen linkages and networks among commun- provincial conference for these committees was ity agencies for the purposes of improving commun- held in November 2011 to facilitate information- ity response to abused women, increase awareness sharing. Forty-two of the 48 committees attended. and prevention, and identify and address gaps in Since the conference, representatives from various VAW services. The committees are typically led by committees have gotten together as a group and a volunteer chair and include representatives from begun attempting to secure funding for the creation various sectors (such as justice and health), as well of a provincial network to support the VAW sector. as from agencies providing VAW services. A website has also been developed where commit- Although the Ministry requires these com- tees can post information, share best practices and mittees to report annually on their activities and stay connected. on what they accomplished for the funding they received, such reporting was inconsistent. To Collaboration with Children’s Aid Societies illustrate, in one of the three regions we visited, none of the committees provided the Ministry with Shelters for abused women with children may need a year-end report that outlined their objectives, to involve a Children’s Aid Society (CAS) in certain deliverables for the year and achievements, as cases, and vice versa. In 2004, the Ministry facili- required under their funding contracts. In the other tated the development of local protocols between Violence Against Women 289

CAS and VAW agencies (both shelters and counsel- In response, the Ministry updated the year-end ling agencies) to identify the situations when the reporting template to be completed by the co-chairs two sectors must involve each other and what of the CAS/VAW collaboration agreement commit- actions should be taken by each. At the time of our tees as part of their annual reporting process to the audit, although there were 46 Children’s Aid Soci- Ministry and, in conjunction with the Ministry of eties in Ontario, only 37 collaboration agreements Children and Youth Services, provided one-time between CAS and VAW agencies were in place in total funding of $200,000 in 2012/13 to support various communities. cross-sectoral initiatives that respond to the needs Over the years, the Ministry was made aware of local communities. With respect to the additional of problems with the collaboration process by funding, we noted that the Ministry required com- such sources as the 2007 Domestic Violence Death mittees to submit proposals that aim to “improve Review Committee report issued by the Office of the collaboration between their sectors, as well as Chief Coroner, the 2009 report from the Domestic service delivery for women who experience vio- Violence Advisory Council, and annual reports lence and their children.” In 2012/13, the Ministry submitted by each CAS/VAW collaboration agree- also prepared a summary of the achievements and ment committee. As a result, in November 2010 the challenges reported by the collaboration agreement Ministry held consultations with representatives committees in their 2010/11 and 2011/12 annual from both the CAS and the VAW sectors to discuss reports. However, according to the summary, many concerns and develop strategies to improve mat- of the concerns identified during the 2010 consulta- ters. Some of the concerns identified included the tions remain, such as a lack of understanding of following: other sectors were required to play a each party’s roles and responsibilities, and in turn role because the needs of women were becoming the need for training to promote a shared under- increasingly complex (for example, mental-health standing of abused women and the agreement. and addiction issues, as well as custody and access issues); links were needed to other children’s Transitional and Housing Support Program services, such as those for mental health; expecta- Referral Agreements tions and requirements for collaboration were not Chapter 3 • VFM Section 3.10 the same for VAW and CAS organizations; current Finding safe and secure housing is important for resources were not adequate to support and nurture women who have left situations involving domestic effective working relationships between the two sec- violence because it allows them to lead independ- tors; and more training was required to promote a ent lives. All 127 Transitional and Housing Support shared understanding of abuse against women. Sug- Program (THSP) providers are expected to have gested actions for improvement included replacing agreements with the 47 local Social Housing Co- the current collaboration agreements with a ordinated Access Centres to help abused women multi-sectoral protocol or collaborative agreements find social housing. These centres, which are oper- with representatives from police, Crown attorneys, ated by local regional or municipal governments probation and parole services and the health sector; and funded by the Ministry of Municipal Affairs developing a common risk assessment process and a and Housing, provide a single point of entry into standardized risk assessment tool to be used by both the local social housing market. At the time of our the CAS and the VAW sectors; providing resources audit, 34 THSP referral agreements were in place. for ongoing cross- or multi-sectoral training on the Therefore, not all communities were covered by application or implementation of any collaboration these arrangements. agreements; and promoting inter-ministerial collab- Victims of domestic violence, whether they are oration and providing inter-ministerial leadership. still living with the abuser or in temporary lodging 290 2013 Annual Report of the Office of the Auditor General of Ontario

such as a VAW shelter, are given priority access to RECOMMENDATION 7 social housing provided they meet specified criteria and submit a written declaration from a community To help improve the co-ordination of service professional (for example, a shelter worker, social delivery for abused women and their children, worker or health-care worker) who confirms their the Ministry of Community and Social Services eligibility for priority status. Eligible applicants are (Ministry) should: placed at the top of the social housing waiting list. • ensure that regional offices undertake effect- The Ministry of Community and Social Ser- ive strategic service planning with agencies vices does not track the number of women who and that the results support the Ministry’s are referred to rent-geared-to-income housing overall goals and priorities; and units designated for victims of domestic violence. • use the annual reports of the Domestic Therefore, it is not aware of how long women in Violence Community Coordinating Commit- the THSP typically wait to receive social housing. tees, and the committees set up to manage Based on an annual survey administered by the the collaboration agreements between Ministry of Municipal Affairs and Housing, in 2011, Children’s Aid Societies and VAW agencies, of the 230,000 people who were waiting for rent- as well as the Transitional and Housing Sup- geared-to-income housing in Ontario, 10,000 (or port Program referral agreements to: 4%) had priority status. During that year, 46% of • summarize the useful information; those with priority status were housed, compared • share the opportunities for service to 8% of those without priority status. According improvements and useful research identi- to the Ministry of Municipal Affairs and Housing, fied; and the average waiting time for social housing for • take corrective action on common issues people with priority status was six months. The identified. Ministry did not have information on waiting times for other groups, but a survey conducted by the MINISTRY RESPONSE Ontario Non-Profit Housing Association that same The Ministry agrees that it needs to ensure that

Chapter 3 • VFM Section 3.10 year found that those housed in 2011—excluding planning activities between regional offices and those in Toronto, which did not report—waited VAW agencies are effective, in order to maintain an average of two to four years, depending on the stability and sustainability of the existing the community. Overall, although the priority system of VAW services and establish priorities applicants made up 4% of the waiting list, they for future system development. The Ministry accounted for almost 24% of the people who will develop a strategic plan that will identify obtained housing. Therefore, the priority policy priorities for improving regional planning was working in getting many abused women and activities and will enhance service system co- their children into housing more quickly. ordination across the VAW sector. As is done with CAS/VAW collaboration agree- In addition, the Ministry will summarize ments, a committee is set up to manage each THSP information from the annual reports submitted referral agreement, and each committee is required by the Domestic Violence Community Coordinat- to report annually to the Ministry on its referral ing Committees and the Children’s Aid Societies/ activities, describing what worked well and what VAW Committees, share useful information did not, and providing resolution strategies. We accordingly and take corrective action where saw no evidence that the Ministry had analyzed the necessary. The Ministry will assess the need for information submitted to identify best practices or continuing the Transitional and Housing Sup- issues to be dealt with systemically. port Program referral agreement reporting. Violence Against Women 291

PERFORMANCE REPORTING AND children. The survey indicated that more than OVERALL EFFECTIVENESS 20% of women who sought shelter on April 15, 2010, did so for reasons other than abuse. The Ministry has set performance measures for all Services that were felt to be needed but not but one of its objectives for the VAW programs and • currently offered, or not offered at the level services. However, the Ministry has not established required to meet the needs of residents, as targets or benchmarks for many of these measures, well as any issues or challenges facing the and does not routinely report results related to shelter or residents, could help the Ministry them. Instead, the Ministry reports activity, such identify service gaps. The survey highlighted as the number of women and children served, the the following service needs: addiction and number of shelters and the number of calls to the mental health counselling, transitional sup- crisis lines. Figure 7 summarizes the objectives, per- port for housing and employment, and pro- formance measures and targets for VAW programs gramming for children. The challenges facing and services, along with the results achieved in the women most frequently reported by the facili- 2011/12 fiscal year (the latest year for which infor- ties included access to affordable long-term mation was available at the time of our audit). housing, lack of services for mental health and The program’s effectiveness cannot be assessed addiction issues, poverty, and access to legal by these measures alone, since three of the five services. Besides the need for more funding, performance measures rely on results from the one of the most frequently mentioned challen- client satisfaction survey. As we noted earlier, these ges facing shelters was the need for more staff results may not be representative of the perceptions training to deal with the increasingly complex of those who access VAW services or the agencies needs of residents. providing VAW services because of the low response The number of women turned away from rate and the fact that few or no surveys were com- • the shelter and the reasons for turning them pleted for 60% of agencies. away could help the Ministry assess its abil- The Ministry may find other information useful ity to meet needs. The survey indicated that to help determine how its services are being used

two-thirds of the women seeking shelter on Chapter 3 • VFM Section 3.10 and whether they are being effective over time and April 15, 2010, were turned away because the in comparison to other jurisdictions. For approxi- shelter was full. mately 20 years, under the federal government’s The number of repeat users could help the Family Violence Initiative, in consultation with • Ministry assess the program’s ability to provincial and territorial governments, Statistics empower women to live free of violence. The Canada has been surveying all residential facilities survey indicated that about 20% of women that provide services to abused women and their residing in shelters on April 15, 2010, had children across Canada and collecting information been at the same shelter before. both on the services provided and on the clientele. The Ministry has access to the public reports This survey, currently called the Transition Home produced by Statistics Canada; but little can be con- Survey, is conducted every other year. A number cluded from those reports, because only selected of the questions, as well as the answers from the information is presented and because that informa- Ontario survey respondents, would provide useful tion may or may not have been included in previous information to the Ministry. For example: reports. The Ministry would be well served to The reasons for seeking shelter that women • request the results of all questions by type of facility reported to the agencies could help the Min- and by province to best identify where improve- istry determine whether emergency shelters ments are needed in Ontario and how it compares are being used for abused women and their to other jurisdictions. 292 2013 Annual Report of the Office of the Auditor General of Ontario

Figure 7: Objectives, Performance Measures, Targets and 2011/12 Results for VAW Programs and Services Source of data: Ministry of Community and Social Services and Office of the Auditor General of Ontario

2011/12 Objectives Performance Measures Targets Results Increase the safety of women who are experiencing • % of women who feel safer 96 87a violence and their children by providing safe shelter, • % of women with a safety plan 100 83b safety plan development and crisis counselling • % of women who sought help and None 81b received service Assist women experiencing violence and their children • % of women who feel more None 87a by enhancing self-esteem and supporting them to confident access resources to live independent of domestic • % of women who gained hope that None 90a violence they could have a better life Enhance the co-ordination of VAW services at the None None N/A community level

a. Source: accumulated results from the 2011 client satisfaction survey. b. Source: calculated by the Office of the Auditor General using 2011/12 service-level data reporting by agencies.

RECOMMENDATION 8 MINISTRY RESPONSE

To assess how effective the Ministry of Com- The Ministry is refining the type of data that munity and Social Services (Ministry) has been transfer-payment agencies are required to report in achieving its objectives for Violence Against to the Ministry on a quarterly basis so that it Women programs and services (see Figure 7), is more consistent, meaningful and reliable. It the Ministry should: will therefore enable the Ministry to measure • establish performance measures for its the performance of programs and services, objective of enhancing the co-ordination of and will support the program planning needs

Chapter 3 • VFM Section 3.10 services, as well as targets for all established of the Ministry and VAW-funded agencies. The performance measures, and regularly report Ministry will reassess the performance measures results related to those measures; and established for the VAW program and the appro- • liaise with Statistics Canada to obtain priateness of reporting results. responses to the biennial Transition Home The Ministry plans to obtain more detailed Survey, by province, and compare pertinent data from the Transition Home Survey and results for Ontario to past performance and other relevant sources to enhance its under- to results in other jurisdictions. standing of violence against women and its assessment of its programs and services, so that it will know how better to meet the diverse needs of women. action isbeingtaken by audited entitiesto address isto helpensure that additionalreporting this chapter.section ofthis Ourobjective inproviding applicable recommendations inthe Committee’s the Accounts of (Committee), we include asummary Standing issuedby Committee the onPublic reports management. by timeasreported recommendations sincethat hasbeentaken to address ofactionthat our status of our value-for-money oninChapter auditsreported 3 recommendations. actions taken by respect management to our with related responses, we follow of status uponthe we rec years after publish the inChapter Tworeports 3ofourAnnualReport. wewhich includewhenwe audit publishthese response recommendation, to a written each broader publicsector toizations inthe provide Crown agenciesask ministries, ofthe andorgan tions inourvalue-for-money and auditreports toIt isourpractice make specific recommenda Chapter 4 Chapter Where hearings on our audits are held and onourauditsare heldand hearings Where Chapter onthe 4provides somebackground 2011 AnnualReport and Reviews Value-for-money Audits 2011 to Follow-up and describes the the and describes ommendations and - - and Related Services on mittee onourspecialreport Com extensiveture. heldby Dueto the the hearings andinany Legisla tohearing subsequent the report the Committee during raised the issuesthat the when future auditsshouldbeconsidered. future auditsandmay impact ourassessmentof onin will bemore fully andreported examined effectively. Thecorrective actionstaken orplanned havecorrective actionsdescribed beenimplemented we cannot provide the ahighlevel that ofassurance work. this Thisisnotwith anaudit,andaccordingly, auditors organization’s alsoassisted internal the Inafew documentation. cases, selected supporting management andreviewies anddiscussionswith of them. for reported about43%of progress substantial with made toward implementing ourrecom mendations we madein2011, isbeing progress activitiesmore fully.Committee’s value-for-money audits.Chapter the 6 describes Committee didnot holdany onour2011 hearings Our follow-up work ofinquir consists primarily We recom for 86%ofthe that are ableto report in both 2012 inboth and2013, the

Ornge Air Ambulance Ornge AirAmbulance mendations, mendations, - - - 293

Chapter 4 Chapter 4 Financial Services Commission of Ontario Section 4.01 Auto Insurance Regulatory Oversight

Follow-up to VFM Section 3.01, 2011 Annual Report

of injury claims in Ontario being about $56,000 Background and five times more than the average injury claim in other provinces, Ontario drivers generally paid much higher premiums than other Canadian driv- The Financial Services Commission of Ontario ers did. However, claims costs in Ontario were also (FSCO), an arm’s-length regulatory agency of the high because Ontario’s coverage provided for one of Ministry of Finance, is responsible for, among other the most comprehensive and highest benefit levels things, regulating the province’s insurance sector. in Canada. FSCO’s auto insurance activities include ruling on We noted in 2011 that the government had applications by private-sector insurance compan- begun taking action to address the high cost of ies for changes in the premium rates that vehicle claims in Ontario. However, we made the following owners pay. FSCO must ensure that proposed observations that outlined some of the challenges premiums are justified based on such factors as an FSCO faced if it was to be more successful in pro- insurance company’s past and anticipated claim actively fulfilling its role of protecting the public costs, expenses and what would be a reasonable interest: expected profit. FSCO also periodically reviews From 2005 to 2010, the total cost of injury the statutory accident benefits available to people • claims under the Statutory Accident Benefits injured in auto accidents, and it provides dispute Schedule (SABS) rose 150% even though the resolution services to settle disagreements between number of injury claims in the same period insurers and injured people about entitlement to increased by only 30%. Benefit payments statutory accident benefits. rose the most in the Greater Toronto Area, In our 2011 Annual Report, we noted that the where drivers also generally paid much higher

Chapter 4 • Follow-up Section 4.01 government must balance the need for a finan- premiums. cially stable auto insurance sector with ensuring FSCO had not routinely obtained assurances that consumers pay affordable and reasonable • from insurance companies that they have paid premiums and receive fair and timely benefits and the proper amounts for claims or that they compensation after an accident. Claims payments have handled claims judiciously. Without such are the largest driver of the cost of auto insurance assurances, there was a risk that consumers premiums and, in 2010/11, with the average cost

294 Auto Insurance Regulatory Oversight 295

would not be treated fairly or that unnecessarily high payouts could help insurers obtain Status of Actions Taken on FSCO approval for higher premium increases. Recommendations • Industry estimates pegged the value of auto insurance fraud in Ontario at between 10% FSCO has made progress in addressing most of our and 15% of the value of 2010 premiums, or recommendations, with significant progress made as much as $1.3 billion. Ontario did not have on several. FSCO was in various stages of imple- significant measures in place to combat fraud, menting changes to help address our recommenda- and the government and FSCO were awaiting tions covering the high cost of auto insurance claims the recommendations of a government- and premiums, auto insurance fraud, the process for appointed anti-fraud task force expected in reviewing insurers’ rates filings and their approv- fall 2012. als, a backlog in its dispute resolution services, In approving premium rates for individual • and oversight of how well insurers complied with insurance companies, FSCO allowed insurers requirements regarding the processing of claims a reasonable rate of return on equity—set at and ensuring approved rates are used. Although 12% in 1996, based on a 1988 benchmark average injury claim costs had declined significantly long-term bond rate of 10%. However, that since 2010, at the time of our follow-up average benchmark had not been adjusted downward, automobile insurance premiums had not. In addi- even though the long-term bond rate had tion, discussions held to date had not resulted in any been about 3% at the time. Furthermore, increase to the amount recovered from auto insurers FSCO needed to improve its documentation for health-system costs incurred to care for people to demonstrate that it treats all insurers’ injured in motor-vehicle accidents. premium-rate-change requests consistently The status of the actions taken is summarized and that its approvals are just and reasonable. following each recommendation. • FSCO’s mediation service was backlogged to the point that resolution of disputes between claimants and insurers was taking 10 to 12 STATUTORY ACCIDENT BENEFITS months, rather than the legislated 60 days. CLAIMS COSTS The Motor Vehicle Accident Claims Fund, • Recommendation 1 administered by FSCO to compensate people In order to ensure that the Financial Services Com- injured in auto accidents when there is no mission of Ontario (FSCO) can effectively monitor insurer to cover the claim, had $109 million Ontario’s auto insurance industry, particularly less in assets as of March 31, 2011, than it claims costs and premiums, and recommend timely needed to satisfy the estimated lifetime costs corrective action to the Minister of Finance when war- of all claims currently in the system. This ranted, FSCO should: unfunded liability was expected to triple by implement regular interim reviews of the Statu-

the 2021/22 fiscal year unless, for instance, • Chapter 4 • Follow-up Section 4.01 tory Accident Benefits Schedule to monitor the $15 fee currently added to every driver’s trends such as unexpected escalating claims licence renewal is doubled. costs and premiums between the legislated five- We made a number of recommendations for year reviews, in order to take appropriate action improvement and received commitments from earlier, if warranted; FSCO that it would take action to address our monitor ongoing compliance with the interim concerns. • Minor Injury Guideline, expedite the work to 296 2013 Annual Report of the Office of the Auditor General of Ontario

develop evidence-based treatment protocols for As part of the 2010 auto insurance reforms, minor injuries, and identify and address any FSCO introduced an interim Minor Injury Guideline lack of clarity in its definitions of injuries; to provide a broader definition of minor injuries, • implement its plans as soon as possible to obtain as well as a $3,500 minor-injuries benefit limit on assurance that insurance companies are judi- the cost of all treatment services and assessments ciously administering accident claims in a fair combined. As of November 2012, FSCO changed and timely manner; and the form used by health-care providers so that it • examine cost-containment strategies and benefit now requires additional information about whether levels in other provinces to determine which the treatment is covered by the Guideline. could be applied in Ontario to control this prov- In July 2012, FSCO retained the consulting ince’s relatively high claims costs and premiums. services of medical and scientific experts who have been working to develop an evidence-based treat- Status ment protocol for the most common injuries from Under the Insurance Act (Act), the Superintendent motor-vehicle accidents. The treatment protocol, of Financial Services (Superintendent) is required if approved by government, could be incorporated to conduct a comprehensive review of the effect- into a Superintendent’s Guideline and used by iveness and administration of auto insurance at insurers and health-care providers when treating least every five years and make recommendations minor injuries resulting from automobile accidents. for improvement to the Minister of Finance. In The protocol will help to reduce disputes in the 2008, FSCO undertook the first statutory five-year auto insurance system and ensure motor-vehicle- review, which led to a report to the Minister of accident victims receive effective, scientifically Finance and to legislative changes in September proven treatment. This is a two-year project. The 2010 to the Statutory Accident Benefits Schedule consultants provide regular updates to the Super- (SABS), a regulation under the Act. In addition to intendent and, as directed in the 2013 Budget, the five-year review, FSCO is required to conduct a FSCO will provide an interim report this year on legislated review every three years of the risk-clas- the progress of the project. We were informed that sification and rate-determination regulations. As the interim Minor Injury Guideline will be assessed well, FSCO participates in a review of the adequacy upon completion of the consultants’ report and will of the SABS every two years. be addressed as part of a future comprehensive On August 16, 2013, the government proclaimed statutory review. legislative changes to consolidate multiple auto In summer 2011, FSCO introduced a new insurance reviews, including the former five-year annual requirement that each insurance company review of auto insurance, the three-year review provide it with a statement from its chief executive of risk-classification regulations and the two-year officer attesting that it had controls, procedures review of the SABS. The new consolidated review of and processes in place to ensure compliance with the auto insurance system will be initiated at least legislative requirements for the payment and once every three years, beginning in 2013.

Chapter 4 • Follow-up Section 4.01 handling of claims. In 2012, on a risk basis, FSCO As part of the two-year SABS review, FSCO conducted on-site examinations of 14 auto insur- provided in December 2012 to the Minister of ance companies representing 46% of the market Finance a report that analyzed the impact of the share and issued a summary report to the industry 2010 reforms and the adequacy of accident bene- outlining the results of this process and identifying fits, including showing that 2011 accident benefits areas for improvement. FSCO expected to have claims costs had decreased following the reforms. visited 16 more companies by August 2013. In 2012, FSCO also introduced a requirement for Ontario Auto Insurance Regulatory Oversight 297 automobile insurers to periodically complete a FRAUD IN AUTO INSURANCE SABS control questionnaire covering claims hand- Recommendation 2 ling, including new questions about treatments To reduce the number of fraudulent claims in covered by the interim Minor Injury Guideline. Ontario’s auto insurance industry and thereby protect FSCO expected that all insurers would be examined the public from unduly high insurance premiums, the within a four-year cycle to verify the responses and Financial Services Commission of Ontario (FSCO) examine insurers’ practices. should use its regulatory and oversight powers to: In addition, new regulations came into force help identify potential measures to combat on January 1, 2013, that provided FSCO with the • fraud, including those recommended by the power to impose administrative fines on insurers Insurance Bureau of Canada and those in effect for not complying with legislative and approval in other jurisdictions, assess their applicability requirements. and relevance to Ontario, and, when appropri- As part of its ongoing policy-development work, ate, provide advice and assistance to the govern- FSCO gathered information through the Canadian ment for their timely implementation; and Council of Insurance Regulators on benefit levels ensure development as soon as possible of an and coverage available in other provinces in an • overall anti-fraud strategy that spells out the effort to identify cost containment strategies that roles and responsibilities of all stakeholders— could be applied to Ontario. A draft summary and the government, FSCO, and insurance compan- analysis was prepared in March 2013, and we were ies—in combatting auto insurance fraud. advised that an updated version would be used for the 2013 review. Status As a result of changes to the SABS in September In 2011, the government appointed the Ontario 2010, the auto insurance industry reports that Auto Insurance Anti-Fraud Task Force (AFTF) to Ontario’s average injury claim cost has decreased determine the scope and nature of automobile more than 50%, from about $56,000 in 2010 to insurance fraud and make recommendations about $27,000 in 2012. The difference between Ontario’s ways to reduce it. As part of the AFTF, the Ministry average injury claim costs and those paid by other of Finance retained consultants to provide research provinces has narrowed, although Ontario’s costs about how other jurisdictions combat fraud, analy- now stand at approximately three times higher sis of the potential range of fraud in Ontario’s auto than those of other provinces. However, lower insurance system and advice on the regulation of accident benefit claim costs have not yet resulted health-care facilities. FSCO actively supported the in corresponding lower average premiums paid in AFTF, including chairing its Regulatory Practices Ontario, where the average premium was $1,551 Working Group and preparing a status report in in 2012, or 8% higher than in 2010, and still the June 2012. highest in the country. In its November 2012 final report, the AFTF In August 2013, the government introduced a said auto insurance fraud was substantial and had

number of initiatives as part of a strategy to reduce a material impact on auto insurance premiums. Chapter 4 • Follow-up Section 4.01 average auto insurance rates by a target of 15%. Estimates of the total amount of fraud ranged Since the passing of legislation in August 2013 from $768 million to $1.56 billion in 2010, which that gave FSCO the authority to order insurance amounts to between $116 and $236 per average companies to file rates, FSCO has required certain premium paid in Ontario in that year. The AFTF insurers to submit detailed actuarial filings so it made 38 recommendations that form an inte- can review claims costs and rates to ensure they grated anti-fraud strategy focused on prevention, are reasonable. detection, investigation and enforcement, along 298 2013 Annual Report of the Office of the Auditor General of Ontario

with enhanced and clearer regulatory roles and industry-created Health Claims for Auto Insurance responsibilities. (HCAI) system mandatory. HCAI is an online data- FSCO and the Ministry of Finance established a base and billing portal to which health-care provid- joint working group to consider the AFTF recom- ers are required to submit billings for injury claims mendations, and action had already been taken before they are forwarded to insurers for payment. to implement several of them. In January 2013, In addition to its role of transferring electronic the government announced regulation changes to documents, HCAI is also a source of valuable data enhance accountability in the auto insurance sec- with the potential to identify fraudulent patterns tor, and FSCO issued an accompanying bulletin to among both providers and claimants. The HCAI support these changes. New regulations came into Anti-Fraud working group piloted three initiatives force on June 1, 2013, which, among other things: to develop electronic tools to assist health-care ser- • require insurers to provide claimants with all vice providers and insurers to identify fraudulent the reasons for which a medical or rehabilita- activity in the system. tion claim was denied; require insurers to itemize expenses in a • RATES FILINGS AND APPROVALS bi-monthly statement to claimants of medical- rehabilitation benefits paid out on a claimant’s Recommendation 3 behalf; To ensure that the Financial Services Commission of • increase the role of claimants in preventing Ontario (FSCO) fairly and consistently authorizes fraud by requiring them to confirm their auto insurance company premium rate changes while receipt of treatment, goods or other services; protecting consumers, FSCO should: and • update and document its policies and proced- • make third-party service providers subject to ures for making rate decisions—particularly sanctions for overcharging insurers for goods for applications that differ from its own assess- and services, and prohibiting them from ask- ments—and for properly assessing rate changes ing consumers to sign blank claim forms. in light of actual financial solvency concerns of Ontario’s 2013 budget proposed to expand and insurance companies; modernize the Superintendent’s investigation and • review what constitutes a reasonable profit mar- enforcement authority (particularly in the area gin for insurance companies when approving of fraud prevention) and give FSCO authority to rate changes, and periodically revise its current license and oversee business practices of health assessment to reflect significant changes; and clinics and practitioners who invoice auto insur- • establish processes for verifying or obtaining ers. These changes to the Act were proclaimed in assurance that insurers actually charge only the August 2013. authorized rates. In January 2013, FSCO launched a project to Status internally review closed mediation files to help FSCO updated its policies and procedures for pro-

Chapter 4 • Follow-up Section 4.01 identify systemic issues that may, in turn, identify cessing and approving rate applications effective patterns of fraudulent behaviour in the mediation May 2012 and told us it had provided staff training system. FSCO told us it was also working with on these new procedures. Rate decisions were stakeholders to develop a consumer engagement based on a defined range that was acceptable when and education strategy, and it launched an anti- a proposed rate differed from the FSCO actuarial fraud hotline in June 2013. service’s assessments. Staff were required to pre- In February 2011, to help streamline the pare an internal briefing note when a difference claims-handling process, FSCO made usage of the Auto Insurance Regulatory Oversight 299 greater than the acceptable range was considered processes and controls that insurers had put into justified. The briefing note was to be reviewed place to ensure they complied with legislative with the Superintendent and the information in it requirements and FSCO-approved rates. FSCO documented in the Rates and Classification Report conducted on-site examinations of insurers during before the filing could be approved. 2012 and 2013, including verification of the degree In October 2012, FSCO retained a consultant to to which it could rely on the information provided review the reasonable profit margin rate that had in a company’s written confirmation and completed been established for auto insurance rate filings, questionnaire, as applicable, and to confirm that including a financial assessment and consultation identified controls were in place and operating with the auto insurance industry. In the final report, effectively. All insurers were to undergo this scru- the consultant recommended that FSCO should tiny at least every four years. consider moving to either a five-year or 10-year rolling average for a return-on-equity benchmark rate. DISPUTE RESOLUTION SERVICES In August 2013, FSCO decided that an eight-year rolling average for a return-on-equity benchmark Recommendation 4 rate would be used going forward. According to To ensure that the Financial Services Commission FSCO, the new methodology generated an 11% of Ontario meets its mandate to provide fair, timely, return-on-equity benchmark for 2013. In addition, accessible, and cost-effective processes for resolving FSCO adopted another benchmark that assesses the disputes over statutory accident benefits, it should: insurer’s premium-to-equity ratio that is consistent • improve its information-gathering to help with federal solvency and capital requirements. explain why almost half of all injury claim- FSCO also has begun a review of the feasibility of ants seek mediation, as well as how disputes moving to a return-on-premium approach, which it are resolved, and to identify possible systemic expects may be relatively more simple and transpar- problems with its SABS benefits policies that can ent than the return-on-equity benchmark. be changed or clarified to help prevent disputes; Since 2012, FSCO has required that the chief and executive officer of an auto insurance company • establish an action plan and timetable for annually attest in writing that it provided auto reducing its current and growing backlog to a insurance in Ontario in accordance with approved point where it can provide mediation services in rates, risk classification systems and underwriting a timely manner in accordance with legislation rules. The Act prescribes the many rules of conduct and established service standards. with which these companies must comply in doing Status their automobile insurance business in Ontario, The government announced in its 2012 and 2013 including having their rates filed with and approved Budgets that a review of the auto insurance dispute by FSCO. resolution system would take place. At the time of FSCO implemented a new annual requirement our follow-up, FSCO was completing an internal for insurance companies to attest that they had Chapter 4 • Follow-up Section 4.01 examination on closed mediation cases and the independent audit processes in place to confirm corresponding insurers’ claims files to gather that approved rates were charged by the insurer. information on the reasons for the high number of These attestations from insurers were due by claimants who were seeking mediation and how October 15, 2013. In addition, in spring 2013 these disputes were resolved. The results of the FSCO began sending out detailed rate verification examination were to be used for the review and questionnaires to auto insurers—some randomly for stakeholder consultations to help identify any selected and some targeted—covering governance 300 2013 Annual Report of the Office of the Auditor General of Ontario

systemic issues that were creating disputes with auto insurance oversight and regulatory activities, the current SABS legislation and policy. In August the Financial Services Commission of Ontario should 2013, the government announced the appointment report timely information on its performance, includ- of an expert to undertake the review and make rec- ing outcome-based measures and targets that more ommendations on transforming the current system. appropriately represent its key regulatory activities An interim report was due in fall 2013 and a final and results. report by the end of February 2014. Status To address FSCO’s growing backlog of cases During the 2012/13 fiscal year, FSCO finalized its involving disputes between insurers and claimants corporate Performance Management Framework on the payment of statutory accident benefits, that details for each of its divisions, including auto Treasury Board approved in December 2011 insurance, a set of performance measures and FSCO’s request for an additional $38.2 million over targets that link to its long-term goals and strategic three years to hire a private dispute-resolution priorities. For example, FSCO’s auto insurance service provider to supplement FSCO’s own staff. performance measures include targets for industry According to FSCO, backlogged mediation cases compliance with SABS benefits and approved auto- were being assigned to the service provider at the mobile insurance premium rates. We were advised rate of 2,000 files per month. New applications that the system has been modified to track the data received on or after November 29, 2012, were needed for reporting on the performance measures, being assigned to FSCO mediators within a couple and that FSCO would report on the measures for of days. On March 31, 2012, there were about the 2013/14 fiscal year in its annual report. The 29,000 cases awaiting assignment. With this con- Performance Management Framework was posted tract help, and with new software that has made on FSCO’s website. mediation scheduling more efficient, all mediation In addition, in June 2012, FSCO posted on its files had been assigned as of August 19, 2013, and website new standards for its turnaround time for the backlog had been eliminated. approving insurers’ filings for private passenger In addition, FSCO had experienced a decrease auto insurance rates and risk classification changes. in the number of applications for mediation The performance results for 2012/13 were posted received each month. In 2012/13, FSCO received on the FSCO website in June 2013. approximately 25,300 new applications for media- As of July 2013, FSCO continued to experience tion, a 29% decrease from the 35,700 applications delays in releasing its annual report to the public, received in 2011/12. FSCO indicated that this and the latest annual report available to the public decrease was likely due to the September 2010 was for the 2009/10 fiscal year. FSCO advised us legislative changes to the SABS that helped reduce that the 2010/11 and 2011/12 annual reports were the number of disputes, as well as the auto insur- submitted to the Minister of Finance, tabled in the ance industry’s increased focus on fraud. FSCO Legislature by the Minister on October 3, 2013, and informed us that with the decreased volume and published on FSCO’s website that same month. It

Chapter 4 • Follow-up Section 4.01 the reduced backlog, mediators could handle new also noted that it had made changes to its internal cases within the prescribed 60-day time limit. processes and it expected the 2012/13 annual report to be delivered to the Minister by Novem- PERFORMANCE MEASURES ber 29, 2013. Under its enabling legislation, FSCO is required Recommendation 5 to publish by June 30 of each year a Statement of In order to provide the public, consumers, stakehold- Priorities setting out its proposed priorities and ers, and insurers with meaningful information on its Auto Insurance Regulatory Oversight 301 planned initiatives for the coming year, and the bankrupt debtors. FSCO noted that any changes to reasons for adopting these priorities. The Statement funding would require amendments to regulations of Priorities includes a report-back section listing and to the existing Motor Vehicle Accident Claims FSCO’s key accomplishments in the previous year. Fund fee on issue or renewal of an Ontario driver’s We noted that the statement for 2013 was available licence, which are the responsibilities of the Min- on its website and included the key auto insurance istry of Finance and the Ministry of Transportation. reforms FSCO was working on, including efforts to increase oversight of insurers, reduce fraud in the OTHER MATTER industry, control claims costs and premiums, and resolve statutory accident benefit disputes backlogs. Assessment of Health-system Costs Recommendation 7 MOTOR VEHICLE ACCIDENT CLAIMS In view of the fact that it has been five years since FUND UNFUNDED LIABILITY the last review of the assessment of health-system costs owed by the auto insurance sector despite the Recommendation 6 significant increase in health-care costs related to To ensure that the Motor Vehicle Accident Claims automobile accidents over the same period, the Finan- Fund (Fund) is sustainable over the long term and cial Services Commission of Ontario should work with able to meet its future financial obligations, the the Ministry of Finance, the Ministry of Health and Financial Services Commission of Ontario should Long-Term Care, and the insurance industry to review establish a strategy and timetable for eliminating the the adequacy of the current assessment amount. Fund’s growing unfunded liability over a reasonable time period and seek government approval to imple- Status ment this plan. The Insurance Act requires all automobile insurers operating in Ontario to pay an annual “assess- Status ment of health-system costs” to recover the costs We were advised by FSCO that, while no changes to the province of providing medical care to had been made to address the unfunded liability of people injured in motor-vehicle accidents. FSCO the Fund, FSCO continues to formally monitor the is responsible for collecting the assessment from status of the Fund, and ongoing Ontario automobile insurers, with each insurer paying a pro-rated share insurance reforms have had a positive impact on of the total. The assessment has not been changed the Fund’s unfunded liability. The Fund’s actuarial since 2006, when it was set at $142 million, even report shows that the unfunded liability was about though, as we reported in 2011, overall health-care $99 million as of March 31, 2013, or about $10 mil- spending and medically related SABS benefits costs lion less than at March 31, 2011. FSCO’s consulting substantially increased since 2006. actuary recently estimated that the Fund will have We were advised that the Ministry of Finance sufficient funds to meet its financial obligations is undertaking to review the current assessment through to the 2020/21 fiscal year. The updated

amount, as noted in the Minister’s August 24, 2013, Chapter 4 • Follow-up Section 4.01 cash-flow analysis was completed in fall 2013, policy statement. following a recent legal decision that will affect the collectability of accounts receivable owed by Chapter 4 Ontario Energy Board Section 4.02 Electricity Sector— Regulatory Oversight

Follow-up to VFM Section 3.02, 2011 Annual Report

However, a number of factors limited the Background Board’s ability to perform these duties to the extent that consumers and the electricity sector might have expected. The Ontario Energy Board (Board) is charged with In our 2011 Annual Report, some of our more overseeing the electricity sector, which provides significant observations were as follows: an essential commodity while operating as a near- The criterion that electricity bills be just and monopoly. The Board is responsible for protecting • reasonable applies only to areas over which the interests of Ontario’s 4.7 million electricity cus- the Board has jurisdiction—only about half of tomers, and for helping to see that the sector is run the total charges on a typical bill. The Board efficiently and cost-effectively, and that it remains can set rates only for the nuclear power and sustainable and financially viable. some of the hydro power produced by Ontario At the time of our follow-up, in May 2013, the Power Generation (OPG), along with trans- Board had about 170 staff and its operating costs mission, distribution and certain other char- for the 2012/13 fiscal year were around $36 million ges. The other half of a typical bill is based on ($35 million in 2010/11), all of which are paid by government policy decisions over which the the entities that it regulates. The Board sets prices Board has no say, and these costs are not sub- for electricity and its delivery, monitors electricity ject to Board oversight. This includes the 50% markets, and approves the administrative costs of of the electricity sold to residential customers the Ontario Power Authority and the Independent that comes from other electricity suppliers Electricity System Operator. and that, in total, constitutes 65% of the cost At the time of our 2011 audit, electricity prices of the electricity component of the typical bill.

Chapter 4 • Follow-up Section 4.02 for the average Ontario consumer had risen about Consumers can purchase electricity through 65% since the restructuring of the electricity sector • their utility at the Regulated Price Plan prices in 1999, and prices were projected to rise another set by the Board or through an electricity 46% by 2015. In light of this, the Board’s role of retailer that sets its own price. As of May 2013, protecting consumers while setting rates that would about 7% of residential customers had signed provide a reasonable rate of return for the industry fixed-price contracts with electricity retailers. was all the more important. These consumers could be paying 35% to 65%

302 Electricity Sector—Regulatory Oversight 303

more for their electricity than they would pay had they not signed those contracts. In the Status of Action Taken on last five years, the Board has received 16,200 Recommendations complaints from the public, the overwhelming majority of them about electricity retailers. Substantial progress has been made on imple- Issues included misrepresentation by sales menting almost all of the recommendations we agents and forgery of signatures on contracts. made in our 2011 Annual Report. For example, the Although the Board follows up on complaints, Board completed an internal review of its current it has taken only a limited number of enforce- processes as we had recommended, examining ment actions against retailers. things such as rate-setting, reporting, and com- In areas in which it has jurisdiction, the Board • munications with ratepayers and industry partici- sets rates using a quasi-judicial process that pants. Information available on its website has been requires utilities and other regulated entities, improved as a result. The Board also has engaged such as OPG and Hydro One, to justify any consultants to assist in its process review. This proposed rate changes at a public hearing. review has yielded a number of additional recom- Many small and mid-sized utilities say the cost mendations (to, for example, establish a standard of this process—$100,000 to $250,000 per process for rate applications with the necessary application—can be as much as half the rev- controls to minimize the instances of deviations and enue increase sought in the first place. These exceptions), and the Board has developed action costs, generally incurred every four years, are plans to address these. Some changes have already recovered from consumers. been implemented and more are planned through- Individuals or organizations wishing to • out the 2013/14 fiscal year. participate in the hearings on behalf of con- The status of the actions that the Board had sumers can obtain intervenor status, and can taken at the time of our follow-up is summarized qualify for reimbursement of their expenses. after each recommendation. However, many of the utilities and other regulated entities that have to reimburse the intervenors say the number of requests that CHARGES SUBJECT TO REGULATORY they receive can be onerous, the cost of pro- OVERSIGHT viding detailed information to the intervenors Recommendation 1 is high, and they want the Board to better To enhance the cost-effectiveness of its rate-setting manage this process. process, the Ontario Energy Board should: We made a number of recommendations for work with the regulated entities to address their improvement and received commitments from • concerns about the cost and complexity of the the Board that it would take action to address our current rate-setting filing requirements and the concerns. impact on their operations; and • better co-ordinate and evaluate intervenor Chapter 4 • Follow-up Section 4.02 participation in the rate-setting process in an effort to reduce duplication and time spent on lower-priority issues.

Status The Board has taken action to substantially address both parts of this recommendation. 304 2013 Annual Report of the Office of the Auditor General of Ontario

The Board completed a comprehensive review of CHARGES NOT SUBJECT TO its rate-application processes with a view to improv- REGULATORY OVERSIGHT ing their effectiveness and efficiency. The review Recommendation 2 considered the respective roles of staff, Board mem- To help ensure that the interests of consumers are bers, applicants and intervenors. protected with respect to those charges not subject to During the 2012/13 fiscal year, the Board Ontario Energy Board (Board) oversight and regula- identified a need for additional improvements to its tion, the Board should: rate-application process and engaged a consultant encourage the Ministry of Energy (Ministry) to assist with a review. The consultant met with • and the Ontario Power Authority (OPA) to con- groups of regulated entities, intervenors, Board sult with it on a more timely basis with respect members and staff to identify the most pressing to the interests of consumers in all energy-supply issues. The consultant’s recommendations were and pricing undertakings by the Ministry and delivered in October 2012 and the Board has the OPA; developed an action plan to address them. work more proactively with the Independent Some initiatives the Board has under way that • Electricity System Operator (IESO) to address specifically address the concerns we raised in our the high-priority recommendations from the 2011 Annual Report include the following: Market Surveillance Panel (MSP); and The Board created a checklist to help rate- • clearly explain the reason for each charge on applicants ensure that electricity cost-of- • consumer power bills, identify the entity receiv- service applications are complete when they ing the proceeds from each charge, and disclose are filed. Having properly completed applica- whether the Board has any oversight role relat- tions at the time of hearings should reduce the ing to the charge. number of interrogatories and other rounds of discovery. The Board was also working on Status amendments to its filing requirements that All three aspects of this recommendation have been would make clearer what must be filed and substantially implemented. to eliminate filing requirements that are not The Board has been meeting with the Ministry material. The revised filing requirements were on a monthly basis and with the IESO and OPA scheduled to be released in late 2013. on a quarterly basis to review issues of common • The Board has been testing a number of dif- interest, including all energy supply and pricing ferent approaches to the discovery process for undertakings of interest to consumers, and to share rate applications to help it determine the most ideas and perspectives on energy supply and related efficient processes to use under different sets issues. In addition, the IESO and OPA are included of circumstances. For example, it completed on several Board-sponsored working groups and a pilot project where Board staff file inter- other forums where their participation has been rogatories first, then responses are received deemed appropriate.

Chapter 4 • Follow-up Section 4.02 from the applicants, and then intervenors file In 2011, the Board began a correspondence with interrogatories. It found that this approach is the IESO regarding the recommendations the MSP most appropriate when there are significant made in its report. It requested and received in technical issues on which Board staff would writing the following information from the IESO: be taking the lead. Another pilot tested having • steps the IESO intends to take in response to Board staff submit their interrogatories once any recommendations made to it in the MSP the applicant has provided a response to inter- report; venors’ interrogatories. Electricity Sector—Regulatory Oversight 305

• estimated timelines for completion of those it reviewed its current communications strategy, steps; and examining best practices in the areas of regula- • whether, in the IESO’s view, any actions or tory and adjudicative communication, consumer market rule amendments beyond those noted education and engagement, and internal processes in the MSP’s report should be taken. for dealing with consumer inquiries and feedback. The Board has undertaken a similar cor- The Board also engaged a consultant to conduct respondence with the OPA regarding any MSP a review of its communications role as well as its recommendations directed at that agency. Board external and internal communications tools and correspondence with the IESO and the OPA practices. The consultant delivered recommenda- regarding MSP reports and recommendations is tions in January 2013 and the Board developed available on the Board’s website. an action plan to address them. The Board imple- The Board has also taken steps to better educate mented the first set of recommended changes consumers about the charges on their electricity during the 2012/13 fiscal year, adopting plain bills. Following our recommendation, it has language for external communications and a visual updated the explanation for each line item on the storytelling approach to explain complex concepts. sample electricity bill on its consumer website to The Board expected many of the remaining changes include the reason for the charge, the entity receiv- to its communications approach to be implemented ing the proceeds from the charge, and whether the during 2013/14, though some changes, such as the Board has any oversight role relating to the charge. redesign of the website, may take longer. The Board also has made substantial progress in addressing our recommendation that it consider CONSUMER PROTECTION limited proactive compliance reviews focusing Recommendation 3 on high-risk areas. It engaged a consultant in To ensure that consumers are protected and that they September 2012 to support the development and have the information they need to understand their implementation of a risk-based approach to compli- electricity bills, the Ontario Energy Board should: ance intended to increase consumer confidence • review its current educational and communi- by ensuring that retailers and marketers are fol- cation programs and make the appropriate lowing customer service and consumer protection adjustments to meet consumer information rules. Under this approach, the Board should, for needs; example, develop key performance indicators to • consider initiating limited proactive compliance ensure that actions taken to combat non-compli- reviews focusing on high-risk areas; ance in areas of high-priority risks are effective. • work with utilities to streamline reporting The Board was also developing a compliance plan requirements, including the timing and fre- outlining initiatives for 2013/14 based on the high- quency of reporting; and priority risks that were identified in the risk assess- • determine whether appropriate deterrent ment. Examples of the initiatives include certificate

actions in those areas that have generated inspections, in-person sales inspections and review Chapter 4 • Follow-up Section 4.02 frequent legitimate consumer complaints can be of marketing materials. It expected to complete the implemented. compliance plan by the end of 2013. The Board has substantially implemented Status our recommendation that it work with utilities The Board has completed its review and is in the to streamline reporting requirements. In 2012 it process of implementing appropriate adjustments completed a review of its reporting and record- to its consumer information materials. In 2012, keeping requirements for electricity distributors, 306 2013 Annual Report of the Office of the Auditor General of Ontario

which looked at possible ways to make the process After the Energy Consumer Protection Act came more efficient by reducing the amount of data that into force in January 2011, the Board completed distributors are required to file with the Board. The inspections of all active retailers and marketers review considered issues such as the timing and fre- in 2011 and 2012 to assess their compliance with quency of reporting, areas of potential redundancy, applicable consumer protection rules. Where those and areas needing clarification. In December 2012, inspections identified instances of non-compliance, the Board implemented a number of amendments the Board undertook enforcement action, which to the requirements that resulted from this review has resulted in administrative penalties totalling that are available on its website. $273,500. Information on enforcement proceed- Released in October 2012, the Renewed ings that the Board has initiated is available on its Regulatory Framework for Electricity report is to website. help guide the Board in setting rates for electricity distributors and transmitters, balancing the Performance Measures need for significant investment in the sector with consumer expectations for reliable service at a Recommendation 4 reasonable price. The Board’s review of distributor To improve the reporting of the effectiveness and performance and benchmarking in the context of costs of its regulatory activities, the Ontario Energy the Renewed Regulatory Framework for Electricity Board (the Board) should develop more results-based is still ongoing, and at the time of our follow-up the or outcome-based performance measures that are Board was in the process of developing a scorecard aligned with its strategic objectives and mandate, and to measure the performance of each distributor in summarize and report all of the costs associated with several key areas, which it expected to implement the Board’s regulatory processes. by the end of 2013. Status The Board has made substantial progress with The Board has made some progress in improving respect to our recommendation on deterrence. the reporting of the effectiveness and costs of its In 2011, it created a dedicated complaints group regulatory activities by developing more outcome- within its Compliance & Consumer Protection busi- based performance measures. It provided examples ness unit to analyze complaint data and identify of initiatives to identify specific performance out- areas or practices that are the subject of frequent comes and determine how to best monitor them: complaints. A summary of complaint numbers and The Board has included in its business plan key issues is available on the Board’s website. • a vision statement regarding the outcomes it The Board has also been using complaint analy- seeks to achieve in the sector, a clear state- sis to identify best practices—for example, in cases ment of the strategic objectives for each year where consumers whose requests to cancel their of the three-year planning period, and a contracts were ignored by suppliers. According to balanced scorecard. The scorecard is to focus the Energy Consumer Protection Act and the Board’s on strategic initiatives and its results are to be

Chapter 4 • Follow-up Section 4.02 code of conduct, the supplier must notify the determined by an independent auditor and energy distributor to cancel a consumer’s contract published in the Board’s annual report. within 10 days of receiving the consumer’s notice of The Board completed a policy evaluation and, cancellation. The complaint analysis has also been • at the time of our follow-up, was in the pro- used to prepare a procedural manual documenting cess of developing a systematic framework to the process analysts are to use to address instances monitor and evaluate the effectiveness of its of non-compliance, which the Board expected to policies. complete by the end of 2013. Electricity Sector—Regulatory Oversight 307

Costs associated with regulatory processes were being summarized, reported and made publicly available on its website: • Regulatory costs in respect of intervenors and the Board’s incremental costs of proceedings are summarized in the Board’s annual reports. • Aggregate costs for intervenors are published every year. Costs for the 2011/12 fiscal year were posted on the Board’s website in July 2012; costs for 2012/13 will be published by the end of 2013. • Regulatory costs incurred by distributors are included under administration costs reported in the Board’s annual yearbooks of electricity distributors. Chapter 4 • Follow-up Section 4.02 Chapter 4 Ministry of Energy Section 4.03 Electricity Sector— Renewable Energy Initiatives

Follow-up to VFM Section 3.03, 2011 Annual Report

an acceptable trade-off given the environmental, Background health and anticipated job-creation benefits. As well, these energy sources are not as reliable as traditional sources, and they require backup from alternative The Ontario government has proposed that the energy sources, such as gas-fired generation. province rely increasingly on renewable energy— In our 2011 Annual Report, we noted the especially wind and solar power. One reason for following: this is to help replace the power lost from the Ontario is on track to shut down its more phasing out of coal-fired generation plants, to • than 7,500 megawatts (MW)—the capacity be completed by 2014. In 2009, the government as of 2003—of coal-fired generation by the enacted the Green Energy and Green Economy Act end of 2014, to be replaced by nuclear power (Act)—now called the Green Energy Act, 2009— to from refurbished plants, an increase of about help attract investments and jobs in renewable 5,000 MW of gas-fired generation, and renew- energy, promote energy conservation and reduce able energy, which is projected to increase to greenhouse gas emissions. 10,700 MW by 2018. The Ministry of Energy (Ministry) has developed Because the Ministry and the OPA aimed to programs and policies to implement the Act, and • implement the Minister’s directions as quickly the Ontario Power Authority (OPA) has played as possible, no comprehensive evaluation was a key role in planning and procuring renewable done on the impact of the billion-dollar com- energy by contracting to buy power from develop-

Chapter 4 • Follow-up Section 4.03 mitment to renewable energy on such things ers of renewable energy projects. Under the Act, the as future electricity prices, net job creation or Minister is provided with the authority to supersede losses across the province, and greenhouse many of the government’s usual planning and regu- gas emissions. latory oversight processes in order to expedite the When the Act was passed, the Ministry said development of renewable energy. • implementing the Act would lead to mod- Wind and solar power will add significant costs est increases in electricity bills of about 1% to ratepayers’ electricity bills. It was felt that the annually. This was later increased to 7.9% higher costs associated with renewable energy were

308 Electricity Sector—Renewable Energy Initiatives 309

annually over the next five years, with 56% Ministry and the OPA reviewed and reduced FIT of the increase due mainly to the cost of prices for new solar and wind power projects by renewable energy. about 20% and 15% respectively in 2012; FIT prices • The OPA was directed to replace a successful for new solar projects were further reduced on aver- program—the Renewable Energy Standard age by one-third in 2013; the OPA has been working Offer Program (RESOP)—with a much more with the Independent Electricity System Operator costly Feed-in Tariff (FIT) program that (IESO) to implement new market rules that require required made-in-Ontario components and renewable generators to turn down or off when encouraged both larger and smaller genera- there is an oversupply of power; and Hydro One has tion projects, but provided renewable energy been upgrading a number of transmission stations generators with significantly more attractive to connect the renewable projects. We noted that contract prices than RESOP. additional work is under way to fully address some • Although the OPA made a number of recom- of our recommendations. For example, the Ministry mendations that could have significantly is planning to launch more online tools and to reduced the costs of FIT, these were held in post more information online to help consumers abeyance until the two-year review of the understand their electricity bills. As well, a review FIT program could be undertaken so as to of Ontario’s Long-Term Energy Plan is scheduled to ensure price stability and maintain investor be finalized before the end of 2013. confidence. The status of the action taken on each of our • A Korean consortium contracted by the Min- recommendations is as follows. istry to develop renewable energy projects is to receive two additional incentives if it COST IMPACT OF RENEWABLE ENERGY meets job-creation targets: $110 million in ON CONSUMERS addition to the already attractive FIT prices; and priority access to Ontario’s already Recommendation 1 limited transmission capacity. However, no To ensure that electricity ratepayers understand economic analysis or business case was done why their electricity bills are rising at a much higher to determine whether the agreement with the rate than inflation, the Ministry of Energy and the consortium was cost-effective, and neither Ontario Power Authority should work together to the Ontario Energy Board nor the OPA was increase consumer awareness of the concept of the consulted about the agreement. Global Adjustment and make more information avail- We made a number of recommendations in our able on the cost impact of its major components. 2011 Annual Report for improvement and received Status commitments from the Ministry and the OPA that In our 2011 Annual Report, we noted that the OPA they would take action to address our concerns. had entered into a number of fixed-price renewable energy contracts that had significantly contributed to higher electricity charges. A number of consumer Chapter 4 • Follow-up Section 4.03 surveys have indicated that although consumers Status of Actions Taken on generally supported renewable energy, they were Recommendations for the most part unaware of its impact on prices. In its responses to our report, the Ministry acknow- Our review indicated that some progress has been ledged that it would increase public awareness made on all of our recommendations, and substan- about energy prices, and the OPA indicated that it tial progress on several of them. For example, the 310 2013 Annual Report of the Office of the Auditor General of Ontario

would provide comprehensive, consistent informa- Status tion about the total cost of electricity. In our 2011 Annual Report, we noted that the Green At the time of our follow-up, we noted that the Energy and Green Economy Act, 2009 provided the Ministry had provided consumers with information Minister of Energy with the authority to direct about Ontario’s energy sector, including electricity certain aspects of planning and procurement of costs, on its website. We also found that the Ministry electricity supply through ministerial “directives” had made changes to its website to make it more and “directions.” The frequent exercise of such user-friendly, such as improving search capabilities, authority had resulted in less thorough analysis and using plain language, increasing accessibility, and assessment of different policy options and the cost- providing updated descriptions of Ontario’s energy effectiveness of alternative approaches. In their sources and provincial programs. The Ministry has responses to our report, the Ministry and the OPA been collaborating with the Ontario Energy Board agreed to work collaboratively to provide decision- to launch other online tools that are intended to makers with the best advice, giving due considera- help consumers understand their electricity bills. As tion to cost, reliability and sustainability. well, the Ministry has used social media tools such At the time of our follow-up, the Ministry as Twitter to educate consumers, and has under- had launched a review of the FIT program on taken market research to further improve consumer October 31, 2011, following the program’s first awareness of the concept of the Global Adjustment two years of operation. During the review, the and its impact on electricity costs. Ministry worked with the OPA, Hydro One and We noted that the OPA has incorporated the the Independent Electricity System Operator to latest 2012 data related to electricity costs in a obtain their input and advice. The Ministry also Generation Procurement Cost Disclosure, which reviewed international best practices, experience was posted on the OPA’s website in September 2013 and perspectives. Both the Ministry and the OPA to provide consumers with more context for and engaged with community groups, municipalities, explanation of electricity costs. The OPA has also the energy industry and associations, Aboriginal revamped its quarterly supply report to make it communities and organizations, environmental more accessible to the average reader. At the same groups, consumer advocacy groups and interested time, the OPA is working with the Ministry and individuals. They received over 2,900 responses other agencies to communicate to consumers initia- from individuals and organizations to an online tives related to electricity costs. survey and about 200 written submissions. The input and advice were reviewed and considered as part of the review of the FIT program. DEVELOPMENT OF ENERGY PLAN AND As part of its review activities, the OPA commis- RENEWABLE ENERGY POLICY sioned two independent consulting companies to produce technical reports regarding the develop- Recommendation 2 ment of renewable projects in Ontario. These To ensure that senior policy decision-makers are pro-

Chapter 4 • Follow-up Section 4.03 reports supplemented the independent analysis vided with sound information on which to base their performed by the OPA’s internal staff and formed decisions on renewable energy policy, the Ministry of the basis of the OPA’s recommendations to the Energy and the Ontario Power Authority should work Ministry regarding proposed changes to the FIT collaboratively to conduct adequate analyses of the program. At the completion of the FIT program various renewable energy implementation alterna- review, the Ministry issued a report on March 22, tives so that decision-makers are able to give due 2012. To address the report’s recommendations, the consideration to cost, reliability, and sustainability. Minister of Energy issued five directives instructing Electricity Sector—Renewable Energy Initiatives 311 the OPA to continue with the FIT program and to to compensate them for any revenue lost as a result implement certain policy changes with regard to of a curtailment instruction. We also found that FIT prices, capacity allocation and prioritization of the normal due diligence process had not been fol- projects. The Ministry’s Renewable Energy Facilita- lowed for the $7 billion Green Energy Investment tion Branch was continuing to meet regularly with Agreement with the Korean consortium and that no the OPA to exchange information, ensure consistent comprehensive and detailed economic analysis or tracking of renewable energy project data, discuss business case had been prepared prior to the Min- policy-related issues and conduct policy analyses. ister entering into that agreement. In its responses to our 2011 Annual Report, the Ministry noted that it would work with the OPA to undertake a manda- PROCUREMENT OF RENEWABLE ENERGY tory review of the FIT program at the two-year Procurement Methods mark, and would continue to work with the IESO to develop new rules and tools to better integrate Recommendation 3 renewable energy sources into the market and pro- To ensure that the price of renewable energy achieves vide full analyses of new investments in renewable the government’s dual goals of cost-effectiveness and energy projects. encouraging a green industry, the Ministry of Energy At the time of our follow-up, the Ministry’s FIT and the Ontario Power Authority should: program review team considered the experiences work collaboratively to give adequate and • of other jurisdictions in setting and adjusting timely consideration to the experiences of other prices for renewable energy, as well as global and jurisdictions and lessons learned from previous local factors that influence pricing for renewable procurements in Ontario when setting and energy projects. The Ministry intends to continue adjusting the renewable contract prices; reviewing FIT programs in other jurisdictions, indi- work with the Independent Electricity System • cating that 92 other jurisdictions have implemented Operator to assess the impact of curtailing FIT programs and that it was common practice to renewables as part of its energy planning in review them regularly. order to identify ways to optimize the electricity In reviewing the FIT program, the OPA also market; and included an assessment by external consultants of ensure that adequate due diligence is under- • price-setting in other jurisdictions, a global scan of taken, commensurate with the size of electricity- jurisdictions and comparable programs, a review sector investments. of stakeholder input and further analysis by the Status OPA. The review recommended reducing FIT prices In our 2011 Annual Report, we noted that there was for some forms of renewable energy. The OPA’s minimal documentation to support how FIT prices recommendations on FIT prices were presented to were calculated and a lack of independent oversight the government during the first quarter of 2012, on their reasonableness. We also noted that there resulting in a new FIT Price Schedule for 2012. To

had been inadequate assessment of the potential balance the interests of all Ontarians while continu- Chapter 4 • Follow-up Section 4.03 costs of curtailing renewable energy (a situation ing to encourage investment, FIT prices for new where the IESO instructs generators to reduce all projects in 2012 were reduced on average by more or part of their output to mitigate an oversupply of than 20% for solar power and by approximately energy) even though there was a strong likelihood 15% for wind power. At the time of our 2011 audit, of curtailment occurring in the future. At the time there were over 3,000 project applications repre- of our 2011 audit, FIT contracts offered renewable senting more than 10,400 MW yet to be commit- energy generators an additional contract payment ted. As of July 2013, about 150 MW of renewable 312 2013 Annual Report of the Office of the Auditor General of Ontario

energy projects had been committed at the new, energy suppliers to deal with the impact of these reduced FIT prices. At the time of our follow-up, market rules on their contracts. The OPA indicated 2013 FIT prices had been further reduced after a that it has reached an agreement with most sup- stakeholder consultation undertaken by the OPA. pliers and intends to continue working with the For example, the price for new solar projects under remaining suppliers. This agreement will provide the FIT program was further reduced by about one- financial certainty to suppliers and reduce costs to third effective August 26, 2013, meaning the new electricity consumers in that suppliers will bear the price is about 50% lower than the original price costs for a certain number of curtailed hours rather when the FIT program was launched. than receiving additional contract payments to Regarding other investments in renewable compensate them for any revenue lost as a result of energy projects, the government has revised its curtailment. According to the IESO, the implemen- Green Energy Investment Agreement with the tation of these market rules is expected to result in Korean consortium, which includes Samsung C&T savings ranging from $70 million to $200 million Corporation. The total commitment for renewable in 2014. energy projects has been reduced from 2,500 MW to 1,369 MW, representing an estimated $3.7 bil- Co-ordination and Planning for the lion reduction from the $9.7 billion contract cost Procurement of Renewable Energy (at the time of our 2011 audit, the estimated amount of the investment was $7 billion). On June Recommendation 4 12, 2013, the Minister of Energy directed the OPA To avoid unintended costs arising out of changes to not to procure large projects (greater than 500 regulatory requirements and changes to supply and kilowatts) under the FIT program and to develop a demand situations, the Ontario Power Authority and new competitive procurement process with input the Ministry of Energy should work collaboratively from stakeholders, municipalities and Aboriginal with other ministries and agencies to ensure that they communities to help identify appropriate locations are made aware on a timely basis of anticipated policy and siting requirements for new large projects. and regulatory changes. On June 17, 2013, the OPA and the IESO launched Status a province-wide initiative to increase awareness In our 2011 Annual Report, we noted several instan- and seek input on regional electricity planning ces where renewable energy initiatives had led to and the siting of large electricity infrastructure. In litigation and potentially unnecessary compensa- September 2013, the OPA submitted interim recom- tion because of conflicts with environmental impact mendations to the Minister following extensive and planning decisions. In their responses to our consultations. report, the Ministry and the OPA acknowledged With respect to the impact of curtailing renew- the importance of close collaboration with other able energy, throughout 2012 the OPA supported ministries and agencies on proposed policy and the IESO on a dispatch management approach regulatory changes.

Chapter 4 • Follow-up Section 4.03 for renewable generation. The OPA engaged with At the time of our follow-up, the Ministry was renewable energy suppliers to address generators’ collaborating with other ministries, including the concerns about the impact of the IESO’s Renewable Ministry of the Environment and the Ministry Integration Market Rule amendments that require of Natural Resources, to streamline regulatory renewable generators to turn down or off when approval processes and eliminate some unneces- there is an oversupply of energy in the system. sary delays or duplication. Specifically: To support the efficient implementation of these market rules, the OPA renegotiated with renewable Electricity Sector—Renewable Energy Initiatives 313

• The Ministry of the Environment has imple- senior officials from relevant ministries to help mented amendments to the Renewable Energy monitor the progress of projects through the Approval regulation as part of the response to approvals process. the FIT program review. Two sets of amend- The OPA has also continued collaborating with ments were enacted, on July 1, 2012, and other agencies and ministries, such as Ontario November 2, 2012, respectively. The amend- Power Generation (OPG), the IESO, Hydro One, ments are intended to clarify requirements the Ministry of Energy, the Ministry of the Environ- and improve turnaround times for applications ment and the Ministry of Natural Resources, to by streamlining the regulatory process while assess and manage the impacts of incorporating maintaining environmental protection. The new generation resources on the electricity system. Ministry projected that these efforts could help During the FIT program review, the OPA formed improve timelines for project approvals by up technical working groups to ensure that multiple to 25%. The Ministry of the Environment has parties were aware of the changes being proposed also initiated a new registry for certain small- and associated solutions. OPA staff are to continue scale ground-mounted solar projects, to align working with the Ontario Energy Board on several requirements with environmental impacts. initiatives, including the Renewed Regulatory This new approach came into effect on Novem- Framework for Electricity (RRFE). The RRFE is a ber 18, 2012. new approach to rate-setting that is intended to • During 2012, the Ministry of Natural Resour- support cost-effective modernization of the elec- ces posted to the Environment Registry tricity network by aligning the needs of the sector for comments proposed policy changes for with the expectations of consumers for reliability renewable energy projects on Crown land. and affordability; offering distributors a choice as The proposed changes are intended to align to how their rates are set to better suit their circum- the release of Crown land with provincial stances; and establishing co-ordinated and optimal energy supply needs and transmission avail- planning through greater harmonization and ability. The Ministry of Natural Resources regional planning processes. OPA planning staff also developed a new Crown Land Site Report are also to continue collaborating with Hydro One, document in order to align access to provincial the IESO and local utility companies on a number Crown land with the updated FIT program. of regional planning and transmission initiatives to • In 2012, the Ministry of Energy created a address local supply adequacy and reliability in the Clean Energy Task Force that included indus- Kitchener-Waterloo-Cambridge-Guelph region, York try experts to advise the Ministers of Energy Region and Toronto. The OPA intends to continue and Economic Development and Innovation its planning activities with the Northwest Ontario (now Economic Development, Trade and First Nations Transmission Planning Committee on Employment) and to help connect all the com- the grid-connection of remote communities. panies in the energy sector. The task force is to

provide advice on ways to increase collabora- Chapter 4 • Follow-up Section 4.03 RELIABILITY OF RENEWABLE ENERGY tion between industry, utilities, academia and government; identify challenges innovative Recommendation 5 companies face when implementing new To ensure that the stability and reliability of clean energy technologies and services in Ontario’s electricity system is not significantly Ontario; and provide advice on export market affected by the substantial increase in renewable opportunities for the clean energy sector. energy generation over the next few years, the • The government created a new Renewable Ontario Power Authority should continue to work Energy Committee in 2012 that included with the Independent Electricity System Operator to 314 2013 Annual Report of the Office of the Auditor General of Ontario

assess the operational challenges and the feasibility DELIVERY OF RENEWABLE ENERGY of adding more intermittent renewable energy into Recommendation 6 the system, and advise the government to adjust the To provide investors who have submitted applications supply mix and energy plan accordingly. for Feed-in Tariff (FIT) projects with timely decisions Status on whether their projects can be connected to the grid In our 2011 Annual Report, we noted that there was and to ensure that adequate transmission capacity a lack of correlation between electricity demand is available for approved projects, the Ontario Power and intermittent renewable energy, resulting in Authority should work with the Ministry of Energy operational challenges such as power surpluses and and Hydro One to: the need for backup power generated from other • identify practical ways to deal on a timely basis energy sources such as natural gas. We also noted with the FIT investors who have been put on that the backup requirements had both cost and hold; and environmental implications. In their responses to • prioritize the connection of approved FIT pro- our report, the Ministry and the OPA agreed that jects to the grid. system reliability and stability is a key element in Status energy system planning and committed to work col- In our 2011 Annual Report, we noted that Ontario’s laboratively with IESO to improve the integration of existing transmission and distribution systems had renewable energy into the supply mix. already been operating at or near capacity when the At the time of our follow-up, the Ministry was FIT program was launched and that this limitation working with the IESO in developing the Renew- had hindered the timely connection of renewable able Integration Market Rules, published in 2012. energy to the grid. In their responses to our report, These rules are intended to enhance the IESO’s the Ministry said it would expedite infrastructure ability to reliably and efficiently manage an elec- upgrades and work with the OPA to prioritize and tricity system that includes a significant amount of effectively connect renewable projects. variable generation from renewable energy sources. At the time of our follow-up, in keeping with the At the time of our follow-up, all of the market rule interest of providing generators with more informa- amendments had come into effect. tion about system availability, one recommendation As the IESO implements the market rules, the arising from the review of the FIT program was OPA intends to continue working with the IESO for the OPA to update its Transmission Availability and renewable energy suppliers on integrating Tables on a regular basis. These tables indicate to renewable energy into the Ontario system and proponents where transmission capacity will be ensuring that renewable energy generators turn available for connecting their renewable energy down or off when there is an oversupply of energy projects. At the time of our follow-up, these tables in the system. had most recently been posted in December 2012 The Ministry was consulting on and working for upcoming small FIT projects. with the OPA and the IESO to develop an updated

Chapter 4 • Follow-up Section 4.03 The Ministry also informed us that Hydro Long Term Energy Plan (LTEP), which was One has been making progress on upgrading a expected to be finalized before the end of 2013. number of transmission stations to enable small- The Ministry developed an interactive tool and scale renewable energy projects. For example, six consumer survey in summer 2013 on the review of upgraded Southwestern Ontario stations have the LTEP, and a series of educational poster boards been placed in service. In June 2012, Hydro One for consultations scheduled to be launched on the announced that it had brought the Bruce to Milton Ministry’s website in November 2013. Transmission Reinforcement Project online six Electricity Sector—Renewable Energy Initiatives 315 months earlier than anticipated. The project is possible, the analysis should give adequate considera- to connect more than 3,000 MW of clean energy tion to both job-creation and job-loss impacts, as from both nuclear and renewable power resources. well as job-related experiences of other jurisdictions Hydro One received approval from the Ontario that have implemented similar renewable energy Energy Board in November 2012 to rewire an initiatives. existing transmission line west of London. This Status project, which is expected to be in service at the In our 2011 Annual Report, we noted that it was end of 2014, is to enable connection of an estimated unclear how the 50,000 new renewable jobs projec- 500 MW of renewable capacity, depending on tion was calculated and whether it was a gross or project type and location. In addition, at the time net number of jobs. We also noted that Ontario’s of our follow-up, Hydro One had initiated upgrades estimate was not consistent with the experiences of to five key transmission stations in Toronto, Ottawa other jurisdictions that have longer histories with and St. Catharines that are to remove limitations renewable energy. In its response to our report, the to connecting more renewable generation in some Ministry said that lessons learned from other juris- areas of the province. dictions with respect to the impacts of job creation With respect to the prioritization of connecting and job losses would be taken into account. FIT projects to the grid, the report released by the At the time of our follow-up, we noted that the Ministry on March 22, 2012, included a detailed Ministry’s calculation of 50,000 jobs to be created discussion on the revised FIT and microFIT applica- through the implementation of the renewable tion and contracting processes. This discussion was energy strategy relied on standard Ontario govern- included to provide clarity to applicants regarding ment methodology, including standard investment the application steps, timelines, and prioritization and job multipliers. This figure of 50,000 has for contracting. The OPA has been working with always been characterized by the Ministry as a mix the Ministry to identify applicants who received of long-term and short-term jobs. The Ministry conditional offers of contracts for their microFIT estimated that by the end of 2012, Ontario’s clean projects but have been unable to connect their energy policies had created over 30,000 jobs in projects at their original locations owing to connec- different areas including construction, installation, tion constraints. The OPA noted that there were 180 energy auditing, operations and maintenance, applicants eligible to participate in the relocation engineering, consulting, manufacturing, finance, IT options. These applicants collectively hold 2,671 and software. The Ministry projected that most new projects accounting for 26.4 MW. Their projects are jobs will be construction- or installation-related, to be relocated to places where they would be able while the remaining jobs are expected to be in oper- to obtain connection. ations and maintenance, equipment manufacturing and engineering design. SOCIO-ECONOMIC, ENVIRONMENTAL The Ministry has been monitoring the develop- AND HEALTH IMPACTS OF RENEWABLE ment of renewable energy in other jurisdictions as ENERGY well as the potential competitive and job impacts Chapter 4 • Follow-up Section 4.03 that higher electricity costs could have on indus- Socio-economic Impacts tries sensitive to energy costs. In addition, the Recommendation 7 government has responded to industries sensitive to To ensure that the provincially reported estimate of energy costs by introducing the Industrial Electri- jobs created through the implementation of the renew- city Incentive Program. Eligible new and expanding able energy strategy is as objective and transparent as industrial companies can qualify for a reduced 316 2013 Annual Report of the Office of the Auditor General of Ontario

electricity rate if they create jobs and bring new Status investment to Ontario. The program is intended to In our 2011 Annual Report, we noted that the encourage existing industrial companies to expand Ontario’s estimated reduction in greenhouse gases their operations and create jobs. Stream 1 of the had not been reduced to take into account the program (for large new investments) closed for continuing need to run fossil-fuel backup power- applications in February 2013, while Stream 2 (for generating facilities. We also noted that the report smaller expansions and new facilities) launched in issued by Ontario’s Chief Medical Officer of Health April 2013. citing no linkage between wind turbine noise and The OPA has been supporting the Ministry’s adverse health effects was not objective. In their job-creation statistics by providing the Ministry with responses to our report, the Ministry and the data available that would be helpful in assessing OPA acknowledged that the impacts of increasing the socio-economic impact of renewable energy. renewable energy should be quantified where pos- Further, as part of the management of the FIT pro- sible and underpinned by objective research. gram, the OPA has conducted random audits on a The Ministry informed us at the time of our number of FIT contracts to ensure compliance with follow-up that the government will continue to rely the terms of the contracts, including the domestic on the Chief Medical Officer of Health to provide content provisions for the purpose of creating jobs in advice on the potential health impacts of renewable Ontario. However, Japan complained to the World energy generators. The Ministry of the Environ- Trade Organization (WTO) in September 2010 ment is to continue monitoring the latest findings that the domestic content requirement breached on low-frequency noise and infrasound from wind world trade rules by being unfairly biased against turbines. In 2010, the Ministry of the Environment non-Ontarian manufacturers. The Ministry advised began providing funding for a five-year term to an us that Ontario intends to comply with the WTO independent research team from the University of rulings and has been given 10 months, from May 24, Waterloo to undertake research on the potential 2013, to bring the FIT program into compliance by health impacts of renewable energy generators. The phasing out the domestic content requirement. team has been studying noise levels at houses near wind turbines and their potential health effects. In May 2012, the OPA expanded and incor- Environmental and Health Impacts of porated environmental performance and social Renewable Energy responsibility into its energy-planning and decision- Recommendation 8 making processes. We noted that the OPA has been To ensure that renewable energy initiatives are tracking CO2 emissions from the electricity sector effective in protecting the environment while having on a regular basis and intends to continue to do so minimal adverse health effects on individuals, the as part of its ongoing energy planning. The OPA’s Ministry of Energy should: latest tracking results at the time of our follow-up • develop adequate procedures for tracking and show that CO2 emissions increased slightly from

Chapter 4 • Follow-up Section 4.03 measuring the effectiveness of renewable energy 12.2 megatonnes (MT) as of December 2011 to initiatives, including the impact of backup gen- 12.6 MT as of December 2012. The increase in erating facilities, in reducing greenhouse gases; emissions was due to an increase in the amount of and energy produced and exported. • provide the public with the results of objective research on the potential health effects of renewable wind power. Electricity Sector—Renewable Energy Initiatives 317

Glossary

additional contract payment—the monetary compensation offered in the Feed-in Tariff (FIT) contract to renewable energy generators for any revenue lost as a result of curtailment. curtailment—a reduction in the output of electricity generators ordered by the Independent Electricity System Operator (IESO) to mitigate an oversupply of electricity. domestic content requirement—a requirement in the Feed-in Tariff (FIT) contract that renewable energy generators use certain made-in-Ontario components; the requirement is intended to promote job creation in Ontario. Feed-in Tariff (FIT)—a program to procure renewable energy launched in September 2009 under the direction of the Minister of Energy, providing renewable energy generators with significantly higher contract prices than the previous procurement initiative, the Renewable Energy Standard Offer Program (RESOP), which it replaced. Generation Procurement Cost Disclosure—an online disclosure of information on electricity costs provided by the OPA for consumers. Global Adjustment—a component of electricity bills whose amount is calculated to make up the difference between the revenues obtained from the electricity market price and the total payments made to regulated and contracted generators (whose prices are guaranteed) and the Ontario Power Authority’s conservation programs. Green Energy and Green Economy Act—the Act enacted in May 2009 with provisions intended to attract investment in renewable energy, promote a culture of energy conservation, create a competitive business environment, increase job opportunities and reduce greenhouse gas emissions. Green Energy Investment Agreement—the 2010 agreement between the Ministry of Energy and a consortium of Korean companies whereby the consortium committed to develop 2,000 megawatts of wind energy projects and 500 megawatts of solar energy projects in Ontario in five phases by 2016, with commitments for equipment to be manufactured in Ontario. Hydro One—the corporation that distributes electricity across the province. Independent Electricity System Operator (IESO)—the entity responsible for the day-to-day operation of Ontario’s electrical system. Ontario Energy Board (OEB)—the entity that regulates Ontario’s electricity and natural-gas sectors. Ontario Power Authority (OPA)—the entity responsible for forecasting electricity demand and procuring electricity supply to meet the province’s power needs. renewable energy—energy generated by natural processes, the four major forms of which are hydro (energy generated from the movement of water), wind (energy generated by turbines from air currents), solar (energy generated by photovoltaic cells that capture radiant light and heat from the sun) and bioenergy (energy generated by burning organic forestry residues and agriculture wastes). Renewable Energy Standard Offer Program (RESOP)—a program to procure renewable energy launched in November 2006, providing fixed, standard prices to generators supplying up to 10 megawatts of renewable energy. Chapter 4 • Follow-up Section 4.03 Chapter 4 Ministry of Finance Section 4.04 Electricity Sector— Stranded Debt

Follow-up to VFM Section 3.04, 2011 Annual Report

electricity sector was restructured on April 1, Background 1999. Less than half of the $38.1 billion was supported by the value of the assets of Hydro One, OPG and the IESO. The remaining $20.9 billion In past Annual Reports, we examined the status not supported by the value of these assets was the of the electricity sector’s stranded debt, defined initial stranded debt. as that portion of the total debt of the old Ontario The government put in place a long-term plan Hydro that could not be serviced in a competitive to service and retire the $20.9‑billion stranded market environment after restructuring of the debt, which included dedicating revenue streams to electricity sector in 1999. We provided the last such OEFC to help pay down this debt: update in our 2012 Annual Report, along with infor- At the time of the restructuring, the estimated mation about the Debt Retirement Charge (DRC), • present value of future payments in lieu of a component of nearly every Ontario ratepayer’s taxes from the electricity-sector companies electricity bill. (OPG, Hydro One and the municipal electrical The stranded debt came into being under the utilities), and of future cumulative annual Energy Competition Act, 1998, which provided the combined profits of OPG and Hydro One in legislative framework for a major restructuring of excess of $520 million a year (the annual the electricity industry. This included the restruc- interest cost of the government’s investment turing of the old Ontario Hydro into four main in the two companies) was estimated at successor companies: Hydro One, Ontario Power $13.1 billion. Generation (OPG), the Independent Electricity The remaining $7.8 billion, called the residual System Operator (IESO) and the Ontario Electricity •

Chapter 4 • Follow-up Section 4.04 stranded debt, was the estimated portion of Financial Corporation (OEFC). OEFC was given the the stranded debt that could not be supported responsibility to manage the legacy debt of the old by the expected dedicated revenue streams Ontario Hydro, along with certain other liabilities from the electricity companies. The Electricity not transferred to Hydro One and OPG under the Act, 1998 (Act) authorized a new Debt Retire- restructuring. ment Charge (DRC), which electricity ratepay- OEFC inherited $38.1 billion in total debt and ers would pay until the residual stranded debt other liabilities from Ontario Hydro when the was retired.

318 Electricity Sector—Stranded Debt 319

The plan was intended to eliminate the stranded Minister had made no such public determination debt in a prudent manner while sharing the debt- of the outstanding amount of the residual stranded repayment burden between electricity consumers debt since April 1, 1999. Our view was that section and the electricity sector. 85 conferred on ministers an obligation to provide a Collection of the DRC began on May 1, 2002, periodic update to ratepayers on the progress their at a rate of 0.7 cents per kilowatt hour (kWh) of payments were making to pay down the residual electricity, a level at which it remains today. Cur- stranded debt. We concluded that a decade was rently, the OEFC collects between $940 million long enough, and suggested the Minister should and $950 million a year in DRC revenue, and provide ratepayers with an update. had collected a total of about $10.6 billion as of March 31, 2013. Our 2011 Annual Report focused on providing details about: Status of Actions Taken on • how much DRC revenue the government had Recommendations collected; the progress in eliminating the residual • In response to these observations, the government stranded debt; and introduced Regulation 89/12 under the Act on when electricity ratepayers might expect to • May 15, 2012, to provide transparency and meet see the DRC fully eliminated. reporting requirements on the outstanding amount Section 85 of the Act, entitled “The Residual of residual stranded debt. The new regulation Stranded Debt and the Debt Retirement Charge,” formally establishes how the residual stranded debt gave the government the authority to implement is to be calculated, and requires annual reporting of the DRC, and this same section specifies when it is the amount in The Ontario Gazette. to end. The key observations from our 2011 Annual We were pleased to see this increased level Report were based on our interpretations of the of transparency was also reflected in the 2012 provisions of section 85, and on our assessment Ontario Economic Outlook and Fiscal Review of whether these provisions had been complied and in the 2013 Ontario Budget; both indicated with in both spirit and form. Specifically, section the Minister of Finance determined the residual 85 requires that the Minister of Finance determine stranded debt to be $4.5 billion as at March 31, the residual stranded debt “from time to time,” and 2012, consistent with the estimate provided in make these determinations public. When the Min- the 2012 Budget. The 2013 Ontario Budget also ister determines that the residual stranded debt has contained a chart, reproduced here as Figure 1, been retired, collection of the DRC must cease. reflecting annual residual stranded debt estimates While the Act did not specify precisely how back to April 1, 1999, and amounts going up to the determination of the residual stranded debt March 31, 2012. Under Ontario Regulation 89/12, was to be done, it does allow the government, by the determination of residual stranded debt as at

regulation, to establish what is to be included in Chapter 4 • Follow-up Section 4.04 March 31, 2013, will be made by the Minister of its calculation. We also observed that the term Finance after the OEFC submits to the Minister “from time to time” was not formally defined, and its annual report, including the audited financial could be left solely up to the government of the day statements, and by no later than March 31, 2014. to determine. Our 2011 Annual Report noted the 320 2013 Annual Report of the Office of the Auditor General of Ontario

Figure 1: Residual Stranded Debt and OEFC Unfunded Liability for Each Fiscal Year Since 1999 ($ billion) Source of data: 2013 Ontario Budget 25

Initial Stranded Debt OEFC Unfunded Liability Residual Stranded Debt 20

11.9

9.9 10 8.5 7. 8 7. 7 7. 5 7. 5 6.7 5.9 6.3 5.6 5.4 5.8 4.5 5

1-Apr-99 2000/01 2001/02 2002/03 2003/04 2004/05 2005/06 2006/07 2007/08 2008/09 2009/10 2010/11 2011/12 1999/2000 Chapter 4 • Follow-up Section 4.04 $10.5 billion peryear. wood product—is estimated to beapproximately sawmill, andengineered wood andvalue-added province’s pulpandpaper, is,the products—that sector forestry valueofOntario’s Canada, the is estimated at142,000 jobs.According to Statistics communities.Employmentindustry ern the within of employment province, inthe especially innorth MNR must adhere. ities onCrown land,subjectto conditionsto which Act under Ontario’s authority ofNatural Resources Ministry (MNR)hasstanding present andfuture generations. In addition, the social,economicandenvironmental needsof the meet management away they insuch their that Crown forests ofOntario’s sustainability and The CFSA isdesignedto provide long-term for the the maintenance andsoon—isgoverned mainly by renewal, management—harvesting, and their forests are 80%ofthe onCrownMore than land, kilometres province. orabouttwo-thirds ofthe forests coverOntario’s 700,000square more than Background Chapter 4 Chapter Ontario’s forest industry is an important source isan important forest industry Ontario’s regarding recurring forestregarding management recurring activ Crown Forest Sustainability Act,1994 4.05 Section Follow-up to VFMSection3.05, to Follow-up Program Management Forest Ministry of Natural Resources ofNatural Ministry Environmental Assessment (CFSA). - - industry had experienced asignificantdeclinedue hadexperienced industry operating under a Sustainable Forest underaSustainable operating Licence. were managed by forest management companies 41of the Forest Management Units (38in2011) into 41 Forest Management Units. Thirty-three follow-up, was divided Undertaking Area ofthe the reasonably terrain. take placedueto the areas where forest management activitiescannot rest comprising provincial landsand private parks, eligible for forest the management activities, with about 190,000 square area kilometres are ofthis ing covers about262,000square kilometres; only Undertak Area ofthe Productive the forest within area isprivately owned. ofthe landsouth of the where accessislimited. Undertaking, Most the are generally Area of not ofthe approved north Forest Undertaking. the management activities 365,000 square kilometres known Area of asthe province is about land occurinanarea ofthe that levels andassociated forest management activities. porarily, resulting inareduction intimberharvest ortemprovince permanently hadclosed,either Asaresult, many millsinthe made inOntario. States, affected demandfor forest which products United economicdownturn inthe dollar andthe Canadian valueofthe mainly increase inthe to the In our At the timeofour At the Most forest management activitiesonCrown 2011 AnnualReport 2011 Annual Report Annual 2011 2011 AnnualReport , we noted the that andour - - 321

3Chapter 4 • Follow-up Section 4.05 322 2013 Annual Report of the Office of the Auditor General of Ontario

Under a Sustainable Forest Licence, which may can accommodate). Harvest blocks were also be granted for up to 20 years, the licence holder held to a silviculture (the practice of control- is responsible for preparing a Forest Management ling the establishment, growth, composition, Plan and implementing the plan by building access health, and quality of forests to meet diverse roads, harvesting trees, renewing/maintaining needs and values) success standard, which the forest, monitoring its forest management is a measure of whether the appropriate activities, and reporting the results of its monitor- or preferred trees have grown back. In the ing to the province. The remaining eight Forest 2008/09 fiscal year, the latest period for Management Units (three in 2011) were managed which information was available at the time of by the Crown. The province also grants Forest our audit, we noted that about a third of the Resource Licences, which allow an individual or licensed forest management companies had company to harvest in a Forest Management Unit. not reported the results of their forest man- Before a Forest Resource Licence can be issued, agement activities, and MNR had not followed the individual or company must come to an agree- up with these companies. The two-thirds that ment with the holder of the Sustainable Forest had reported indicated that although 93% of Licence. The Forest Resource Licence holder will the total area assessed by the companies had generally not be responsible for any forest renewal/ met the province’s minimum 40% stocking maintenance activities subsequent to harvesting, standard, only 51% of the total area assessed because this responsibility typically remains with had achieved silviculture success. the Sustainable Forest Licence holder. The province • MNR’s 40% stocking standard had not has granted nearly 3,400 Forest Resource Licences changed since the 1970s. Several other juris- (nearly 4,000 in 2011), which have a maximum dictions in Canada hold the industry to higher term of five years. standards. In fact, we noted that one MNR Under the CFSA, licensed forest management region, on its own initiative, held companies companies are responsible for overall forest sustain- managing Crown forests in its jurisdiction to a ability planning and for carrying out all key forest higher stocking standard. management activities, including harvesting and • Before planting, seeding or even natural forest renewal, on behalf of the Crown. The prov- regeneration can take place, it is often neces- ince’s role in ensuring the sustainability of Crown sary to prepare a site to allow for regenera- forests has increasingly become one of overseeing tion to take place under the best possible the activities of the private-sector forest manage- conditions, thereby increasing the likelihood ment companies. of success. It is also often necessary to sub- Overall, we concluded in our 2011 Annual Report sequently tend the site, usually by spraying that improvements are needed if the Ministry of to kill off competing vegetation, to further Natural Resources and the Ministry of Northern increase the likelihood of regeneration suc- Development, Mines and Forestry (MNDMF) were cess. On average, between the 2004/05 and

Chapter 4 • Follow-up Section 4.05 to have adequate assurance that the key objective of 2008/09 fiscal years (the latest periods for the CFSA—to provide for the long-term sustainabil- which information was available at the time ity of Ontario’s Crown forests—was being achieved. of our initial audit), only about a third of the Our specific observations were as follows: area targeted for regeneration either natur- • The province considered a one-hectare har- ally or by direct seeding or planting was pre- vest block to have regenerated successfully pared and/or subsequently tended. Moreover, if it was stocked with a minimum of 1,000 the average decreased over that five-year per- trees (that is, 40% of what the harvest block iod. In accordance with the CFSA, all Crown Forest Management Program 323

forests are subjected to an Independent Forest Independent Forest Audits, but that defi- Audit once every five years. Independent For- ciencies detected during such audits were est Audit reports completed in the 2008 and not being addressed in some cases. 2009 calendar years expressed concern about • The average annual harvest between 2004/05 inadequate site preparation or about non- and 2008/09 had been only about 63% of existent or inadequate tending practices that what was planned, and had decreased from were leading to reductions in growth, yield almost 80% of what was planned in the and stand densities, as well as to an increase 2004/05 fiscal year to about 40% of what in the time required for stands to reach free- was planned in the 2008/09 fiscal year. The to-grow status (meaning that the trees are shortfall was usually due to existing licensees free of insects, diseases and high levels of with sole rights to harvest Crown timber not competing vegetation). having a market for the timber. There were • We noted that Forest Management Plans indications that other companies that did not had been completed in accordance with the have access to timber in Ontario’s Crown for- requirements of the CFSA and reviewed and ests could market Ontario wood. A November approved by MNR staff. However, MNR had 2009 competition for unused Crown wood not ensured that the most accurate and up- initiated by MNDMF resulted in the allocation to-date information on forest composition, of approximately 5.5 million cubic metres of wildlife habitat and the protection of these timber that otherwise would not have been habitats was made available at the time the harvested. About 25% of the winning propon- plans were prepared. ents were new mills that planned to invest in • With respect to the province’s monitoring of the province as a result of this competition. At the forest industry, we noted the following: the time of our audit, MNDMF had no plans to • MNR did not maintain a complete list of hold similar competitions in the near future. active harvest blocks in its compliance In fact, we noted that MNDMF did not monitor system to ensure that all harvest blocks whether there is an excess supply of Crown could be identified for possible inspection, wood that could be reallocated to others who and not all of MNR’s district offices used a might be able to market the timber. risk-based approach for selecting blocks for • Measures and controls did not fully ensure inspections. Where problems were noted, that Crown forest revenue was appropriately repeat offenders often did not receive calculated and received on a timely basis and appropriate remedies such as a penalty or a that trusts established to fund forest renewal stop-work order. expenditures incurred by forest management • The forest industry is required to report companies were administered and funded its renewal activities annually to MNR. To adequately. verify the accuracy of the reporting, MNR We also noted that MNR could enhance the use-

implemented a Silviculture Effectiveness fulness of the information presented in its annual Chapter 4 • Follow-up Section 4.05 Monitoring program. However, its district report on forest management by comparing actual offices were not completing many of the levels of key activities—such as harvesting, regener- required “core tasks” in the program. ation (whether occurring naturally or assisted by Where problems were noted, little follow- planting or seeding), site preparation and tend- up action was being taken. ing—to planned levels and providing explanations • We noted that a good process was in place for significant variances. to select the team that conducted the 324 2013 Annual Report of the Office of the Auditor General of Ontario

We made a number of recommendations for meeting the province’s stocking and silviculture improvement and received commitments from standards; and MNR and MNDMF that they would take action to • conduct scientific studies and research into address our concerns. (At the time of our audit both practices in other jurisdictions to ensure that ministries were responsible for the management of the stocking standard is adequate to ensure Ontario’s Crown forests. However, in October 2011 that forest management companies are held to that responsibility fully reverted to MNR.) a regeneration standard that will successfully renew harvested areas with the desired species. Where forest management companies opt for lower-end regeneration activities, MNR should, as Status of Action Taken on part of its review of Forest Management Plans, ensure Recommendations that there is adequate justification for these less- expensive treatments and assess whether the treatments will achieve planned renewal objectives. According to the information provided to us by MNR, some progress has been made in addressing Status several of the recommendations we made in our In our 2011 audit, we recommended that MNR fol- 2011 Annual Report. For example, it had taken steps low up with those forest management companies to better ensure the sustainability of the Forest that have not regularly reported on the results of Renewal and Forest Futures trusts, as well as to their forest management activities. Forest manage- better manage the use of available wood supplies. ment companies are required to report to MNR However, others will require more time to be fully the results of assessments completed on areas har- addressed. For example, our recommendations vested seven to 10 years previously within Forest to better ensure the successful regeneration of Management Units and whether these areas have Crown forests after harvesting have yet to be fully achieved the province’s stocking and silviculture addressed. At the time of our follow-up, MNR was standards. In the 2008/09 fiscal year (the latest still in the midst of reviewing its current regenera- year for which information was available at the tion standards and hoped to finish its review and time of our 2011 audit), we noted that about a develop a revised direction, subject to approval, by third of the forest management companies had not April 2014. reported the results of their forest management The status of actions taken on each of our rec- activities in 2008/09, and MNR had not followed ommendations was as follows. up with these companies. In its response to our 2011 recommendation MNR indicated that forest management companies are required to report SUSTAINABLE FOREST MANAGEMENT annually to MNR on the results of any assessments Forest Renewal that they have completed, but are not required to conduct assessments annually; instead, they may

Chapter 4 • Follow-up Section 4.05 Recommendation 1 accumulate larger harvest blocks for assessment To better ensure that the province’s Crown forests are once every few years. Because of this, it is expected successfully regenerated after harvesting, the Ministry that not all companies will report each year. MNR of Natural Resources should: agreed to follow up with any companies that have follow up with those forest management com- • not reported regularly to ensure that they have a panies that have not regularly reported on the reasonable rationale for not doing so. In 2010/11 results of their forest management activities in and 2011/12, MNR did follow up with forest Forest Management Program 325 management companies that had not reported by Status sending letters reminding them of their reporting At the time of our 2011 audit, MNR intended to requirements, including noting where no assess- have updated its Forest Resource Inventory, which ments had been undertaken. The Ministry informed among other things provides information on the us that all 2010/11 and 2011/12 annual reports had composition, age, height and stocking of individual been submitted. trees within a forest, by 2014. At the time of our In regard to our recommendation that the follow-up, MNR informed us it was still on target to Ministry research practices in other jurisdic- meet this timeline. tions to ensure that its regeneration standards Also, at the time of our 2011 audit, MNR had are adequate, MNR began in 2012 to review and determined that 42 endangered and threatened develop guidelines for improving regeneration stan- species were dependent on the province’s Crown dards by commissioning studies that evaluated its forests and likely to be affected by forest manage- current methodologies for assessing regeneration ment operations, and therefore needed protection. success, and compared standards and approaches We noted that for six of these species, no provincial used in other provinces. MNR hoped to finish this prescriptions (that is, documents specifying the way review and develop revised direction, subject to the species should be protected—for example, by approval, by April 1, 2014. setting up buffer zones between the species and for- MNR also informed us that it continues to est management operations) had been developed. monitor whether forest management companies are At the time of our follow-up, MNR informed us that achieving planned objectives by opting for lower- it had finalized habitat regulations for five of these end regeneration activities such as natural regenera- endangered, forest-dependent species and was tion (instead of seeding or direct planting) through consulting on habitat regulation proposals for the its Silviculture Effectiveness Monitoring program. remaining species. This program consists of a number of “core tasks” Finally, in 2005, MNR had reviewed its silvi- that MNR’s district offices are to carry out to assess culture guides used by the forest industry when industry renewal efforts. In 2010/11 and 2011/12, preparing Forest Management Plans and concluded the program was undertaken on 80% of the forest that all but one required revision. At the time of our units. However, at the time of our follow-up, MNR 2011 audit, MNR was still revising the guides, which was still analyzing the data. prompted us to recommend that MNR update them on a timelier basis. At the time of our follow-up, MNR had still not finished the work and indicated it Forest Management Plans was on track to be completed by fall 2013. Recommendation 2 In order that Forest Management Plans meet their Monitoring objectives in ensuring the future sustainability of Crown forests, the Ministry of Natural Resources Inspection and Enforcement

should ensure that accurate and up-to-date informa- Recommendation 3 Chapter 4 • Follow-up Section 4.05 tion on forest composition and wildlife habitat and To improve its monitoring of forest management the protection of these habitats is made available at companies’ operations for compliance with applicable the time the plans are prepared. MNR should also legislation, regulations, and policies, the Ministry of update any silviculture guides used in forest manage- Natural Resources should: ment planning on a timelier basis. • review its current compliance database to ensure that appropriate linkages are made to 326 2013 Annual Report of the Office of the Auditor General of Ontario

complete harvest block listings so that all har- With respect to our recommendation to MNR to vest blocks can be identified for possible inspec- provide guidance to its district offices in adopting tion; and a risk-based approach for selecting harvest blocks • provide guidance to its district offices in adopt- for inspection, MNR had developed draft guiding a risk-based approach for selecting blocks for ance on risk-based planning for consideration in inspection. the monitoring of industry forest operations. The MNR should also ensure that its district offices are guidance included direction on determining how more consistent and effective in the use of appropri- risk is to be assessed, and managed. At the time of ate remedies to encourage compliance, especially for our follow-up, MNR informed us that the guidance repeat offenders. had been finalized and will be sent to forest managers for implementation starting April 1, 2014. Status In our 2011 Annual Report, we noted that repeat In Ontario, the forest industry is required to inspect offenders often received verbal or written warn- all harvest blocks and report to MNR all suspected ings instead of remedies that might act as more of incidents of non-compliance. MNR then investi- a deterrent—such as an administrative penalty or gates and determines the appropriate remedial cancellation of the forestry licence in serious cases. action for any non-compliance. This prompted us to recommend that the Ministry In our 2011 Annual Report, we noted that should ensure its district offices are more consistent MNR’s database did not contain a complete list- and effective in the use of appropriate remedies to ing of active harvest blocks and listed only those encourage compliance, especially for repeat offend- that had been inspected by forest management ers. At the time of our follow-up, MNR had updated companies. As a result, MNR could not readily its Forest Compliance Handbook to provide more compare all active harvest blocks with those that clarity to district offices in this area. For instance, had been inspected and follow up with companies the Ministry combined two previously separate regarding uninspected blocks. This prompted us to procedures on determining and applying remedies recommend that MNR review the completeness of into one, which, according to the Ministry, reduced its compliance database to ensure that all harvest the complexity of the direction provided to district blocks can be identified for possible inspection. At offices. The Ministry hoped this would improve the the time of our follow-up, MNR had completed such consistency in the application of remedies among a review to determine whether making appropriate district offices. A new procedure was also imple- linkages with harvest block data in Forest Manage- mented on April 1, 2012, to guide field staff in the ment Plans would be warranted; it determined appropriate use of written warnings as a remedy. that the system change would cost approximately $300,000 and take roughly two years to complete. MNR concluded that since the harvest block data is Silviculture Effectiveness Monitoring Program available in Forest Management Plans and Annual Recommendation 4 Work Schedules, ensuring the completeness of this To ensure that the Silviculture Effectiveness Mon- Chapter 4 • Follow-up Section 4.05 data in its compliance database was not warranted. itoring (SEM) program adequately assesses the In this regard, we note that unless MNR takes the effectiveness of industry-reported renewal efforts in time to compare all harvest blocks listed in each regenerating Crown forests, the district offices of the individual Forest Management Plan with those that Ministry of Natural Resources should complete all have been inspected by the forest industry, it will core tasks as outlined in the program and follow up not be able to attain the necessary assurance that with forest management companies on sites found not the forest industry has inspected all harvest blocks. to have met the free-to-grow criteria to ensure that the Forest Management Program 327 companies subsequently took appropriate remedial the extent to which previous recommendations were regeneration measures. satisfactorily addressed. To further enhance the effectiveness of the SEM pro- Status gram, MNR should consider prescribing penalties that Every Forest Management Unit in Ontario is district offices can apply to encourage compliance. subjected to an Independent Forest Audit, which Status assesses a Forest Management Unit’s sustainable The Silviculture Effectiveness Monitoring program forest management practices, at least once every consists of a number of core tasks that MNR’s five years. Upon the audit’s completion, MNR district offices are to carry out to assess the forest and the Forest Management Unit must submit an industry’s efforts in renewing forests. In our 2011 action plan to address reported deficiencies within audit we noted that, for the 2008/09 and 2009/10 two months of receiving the final report, and then fiscal years, district offices that we had visited had complete a status report two years after submitting on average completed only 40% of the core tasks the action plan. In our 2011 Annual Report, we prescribed in the Silviculture Effectiveness Monitor- noted that forest management companies had not ing program. In response to our recommendation, completed a number of the action plans and status MNR agreed to take steps to improve the comple- reports for audits previously conducted on a timely tion rate of the core tasks. However, at the time of basis. In its response to our recommendation, MNR our follow-up, MNR statistics indicated that, for said it would formally review Independent Forest the fiscal year 2011/12, the latest year for which Audit processes and protocols in 2011, and the statistics were available, the completion rate of the results would inform ongoing improvements to core tasks by district offices had only marginally the Independent Forest Audit process, including improved, to 48%. the process in place to assess the extent to which With respect to our recommendation that MNR previous recommendations by auditors have been should consider prescribing penalties to encour- addressed. Shortly thereafter, MNR held early age compliance, MNR agreed in 2011 to evaluate meetings with Sustainable Forest Licence holders the Silviculture Effectiveness Monitoring program on action plan development and streamlined review to ensure that the appropriate incentives were in processes to ensure action plans and status reports place to make sure that when remedial regenera- were completed on a timely basis. The Ministry tion measures are required, these measures are indicated that the action plans and status reports completed by the forest industry. To this end, MNR that were most recently due were, on average, sub- completed a review of its Silviculture Effectiveness mitted on time and, for the most part, the actions Monitoring program in May 2013, but indicated taken had satisfactorily addressed the recommen- that changes stemming from the evaluation will be dations of previous audits. proposed as part of the next revision to the Forest Management Planning Manual and other guidance Planned Versus Actual Harvest documents scheduled to begin in early 2014. Recommendation 6 Chapter 4 • Follow-up Section 4.05 To help ensure that forests are being managed on a Independent Forest Audits sustainable basis and that harvest operations are Recommendation 5 carried out in accordance with approved plans, the The Ministry of Natural Resources should ensure that Ministry of Northern Development, Mines and For- action plans and status reports that address the rec- estry should: ommendations of the Independent Forest Audits are completed on a timely basis and ensure that it assesses 328 2013 Annual Report of the Office of the Auditor General of Ontario

• enhance its ability to monitor on an ongoing Licence, or a supply agreement or commitment. basis the excess supply of Crown wood that can Discussions with forestry industry representatives be reallocated to new companies that can use or were ongoing and at the time of our follow-up, MNR market the wood; and could not provide a timeline for the regulation’s • conduct research into successful practices used implementation. in other jurisdictions to address significant variances between planned and actual harvests. CROWN FOREST REVENUE Status Stumpage Fees In 2011, we noted that in those forest management units where licensees had sole rights to harvest Recommendation 7 Crown timber, but did not have a market for that To ensure that the province receives the proper timber, the actual harvest tended to fall well short amount of revenue for the use of Crown forest resour- of the planned harvest. There were indications that ces, the Ministry of Northern Development, Mines and other companies that did not have access to timber Forestry should: in Ontario’s Crown forests could market Ontario • develop overall provincial guidance for estab- wood, which prompted us to recommend that the lishing wood measurement factors to ensure Ministry should better monitor the excess supply consistency and accuracy among the regions of Crown wood that can be reallocated to new when determining stumpage fees; companies that can use or market the wood. In its • increase the number of scaling audits performed response to our recommendation, the Ministry indi- each year to ensure that all mills are subject to cated that, in the longer term, it had undertaken an the required audit every five to seven years in initiative to modernize its tenure and pricing sys- accordance with MNDMF guidelines; and tem in an effort to allow better access to Ontario’s • design and implement system controls in the wood supply, thereby improving the likelihood that stumpage fee information system so that invalid planned harvest volumes will actually be used. licence holders, and mills and haulers that are In the meantime, in October 2011, a database not authorized to receive and transport wood, and reporting tool called Trackwood was released are identified for appropriate follow-up. to monitor the wood supply and identify surpluses. MNDMF should also formally assess the implica- The information in Trackwood is updated as it tions of renewing harvest licences where significant becomes available. Monthly updates of the avail- stumpage fees are outstanding. able wood supply are now shared with licensees, Status existing mills, new industry proponents, commun- To calculate stumpage fees, the mills measure ities and the government’s economic development nearly all Crown timber harvested and report to staff. MNR also posts these updates on its website MNR on the species of trees and the respective where the public may view them. volumes received. MNR estimates the percentage of At the time of our follow-up, MNR had also

Chapter 4 • Follow-up Section 4.05 defective or undersized wood, which does not incur researched practices used in Quebec and British stumpage fees, by checking the number of under- Columbia relating to the promotion and full use of sized logs in a sample of loads received by mills. In the available wood supply. MNR indicated to us that, our 2011 Annual Report, we noted that there was as part of the initiative to modernize its tenure and no overall provincial guidance on how these esti- pricing system, it was still working with the forest mates should be done, and that all three regions we industry to develop a regulation that, if the avail- visited used different methods. At the time of our able wood supply was not sufficiently used, would follow-up, MNR had developed a 10-year provincial provide for the cancellation of a Sustainable Forest Forest Management Program 329 sampling plan to eliminate these inconsistencies the time of our follow-up, MNR expected to com- and provide the framework for new regional plete these changes to the system in the 2013/14 sampling plans. At the time of our follow-up, two fiscal year. regions had finalized their plans and the third was With respect to our recommendation to assess in the process of finalizing its plan. the implications of renewing the harvest licences of According to MNR guidelines, all mills are to be companies with outstanding stumpage fees, MNR audited every five to seven years to verify that they sent a memo to its regional directors in March 2012 have adequate procedures to accurately measure recommending withholding licence approval to the Crown timber they receive. In our 2011 Annual companies in arrears until a repayment arrange- Report, we noted that an average of only 10 such ment was in place. MNR also provided us with an audits had been carried out annually in the preced- example of harvest approvals being withheld for ing nine years. At this rate, given that more than a large company in April and May 2013 until the 200 mills in the province receive and measure company had paid its outstanding stumpage fees. Crown timber, it would have taken more than 20 In 2011, $45 million in stumpage revenue was in years to audit them all. In response to our recom- arrears. As of June 2013, this amount had dropped mendation to increase the number of scaling audits somewhat to $40.6 million, and $13.6 million each year, MNR indicated that it had reduced the of it had been approved for write-off by an scope of the audits on larger mills in cases where Order-in-Council. the audit team felt it did not compromise the audit’s objective. On these larger mills, MNR examined Forest Renewal and Forestry Futures Trusts documentation covering periods of six to 12 months, and has since reduced this period to three Recommendation 8 to six months. MNR indicated that this reduction To ensure that the Forest Renewal Trust and the enables it to audit the larger mills more quickly and Forestry Futures Trust are sufficiently funded for their hence allows it to do more audits overall. However, intended purposes, the Ministry of Natural Resources for the 2012/13 fiscal year, we noted that MNR had should: audited 10 mills, the same as the average number • review the significant variances in renewal that were being audited at the time of our 2011 rates calculated by district offices for the same Annual Report. When we questioned the Ministry species of trees to ensure that such variances are on this, it indicated that in addition to shortening justified; the review period on audits, it is also pursuing • review the overall minimum balance that is to other options, such as training more staff to audit be maintained in the Forest Renewal Trust to mills. This will enable it to conduct more audits in ensure that the amount is a true reflection of the future. the actual annual forest renewal obligation and In 2011 we noted that many invoices had been ensure that licensees annually maintain their processed for species that forest management portion of the minimum balance; companies did not have a licence to harvest, or • review the Forestry Futures Trust charge to Chapter 4 • Follow-up Section 4.05 haulers were not authorized to haul. In response ensure that it is sufficient to fund the initiatives to our recommendation, MNR indicated that it has that the trust is intended to fund; and proposed changes to the system that would flag • consider requiring Sustainable Forest Licence any unauthorized receipt/transport of wood as an holders to provide some form of financial assur- “invalid tally.” Once a tally is flagged, the system ance that can be used to cover potential silvicul- would not allow it to be processed until it is veri- ture liabilities if a licensee becomes insolvent or fied manually and followed up appropriately. At surrenders its licence. 330 2013 Annual Report of the Office of the Auditor General of Ontario

Status number of options for requiring Sustainable Forest In our 2011 Annual Report, we found that levies Licence holders to provide some form of financial deposited to the Forest Renewal Trust, established assurance. One option that MNR was considering to fund forest renewal expenditures incurred by at the time of our follow-up was an insurance fund forest management companies, varied significantly as a hedge against the event of bankruptcy or some across district offices even for the same species of other occurrence that would prevent a licensee tree. At the time of our follow-up, MNR indicated from completing regeneration activities. An actu- that the authority to establish these forest renewal arial analysis of the regeneration liability and the levies had been taken from district managers and probability of default would determine the size of given to regional directors, and its renewal charge- the fund and the premiums. Since consultations setting process was revised in December 2012 to with stakeholders would first need to be conducted reflect this change. For the 2013/14 fiscal year, the on the various options under consideration, MNR revised process now must consider a licensee’s past could not provide a timeline for when potential reimbursements of eligible renewal expenditures, changes might be made. a forecast of the volume and species of trees to be harvested by the licensee, and the amount of REPORTING future reimbursements. In our 2011 Annual Report, we noted that as of Recommendation 9 March 31, 2011, five licensees had not maintained To enhance the value of its annual report on forest their minimum balance totalling $4 million in the management, the Ministry of Natural Resources Forest Renewal Trust, contravening the terms of should compare actual levels of key forest manage- their licences. At the time of our follow-up, three ment activities—such as harvest and regeneration Sustainable Forest Licence holders did not meet their (that is, natural, planting, seeding, site preparation, minimum balance requirement, totalling a little over and tending)—to planned or target levels and should $230,000. MNR indicated that it was actively pursu- provide explanations for any significant variances. ing the recovery of shortfalls from these licensees. Status MNR had also begun to develop a process for quan- In its 2009/10 annual report on forest manage- tifying and maintaining a statement of outstanding ment (tabled in the Legislature December 2012), forest regeneration liabilities in order to evaluate MNR included planned harvest area and volume whether funds held in individual trust accounts levels. MNR has yet to table the 2010/11 annual are sufficient to cover these liabilities. This involves report. When we questioned why the report had analyzing annual report data submitted by licensees not yet been tabled, MNR indicated that it is up to assess whether all regeneration obligations have to the discretion of the government since there is been fulfilled. no legislative timeline for tabling. The 2010/11 Since our 2011 audit, MNR has also completed annual report, when tabled, will include planned a review of the Forestry Futures Trust charge to regeneration levels as well. MNR indicated that

Chapter 4 • Follow-up Section 4.05 assess whether it is adequately funded. At the time future reports will include a more detailed analysis of our follow-up, MNR was considering an adjust- of actual versus planned levels of harvest and ment to the Forestry Futures Trust charge for infla- regeneration, and explanations of any significant tion as a result of the review. variances. As part of an overall strategic and operational review of both trusts commissioned by MNR in March 2012, the Ministry was considering a Forest Management Program 331

OTHER MATTER that demonstrates that a prospective mill has the ability to finance, operate and manage the facility, Licensing of Mills it is required to submit, among other things, aud- Recommendation 10 ited financial statements for the past three years, The Ministry of Northern Development, Mines and pro forma income statements, balance sheets and Forestry should ensure that forest resource process- cash flow statements for the first five years of oper- ing facility licences are granted only to those forest ation, credit rating and the name of the financial resource processing facilities that demonstrate that institution supporting its application. they have sufficient financial resources to operate, Mills are also required to submit an annual and ensure that forest resource processing facilities return that reports on the facility’s operations submit the required annual returns on a timely basis. based on the volume processed. In our 2011 Annual Report, we also noted that about two- Status thirds of the annual returns were either not To obtain a licence, mills are required to submit a submitted on a timely basis, or not submitted at business plan to the Ministry, which must be satis- all. In March 2013, MNR completed a project to fied that the applicant has the ability to finance, improve the timeliness of the submission of annual operate and manage the facility. In our 2011 Annual returns by forest resource processing facilities. The Report, we noted that licences had been issued project simplified the submission and approval to some mills that had submitted business plans processes in the electronic system that handles that did not demonstrate the applicant’s ability to facility annual returns (eFAR). At the time of our adequately finance the facility. follow-up, the Ministry informed us that for the At the time of our follow-up, MNR had instituted 2007–2011 period, returns covering 87% of the new requirements for the assessment and docu- volume processed had been submitted. The due mentation of the financial resources of new forest date for 2012 returns was September 30, 2013, resource processing facilities (mills) applying for and at the time of our follow-up, returns covering a licence. For instance, as part of a business plan 57% of the volume processed had been submitted. Chapter 4 • Follow-up Section 4.05 Chapter 4 Ministry of Health and Long-Term Care Section 4.06 Funding Alternatives for Family Physicians

Follow-up to VFM Section 3.06, 2011 Annual Report

in alternate funding arrangements and more than Background 90% of enrolled patients. Alternate funding arrangements are generally established and modified by the Physician Services In the past, Ontario’s family physicians were Agreement between the Ministry and the Ontario traditionally paid almost entirely on a fee-for- Medical Association (OMA), which bargains on service basis from the Ontario Health Insurance behalf of physicians in Ontario. This agreement Plan (OHIP) for providing medical services. Over specifies the services that physicians must provide the past 10 years, the Ontario Ministry of Health and the compensation that the province will pay for and Long-Term Care (Ministry) has significantly services rendered. Up to now, it has generally been increased its use of alternate funding arrangements negotiated every four years, but the latest agree- for family physicians in order to, among other ment was for a two-year period only and therefore things, improve patients’ access to care and provide will be renegotiated in 2014. income stability for physicians. By the end of the 2012/13 fiscal year, 8,100 There are 17 types of alternate funding arrange- of the province’s 12,500 family physicians were ments for family physicians. Under many of them, participating in alternate funding arrangements instead of receiving a fee for each service per- (7,700 of almost 12,000 family physicians in formed, physicians are paid an annual fee (called 2010/11), and 10 million Ontarians had enrolled a capitation fee) to provide any of a specific list with these physicians (9.5 million in 2010/11). Of of services to each enrolled patient (that is, each the $4.2 billion in total payments made to the prov- patient who agrees to see the physician as his or ince’s family physicians in 2012/13 ($3.7 billion her regular family physician). Physicians may bill in 2009/10), $3.4 billion was paid to physicians

Chapter 4 • Follow-up Section 4.06 for additional services, as well as for services to participating in alternate funding arrangements non-enrolled patients, on a fee-for-service basis (more than $2.8 billion in 2009/10), with $2.2 bil- (for a list of the types of payments physicians can lion of this amount related to non-fee-for-service receive, see Figure 1). As was also the case at the payments, such as annual capitation payments time of our 2011 audit, the Family Health Group ($1.6 billion in 2009/10). (FHG), Family Health Organization (FHO), and In our 2011 Annual Report, we found that most Family Health Network (FHN) arrangements family physicians participating in alternate funding account for more than 90% of family physicians

332 Funding Alternatives for Family Physicians 333

Figure 1: Selected Types of Payments under Alternate Funding Arrangements for Family Physicians Prepared by the Office of the Auditor General of Ontario

Type of Payment Description Base capitation a fixed amount paid for each enrolled patient, based on age and sex, for providing services listed in the contract, regardless of the number of services performed or the number of times the patient visits the physician (for example, base capitation for FHOs ranges from about $58 to $521 per patient, and for FHNs from about $52 to $367) Access bonus a portion of the base capitation that is reduced when enrolled patients seek care for services listed in the alternate funding arrangement from a physician outside the group the patients are enrolled with Comprehensive care a fixed amount paid for each enrolled patient, based on age and sex, for being responsible for a capitation patient’s overall care and co-ordinating medical services, such as referrals to other health-care providers Complex capitation a fixed amount paid for enrolling a “hard-to-care-for” patient Enhanced fee-for-service physicians bill OHIP and are paid at a rate higher than the traditional fee-for-service value for each patient service provided; the amount in excess of the traditional fee-for-service value is referred to as a “top-up” payment Fee-for-service physicians bill OHIP and are paid the established fee per the OHIP fee schedule for each service provided to a patient Incentives additional payments to physicians to provide specific services, such as patient care on weekends, preventive care and diabetes management; encourage certain activities (e.g., enrolment of certain types of patients, such as hard-to-care-for patients); and compensate physicians for continuing medical education courses Shadow billing physicians who receive base capitation funding can bill OHIP and be paid a percentage of the traditional fee-for-service amount for patient services listed in the alternate funding arrangement; physicians are generally eligible for either shadow billing or enhanced fee-for-service arrangements in 2007/08 were being paid at least times to see a physician had not changed 25% more than their counterparts in the fee-for- significantly. Although more than 40% of service system. By 2009/10, the 66% of family patients got in to see their physician within a physicians who participated in alternate funding day, the rest indicated that they had to wait up arrangements were receiving 76% of the total to a week or longer. amount paid to family physicians. The Ministry had • Of the 8.6 million patients enrolled with not tracked the full cost of each alternate funding either an FHO or an FHG, 1.9 million (22%) arrangement since 2007/08, or analyzed whether did not visit their physician’s practice in the the expected benefits of these more costly arrange- 2009/10 fiscal year, yet the physicians in these ments had materialized. practices received $123 million just for having Some of our other significant observations these patients enrolled. Furthermore, almost included the following: half of these patients visited a different phys- • Based on a survey it commissioned, the ician, and OHIP also paid for those visits. Chapter 4 • Follow-up Section 4.06 Ministry estimated that various initiatives, • The annual capitation fee for each patient including alternate funding arrangements, enrolled in an FHO could be 40% higher than had resulted in almost 500,000 more Ontar- the annual fee for patients enrolled in an FHN, ians having a family physician in 2010 than because almost twice as many services were in 2007. However, the survey also found that covered under FHO arrangements. Neverthe- patients generally indicated that the wait less, in 2009/10, 27% of all services provided 334 2013 Annual Report of the Office of the Auditor General of Ontario

to FHO patients were not covered by the ESTABLISHING ALTERNATE FUNDING arrangement, and the Ministry paid an addi- ARRANGEMENTS tional $72 million to physicians for providing Recommendation 1 these services. Thirty percent of these services To help ensure that alternate funding arrangements were for flu shots and Pap-smear technical for family physicians meet the goals and objectives services, yet the Ministry had not assessed of the Ministry of Health and Long-Term Care (Min- whether it would be more cost-effective to istry) in a cost-effective manner, the Ministry should: have the annual capitation payment include periodically analyze the costs and benefits of coverage for these and other relatively routine • existing alternate funding arrangements to medical services. determine whether the incremental costs of these We made a number of recommendations for arrangements are justified compared to the improvements and received commitments from the traditional fee-for-service model; Ministry that it would take action to address our when negotiating alternate funding arrange- concerns. • ments with the Ontario Medical Association (OMA) ensure that it has good information on the relative costs and benefits of new arrangements being considered as compared to the trad- Status of Actions Taken on itional fee-for-service compensation model, so Recommendations that it is able to take a well-informed bargaining position; and The Ministry provided us with information in the • require all physicians to sign a contract before spring and summer of 2013 on the current status commencing participation in an alternate fund- of our recommendations, indicating it had made ing arrangement. some progress in implementing the recommenda- Status tions in our 2011 Annual Report. For example, the The Ministry has started a formal evaluation of Ministry has started to periodically monitor whether the two main alternate funding arrangements: the physician groups are meeting their after-hours Family Health Groups (FHGs) and Family Health service requirements. However, it will take longer Organizations (FHOs). The evaluation is expected to implement most other recommendations, such as to measure the effectiveness of the models against monitoring the frequency and nature of physician identified objectives and establish baseline informa- services provided to patients, tracking the average tion on the performance of FHG and FHO models amount paid to a family physician participating in in comparison to the traditional fee-for-service an alternate funding arrangement, reviewing the model. The evaluation is expected to include a com- impact of enrolment size on patient access to care, prehensive jurisdictional literature review, analysis and reviewing the impact of existing financial incen- of data from the claims-payment system, and tives on hard-to-care-for patients. The Ministry and

Chapter 4 • Follow-up Section 4.06 surveys of patients and physicians. At the time of the OMA have agreed to conduct a number of joint our audit, the Ministry told us that work was under studies to look at many of our concerns regarding way on the first two components of the evaluation patient access to care. They expect to complete the (literature review and data analysis), and that it studies by April 2014 to inform the negotiations was considering using its new Health Care Experi- between the Ministry and the OMA in 2014. ence Survey to obtain the views of patients and The status of the actions taken on each recom- physicians. The Ministry expects to complete the mendation is described in the following sections. evaluation by January 2014. Funding Alternatives for Family Physicians 335

The Ministry also said it will continue the • review the impact of its policy that allows practice of fully costing any new alternate funding practices with more than five physicians to enrol arrangements, and any amendments to existing only 4,000 patients in total, rather than the 800 arrangements, prior to negotiations. Since our patients per physician required by practices with audit, there have not been any new types of alter- fewer physicians, to determine the impact this nate funding arrangements. The Ministry informed policy has on access for people with no family us that, for the purpose of negotiating the 2012 physician; and Physician Services Agreement with the OMA, it • review the number of patients being de-enrolled prepared a series of proposals on various aspects by their physician to determine whether a of alternative funding arrangements for family significant number of these patients are in the physicians. These proposals were designed to hard-to-care-for category, and, if so, whether simplify or reduce the different types of payments the current financial incentive arrangements under the contracts, achieve savings, better define should be revised. service expectations and performance measures, Status and improve access to care and quality. In most The Ministry informed us that it plans to review its cases, these proposals contained information on the policies regarding: expected costs of the proposed changes. Changes the appropriateness of paying capitation pay- made to the 2012 Physician Services Agreement as • ments for enrolled patients who do not visit a result of these proposals are referred to through- the physician with whom they are enrolled for out this status update where appropriate. at least a one-year period; The Ministry also informed us that it has refined the impact on access to care resulting from its registration procedures to include a checklist • controls on minimum enrolment size; and of all documentation required, including signed the linkage between de-enrolment and patient contracts and declaration forms, prior to commen- • complexity, and whether enhanced/modified cing funding to physicians under alternate funding payment incentives are required to ensure arrangements. This process should help ensure that continued access to care. signed contracts and declaration forms are in place The Ministry has identified the data and resour- for new arrangements or for physicians joining ces needed to perform the reviews, but has not existing arrangements. The Ministry told us that yet extracted the data to begin the analyses. The it did not ensure signed contracts or declaration Ministry advised us that any proposed changes forms were in place for existing physicians. resulting from the policy reviews would have to be negotiated with the OMA, either as part of the next ENROLLED PATIENTS round of negotiations for the upcoming 2014 Phys- ician Services Agreement, or through the contract Recommendation 2 amendment process set out in the current 2012 To better ensure that alternate funding arrange- Physician Services Agreement. ments are cost-effective and that patients have access Chapter 4 • Follow-up Section 4.06 In the Ministry’s 2011 response to our audit to family physicians when needed, the Ministry of recommendation, it indicated that work was under- Health and Long-Term Care should: way by a joint ministry/OMA working group, with periodically review the number of patients who • support from the Institute for Clinical Evaluative do not see the physician they are enrolled with, Sciences, to evaluate options for modifying the and assess whether continuing to pay physicians capitation rate in order to resolve issues related to the full annual capitation fee for these patients maintaining complex patients in capitation-based is reasonable; funding models (the rate currently only takes into 336 2013 Annual Report of the Office of the Auditor General of Ontario

account the age and sex of a patient). The study improvement in compliance rates over the last two formed the basis for an interim acuity modifier years, as illustrated in Figure 2. An exemption from included in the 2012 Physician Services Agreement, providing after-hours services can be obtained from which is mentioned in the next recommendation. the Ministry if more than 50% of physicians in the The Ministry informed us that it expects to negoti- group provide certain other services outside regular ate a permanent acuity modifier in the next round hours, such as emergency room coverage. The of negotiations with the OMA. Ministry advised us that, since 2011, it has required all physician groups who meet exemption criteria and wish to be exempt from providing after-hours PATIENT ACCESS TO PRIMARY-CARE service to re-apply annually for the exemption to SERVICES ensure they continue to be eligible for it. According Recommendation 3 to the Ministry, over the last two years there has To ensure that alternate funding arrangements are also been an almost 40% increase in the number meeting their goal of improving access to family phys- of FHOs required to perform after-hours services, icians, the Ministry of Health and Long-Term Care which is likely to improve access to services, and (Ministry) should: virtually no change for FHGs and FHNs. • periodically monitor whether physicians par- The Ministry advised us that it had completed ticipating in alternate funding arrangements an inventory of all current contract requirements provide patients with sufficient and convenient in the FHG and FHO alternative funding arrange- hours of availability, including after-hours avail- ments, and had assessed the impact associated with ability, as required by the arrangements; and each contract requirement in terms of financial risk • conduct a formal review of whether alternate and risk to patient access. The Ministry’s evaluation funding arrangements are meeting the goal identified two contract requirements as high risk, of improving access, especially given that the for which the Ministry had no monitoring processes Ministry’s Primary Care Access Survey indicates in place. One was physician services (the ability, little change in the last three years in the wait for example, to provide patients with comprehen- times for seeing a family physician. sive medical care) and the other was maintaining regular business hours. The Ministry informed us Status that developing monitoring processes for these two At the time of our follow-up, the Ministry had areas are a priority, and that it expects to have them implemented an annual monitoring process to in place by January 2014. evaluate the provision of after-hours services by According to the Ministry, improving patient family physicians in alternate funding arrange- access to primary care services was a key theme in ments, and had developed a process to encour- the 2012 negotiations with the OMA. To that end, age non-compliant physicians to take corrective the 2012 Physician Services Agreement includes a action. Contracts define “after-hours” as Monday to Thursday after 5 p.m. and anytime from Friday

Chapter 4 • Follow-up Section 4.06 Figure 2: Percentage of Physician Groups in through Sunday. At the time of our audit in 2011, Compliance with After-hours Services the Ministry conducted an ad hoc review of claims Source of data: Ministry of Health and Long-Term Care for after-hours services submitted by FHNs, FHOs and FHGs for June 2010 to determine whether Funding June 2010 June 2011 June 2012 Arrangement (%) (%) (%) physician groups had complied with the after-hours FHG 75 79 76 service requirements. The Ministry informed us FHN 41 57 50 that it repeated the exercise for June 2011 and FHO 60 72 62 June 2012 and found that there has been a slight Funding Alternatives for Family Physicians 337 number of provisions to improve access to family Both studies are to be conducted by a joint com- physicians, such as: mittee of the Ministry and the OMA that is expected • bonuses to encourage more house calls; to report back by April 2014. The Ministry informed • implementation of an interim acuity modifier us that it would consider recommendations from in capitation payments to take into account the two joint studies in developing proposals for the the seriousness of a patient’s medical condi- 2014 round of negotiations with the OMA. tion; and enhanced after-hours service requirements • PAYING FAMILY PHYSICIANS for groups with more than 10 physicians. For example, under the 2012 agreement, practices Recommendation 4 with 10 physicians are required to provide a To facilitate the administration of the current complex minimum of seven three-hour blocks of after- alternate funding arrangements for family phys- hours services each week, while practices of icians, the Ministry of Health and Long-Term Care 100 physicians must provide 20 three-hour (Ministry) should consider reducing the number of blocks. Under the previous agreement, all arrangements and simplifying the types of payments. practices of more than five physicians were Further, to better ensure that the alternate funding required to provide a weekly minimum of only arrangements are cost-effective, the Ministry should: five three-hour blocks. • review the fee-for-service payments to physicians The Ministry intends to monitor house calls for services not covered by the annual capitation through fee-for-service claims and/or shadow payment, and determine whether significant billings, and the enhanced after-hours services for savings may be possible by having them covered large groups through the annual monitoring pro- by the capitation payment; and cess described above. Since the acuity modifier is a • consider negotiating a reduction in capitation one-time calculation and payment, no monitoring payments for patients who never or seldom see activity is expected. the physician they are enrolled with, as well as The 2012 Physician Services Agreement also a further reduction in capitation payments to included commitments by the Ministry and the better reflect the cost of non-emergency services OMA to conduct two joint studies related to patient that patients obtain from physicians who are access to primary care physicians, as follows: not part of the practice they are enrolled with. a study of daytime access to primary care • Status physicians in the various alternate funding The Ministry advised us that during the 2012 nego- arrangements, including recommendations tiations with the OMA, it proposed moving towards on possible guidelines on daytime operations a single capitation payment model that would cover that could include standards for group size, more clinical services than before. It also proposed and strategies and support for same-day or to simplify the types of payments under the various next-day access; and contracts. However, negotiations with the OMA did

a policy review to consider the value of access Chapter 4 • Follow-up Section 4.06 • not result in any changes in the number of arrange- bonuses (the amounts deducted from capita- ments, nor in the list of services covered under each tion payments to physicians in FHNs or FHOs type of arrangement. when their enrolled patients seek non-emer- The 2012 negotiations with the OMA did, gency treatment outside the practice), the however, result in some changes in the types of impact on emergency departments, exemption payments made to physicians. According to the for urgent care centres and GP-focused practi- 2012 Physician Services Agreement, some types of ces, and the impact of walk-in clinics. 338 2013 Annual Report of the Office of the Auditor General of Ontario

payments were eliminated for all types of funding MONITORING arrangements, while other types of payments were Recommendation 5 eliminated for only some arrangements. Overall, To provide the Ministry of Health and Long-Term Care this reduced the number of different types of pay- (Ministry) with information that would facilitate ments to physicians in FHGs to 37 from 42, and to better monitoring of the benefits and costs of each physicians in FHOs to 52 from 61. alternate funding arrangement for family physicians, In addition, the Ministry told us that it plans the Ministry should: to initiate a review of all bonus and premium periodically review shadow billing data to payments under the various contracts to identify • determine the frequency and nature of services opportunities to further simplify payments. It provided by physicians in each arrangement; expects to use the results from this review to pro- track the total amount paid to physicians par- pose changes to the OMA in 2014. • ticipating in each arrangement; and With regard to the issue of physicians being track the average amounts paid to each paid a capitation rate for patients they seldom or • physician both for reasonableness and for the never see, the last round of negotiations with the purposes of comparing them to physician com- OMA did not result in a reduction in capitation pensation under the traditional fee-for-service rates for these patients. Instead, the Ministry hopes funding model. to establish an acuity modifier that will address service utilization under the capitation-based pay- Status ment models. The last round of negotiations also At the time of our follow-up, the Ministry was still did not produce an increased penalty in capitation in the process of developing monitoring activities payments (that is, the access bonus) to physicians that would address the recommendations above and when their enrolled patients seek non-emergency support future program or policy design changes for services from outside the practice. As noted in the capitation-based models, including FHNs and FHOs. previous recommendation, the Ministry and OMA The Ministry informed us that it had identified its have committed to jointly conduct a policy review data needs and extracted data for initial analysis, of the access bonus payment for capitation-based and was in the process of developing regular pro- models like FHNs and FHOs. In its 2011 response, duction reports for payment tracking and analysis. the Ministry stated that a similar review was under The Ministry expects regular production reports way at that time. However, it was put on hold once to be developed by late autumn 2013 and regular negotiations started with the OMA. monitoring activities to begin soon after. Chapter 4 • Follow-up Section 4.06 services agreement between the Ministry and the andthe between agreement Ministry the services are alsosubject to provisions physician inthe forties. Alternate fundingarrangements specialists and universi ashospitals organizations such other andmayon behalfofphysicians include inOntario) bargains organization that the Association (OMA, Medical Ontario physicians, andin most casesthe betweentual agreements Ministry, of the agroup in hospitals. family physicians) whoprovide emergency services forfunding arrangements physicians (generally In 1999, introduced specialist alternate Ministry the newtraining physicians andconductingresearch. including asacademicservices, basis,such service were not compensated fee-for- existing underthe they for which toage services provide them certain remote province, areas ofthe aswell asto encour ments to encourage specialist physicians to work in introduced alternate fundingarrange (Ministry) andLong-Term1990s, ofHealth Ministry the Care Plan(OHIP)billings.Inthe Insurance Health incomefrom fee-for-service most oftheir Ontario andgenerally andemergency obtain atrics services, 60 areas, includingcardiology, pedi orthopaedics, inmoreSpecialist physiciansthan provide services Background Chapter 4 Chapter Alternate funding arrangements areAlternate fundingarrangements contrac Section 4.07

Follow-up to VFMSection3.07, to Follow-up Physicians Specialist Funding Alternativesfor Ministry ofHealthandLong-TermMinistry Care - - - - - in 2012. signed most recent agreement the since 2000,with hasbeennegotiated four which years every OMA, between 2006/07 and2009/10, number whilethe physicians increased bydepartment almost 40% paymentsfound, for to that instance, emergency were arrangements the cost-effective.whether We as improving accessto patients’ specialists—or expected benefits—such specialists hadyieldedthe for alternate fundingarrangements the whether hadconductedanalysis of littleformal Ministry arrangement. aspecialist alternate funding through least inpart, province physicians inthe weredepartment paid,at cialist 90%ofemergency physicians andmore than 2010 latest available 50%ofspe (the information), (17% of$6.3 billionin2009/10). 31, AsofMarch specialists andemergency year room physicians that $7.1 paidto all Ministry the billion intotal that in 2009/10), accountsfor about18% which ofthe toarrangements physicians (almost $1.1 billion $1.3 billion underspecialist alternate funding 2012/13tion. Inthe fiscal year, paid the Ministry in2012/13arrangements intime for publica and emergency room physicians paidunderthese numberofspecialists able to provide the uswith was not Ministry butthe funding arrangement, physicians were fundedunderaspecialist alternate In our 2009/10In the fiscal year, than 9,000 more 2011 AnnualReport 2011 Annual Report Annual 2011 , we found that the , we the found that - - 339

Chapter 4 • Follow-up Section 4.07 340 2013 Annual Report of the Office of the Auditor General of Ontario

of physicians working in emergency departments increased by only 10%, and the number of patient Status of Actions Taken on visits increased by only 7%. Recommendations Some of our more significant observations were as follows: The Ministry provided us with information in Specialists could earn numerous types of pay- • spring and summer 2013 on the status of our ments and premiums under alternate funding recommendations. According to this information, arrangements (formed through arrangements some progress has been made in implementing among hospitals, universities with a medical most of the recommendations in our 2011 Annual school and physicians), making it difficult for Report. For example, the Ministry has developed a the Ministry to monitor contracts and related template to facilitate comparison of alternate fund- payments. For example, for academic services ing arrangements with the fee-for-service model. at Academic Health Science Centres, there However, tracking the full cost of alternate funding were as many as nine different categories of arrangements will take longer to implement. Some payments. actions, such as incorporating performance meas- Ten Academic Health Science Centres • ures into contracts and significantly simplifying the received “specialty review funding” total- different types of payments under the academic ling $19.7 million in 2009/10 as an interim contracts, will depend on further negotiations with measure to alleviate shortages in five specialty the OMA, as they were not addressed in the 2012 areas. Yet similar interim funding had been negotiations. given annually since 2002. The status of actions taken on each of our The Ministry paid $15,000 each to 234 north- • recommendations is described in the following ern specialists who gave the Ministry permis- sections. sion to collect information on income they earned from provincial government–funded sources. CONTRACTING WITH SPECIALISTS In order to monitor whether specialists funded • Recommendation 1 under academic contracts performed the To help ensure that compensation arrangements for required services, the Ministry provided the specialists meet the Ministry of Health and Long-Term specialists with a checklist to self-evaluate Care’s goals and objectives in a financially prudent their performance. But the checklists were manner, the Ministry should: never requested back, and minimal other assess and document the anticipated costs and monitoring had been done. • benefits of each alternate funding arrangement, In April 2008, the Ministry paid more than • compared to the standard fee-for-service com- $15 million to 292 physicians who signed a pensation method, before entering into a formal document indicating that they intended to agreement;

Chapter 4 • Follow-up Section 4.07 join a northern specialist alternate funding incorporate specific performance measures into arrangement. However, 11 of the physicians, • the contracts, such as the number of patients who were paid a total of $617,000, did not to be seen or the wait times to access care, to subsequently join such an arrangement, yet enable the Ministry to periodically assess what they were allowed to keep the funding. benefits are received for the additional cost of We made a number of recommendations for the arrangement; and improvement and received commitments from the Ministry that it would take action to address our concerns. Funding Alternatives for Specialist Physicians 341

• require physicians to sign that they agree to the PAYING SPECIALISTS terms of the contract before commencing par- Recommendation 2 ticipation in an alternate funding arrangement. To better ensure that payments made under alternate Status funding arrangements among similar specialist The Ministry informed us that in 2012 it developed groups are in accordance with the underlying con- a cost/benefit analysis template to facilitate com- tracts, the Ministry of Health and Long-Term Care parisons between alternate funding arrangements should: and the fee-for-service model. Since the time of our • simplify the numerous different types of pay- audit, the Ministry has not entered into any new ments under the academic contracts; and agreements with specialist physicians and hence • review situations where additional funding is has not used the template. In addition, although consistently being provided or where overfund- almost all contracts that were in place during our ing or duplicate payments have occurred in 2011 audit have since expired, none have been order to determine whether the funding should renewed; therefore, none have been subject to a be adjusted or recovered. cost/benefit analysis. Payments continue to be Status made as per the terms and conditions of the expired The Ministry established a working group in August agreements. At the time of our follow-up, the Min- 2012 to review opportunities for streamlining aca- istry and the OMA were negotiating new standard- demic payment categories, and also to review pay- ized contracts. ment categories under other alternate payment/ The Ministry informed us that no performance funding arrangements. As a result of its review, measures have been incorporated in any existing the working group recommended eliminating two contracts, but it has begun a process of reviewing funding categories for Academic Health Science existing agreements to identify what performance Centres and four funding categories linked to other measures should be in place. The Ministry also alternate payment/funding arrangements. The informed us that the addition of any new perform- working group did not recommend eliminating any ance measures must be negotiated with the OMA. other funding categories, because they are linked to The Ministry indicated that it has reviewed the payment requirements set out in the alternate fund- declaration and consent requirements by contract ing agreements and the physician services agree- type and that currently all physicians are required ments for 2004 and 2008 between the province to sign that they agree to the terms of the contract and the OMA. The various funding categories were before commencing participation in an alternate not consolidated into the 2012 Physician Services funding arrangement. The Ministry also advised Agreement. According to the Ministry, the 2012 us that it has ensured that signed declaration and negotiations with the OMA did not focus on individ- consent forms are on file for all agreements requir- ual agreements with specific specialist groups. The ing them, except for the agreements involving the working group recommended that implementation approximately 3,000 emergency room physicians

coincide with the start of the 2013/14 fiscal year. At Chapter 4 • Follow-up Section 4.07 paid through alternate funding arrangements. the time of our follow-up, an implementation date Declaration and consent forms for emergency room had not yet been determined. physicians had been held at OHIP district offices The Ministry informed us that recovery that have been since closed, and therefore they practices had been reviewed to ensure that docu- were not available for verification. The Ministry mentation is in place to support decisions related expects all physicians in alternate funding arrange- to non-recoveries. Overpayments to emergency ments to sign new declaration and consent forms departments, which occur when patient volumes once standard contracts are negotiated. 342 2013 Annual Report of the Office of the Auditor General of Ontario

are lower than expected, totalled $972,000 for review whether its overall goals and objectives for such the 2010/11 fiscal year. By December 2012, the arrangements are being met in a cost-effective manner. Ministry advised us, it had recovered $315,000 Status and would be recovering the remaining $657,000 The Ministry advised us that it was developing a from emergency physician groups at two hospitals process to review billing data on a regular cyclical over an extended period of time in order to lessen basis, which would enable it to determine, for the impact of a lump sum recovery, which could example, whether academic physicians are provid- jeopardize the ability of those emergency departing a minimum level of clinical services, including ments to provide services 24 hours a day, seven seeing a minimum number of patients. However, days a week. the Ministry had not set an implementation date. Furthermore, as was its practice in 2011, it was MONITORING ALTERNATE FUNDING continuing to analyze billing claims only when a ARRANGEMENTS physician group funded under an alternate funding arrangement asked to have physicians added to the Recommendation 3 group, and to identify overpayments and underpay- To better ensure that Ontarians have access to special- ments to emergency departments whose payments ist physician care, consistent with the overall objective were based on patient volume. of alternate funding arrangements, the Ministry of At the time of our follow-up, the Ministry Health and Long-Term Care should monitor whether advised us that it was not yet tracking the full cost specialist groups are providing patient care and other of each alternate funding arrangement, but was services in accordance with their contracts. working with its information technology staff to Further, to ensure that the benefits of the special- develop an automated report that would track ist alternate funding arrangements outweigh the all physician payments to each alternate funding costs, the Ministry should track the full costs of each arrangement group, including base payments, pre- alternate funding arrangement, including total fee- mium payments and fee-for-service payments, as for-service billings paid to physicians, either directly applicable to each arrangement. or indirectly, and use this information to periodically Chapter 4 • Follow-up Section 4.07 specific category ofproduct. Itcanalso buyprod specific category a callto known suppliers, asa“needsletter,” for a line,is tagesto issue finewineandpremium spirits forone, both general-list Vin products andfor the ods to selectandbuy new products. Theprincipal 2012/13the fiscal year.three meth The LCBO uses 42% ofVintages products were newly acquired for 630 stores. 13% About ofgeneral-list products and available by order—available private atmore than list, 6,700Vintages products and12,400 products products—approximately 4,100 items onitsgeneral provincethe have goneup74% time. inthat the dividendsitpays years ago,anditsprofit and to province. LCBO saleshave increased 57% from 10 the profit LCBOto remitted allofthat virtually was $1.7 billion ($1.56 billion in2010/11). The $4.9 billion ($4.6in2010/11), anditsprofit incomewereLCBO’s salesandother approximately retailer.profitable the 2012/13For fiscal year, the innovativeresponsible, performance-driven, and to beasociallyalcohol products inOntario—is to buy, andsellbeverage distribute, import, (LCBO)—a Crown powerOntario the agency with LiquorThe mandate Control Board ofthe of Background Chapter 4 Chapter The LCBO offers consumers more than 23,200 more than consumers The LCBO offers 4.08 Section

Follow-up to VFMSection3.08, to Follow-up Procurement LCBO NewProduct Liquor Control Board ofOntario Board Liquor Control - - - directly from suppliers. ucts onan“adhoc”basisor, caseofVintages, inthe in place for the purchase ofnew products, and purchase in placefor the had adequate systems, policiesandprocedures wine prices). lowest andbeer, for spirits prices lowest andthe 2011 third- LCBO hadthe the found that survey and beer, lowest April and the (the wineprices second-lowest the for spirits with prices prices), lowest LCBO hadthe the found overall that alcohol 2011 (anApril jurisdictions all those also survey lowest LCBO hadthe the overall of alcoholprices 2013 aJanuary jurisdictions, found that survey sells areatlower offered Canadian inother prices Unitedthe States. in alcohol products are generally those higherthan lowestucts atthe for soretail prices possibleprices, doesnot sell itsprod Canadianjurisdictions, other social responsibility. LCBO, like the Thismeansthat forout minimumretail alcoholto prices encourage province.for the Ontario’s consumption ofalcohol whilegenerating revenue date to promote saleand socialresponsibility inthe products itsells,guidedby for its man the prices powerFinance, LCBO hasthe the to retail set the of Ministry the with inconsultation established Within the framework policy the pricing Within ofthe Our 2011 LCBO auditfocused the onwhether LCBO the products that someofthe Although 2011 Annual Report Annual 2011 Liquor Control Act sets - - 343

Chapter 4 • Follow-up Section 4.08 344 2013 Annual Report of the Office of the Auditor General of Ontario

whether such purchases were acquired and man- lower profits, something that runs against aged effectively and in compliance with applicable the LCBO’s mandate of generating profits for legislation, government directives and LCBO pro- the province and encouraging responsible curement policies. consumption. In our 2011 Annual Report, we noted that the • The LCBO does have many well-established LCBO had many well-established purchasing purchasing practices. However, it could practices consistent with those in other Canadian improve some of its processes relating to jurisdictions and other government monopolies. purchasing and monitoring of product per- Nevertheless, our 2011 audit suggested certain formance to better demonstrate that these changes that could be made to improve some of processes are carried out in a fair and trans- the LCBO’s processes related to purchasing and the parent manner. subsequent monitoring of product performance, to We made a number of recommendations for better demonstrate that these are carried out in a improvement and received commitments from fair and transparent manner. Our findings included the LCBO that it would take action to address our the following: concerns in all but one area. This area is noted later • In the private sector, large retailers use their in the section on recommendation 4 regarding the buying power to negotiate lower costs with lack of documentation around the reasons for selec- suppliers. However, the LCBO, despite being tion or elimination of products at the prequalifica- one of the largest purchasers of alcohol in the tion stage. world, does not focus on getting the lowest cost it can for a product. Rather, the cost it pays is driven by the retail price it wants to charge for a product. The LCBO gives suppli- Status of Actions Taken on ers a price range within which it wants to sell Recommendations a product. Suppliers’ product submissions include, among other things, the retail price According to information the LCBO provided at which they want their product to sell in to us in spring 2013, it has fully implemented LCBO stores, and they then work backwards, most of the recommendations we made in our applying the LCBO’s fixed-pricing structure 2011 Annual Report. For example, the LCBO has to determine their wholesale cost. We noted updated its internal policies and procedures for that in some instances suppliers submitted each procurement method, including the evalua- wholesale quotes that were significantly lower tion criteria and processes to be used in assessing or higher than what the LCBO expected, in new-product submissions. However, a couple of which cases the suppliers were asked to revise the components of our recommendations that the amount of their quotes in order to match involve working with other organizations, such the agreed-upon retail price, which effectively as the Ministry of Finance, will require more time

Chapter 4 • Follow-up Section 4.08 either raises or lowers the price the LCBO pays to be fully addressed. The two particular areas the supplier for the product. that have yet to be substantially addressed are our The LCBO does not negotiate volume dis- • recommendation to consider using, on a trial basis, counts. This is also true of other Canadian a variable markup when purchasing new products, jurisdictions we looked at. The LCBO’s fixed- and our recommendation to determine the most pricing structure gives it no incentive to nego- appropriate organization to monitor compliance tiate lower wholesale costs; doing so would with the Liquor Control Act’s minimum retail price result in lower retail prices, and, in turn, LCBO New Product Procurement 345 requirements. Progress on these is not expected The Cost of Beverage Alcohol Products until late 2013. The LCBO noted that in the mean- Recommendation 2 time it continues to operate within the pricing and In keeping with its mandate to generate sufficient procurement parameters that have been set by the profits and adhere to the government’s policy direc- Ministry of Finance. tion of maintaining a retail pricing mechanism that The status of actions taken on each of our rec- encourages responsible consumption, the LCBO ommendations at the time of our follow-up was as should consider, in consultation with the Ministry of follows: Finance, the following strategy on a trial or pilot basis to take advantage of its being one of the largest pur- RETAIL PRICES OF BEVERAGE ALCOHOL chasers of beverage alcohol products in the world: PRODUCTS • once product categories and their related retail price ranges have been determined, allow sup- Legislated Minimum Retail Prices pliers to offer a product at whatever cost they Recommendation 1 are willing to accept to have it sold at the LCBO, To better inform Ontarians about how beverage and then use a variable markup to arrive at the alcohol prices are set, the LCBO should provide more desired fixed retail price; and information to the public on its pricing policy, includ- • calculate the gross profit margin for a particular ing how its mandate and provincial policy objectives product based on the supplier’s cost quote, and affect pricing, and details about its pricing structure. take this into consideration in making decisions As well, the LCBO, in conjunction with the Ministry on which new products to purchase along with of Finance, should establish a process for ensuring the other evaluation criteria currently used, that all stores are complying with the Liquor Control such as the quality of the product. Act’s minimum retail price requirements and consider Status whether the LCBO is the most appropriate organiza- The LCBO informed us that it has provided the tion to monitor this compliance. Ministry of Finance with options on how to proceed Status with piloting a purchasing strategy as suggested in The LCBO has expanded its website to include more our recommendation. The Ministry requested that information on its pricing policy, including how the LCBO first consult with stakeholders before its mandate and provincial policy objectives affect deciding whether to proceed. The LCBO expected pricing, and details about its pricing structure. to report back to the Ministry with stakeholder The Liquor Control Act’s minimum price require- feedback by late 2013. A pilot program could begin ments apply to both LCBO retail stores and non- in the 2013/14 fiscal year. LCBO stores including The Beer Store and winery, The LCBO also informed us that it was no longer brewery and distillery retailers. A working group asking suppliers to raise quotes that, perhaps made up of staff from the LCBO, the Ministry of because of an error in calculation or changes in

Finance, the Ministry of the Attorney General, and freight or exchange rates, were too low to produce Chapter 4 • Follow-up Section 4.08 the Alcohol and Gaming Commission of Ontario has the agreed-upon retail price. It would, however, been tasked with assessing where responsibility for continue to ask suppliers to lower any quote that regulating the Liquor Control Act’s minimum price was higher than expected. requirements should reside. The LCBO expected the working group to have completed its interim report by late 2013. 346 2013 Annual Report of the Office of the Auditor General of Ontario

IDENTIFYING PRODUCT NEEDS As noted in the LCBO’s response to our audit recommendations, the LCBO had concerns with Recommendation 3 documenting the reasons for selection or elimina- To help ensure that purchases reflect corporate sales tion of products at the prequalification stage. The objectives and meet customer demand, the LCBO LCBO said it believed this process would entail should develop detailed annual category plans for the either limiting the number of submissions it would major beverage alcohol categories. accept, or hiring additional staff because of the high Status volume of submissions it receives. As a result, the The LCBO has developed detailed annual category LCBO does not believe it is practical to document plans for the major beverage alcohol categories for these decisions at the prequalification stage of the fiscal years 2013 and 2014. The LCBO indicated procurement process. According to the LCBO, as of that it would continue to develop annual category spring 2011, management has commenced oversee- plans as part of the LCBO’s annual business plan- ing and approving prequalification selections. ning process in the future. The LCBO also advised us that it has also been documenting the reasons for the selection or elimination of products for the submission stage METHODS OF PURCHASING NEW onward, as well as the required management BEVERAGE ALCOHOL PRODUCTS approvals, since spring 2011. Recommendation 4 To ensure that it can demonstrate to suppliers and ONGOING MONITORING OF PRODUCT other stakeholders that purchases are acquired PERFORMANCE through an open, fair and transparent process, the LCBO should: Recommendation 5 • develop written policies and procedures for each To help ensure that products not meeting acceptable procurement method, including the evaluation sales targets are identified in a timely manner, the criteria and process to be used in assessing LCBO should: submissions at the various stages of the procure- • regularly review and assess sales targets for each ment process; product category to ensure that they continue • disclose its evaluation criteria to suppliers, to be reasonable and appropriate for identifying including a clear articulation of all mandatory underperforming products; requirements, an indication of the relative • establish clear guidelines for the nature and weighting for rated requirements where applic- timing of action to be taken when a product is able, and a description of the shortlisting pro- identified as underperforming; and cess; and • establish policies for documenting decisions on • ensure that reasons for selection and required delisting and requesting supplier rebates. management approvals are appropriately Status

Chapter 4 • Follow-up Section 4.08 documented. The LCBO indicated that it has been setting sales Status targets annually and reviewing them for appropri- The LCBO has developed written policies and pro- ateness throughout the year. It has also developed cedures for each procurement method, including the guidelines for actions to be taken when a product evaluation criteria and process it uses in assessing is identified as underperforming, and policies for submissions from suppliers. This information has documenting decisions on delisting and requesting been published both on the LCBO’s main website supplier rebates. and on its trade resources website that suppliers use. entitling people to legal aid per capita than most most people to than legal aidpercapita entitling andissuedfewereligibility thresholds certificates lowestince, evenithadone ofthe though income any prov other than per capita legal aidsupport hadspentmore on past decade,Ontario least the government. from 2010/11, provincial from the most ofthat 2012/13the fiscal year, was unchanged which received $354 million infundingduring Ontario aslandlord–tenantissues, such disputes. Legal Aid representation issuesandtribunal assistance ity legal government clinicsto assist peoplewith it fundsandoversees 77 independentcommun alawyer; without andfamily and courts at criminal for peoplewhoarrive provide dutycounselservices manages about1,500 lawyers andcontract to staff itpays and services; for those bill Legal AidOntario retain lawyers private people whothen whointurn ways: itissueslegalin three aidcertificates to foundation for providingas the services. such bar(private-sector lawyers) private the andclinics effective andefficientmanner, while recognizing consistently inacost- high-quality legal aidservices amandate towith provide low-income peoplewith Attorney General ofthe Ministry toaccountable the isanindependentcorporation Legal AidOntario Background Chapter 4 Chapter In our provides toLegal assistance AidOntario people 4.09 Section 2011 AnnualReport

Follow-up to VFMSection3.09, to Follow-up Legal AidOntario Ministry of the Attorney General oftheAttorney Ministry , we noted for at that - - more efficient. We notedthat ithada well-defined courts andhelpmake the access to itsservices make more itsoperations cost-effective, improve deficits, ofoperating needto address ahistory the provinces. acknowledgedother Legal AidOntario cessful inmeeting itsmandate: neededto addressprogram ifitwas to befully suc However, following the were areas the someofthe direction. right strategyterm was heading inthe itsnew callcentre. houses andthrough useofdutycounselavailableexpanded atcourt through as such issuingofcertificates, beyond the it hadmoved to increase accessto legal aidservices strategylong-term issuesandthat to address these • • We multi-year long- Legal AidOntario’s felt that

Canada to help ensure that legal services legalCanada to services helpensure that Law Society the ofUpper inplacewith gram had not hadaquality auditpro assurance services. websitefrom Legal AidOntario’s for legal duty counsel,legal adviceandinformation and more clientshadbeenrequired to rely on of years hadbeenprovided certificates with meant fewer peopleover previous couple the age issued, legalcertificate billingfor each aver anescalationinthe This, combinedwith changed since1996 and1993, respectively. for qualifying hadnotcial eligibilitycut-offs finan from communitylegal clinics,andthe fied or for legal forassistance aidcertificates Since itsinception in1999, Legal AidOntario minimalornoincomequaliOnly peoplewith 2011 Annual Report Annual 2011 ------347

Chapter 4 • Follow-up Section 4.09 348 2013 Annual Report of the Office of the Auditor General of Ontario

provided by contract and staff lawyers to its measures developed. However, some recommenda- low-income and vulnerable clients were of a tions required more time to fully implement. Legal high standard. Aid Ontario was in the process of implementing its • At the time of our audit, Legal Aid Ontario strategy for modernizing and expanding financial was working to address deficiencies with its eligibility for legal aid, and reviewing the efficiency lawyer payment system. Most importantly, and effectiveness of the community legal clinic strengthening of controls was required to system in Ontario. In reforming the clinic system, ensure that all payments, which then totalled Legal Aid Ontario developed proposed clinic ser- $188 million annually, were justified. vice delivery ideas with the objective of enhancing • Legal Aid Ontario’s efforts to extract greater service levels and providing a greater range of efficiencies from community legal clinics had services within the clinics. strained its relationship with the clinics. The status of action on each of our recommen- • With the significant amount of solicitor–client dations was as follows. privileged information on Legal Aid Ontario’s information technology systems, we expected RECENT INITIATIVES it to have performed recent and comprehensive privacy and threat risk assessments of its Recommendation 1 computer databases. However, the last privacy To better inform the Legislature and the public of assessment was in 2004, and its systems had its strategic priorities and success in achieving its changed significantly since then. mandate of providing legal assistance to low-income As with our 2001 audit, we again noted that Ontarians, Legal Aid Ontario should develop and Legal Aid Ontario was lacking key performance implement meaningful performance measures on its measures on the services it provides to its clients key services and program outcomes, and enhance and stakeholders, and its publicly available annual both the information in its annual report and on its report to the Attorney General of Ontario was three website. It should also work with the Ministry of the years overdue. Attorney General to ensure that its annual report is We made a number of recommendations for made public on a more timely basis. improvement and received commitments from Status Legal Aid Ontario that it would take action to At the time of our follow-up, performance measures address our concerns. on Legal Aid Ontario’s key services and program outcomes had been developed and approved by its board of directors. These measures would be used to help reduce the number of client complaints and Status of Actions Taken on the cost per client. These measures are included in Recommendations its annual report for the 2012/13 fiscal year, which had been approved by Legal Aid Ontario’s board of Chapter 4 • Follow-up Section 4.09 Legal Aid Ontario provided us with information in directors, but had not yet been tabled in the Legis- spring and summer 2013 on the status of our rec- lature and therefore was not yet available to the ommendations. At the time of our follow-up, Legal public. Additional work on data collection systems Aid Ontario had taken action on all of the recom- was planned to enable Legal Aid Ontario to capture mendations we made in 2011. A review of financial and report on additional performance measures in eligibility guidelines and access to legal aid ser- future years. vices had been completed and new performance Legal Aid Ontario 349

Starting in the fall of 2011, Legal Aid Ontario analysis of Legal Aid Ontario’s financial eligibility began producing a new Quarterly Performance guidelines, its relationship to legal aid services, Overview Report that included updates on the access to justice and the broader justice system, and organization’s financial position, client services, an analysis on the impact of its financial eligibility legal aid certificates and payments to lawyers. The on low-income Ontarians. As a result of the study, reports are distributed to stakeholders by email and Legal Aid Ontario completed a Financial Eligibility posted on Legal Aid Ontario’s external website. In Modernization Plan for 2013/14 that includes a addition, an email-based newsletter, called LAO strategy for modernizing and expanding financial Express, that details current issues, projects and eligibility for legal aid. For example, Legal Aid other events concerning Legal Aid Ontario was Ontario is planning three financial eligibility pilot more frequently issued to stakeholders beginning projects to address legal needs in youth criminal January 2012 and also posted to Legal Aid Ontario’s justice, family law and clinic law. The strategy to external website. expand financial eligibility was approved by its The annual reports for fiscal years 2009/10 board of directors in June 2013. to 2011/12 have now been tabled in the Legisla- In spring 2012, Legal Aid Ontario began a ture and posted on Legal Aid Ontario’s external comparative analysis of its per capita costs with website. Legal Aid Ontario’s three-year strategic those of other Canadian legal aid plans. This business plan for fiscal years 2013/14 to 2015/16 project, which is expected to be completed by the has been approved by the Ministry of the Attorney end of the 2013/14 fiscal year, examines legal General and was made available to the public in aid programs in other provinces to identify the September 2013. factors and best practices that contribute to their lower costs. Legal Aid Ontario advised us that data had been analyzed and results shared with other MEETING DEMAND FOR LEGAL AID legal aid plans in January 2013. Legal Aid Ontario Recommendation 2 developed a framework for conducting an analysis To help ensure that its multi-year efforts to modernize of legal aid services across Canada and presented legal aid services result in delivering cost-effective ser- its framework at the Association of Legal Aid Plans vices to those in need, Legal Aid Ontario, in collabora- of Canada annual meeting in June 2013. tion with the Ministry of the Attorney General, should: We were advised that Legal Aid Ontario is • study the impact on low-income individuals of continuously developing service delivery options. its current financial eligibility threshold, which For instance, at the time of our follow-up it was has not been raised since 1996, and its shift to developing a new service delivery model for refu- using less costly legal aid support services; gee legal aid services to address recent changes to • assess legal aid programs in other provinces to the federal government’s new refugee and immi- identify the factors and best practices contribut- gration legislation, and pilot programs of refugee ing to their lower costs that can be applied in claims matters handled by community legal clinics

Ontario; and had begun in two locations. Chapter 4 • Follow-up Section 4.09 • continue to identify alternative ways to meet the In addition, in its 2013 budget, the provincial legal needs of low-income individuals in a cost- government announced that it is investing $30 mil- effective manner. lion over three years into Legal Aid Ontario to improve access to justice and enhance outcomes Status for low-income families, victims of domestic vio- Legal Aid Ontario’s board chair headed an aca- lence and other vulnerable groups to respond to demic study group that in April 2013 completed an evolving needs. 350 2013 Annual Report of the Office of the Auditor General of Ontario

QUALITY OF LEGAL SERVICES management that tracks changes over time. As a result of introducing these new measures, the num- Recommendation 3 ber of lawyers whose status has been conditional To strengthen its ability to ensure that consistently on at least one panel for more than two years has high-quality legal aid services are being provided as dropped from more than 800 as per our 2011 audit required by legislation, Legal Aid Ontario should: report to 230 as of March 31, 2013, a 71% decrease. assess the reasons for a high number of lawyers • In addition, the percentage of lawyers on panels on being on conditional status for panel member- a conditional basis has been reduced from 22% as ship beyond the two-year maximum time of July 2011 to 14% as of April 2013. allowed, and take timely action to ensure that Since 2007, lawyers have certified their compli- those not meeting requirements are appropri- ance with Legal Aid Ontario’s panel standards, ately followed up on; and including meeting experience requirements, by sub- either address long-standing impediments to • mitting the Lawyer’s Annual Self Report. Starting establishing a quality assurance audit program in 2012, all certificate lawyers and per diem duty with the Law Society of Upper Canada or seek counsel were required to report on activities related changes to its legislation that would allow to their conditional status. alternative means of developing and imple- We were told that, starting in September 2013, menting a quality assurance audit program Legal Aid Ontario will perform random audits of to oversee lawyers, including considering best lawyers to ensure their self-reporting has been practices in other jurisdictions. accurate, and new updated panel rosters will be Status generated based on this self-reporting. This will In 2004, Legal Aid Ontario began phasing in allow Legal Aid Ontario to follow up with lawyers standards that require lawyers to demonstrate a who fail to self-report to determine their compli- specific level of knowledge, skill and experience ance and ongoing intention to continue panel in the area of law they practice. Those who meet membership. We were also advised that a meet- the requirements are assigned to one or more of 10 ing was scheduled for September 2013 between panels to provide service in specific areas of law. Legal Aid Ontario and the Law Society of Upper New lawyers or lawyers new to a particular area of Canada to discuss introducing a quality assurance law who do not meet the experience requirement audit program. Depending on the outcome of the can be conditionally admitted to a panel if they meeting, Legal Aid Ontario may consider pursuing agree to meet the minimum experience level within legislative changes. 24 months. A conditionally approved lawyer must attend training and be mentored, as determined by BILLINGS BY LAWYERS a district area director. Conditionally approved lawyers are authorized to accept legal aid certificates. Recommendation 4 To improve monitoring of lawyers whose status To help ensure that internal controls over lawyer bill-

Chapter 4 • Follow-up Section 4.09 is conditional, in April 2012, Legal Aid Ontario’s ing and payment processing are appropriate, Legal Quality Services Office began issuing quarterly Aid Ontario should: reports to its nine district offices listing lawyers • assess the recoveries achieved in the most recent who have been admitted to the certificate panel on year’s billings using the new targeted, risk-based a conditional basis, and those who have been con- approach, and on that basis decide whether or ditional for more than two years. District managers not to proceed with an examination of billings are expected to follow up on those lawyers. The from additional prior periods; and data is summarized in a quarterly report to senior Legal Aid Ontario 351

• assess the cost-effectiveness of its investigation COMMUNITY LEGAL CLINICS activities and continue to work with the Min- Recommendation 5 istry of the Attorney General for timely access to To better address the legal needs of low-income indi- court information that is needed for verifying viduals served by community legal clinics, Legal Aid lawyers’ billings. Ontario should: Status • assess the impact of not increasing the clinics’ Legal Aid Ontario advised us that in June 2012 income threshold for determining financial its Audit and Compliance Unit implemented a eligibility since 1993; revised risk-based approach to review payments to • consider requiring clinics to capture and report lawyers. Accounts that are at high risk for inappro- on the number of applicants who are denied priate payment are selected for examination. In assistance and the reasons they are denied; addition, the Unit had developed an improved • improve the timeliness of the clinic budget risk-based fraud detection tool, which would use review and approval process; and computer analyses of past payments to identify • develop and implement performance measures inconsistencies. We were advised that the tool was for clinics that are reflective of the outcomes in use starting in fall 2013. Legal Aid Ontario told achieved, together with a quality assurance us it planned to use the new risk-based approach program that includes the quality of legal advice and fraud detection tool to examine billings going and services delivered to clinic clients. forward, but it decided that it would not be cost Legal Aid Ontario, in conjunction with representa- effective to systematically examine all accounts tives of community legal clinics, should assess the from prior years. overall effectiveness of the local clinic structure and Legal Aid Ontario’s Investigations Department consider whether any changes are possible that would had implemented an improved case file manage- help serve more clients using available funding. ment system, and we were advised that the new Status system allows staff resources to be more cost An analysis of clinic financial eligibility guidelines effectively allocated. was incorporated in Legal Aid Ontario’s overall The Ministry of the Attorney General has pro- financial eligibility study completed in April 2013. vided Legal Aid Ontario with access to its court The study noted that the clinic eligibility test, information for certain cases. A memorandum including its outdated financial eligibility thresh- of understanding governing the data-sharing olds, is likely posing a significant barrier to clinics relationship between the Ministry and Legal Aid being able to meet the needs of low-income people. Ontario is in place for the five-year period ending Legal Aid Ontario’s Financial Eligibility Moderniza- in 2016. Although access to court information that tion Plan for 2013/14 also includes a strategy for is needed to verify lawyers’ billings has improved, modernizing and expanding financial eligibility for Legal Aid Ontario advised us that the information legal aid at clinics. from the Ministry is still somewhat limited, and it

At the time of our follow-up, in addition to Chapter 4 • Follow-up Section 4.09 was continuing to negotiate with the Ministry for refreshing the information technology infra- further information at the time of our follow-up. structure in clinics, Legal Aid Ontario was implementing its Clinic Information Management System (CIMS) to modernize how clinics track and report services to Legal Aid Ontario. CIMS will require clinics to better capture and report on the 352 2013 Annual Report of the Office of the Auditor General of Ontario

number of applicants who are denied assistance with clinics from August to November 2013 to and the reasons they are denied. Draft perform- obtain more feedback. ance measures had been developed and were being consulted on with clinics. Legal Aid Ontario INFORMATION TECHNOLOGY told us that CIMS was scheduled for implementation in 2014. Recommendation 6 Legal Aid Ontario had made some progress in To ensure that information technology systems meet improving the timelines of the clinic budget review privacy, security and service level standards, Legal Aid and approval process, and had set a new target date Ontario should: of June 30 of each fiscal year to approve all budgets. • periodically assess threats and risks associated Clinic budgets for the fiscal years beginning on with its sensitive information and assets and April 1, 2012 and April 1, 2013 were finalized by take steps to manage the issues identified; and regional vice-presidents for those years by June • engage the users of the information technology 2012 and early July 2013. services in the development of key performance Legal Aid Ontario has been reviewing the measures that would provide management with efficiency and effectiveness of the clinic system information on their progress in meeting user in Ontario over the last four years. In May 2012, needs. Legal Aid Ontario released a paper called Ideas Status for the Future Development of Clinic Law Delivery In July 2012, Legal Aid Ontario established its Services in Ontario, as part of its strategic planning Privacy Impact Assessment/Threat Risk Assessment process. In addition, Legal Aid Ontario developed (PIA/TRA) program, which adopted principles proposed clinic service delivery approaches for and methodology required by all agencies and both general and specialty legal clinics, with the ministries of the Ontario government. We were objective of enhancing service levels and providing informed that PIA/TRAs had been performed with a greater range of services within the clinics. Legal all system changes effective mid-2012. At the time Aid Ontario retained third-party consultants to of our follow-up, a PIA/TRA for Legal Aid Ontario’s evaluate the proposals and were presented with an key accounting and case management IT system evaluation report in December 2012. A consulta- was nearly completed. Legal Aid Ontario had hired tion paper on clinic performance measures was a consulting firm to review the overall PIA/TRA finalized and published on the Legal Aid Ontario program and to evaluate the PIA/TRA test plans for website in March 2013. On May 16, 2013, Legal the accounting and case management Information Aid Ontario released its strategic direction for the Technology (IT) system. A report was received in delivery of clinic law services over the next five September 2013. years. The Clinic Law Service Strategic Direction Legal Aid Ontario’s IT department completed outlines the key objectives and principles that will an internal threat risk assessment on its produc- shape how the future of clinic law will be further tion servers, where a list of short and longer term

Chapter 4 • Follow-up Section 4.09 developed to improve client service in the most safeguards were identified. At the time of our cost-effective way. We were advised that how follow-up, the IT department was in the process of change will be implemented will be the subject of establishing a plan to address each safeguard. In much consultation and discussion with clinics and February 2013, a third-party consultant completed others in the months and years ahead. At the time a review of the production servers and confirmed of our follow-up, Legal Aid Ontario had plans to the setup and configuration was correct for conduct teleconferences and in-person meetings reducing the risk of threats. Legal Aid Ontario 353

Legal Aid Ontario established a new, more holistic and formal strategic corporate security program, including a plan for increasing security over its external and internal IT infrastructure that was completed in fall 2013. Its internal auditor also had plans to review the implementation of the program. IT key performance measures were formally established in 2012, and include measures for its services such as maximum phone wait times, number of incidents opened and resolved on first contact, time required to resolve issues and availability of core business application systems. Legal Aid Ontario advised us that results of performance measures are reviewed quarterly to analyze trends and identify deficiencies. In addition, an IT end- user satisfaction survey was sent out to Legal Aid Ontario staff and clinic staff in April 2013, and we were informed that the results will be used to further refine its key performance measures. Chapter 4 • Follow-up Section 4.09 Chapter 4 Ministry of the Attorney General Section 4.10 Office of the Children’s Lawyer

Follow-up to VFM Section 3.10, 2011 Annual Report

tion services is significant. The Office is unique—no Background other jurisdiction in Canada provides children with the same range of centralized legal services. Over- all, the legal and investigative work done by the The Office of the Children’s Lawyer (Office), which Office is valued by the courts, children and other is part of the Ministry of the Attorney General stakeholders. However, these services are often not (Ministry), provides children under the age of 18 assigned or delivered in a timely enough manner. with legal representation in child protection cases, We identified several areas in which the Office’s custody and access cases, and property rights systems, policies and procedures needed improve- matters such as estate matters and personal injury ment. Among our more significant findings: claims. The Office must provide legal representa- The Office’s case management system was tion for children when appointed by the court or • not meeting its information needs, and the when required by legislation in child protection and Office did not have an adequate process in property rights cases; however, it has discretion in place for evaluating the cost-effectiveness of accepting cases when the court requests its involve- its operations. For example, the Office had ment in custody and access matters. not adequately analyzed why its payments In the 2012/13 fiscal year, the Office carried out to panel agents had increased by more than its duties with approximately 85 staff (also 85 in $8 million, or 60%, over the last 10 years 2010/11), including lawyers, social workers and even though new cases accepted each year support staff. The Office also engages what it calls decreased by 20% and the Office’s overall “panel agents”— approximately 450 private law- active caseload did not change significantly yers (440 in 2010/11) and 245 clinical investigators over the same period.

Chapter 4 • Follow-up Section 4.10 (180 in 2010/11)—on an hourly fee-for-service In the 2010/11 fiscal year, the Office exercised basis. For the 2012/13 fiscal year, the Office’s • its discretion to refuse more than 40% of child expenditures totalled approximately $40 million custody and access cases referred to it by a ($32 million in 2010/11). The Office accepts about court. We found, however, that the Office had 8,000 new cases a year and, as of March 31, 2013, it not adequately assessed the impact of these had more than 10,300 open cases (11,000 in 2011). refusals on the children and courts. Many of In our 2011 Annual Report, we noted that the decisions to refuse cases were made pri- demand for the Office’s legal and clinical investiga- marily because of a lack of financial resources.

354 Office of the Children’s Lawyer 355

• Although the Office has substantially reduced other recommendations that we made in 2011. For the time it takes to accept or refuse custody example, it established new and improved criteria and access cases—from 68 days in 2008/09 to for tracking the reasons for accepting and refusing 39 days in 2010/11—it still was not meeting custody and access cases, and was looking more its 21-day turnaround target. closely into its reasons for refusing cases and into • In custody and access cases in which the reducing its refusal rates. The Office was in the Office is asked to investigate and then provide process of implementing its new CHILD case man- the court with a report and recommendations, agement system, which would help it capture and Family Law Rules require it to do so within 90 report the information it needs to address several days. However, the Office met this deadline of our recommendations. Staff had begun using the less than 20% of the time and did not have case management system and a new agent billing any formal strategy in place for improving its system was scheduled to go live in December 2013. performance in this regard. The status of the actions taken on each of our • The Office had a sound process for ensuring recommendations at the time of our follow-up was that personal rights lawyers and clinical inves- as follows. tigators were well qualified and selected fairly. However, there was no open selection process INTAKE AND REFERRAL OF CASES in place for the almost 100 property rights lawyers the Office engaged. Recommendation 1 • In 2011, the Office permitted property rights To ensure that its intake and referral services make panel lawyers to charge up to $350 an hour appropriate and timely decisions on whether to when recovering their costs from a child’s accept or reject a custody and access case and whom interest in an estate, or from trust or settle- to assign a personal rights case to, the Office of the ment funds. Yet if the same lawyers charged Children’s Lawyer (Office) should: the Office directly for their services, they were • establish criteria for accepting cases based on paid $97 an hour. the best interests of the children involved and the • The Office’s programs for reviewing the qual- benefits provided by the Office’s involvement, ity of the work performed by panel agents did and track these reasons for accepting them—the not include an assessment of whether the fees reasons for refusing cases should also continue charged were reasonable. to be tracked, but recorded more accurately, We made a number of recommendations for including noting when funding limitations improvement and received commitments from affect the decision to refuse a case; the Office that it would take action to address our • examine the impact on children and the courts concerns. of its refusal rate of more than 40% for custody and access cases referred to the Office by the courts;

• monitor the number of cases assigned to each Chapter 4 • Follow-up Section 4.10 Status of Actions Taken on in-house lawyer and panel agent, and ensure Recommendations that higher-than-normal caseloads receive the required authorizations; and establish recording and reporting systems that The Office of the Children’s Lawyer has substan- • allow management to adequately track and tially addressed some of our recommendations and monitor the time it takes to accept or reject a made progress in addressing the majority of the custody and access case as well as to assign an 356 2013 Annual Report of the Office of the Auditor General of Ontario

accepted case, and use this information to iden- staff should they attempt to assign new cases to tify any systemic reasons for delays. panel agents with more than the set maximum of assigned active files. The Office’s policy requires Status its staff to obtain prior approval from a director to The Office introduced the first phase of its new exceed the set maximum. In addition, the Office has case-management system—the Children Informa- retained additional panel agents in certain districts tion and Legal Database (CHILD)—in October that have historically experienced high caseloads 2012. The second phase, which is to include a per panel agent. portal invoicing system to allow panel agents to bill With these efforts, the Office was able to reduce the Office online, is to be implemented in December the number of legal agents carrying more than 2013. At the time of our follow-up, CHILD was not 50 cases from 15 agents in 2011 to 12 as of June able to generate reliable reports on case timelines 2013, and the most cases given to any one legal and common reasons that cases were accepted agent was reduced from 123 to 74 over the same or refused. The Office was modifying CHILD to period. We were advised that the Office intended improve reporting and this work was scheduled to to reduce this number even further by the end of be completed by March 2014. We were advised that, the 2013/14 fiscal year. For clinical cases, the Office once it becomes fully functioning, CHILD will help no longer enforces its policy of requiring clinical address several of our recommendations. agents to be assigned to prepare no more than two In February 2013, the Office established a new Children’s Lawyer Reports per month. Instead, it set of 26 criteria for determining acceptance of now uses new reports from its CHILD system to custody and access cases. The Office also increased regularly monitor that the number of cases assigned the existing 13 criteria it uses for refusing cases to to clinical agents is based on their experience, 23. The 23 criteria provide a broader number of supervision needs, writing skills and promptness reasons why a case may be refused, including when in completing assignments. In addition, the Office funding limitations are a factor. The Office has also increased its panel by 35% since 2010/11 to 245 been able to reduce its rates of refusal of new cus- clinical investigators in order to accept more cases, tody and access cases from 41% in 2010/11 to 35% increase agents’ availability in certain regions and in 2012/13, and reduce the variance of refusal rates better manage agents’ workloads, particularly so among its nine regions throughout the province. that custody and access cases could be completed The Office did not act on our recommendation in a more timely manner. The Office informed us to examine the impact on children and the courts of that it had introduced new measures to expedite its then-current refusal rate of more than 40% for senior management decisions on accepting or refus- custody and access cases referred to the Office by ing cases within five days. However, until CHILD the courts. The Office felt this examination would system reporting improvements are completed, the be time- and resource-intensive and it would be Office is unable to determine if it is consistently difficult to isolate the impact of the refusal of cases meeting its 21-day target turnaround time for from the many factors that determine the outcomes

Chapter 4 • Follow-up Section 4.10 deciding whether to accept or refuse a case. of children’s lives. To help improve case completion times, the The Office has taken steps to track and monitor Office’s intake processes were changed to obtain the number of legal cases assigned to each in-house earlier consent to gather personal information from lawyer and panel agent. The CHILD system gener- and about clients. In addition, it implemented a ates a report that enables the legal director to view new procedure that requires a director to review on a weekly basis the number of cases each lawyer weekly any cases that are not assigned to panel is assigned. CHILD provides a warning to Office agents to determine if there are delays. Office of the Children’s Lawyer 357

TIMELINESS OF COURT REPORTS tariff rates for panel lawyers would be the same as the rates paid by Legal Aid Ontario; Recommendation 2 assess whether alternatives may be available to To help improve its performance in meeting a regu- • retain appropriate lawyers for property rights lated 90-day deadline for filing Children’s Lawyer work to enable at least some reduction in the Reports with the court, the Office of the Children’s current significant premium rates being paid for Lawyer should establish a formal strategy that services billed directly to the estates/trusts or addresses the changes needed to its systems and pro- out of settlement funds belonging to the child; cedures in this area. • implement better systems and procedures for Status scrutinizing legal fees, such as post-payment As of June 2013, the Office was still unable to examinations and assessing the reasonableness generate from its CHILD system reliable reports of invoices, and for paying them within targeted for management on whether it was consistently time periods; and meeting the 90-day deadline for filing a Children’s • in conjunction with its stakeholders, research Lawyer Report to the courts. It expected system and evaluate alternative methods of payment modifications to be made by March 2014 to per- to its panel agents, such as block-fee payments, mit this. The Office has nevertheless developed that would increase financial certainty in pay- a formal strategy and action plan to improve its ments and reduce administrative processing performance in meeting the 90-day deadline and requirements and costs for the Office. some actions have already been taken. For example, Status in fall 2012 the Office began issuing interim Chil- The Office told us it could not apply the same dren’s Lawyer Reports to the courts. Specifically, empanelment process for selecting and prequalify- the interim reports are issued when circumstances ing lawyers for property rights cases as it does for prevent parties from engaging in the clinical pro- personal rights cases. The Office said that property cess or when required additional information or rights cases were more varied and it had to find assessments cannot be completed within the 90-day lawyers with suitable expertise and experience in time frame. The interim reports inform the court different areas of law, depending on the case. The and parties of the status of the work completed thus Office surveyed other jurisdictions across Canada far and invite further involvement of the Office at a and found that no formal process was used to later date if deemed necessary. identify and select panels of lawyers for property rights cases. Instead, the Office recently established PANEL AGENTS a draft plan that outlines the process and criteria for recruiting, selecting and retaining panel lawyers Recommendation 3 for property rights cases. As of June 2013, the To ensure that it has adequate systems, policies, and Office was reviewing its current external legal panel procedures for acquiring, reimbursing, and managing against these criteria and was planning to invite its legal and clinical panel agents, the Office of the Chapter 4 • Follow-up Section 4.10 additional lawyers to this pre-qualification process Children’s Lawyer (Office) should: in the fall of 2013. develop a more open empanelment process for • Following our 2010/11 audit, the Ministry lawyers hired for property rights cases similar to approved, retroactive to July 1, 2011, an increase to the sound process already in place for personal the tariff rate for panel lawyers to match the rates rights panel agents; paid by Legal Aid Ontario to its lawyers. Tariff rate further consult with the Ministry of the Attorney • increases for panel lawyers were again approved for General on establishing a process whereby the 358 2013 Annual Report of the Office of the Auditor General of Ontario

the 2012/13 and 2013/14 fiscal years. The Office PROGRAM COSTS has requested a tariff rate increase for 2014/15, but Recommendation 4 there has been no approval for matching Legal Aid To ensure that it has adequate management infor- Ontario’s rates beyond 2013/14, nor has a policy on mation on costs for services to enable it to more this been established. accurately assess the efficiency of both in-house The Office continues to pay property rights staff and panel agents over time, the Office of the lawyers a $350 hourly rate when they bill their ser- Children’s Lawyer should collect information on the vices directly to a child’s interest in an estate or to actual costs of completing its different types of cases settlement funds belonging to the child. The Office and other activities. It should also explore opportun- compared the rates paid to property rights lawyers ities for reducing its costs or enhancing its adminis- with the rates of similar organizations as well as trative capacity by collaborating with Ontario Public the private sector and its existing external panel Sector organizations that do similar legal work in lawyers. The Office concluded that the Office’s areas like property rights and in fields such as train- rate was already significantly less than the rates of ing, quality assurance and empanelment processes. private lawyers and that it would not be workable to reduce its rate and still maintain the current level Status of service. At the time of our follow-up, the Office was The Office informed us that the implementation developing a new model to improve its ability to of the second phase of CHILD in December 2013 estimate the average cost per each type of case. would improve procedures for scrutinizing legal This would help the Office decide how many new fees because it would enable electronic billing with cases it will be able to accept within its funding automated controls and verification. Invoices with limitations. The Office planned to test the model at additional hours would need pre-approval before the end of 2013 and was to then decide if it should they could be put into the system for payment be incorporated into CHILD. However, the Office to be processed. This is designed to expedite the had not determined the actual costs for completing payment process and ensure additional fees are pre- different types of cases from beginning to end and approved and reasonable. had no plans to implement a process to monitor the As of July 2013, the Office was in discussions cost of handling cases in-house and compare it to with Legal Aid Ontario on several alternative panel lawyers’ costs to assess cost-effectiveness for methods of payment, including block-fee payments, different types of cases. which are fixed fees paid for common types of The Office has not established any substantial services and which Legal Aid Ontario uses to pay new collaborative arrangements with other Ontario criminal law lawyers for certain services. Legal Public Sector organizations to reduce its costs, Aid Ontario was researching the viability of these although it told us that it regularly engages with alternative methods of payment for its family stakeholders both formally and informally, and that law lawyers. The Office planned to use Legal Aid the discussions have influenced efficiencies in some

Chapter 4 • Follow-up Section 4.10 Ontario’s research once it is completed to assess if day-to-day operations. For example, between May they would be suitable for the types of legal services and October 2012, the Office consulted with vari- provided by the Office. ous stakeholders both outside and within the public sector, identifying and implementing several recommendations to improve the Office’s child protection service delivery and make more efficient use of panel agents’ time. Also, the Office informed us that Office of the Children’s Lawyer 359 it held informal meetings with Legal Aid Ontario TRANSITION TO ADULTHOOD on a regular basis and that the offices often share Recommendation 6 information on training plans. The Office and Legal To help ensure that children’s interests continue to be Aid Ontario have jointly delivered a training session adequately protected when they turn 18 and no longer to lawyers, and further joint training sessions were qualify for the legal services offered by the Office of the scheduled for fall 2013. Children’s Lawyer (Office), the Office should establish processes that include developing and communicating INFORMATION MANAGEMENT SYSTEMS transition plans for each child, including referrals to appropriate support services. Recommendation 5 To ensure that the new case management informa- Status tion system—Children Information and Legal In 2012, the Office established a committee to Database (CHILD)—being developed will resolve examine ways it could better support youth it deficiencies in the system it is replacing and meet no longer provided services to. As a result, in current business and user requirements, the Office July 2013 the Office approved and implemented a of the Children’s Lawyer, in conjunction with Justice new policy and procedure on minors turning 18. Technology Services (JTS) project managers, should Lawyers have begun to apply it in ways such as prepare an interim report for senior management meeting with minors at age 18 to provide informa- comparing the deficiencies of the existing system to tion about their cases and resources available to the intended functionality of the new system and them, and, where litigation will continue past a identify any expected gaps or limitations in CHILD’s minor’s 18th birthday, retaining an agent who can design. The interim report should also address how continue to act for the youth after he or she turns the new system will improve safeguards for confiden- 18. The Office also revised the letter it uses to tial information and improve data integrity and case inform minors who are turning 18 that the Office file management and controls. can no longer provide services to them. The letter was recast in plain language, included definitions Status of the legal terms it used, and included referrals The Office and the Ministry’s Justice Technology to other appropriate support services such as the Services (JTS) Project Team prepared an interim Community Legal Education Ontario website, the report and gap analysis in October 2011 to describe Law Society of Upper Canada referral service and how CHILD would address existing system deficien- the Law Help Ontario website. cies. The Office informed us that by the end of the 2013/14 fiscal year, once the second phase of CHILD was implemented, the system would meet QUALITY ASSURANCE AND TRAINING 94% of the original documented business require- PROGRAMS ments. Certain desired functions were not part of Recommendation 7 the original scope of the project, such as tracking

To ensure that it is reaping the full benefits of in-house Chapter 4 • Follow-up Section 4.10 the time Office lawyers spend on each case (time- training and continuing education requirements for docketing), and other functions required further its panel agents and its own staff, the Office of the consideration. As well, the Office conducted a Children’s Lawyer should better document attendance threat risk assessment in May 2012 to ensure that at training and professional development activities the new system improved data integrity and safe- so that such activities can be considered in its panel guards for confidential information. agents’ and staff performance evaluations. 360 2013 Annual Report of the Office of the Auditor General of Ontario

Status Status The Office revised its agreements with panel The Office developed a revised set of key perform- agents to include a mandatory training require- ance indicators that it expected would be implement and had begun keeping track of panel agents’ mented by the end of the fiscal year when the new and in-house lawyers’ attendance at mandatory CHILD system becomes fully functional and is able in-class or online training sessions. Panel agents to capture and report on the necessary informa- or in-house lawyers who miss mandatory training tion. The indicators were aligned with key operat- sessions are contacted and instructed to attend ing goals and strategies of the Office and covered alternate sessions. four key areas: processes, people, financials and The Office had revised its panel members’ clients/stakeholders. 18-month performance evaluation processes to The Office advised us that plans were in place include a consideration of professional develop- as of June 2013 to develop by the end of 2013 ment and attendance at mandatory training ses- a formal stakeholder consultation strategy that sions. We were advised that annual in-house staff would include addressing how youth engage- evaluations continue to include an assessment of ment feedback will be obtained. The Office also professional development activities undertaken. informed us that it was continuing its practice of having regular informal discussions with the judiciary across Ontario on improving court processes MEASURING PERFORMANCE and making them more efficient. For example, Recommendation 8 feedback from the judiciary resulted in changes To help assess whether it is efficiently and effectively to the intake forms that parties complete and to meeting the needs of its clients and stakeholders, the the standard form orders completed by judges. In Office of the Children’s Lawyer should continue to 2012, the Office engaged in 24 consultations with develop and report key performance indicators that a variety of stakeholders, such as school boards, are clearly defined and objectively measured, estab- family justice organizations, children’s aid societies lish realistic targets, and measure and report on its and law associations, to inform them of the duties success in meeting such targets. It should also imple- of the Office and to provide opportunities for them ment a more formal process of obtaining periodic to give feedback. feedback from stakeholders, such as its child clients and the judiciary. Chapter 4 • Follow-up Section 4.10 organizations shouldbefunded. team—toteers oneach vote projects or onwhich review province—18 teams across the to 24 volun 300volunteersmore may than benamedto grant head officeand 16 regional offices.Inaddition, locatedand about120 atitsToronto full-timestaff people working organizations. inthese money paysand wages for grant salaries of the andrecreation.environment, Most of andsports andculture, arts of humanandsocialservices, organizations working areas charitable inthe ($110 million in2010/11) to not-for-profit and in 2010/11) totalling $116 more than million overdistributed 1,300 (about1,500 grants ments incommunity-based initiatives.” sector, voluntary invest capacityofthe through the by strengthening Ontario communities throughout Itsmissionisto build“healthyandvibrant ernment. in1982established gov Ontario asanagency ofthe TrilliumThe Ontario Foundation (Foundation) was Background • Chapter 4 Chapter In our The agency hasavolunteer board ofdirectors 2012/13In the fiscal year,the Foundation

$100 year to million each communitynot-for- is to give outitsallocationofmore than One of the Foundation’sOne ofthe mainresponsibilities Section 4.11 2011 Annual Report

Follow-up to VFMSection3.11, to Follow-up Foundation TrilliumOntario , we noted: - - - • • •

they support the localcommunityandrelate the support they range ofprojects canbefunded,aslong organizations. Awide charitable profit and organizations about the availability of grants availabilityorganizations ofgrants aboutthe dation coulddomore to community inform hensive andinformative. However, Foun the Foundation. to reported the they information not substantiate andperformance expenditure assessed spendingby recipients. approved, andeffectively monitored and amountsrequested grant and ableness ofthe proposals, adequately reason assessedthe compared relative ofdifferent the merits Foundation the demonstrate objectively that available littledocumentation towas often there factthat amounts. Thiswas dueto the projectsworthy were fundedfor reasonable most didnot the demonstratetion often that underlying process andresulting documenta applicantsreceive the deciding which grants, applicationandreview process forgrant operates. Foundation the context that this it iswithin may well eyebeholder, beinthe of and of valuefor money received grant for each areas listedto above. the Thedetermination We Foundation’s felt the website was compre recipients we grant Many visited ofthe could Foundation the hasawell-definedAlthough 2011 Annual Report Annual 2011 - - - - 361

Chapter 4 • Follow-up Section 4.11 362 2013 Annual Report of the Office of the Auditor General of Ontario

and about the application process. It could, GRANT PROMOTION for example, consider advertising in local and Recommendation 1 ethnic-community newspapers. To ensure that all qualified organizations get a fair Although the Foundation’s administrative • chance to learn about and apply for its grants, the expenditures are relatively modest compared Ontario Trillium Foundation should: to most other government agencies we have publicly advertise information about its grants, audited, it nevertheless needs to tighten • application deadlines, and its website; and up certain of its administrative procedures investigate ways to reduce or eliminate perceived to ensure that it complies with the govern- • or real conflicts of interest by ensuring that the ment’s procurement and employee-expense people who encourage organizations to apply guidelines. for grants are not the ones who subsequently We made a number of recommendations for help select which applications will be funded. improvement and received commitments from the Foundation that it would take action to address our Status concerns. In our 2011 Annual Report, we noted that the Foun- dation did not publicly advertise the availability of grants, and, as a result, there was little assurance that all eligible organizations were aware of the Status of Actions Taken on Foundation and its programs. We also noted that Recommendations the solicitation of applications by staff and Founda- tion volunteers raised issues of potential conflict of interest, as these same staff later reviewed applica- We concluded that the Foundation had made tions to determine who got funded. substantial progress on most of our recommenda- With respect to publicity, the Foundation tions. For example, the Foundation developed a established a new outreach and promotion target new corporate strategy with new performance as part of its performance measurement process. measurement indicators and targets; expanded Achievement of this target is measured by the ratio its promotional activities; and developed new of applications submitted to the number of applica- approaches to grant monitoring, including more tions granted. In an effort to increase this ratio, the site visits to grant recipients. It also strengthened Foundation purchased Google ads for several one- its conflict-of-interest guidance and monitoring. month periods prior to grant application deadlines At the time of our follow-up, work remained to to direct visitors to the Foundation’s website. This be done to fully implement improved goods and resulted in more than 1,000 “click-throughs” from services procurement practices, and to complete an Google to the Foundation site in each of the months assessment tool to help staff review the reasonable- of January, February and October 2012, and more ness of grant requests. than 1,000 again in February 2013. The Foundation The status of the actions taken on each of our

Chapter 4 • Follow-up Section 4.11 also developed a brochure that provided informa- recommendations was as follows. tion about its new strategic framework and sector priorities, and directed interested parties to its website and staff. At the time of our follow-up, 8,000 copies of the brochure had been produced and distributed amongst staff in the 16 catchment areas for use in their outreach activities. The Foundation identified low-demand areas in the province and Ontario Trillium Foundation 363 initiated outreach and communications activities • maintain documentation that provides a basis in those areas, including placing advertisements in for comparing one project to another to clearly local media. demonstrate why some projects were funded and The Foundation planned to continue placing others not. strategic advertisements across the province prior Status to grant deadline dates in 2013/14. It also planned In our 2011 Annual Report, we noted that although to research and pilot-test related initiatives, such regional offices were required to complete a as publishing electronic newsletters and increas- 15-question first review for each application that ing its presence on social media such as Facebook, passed an initial technical review, many of the case LinkedIn and Twitter. files we reviewed contained no evidence that this With respect to conflict of interest, the Founda- had been done. Even when the review was on file, tion put in place a new process to ensure that its it was improperly completed in half of the cases representatives who encouraged organizations to reviewed. We further noted that five of the eight apply for grants were no longer the ones who sub- offices we visited did not use the total score from sequently helped select which applications would the first review to rank projects, as intended by be funded. A new application assessment process the procedure. In the three that did, there were developed in 2012 codified this separation and unexplained instances of lower-ranked projects reinforced distinct roles for staff and volunteers by advancing in the process while higher-ranked pro- stipulating that volunteer grant review teams were jects did not. We also found the due-diligence work responsible for determining which applications completed on applications and the documentation would be approved, while staff, who conduct all of this work was often inadequate and varied sig- outreach and promotion activity, would act only as nificantly, and in some cases the required site visits advisers to these teams. were either poorly documented or not done at all. The Foundation further developed a questions- There was also no comparative documentation to and-answers document and distributed it to staff indicate why some projects were recommended for and volunteers to provide guidance on conflict-of- funding while others were not. interest issues. As well, all board members, staff At the time of our follow-up, the Foundation and volunteers had recently completed training on informed us that in early 2013 it had implemented the new conflict-of-interest requirements. a new application assessment process, aligned with its new strategic framework, to ensure that granting GRANT REVIEW AND APPROVAL decisions are based on a more objective and rigor- PROCESS ous process. The new process addresses eligibility, sector impact, community impact, feasibility assess- Recommendation 2 ments, internal review meetings and documenta- To help ensure that grant decisions are objective and tion requirements. To improve transparency, the supportable, the Ontario Trillium Foundation should: new process provides a basis for comparing one

make sure each of its regional offices completes Chapter 4 • Follow-up Section 4.11 • project to another and required documentation as the 15-point questionnaire and uses it to assess to why applications were either funded or declined. and prioritize grant applications; Finally, it indicates those steps that are to be com- develop consistent guidelines, policies, and • pleted with the help of volunteers and those that procedures for staff and grant-review teams to are not. follow when assessing grant applications, and make sure any required site visits are conducted; and 364 2013 Annual Report of the Office of the Auditor General of Ontario

The new application assessment process is sup- ensure funding is reasonable. First, in cases where ported by an online grants management system grants have been previously given to an organ- that staff use to: ization, program managers can assess the new • enter scores against each assessment submitted proposal by comparing it with the prior criterion; grants. Second, as they did even at the time of our • automate certain calculations, such as the audit, grant review teams made up of community leverage ratio of money potentially raised members have the authority to question and reject from other sources; and unreasonable amounts requested, and they often • produce reports ranking applications by score do so. Third, the Foundation had initiated a project for both the first and second review meeting, to collect external validation data for goods and and summaries for each application. services that were frequently funded (for example, salaries for various positions, consultant fees for common projects and information technology REASONABLENESS OF AMOUNTS hardware costs). A working group compiling this APPROVED data was planning to incorporate the information Recommendation 3 it gathered into the development of an assessment To help ensure that grant amounts are reasonable and tool for staff use. The Foundation planned to have commensurate with the value of goods and services to this tool available by fall 2013. be received, the Ontario Trillium Foundation should: We were informed that the lead reviewer and • assess and adequately document the reasonable- the program manager, in consultation with the ness of the specific services or deliverables organ- grantee, now conduct an assessment of the resour- izations say they will provide with the money ces required to meet the objectives of the grant they are requesting; and based on the specific expected outcomes or the • objectively assess the required work effort or grant activities. In this assessment, staff and volun- other resources needed to meet the stated object- teers use their knowledge and experience and the ives of the grant application. database of previous grants.

Status In our 2011 Annual Report, we noted that although GRANT MONITORING the biggest component of many funded projects Recommendation 4 was salaries and fees, grant files often did not The Ontario Trillium Foundation should strengthen contain the appropriate information needed from its monitoring efforts to help ensure that funds are applicants to assess the reasonableness of these used for their intended purpose, and that reported proposed costs. We also were often unable to deter- purchases were actually made, by: mine whether the grant amounts were commensur- implementing periodic quality assurance ate with the services to be provided. We also found • reviews of grant files to ensure compliance with a number of cases where there was no evidence

Chapter 4 • Follow-up Section 4.11 internal policies and requirements, and assess- that grant recipients had obtained the competitive ing the appropriateness of decisions made by bids required when buying items that cost more granting staff; than $5,000. expanding on the process undertaken by the At the time of our follow-up, the Foundation • contracted individual to include more thorough informed us it used a three-pronged approach to reviews of granting information; • requiring organizations to submit sufficiently detailed information to enable the Foundation Ontario Trillium Foundation 365

to assess the reasonableness of the amounts sultant made a number of recommendations, and in spent; response to these the Foundation said it had: • conducting more audits of progress and final • developed a risk-assessment tool to assess the reports submitted by grant recipients; and degree and type of monitoring required for • conducting site visits, where applicable, to see each grant, to be used for all applications; how grant money was spent. • placed a renewed emphasis on site visits for higher-risk grants; and Status refined a model for monitoring grant recipi- In our 2011 Annual Report, we noted that although • ents’ progress against expected results as part grant recipients were required to submit annual of the Foundation’s new performance meas- progress reports on how they used provided urement framework. funds, the process was inadequate for ensuring The Foundation further informed us it had that money was spent on its intended purpose. restructured its organization to allocate part of For example, we noted that in a number of cases its operating budget to support a new monitoring there was insufficient detail in the reports to enable function and was in the process of developing a assessment of the reasonableness of amounts spent Quality Assurance Unit that would be responsible or whether, in fact, organizations were simply for monitoring grant recipient expenditures, ensur- reporting the original budget amounts as the ing compliance with grant conditions and internal amounts eventually spent. The Foundation also audits. The Foundation said its performance score- rarely requested invoices or other documentation to card also included several new compliance targets, substantiate reported expenditures. As well, recipi- including a target percentage of grant recipients to ents were not required to substantiate performance be audited. information they provided to the Foundation, The Foundation told us that it now more often progress reports submitted by grant recipients were evaluates the reasonableness of grant recipients’ often late and there was often inadequate evidence spending, doing so both at the interim-progress- of questioning by Foundation staff about those report stage and when the funded project is com- reports. Furthermore, few site visits were made to pleted. If it needs to, it requests more information. directly assess the use of Foundation funds. Our site The most comprehensive review is at the end of the visits identified a number of instances where grant grant process, when the total amount spent, as well spending was inadequately documented by recipi- as the achievements accomplished with the grant, ents, where amounts spent appeared excessive, are reported on. Staff assess these achievements or where funds were used for purposes other than and whether value for money was received. In cases those approved. where adequate value does not appear to have been At the time of our follow-up, the Foundation had received, the reasons are explored with the grant introduced an enhanced audit plan for 2012/13 recipient, and next steps, such as requesting further that included more audits of progress and final supporting documents or using the assessment to reports, more thorough reviews of granting infor-

inform future grant decisions, are taken. Chapter 4 • Follow-up Section 4.11 mation, and site visits to grant recipients on a pilot- program basis. A total of 70 grants were subject to a new audit process, and the consultant conducting PERFORMANCE MEASURES the audits concluded in a report to senior staff and Recommendation 5 the board that, although some exceptions were To help assess whether the Ontario Trillium Founda- noted, the “vast majority” of grant recipients had tion (Foundation) is meeting its stated objectives, and spent Foundation money appropriately. The con- 366 2013 Annual Report of the Office of the Auditor General of Ontario

to help identify in a timely manner those areas need- GOODS AND SERVICES PROCUREMENT ing improvements, the Foundation should: Recommendation 6 establish meaningful operational indicators • To help ensure that the Ontario Trillium Foundation and realistic targets, and measure and publicly (Foundation) follows the government’s directives report on its success in meeting such targets; on the acquisition of goods and services, as well as and travel, meal, and hospitality expenses, the Foundation substantiate, at least on a sample basis, the • should reinforce with staff the need to comply with information obtained from grant recipients that the directives, and consider having the Ministry of is used to evaluate success in meeting targets. Finance’s Internal Audit Division periodically review Status compliance and report the results of such reviews to In our 2011 Annual Report, we noted that while the Foundation’s Board. the Foundation had developed a set of perform- Status ance measures for assessing its performance and In our 2011 Annual Report, we noted that although providing information to the public, these measures Foundation staff appeared to have an institutional were insufficient for assessing the Foundation’s mindset that emphasized cost containment, about success in meeting its objective of funding worthy half of a sample of consulting and goods-and-servi- projects in the right amounts or for identifying ces acquisitions that we reviewed and that required internal operational areas in need of improvement. a competitive selection process were instead Our evaluation of the measures that were in place single-sourced with inadequate documentation noted that they were too broad to yield meaningful justifying this single-source decision. Further, about assessments. one-quarter of these contracts were not approved at At the time of our follow-up, the Foundation the appropriate management level. We also noted informed us that as part of its new strategy it had for some employee claims for travel, meals and hos- enhanced its approach to measuring the impact of pitality a lack of detailed information supporting its grants by developing an enterprise-wide “bal- the amounts claimed and proving that they were anced scorecard” to monitor performance, based on business-related. a review of international best practices for granting At the time of our follow-up, the Foundation had organizations. The scorecard established indicators developed enhanced and clearer guidelines for pro- for measuring both the organizational performance curement and travel expenses. New travel expense of the Foundation as well as the performance of policies were finalized and communicated to staff grant recipients. New performance targets were in January 2012 and new procurement policies included in the Foundation’s January 2013 business approved and distributed in February 2012. A pro- plan, and each board meeting is now to include curement specialist had been hired for a six-month time for a discussion on these targets and indica- period to further review and strengthen procure- tors to ensure accountability. The Foundation said ment practices and continue to refine its guidelines it was aiming to complete development of new

Chapter 4 • Follow-up Section 4.11 and procedures in this area. The Foundation said it sub-indicators for this performance management expected to finish its work addressing this recom- system in 2013/14, and the new Quality Assurance mendation by fall 2014. Unit would be substantiating grant recipients’ success on a sample basis. Ontario Trillium Foundation 367

OTHER MATTER At the time of our follow-up, the Foundation had adopted the Public Service of Ontario Act’s Conflict-of-interest Declarations Conflict of Interest Policy. It had also developed a Recommendation 7 question-and-answer guidance document, which To help ensure that its conflict-of-interest policy is used real-world examples of complex potential effectively enforced, the Ontario Trillium Foundation conflict situations in the context of the Foundation’s (Foundation) should more effectively oversee and work. It intended to revise the document as staff monitor compliance with its conflict-of-interest policy knowledge of and experience with conflict issues by staff, members of the board of directors, and grant- grew. The Foundation said new declaration forms review team members. It should also require them to had also been developed and completed by all staff, update or renew their conflict-of-interest declarations volunteers and board members, and a process had annually, and include a listing of individuals and been put in place for annually updating them. In organizations with whom they have a potential con- addition, declarations of conflict of interest were flict of interest. made standing items on the agendas of the board and grant review team. In early 2013, members of Status the senior management team and board members In our 2011 Annual Report, we noted that while attended grant review team meetings to deliver Foundation staff and volunteers were required to training on the new policy and facilitate a discus- sign conflict-of-interest declarations and agree in sion on compliance. As well, conflict-of-interest writing to comply with the Foundation’s conflict- training was incorporated into a new board, staff of-interest policy, they were not required to identify and volunteer orientation process. people or organizations with whom they may The Foundation now provides program man- have a potential conflict of interest, nor were they agers with a summary report that includes the required to periodically update or renew these dec- declared conflicts of interest from the annual declarations. Also, some conflict-of-interest declara- larations for each of the volunteers on their grant- tions could not be located. review team or committee. Program managers In its response to our report, the Foundation can then direct specific applications to volunteers indicated it had instituted the annual signing of without conflicts on those files. conflict-of-interest declarations, and would investigate best practices in relation to the creation and maintenance of a list of organizations with which individuals had a potential conflict of interest. Chapter 4 • Follow-up Section 4.11 Chapter 4 Ministry of Training, Colleges and Universities Section 4.12 Private Career Colleges

Follow-up to VFM Section 3.12, 2011 Annual Report

addition, in the last three academic years (2010/11 Background through 2012/13), almost $200 million in provincial loans and grants was provided to an annual average of 13,500 students through the Ministry’s Private career colleges are independent organiza- Ontario Student Assistance Program ($200 million tions that offer certificate and diploma programs in 2007/08 through 2009/10, for an average of in fields such as business, health services and 9,500 students). information technology. They often cater to adult At the time of our audit in 2011, the Ministry students who need specific job skills to join the had undertaken a number of good initiatives to workforce or become more competitive in the job improve its oversight of private career colleges market. As of January 2013, there were 427 regis- and strengthen protection for students. However, tered private career colleges in Ontario (about 470 further improvements were needed to ensure com- in 2010/11) serving approximately 67,800 students pliance with the Act, its regulations and ministry (60,000 in 2010/11). policies, and to protect students. The following The Ministry of Training, Colleges and Univer- were some of our more significant observations: sities (Ministry) administers the Private Career Although it had taken steps to identify and Colleges Act, 2005 (Act). The Act focuses on pro- • act on unregistered colleges, the Ministry tecting the rights of students. Through the Training could have made better use of information it Completion Assurance Fund, the Act also provides already had on hand to identify colleges that students with the right to complete their training at continued to operate illegally. For example, another institution or receive a refund if the private the Ministry did not routinely check to see career college they are attending ceases operations. that schools that had been closed remained Although the Ministry does not fund private closed. We reviewed a sample of schools that career colleges directly, it provides significant fund- had been closed and found that a number ing to the sector through its employment training

Chapter 4 • Follow-up Section 4.12 appeared to be offering courses. and student assistance programs. Over the past In 2006, the Ministry stopped collecting infor- three fiscal years (2010/11 through 2012/13), a • mation on graduation rates and employment total of approximately $191 million was provided upon graduation for private career colleges, through the Ministry’s Second Career Program something it does for public colleges. More for almost 24,000 students to pay for their tuition than 85% of the private career college gradu- to attend private career colleges ($122 million in ates who responded to our survey said that 2007/08 through 2009/10, for 13,000 students). In such student outcome data would have been

368 Private Career Colleges 369

useful in helping them with their choice of risks for each private career college based on college and courses. benchmarks developed for schools offering similar • The Ministry did not have adequate processes programs. This assessment highlights risk areas in place for assessing the financial viability that may need to be addressed before new colleges of colleges when they sought to renew their are registered or during the annual registration annual registration. One college with signifi- process for existing colleges, and helps prioritize cant losses had its registration renewed with- private career colleges for inspection purposes. We out any evidence that its financial viability were informed that the Ministry completed inspec- had been reviewed. The college subsequently tions of all schools deemed to be high risk, and all closed, costing the Training Completion medium-risk schools were to be inspected within Assurance Fund more than $800,000. the first half of 2013. Consequently, the Ministry • The Ministry can enter and inspect the prem- substantially achieved its goal of processing voca- ises of a registered private career college or an tional program applications within six months, an unregistered institution that should be regis- improvement since 2011 when almost one-third of tered. Although a risk assessment done by the applications had been outstanding for more than Ministry identified 180 private career college six months. A new information system planned campuses with multiple compliance risk fac- for implementation in the 2014/15 fiscal year will tors, the Ministry could not demonstrate that further improve program administration. it had done enough inspections to manage The status of actions taken on each of our recom- the risk of non-compliance with the Act and mendations is described in the following sections. its regulations. During the 2010/11 fiscal year, there were approximately 470 registered UNREGISTERED PRIVATE TRAINING private career colleges with 650 campuses in INSTITUTIONS Ontario, but the Ministry estimated that only 30 campuses had been inspected in 2010. Recommendation 1 We made a number of recommendations for To enhance protection for current and prospective improvement and received commitments from the students of private career colleges, the Ministry of Ministry that it would take action to address our Training, Colleges and Universities (Ministry) should: concerns. • use the information at its disposal to proactively identify possible unregistered private training institutions offering or advertising unapproved vocational programs and establish a targeted Status of Actions Taken on time frame for completing investigations; and Recommendations • consider establishing standardized follow-up procedures and timelines to ensure that the unregistered institutions against which it has According to information provided by the Ministry,

previously taken enforcement action continue to Chapter 4 • Follow-up Section 4.12 substantial progress has been made on imple- comply with the Ministry’s requirements. menting many of the recommendations in our 2011 Annual Report. For example, the Ministry Status now requires all private career colleges to annually The Ministry informed us that it has reviewed all submit audited financial statements. Information vocational program pre-screening applications from these statements and other sources helps received from 2007 to 2011 to determine if any the Ministry to assess the financial and business unapproved vocational programs were being 370 2013 Annual Report of the Office of the Auditor General of Ontario

offered and has taken enforcement action where if this later review determines that the institution is necessary. In the future it intends to review all in full compliance with the Act. pre-screening applications from the previous year during the first quarter of the subsequent year to PERFORMANCE MEASURES ensure that schools are not contravening the Act. Also, the Ministry indicated that since 2011 it has Recommendation 2 been proactively identifying possible unregistered To help prospective students make informed decisions institutions and unapproved vocational programs on which private career college and which program to by surveying available media, including the Inter- enrol in, the Ministry of Training, Colleges and Univer- net, and investigating suspected non-compliance sities (Ministry) should collect, validate, and publish identified by students or registered institutions. student outcome data such as information on gradua- The Ministry has developed a risk-based frame- tion rates and employment in their field of study. In work to prioritize investigations of unapproved addition, the Ministry should use these data to assist vocational programs being advertised and/or deliv- in its oversight of the private career college sector. ered whether by unregistered or registered private Status career colleges. Investigations are categorized into The Ministry engaged a consultant to undertake a one of three levels: high-risk (involving student review of the performance indicator process that or public safety issues, such as training for truck was used before 2006, when the Ministry put a drivers and dental hygienists), complex (requiring moratorium on the collection of private career investigation jointly with other agencies, or forensic college data. After consulting with private career examinations of computer files or student and finan- colleges, current and former students and other cial records), and basic (involving schools advertis- stakeholders, the consultant recommended col- ing or offering unapproved vocational programs). lecting and reporting a number of performance The Ministry has informed us that it refines best measures. As a result, the Ministry adopted a num- practices for each level on an ongoing basis and has ber of Key Performance Indicators for the sector: incorporated them into its investigation manual. graduation rate, graduate employment rate, gradu- The Ministry has introduced protocols to track ate employment rate in the field of study, graduate all unregistered institutions against which it had satisfaction and employer satisfaction. The Ministry previously taken enforcement action. A one-time indicated that these performance indicators are review of all such private career colleges was comparable to those reported by the public college undertaken in 2011 to ensure that these businesses sector. were not engaged in any new activity that contra- In November 2012, the Ministry posted a vened the Act. In 2012, this one-time review was request for proposal for services to conduct and replaced with a standardized review process for all report on a graduate outcomes survey, an employer enforcement files. Institutions that are the subject satisfaction survey and other graduate outcomes of enforcement action will be required to confirm of Ontario’s public and private career colleges.

Chapter 4 • Follow-up Section 4.12 in writing that they are in compliance with the Act. Since that time, the Ministry has worked with the The investigator will then conduct a website check, consultant, an internal working group and a sector site visit or other check to verify within 30 days that advisory group to finalize survey materials and col- the institution is in compliance. Investigators will lection methodology. The performance-reporting keep the enforcement file open and conduct a sec- process commenced in July 2013 with private ondary review to confirm that the institution is still career colleges approved under the Ontario Student complying with the Act. The file will be closed only Assistance Program (OSAP) for graduates of 2013. Private Career Colleges 371

In this first year of implementation, the Ministry in contravention of the Private Career Colleges intends to report on three indicators (graduate rate, Act, 2005; and graduation employment rate and graduate employ- • ensure the timely review of applications for ment rate in the field of study) for OSAP-approved registration renewal, including an adequate private career colleges. assessment of financial and other application The Ministry also noted that the existing Regis- information. tration Information for Career Colleges system has Status undergone a number of upgrades and has reached The Ministry is now more rigorously assessing new its functional capacity. The cost of maintaining this applicants to run private career colleges, including vendor-owned system has become significant. Due both start-up colleges and prospective purchasers to the need for a system that can support enhanced of existing colleges. For example, applicants are regulatory oversight and reporting, quality assur- asked for business plans in addition to the routine ance, and key performance indicators, the Ministry information previously required for registration. As has begun to develop the new government-owned well, applicants are required to provide pro forma Program Approval and Registration Information financial statements prepared by a licensed public System (PARIS). PARIS is being designed to enhance accountant, a level of additional scrutiny that often the Ministry’s ability to track program applications brings to light risks that could affect the protection and help improve turnaround times for registration students are given, including a school’s financial and program approval. The new system is expected viability. to let key electronic data be linked to each college’s The Ministry has also developed and imple- record and help eliminate manual tracking, and to mented a private career college capacity assess- highlight errors and incomplete program applica- ment to assess an institution’s financial and tions. The Ministry plans to implement the new business risks. It has developed benchmarks for system during the 2014/15 fiscal year. various subdivisions of the private career college sector (schools offering similar programs) based REGISTRATION on financial indicators gathered from audited financial statements of all private career colleges. Recommendation 3 These benchmarks are to be updated annually To safeguard government funding provided to and used during the financial review portion of students and the money in the Training Completion the capacity assessment. The assessment for new Assurance Fund as well as to enhance the protection applicants includes: offered to prospective students of private career col- a credit check to assess each applicant’s finan- leges, the Ministry of Training, Colleges and Universi- • cial viability; ties should: a comparison of applicants’ financial positions ensure that its review of applications for • • with the expected initial cash flow required to private career college registrations is initiated operate a new college;

on a timely basis and includes an appropriate Chapter 4 • Follow-up Section 4.12 an Ontario Business Information System assessment of the applicant’s forecast financial • search to ensure the accuracy of corporate information, and checks on the applicant’s information supplied by new applicants; and references, credit, and criminal record; a review of ministry records related to each maintain a record of rejected applications to • • applicant to ensure there is no history of facilitate management follow-up to ensure that non-compliance. rejected institutions do not subsequently operate 372 2013 Annual Report of the Office of the Auditor General of Ontario

Where risks are identified, applicants are invited strategies. These strategies often include conditions to meet with ministry management to develop miti- on registration such as more frequent enrolment gation strategies such as limited program approval reporting, preparation of business plans and periods and conditions of registration (e.g., addi- increased financial security. tional reporting requirements, increased financial To enhance transparency and the protections security). Beginning in 2013, as part of this new offered to students of private career colleges process, the Superintendent of Private Career Col- through the Training Completion Assurance Fund, leges will meet with new applicants for registration the Ministry engaged a licensed public accountant on a selective basis to clearly articulate ministry to audit the 2011 financial statements of the Fund. expectations on compliance. The Superintendent The Ministry publicly released the audited state- may implement further conditions on a school’s ments in September 2012, which showed a balance initial registration to mitigate student risk. of almost $10.5 million as at December 31, 2011, In 2012, the Ministry developed operational and intends to have the Fund audited annually. In policies and benchmarks to assist with meeting ser- addition, in November 2012, the Ministry released vice timelines for the initial review of registration the results of a satisfaction survey of students applications of new private career colleges. During affected by a school closure and financially pro- the 2012/13 fiscal year, the Ministry approved 18 tected by the Fund. The survey indicated that 74% new private career colleges and 13 new campuses. of students affected by a closure were satisfied with Over the last year, many private career colleges that the administration of the Fund and that students have had difficulty submitting complete applica- were generally satisfied with services provided by tions for registration have been offered tutorials the Ministry and by the institution at which they with ministry management, because the Ministry’s completed their training. ability to meet its service commitments is directly related to the quality of the applications received. PROGRAM APPROVAL The Ministry plans to continue working with applicants to improve the quality of submitted registra- Recommendation 4 tion applications. To enhance the quality of private career college pro- Since February 1, 2011, the Ministry has been grams and to ensure that all programs, regardless of tracking and periodically following up on refused, which college is offering them, provide the skills and rejected and abandoned applications for new knowledge currently necessary to obtain employment private career colleges and campuses to ensure in the prescribed vocation, the Ministry of Training, that they are not operating in contravention of the Colleges and Universities should: Act and its regulations. It has started to perform a • review the processes in place to assess the similar review of all private career colleges that had qualifications and independence of the general cancelled their registration during the previous year. third-party program assessors that provide The Ministry now requires registered private recommendations for program approval;

Chapter 4 • Follow-up Section 4.12 career colleges to provide audited financial state- • maintain a record of rejected program applica- ments to allow it to highlight high-risk institutions. tions and consider implementing follow-up The Ministry is also using the new capacity assess- procedures to ensure that such programs are not ment process to assess the financial and business offered despite their not being approved; risk of each private career college at the time its • build on the progress made to date in improving registration is renewed. When a private career col- the timeliness of the program approval process lege is deemed to be a high risk, the Ministry works and develop a plan for program re-approvals; with the institution to develop risk-mitigation and Private Career Colleges 373

• enhance its system so that it can provide the tion to mandatory updates resulting from changes information needed to effectively manage the to the standards, the Ministry indicated that it program approval process. would explore options for re-approving existing programs in time to submit recommendations in Status September 2013 for legislative review. In September 2010 the Ministry enhanced its quality assurance processes by requiring every new vocational program to receive a favourable assess- LEGISLATIVE COMPLIANCE ment from both an adult education expert and a Recommendation 5 program subject-matter expert. The Ministry has To enhance the level of compliance with the Private also improved the third-party program evaluation Career Colleges Act, 2005 and its regulations, and to process by redesigning ministry forms and informa- provide better protection to students and prospective tion materials as well as standardizing the process students of private career colleges, the Ministry of for validating assessors’ credentials with regulatory Training, Colleges and Universities should: institutions. The Ministry now tracks those asses- undertake enough inspections to adequately sors for future reference and evaluation. • manage the risk of non-compliance; In 2010, recognizing that incomplete program clarify the focus and extent of testing that applications need significantly more time to process • inspectors should perform during the course of and can result in approval delays, the Ministry an inspection of a college; launched a series of communications to inform implement appropriate management oversight private career college administrators of common • procedures to enhance the quality and con- issues related to incomplete applications. The sistency of college inspections; and Ministry also contacted private career colleges to aggregate and analyze inspection results to inquire about the status of incomplete program • identify trends and systemic issues that warrant applications. As a result, many of these applica- further attention. tions were withdrawn. Incomplete submissions account for the majority of program applications Status that have not yet been processed. Also, since In 2011, the Ministry assessed all registered February 1, 2011, the Ministry has been tracking private career colleges for inspection against a refused or rejected program applications as part of risk-management framework and ranked each its ongoing monitoring and will continue to do so school as a high, medium or low risk. All high-risk in future inspections to ensure that these programs schools were inspected. The Ministry informed us are not being offered in contravention of the Act. that all medium-risk schools were inspected in the During the 2012/13 fiscal year, the Ministry first half of 2013. The Ministry stated that in the approved 570 program applications. The majority future it would ensure that all high-risk schools of unresolved applications had been outstanding are inspected within three months of being identi- for less than six months. Consequently, the Ministry fied as such and that all medium-risk schools are has substantially achieved its goal that no program inspected within 24 months of being identified. Chapter 4 • Follow-up Section 4.12 application would await review for more than six The Ministry also indicated that it will implement months. The Ministry has developed new common a system in the 2014/15 to 2015/16 fiscal years to standards for certain programs and has required prioritize low- and medium-risk schools for inspec- all private career colleges offering those programs tion. The Ministry also noted that the new capacity to resubmit applications for review to ensure their assessment process would ensure that each private programs comply with the new standards. In addi- career college has its financial and business risks reassessed when it renews its annual registration. 374 2013 Annual Report of the Office of the Auditor General of Ontario

The Ministry developed a new pre- and post- • more effectively communicate to students that inspection checklist to spell out the extent of testing they are entitled to escalate unresolved com- that inspectors should perform. To clarify the focus plaints to the Ministry. and extent of the testing, the Ministry stated that Status inspectors now meet weekly with management to The Ministry receives complaints from sources review recent inspection reports and discuss any including students and the general public. When actions required. students have not gone through their college’s The Ministry indicated that it developed a basic internal complaints process, they are directed case management program to track current inves- back to the college. When a student complaint is tigations and to archive closed files in a searchable submitted after the student has gone through the database. The Ministry is adapting this program to college’s complaints process, the Ministry informed generate reports to track all investigations. Enforce- us that an initial acknowledgement is now sent to ment staff have participated in the development of the student within four business days. The Ministry the PARIS system to help ensure that it will enhance contacts the college regarding the complaint and oversight by linking key electronic data, such as requests a response, including relevant documen- administrative penalties and conditions of registra- tation, to be submitted within 15 business days. tion, directly to each college’s record. The Ministry Once the Ministry receives the college’s response, it expects that the efficiencies provided by the new determines whether the evidence shows that the Act system will enable its staff to focus more of their has been violated. If it does, the Ministry notifies the efforts on compliance and enforcement, as well as student and instructs the college to take corrective on general oversight and student protection. action. If the evidence does not show a violation of The Ministry has developed a new database the Act, the student and the college are notified, to track compliance trends and identify systemic and the student may be provided with contact issues that warrant further attention. The database information to take further action, if appropriate, facilitates management review of inspection reports such as pursuing the matter in small claims court or and relevant inspection details. Trend reports are to with the Ministry of Consumer Services. Approxi- be used to address common compliance issues and mately 40% of student complaints that have gone inform the legislative review. through private career colleges’ complaint processes cannot be addressed under the Act. The Ministry STUDENT COMPLAINTS informed us that it has developed a formal protocol for addressing student complaints and expects this Recommendation 6 protocol to be in place by the end of 2013. To help ensure that the protections offered by legisla- The Ministry has established a student com- tion to students of private career colleges are effective plaints tracking process to help identify trends and and to enhance management’s ability to oversee the issues that may require follow-up. This tracking complaints process, the Ministry of Training, Colleges process has been put into a database (similar to

Chapter 4 • Follow-up Section 4.12 and Universities (Ministry) should: that used to track inspections) that has been in use establish target time frames for resolving • since April 2013. The Ministry noted that student complaints and for receipt from colleges of the complaints that reveal non-compliance at a private information necessary to address complaints; career college are one of the risk factors now used in analyze complaints to identify possible issues or • its new risk-management framework for inspections. trends that may require more focused action; As a condition of registration, each private and career college is required to provide every student Private Career Colleges 375 with a copy of the school’s student complaints private career college and programs that best procedure, its expulsion policy and the regulation meet their vocational goals; and outlining refund policies. The student complaints • work with private career colleges and their asso- procedure must outline the personnel students can ciations to ensure that student-oriented com- contact to file a complaint, the manner in which munications are user friendly and communicate complaints should be submitted and the process the in a fair and transparent manner the protec- college follows in dealing with student complaints. tions offered to students who attend registered As well, every student must be given the Ministry’s colleges and programs. “Statement of Students’ Rights and Responsibil- Status ities,” which further outlines the steps a student The Ministry stated that it continues to review can take in filing a complaint, including filing a its communications and consultation strategy to complaint with the Ministry. ensure that all stakeholders are informed of upcom- The Ministry indicated that it would continue ing initiatives and new or changing regulatory to partner with the private career college sector requirements. The Ministry has implemented regu- and with other consumer protection institutions lar quarterly sector updates from the Superintend- to ensure that students are provided with consistent of Private Career Colleges to better disseminate ent and accurate information on the protections information to the sector. The Ministry also indi- afforded to them under the Act. Also, the Ministry cated that outputs from the new system (PARIS) obtains information from agencies and regulatory would provide information on Key Performance bodies at all levels of government, including units Indicators to the public, including students, their within the Ontario government, federal government families and employers, so that students can make agencies and student-funding agencies, to identify informed decisions when choosing a program. student-protection issues such as concerns about The Ministry stated that it would continue to program quality and equipment used in courses. work with private career colleges and their associa- The Ontario Public Service has been working tions to ensure that information for students is easy on the development of a new website intended to to find and helps them understand the protections facilitate public access to such information. The new afforded to them under the Act. In August 2011, website is intended to outline the student complaint the Ministry partnered with the largest sector process at private career colleges, including how to association to complete a number of enhancements submit complaints to the Ministry, if necessary. We to the Ministry’s public website. Also, to promote were told that this initiative would be implemented student awareness of the standards required of in phases throughout 2013 and onwards. some higher-risk vocational programs, the Ministry has posted its vocational program standards on its PUBLIC AWARENESS public website. The Ontario Public Service website under development is being designed to bundle Recommendation 7 ministry content by theme and audience to better

To enhance protection offered to students and pro- Chapter 4 • Follow-up Section 4.12 capture initiatives that concern more than one spective students, and to ensure that the private career ministry. The new website is intended to allow college sector is not unfairly affected, the Ministry of private career colleges, students and other sector Training, Colleges and Universities should: stakeholders to more easily access the information periodically evaluate the effectiveness of its com- • they require in a timely manner. munication strategy to identify opportunities for improvement in helping students choose the Chapter 4 Ministry of Education Section 4.13 Student Success Initiatives

Follow-up to VFM Section 3.13, 2011 Annual Report

at 81% for the 2009/10 school year, and for the Background 2011/12 school year the Ministry announced a graduation rate of 83%. However, in our 2011 Annual Report we did note areas where refinements Ontario’s Student Success Strategy is a collection of to the initiatives would help ensure that the Ministry initiatives that the Ministry of Education (Ministry) can meet its objectives and that students acquire has implemented since 2003 to help secondary the knowledge and skills they need to go on to post- school students graduate with their high school secondary education or employment. Some of our diplomas. At that time, the Ministry reported a five- observations at that time included the following: year cohort graduation rate (the graduation rate as Ontario school boards we visited track risk fac- a percentage of the grade 9 population five years • tors such as gender, absenteeism and course previously) for the 2003/04 school year of 68%. success to help identify students at risk and The Student Success Strategy aimed to raise the then provide them with supports. However, we graduation rate to 85% by the 2010/11 school year. noted that some other jurisdictions have found The Ministry’s Student Achievement Division that targeting supports to specific groups of is responsible for developing and monitoring the students based on factors such as ethnicity, Student Success Strategy, while school boards disability and economic status has been very and schools are responsible for delivering the effective in improving graduation rates. strategy’s initiatives. Every board receives funding The Ministry’s reported graduation rate is for one student success leader to help implement • based on calculating the percentage of Grade 9 programs in its schools, as well as funding for one students who graduate within five years. student success teacher per secondary school who However, the 2009/10 graduation rate would

Chapter 4 • Follow-up Section 4.13 is responsible for providing supports to students at have been only 72% if it had been based on risk of not graduating. In the 2012/13 school year, graduation within the four-year span of high the Ministry provided approximately $150 million school. On the other hand, the graduation rate to school boards for the delivery of student success would have been 91% if it had been extended initiatives (almost $130 million for the 2010/11 to when students reach the age of 25. school year). The Ministry relies primarily on tracking The Ministry has made steady progress toward • changes in the graduation rate to measure the goal of an 85% graduation rate. The rate stood the outcome of the Student Success Strategy.

376 Student Success Initiatives 377

However, graduation rates are generally not study was conducted to determine why students publicly available by school board, and boards disengaged (i.e., left school before graduation), do not use a consistent method of calculating what was needed for them to successfully return graduation rates, so it is difficult to meaning- to school and what preventive measures could be fully compare rates across the province. Better put in place that would decrease the probability of information is also needed on graduates’ level students disengaging in the first place. The Ministry of preparedness for post-secondary studies informed us that, through this initiative, more than and employment. 8,000 students returned to school in the 2011/12 • We noted situations where the work place- school year to work toward their diplomas. As a ments in the Cooperative Education program result, funding for this initiative was continued for did not appear to complement the students’ 2013/14. Also, the Ministry has performed much of curriculum requirements for in-class learning. the initial work required to begin publishing school Students earned credits in a wide range of board graduation rates and to better track gradu- placements, such as clothing stores, fast-food ates who pursue post-secondary education and outlets, coffee shops and laboratories. assess graduate outcomes. • In the 2009/10 and 2010/11 school years, The status of actions taken on each of our recom- only $15 million of the $245 million the mendations is described in the following sections. Ministry provided to school boards for student success initiatives was allocated based on a MEASURING AND REPORTING ON direct assessment of student needs. Much of STUDENT OUTCOMES the remaining funding was allocated based on the number of students in each board, rather Recommendation 1 than being targeted to the boards, schools and To help the Ministry of Education (Ministry), school students most in need of support. boards, and schools generate timely data for decision- We made a number of recommendations for making purposes that are consistent and comparable, improvement and received commitments from the the Ministry and the province’s school boards should: Ministry that it would take action to address our • set reasonable targets for graduation rates and concerns. student success indicators in line with overall provincial goals and require more formal reporting on the achievement of these targets at the provincial and school board levels; Status of Actions Taken on • develop a common method for school boards to Recommendations calculate and report graduation rates and other student success indicators; help school boards share best practices that According to information received from the Min- • would assist in the more timely verification and istry, progress has been made on implementing

submission of student data; Chapter 4 • Follow-up Section 4.13 all of the recommendations in our 2011 Annual consider collecting information on high school Report, with substantial progress made on several. • graduates to identify any gaps in knowledge or For example, the Taking Stock report that monitors skills that may require attention; and students as they progress through secondary extend the use of the proposed student identifier school has been revised to better identify students • number to include private career colleges. considered potentially at risk of not graduating. Also, as part of the re-engagement initiative, a 378 2013 Annual Report of the Office of the Auditor General of Ontario

Status The Ministry indicated that it has enhanced its The Ministry informed us that it has worked with data confirmation procedures to support school school boards to identify and share effective data boards with their submission of consistent and collection, submission and verification practices to comparable student data. In addition, the Ministry assist in establishing reasonable targets for board implemented a strategy to mitigate barriers boards graduation rates, creating common calculation face in verifying and submitting student data. As a and reporting methods, and monitoring students result, starting with the March 2012 submission, beyond secondary school. For example, the Min- data collection for publicly funded school boards istry revised its Taking Stock report in the 2011/12 was 100% complete in four months. and 2012/13 school years to collect more refined Since the Ministry plans to publish board-level data on secondary school students, including: graduation rates in March 2015, it will need accur- • the number of students identified to be poten- ate, timely data from all school boards. The Min- tially at risk of not graduating; istry provided funding to 10 school boards through • the progress of all students toward the a data quality improvement effort in 2011/12 to Ontario Secondary School Diploma (OSSD) identify best practices for data submission and requirements; verification, and in February 2013 the best practices • the progress of students enrolled in the Super- identified through this project were shared with all vised Alternative Learning (SAL) program; school boards. and The collaborative College Mathematics Project • the progress of part-time students under age analyzed mathematical achievement of first-semes- 18 and re-engaged students (i.e., returning to ter college students in relation to their secondary school after leaving the system). school mathematics backgrounds since 2008. The To improve the monitoring of students at risk results, along with ways to increase student success of not graduating, the Ministry held sessions with in college mathematics, were discussed with col- Student Success School and Cross Panel Teams on leges and high schools. The Ministry informed us using and sharing student success indicator and that this project has been expanded to include both Taking Stock data, and it also works with Managing language and mathematics achievement of recent Information for Student Achievement leaders in Ontario graduates in their first and second semester the boards. of college. In addition, for 2013/14, the Ministry Starting January 2013 the Ministry added the plans to undertake a study of other jurisdictions to Grade 11 credit accumulation indicator for all identify effective practices in identifying gaps in school boards to the board progress report on the high school graduates’ knowledge and skills. Ministry’s public website. The Ministry has also Since April 2011 the Ministry of Training, decided on a plan in principle for implementing Colleges and Universities (MTCU) has been work- a common method for calculating and reporting ing with the post-secondary sector to extend the board-level graduation rates, which it believes may Ontario Education Number (OEN), and all Ontario

Chapter 4 • Follow-up Section 4.13 motivate continued momentum in Student Suc- colleges and universities have enhanced their cess initiatives, act as a performance measure for systems to accommodate the OEN, validate educa- the boards and provide greater transparency. The tion numbers and request the assignment of new Ministry’s next steps are to calculate school board numbers. With this new capacity, post-secondary graduation rates for the 2012/13 school year in institutions have been processing their existing March 2014, and to publish school board graduation student files and have been able to assign 307,000 rates for the 2013/14 school year in March 2015. new OENs to students for whom an OEN did not previously exist. To ensure that the same student Student Success Initiatives 379 number follows a student from high school to post- Status secondary education, institutions need to update The Ministry revised its Taking Stock data report privacy notices for student data collection and con- for the 2012/13 and 2013/14 school years to tact students to resolve mismatches. The Ministry include more explicit guidelines to identify students indicated that 90% of funded graduate and under- considered potentially at risk of not graduating. graduate students are expected to be set up by the To ensure consistency in board definitions, the fall of 2013. The 2014 student enrolment report is revised report requires boards to deem all students expected to contain OENs for all students registered who failed a specified number of courses to be at with Ontario colleges and universities. risk of not graduating. In April 2013, regional ses- The Ministry informed us that a review with sions were held to present school boards with the MTCU of privacy legislation is planned in 2013/14 provincial perspective on identifying students as to provide a clear authority to link personal infor- potentially at risk. mation associated with the OEN from high school The Ministry indicated that identifying students to post-secondary education. The Ministry noted or student groups at risk of not graduating on the that the data collection systems and business pro- basis of student ethnicity, language and socio- cesses at private career colleges will likely not meet economic status raises issues of privacy, fairness its authentication requirement. Consequently, it is and sensitivity. Taking these considerations into assessing the feasibility of enhancing system secur- account, the Ministry completed in the summer of ity to allow for a self-authentication and retrieval 2012 a detailed analysis of options and recommen- model for students at private career colleges. In dations for reporting on the achievement of self- addition, the Ministry is considering amending the identified Aboriginal students. The Ministry is also regulations to provide the legal authority for OEN working with the Ministry of Children and Youth use, assignment and information collection with Services to improve the educational outcomes of private career colleges. children and youth in the care of, or receiving services from, Children’s Aid Societies. This includes promoting the development of protocols between STUDENTS AT RISK OF NOT GRADUATING the education and child welfare sectors and Recommendation 2 identifying opportunities for better linkages and To help identify students and student groups at risk programs to benefit these students. of not graduating who may benefit from additional In spring 2013, the Ministry prepared materials and specific supports and programs, the Ministry of on standardized criteria for analyzing and reporting Education and the province’s school boards should: on data on student sub-populations. The Ministry • establish a common definition for reporting stated that it is analyzing options for producing grade 9 and grade 10 students considered at risk student success indicators for sub-populations, of not graduating; such as students attending schools with particular • assess the viability of calculating student success socio-economic attributes. In addition, the Ministry

indicators by a variety of attributes such as eth- intends to use student success indicator data to pre- Chapter 4 • Follow-up Section 4.13 nicity, language, and socio-economic status, and pare reports at the provincial, board and school lev- consider a system or process for collecting data els that group students by gender, special education based on student self-identification; and status and English language learner performance. • review the processes used to record students In fall 2012, the Ministry approved a proposed who leave school without a diploma so that the list of new student mobility codes that are intended reasons students leave school can be determined. to improve the quality of the data collected from school boards when a student transfers or leaves 380 2013 Annual Report of the Office of the Auditor General of Ontario

school. The new codes should assist the Ministry in Status better understanding the destinations of students The Ministry continues to assess the re-engagement who leave Ontario schools. Throughout the winter initiative by comparing school board data submit- of 2012/13, the Ministry met with representatives ted through the Taking Stock report and found that from school boards to communicate the proposed in the 2011/12 school year more than 16,000 stu- changes in student mobility codes and receive feed- dents (10,000 students in 2010/11) were contacted back. The Ministry plans to begin using the new to return to school and more than 8,000 students mobility codes for the 2013/14 school year. (5,000 students in 2010/11) were re-engaged The Ministry’s new re-engagement initiative— through this initiative. Although funding for the Next Steps—builds on the success of a 2010 initiative re-engagement initiative was expected to wind to re-engage students who were close to graduating down, the Ministry informed us that, given the by integrating information gathered from a variety encouraging results to date, funding will again be of sources. The initiative focuses on how to encour- provided to school boards in the 2013/14 school age students with significant credit accumulation to year to contact students who are near to graduation successfully return to school and what preventive but have left the system and then mentor and mon- measures would decrease the probability of such itor their transition back to the school system. The students disengaging in the first place. Ministry also noted that it was planning to make a concerted effort to re-engage all self-identified Aboriginal students who have left the system or STUDENT SUCCESS STRATEGY are at risk of not graduating because they have not INITIATIVES accumulated sufficient credits. Recommendation 3 All Ontario secondary students must complete To ensure that student success initiatives increase the 40 hours of community involvement as a graduation number of students who obtain their Ontario Second- requirement. In the summer of 2012, the Ministry ary School Diploma and are adequately prepared for requested assistance from members of the Ontario college, university, apprenticeship, or the workforce, School Counsellors’ Association (OSCA) in col- the Ministry of Education and the province’s school lecting best practices for helping students achieve boards should: their community involvement hours. In fall 2012, • assess the re-engagement initiative to determine the Ministry participated in working groups with if the benefits that boards had noted justify the several organizations (Volunteer Canada, Change cost of maintaining the program in future years; the World/MCI, the Ontario Volunteer Centre Net- • disseminate best practices or guidance for help- work, Charity Republic, OSCA, Free the Children) ing students achieve their community service interested in helping students find volunteer oppor- hours before graduation; tunities and promoting the spirit of volunteerism • better link work placements in cooperative edu- within the schools. At the time of our follow-up, cation with course expectations to ensure that three separate organizations were developing

Chapter 4 • Follow-up Section 4.13 the placements complement the in-class experi- systems designed to enable students to track their ence as required; and volunteer hours online, provide an approval process • assess the Credit Recovery program to determine for charities to acknowledge student participation, whether students are achieving the required and enable school administrators to monitor student course expectations, and consider more detailed volunteer hours. These systems were expected to guidelines to ensure consistent program delivery be piloted in the 2013/14 school year. In May 2013, across the province. the Ministry conducted regional training sessions with one guidance counselor from each secondary Student Success Initiatives 381 school to provide information on these systems and • allocate demographic funding based on the most share promising practices for supporting students in recent data available; and acquiring the 40 hours of community involvement • assess the cost and benefits of ministry delivery required to graduate. of the Dual Credit program. The co-operative education program allows Status students to earn secondary school credits while To help target funds to those students most in need completing a work placement in the community. of assistance, the Ministry modified its funding In spring 2012, ministry staff engaged the Ontario approach in some programs and sought models that Cooperative Education Association executive and show positive outcomes for vulnerable groups of other regional co-operative education associations students that could be adopted across the province. in discussions on practices related to personalized For example: placement learning plans and improving linkages The Ministry informed us that to address the between students’ co-operative education experi- • needs of additional low-performing schools, ence and in-class credits. In the summer of 2012 the it expanded the School Support Initiative Ministry contracted four school board co-operative (SSI) from three boards (27 schools) in education co-ordinators to document best practices. the 2008/09 school year to 26 boards (116 In the spring of 2013 the Ministry released a new schools) in 2012/13. As well, it determined resource for co-operative education teachers and that 55 of the participating schools demon- school and board administrators highlighting best strated increased achievement in 2011/12 practices, including matching placements to cur- such that they no longer receive SSI funds. riculum expectations. The Ministry also informed us that, for the To help ensure consistency throughout the prov- 2013/14 school year, the program would pro- ince in the Credit Recovery program, which gives vide additional funding to current SSI-eligible students who have failed a course another chance boards to better target the gaps in student to demonstrate understanding of specific topics learning and achievement for specific identi- rather than retaking the entire course, the Ministry fied sub-populations such as self-identified has drafted a proposal to review the program for Aboriginal students. The model is to be the 2012/13 and 2013/14 school years. reviewed and may be further revised pending an assessment of the 2012/13 funding model. STUDENT SUCCESS FUNDING • In June 2012, the Ministry contacted all boards to find additional participants for the Recommendation 4 second year of a pilot to explore effective To ensure that Student Success Strategy funding is practices to assist students in the Supervised spent efficiently to address the specific needs of stu- Alternative Learning program. This program dents at risk of not graduating, the Ministry of Educa- attempts to re-engage students 14 to 17 tion and the province’s school boards should: years old who are not attending school. For

adopt funding methods that target more money Chapter 4 • Follow-up Section 4.13 • 2012/13 this pilot program was expanded for schools and boards where students at risk from five to 10 boards to continue the focus on most need the assistance and work with the developing strategies and templates for track- boards and schools to better estimate student ing Supervised Alternative Learning students’ participation in application-based programs; achievements in non-credit learning activities. improve existing processes to monitor board • In January 2013, the Ministry invited all school expenditures and ensure that overfunding is • boards to submit applications for funding properly accounted for; 382 2013 Annual Report of the Office of the Auditor General of Ontario

to run pilot programs focused on innovative support the implementation of the School Support delivery models to improve educational out- Initiative. In addition, to help ensure funding is comes for secondary school students in the spent appropriately, education officers reviewed care of, or receiving services from, Children’s and analyzed the 2011/12 board expenditure Aid Societies. The Ministry noted that the reports with respect to the specific student success educational outcomes of children and youth in strategies being implemented. In December 2013, care are significantly lower than those of the the Ministry plans to conduct a similar review of general population, and that it is committed the 2012/13 expenditures. The Ministry also stated to providing targeted, personalized support that unspent funding in Dual Credit programs to meet the learning needs of these students. was being reallocated to other programs and the Funding has been allocated for the boards that Council of Ontario Directors of Education returned will be selected to run pilot programs in the $6.3 million of unspent funds to the Ministry of 2013/14 school year. Finance in the fall of 2012. • Now that sufficient self-identified Aboriginal Using the most recent census data available to student data is available, student success calculate demographic funding resulted in signifi- activities are being enhanced in boards and cant redistributive impacts on school boards. To schools where there are larger numbers or limit the impact on individual school boards, the proportions of self-identified First Nations, Ministry is phasing in the funding reallocations Métis and Inuit students. over four years. During the phase-in period some The Ministry has been looking at historical boards will see their funding decrease and other trends in the number of students enrolled in the boards will see progressive funding increases. The two major application-based programs: Specialist redistribution of demographic funding will be com- High Skills Major (SHSM) and Dual Credit. The pleted in the 2014/15 fiscal year. Ministry reduced funding to these programs after In June 2012, the Ministry engaged a consultant identifying that the projected numbers of students to conduct a review and assessment of the manage- exceeded the actual number participating in previ- ment of the Dual Credit program and to provide ous years. Some programs with consistently low options and recommendations for its future man- enrolment in comparison to the overall school agement. The focus was on analyzing the costs and population are to be reviewed for the 2013/14 benefits of transferring the delivery of the program school year. The Ministry also indicated that adjust- from the Council of Ontario Directors of Education ments to funding for school boards and colleges for (CODE) to the Ministry. The consultant provided a the SHSM and Dual Credit programs will continue number of program recommendations and manage- to be based on student participation data. ment options, outlining the implications of each. The Ministry informed us that it revised The Ministry selected the option to strengthen and expenditure reporting templates, which are now improve the current delivery process but transfer monitored more frequently to ensure that school some of CODE’s responsibilities to the Ministry. The

Chapter 4 • Follow-up Section 4.13 boards are spending funding according to the stated Ministry noted that several of the report’s recom- contract deliverables and in a timely manner. For mendations relating to CODE’s responsibilities have example, in the 2012/13 school year, ongoing visits been put into practice and used to develop a 2013 were conducted to monitor board practices and to action plan for the program. Passport provides supports and services exclusively andservices provides Passport supports 1, As ofApril 2012, andsupports. apply for services as“single(DSOs) now windows” serve for adultsto organizations Ontario Nine Developmental Services to andsupports. apply for developmental services implemented anew Ministry process for people the TransformationDevelopmental Services project, over 15,300 serving program, adults. spentover Ministry $96 milliononitsPassport the Aswell, program). 24,000 familiesunderaformer (in2010/11,children itspent$99 millionserving received approximately $42 millionto serve 12,500 atHome(SSAH) program istered SpecialServices people (134,000 in2010/11). TheMinistry-admin to about132,000 providedthat services eligible payment agencies (412 in2010/11) innineregions approximatelythrough transfer- with 390contracts including $422 million($472 million in2010/11) lion ($571 million in2010/11) programs, onsuch 2012/13 fiscal year, spent$561the Ministry mil inawiderangeand participate ofactivities. Inthe abilities live athome,work communities intheir developmental to dis helppeoplewith programs services ofsupportive fundsavariety (Ministry) ofCommunityandSocialServices The Ministry Background Chapter 4 Chapter In July 2011, long-term Ministry’s ofthe aspart Section 4.14

Follow-up to VFMSection3.14, to Follow-up People withDisabilities for Services Supportive Ministry ofCommunityandSocialServices Ministry - - - ces exclusively andyouth. for children for adults,andSSAH andservi provides supports midst ofacomprehensive Developmental Services was inthe Ministry payment the agencies. Although public fundswere properly managed by transfer- were quality provided services ensure that andthat oversight procedures were still not adequate to developmental disabilities. TheMinistry’s with inacost-effectivelevel mannerto ofsupport people cies were providing andconsistent an appropriate agen adequate itsservice-delivery that assurance still didnot havefactorily addressed. TheMinistry 15program years earlier still hadnot beensatis noted concerns many inourauditofthe ofthe that respiterelief care. through developmentenhance personal andprovide family to are designedprimarily by families,andthat typically beyond provided those and services ports ect fundingto sup eligiblefamiliesfor purchasing family/caregiver. TheSSAHprovides program dir adevelopmental disabilityandtheir adults with andcaregiver respitemunity participation for provideswhich direct fundingto familiesfor com program, Agencies Passport alsoadminister the andrespitebehaviour therapy care. intervention andlanguageand counselling,speech therapy, asassessment provide services for orarrange such Agencies that receiveAgencies that transfer-payment funding At the timeofour At the 2011 Annual Report Annual 2011 2011 AnnualReport , we found ------383

Chapter 4 • Follow-up Section 4.14 384 2013 Annual Report of the Office of the Auditor General of Ontario

Transformation project intended to address these • As of March 31, 2011, there was a waiting list and other areas, we found it would take several of almost 9,600 people who met the SSAH eli- years for many of the issues we identified to be gibility criteria but were still waiting for SSAH addressed effectively. Among our more significant funding. findings were the following: We made a number of recommendations for • In half the cases we reviewed, agencies lacked improvement and received commitment from the supporting documentation to adequately Ministry that it would make changes consistent demonstrate a person’s eligibility or needs. with our recommendations. As a result, agencies could not demonstrate, and the Ministry could not assess, whether the individual was getting the appropriate level of service or was in need of additional support. Status of Actions Taken on • The Ministry had not established acceptable Recommendations standards of service or the necessary processes to properly monitor the quality of servi- The Ministry has made some progress in address- ces provided. Consequently, it could not assess ing all of the recommendations in our 2011 Annual whether it was receiving value for money for Report. For instance, the Ministry has clarified the funding provided to community-based the definition of developmental disability and the agencies. Ministry staff rarely visited agencies criteria and documentation needed when applying for these purposes. for supports and services. It has also conducted The Ministry was not aware of the number of • site visits to agencies and Developmental Services people waiting for agency-based supportive Ontario organizations to assess their compliance services, information that was necessary for with quality assurance measures and policy direc- assessing unmet service needs. tive requirements. Our concerns with regard to the Although it would be reasonable to expect • Passport guidelines and process of reimbursing a consistent set of rules about what were expense claims have been partially addressed, but appropriate services and, therefore, allowable will require more time to be addressed fully. The expenditures under the Passport program, the status of actions taken on each of our recommenda- Ministry had not set such rules. As a result, tions at the time of our follow-up was as follows. expenses for services that were reimbursed in one region were deemed ineligible for reimbursement in another. SERVICES PROVIDED BY TRANSFER- • In practice, annual agency funding continued PAYMENT AGENCIES to be based primarily on historical rather than Eligibility and Access to Services needs-based levels, exacerbating previous funding inequities. As a result, some hourly Recommendation 1

Chapter 4 • Follow-up Section 4.14 service costs appeared excessive, and the To help ensure that eligibility is determined consist- range of costs per hour for similar services ently and equitably across the province, and that varied widely across the province. individuals receive the appropriate support, the Min- • The Ministry had little knowledge of whether istry of Community and Social Services (Ministry) the agencies it funded and their boards of should provide guidance to agencies regarding the directors had effective governance and control criteria and documentation required to demonstrate a structures in place. person’s eligibility and needs. The Ministry’s regional offices, as part of their oversight responsibilities, Supportive Services for People with Disabilities 385 should then periodically review whether transfer- time of our follow-up. The Ministry has the author- payment agencies are assessing people on a consistent ity under SIPPDA to complete compliance inspec- basis and matching their needs to the most suitable tions for all Ministry-funded services and supports. available services. However, inspectors currently verify compliance only with applicable legislation and policy require- Status ments that outline the DSOs’ role in confirming As noted in our 2011 Annual Report, the Ministry eligibility; they include no specific criteria related implemented a new process in July 2011 for people to matching services to needs. applying for developmental services and supports. The Ministry completed compliance inspec- Nine Developmental Services Ontario organizations for all nine DSOs in 2012/13. We noted that tions (DSOs) are now the “single windows” through no DSOs were found to be 100% compliant upon which adults with developmental disabilities and inspection, and the highest rate of non-compliance their families apply for Ministry-funded services related to individuals’ records. The Ministry and supports. Eligibility criteria and documentation informed us that 78% of the total non-compliance requirements were revised to promote consistent requirements had been addressed within 10 busi- decisions for support across the province, and the ness days of the inspections, and all requirements new Services and Supports to Promote the Social had been met at all DSOs by June 2013. Inclusion of Persons with Developmental Disabilities The Ministry also informed us that service agen- Act (SIPDDA) includes a new definition of develop- cies are required to develop and annually update mental disability. support plans for each individual receiving ministry The new eligibility criteria require that appli- services. The agency works with the individual to cants provide their DSO with the following: develop a support plan that includes strategies to an assessment or report, signed by a psycholo- • reach their goals, and the services and supports gist or psychological associate, that confirms that are needed to help execute those strategies. they have a developmental disability; The Ministry inspected 370 transfer-payment documentary proof, such as a copy of a pass- • agency sites (some agencies operate multiple sites) port or birth certificate, that they are 18 years between June 1, 2012, and March 31, 2013, and of age or older; and found that more than half had compliance issues. documentary proof, such as a bank statement, • At the time of our follow-up, many of these agen- utility bill or rental agreement, that they live cies were still addressing these issues. in Ontario. Applicants for services and supports can request A new policy directive also outlines procedures a review if they disagree with a DSO eligibility deci- to be used by DSOs to confirm applicant eligibil- sion, and a new policy directive sets out the review ity. The Ministry further engaged clinicians from process. the Centre for Addiction and Mental Health in June 2011 and May 2012 to develop and provide DSO staff with training on the new eligibility Quality of Services Provided criteria. It also distributed guides, a checklist and Chapter 4 • Follow-up Section 4.14 Recommendation 2 other tools to support the DSOs in administering To ensure that services are appropriate, are of an the new process. acceptable standard, and represent value for the Periodic reviews of the success in matching the money spent, the Ministry of Community and Social most suitable services to the identified needs have Services should: not been implemented, but the process was under establish acceptable standards of service; and collective review by the Ministry and DSOs at the • 386 2013 Annual Report of the Office of the Auditor General of Ontario

• periodically evaluate the appropriateness and Review work included cost analysis, assessment of cost-effectiveness of the services provided by outcomes, eligibility determination and administra- transfer-payment agencies. tion practices, and will continue into 2014. The Ministry informed us that significant Status progress had been made on the sector-led Develop- As noted in our 2011 Annual Report, the Ministry mental Services (DS) Human Resources (HR) introduced a new regulation in January 2011 to strategy, which aims to recruit and retain qualified establish more robust and consistent quality assur- professionals in the DS sector. Core competencies ance standards for agencies. The regulation was were identified for seven standard developmental intended to help evaluate the appropriateness and services agency positions, and three training mod- cost-effectiveness of the services being provided. ules were developed to help staff understand and The Ministry developed additional policy use them. By incorporating the required core com- directives for service agencies in November 2011 petencies into HR hiring and screening processes, to address complaints and establish behavioural DS education and qualifications, and management intervention strategies. The Ministry further feedback and coaching for staff, the strategy aims updated policy directives for DSOs in August 2013. to ensure a well-trained and qualified workforce, The directives are intended to help ensure consist- which will result in quality support for people with ent customer service levels across the province and, developmental disabilities. in the event of relocation, to help make it easier to Lastly, the Ministry launched a project in 2012 to transition between DSOs. The directives provide identify cost drivers in the developmental services instructions for the DSOs on: sector. Based on new and existing financial and the information they provide to the public • service data, the project is aimed at developing unit and applicants about available supports and costs for Ministry-funded services and identify- services, and the application process; ing variables that explain cost differences across confirming eligibility for supports and services • the province. The final report on this project was for the first time; expected in winter 2013–14. • responding to questions and concerns about the application process or services provided; • following consistent steps and using the same Wait Lists tools to assess all applicants using the Applica- Recommendation 3 tion Package; and To help monitor and assess unmet service needs, and reporting requirements to the Ministry. • help allocate funding more equitably, the Ministry of As noted above, the Ministry recently inspected Community and Social Services (Ministry) should a number of service agencies and all of the DSOs in work with agencies to ensure that they prepare and order to assess compliance with its quality assur- periodically forward to the Ministry accurate wait-list ance measures and policy directives. The Ministry information on a consistent basis. informed us that it will continue to do regular com- Chapter 4 • Follow-up Section 4.14 pliance inspections and, as noted, the process of Status matching individuals to services and supports was Agencies no longer maintain wait lists because all under review at the time of our follow-up. individuals applying for supports and services now In January 2013, the Ministry also completed do so through the DSOs. The agencies report to an evaluation of its Passport Mentoring Program the DSOs on their vacancies and the DSOs match for youth and young adults with developmental eligible and prioritized individuals with available disabilities who are transitioning from school. supports and services. In order to improve wait-list Supportive Services for People with Disabilities 387 information as well as system planning and fore- information collected by DSOs to identify the sup- casting, the Ministry is working to consolidate all port needs of individuals referred to them. The information about individuals receiving or waiting DSOs transfer completed application packages to for adult developmental services. This information the Passport agencies on an ongoing basis so they will be moved into the Developmental Services have an updated list of individuals waiting for Consolidated Information System (DSCIS) to pro- Passport services. vide a reliable count of all individuals currently on wait lists. Through the DSCIS, DSOs will be able to Passport Program obtain wait-list reports, including one report that provides data on the number of individuals wait- Recommendation 4 ing for each type of funded adult developmental To ensure that families are being reimbursed only for service. The information will be shared with the reasonable cost for eligible activities, the Ministry community planning groups and transfer-payment of Community and Social Services should clearly agencies. The Ministry plans to complete this pro- define what are eligible expenditures and ensure that ject later in 2013. agencies are approving and reimbursing expense The Ministry informed us that it is developing claims on a consistent basis across the province. a prioritization tool to ensure consistent processes Status across the province. This prioritization tool will use Between September and December 2012, the Min- common risk factors to determine the immediacy of istry solicited stakeholder feedback on proposed an individual’s needs for services and/or supports. changes to the Passport guidelines relating to the In December 2011, the Ministry issued interim issues of eligible and ineligible expenses, respite, guidelines to the agencies for regional prioritiza- and accountability requirements. The Ministry tion for implementation by fall 2013. As well, the informed us that it was considering the feedback Ministry informed us that a funding entity will be and continuing consultations on a revised guide- created as part of the phased implementation of line. In the interim, an addendum effective July 1, SIPDDA. While the Ministry will focus on policy 2013, was added to the Passport guidelines to make setting and overall management of the program, some program changes and provide some clarity the new entity will make funding decisions at the and examples of eligible and ineligible expenses. individual level, by prioritizing the supports, servi- The key change in the addendum with respect to ces and funding for each applicant deemed eligible eligible expenditures was the addition of caregiver under the Act. The timeline for the creation of this respite services and supports. Caregiver respite funding entity has not been determined. refers to services and supports provided to, or At the time of our follow-up, the Ministry was for the benefit of, a person with a developmental developing a resource distribution model for dis- disability by someone other than the primary tributing resources aimed at improving fairness and caregiver to give that primary caregiver some relief. equity, as well as local accountability and flexibility. Indirect respite refers to short-term arrangements

The Ministry was planning to engage with stake- Chapter 4 • Follow-up Section 4.14 that help the primary caregiver manage household holders beginning in fall 2013 to receive feedback and family responsibilities that are not directly and make further refinements to the model. related to caring for a person with a developmental Lastly, the Ministry implemented a new disability. Indirect respite was not added as an eli- Passport Mapping Tool in May 2013 to help Pass- gible Passport expense. However, Passport agencies port agencies maintain accurate wait lists and can pre-approve temporary use of Passport funds determine individual funding allocations. The for indirect respite in extenuating circumstances. tool helps Passport agencies use the application 388 2013 Annual Report of the Office of the Auditor General of Ontario

There is a one-year grace period for adults who Status transitioned to Passport from SSAH before April 1, New Transfer Payment Reporting Standards were 2013, as indirect respite was an eligible expense introduced in the 2012/13 fiscal year. The stan- under SSAH. As well, in extenuating circumstances, dards were intended both to meet the requirements the Passport agency can approve the continued use of the legislation and to address our concerns by of funds for indirect respite beyond the deadline for improving the Ministry’s ability to compare agency these individuals. costs of similar programs. The Ministry engaged The Ministry added a “tip sheet” to its website to some of its stakeholders to develop, oversee and help individuals and families understand expenses deliver province-wide training of agencies in the that are now covered under Passport. The Min- DS sector on the new standards. Ministry staff, istry’s July 2013 bulletin, Spotlight on Transforma- regional leads, program supervisors and agencies tion, which was posted on its website and sent to received this training in December 2012. stakeholders, also highlighted the changes. The new standards were introduced in two We noted that during its consultation on phases. In the first phase, to improve the con- the Passport guidelines, the Ministry proposed sistency and accuracy of the information reported, changes to accountability requirements, such as the number of detail codes was reduced from 30 moving to quarterly reporting of expenses and/ to 16, and definitions were clarified. In the second or performing random or risk-based audits of phase, the Ministry standardized the financial invoices and receipts. However, the Ministry did not information collected through the service con- include any changes with respect to reviewing or tracting process and the Transfer Payment Budget reporting expenses in the addendum. The Ministry Package. The new standardized categories align informed us that it will be undertaking additional with the Ministry’s chart of accounts and are consultations with stakeholders to develop policy intended to improve consistency in expenditure and guidelines regarding admissible expenses and reporting. Agencies can now only use the expendi- activities and accountability requirements. ture categories provided by the Ministry. As noted earlier, the Ministry launched a project in 2012 to improve its ability to analyze and com- MANAGEMENT AND CONTROL OF pare costs of services and enable it to investigate TRANSFER-PAYMENT CONTRACTS and explain variances from budget that seem Budget Submissions and Annual Service unjustified. Unit costing work at the agency level Contracts was completed in March 2013 and an interim report was provided to the Ministry in April 2013. The Recommendation 5 Ministry expects the final report in winter 2013–14. To ensure that funding provided to transfer-payment At the time of our 2011 audit, the Ministry was agencies is commensurate with the value of services developing a new funding-allocation model to provided and that funding is primarily provided improve transparency and equity in the allocation of based on local needs, the Ministry of Community and

Chapter 4 • Follow-up Section 4.14 funds. The goal was to distribute resources based on Social Services should: individuals’ assessed needs using consistent criteria. reassess its current budget submission, review • Under the new model, each individual will have a and approval process and revise it to ensure that unique and portable budget, and will be able to pur- the approved funding to agencies is appropriate chase the services that offer the greatest value and for the expected level of service; and best meet his or her needs. Work on the new model analyze and compare the agency costs of similar • is still ongoing, with the Ministry planning to hold programs across the province, and investigate significant variances that seem unjustified. Supportive Services for People with Disabilities 389 stakeholder discussions before starting pilot testing regional offices. As well, agencies are notified in late in the 2013/14 fiscal year. advance of the compliance inspections; this is not consistent with our recommendation to implement spot audits. Ministry Oversight and Control Annual in-class and online training sessions Recommendation 6 are provided each year to inform agencies about To ensure adequate oversight of transfer-payment changes to the Transfer Payment Budget Package agencies and to improve accountability within the and any new reporting requirements. Ministry and supportive services program, the Ministry of Com- agency staff are invited to participate, and the train- munity and Social Services should: ing material is accessible online. However, actual • review all agency quarterly reports and year-end participation is poorly tracked. TPAR [Transfer Payment Annual Reconciliation The Ministry annually reviews its agencies’ report] submissions for unusual or unexplained reporting policies; however, there have been no sig- variances from previous years and from contrac- nificant changes since the time of our audit. Agen- tual agreements, and follow up on all significant cies are still required to self-identify variances from variances; budget in their quarterly and annual reports and to • perform spot audits on agencies to validate the submit a variance report to the regional office. information provided in the quarterly reports With respect to our recommendation about and TPAR submissions; and reviewing the level of financial expertise among • assess whether each regional office has the level staff at regional offices, managers develop learn- of financial expertise required, and, where lacking plans for all staff as part of their annual per- ing, determine the best way of acquiring this formance planning and review cycle. Employees expertise. work with their manager to identify training needs and goals and develop appropriate plans for the Status year to reach them. As well, staff are trained on The new Transfer Payment Reporting Standards the Ministry’s information system, which man- discussed earlier are intended to enhance the ages financial and service information related to Ministry’s ability to assess value for money and to transfer-payment agencies, and on a web-based investigate significant variances. The Ministry has application that develops reports. Although no implemented two new transfer-payment frame- new initiatives have been undertaken to address works—one for ministry staff and the other for this aspect of our recommendation, the Ministry service agencies—that were being developed at the is reviewing and improving the current training time of our 2011 Annual Report. The frameworks model and updating its online training modules to consolidate the Ministry’s existing business practi- build capacity in the regions. ces and its requirements for appropriate management of government funds. The Ministry indicated that as part of its Governance and Accountability compliance program, discussed earlier, inspectors Chapter 4 • Follow-up Section 4.14 Recommendation 7 verify compliance with quality assurance measures To ensure that agencies have the capabilities to and the Policy Directives for Service Agencies. properly administer the spending of public funds, the This includes reviewing financial records. How- Ministry of Community and Social Services should ever, the inspectors do not validate quarterly encourage the regional offices to play a more hands-on reports or Transfer Payment Annual Reconciliation role in ensuring that agencies have appropriate exper- report submissions; these are reviewed by staff in tise and governance structures and accountability 390 2013 Annual Report of the Office of the Auditor General of Ontario

processes, including those smaller agencies that receive As noted earlier, the Ministry was at the less funding but may have more difficulty maintaining time of our 2011 Annual Report working on two proper financial controls. Transfer Payment Governance and Accountability Frameworks—one for staff and the other for agen- Status cies. The frameworks consolidated the Ministry’s Following an internal review of risk-assessment existing business practices and expectations into processes, the Ministry introduced a revised Risk two documents. The Ministry’s framework consoli- Assessment Methodology and Tools in fall 2011 to dated all of the Ministry’s business practices and strengthen oversight, governance and accountabil- tools for transfer-payment oversight. The service- ity while attempting to control costs by focusing on provider framework focused on what agencies must areas of higher risk. Key changes included: do to meet ministry governance and accountability Business cycle: High-risk agencies would now • requirements. Both frameworks were implemented be reviewed by the Ministry every 12 months in March 2012. Regional directors were tasked instead of every six months, and low-risk with disseminating and discussing the Ministry’s agencies would be reviewed every 24 months expectations, as outlined in the framework, during instead of every 18 months. Mid-way reviews regular budget negotiations and planning meetings were added to the review schedule and a firm in spring 2012. timeline was created to ensure assessments With respect to our recommendation about would be complete before annual contracting ensuring that agencies have appropriate expertise decisions were made. and governance structures, the ongoing human- Risk Dimension and Weighting: The avail- • resources efforts and core-competencies strategy able responses on the risk assessment were will help agencies recruit and retain qualified pro- expanded and questions were streamlined to fessionals. To date, the main focus of the strategy improve comparability. has been on improving the competencies of direct Risk Mitigation Strategy: Risk mitigation was • support staff. However, core competencies have integrated into the risk assessment. For each also been identified for executive directors, and this medium or high risk identified, agencies must information was shared with all regional offices, develop a mitigation strategy to reduce the and executive directors and boards of directors of likelihood or severity of that risk. all DS agencies. The agencies are encouraged to Risk-Rating Scale: The number of factors that • use this information when hiring, setting perform- determine risk ratings was expanded. ance expectations or reviewing performance. The Business Process and Tool Usability: The • Ministry indicated the sector had also developed a process for agencies that receive funding from core competencies “dictionary” to establish a com- multiple ministries or regions was formalized, mon language for performance expectations and and full assessments for new service providers benchmarks for hiring, learning and development. were added. As well, the Ministry is continuing to contribute The Ministry introduced the new methodol-

Chapter 4 • Follow-up Section 4.14 annual funding to a leadership program at Queen’s ogy and tools through teleconference and online University for executive directors and other leader- sessions. It also trained a number of managers ship positions. and regional leads who then facilitated training sessions for regional staff and service providers. Training and implementation were completed by December 2011. Supportive Services for People with Disabilities 391

SPECIAL SERVICES AT HOME (SSAH) the addition of respite to the Passport guidelines. Passport agencies are not required to use the tem- SSAH Reimbursements plate provided by the Ministry, and may create their Recommendation 8 own invoice for individuals and families to submit To ensure that Special Services at Home (SSAH) receipts. Passport agencies set their own policies reimbursements to families are consistently made and practices for reimbursing individuals and only for legitimate and eligible expenses, the Ministry families for eligible expenses. The Ministry does not of Community and Social Services (Ministry) should prescribe how or when Passport agencies reimburse establish and communicate clear criteria for what individuals and families, but it does hold agencies constitutes an eligible expense. accountable to its transfer-payment standards and In addition, the Ministry and agencies that admin- requirements. The criteria for Passport’s eligible ister SSAH funding should obtain sufficiently detailed expenses and the approval of expenditures were invoices—and, where applicable, receipts—to ensure addressed earlier in this section. that the amounts claimed are in fact eligible and reasonable before funds are disbursed. OTHER MATTERS Status Travel, Meal and Hospitality Expenditures As noted previously, Special Services at Home (SSAH) serve only children and youth as of April 1, Recommendation 9 2012, and all adults seeking direct funding support To help ensure that all agencies that are required to must apply through the DSOs for direct funding do so implement the government’s new directive on under the Passport program. The scope of our 2011 travel, meal and hospitality expenses, and that all Annual Report was limited to supportive services for other agencies follow the spirit of the directive, the adults with disabilities; however, we noted that the Ministry of Community and Social Services should Ministry updated the SSAH invoice template and reinforce the requirements to do so and consider hav- “Managing your Funding” guide in December 2011 ing the agencies’ board chairs annually attest to such to require more detailed expense submissions. compliance. The invoice template now requires information Status about the type of service and/or program that was Compliance reporting is required of all agencies provided, and a separate invoice must be submitted receiving $10 million or more in transfer-payment for each worker. A reminder was added to the guide funding, in accordance with the Broader Public Sec- about eligible expenses and the sign-off section tor Accountability Act, 2010 (BPS Act). The Ministry was amended to clarify the accountability require- provided its regional directors with a template let- ments for the individual submitting the claim and ter for distribution to those agencies affected by the for the support worker who provided the service. requirements. The letter outlined the new reporting The new guide was distributed to all 2011/12 SSAH requirements, including the deadlines for compli- recipients young enough to be eligible for SSAH

ance, and links were provided to online versions of Chapter 4 • Follow-up Section 4.14 during 2012/13. The Ministry indicated that all the BPS Act and its associated directives. SSAH invoices are approved by ministry staff prior Agency compliance reporting consists primarily to disbursement. of an annual attestation of compliance signed by The Passport reimbursement process has not the chief executive officer or equivalent, and by been revised since our 2011 Annual Report. The the chair of the board of directors. The attestation Ministry provides Passport agencies with a sample form requires each agency to report whether it is in invoice template, which has been updated to reflect compliance with nine requirements set out in the 392 2013 Annual Report of the Office of the Auditor General of Ontario

BPS Act or its directives. One of these requirements SSAH Program Administration is that the organization manage its travel, meal and Recommendation 10 hospitality expenses according to policies that com- Given the similarities in overall staffing levels at the ply with the Broader Public Sector Expenses Direc- regional offices dedicated to the Special Services at tive. For any issues of non-compliance, the agency Home (SSAH) program, the Ministry of Community is required to report on the corrective action that it and Social Services should assess the need for the will take. The Ministry distributes this attestation additional administration costs being paid out to form along with the Transfer Payment Budget Pack- agencies and ensure that all costs incurred are reason- age every February. Agencies were first required to able and necessary. return the compliance forms to their regional office in June 2012. Status The Ministry informed us that it communicated At the time of our follow-up, the Ministry had the requirements of the BPS Act to those agencies reviewed the administrative models of SSAH and that are required to comply with it. As well, it Passport as part of the transition to a single direct- encouraged other agencies to voluntarily comply funding program. The review considered the type with it. and level of administrative support offered to fam- The Internal Audit Division recently launched ilies transitioning from SSAH to Passport in the fis- a review of the actions that the Ministry has taken cal year 2012/13 and the source of administrative to address our recommendations relating to travel, funding for agencies. The review identified strat- meal and hospitality expenditures. A report was egies to reduce the provincial variations in adminis- expected later in 2013. tration costs and to increase cost effectiveness. The The Ministry expects that the revised risk-assess- Ministry established one common formula for fund- ment process discussed earlier will help improve ing administration costs for all Passport agencies: program management and mitigate risks. The it is now determined as a percentage of the total risk-assessment documentation includes a section annual funding of each Passport agency. Passport related to the implementation of policies consistent agencies are no longer required to determine pro- with the Broader Public Sector Expenses Directive. gram eligibility, assist applicants in the completion This section requires an assessment of, among other of a Passport application, or interview program things, whether the agency has and makes use of applicants. The new formula reflects this new role. financial policies and procedures covering pro- As part of this transition, the Ministry worked curement, meals/hospitality and travel; whether with regional offices to develop transition strategies cheque authorization and expenditure approvals and timelines that would help implement the new are independent; and whether financial oversight formula and minimize its impact on families and responsibilities are segregated to reduce the risk individuals. Three regional offices requested and of errors or irregularities going undetected. The received one-time additional funding during the Ministry completed risk assessments of all transfer- transition year to support service or transition co-

Chapter 4 • Follow-up Section 4.14 payment agencies in the developmental services ordinators, to provide workshops and training for sector between November 2011 and January 2012. recipients, and to maintain existing HR and administrative supports. The Ministry has not reviewed or changed the administration funding for SSAH. we have done over past fiscal year. the annually Legislative work to Assembly the onthe Act andthe in the net mediumhadtaken asanadvertising off. Act, Inter introduced atatimebeforeOntario the would have still inthe exists closedaloopholethat Auditorto the review General’s mandate. This version would have addedInternet advertising passed. neither although Actintroduced billsclosely in2013, basedonthe ColumbiaandNovain British Scotia, for example, legislation.Oppositionlegislatorsmodel for such asa and continuesto becited jurisdictions inother to ensure itisnot partisan. broadcast andoutdooradvertising print, ernment requires(Act), which my Office to review most gov duction ofthe intro 10th ofthe This year the marks anniversary INTRODUCTION 2012/13 Advertising Chapter 5 Chapter This chapter satisfies the legislativeThis chapter satisfies requirement Columbia the British that It was significant, too, law onlyThe Act such inCanada, remains the Government Advertising Act, 2004 Act,2004 Advertising Government Auditor General Act Review Activity, Advertising Review ofGovernment to report to report

- - - a total valueof$30.1 million. Thiscompares to 565 itemsual advertising in130 finalsubmissions,with 2012/13In the fiscal year, we reviewed 572 individ for review, asfollows: them to ouroffice submitting first ments without viously contravened Actadvertise by the running We did,however, hadpre three ministries that find Act andreceived standards ofthe met approval. the adssubmitted toof the usinfinal version for review we is,all found nosubmissionsinviolation—that past fiscal year the was 3.1time during days. average the work and other turnaround priorities, complexity ofan ad the with for adecisionvaries timerequired the business days orless.Although of $34.8 million last year. individual adsin121 atotal value submissionswith RESULTS OFOURREVIEWS • • For the first time since the For timesince first the Inallcases,we gave seven ourdecisionwithin

submitted in February three videosonflu three submitted inFebruary Act. of the have inviolation ofSection6(1)3 found them ads beensubmitted to usfor review, we would contravention Act. Hadthese ofthe further in General, Attorneys then namesofthe the review. Inaddition,15 adscontained ofthe them for submitting first lications, without between 2006and2011, mostly inlegal pub 17 David. W. ads about the MundellMedal The Ministry of Health andLong-Term ofHealth The Ministry Care Attorney ran General ofthe The Ministry 2007 AnnualReport - - , - - 393

Chapter 5 Chapter 5 394 submissions. In the past calendaryear,submissions. Inthe someof them As noted above, we regularly process pre-review pre-reviews. year’s discussionofthis further 9.1 Matters” below businessdays. See“Other for a average submissionswas time for these turnaround to complete areasonable time.The within them Nonetheless,review we period. effort make every seven-business-day the to issueadecision within requirements Act, we ofthe tory are not required statu andoutsidethe onourpart are voluntary stage ofdevelopment.preliminary Aspre-reviews past fiscal year comprising 22adsata sions this basisofany adsitplaces. on amonthly such to adviseus Act andrequires Ministry under the the the nature ofurgent matters clarifies that Ministry 2013 of Ontario of the OfficeGeneral of the Auditor Annual Report Pre-review Submissions Pre-review OTHER • In addition,we 10 examined pre-review submis Subsequently, we developed the aprocess with

paid for government by ofOntario. the includeastatement saying hadbeen they they proviso approved the that content, with their submitted to usfor review, we would have adsbeen Act.intended Hadthese underthe tion adswereof these aboutforest-fire Act. Wethe some timethat determined atthe exempt from review underSection2(5)of matters (forest fires) and thereforewere adsregarded urgent claimedthe Ministry the of contraventions identifiedlast year after last of aseries believe eightadsare the these them. submitting first We in print—without forest-fire safetyradio andtwo ads—sixon Act. Sections 2(2)and2(3)ofthe previous fall.Thiswas incontravention of the hadalready begunrunning they that acknowledged videos whenitsubmitted the and we approved However, them. Ministry the prevention intended for useinmedicaloffices, The Ministry ofNatural Resources eight The Ministry ran andsowere not ofanurgent nature as MATTERS preven - - - meet the standards of the Act because they were Act becausethey standards ofthe meet the budget. We adswould the likely concludedthat not proposed amendmentsto the in connectionwith submitted adsfor four pre-review proposed print campaign andresubmitted it,andwe approved it. subsequently revised the TheMinistry of itscritics. andanegative governmentimage one party ofthe may they have adswas that fosteredthe apositive 2013 provincial budget. about Oneofourconcerns ofFinance aboutthe proposed toMinistry run example, adsfor contained the acampaign that occasionally helpsassessproposed government ads. that group ReviewAdvertising Panel, anadvisory and required more intensive by examination the were quite politicalcontext, current complex inthe limits of the Act. limits ofthe attimes,pre-reviews arethat, beingusedto test the sums ofmoney to it.However, we are concerned Act beforecommitlarge standards ofthe they the aproposed campaigntion ofwhether would meet only todo these give government officesanindica ultimately not to chose proceed. Ministry the and Act becauseitwasstandards partisan, ofthe proposed campaignthe would likely not meet the Budget.”ance Ontario’s We again deemedthat for acampaign entitled“Making Choicesto Bal province)intended across the for bulkdistribution andaproposed “householder”(abookletscript submitted for pre-review are-worked television later, were Somemonths partisan. Ministry the they Act things, because,amongother dards ofthe we adslikely found these would not stan meet the ant to people.”extensive After review Panel, by the deficit whileprotecting import the what’s “tackle and“plan”to “economic blueprint” government’s for script and aradio the pre-review dealingwith Development andInnovation submitted atelevision would they havecontext inwhich run. minority-government particularly inthe partisan, The following month, the Ministry ofFinance Ministry The following the month, One pre-review for submissioninApril, The Act makes noprovision for pre-reviews; we ofEconomic Ministry occasion,the On another - - - - lowed bythe ourOfficecanbe found in on.ca party. TheAct canbefound at governing interestsis to promoteofthe partisan the objective ofanitem aprimary determining whether to consideradditionalfactors in authority tionary governingitical interests party.” ofthe itemobjective pol isto ofthe promote partisan the Auditor opinionofthe aprimary General, in the if, must “anitem meet ispartisan andstates that innature. partisan be interpreted asbeing,primarily is,orcould adsmust not that anything such contain meetensure legislated they standards. Above all, for reviewing specifiedtypesofgovernment ads to The Auditor Act isresponsible General underthe include Internet advertising. government Acthas comefor to to the amendthe include any Internet costs. We believetime the chapter doesnot endofthis atthe ture reporting met for allgovernment Ourexpendi advertising. Act intent isto ofthe be should beaddressed ifthe that the legislation become asignificantloopholein increases, weadvertising believe itsexemption has online government’s Asthe for inprint. advertising itdid ment spentmore than onInternet advertising for review. 2012/13 Inthe fiscal year,the govern hadbeensubmitted Act to ifthey of the ourOffice wouldonline campaigns that have beeninviolation past, we Inthe advertising. have seengovernment entirely online.TheAct doesnot cover online online component andsomecampaigns even run large campaigns advertising ment’s includean most marketing campaigns. govern Many ofthe of part hasbecomeanintegral Online advertising Review Function Overview Online Advertising The Act Auditor alsogives the discre General advertisements The Act outlinesstandards that , and more details about the processes fol, andmore details aboutthe of

the Advertising www.e-laws.gov. Government Government

------ted to my Office for review andapproval before known as“reviewable” items andmust besubmit meetingAdvertisements definitionsare any ofthese ofbulkdelivery. method bulk mailoranother usingunaddresseduted to householdsinOntario government officeproposes to pay to have distrib television. a Italsoappliesto matter printed that displayed onabillboard, orbroadcast or onradio to have publishedinanewspaper ormagazine, the Premier—propose to pay Officeof and the specifically, governmentCabinet Office ministries, The Act government appliesto offices— adsthat on.ca/adreview Guidelines Review Advertising run the item. the run Act, anditmay item standards ofthe meets the the ment officeisdeemed to have receivedthat notice seven governa decisionwithin business days, the hasbeenapproved. advertisement the notice, that receives notice, orisdeemedto have received the deputyminister, office,usually headofthat the submitted ordisseminate itemdistribute the until ment officecannot publish,display, broadcast, reviewableevery item to our Office.Thegovern undue delays ingetting message out. the seven-business-daynormal process would impose orsafety, publichealth matters affecting where the those regardinga government urgent office,and to provisionare adsonthe ofgoodsandservices notices publicrequired to the by law. Alsoexcluded canrun. they WHAT • • If the AuditorIf the Officedoesnot General’s render The Act requires government offices to submit As well, following the are not Act: subjectto the The Act excludes from review jobadsand

unaddressed mail). magazine ornewspaper, by or distributed bulk ina publications (unlessusedasapaidinsert or similarmaterials andother ments, reports brochures, newsletters,brochures, docu consultation and online advertising; FALLS . UNDER THE Review of Government Advertising at ACT www.auditor. - - - - - 395

Chapter 5 Chapter 5 396 ment meets the standards of the Act, which are: Act, which standards ofthe ment meets the the proposed advertise Office determines whether In conductingitsreview, Auditor the General’s Act. standards ofthe meets the soweished form canreview itstill itto ensure that government officemust submititinfin used, the office. However, canbe the advertisement before Act, we governmentstandards ofthe soadvisethe governmentto the office.Ifitappears the tomeet to Act, violate we the provide explanation abrief Act. the requirements of statutory andis outsidethe part onour Act. Apre-reviewthe voluntary isstrictly couldbedeemedobjectionableunder that material time andmoney spentto develop adscontaining appear whencompleted. Pre-reviews help limitthe the itemit reasonably asitisintended reflects to provided orstoryboard, ad. Thiscanbeascript that offices wishingus anearlyexamine to ofan version adhasbecomepartisan. impression the that if we new leave circumstances determine that the year, my although Officecan rescind anapproval final. Auditor Act, alldecisionsofthe are General the Office hasseven days to render adecision. Under review.another submission, first my the Aswith rejectedsubmit arevised version item ofthe for not beused.However, government officemay the item item may doesnot standards, the the meet the 2013 of Ontario of the OfficeGeneral of the Auditor Annual Report ADVERTISEMENTS FOR STANDARDS • If material submittedIf material for pre-review appears A pre-review isalsoavailable to government Approval isvalidfor ofanadvertisement one If my that the Officenotifies office government

• • objectives: following oneormore ofthe achieving The item must beareasonable meansof

to inform the public of its rights and publicofits rights to the inform orproposed publicofcurrent to the inform responsibilities under the law;responsibilities underthe orservices; government policies,programs PROPOSED

- - certain undesirable ones,asfollows: undesirable certain andavoids attributes desirable it includescertain to beperceived. isgiven Consideration to whether impression conveyed adandhow by the itislikely additionalfactors relate overall these toeral, the governing interests party. ofthe partisan Ingen objective ofanitem isto promote a primary the consider additionalfactors to determine whether above, Act allows Auditor the the to General standards specific statutory In additionto the OTHER • • • • •

• • • • governing party, oranegative impression ofa of fostering apositive impression ofthe exempt requirement). from this item is casethe inwhich outside Ontario, target audienceislocated primary (unless the Legislative Assemblyor amemberofthe Executiveimage ofamemberthe Council paid for government by ofOntario. the partisan interests of the governing interests party. ofthe partisan promotion objective ofthe the as aprimary Auditoropinion ofthe itcannot General, have government. ofthe orentitycritical person The item objective must not have aprimary The item name,voice must not or includethe The itemitis must includeastatement that Each item should: Each is,inthe The item that must not bepartisan;

to promote Ontario, or any part of the provto ofthe orany promote part Ontario, to encourage ordiscourage specificsocial present information objectively,present information intone and subjectmattercontain relevant to govern activity or sector of Ontario’s economy.activity orsector ofOntario’s study orvisit,to promote any economic ince, asagoodplaceto live, work, invest, publicinterest;behaviour inthe and/or content, with facts expressed clearly factsexpressed content, with and item); inthe dealt with specificmatters responsibilities for the ment shouldhave direct andsubstantial govern is,the ment responsibilities (that FACTORS - - - - tion over years. the that have significantareas requiredof the clarifica Act issilent.Whatfollows the description isabrief government, areas where the with in co-operation my advertising, to clarify, Officehastried ernment review onresponsibility forSince taking ofgov the OTHER •

• • • • • • • • • Items shouldnot:

deliver self-congratulatory orimage- deliver self-congratulatory intentionally promote, orbeperceived as criti ridicule, directly orindirectly attack, logosand/orslogans com use colours, audienceto distinguish betweenenable the emphasize factsand/orexplanations, provide the ofboth abalancedexplanation use a uniform resource locatoruse auniform (URL)to or present pre-existing policies,services als; and ofpropos politicalmerits the than rather has beenmade; policyproposals where nodecision with benefits anddisadvantages whendealing language; accurately, usingunbiasedandobjective following section). Act (see“Websites”standards ofthe inthe may contentpage not that meet with the direct viewers readers, orlisteners to aweb wereactivities asifthey new; or building messages; message willbecommunicated);the overallat andthe environment inwhich audienceitisaimed message, the ing ofthe isalsogivenend, consideration tim to the interestspromoting, (to political-party this government; ofthe critical of those views, policiesoractions cize orrebut the governing party; monly the associated with other.or analysis onthe onehandandcomment,opinion fact onthe REVIEW PROTOCOLS ------opinions of others. opinions ofothers. policiesor the messages orany attacks content that nameorphoto, minister’s any self-congratulatory Act. Forthe page example, must the not includea for any may content not that standards of meet the case we page. next review We the page this examine isa gateway click first unless that page, inwhich We click, first content consideronly ofthe the reviewable item would beincludedinourreview. page, first ofa or“click,” the website citedthat ina legislationwas passed the soonafter agreement government, we the cametodiscussions with an ad.Following isanextension ofthe advertisement Response Codeorsimilarlinkage usedinan Act, weunder the believeawebsite, that Quick websitesAlthough are not specifically reviewable sites. Although the Act is silent on this, we Act issilent onthis, reached the sites. Although presence onsocial-media government’s ing to the Office receives ads for approval iconspoint with exponentially inrecent years. Increasingly, our came into However, effect. itsusehasgrown Act Social mediawas initsinfancywhen the following ofthe criteria: three tising must besubmitted for review ifitmeets all adverthird-party and mythat Officehave agreed sometimes usedfor Thegovernment advertising. Government are fundsprovided toparties third Websites Social Media Advertising Third-party • • •

the item. the provincialthe item; and visualidentifierin official logooranother Ontario sion to usethe item; the distributing cost ofpublishing,displaying, broadcasting or fundsintended to orallofthe with pay part the government officeapprovesthe the content of permis party third the government grants the thirda government party officeprovidesthe Review of Government Advertising - - - 397

Chapter 5 Chapter 5 398 2012/13 fiscal year: the to my providedadvisers Officeduring services review ofselectedin the submissions. The following engage to advisers fourassist usasneeded external ing Act,2004 the the requirementsfulfill of Act The Auditor can,underthe General government office. controlbe beyondofthe the networksthese changes frequently andcanattimes references. However, we content on recognize that are nopartisan cited there inanadto ensure that aninitialscanofany social-mediachannel perform we government that will the with an agreement 2013 of Ontario of the OfficeGeneral of the Auditor Annual Report External Advisers • • , appoint an Advertising Commissionerto, appointanAdvertising help

and governments useadvertising. way politicalparties onthe inwhich articles inCanadaandanumberof ment advertising Professor Rose abookongovern haswritten andCanadianpolitics. in politicaladvertising interests a leadingCanadianacademicwith Political Studies atQueen’s University. Heis working industry. advertising inthe andcommunicationswhile media, advertising acquired acomprehensive in background dards Council.Before studying law, Mr. Engle StanCanada, andasChairofitsAdvertising side legal counselfor Standards Advertising not-for-profitthe out sectors. Healsoactsas for-profit ofclients inthe diverse group and law formunications andentertainment a marketing,specializing inadvertising, com Rafe EngleRafe (J.D.,L.L.M.)isaToronto lawyer Jonathan Rose isAssociateJonathan Professor of . However, we have instead to chosen Government Advertis Government Auditor General - - - - - our review past year. ofgovernment this advertising in ourreview work year. this review.General’s We found nomatters ofconcern Auditor useofitems pendingthe prohibition onthe submission requirementsAct. Thesedealwith and requirements to the ofsections 2,3,4and8the compliance respect procedures with certain formed We atselecteddocumentation ministries. alsoper supporting andtheir tising andcreative services reviewed selected payments to ofadver suppliers my expenditures, advertising reported Office the In order to completeness test the of andaccuracy tising campaigns to reported ministry. usby each Act. under the matterfor andprinted reviewable advertisements ally Legislative to Assembly the onexpenditures The Matter Advertisements Expenditures • • These advisers providedThese advisers in valuableassistance Figure details expenditure ofadver 1contains Auditor General Act

and overseeing ituntilhisretirement in2010. implementing review ouradvertising function ations inmy Office.He was instrumental andtelevision. national politicsinprint ist, editor and andproducer covering Ontario asajournal decadesofexperience three with John Sciarra is the former director former ofoper isthe John Sciarra Joel Ruimy isacommunicationsconsultant requires annu meto report on

and Printed -

- - - - - This pageblank. intentionally left Review of Government Advertising 399

Chapter 5 Chapter 5 400 3. 2. 1. 2004 Act, offices government Ontario Source ofdata: Advertising Government underthe Matter andPrinted ReviewableFigure 1:Expendituresfor Advertisements 2013 of Ontario of the OfficeGeneral of the Auditor Annual Report Attorney General Foodland Ontario andFood Agriculture Public Notice–AlgonquinLandClaim Program Grants Aboriginal CommunityCapital Aboriginal BusinessDirectory Affairs Aboriginal Title Ministry/Campaign ServiceOntario OfficeChanges ServiceOntario OfficeChanges ServiceOntario Government Services TrilliumOntario Benefit Healthy HomesRenovation Tax Credit 2012 Savings Bonds Ontario Finance Full-day Kindergarten Education Your NextBigIdea Your NextBigIdea Your NextBigIdea Economic Development, Trade andEmployment RIDE Investigations Public Notice–Death Safety Internet Safety Internet Alive Arrive andCorrectional Services Community Safety ODSP Office Relocation Community andSocialServices Remembrance Day Ceremony Order ofOntario Global ExperienceOntario Citizenship andImmigration Mundell MedalAwards ServiceChanges Courthouse Costs basedonestimates. Costs in2012,Reported in2013. butmorecosts in2013,Reported in2014. but morecosts 3 2 2 1 1 2 , April1, 2012–March 31, 2013 Submissions # of — — — 7 6 1 4 1 1 1 1 1 2 1 1 2 2 1 2 8 2 5 3 2 Items # of 39 36 44 26 40 23 12 13 18 71 27 27 — — — 2 4 1 2 8 1 1 1 2 Fees Agency 230,675 195,870 318,743 65,285 51,620 18,414 — — — — — — — — — — — — — — — — — — Production 284,342 188,597 281,211 38,229 38,629 18,000 63,467 ($) Costs Third-party 3,800 3,225 4,031 — — — — — — — — — — — — — —

7,00 177,10 30,952 35,494 98,150 56,787 Talent 2,981 — — — — — — — — — — — — — — — — — — Bulk Mail — — — — — — — — — — — — — — — — — — — — — — — — 51,030 16,570 11,970 5,480 2,686 5,250 3,183 Other 284 601 378 60 — — — — — — — — — — — — — 1,060,540 1,324,258 1,265,098 2,437,652 52,030 13,550 87,093 TV — — — — — — — — — — — — — — — — — 1,579,352 196,053 547,378 ($) Media Costs 21,019 Radio — — — — — — — — — — — — — — — — — — — — 1,835,360 1,904,995 1,716,041 646,026 132,358 209,491 839,847 799,181 25,537 24,510 15,378 3,858 1,530 1,250 1,962 3,310 1,138 Print 931 — — — — — — Out-of-Home* –189,998 456,184 118,997 69,630 60,270 26,162 — — — — — — — — — — — — — — — — — — Ad Value 4,750 1,413 7,285 665 ($) — — — — — — — — — — — — — — — — — — — — † 4,808,895 2,706,047 1,759,247 Campaign 1,411,853 1,510,773 1,102,875 3,047,712 2,587,112 Total ($) 138,766 122,871 218,541 45,529 70,030 13,550 25,915 15,979 2,663 3,858 1,530 1,962 4,031 3,310 1,198 931 ServiceOntario OfficeChanges ServiceOntario OfficeChanges ServiceOntario Government Services TrilliumOntario Benefit Healthy HomesRenovation Tax Credit 2012 Savings Bonds Ontario Finance Full-day Kindergarten Education Your NextBigIdea Your NextBigIdea Your NextBigIdea Economic Development, Trade andEmployment RIDE Investigations Public Notice–Death Safety Internet Safety Internet Alive Arrive andCorrectional Services Community Safety ODSP Office Relocation Community andSocialServices Remembrance Day Ceremony Order ofOntario Global ExperienceOntario Citizenship andImmigration Mundell MedalAwards ServiceChanges Courthouse Attorney General Foodland Ontario andFood Agriculture Public Notice–AlgonquinLandClaim Program Grants Aboriginal CommunityCapital Aboriginal BusinessDirectory Affairs Aboriginal Title Ministry/Campaign 3 2 2 1 1 2 Submissions # of — — — 8 2 5 3 2 7 6 1 4 1 1 1 1 1 2 1 1 2 2 1 2 Items # of 36 44 26 40 23 39 12 13 18 71 27 27 — — — 8 1 1 1 2 2 4 1 2 Fees Agency 230,675 195,870 318,743 65,285 51,620 18,414 — — — — — — — — — — — — — — — — — — Production 284,342 188,597 281,211 38,229 38,629 18,000 63,467 ($) Costs Third-party 3,800 3,225 4,031 — — — — — — — — — — — — — — 7,00 177,10 30,952 35,494 98,150 56,787 Talent 2,981 — — — — — — — — — — — — — — — — — — Bulk Mail — — — — — — — — — — — — — — — — — — — — — — — — 51,030 16,570 11,970 5,480 2,686 5,250 3,183 Other 284 601 378 60 — — — — — — — — — — — — — † *

Ad Value denotes the value of an ad space provided at no cost, often where the government hasprovidedfundingforarelatedevent. government wherethe Ad Value often valueofanadspaceprovidedatnocost, denotesthe posters. includesbillboardsandtransit Out-of-Home advertising 1,060,540 1,324,258 1,265,098 2,437,652 52,030 13,550 87,093 TV — — — — — — — — — — — — — — — — — 1,579,352 196,053 547,378 ($) Media Costs 21,019 Radio — — — — — — — — — — — — — — — — — — — — 1,835,360 1,904,995 1,716,041 646,026 132,358 209,491 839,847 799,181 25,537 24,510 15,378 3,858 1,962 1,530 1,250 3,310 1,138 Print 931 — — — — — — Out-of-Home* –189,998 456,184 118,997 69,630 60,270 26,162 — — — — — — — — — — — — — — — — — — Ad Value 4,750 1,413 7,285 Review of Government Advertising 665 ($) — — — — — — — — — — — — — — — — — — — — † 4,808,895 2,706,047 1,759,247 Campaign 1,411,853 1,510,773 1,102,875 3,047,712 2,587,112 Total ($) 138,766 122,871 218,541 45,529 70,030 13,550 25,915 15,979 3,858 2,663 1,962 1,530 4,031 3,310 1,198 931 401

Chapter 5 Chapter 5 402 5. 4. 2. 1. 2013 of Ontario of the OfficeGeneral of the Auditor Annual Report Growing the Greenbelt Growing the andHousing Municipal Affairs andSafetyatWorkHealth Labour Stroke Warning Signs Smoke-Free Ontario Seasonal Influenza Seasonal Influenza Healthy Changes CareOptions Health Health andLong-Term Care Title Ministry/Campaign Student Permit Regulations Student Permit Tuition Ontario 30% Off Training, CollegesandUniversities Parks Huronia Historical Parks Huronia Historical Park WilliamHistorical Fort Park WilliamHistorical Fort Park WilliamHistorical Fort Tourism, CultureandSport New MiningActRegulations Development andMines Northern Restoration Wildlife Habitat Wildfire Prevention Water ManagementPlan Waste DisposalSiteClosure Office Service ChangestoDistrict Card Outdoors Parks Ontario Parks Ontario Parks Ontario Land Management Management Plan Fisheries Contest Art Fish WildfirePrevention FireSmart Members CommitteeSeeksNew Advisory 50 MillionTrees Program Natural Resources Review Policy Statement Contravention—ad published before being reviewed, then submittedandapproved. Contravention—ad publishedbefore beingreviewed,then by Ministry. Contravention—not reviewed,butreported in2012,Reported in2013. butmorecosts in2013,Reported in2014. but morecosts 2 1 4 1,5 2 1 1 2 1 1 1 2 Submissions # of 18 16 — — — — — 1 1 2 1 2 1 1 6 1 2 1 1 3 1 1 1 1 1 3 1 2 1 2 1 2 Items # of 24 22 34 16 — — — — 1 8 2 5 1 8 3 2 4 1 1 4 2 2 8 3 1 2 1 2 7 1 4 1 Fees Agency 129,200 296,712 86,615 3,875 — — — — — — — — — — — — — — — — — — — — — — — — — — — — Production 323,025 449,014 441,082 32,936 13,795 50,537 ($) Costs Third-party 2,060 2,720 1,000 600 480 275 175 — — — — — — — — — — — — — — — — — — — 171,015 61,052 22,476 77,507 Talent — — — — — — — — — — — — — — — — — — — — — — — — — — — — Bulk Mail 6,388 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — 2,092 2,697 Other 5,177 50 89 89 35 — — — — — — — — — — — — — — — — — — — — — — — — — 2,406,903 1,273,365 849,000 960,826 548,243 44,171 TV — — — — — — — — — — — — — — — — — — — — — — — — — — 599,197 22,708 30,011 ($) Media Costs 37,923 Radio — — — — — — — — — — — — — — — — — — — — — — — — — — — — 23,620 43,253 30,842 10,526 13,661 12,120 36,897 4,895 5,966 8,443 1,699 1,695 1,528 1,210 1,115 Print 334 505 — — — — — — — — — — — — — — — Out-of-Home* 166,835 411,415 41,872 150 — — — — — — — — — — — — — — — — — — — — — — — — — — — — Ad Value 22,894 16,682 12,516 4,100 ($) — — — — — — — — — — — — — — — — — — — — — — — — — — — — † 1,394,006 3,914,909 Campaign 1,151,025 1,617,505 849,000 Total ($) 444,351 113,799 101,509 23,620 30,842 22,894 10,526 13,661 12,120 24,407 12,791 46,767 16,857 6,055 8,958 2,060 1,695 1,528 5,100 1,210 1,115 600 334 505 50 89 0 Student Permit Regulations Student Permit Tuition Ontario 30% Off Training, CollegesandUniversities Parks Huronia Historical Parks Huronia Historical Park WilliamHistorical Fort Park WilliamHistorical Fort Park WilliamHistorical Fort Tourism, CultureandSport New MiningActRegulations Development andMines Northern Restoration Wildlife Habitat Wildfire Prevention Water ManagementPlan Waste DisposalSiteClosure Office Service ChangestoDistrict Card Outdoors Parks Ontario Parks Ontario Parks Ontario Land Management Management Plan Fisheries Contest Art Fish WildfirePrevention FireSmart Members CommitteeSeeksNew Advisory 50 MillionTrees Program Natural Resources Review Policy Statement Greenbelt Growing the andHousing Municipal Affairs andSafetyatWorkHealth Labour Stroke Warning Signs Smoke-Free Ontario Seasonal Influenza Seasonal Influenza Healthy Changes CareOptions Health Health andLong-Term Care Title Ministry/Campaign 2 1 4 1,5 2 1 1 2 1 1 1 2 Submissions # of 18 16 — — — — — 1 2 1 2 1 1 2 1 2 1 1 6 1 2 1 1 3 1 1 1 1 1 3 1 2 Items # of 22 34 24 16 — — — — 1 1 4 2 2 8 3 1 2 1 2 7 1 4 1 1 8 2 5 1 8 3 2 4 Fees Agency 129,200 296,712 86,615 3,875 — — — — — — — — — — — — — — — — — — — — — — — — — — — — Production 323,025 449,014 441,082 32,936 13,795 50,537 ($) Costs Third-party 2,060 2,720 1,000 600 480 275 175 — — — — — — — — — — — — — — — — — — — 171,015 61,052 22,476 77,507 Talent — — — — — — — — — — — — — — — — — — — — — — — — — — — — Bulk Mail 6,388 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — 2,092 2,697 Other 5,177 50 89 89 35 — — — — — — — — — — — — — — — — — — — — — — — — — † *

Ad Value denotes the value of an ad space provided at no cost, often where the government hasprovidedfundingforarelatedevent. government wherethe Ad Value often valueofanadspaceprovidedatnocost, denotesthe posters. includesbillboardsandtransit Out-of-Home advertising 2,406,903 1,273,365 849,000 960,826 548,243 44,171 TV — — — — — — — — — — — — — — — — — — — — — — — — — — 599,197 22,708 30,011 ($) Media Costs 37,923 Radio — — — — — — — — — — — — — — — — — — — — — — — — — — — — 23,620 43,253 30,842 10,526 13,661 12,120 36,897 4,895 5,966 8,443 1,699 1,695 1,528 1,210 1,115 Print 334 505 — — — — — — — — — — — — — — — Out-of-Home* 166,835 411,415 41,872 150 — — — — — — — — — — — — — — — — — — — — — — — — — — — — Ad Value 22,894 16,682 12,516 4,100 Review of Government Advertising ($) — — — — — — — — — — — — — — — — — — — — — — — — — — — — † 1,394,006 3,914,909 Campaign 1,151,025 1,617,505 849,000 Total ($) 444,351 113,799 101,509 22,894 23,620 30,842 10,526 13,661 12,120 24,407 12,791 46,767 16,857 2,060 6,055 8,958 1,695 1,528 5,100 1,210 1,115 600 334 505 50 89 0 403

Chapter 5 Chapter 5 404 1. 2013 of Ontario of the OfficeGeneral of the Auditor Annual Report Total Veteran LicencePlates Graphic Safety Pedestrian Transportation Title Ministry/Campaign Reported in2013,Reported in2014. butmorecosts 1 Submissions 130 # of 1 1 Items 572 # of 2 2 Fees Agency 1,405,009 8,000 — Production 2,242,370 ($) Costs Third-party 1,140 — 740,243 Talent 6,729 — Bulk Mail 6,388 — — 108,038 Other 317 — 12,907,154 584,425 TV — 3,033,641 ($) Media Costs Radio — — 8,371,760 10,748 Print — Out-of-Home* 1,161,517 — — Ad Value 75,305 5,000 ($) — † 30,051,425 Campaign Total ($) 611,359 5,000 Total Veteran LicencePlates Graphic Safety Pedestrian Transportation Title Ministry/Campaign 1 Submissions 130 # of 1 1 Items 572 # of 2 2 Fees Agency 1,405,009 8,000 — Production 2,242,370 ($) Costs Third-party 1,140 — 740,243 Talent 6,729 — Bulk Mail 6,388 — — 108,038 Other 317 — † *

Chapter 5 Chapter 6 406 on itsbehalf. aspecialassignmentPublic Accounts orundertake itor any examine General matter inrespect ofthe Act Under sections16 and17 ofthe Legislativeand recommendations to Assembly. the andpresents itsobservations our specialreports relating to or matters inourAnnualReport raised year the throughout holds anumberofhearings objectives. TheCommittee their typicallyachieving in effectiveness ofprograms andthe operations efficiency ofgovernment andbroader public-sector economyon anumberofissues,includingthe and Legislative to Assembly the assesses andreports becomeavailable.they TheCommittee examines, Committee referredto as been permanently the are PublicAccounts,deemedto which havethe Auditor from the andon General tions onreports opinionsandrecommenda House itsobservations, mittee) isempowered to to review the andreport The Standing Committee onPublicAccounts (Com Role Chapter 6 Chapter , the Committee Aud may, the alsorequest the that of

the Committee on PublicAccounts The StandingCommittee Auditor General - - - Conservative Assembly, following the members: with tion 113 Legislative Standing Ordersofthe ofthe reactivated by Legislature under Sec motion ofthe Assembly hadbeenprorogued. Committee hadbeendissolved, Legislative andthe tion to itsdissolution. sessionimmediatelyfirst following a elec general Parliament, openingofits ofthe duration from the fortie. TheCommittee established the isnormally Chairmaymotions, vote whilethe only to break a except Chairaremembers entitledto vote the on Legislative Assembly.representation inthe All that party’s from any reflects given politicalparty Legislature. Thenumberofmembers motion ofthe Committee are appointed ofthe byMembers a of theCommittee Appointment andComposition Jagmeet Singh,New Democrat Liberal Shafiq Qaadri, Ouellette, Progressive Conservative Jerry Phil McNeely, Liberal Liberal Helena Jaczek, France Gélinas,New Democrat Damerla,Dipika Liberal Toby Vice-chair, Barrett, Progressive Miller,Norm Chair, Progressive Conservative 20,2013,On February Committee was the last year, timeofourAnnualReport At the the

- - ical parties annually selectstwo audits or ical parties orthree Normally,Annual Report. polit three ofthe each value-for-money from the reports chapter ofour Ittypically reviews onthem. and conductshearings and ourspecialreports, from ourAnnualReport proceedings. Legislative andother debates, Assembly speeches ofgovernment official Hansard, verbatim the report All publiccommittee proceedings are recorded in the preparation ofits reports. or briefing report agenda, Committee’s audit the dealingwith those All meetings publicexcept are opento the for House,atany timeofitschoosing. other of the approvaltive the Assembly issitting,and,with The Committee may meet weekly Legisla whenthe our Office. Public Accounts andany issuedby specialreports relating to Ontario’s ourAnnualReport, hearings itsreviews Committeeings to and with assist the attends allcommitteepanied by meet seniorstaff, General Act section16In accordance with ofthe Conservative was revised by Legislature asfollows: amotion ofthe OPERATIONS COMMITTEE WITH THECOMMITTEE AUDITOR GENERAL’S The Committee identifies matters ofinterest Jagmeet Singh,New Democrat Ouellette, Progressive Conservative Jerry Phil McNeely, Liberal Bill Mauro, Liberal Liberal Helena Jaczek, France Gélinas,New Democrat Lorenzo Berardinetti, Liberal Toby Vice-chair, Barrett, Progressive Miller,Norm Chair, Progressive Conservative On September 9,2013, Committee membership , the Auditor, the sometimes General, accom PROCEDURES

ADVISORY ROLE AND Auditor

- - - - tions to the Legislativetions to Assembly. the mittee provides itscommentsandrecommenda are completed,attend. Onceitshearings Com the policymatters, ministers are rarelythan asked to administrativeoperational, rather andfinancial dealswith BecauseourAnnualReport members. andrespond to questionshearings from committee the auditee(s)the senior officialsfrom to appearat auditorreview.the TheCommittee typically asks subjectof public-sector was the organization that ministry,tions from the Crown agency orbroader responses to ourfindingsand the recommenda and applicablesectionfrom ourReport on the researcher, Committee Committee’s the the briefs tee review. for sectionsfromCommit ourAnnualReport other Committee oronlineat Clerkters are publicly ofthe the available through andlet allcommitteedations. Oncetabled, reports commentsandrecommen Committee’s include the itsmeetings Committee and by during gathered the information the andletters summarize reports work Legislative Assembly. for inthe tabling These andlettersThe Committee onits issuesreports tee andrespond to questions. witnesses were calledto testify Commit before the meetings includedanumberofsessionsinwhich Ornge AirAmbulanceandRelated Services related 2012 March to ourOffice’s special report, ings were administrative others innature andthe 2013–September 2013 meet Afew period. ofthe The Committee met 14 February the timesduring inourreports. raised recommendations madeandconcerns address the to are Committee taking update onactionsthey the were auditees that notthose selected for hearings REPORTS MEETINGS At each hearing, the Auditor the alongwith General, hearing, At each The Clerk of the CommitteeThe Clerk alsorequests ofthe that The Standing Committee on Public Accounts Committee The Standing OF HELD THE COMMITTEE www.ontla.on.ca . These . ------407

Chapter 6 Chapter 6 408 mittee for intends to report issueasecondinterim the Committee. identified TheCom by concerns heldin2012,mittee hearings includingongoing 20com results ofthe the summarized This report report, 2012 March onourOffice’s report interim special area. operational tion orauditsofthat into inany consideration subsequent follow-up sec recommendations andresponses, andwe take them a stipulatedthese OurOffice timeframe. reviews provide Committee responses within Clerk the with istry, agency orbroader public-sector organization min dations andrequestmanagement ofthe that 2013 of Ontario of the OfficeGeneral of the Auditor Annual Report In June2013, Committee released itsfirst the typicallyCommittee includerecommen reports Ornge AirAmbulanceandRelated Services - . - - - - to 27, 2013. andwas heldinReginakatchewan from August 24 Auditors to discussissuesofmutualinterest. CanadianCouncilofLegislative the conference with tees from across Canada.CCPAC holdsajointannual publicaccountscommit provincial andterritorial tees (CCPAC) consists ofdelegates from federal, The CanadianCouncilofPublicAccounts Commit recommendations. Committee’s 2013the the containing andafinal report hearings, COMMITTEES ACCOUNTS CANADIAN The 34th annualconference wasThe 34th hosted by Sas COUNCIL OFPUBLIC

- - - ance of the province fiscal ance ofthe overyears. three next the financialperform for the onitsexpectations report pre-election government’s reasonableness ofthe required to review and deliver an opiniononthe gov.on.ca. actscanbefoundfunction). Both at review advertising Office’s for more details onthe proposed government (seeChapter advertising 5 ornotdeciding whether to approve typesof certain Auditor isresponsible General for reviewing and under the ofthe authority taxpayers. Ontario money spentonbehalfof for obtain the value they spendpublicfunds,andfor the they prudently for recipients accountable howtors andgrant government,Assembly itsadministra holdthe the Legislative Officehelps Insodoing,the them. onand financialaudits reviews, and reporting by conductingvalue-for-moneycitizens ofOntario the Legislativethe Assembly and (Office) serves the AuditorThe Officeof ofOntario General Chapter 7 Chapter In an election year the AuditorIn anelectionyear the isalso General the under The work Officeisperformed ofthe Government Advertising Act,2004 Advertising Government Auditor General Act ofOntario General The OfficeoftheAuditor . Inaddition, www.e-laws. , the , the - - considered in this assessment: considered inthis to be 12(2)(f)(iv)the criteria and(v)]identifies following entities: to conductvalue-for-moneyauthority auditsofthe the TheOffice has service. effectiveness ofthe relativebeing delivered publicandthe cost- to the underlying level to operations assessthe of service Value-for-money auditsdelve auditee’s into the manages oractivities. andadministers itsprograms well we agiven entitythat audit) “auditee” (the value-for-money assesseshow auditing,which Office’s work two-thirdsAbout ofthe relates to VALUE-FOR-MONEY General ANNUAL REPORT • • • • • The

economy. boards, anduniversities). homes,school long-term-care hospitals, aidsocieties,children’s communitycolleges, services, provideagencies that mental-health receive (forthat government example, grants Money should be spent with dueregardMoney for should bespentwith broader publicsectororganizations inthe Crown-controlled and corporations; Crown agencies; government ministries; Ontario Auditor General Act Overview AUDITS (Act) [insubclauses IN THE

409

Chapter 7 Chapter 7 410 money auditsis: work is typically includedinourvalue-forthat compliance(discussed inalater section).Other into value-for-money both auditsand“attest” audits pliance” auditwork, butisgenerally incorporated istechnically “com risks controlling against these following whereinstances the was observed: developed value-for-money for each audit. ororganization beingauditedprogram are also ministry, particular directly ofthe operations to the havethat relate not beenmet. More specificcriteria value-for-money three where the above criteria on any heorshemay instances have observed 2013 of Ontario of the OfficeGeneral of the Auditor Annual Report • • • • • • • Assessing the extent to which the auditee was extent the to which Assessing the on The Act alsorequires heorshereport that Auditor The Act requires the report General that

• • • requirements. key these legislationand authority with management auditee’s hascomplied the that to reasonable assurance obtain necessary responsible for and administering; management auditee’s is the that as those andactivitiesaswell programs auditee’s the auditee govern or the that authorities and the itwas onesfor appropriated. which the ficient to: and procedures appliedwererules not suf money was not fully accounted for. programs. effectiveness of onthe measure andreport efficiency. Money was expended for purposes other than than other Money was for expended purposes Essential records were not orthe maintained Accounts were not properly kept orpublic proceduresAppropriate shouldbeinplaceto dueregardMoney for shouldbespentwith performing the tests andprocedures the we deem performing keyidentifying the provisions inlegislation

ensure that expenditures were expenditures ensure that madeonly assessment,collection effectively the check safeguard andcontrol publicproperty; as authorized. and proper allocationofrevenue; or - - Canadian Institute Accountants). ofChartered Professional Accountants ofCanada(formerly the professional Chartered by standards established the value-for-money work, we follow relevant the reviews estimates. oflegislationandexpenditure legislativequestions sessionsandthrough during government andchallenging policiesthrough itoring foraccountable policymatters by continually mon government Legislative holdsthe Assembly that ofgovernment policy. merits on the Rather, itisthe note, however, indoingsowe that donot comment ing government to policydecisions.Itisimportant how well management andexecut isadministering ourvalue-for-moneycould say that auditsfocus on result ofgovernment policydecisions.Thus,we choose higher-riskchoose audits,we as: considerfactors such Toconsequences public itserves. for the help us results inpotential which negative money criteria, value-for- anauditee isnot three that meeting the year,to auditeach is we risk considerhow the great well asseveral large Crown-controlled corporations. agencies), as dren’s aidsocieties andsocialservice (chil universities andcolleges), andsocial services boards, providers), education(school service health homes,andmental- long-term-care (hospitals, range oftopics inseveral sectors, includinghealth auditees, ouraudits haveine these covered awide toour mandate in2004 expanded allow usto exam activitiesare numerous anddiverse. Since and their are samecycle somany because there the ofthem, public sector andCrown-controlledon corporations vals. We broader donot auditorganizations inthe activities atapproximately five- to seven-year inter and programs The Officeauditsmajor ministry followingin the sections. work. processes we Someofthe useare described quality, the for ensuring and valueofour integrity These standards require we that have processes Selecting What to AuditSelecting What to In planning, performing and reporting onour andreporting In planning,performing andactivitiesareGovernment the programs In selecting what program, activity or organization activityororganization In selectingwhatprogram, - - - - - recommendations. Each audit report hasasection auditreport recommendations. Each and our auditobjective andmake observations we aconclusionregarding sothat canreach teria, procedures to address ourauditobjective andcri audit. planningstagemanagement ofthe atthe we senior auditee’s develop the are discussedwith able laws, authorities. regulations andother fully auditsorreviews; appliedinother andapplic success policies andprocedures; applicable criteria own management’s andservices; similar programs delivering bodiesorjurisdictions other of expertise; sive into asrecognized bodies research sources such ing effectively. involves Developing criteria exten shouldbeinplaceandoperat and procedures that cover key that the systems, policies audit criteria what we want to achieve. We develop then suitable When we beginanaudit,we set anobjective for auditworkresults ofinternal inourauditreport. of ouraudit,butwe dorely onandpresent the cases,we Inother scope donot diminishthe effort. scopeto avoidor change ouraudit’s duplicationof workwhat that consists of,we may defer anaudit auditors have completed orplanned.Depending on work internal auditee’s selection process isthe the auditjustify itscost. ducting the and Assurance Levels and Assurance Audit Objectives, Audit Criteria,Setting • • • • • • The next stepThe next isto designandconducttests and To suitability, ensure their criteria further the factorAnother we take into accountinthe We benefitsofcon the alsoconsiderwhether

operations; and operations; operations; public; ization onthe follow-ups; identify. the significance of the issuesanauditmight significanceof the the auditee’s changesrecent significant in complexity auditee’s anddiversitythe ofthe activityororgan program, impact ofthe the total revenuesthe involved; orexpenditures results ofprevious auditsandrelatedthe ------it isconclusive, andwe must rely onprofessional concluding onourobjective ismore persuasive than evidence available for ofthe Also,much arise. so we problem willnever cannot the guarantee that might beableto circumvent control systems, such butauditee management orstaff from occurring, working effectively to prevent problem aparticular system was inplacefor aprocess orprocedure that auditeewe hadacontrol the may concludethat For to factors alsocontribute this. Other example, our auditwork identifiesallsignificantmatters. what iscalledan“absolute level that ofassurance” sowemation hasitslimitations, cannot provide is explained. audit objective scopeofourwork isstated andthe entitled “Audit Objective andScope,” the inwhich inquiries and discussions with management; anddiscussionswith inquiries through ate level primarily obtained ofassurance, anaudit.Areview provides than a moder rather Act. responsibilities underthe to wefulfillour decisions that deemnecessary inCabinet contained submissionsor information However, all Officecanaccessvirtually other the do not ofCabinet. deliberations seekaccessto the ofCabinet privilege,we principle respect for the ofourduties. Outof performance to the necessary to andrecords accessallrelevant information ment provides, Act we underthe are entitled andadvice. assistance independent expert we ahighly are area, examining technical obtaining because independent sources; and,where necessary facts with procedures confirming andtransactions; mation itprovides; andtesting examining systems, management infor andanalyzinginterviewing the cifically, anauditlevel isobtained ofassurance by usingourregular auditprocedures.obtain Spe highest reasonable we level that ofassurance can for ourwork to provide isatan“auditlevel”—the interpreting information. ofourwork—forjudgment inmuch example, in Conducting tests andprocedures infor to gather Infrequently, a review Officewillperform the manage that information respect toWith the we that assurance plan For reasons, the allthese The Office of the Auditor General of Ontario The General Office ofthe Auditor - - - - - 411

Chapter 7 Chapter 7 412 minister or head of the agency,minister orheadofthe or corporation the deputy Auditor the finalizeswith General which report, draft into the discussed andincorporated ten responses to ourrecommendations. Theseare senior management, providesauditee’s which writ the prepared anddiscussed with isthen report results. Adraft onourpreliminary ment isbriefed conclusionofon-site work,cation. At the manage review andensure open linesofcommuni progress audit orreview, management meet to with ourstaff focus, ofourwork the ingeneral During terms. andthe objectivement to and criteria, discussthe manage meetEarly with process, ourstaff inthe value-for-money the throughout auditorreview. seniormanagement auditee’s the municate with from ourOfficecom tions andconclusions,staff To ofourobserva factualaccuracy helpensure the process more effective. the Estimates,annual expenditure andways to make process usedto review and approve province’s the liability.reduce that In2012, we reviewed the actionsbeingtakenan ongoinginterest to inthe Committee hasshown onPublicAccounts, which BoardInsurance was well received Standing by the Unfunded Workplacethe Liabilityofthe Safety and sector. electricity sight ofthe Our2009review of of renewable over energy initiatives andregulatory complemented ourrelated value-for-money audits sector stranded debt,which electricity review ofthe reviews when,for example: tems, procedures We andtransactions. perform and only limited andtesting examination ofsys providedanalyses ofinformation by management; 2013 of Ontario of the OfficeGeneral of the Auditor Annual Report Communicating with Management • • In the 2011In the audityear, we a conducted such

to anaudit. conductareview than oractivitymake itmoreprogram appropriate ance; or to provide ahigherlevelunnecessary ofassur other factors relating to the nature of the factors nature relating ofthe other to the it would beprohibitively expensive or ------the PublicAccounts. Underthe section17, Legisla the onany andreport mattermust examine respecting lic Accounts may Auditor resolve the General that Under section16, Standing Committee the onPub additionalspecialwork. toGeneral undertake AnnualReport. not untilthe bedeferred Auditor opinionofthe should General, inthe that, Legislative Assemblythe atany time,onany matter Officemay makeIn addition,the to aspecial report Legislative to Assembly.its inanAnnualReport the onitsaud As required Office reports Act, by the the is publishedinChapter AnnualReport. 3ofthe report the which organization, after grant-recipient tion 17, we onthe prepared aspecialreport Public Accounts: under section17 Standing by Committee the on followingyear were the specialreports requested on completion time.This andmadepublicatthat Legislature inthe willbetabled specialreport the that requester’s agreement has beento the obtain request undersection16 or17, practice ournormal duties. believes other itconflictswith an assignmentrequested by aminister ifheorshe Auditor duties, andthe candecline such General to take precedence over Auditor the other General’s ment. However, specialassignmentsare not these Auditor aspecialassign the undertake General that Accounts, Crown oraminister ofthe may request tive Assembly, Standing Committee the onPublic SPECIAL REPORTS • • • Two Auditor the Act authorize sectionsofthe At the requestPremier,At the of alsoundersec In recent years whenwe have received aspecial

progress at the time this chapter was written). timethis atthe progress Tracks Slots (in atRace lation ofthe Program PlanImplementation andCancel ernization chapter and was written); timethis at the sion Divestiture Savings Estimates (inprogress 2013); inApril (tabled Mississauga Power PlantCancellationCosts Ontario Lottery and Gaming Corporation Mod andGamingCorporation Lottery Ontario Transportation Northland Commis Ontario

Oakville -

------(2), and(3)]requires that: Crown. Specifically,the Act [insubsections9(1), province accountsofmany agencies andthe ofthe year,Every the we financial statements of audit the intended for. tures; andnot spendingmoney onwhatitwas properly expendi allocatingrevenue; unauthorized deficient procedures for assessing,collectingand ing; inadequate safeguarding ofpublicproperty; unaccounted-for ofrecordkeep publicmoney; lack relating to improperlyaging risks kept accounts; work. Specifically, the controls for man we assess audit work into attest incorporated audit isoften overview ofvalue-for-moneythe audits,compliance accepted Asmentionedin accountingprinciples). Canadiangenerallypolicies (inmost cases,with accounting certain complies with is fairandthat andfinancialpositionina operations that way auditee’s cial statementsonthe present information finan the hisorheropiniononwhether expresses the auditor audits, financial statements. Insuch Attest ofanauditee’s auditsare examinations October 2013. Power PlantCancellationCosts The Auditees ATTEST • • •

General; and General; Auditor results to the their andreport General Auditor direction ofthe auditsunderthe their Crown agencies ofthe perform of certain not audited by auditor; another Crown agencies are ofthe ments ofthose that orotherwise; trust Consolidated Revenue Fund, heldin whether province’s ofthe part public money forming receipt of records anddisbursement ofthe public accounting firms appointed asauditors public accountingfirms Auditor financial state the auditthe General Auditor accountsand the auditthe General AUDITS

, which was tabled in was tabled , which - - - - - related agencythe deputy financial statements to andofthe reports of ourindependentauditor’s agency.sible for the OurOfficealsoprovides copies agency’s minister(s)to board respon the andthe Auditor responsibilities are the reporting General’s Agency stipulates legislationnormally report. that agencies andCrown-controlledinthis corporations province’s consolidated financial statements. financial statements. the province’s consolidated andincludedin firms accounts are also audited by publicaccounting broader publicsectororganizations whose inthe 2012/13 the during audit year. Exhibit3lists respectively, audited publicaccountingfirms that Crown Crown-controlledthe andthe corporations, agencies 2ofExhibit1andlist of the Part behalf. Office’s agencies onthe number ofthese to audita publicaccountingfirms with contracts 2012/13 the audited during audityear. TheOffice our AnnualReport. Legislature, weattention ofthe would includeitin that significant we the felt itshouldbebrought to audit committee Ifamatter (ifoneexists). were so the agency’s isdiscussedwith which a final report, auditor prepares the ment responds to itinwriting, agency’s iscleared seniormanage andthe report the draft that discussion.After the results of reflect seniormanagement.isrevised to Thereport it with anddiscusses findings report itor prepares adraft ing management to make improvements, aud the Treasury ofthe Board. Secretary associatedminister ministry, ofthe aswell asto the • We results ofattest donot auditsof discussthe attestChapter auditofthe year’s 2discussesthis Part 1 of Exhibit 1 lists the agencies that were agencies 1ofExhibitlists that the Part When anagency attest auditnotes areas requir

contained inamanagementcontained letter). recommendations to management (typically ofitsfindingsand report accounting firm’s andacopy ofthe corporation ments ofthe audited acopyGeneral financial state ofthe deliver Auditor tocontrolled corporations the public accounting firms auditingCrown-public accountingfirms The Office of the Auditor General of Ontario The General Office ofthe Auditor - - - - - 413

Chapter 7 Chapter 7 414 its recommendation to the Legislativeits recommendation to Assembly. the accounting profession. made TheCommittee then tive memberfrom the Assembly; andanexternal Legisla Clerk Speaker ofthe Accounts; andthe the Standing Committee Chairofthe onPublic ing the Assembly, inthe politicalparties three includ the conducted by acommittee ofrepresentatives from interviews in nationalpublicationsandin-depth position the includedadvertising conducted, which past, anopen-competitionthe selectionprocess was Committee, seeChapter onthe information 6).As in consulted appointmentismade(for before more the Assembly, officialopposition—be isamemberofthe Legislative Standing Ordersofthe under the Standing Committeethe onPublicAccounts—who, Assembly. Chairof TheAct alsorequires the that appointee must bealsoapproved Legislative by the Assembly,” address ofthe “on the the meaningthat Cabinet). Theappointmentismade Council (the ExecutiveAuditor adviceofthe onthe General Lieutenant Governor is,the appoints the cil—that Assembly Lieutenant by Governor the inCoun the Legislative isappointedGeneral asan officerof Under the year auditmandate. ofourexpanded eighth the This government grants. receive broader publicsector that izations inthe value-for-moneyOffice’s auditmandate to organ expansion ofourchangesthe these was nificant of Legislative Assembly. the to serve Themost sig changes totain the Amendment Act was to ofthe make cer purpose received Royal AssentonNovember The 30,2004. AmendmentAct Law sage onNovember ofthe 22, 2004, The 2013 of Ontario of the OfficeGeneral of the Auditor Annual Report Appointment of Auditor General General Appointment of Auditor GENERAL OTHER Auditor General Actc OF STIPULATIONS Auditor General Act ACT

Audit Act (Amendment Act), which ame about with the pas the ame aboutwith 2013 AnnualReport to enhanceourability (Act), the Auditor(Act), the THE Audit Statute AUDITOR marks marks - - - - -

- - government’s administrativegovernment’s process—reviews and legislative isindependentofthe committee that political pressure. legislated mandatethe Office’s without to fulfill Legislative free Assembly inthe andisthus parties political government andthe from the distance Auditor anarm’s-length the maintains General Legislativecause by Assembly. the Consequently, non-renewable andcanbe dismissedonly term, for responsibilities objectivelyreporting andfairly. Office to fulfillitsauditingand enablesthe that tion. Thisindependenceisanessentialsafeguard government anditsadministra independent ofthe Officeare The Auditor ofthe andstaff General Office, thus further ensuring confidentiality. ensuring thus further Office, ofanaudit,cannotcourse beaccessedfrom our the from anauditee obtained during information andauditworkingreports includingall papers, and Protection Act ofPrivacy exemptOffice is the from Assembly orany ofitscommittees. Aswell, our donotpapers have to Legislative belaid before the Underpapers. section19 working Act, these ofthe ofourauditworking part considered anintegral that are and findings reports auditreports draft activities,we ofourreporting prepare course In the chapter. endofthis at the year forcussion ofexpenditures are presented the Assembly. Theaudited statements andrelated dis Legislativeand subsequently inthe must betabled Officeare submittedthe Boardstatements ofthe to audited financial andthe accountants, chartered 2012/13 fiscal year have beenaudited of by afirm expendituresthe relating Office’s to Act,by the the Legislativelaid before Assembly. the Asrequired approves budget, Office’s issubsequently which the Independence CONFIDENTIALITY The Board of Internal Economy—an all-party Economy—anThe Board ofInternal all-party The Auditor isappointed General to a10-year, OF Freedom ofInformation WORKING , which means our draft meansourdraft , which PAPERS -

- agement Committee. SeniorMan Office’s makement Advertising upthe Resources andofCommunicationsGovern Managers Directors,ofHuman andthe the eral, (seeFigurestaff 1). audit other number ofauditManagers andvarious teams Directors are a androunding outthe the Assisting assignedportfolio. the auditswithin the by aDirector, whooversees andisresponsible for headed organization, are owneach ment’s ministry ity. govern loosely basedonthe Theportfolios, areas ofauditactiv various inthe foster expertise intended related to auditentitiesandto alignwith The Officeisorganized intoteams, portfolio fivethereafter.upon beinghired andevery years check andundergodeclaration apolicesecurity required to complete anannualconflict-of-interest conflict-of-interest situations.Allemployees are sure requirements steps to andthe betaken to avoid itees. TheCodealsoprovides guidanceondisclo Legislative Assembly,the publicandouraud the responsibilities Office’s to the describes and further work.competence intheir andintegrity higheststrive to standards ofbehaviour, achieve the conduct themselves inaprofessional mannerandto for employees Office,whohave ofthe aduty to philosophy, regarding conduct and rules principles The Codeisintended to beageneral statement of dards andensure aprofessional work environment. to highprofessionalencourage maintain stan staff The OfficehasaCodeofProfessional Conduct to Personnel Office CODE The Auditor DeputyAuditor the General, Gen exist whyexpectations The Codeexplains these OF Organization PROFESSIONAL

CONDUCT and

------sharing ideasandexchanging information. sharing andprovides for ausefulforum inces andterritories, federal provAccounts government from the andthe Standing Committees ofthe onPublic members togethermittees. Itbrings legislative auditors and CanadianCouncilofPublicAccounts Com of the annualconference the years, beenheldjointly with 2013. has,for anumber of Thisannualgathering inRegina, from AugustAuditors 25to (CCOLA) 27, CanadianCouncilofLegislativemeeting ofthe This year, 41st hosted Saskatchewan the annual section 17, andcompleted majority ofwork the on its (seeChapter 3),issued two under specialreports additional work requested year. this Office,particularlythe unprecedented givenfor the The 2012/13 fiscal successful year was another year zania andVietnam. from Taiwan, Bangladesh, Cameroon, Ghana,Tan and Yunnan Province inChina,aswell asvisitors Vietnam (2),Gauteng Province Africa, inSouth Office hosted delegations from the report, by this audityear covered the During audit experiences. Office and to share ourvalue-for-money and other roles andresponsibilities ofthe to discussthe to visitors anddelegations meet from abroad with receives Officeperiodically requestsauditing, the As anacknowledged leaderinvalue-for-money Legislative Canadian Council Office Results Produced International In total, we conducted 10 value-for-money aud The Office of the Auditor General of Ontario The General Office ofthe Auditor This Year Auditors Visitors

by the of

- - - - 415

Chapter 7 Chapter 7 416 4. 3. 2. 1. 30,2013 September Figure 1:OfficeOrganization, 2013 of Ontario of the OfficeGeneral of the Auditor Annual Report Deputy AuditorGeneral Gar Financial StatementAuditPo Va Bonnie L Megan Sim Mar K To Wa John McDowell,Director Crown A and Revenu Community andSocialSer Auditor Genera Michael Okulicz Jennifer Lee Inna Guelfand Ti Celia Y Sandy Chan V Nick St Kim Cho,Manager Gus Chagani,Director Kris Lauren Hanna Ka Marcia DeSouza Bar Environment andNaturalResources A member of the portfolio who contributed to this Annual Report but left the OfficebeforeSeptember30,2013. the butleft AnnualReport whocontributedtothis portfolio A memberofthe Manager’s timeisdivided50/50between twoportfolios. auditworkloadpressures. statement toaddressseasonalfinancial between portfolios belowmanagerlevelshift Staff September3,2013,Effective JimMcCarter. replacingretiring AuditorGeneral andy Fletcher anna Gotsis,Director no Bove m Chatzidimo lue-for-money AuditPo lter Allan,Manager trina Exalta y Peall y R tosz Amer ty Ho eung, Manager omano av gencies (1),Finance ysyk ropoulos, Manager e 3 1 , Manager ski cion s l vices, Janet Wa Zhen Gurinder Par Li-Lian Koh Jing W Alexander Tr Shre rf olios andStaf ya Shah ya Stekovic rt ang folios and n uong mar Quality AssuranceandSpecialProjects Human Resources Kris Paul Amodeo,Manager Shariq Saeed Sha Barbar St Re andar becca Yo f St tin Snowden,Manager yna Whiteford 2 af a Stur f ds andResearch 2 Jennifer Fung Dimit Sally Chang Kevin Aro Naomi Herber We Susan Klein,Director Cons Mar Izabela Beben T Laur Health andLong-term-careProvider Crown A Rumi Janmohame Education andTr Johan Boer Michael Baxter Emanuel T Zahr Ger Nina Khant Michael K sipovich, Manager eresa Carello,Manager rock, Manager ndy Cumbo,Manager garet Chen ard F a Bell,Director a Jaf t ar Dimitrov antino DeSousa gencies (2 itzmaurice, Director fer atsevman sikritsis, Manager , Manager(Acting) g, Manager aining d ) 4 Ti Linde Qiu V Ingrid Goh Cynt Zachar Ro Myt Ta Dor Ellen T Mark Smit eronica Ho ff ra ger Munroe an a Ulisse hili Pr hia Ts Petroff y Ya y Thomas epelenas u at s o h heeskar Communications andGovernmentAdver Administratio Information T Operations Chris Shant Sohani Myer Maureen Bissonnette Peter Lee Shams Ali T Shirle Mariana Green Chris iina R Energy andHealth Mar Alfred Kiang T Kir Ra Jesse Dufour Vivian Sin,Manager Fr Rick MacNeil,Manager Vince Mazzone,Director Ar Denise Y Gigi Y Rudolph Chiu,Director Helen Chow Anit Ariane Chan Justice andRegulator tine Wu tine Pedias,Manager Allen Fung Lorett Geor Sandy Chan Bill Pelow Whitne Public Accounts anmay y McGibbon a Per aser Ro andoja ujunan Balakrishnan shmeet Gill an Grewal garet Lam echnology a Cheung n gegiana T ip, Manager a Cheung saud y Wa Gupt s oung, Manager gers , Director 4 h 3 a , Manager , Manager anudjaja y Brian W Michael Ya Ro Ruchir Patel Alice Nowak We Lisa Li Alla V Pasha Sidhu Oscar R b ndy Ng yn Wilson tising Review olodina 4 anchuk odriguez rm olinsky Prepared by the Office of the Auditor General ofOntario the AuditorGeneral Prepared by Officeof the Figure 2:Five-year ComparisonofSpending(AccrualBasis)($000) the financial results for Office’s ments outlinethe The following discussionandourfinancial state advisers. andexpert agent auditors, staff contract aswell ofour asthat and dedicationofourstaff, clearly not hard work have the beenpossiblewithout discussed inChapter 5. further under the conducting ourfinancial-statement audits. for standardsand assurance andmethodology fully implement significant revisions to accounting continuing ourinvestment to intraining success our key financial-statement auditdeadlineswhile 40Crownmore than agencies. We again met allof cussed inChapter 2),aswell statements of asthe dis consolidated financial statements (further tion, we province’s are responsible for auditingthe lineonourexpenditures. while holdingthe more urgenttion ofthe specialwork requested, all followingbe postponed to the year to allow comple planned value-for-money hadto audit,CivilCourts, to follow. reports two requeststhe other with One Rent services Professional andother Salaries andbenefits Actual expenses budget Approved * Returned province* to Total Other Travel andcommunications Financial Accountability assets, deferred leaseinducementsandemployeeassets, deferred futurebenefitaccruals). ofcapital excessofappropriationoverexpensesasaresultnon-cash (suchasamortization the than These amountsaretypicallyslightlydifferent The results produced by the Office this year Office The results produced bywould the We successfully met ourreview responsibilities earlier AttestAs mentionedinthe Audits sec Government Advertising Act,2004 Advertising Government

2008/09 , as 14,534 16,245 - 10,279 1,561 1,096 1,051 1,776 - - - 332 - statutory responsibilities under the responsibilities underthe statutory agencies. Theremaining 1%was devoted to our provincestatements ofthe andsome40ofits to annualfinancial completing auditsofthe the wason PublicAccounts. devoted one-third About Standing Committee ofthe audits, astated priority were value-for-money usedto perform andspecial almost two-thirdsindicates ofourresources that is provided inNote financial statements and 9to the review ofgovernment Thisbreakdown advertising. the special audits,financial-statement audits,and our Officeis responsible for: value-for-money and a breakdown mainactivities by ofourexpenses the standards, we these haveaccordance with presented public-sector accountingstandards. In ance with financial statements have beenprepared inaccord 2012/13 fiscal year.the second year our Thisis 2011/12) andbenefitcosts relatedfor to salary spending andshows over that 74% (72% in Figure major 3presents the components ofour budget over andexpenditures last five the years. Accounting Standards time. first for the year to helpseveral agencies to adopt PublicSector was required Pre-Election Report needed for the this year was not required year. this Thetimepreviously early Pre-Election 2011/12 Report the inthe fiscal slightly from 2012, review mostly of becausethe Act Advertising 2009/10 Figure 2provides acomparison ofourapproved 14,853 16,224 10,862 1,498 1,489 1,069 1,073 360 The Office of the Auditor General of Ontario The General Office ofthe Auditor . Thesepercentages changed only 2010/11 16,224 15,163 11,228 1,222 1,036 1,491 1,071 337 2011/12 16,224 15,241 11,039 1,667 1,016 1,216 303 997 Government Government 2012/13 15,346 16,224 11,390 1,000 1,643 1,015 309 989 - 417

Chapter 7 Chapter 7 418 following previous year. a5.2%increase the year, prior 3.2% the whilebenefitcosts rose 0.9% adecline of costs increased 3.7%after Our salary challenges we andsome ofthe expenses facefollows. increases to ourbudget over last four years. the have there beenno factthat andthe circumstances, province’s tight fiscal upbecauseofthe to fully staff sector.private Inaddition,we have beenreluctant professionalspensation increases for inthe such ranges havesalary simply not kept com pacewith competitive Toronto jobmarket—our public-service the and retaining qualified in professional staff wethat have historically facedchallenges inhiring totalling is $6.3 million.Themainreason for this Office mandate unspentfunds while returning five years, yet we have successfully fulfilledour budget. Ourbudget hasbeenfrozen over last the in 2011/12) andwere again significantly under haveportions beenrelatively inrecent stable years. remainder.rent, comprised most ofthe Thesepro and whileprofessional services, andother staff, ofOntario the AuditorGeneral Prepared by Officeof the 2012/13) Figure 3:Spendingby MajorExpenditureCategory, 2013 of Ontario of the OfficeGeneral of the Auditor Annual Report SALARIES A more detailed discussion of the changes in our changes inour A more detailed discussionofthe Overall, increased just ourexpenses 0.7%(0.5% AND BENEFITS

(7 benef Salaries and (1 ot Professional and Re Ot (2.0%) communications Tr av her services 4.2%) 0.7%) her (6.6%) nt (6.5%) el and its - - 120 accountants rise fairly five rise first accountants quickly inthe years for qualified becausesalaries paid astrainees, they than considerablywere morequalified staff us.Towith becompetitive, we must pay ournewly year remained the accounting designationduring professional their asstudentstrainees, whoearned in Figure 4.We continueto employ fewer student to 106staff year of104, prior from the asshown year, ouraverage with levelrising staffing by two increasenew this hires accounted ofthe for much However, staff. in replacing anddeparting retiring responsibilities, by continuedto delays beoffset abilitytowho demonstrated take onadditional the year, the designations during staff andfor those professional accounting their ees whoobtained any increases dueto by promotions train earned Prepared by the Office of the Auditor General ofOntario the AuditorGeneral Prepared by Officeof the 2005/06–2012/13Figure 4:Staffing, junior levels, andbenefitsare where oursalaries atmore continuesto beprimarily ties. Ourhiring remained fairly robust despite economicuncertain marketas the for professional has accountants continue departures arose. Staff as opportunities butmore juniorstaff experienced andhiring staff tious by replacement senior delayingofretiring the ing needfor we cost containment, remained cau following qualification. 10 10 110 115 70 80 85 90 95 75 2005/06 0 5 With the legislated freeze on salary ranges, legislated freeze onsalary the With With the economic uncertainty and the continu andthe economicuncertainty the With 105 16 89

2006/ Actual a Unf 115 illed positions 16 07 99

20 ver 07 age FTEs 117 /0 110 8 77 2008/09 1171 110

2009/1 10 13 0 17 4 20 10 117 /1 10 1 13 4 20 11 117 10 /1 15

2 2 20 12/1 117 10 13 - -

3 4 - - izing the financial-statement auditsofCrownizing the agen more complex work andtightdeadlines for final sionals to meet ourlegislated responsibilities given arrangements. andsecurity structure previous year to inacontract review ourITinfra year by was ourone-timeinvestmentthis offset the Anincrease professional incontract help months. help usmanage summer peakworkloads the during to cover for andunexpected parental leaves, andto last year,than butwe continueto staff usecontract total Thesecosts expenditures. were slightly less most significantspendingarea, atalmost 11% of CPAand contract and represent ournext firms, professionals contract includeboth These services disclosed inNote 7to ourfinancial statements. 2012 inthe highest-paid staff calendaryear are ofour for professional Thesalaries accountants. competition($150,400), are ourprimary which CPAlower atprofessional service those firms than sector ($129,900) and,most importantly, 26% were 14% not-for-profit lower inthe those than average for CPAs salaries ingovernment ($111,200) sional Accountants ofCanada),publishedin2011, Accountants (now Profes known Chartered asthe CanadianInstitute by ofChartered the survey sectors offer.private According most recent to the not-for-profit the both that the salaries and the with government. Theseranges remain uncompetitive ranges ofsimilarpositionsinthe salary able to the retire overenced staff few next years. the its willonly increase asmore ofourmost experi aud these and enhancingourcapacityto perform aspossible.Thechallenge ofmaintaining as much audits requires highlystaff qualified,experienced growingunfilled positions.The complexity ofour asFigurethat, 4shows, we still have anumberof enced professional Thisisonereason accountants. broader-public-sector scalesfor more experi salary competitive. We quickly and fallbehindprivate- PROFESSIONAL We continueto have to rely profes oncontract levels mustUnder becompar Act, oursalary the AND OTHER SERVICES - -

------to implement ongoingchanges to accountingand required additionalhours andthe staff pay their they highersalaries remain higherbecauseofthe up to ourapproved complement of117 staff. of budget freezes, we cannolonger to afford move Further,post-qualifying four years experience. after particularly for several professionals years with of complement, given ouruncompetitive levels, salary remained difficult for us ourapprovedto reach full economicdownturn, ithas Also, even the during tion ifin-year cutsto ourbudget are requested. itprovidesthat andlessdisrup more flexibility economictimes,in uncertain particularly during to leaves,parental approach staffing, isaprudent as to needs,such fill moretemporary staff contract province.cies andthe We alsobelieveusing that our communications costs through achange in our communicationscosts through increase intravelsome ofthe costs by reducing relative to last year. However, we were ableto offset out generallyits we required carried more travel audits selected. Thisyear, value-for-money the aud ever, year dependingonthe each willvary these audit broader-public-sector organizations. How travel ofour mandate expansion to costs sincethe year. Ingeneral, significantly wemore are incurring adeclineofover 2%after 10%than previous the Our travel andcommunications costs increased less renewal completed fallof2011. inthe lease ofthe terms underthe or declinefurther centage oftotal spendingandshouldremain stable utilities. Accommodation costs declinedasaper costs,decline inbuildingoperating particularly previous year, the less than to owing a primarily Our costs for accommodationwere again slightly have achieved savings insomecases. andwe expire, ascontracts ket services for such standards.assurance We continueto mar test the TRAVEL ANDCOMMUNICATIONS RENT Contract costsContract for CPA firms we work with

The Office of the Auditor General of Ontario The General Office ofthe Auditor - - - - - 419

Chapter 7 Chapter 7 420 year aswe continuedto carefully manage ourcosts. tising Act forexpenses the administering equipment maintenance, andstatutory training, 2011. Ourcosts forsuppliesand asset amortization, we which oninJune reported Election Report, to complete 2011 review ofthe ourstatutory Pre- 2011/12 fiscal year, we requiredexpertise contract year. from specialists contract this expertise Inthe last year, becausewe required primarily farless costs were Such expenses. 16.5%tory lower than and equipment maintenance, andstatu training, supplies costs includeasset amortization, Other review conductedsecurity last year. and the ITinfrastructure identified by opportunities providers, andby implementingservice cost-saving 2013 of Ontario of the OfficeGeneral of the Auditor Annual Report OTHER

were also slightly less than the previouswere the alsoslightly lessthan Government AdverGovernment - - FINANCIAL STATEMENTS The Office of the Auditor General of Ontario The General Office ofthe Auditor 421

Chapter 7 Chapter 7 422 2013 of Ontario of the OfficeGeneral of the Auditor Annual Report The Office of the Auditor General of Ontario The General Office ofthe Auditor 423

Chapter 7 Chapter 7 424 2013 of Ontario of the OfficeGeneral of the Auditor Annual Report See accompanying notes tofinancial statements. deficit,Accumulated of end year deficit,Accumulated beginning ofyear Net deficiency operations returned Province tothe Less: Excess of appropriation over expenses Revenue Expenses 2013 31, March Ended Year the For Deficit Accumulated and of Operations Statement Ontario of General Auditor the of Office

Consolidated RevenueConsolidated Fund Statutory expenses: Transfer payment: CCAF and equipment Supplies Training development and andTravel communication Amortization of capital assets Office rent Professional and other services Employee benefits (Note Salaries wages and

Total expensesTotal (Notes

Statutory services Government Advertising Act Auditor General Act

5 -

FCVI Inc. )

[Note –

8 Voted appropriation

and 2 (B)]

)

[Note 2 (B)]

16,224,100 16,224,100 1,062,400 1,714,500 2,041,200 9,755,400 ( Note 11) 377,500 378,600 418,800 130,000 242,700 Budget 73,000 30,000 2013 — $

16,224,100 15,346,229 (2, (2,561,021) 1,642,632 2,103,948 9,286,283 1,000,115 ( 683,265 122,244 196,550 150,417 308,567 316,462 989,446 877,871 245,732 72,989 Actual 24,578 8,625 2013 $ ) )

16,224,100 15,240,592 (2,561,021) (2,547,096) 1,016,280 1,666,589 2,085,050 8,953,561 208,311 165,152 303,072 324,489 997,433 983,508 187,582 246,575 (13,925) 72,989 Actual 10,942 2012 $

See accompanying notes tofinancial statements. Cash Cash Increase (decrease) in cash Capital transactions operatingCash by provided transactions non Changes in O 2013 31, March Ended Year the For Cash of Flows Statement Ontario of General Auditor the of Office

perating , , Purchase tangible of capital assets Increase Increase (d inducement leasein receivable Increase (increase)Decrease duein from Consolidated Decrease harmonizedin sales recoverable taxes employeeAccrued benefits Amortization of Net operations deficiency

end of end of year beginning transactions year (decrease) ecrease - cash working capital

capital assets

)

in deferredin lease inducement accountsin payable accruedand liabilities

expense

Revenue Fund

The Office of the Auditor General of Ontario The General Office ofthe Auditor ( (318,598) 122,244 375,218 181,000 316,462 411,965 290,695 121,270 439,868 (32,223) 26,786 59,171 10,916 64,650 20 — 1 $ 3

)

(209,475) (416,564) (287,475) (322,225) (188,122) 290,695 500,170 207,089 308,799 494,564 184,000 324,489 (92,250) (13,925) 6,323 20 1 $ 2

425

Chapter 7 Chapter 7 426 2013 of Ontario of the OfficeGeneral of the Auditor Annual Report As required by the Fiscal Transparencyand Accountability Act, 2004 bothUnder Acts, the Auditor General reports directly to the Assembly.Legislative Act. the by required standards the meet printedadvertising, matter or reviewable messages proposed by government offices to determine whether they tangible tangible of amortization and capitalization the including accounting, cashmodified basis, an excess o Additionally, under the under Additionally, ions. organizat Act Advertising to th help required advice or assistance expert any of costs for statutory expenses voted appropriation isThe intended to cover the salary of Auditorthe General as well the as appropriations. future from of financialThe statements beenhave prepared in accordance with 2. S to re value and accountability promotes fairness of the financial statements of the Province numerousand ofagencies the Crown. In doing so, the Office that re sector public broader the in institutions of programs, government of Act ofGeneral Auditor the provisions the with accordance In Nature1. Operations of 2013 31, March Ended Year the For Statements Financial to Notes Ontario of General Auditor the of Office 17 of Act. theGeneral Auditor are are returned to the Province’s Consolidated Revenue Fund each year. As the voted appropriations Unspent of Province the Ontario. from voted appropriations annual through isfunded Office The VOTED(B) APPROPRIATIONS year that the events giving rise to the expense occur and resources consumed.are financialThese statements are accounted for on an basisaccrual whereby expenses are recognized in the fiscal ACCRUAL(A) BASIS The significant accounting policies are as follows: General Auditor the the port on theport of reasonableness a ummary of S lease inducement

and the the and ,

through the Office of the Auditor General of Ontario Office) (the Ontario of Auditor the General of Office the through

ignificant Accountingignificant Policies Fiscal Transparency and Accountability Act

Government Advertising Act,2004 Government Advertising and the recognition of employee benefit

r deficiency of revenue over expenses arises from the application of accrual of accrual the from application expenses arises of over revenue r deficiency - for Pre

- money in government operations and in broader public sector sector public broader in and operations government money in - Elec

tion Report prepared by the Ministry of Finance. of Ministry bythe prepared tion Report

, the Office the , Government e Office meet its responsibilities under the Government

and various other statutes and authorities, the the authorities, and statutes other various and

s expenses Canadian , or to conduct special assignments under Section under Section assignments , or toconduct special , in in ,

is is required to specified review types of capital assets capital an election year an ceive government grants, and of the the governmentceive of and grants, public s public earned to date but that will be thatfunded to but earned will date

, the deferral and amortization amortization and deferral the , ector ector appropriation appropriation conductsindependent audits the the a ccounting Office

is is is

also re also prepared standards quired

on a a on . • • Office’sThe financial assets and financial liabilities are accounted for as follows: on straight the isrecorded assets capital capitalTangible assets are recorded at historical cost less accumulated amortization. Amortizati TANGIBLE(C) CAPITA L ASSETS S 2. 2013 31, March Ended Year the For Statements Financial to Notes Ontario of General Auditor the of Office The The DEFERRED(E) LEASE INDUCEMENT arising from its financial instruments due to their nature. • • (F 10- rate, to interest any not Office exposed the is that opinion ismanagement’s It these finathese are reviewed annually to reflect new information as it becomes available. Measurement uncertainty exists in Estimates basedare on the best information available theat time of preparation of financial the statements and obligation. Items therequiring of use significant estimates include: useful life of capital assets and accrued employ period. the reporting during expenses and of revenue amounts reported the and statements, financial the of date management to make estimates and assumptions that the affect reported amou standards sector accounting public Canadian with accordance in statements financial of preparation The (D) FINANCIAL(D) INSTRUMENTS

) MEASUREMENT UNCERTAINTY year lease Due from Consolidated Re from Consolidated Due value. fair value approximates carrying so value in change of risk to insignificant an issubject Cash amount. be paid has not been made as it is not expected that there would be significanta fromdifference the recorded 201 31, March Accrued employee benefits obligation is recorded costat based on the entitlements earned by employees up to cost. at recorded are liabilities accrued payable and Accounts deferred lease inducement is being amortized as a ofreduction rent expense on a straight ummary of S ncial statements.ncial

period that commenced November 1, 2011. 3 . A fair value estimate based on based actu estimate . A value fair ignificant Accounting Policies (Continued) Leasehold improvements andFurniture fixtures Computer software Computer hardware Actual results could differ from from differ could results Actual

venue Fund is recorded at cost. at recorded is Fund venue

- line methodline over the estimated useful lives of the follows:assets as

The remaining term 5 years 3 years 3 years arial assumptions about when these benefits actuallythese will arial assumptions when about

these estimates.

The Office of the Auditor General of Ontario The General Office ofthe Auditor of the leaseof

currency, liquidityorcredit risk nts of assets and liabilities at the at liabilities and of assets nts

- line basis overthe on of tangible tangible of on

ee benefits requires requires

427

Chapter 7 Chapter 7 428 2013 of Ontario of the OfficeGeneral of the Auditor Annual Report Net Book Value, March 31, amortizationAccumulated Cost Net amortizationAccumulated Cost 3. 2013 31, March Ended Year the For Statements Financial to Notes Ontario of General Auditor the of Office Balance, endof year Balance, beginning ofyear Balance, endof year Balance, beginning ofyear Balance, endof year Balance, beginning ofyear Balance, Balance, beginning of Book Value, March 31,201 Tangible Tangible

Disposals Amortization Disposals Additions Disposals Amortization Disposals Additions end

of year

Capital Assets

year

2012

Computer Computer hardware hardware (153,590) (153,590) ( ( 174,113 174,113 171,482 403,848 678,777 165,520 687,370 283,522 403,848 159,472 397,966 687,370 243,826 597,134 277,560 401,217 $ $ ) )

Computer Computer software software

104,288 210,495 396,107 352,985 142,490 210,495 100,245 147,169 352,985 340,833 136,766 259,341 ( (36,919) (36,919) ( 55,442 55,442 98,564 49,072 $ $ ) )

and and and and Furniture Furniture (176,288) (176,288) 152,204 146,025 211,914 152,204 294,185 211,914 378,491 107,739 f f ( ( ixtures ixtures 71,017 71,017 26,552 59,710 34,307 38,286 5,128 9,711 $ $ ) )

i i mprovements mprovements Leasehold Leasehold (235,868) (235,868) 241,566 163,341 349,823 108,257 241,566 211,101 349,823 113,955 235,868 143,503 14,140 49,386 30,465 19,838 — — $ $

1,008,113 1, 1, 1,008,113 1,050,421 1,602,092 1,552,325 ( (366,797) (366,797) ( 536,440 536,440 316,462 318,598 593,979 324,489 416,564 596,115 788,135 384,250 602,092 Total Total 2013 2012 $ $ ) ) and accrued liabilities, as follows: liabilities, as accruedand payable in accounts included which oneare year, payable within amounts any less obligation, benefits employee Statement of Operations Accumulatedand Deficit. total The forliability these costs is reflected in the accrued during T ACCRUED(B) EMPLOYE Deficit. Accumulated recorded based o based recorded commercialstandard terms. Accru Accounts4. Payable and Accrued Liabilities 2013 31, March Ended Year the For Statements Financial to Notes Ontario of General Auditor the of Office $ . Office Office’s the of The required annual or obligations payment not assets are valuations funding of actuarial surpluses any viable, funds arefinancially the pension that orensuring unfunded liabilities arising from statutory sponsor of the PSPF, determines the Office’s paymentsannual to for the fund.responsible is As sponsor the sole the is which Ontario, of Province The provincial agencies. many ofand Province the for employees plan Office’sThe employees participate in the Public Service Pension Fund ( PENSION(A) BENEFITS In the Office’s financial statements, t employees that earnedhave theseProvince’sconsolidatedthe recognizedfinancialstatements. benefits is in and future forliability benefits earned by the Office’s employees is included thein estimated forliability provincialall Act General Although the Office’s employees notare members of the Ontario Public Service, provisionsunder in the Auditor 5. Obligation for Employee Future third with transactions business to normal largely relates payable Accounts 7 he costs of compensatedlegislated, severance absences 54, 442

the year amounted to $2 to amounted year the (20 , the Office’s employees entitledare to the same benefits as Ontario Public Service employees. The 1 2 severance,Accrued vacation and benefits and salaries Accrued Accounts payable

- n employmentn legislat arrangements and

$ 719,119 ),

E BENEFITSOBLIGATIO is

61 included employeein benefits expense in the Statement of Operations and

,000 (201 salaries and for salaries benefits als

hese benefits are accounted for as follows: benefitsfor hese accounted are as

2 Benefits N

– other

$ 274,000

credits

and unused vacation entitlements earned b earned entitlements vacation unused and

) and are included in employee benefits in the the in benefits employee in included are and )

ed entitlements .

and severance, vacation and other credits other vacation and severance, and

The Office of the Auditor General of Ontario The General Office ofthe Auditor PSPF) which is a defined benefit pensionbenefitdefined a PSPF) is which 1, 932 419,860 270,967 622,827

party vendors and is subject to to subject is and vendors party 201 ,000

$ 3

1,647,041 983,000 409,284 254,757

201 $ 2

y employees

are are 429

Chapter 7 Chapter 7 430 2013 of Ontario of the OfficeGeneral of the Auditor Annual Report ACCRUED(B) EMPLOYE 5. Obligation for Future Employee Benefits (Continued) 2013 31, March Ended Year the For Statements Financial to Notes Ontario of General Auditor the of Office $5 to approximately amounting the for premises expenses operating taxes and of share realty proportionate pay to committed its Office is The also thethefor as follows: of remainingleaseis termcommitment on 31, October 2021 expires which premises torent lease Office operating an has The 6. statements. financial these in not included is accordingly Services and of Government Ministry Ontario othernon of cost The (C) OTHER NON-PENSION POST

s Commitment obligation employeeAccrued benefits Less: severance,creditsliability for vacation andMCO Total

accounts payable and accrued liabilities Due one within year and included in - - post pension

E BENEFITSOBLIGATIO -EMPLOYMENT BENEFITS ,000 during 2013. during 06,000 retirement benefits is determined and funded on ongoingan basis by the

201 2017 201 201 201 201 8 6 5 3 4–1 – – – – –

1 18 1 1 1 9 7 6 4 5

N and beyond

1,897,800 514,200 508,800 501 48 495 8 2,404 3,336

, , ,900 3 4 932 00 00 $ 201

,000 ,000 ,000 $ 3

. The minimum The rental . 2,172,000 3,155,000 983,000 201 $ 2

earning Section 3(5) of this Act requires disclosure public paid Ontario of to benefits all and salary the Public7. Sector Salary Disclosure Act, 1996 2013 31, March Ended Year the For Statements Financial to Notes Ontario of General Auditor the of Office

an annual salary in excess of $100,000 of excess in salary annual an Wanchuk, Brian Tepelenas, Ellen TanmayGupta, Bove, Tino Boer, Johan AnnemarieWiebe, Young, Denise Yip, Gigi Tsikritsis, Emanuel Stavropoulos, Nick Rogers, Fraser MacNeil, Richard Herberg, Naomi Cumbo, Wendy Cho, Kim Chan, Sandy Carello, Teresa Allan, Pelow, William John McDowell, Mazzone, Vince SusanKlein, VannaGotsis, Fitzmaurice, Gerard Chiu, Rudolph Chagani, Gus Bordne, Walter LauraBell, Gary Peall, McCarter, Jim Name

Walter

Audit Supervisor Audit Supervisor Audit Supervisor Audit Supervisor Audit Supervisor Manager, Human Resources Audit Manager Audit Manager Audit Manager Audit Manager Audit Manager Audit Audit Manager Audit Manager Audit Manager Audit Manager Audit Manager Audit Manager Director Director Director Director Director Director Director Director Director Director Deputy Auditor General Auditor General Position

Manager

. This disclosure for thefor 201 This disclosure .

The Office of the Auditor General of Ontario The General Office ofthe Auditor 2

calendar year is as follows: as is year calendar 139,934 142,913 127,818 139,934 127,818 189,866 245,732 103,656 105,660 100,211 102,148 103,656 115,624 113,214 112,821 113,214 113,214 113,214 113,214 113,214 113,214 110,711 113,214 113,214 110,263 121,834 139,934 139,934 142,913 121,834 Salary $

- sector employees Benefits Taxable 3,865 187 187 174 187 174 239 143 143 142 143 143 152 152 151 152 152 152 152 152 152 149 152 152 152 164 187 187 187 164 $

431

Chapter 7 Chapter 7 432 2013 of Ontario of the OfficeGeneral of the Auditor Annual Report acquisition rather than being capitalized and amortized over their useful lives. under Economy, which purchases Board of Internal ofto the computers softwareand are expensed in the year of the followed for policies the with accounting consistent basis Office’sThe Statement of Expenses presented in Volume 1 of the Public Accounts of Ontario was prepared on a Reconciliation8. to AccountsVolumePublic 1Basis Presentationof 2013 31, March Ended Year the For Statements Financial to Notes Ontario of General Auditor the of Office

statements. accrued financial statements is financial obligation for employee futureobligation benefits and deferred lease inducement

A reconciliation of total expenses reported in V reported in expenses total of A reconciliation Accumulated Deficit expensesTotal per expensesTotal per Public Accounts Volume 1 amortization ofdeferred lease inducement change accruedin future employee benefit costs amortization ofcapital assets purchase of capital assets as follows: the Statementthe of Operations and

olume 1 to the total expenses reported in these preparation of the Estimates submitted for approval for approval submitted of Estimates the preparation

15, 15, ( 318,598 146,641 ( 181,000 316,462 199,588 346,229 32,223 20

o1 als Volume excludes the d in these financial financial these d recognize in 1 $ 3 ) )

15,240,093 15,240,592 (416,564) 106,000 324,489 (13,426) 20 499 1 $ 2

. Expenses9. Activityby 2013 31, March Ended Year the For Statements Financial to Notes Ontario of General Auditor the of Office been audited. 8. Note in discussed as Standards, Accounting accounting for presentation in Volume of1 the Public Acc bythe of Economy Board Internal were approved figures Budgeted Budgeted1. Figures expenserent on straighta - accommodation tofuture applied be pa As Deferred Inducement Lease 10. billings. on based actual to that activity areallocated services, professional costs and travel such most as activity, for onlyone incurred Expenses activity. aspecific with be identified otherwise not could that costs overhead and charged to each activity as recorded by staff in the Office’s time accounting system, including administrative time Expenses beenhave allocated to the Office’s three (2012 – Government Advertising Pre Election Report Financial Statement audits auditsand special for money Value Government Advertising Financial Statement audits auditsand special for money Value rt ofrt the lease arrangements for its office premises, Office the negotiated a lease inducement of $322,225 to

line bas line

is overis the 10

% %

costs. This deferred lease inducement is being amortized as a reductionof abeing amortizedlease inducementis as costs.This deferred

Salaries and Salaries and and Receivable 11,038,611 11,390,231

3, 7,417,946 3,565,142 7,699,796 Benefits Benefits 121,425 143,502 355,738 125,293

They are - 72.4 74.2 year period lease thatcommenced 1, 2011 November

presented for information purposes only have only and purposes for information presented Operating Operating ounts of Ontario. This differs from Public Sector Sector Public from ofThis Ontario. differs ounts Expenses Operating Operating Expenses 3,677,063 1,604,031 2,044,794 3,756,882 1,445,314 2,233,192 four) 28,238 31,595 46,781

Other Other Other Other

24.0 24.7

main basedactivities primarily on the hours 2012 2013

and were prepared on a modified cash basis of

The Office of the Auditor General of Ontario The General Office ofthe Auditor Expenses Expenses Statutory Statutory 278,935 227,785 445,099 131,756 265,415 20,912 30,238 23,271 24,657 1.8 2.9

15,240,592 15,346,229 4,825,709 9,916,553 5,199,411 9,972,375 176,291 322,039 174,443 100 100 Total T otal .0 .0

100.0 100.0

not not 31.7 65.1 33.9 65.0 1.1 2.1 1.1 % %

433

Chapter 7 Exhibit 1 434 * Financial Electricity Corporation Ontario Educational CommunicationsAuthority Ontario Development Corporation Ontario CleanWaterOntario Agency (December31)* the Ombudsman Office of andPrivacy the Information Office of the Environmental Commissioner Office of the Children’s LawyerOffice of the Assembly Office of Fund Heritage Corporation Ontario Northern Livestock Financial Protection Board, Fund for Liquor Control Board ofOntario Legal AidOntario Investor Securities EducationFund, Ontario FinancialGrain Protection Board, Funds for CommissionofOntario Financial Services Election Fees andExpenses, Chief Electoral Officer, Centennial Centre ofScienceandTechnology Cancer Care Ontario Algonquin Forestry Authority Agricorp by theAuditor whose 1. Agencies Dates in parentheses indicate fiscal periods endingona indicateDates fiscalperiods inparentheses date other than March 31. March date than other Commissioner Livestock Producers Commission Canola Soybeans, Corn, ofGrain Wheatand Producers Exhibit 1 General Election Finances Act Finances Election accounts areaudited Agencies oftheCrown Agencies Election Act Election

Ontario Mortgage andHousingCorporation Ontario MediaDevelopment Corporation Ontario Investor Immigrant Ontario Corporation Trust Heritage Ontario FoodOntario Terminal Board Financing Authority Ontario Energy Board Ontario Workplace Safety Board andInsurance St. Lawrence Parks Commission Niagara Parks Commission(October 31)* Motor Vehicle Accident ClaimsFund the by another whose 2. Agencies Public Guardian andTrustee Province for the of Provincial Judges Pension Fund, Provincial Judges Provincial Advocate for Children andYouth Arts CouncilforProvince the ofOntario Pension BenefitsGuarantee Fund, Financial Commission Securities Ontario Commission Racing Ontario (December 31)* PlaceCorporation Ontario Transportation Northland Commission Ontario (December 31)* Ontario Pension Board CommissionofOntario Services Auditor under auditor General accounts areaudited the direction

1. Metropolitan Toronto Convention Centre Metrolinx Collection CanadianArt McMichael System OperatorIndependent Electricity Hydro OneInc.(December31) Centre Human RightsLegal Support Higher EducationQualityCouncilofOntario MarketingHealthForceOntario andRecruitment Hamilton Niagara LocalHealth HaldimandBrant Network St.Integration Erie ClairLocalHealth Ontario eHealth Education QualityandAccountability Office ofOntario Corporation Deposit Insurance Network Integration Champlain LocalHealth WestCentral Network Integration LocalHealth Network Integration LocalHealth Central Network Integration EastCentral LocalHealth Board ofFuneral Services Institute Research ofOntario Agricultural and General with General, ited whose Corporations Dates in parentheses indicate fiscal periods endingona indicateDates fiscalperiods inparentheses date other than March 31. March date than other Corporation (December 31) Agency NetworkIntegration (December 31) Exhibit 2 other by an audit to related otherthan auditor full 1 1 reports, access documents accounts areaud Corporations Crown-controlled by theAuditor working 1 the as required Auditor papers -

North West NetworkNorth Integration LocalHealth Integration SimcoeMuskoka LocalHealth North Network Integration East Local Health North AssessmentCorporation Municipal Property Integration Mississauga Halton LocalHealth 2. Toronto IslandsResidential CommunityTrust Toronto Network Integration LocalHealth Central WestSouth Network Integration LocalHealth Network Integration East LocalHealth South Science North Royal Museum Ontario Ontario Public Health Owen SoundTransportation Company Limited Ottawa Convention Centre Corporation TrilliumOntario Foundation TourismOntario Marketing Partnership PowerOntario Inc.(December31) Generation PowerOntario (December 31) Authority Pension Board (December31)Ontario andGamingCorporation Lottery Ontario andLandsCorporation Infrastructure Ontario QualityCouncil Health Ontario French-languageOntario Educational Growth Corporation Capital Ontario Health Ontario. Health Protection andPromotion changed itsnameto Public Effective June14, 2011, Agency for Health Ontario the Network Network Corporation Corporation Communications Authority 2 1 1 1 435

Exhibit 2 Exhibit 2 436 fiscal year: The following 2012/13 changes were the madeduring Note: Waterloo Wellington Integration LocalHealth Walkerton CleanWater Centre Trillium ofLife Network Gift Toronto Waterfront Revitalization Corporation 2013 of Ontario of the OfficeGeneral of the Auditor Annual Report Waterfront Regeneration Trust Agency University ofOttawa Foundation Trent University Foundation McMaster University Foundation Foundation atQueen’s University atKingston ImprovingEcho: Women’s inOntario Health UniversityBrock Foundation Deletions: Network * This exhibit only includes the more financially significant organizations in thebroaderpublicsector.organizations in morefinanciallysignificant * Thisexhibitonlyincludes the Georgian Bay Hospital General Four Services CountiesHealth Hospital Espanola General Inc. Hospital andDistrict Englehart Centre Regional Health Dryden Corporation Deep River Hospital andDistrict Cornwall CommunityHospital Collingwood Hospital andMarine General Clinton PublicHospital Ontario ofEastern Children’s Hospital Alliance Health Chatham-Kent Casey HouseHospice Carleton Hospital Memorial PlaceandDistrict Campbellford Hospital Memorial Hospital Cambridge Memorial ContinuingCareBruyère Inc. Hospital General Brockville Bridgepoint Hospital System CommunityHealthcare Brant Bluewater Health Centre Blind River Health District Hospital Bingham Memorial Baycrest Care Centre for Geriatric Atikokan Hospital General Hospital Memorial andDistrict Arnprior Hospital Anson General Almonte Hospital General Hospital &General Marine Alexandra Ingersoll Hospital Alexandra working than the organizations Broader-public-sector PUBLIC HOSPITALS Exhibit 3 andotherrelated papers Auditor with General, (MINISTRY Broader PublicSector Broader inthe Organizations OF HEALTH ANDLONG-TERM full documents access whose by theAuditor accounts areaudited Grand RiverGrand Hospital Hospital Geraldton District Lady Minto atCochrane Hospital Centre Lady DunnHealth Kirkland Hospital andDistrict Kingston Hospital General Kemptville Hospital District Hospital Joseph Brant Humber River Regional Hospital Hôtel-Dieu Cornwall Hospital, Hôtel-Dieu Hospital Grace Children forHospital Sick CommunityHospital Hornepayne NotreHôpital (Hearst) DameHospital Montfort Hôpital Hospital Memorial Glengarry Hôpital andDistrict deHawkesbury Général Hôpital KidsRehabilitation Hospital Holland Bloorview SciencesNorth Health Care Centre Headwaters Health Hanover Hospital &District SciencesCorporation Hamilton Health Corporation Services Halton Healthcare Corporation Services HighlandsHealth Haliburton Haldimand War Hospital Memorial Hospital Guelph General Groves CommunityHospital Memorial Services Health Grey Bruce as General Hospital Inc. Hospital General required*

CARE) General to by an audit other auditor reports,

437

Exhibit 3 Exhibit 3 438 Scarborough Hospital Sault Area Hospital Salvation ArmyToronto Hospital Grace Runnymede Centre Healthcare Royal Centre Regional Victoria Health Rouge Valley System Health Hospital Ross Memorial Care FacilitiesRiverside Health Inc. Renfrew Hospital Victoria ofSt.HotelReligious Hospitallers Josephofthe ofSt.HôtelReligious Hospitallers Josephofthe Red Lake Hospital Margaret Memorial Cochenour Corporation Quinte Healthcare Queensway-Carleton Hospital Providence Healthcare Providence Care Centre (Kingston) Peterborough Centre Regional Health Falls andSmiths Hospital District Perth Pembroke Regional Inc. Hospital Ottawa Hospital Hospital Memorial Soldiers’ Orillia HillsHospital Northumberland YorkNorth Hospital General WellingtonNorth Care Corporation Health Bay Centre Regional Health North Norfolk Hospital General Hospital Memorial Nipigon District System Niagara Health Muskoka Algonquin Healthcare Mount SinaiHospital McCausland Hospital Mattawa Hospital General Markham Stouffville Hospital Manitouwadge Hospital General Centre Manitoulin Health Health Mackenzie SciencesCentre London Health Listowel Hospital Memorial Lennox andAddington Hospital CountyGeneral Hospital Memorial Leamington District Lakeridge Health WoodsLake ofthe Hospital District 2013 of Ontario of the OfficeGeneral of the Auditor Annual Report Dieu ofSt. Catharines Dieu ofKingston Corporation St. Mary’s General Hospital General St. Mary’s St. Hamilton Joseph’s Healthcare St. Centre (Toronto) Joseph’s Health St. Centre (Guelph) Joseph’s Health St. Care, London Joseph’s Health St. Elliot Joseph’sHospital, General Lake St. Joseph’s ContinuingCare Centre ofSudbury St. Joseph’s Care Group St. Hospital Francis Memorial Centre RegionalSouthlake Health Association Huron Hospital South Centre Grey Health Bruce South Falls Rock Smooth Hospital Sioux Lookout Meno-Ya-Win Centre Health Services desanté deChapleauHealth Services Sensenbrenner Hospital CommunityHospital Seaforth Woodstock Trust Hospital General Women’s College Hospital Hospital Wingham andDistrict Windsor Regional Hospital Hospital Memorial Winchester District Hospital General Wilson Memorial System William OslerHealth Centre West SoundHealth Parry West Park Centre Healthcare West Hospital NipissingGeneral West Hospital LincolnMemorial West Hospital HaldimandGeneral Weeneebayko Authority Area Health University Institute ofOttawa Heart University Network Health Trillium Partners Health Toronto East Hospital General Timmins Hospital andDistrict Hospital Tillsonburg Memorial District Thunder Bay SciencesCentre Regional Health Temiskaming Hospital Sunnybrook SciencesCentre Health Strathroy Hospital General Middlesex Stratford Hospital General Stevenson Hospital Memorial St. Hospital Thomas-ElginGeneral St. Hospital Michael’s Hospital Memorial St. Mary’s Conseil scolaire de district du Grand Nord duGrand Conseil scolaire dedistrict de desécolescatholiques Conseil scolaire dedistrict Franco-Nord catholique Conseil scolaire dedistrict du catholique Conseil scolaire dedistrict duCentre-Est catholique Conseil scolaire dedistrict desGrandes catholique Conseil scolaire dedistrict desAurores catholique Conseil scolaire dedistrict del’Est catholique Conseil scolaire dedistrict Centre-Sud catholique Conseil scolaire dedistrict Conseil desécolespubliques del’Est del’Ontario Board Ontario ofEastern School District Catholic Campbell Authority Children’s School Board School District Catholic Bruce-Grey School District HaldimandNorfolkBrant Catholic BoardBluewater School District Authority MacMillanSchool Bloorview Avon Board School MaitlandDistrict School Algonquin District andLakeshore Catholic Board School Algoma District Mississauga Halton CommunityCare Access Centre Hamilton Niagara Community HaldimandBrant St.Erie ClairCommunityCare Access Centre Champlain CommunityCare Access Centre WestCentral CommunityCare Access Centre EastCentral CommunityCare Access Centre CommunityCareCentral Access Centre Sciences Shores Centre for Health Mental Ontario Centre for Health Addiction andMental SCHOOL COMMUNITY SPECIALTY l’Ontario du Sud-Ouest Nouvel-Ontario de l’Ontario Rivières boréales ontarien Board Board Care Access Centre BOARDS PSYCHIATRIC ACCESSCARE (MINISTRY HOSPITALS (MINISTRY CENTRES OF EDUCATION) (MINISTRY OF OF HEALTH ANDLONG-TERM HEALTH ANDLONG-TERM Conseil scolaire de district du Nord-EstConseil scolaire dedistrict del’Ontario Waterloo Wellington CommunityCare Access Centre Toronto CommunityCare Central Access Centre WestSouth CommunityCare Access Centre East CommunityCare Access CentreSouth WestNorth CommunityCare Access Centre SimcoeMuskoka CommunityCare Access North East Community Care Access Centre North Waypoint Care Centre for Health Mental Royal Care Group Ottawa Health Moosonee District School Area Board School Moosonee District Area BoardMoose Factory School Island District Board School Catholic London District Board Limestone School District Board Lambton School Kent District Board School Lakehead District Authority School KidsAbility Board School District Kenora Catholic Board School Keewatin-Patricia District Board PineRidge School District Kawartha John McGivney Authority Children’s Centre School James Bay Board Lowlands School Secondary Board School District Catholic Huron-Superior Board School District Catholic Huron-Perth Board School Edward District Hastings andPrince Hamilton-Wentworth Board School District Hamilton-Wentworth Board School District Catholic Board School Halton District Board School District Halton Catholic Board School Greater CountyDistrict Essex Board School District Erie Grand Board Durham School District Board School District Durham Catholic Board School District Catholic Dufferin-Peel East North Board Ontario School District Board ofNiagara School District Conseil scolaire Viamonde Centre

CARE)

CARE) Exhibit 3 439

Exhibit 3 Exhibit 3 440 Georgian College of Applied Arts andTechnologyGeorgian College ofAppliedArts George Brown and College ofAppliedArts Fanshawe andTechnology College ofAppliedArts Durham andTechnology College ofAppliedArts Confederation and College ofApplied Arts Conestoga College Institute ofTechnology and appliquésCollège et LaCité d’arts detechnologie Collège appliqués et Boréal d’arts de technologie Centennial andTechnology College ofAppliedArts Canadore andTechnology College ofAppliedArts andTechnology CollegeCambrian ofAppliedArts Algonquin andTechnology College ofAppliedArts Board Renfrew School CountyDistrict Board School Renfrew District CountyCatholic BoardRainy River School District Board School Rainbow District Board School District Catholic Clarington Peterborough and Northumberland Victoria Penetanguishene Protestant Board Separate School Board Peel School District Ottawa-Carleton Board School District Ottawa Children’s Treatment Centre School Board School District Ottawa Catholic Board School District Catholic Northwest Board School District Catholic Northeastern School District Sound Catholic Nipissing-Parry Niagara Peninsula Children’s Centre School Board School District Niagara Catholic Board School District Near North 2013 of Ontario of the OfficeGeneral of the Auditor Annual Report COLLEGES Technology Technology Advanced Learning collégiale Authority Board Authority (MINISTRY OF TRAINING, COLLEGES AND UNIVERSITIES) Windsor-Essex Board School District Catholic Wellington Board School District Catholic Waterloo Board School Region District Waterloo Board School District Catholic Board Upper School District Grand Board Upper School CanadaDistrict Trillium Board School Lakelands District Toronto Board School District Toronto Board School District Catholic Board School District Thunder Bay Catholic Thames Valley Board School District Superior-Greenstone Board School District Board School District Catholic North Superior Board School District Catholic Sudbury Board School St. District ClairCatholic Board School District Simcoe Muskoka Catholic Board School Simcoe CountyDistrict St. Lawrence and College ofAppliedArts St. andTechnology ClairCollege ofAppliedArts Sir Sandford and FlemingCollege ofAppliedArts CollegeSheridan Institute ofTechnology and andTechnologySeneca College ofAppliedArts andTechnologySault College ofAppliedArts andTechnology College ofAppliedArts Northern Niagara andTechnology College ofAppliedArts Mohawk andTechnology College ofAppliedArts Loyalist andTechnology College ofAppliedArts Lambton andTechnology College ofAppliedArts Humber College Institute ofTechnology and York Board School Region District York Board School District Catholic Technology Technology Advanced Learning Advanced Learning ized and the amount expended. Theseare outlined amountexpended. ized andthe order, date ofeach the stating amountauthor the payments inexcess ofappropriations, authorize Treasury allordersofthe Boardreport madeto Act Under subsection12(2)(e) ofthe Children andYouth Services Cabinet Office General Attorney Affairs Agriculture, FoodandRural Aboriginal Affairs Ministry , the Auditor, the isrequired General to annually Exhibit 4 Treasury Board Orders Board Treasury Auditor General - Date ofOrder Mar 19, 2013 Jan 10, 2013 Sep 10, 2012 Aug 20,2012 Jun 14, 2012 Apr 19, 2012 Apr 16, 2013 Mar 21, 2013 Nov 19, 2012 Mar 19, 2013 Mar 19, 2013 Mar 18, 2013 Feb 26,2013 Feb 26,2013 Dec 6,2012 Nov 19, 2012 Apr 19, 2012 Apr 16, 2013 Apr 16, 2013 Mar 14, 2013 Jan 10, 2013 at the voteat the anditem level. expenditures such summarizes level, schedule this detail sub-vote by creating accountsatthe anditem related expenditures ordersinmore totrack these may following ministries in the Although table. ($) Authorized 104,963,600 62,503,400 36,863,600 27,578,000 32,700,000 42,780,900 10,000,000 21,000,000 13,091,600 1,533,100 6,800,000 2,422,500 6,585,400 4,000,000 8,250,000 1,000,000 1,110,000 7,901,000 540,000 500,000 650,000 500,000 280,000 108,100 35,000 Expended ($) 23,022,002 24,102,344 23,022,002 12,994,525 42,416,488 32,603,919 2,360,000 4,220,030 7,452,569 6,887,789 439,414 383,261 56,153 — — — — — — — — — — — — 441

Exhibit 4 Exhibit 4 442 2013 of Ontario of the OfficeGeneral of the Auditor Annual Report Finance Environment Energy Education Economic DevelopmentandInnovation Services Community SafetyandCorrectional Community andSocialServices Citizenship andImmigration Ministry Nov 19, 2012 Apr 16, 2013 Feb 1, 2013 Apr 16, 2013 Mar 21, 2013 Dec 21, 2012 Sep 13, 2012 Jun 14, 2012 May 17, 2012 Mar 28,2013 Dec 6,2012 Aug 15, 2012 Date ofOrder Jul 15, 2013 Apr 16, 2013 Mar 19, 2013 Mar 19, 2013 Mar 19, 2013 Jan 17, 2013 Dec 21, 2012 Dec 6,2012 Apr 19, 2012 Mar 19, 2013 Mar 19, 2013 Mar 19, 2013 Dec 13, 2012 Apr 16, 2013 Mar 19, 2013 Mar 19, 2013 Mar 19, 2013 Mar 19, 2013 Apr 16, 2013 Mar 19, 2013 Mar 19, 2013 Jan 21, 2013 Dec 13, 2012 1,426,541,700 ($) Authorized 359,000,000 117,246,200 317,668,500 715,594,200 143,168,700 307,382,500 138,557,500 19,032,900 57,066,700 40,547,000 90,699,100 22,000,000 35,000,000 40,000,000 20,000,000 17,606,600 15,000,000 19,700,000 16,332,900 12,911,800 37,000,000 50,297,400 12,637,200 77,787,300 5,000,000 5,000,000 8,693,500 2,965,200 8,000,000 2,700,000 2,790,000 2,750,000 1,000,000 1,780,800 2,478,800 7,300,000 7,606,600 3,987,800 540,000 429,500 470,000 547,000 Expended ($) 122,218,068 384,228,016 359,000,000 117,473,068 22,999,000 90,227,682 31,328,562 43,884,326 22,000,000 10,263,316 15,000,000 18,720,335 72,817,251 31,328,562 12,911,782 77,315,900 5,000,000 2,964,200 7,511,937 1,780,800 2,234,712 4,725,193 2,478,800 7,296,000 7,999,000 7,511,937 993,304 538,123 437,790 — — — — — — — — — — — — — — Natural ResourcesNatural andHousing Municipal Affairs Labour Infrastructure andLong-TermHealth Care Services Government Ministry Mar 19, 2013 Mar 19, 2013 Mar 19, 2013 Jan 21, 2013 Jan 14, 2013 Nov 19, 2012 May 17, 2012 May 17, 2012 Apr 9,2013 Feb 26,2013 Feb 21, 2013 Dec 6,2012 Sep 27, 2012 Sep 13, 2012 Sep 13, 2012 Jun 14, 2012 Apr 19, 2012 Date ofOrder Mar 19, 2013 Nov 19, 2012 Oct 18, 2012 Jul 18, 2012 Jul 18, 2012 Jun 27, 2012 Mar 19, 2013 Jan 10, 2013 Dec 13, 2012 Dec 6,2012 Nov 29,2012 Aug 15, 2012 Jun 7, 2012 May 17, 2012 Aug 15, 2012 Jun 14, 2012 Mar 19, 2013 Apr 16, 2013 1,879,910,800 1,414,443,100 ($) Authorized 152,445,400 106,160,400 397,450,100 86,159,400 20,000,000 40,000,000 12,828,000 12,000,000 72,000,000 15,000,000 16,000,000 31,500,000 11,600,000 58,527,700 49,767,700 11,597,100 1,999,000 7,332,500 5,500,000 8,083,800 3,925,000 6,750,000 1,500,000 1,304,400 1,000,000 1,000,000 1,000,000 1,000,000 2,120,500 9,881,300 8,415,500 1,476,600 1,475,000 7,724,900 300,000 999,000 410,300 95,000 1,680,109,347 1,319,035,406 Expended ($) 140,225,407 101,459,675 350,289,542 40,000,000 12,000,000 72,000,000 67,172,266 15,000,000 15,898,000 10,992,800 10,784,399 11,600,000 41,988,019 58,527,700 6,561,046 2,362,253 6,028,489 3,832,253 3,925,000 1,000,000 2,544,531 6,091,424 1,475,000 1,287,000 1,027,422 Exhibit 4 438,299 438,299 410,300 67,157 8 — — — — — — — — — 443

Exhibit 4 Exhibit 4 444 2013 of Ontario of the OfficeGeneral of the Auditor Annual Report Total Orders Board Treasury Transportation Training, CollegesandUniversities Tourism, CultureandSport Affairs Office ofFrancophone DevelopmentandMines Northern Ministry Mar 28,2013 Mar 19, 2013 Mar 19, 2013 Mar 19, 2013 Jan 10, 2013 Nov 19, 2012 Oct 18, 2012 Aug 20,2012 Jan 10, 2013 Apr 16, 2013 Mar 26,2013 Feb 5,2013 Dec 6,2012 Jul 18, 2012 Jun 27, 2012 Date ofOrder Mar 19, 2013 Mar 18, 2013 Jul 13, 2012 Jun 14, 2012 Apr 16, 2013 Mar 19, 2013 Mar 19, 2013 Mar 19, 2013 Feb 15, 2013 Jul 18, 2012 Jun 14, 2012 Jun 14, 2012 Jun 14, 2012 Jun 14, 2012 May 30,2012 Jul 15, 2013 5,088,162,000 ($) Authorized 102,373,200 381,470,400 103,873,400 247,600,000 42,260,000 12,350,000 12,400,000 44,474,100 15,273,600 13,410,400 36,612,100 14,370,500 42,517,400 3,950,000 2,500,000 3,800,000 5,000,000 9,000,000 4,800,000 5,800,000 2,000,000 2,700,000 1,036,600 1,430,000 1,750,000 6,010,000 4,701,700 1,100,000 7,500,000 7,000,000 500,000 250,000 730,000 612,000 50,000 3,269,310,653 Expended ($) 361,301,204 103,873,400 247,599,987 15,500,669 39,657,064 25,311,596 12,199,785 35,212,100 2,000,000 2,469,842 9,344,985 1,430,000 4,299,718 3,375,345 2,868,617 1,931,009 8,501,257 4,857,975 500,000 729,000 432,267 145,246 — — — — — — — — — — — — —