2012 Annual Report

Ofﬁce of the Auditor General of Ontario

To the Honourable Speaker of the Legislative Assembly

In my capacity as the Auditor General, I am pleased to submit to you the 2012 Annual Report of the Office of the Auditor General of Ontario to lay before the Assembly in accordance with the provisions of section 12 of the Auditor General Act.

Jim McCarter, FCA Auditor General

Fall 2012 Copies of this report are available for $9.00 from Publications Ontario: (416) 326-5300 or toll-free long distance 1-800-668-9938. An electronic version of this report is available on the Internet at www.auditor.on.ca

Ce document est également disponible en français.

Cover photograph credits: ISSN 1719-2609 (Print) top right: Dick Hemingway Photographs ISBN 978-1-4606-0348-2 (Print, 2012 ed.) middle left: © Tetra Images/INMAGINE.com middle centre: © iStockphoto.com/ssuaphoto ISSN 1911-7078 (Online) bottom left: Office of the Auditor General of Ontario/Mariana Green ISBN 978-1-4606-0349-9 (PDF, 2012 ed.) bottom centre: © iStockphoto.com/mevens, HooRoo Graphics Table of Contents

Chapter 1 Overview and Summaries of Value-for-money Audits and Reviews 5 Chapter 2 Public Accounts of the Province 27 Chapter 3 Reports on Value-for-money Audits and Reviews 45 Section 3.01 Cancer Screening Programs 46 Section 3.02 Criminal Prosecutions 65 Section 3.03 Diabetes Management Strategy 82 Section 3.04 Drive Clean Program 107 Section 3.05 Education of Aboriginal Students 129 Section 3.06 Independent Health Facilities 149 Section 3.07 Legislative Estimates Review Process 171 Section 3.08 Long-term-care Home Placement Process 185 Section 3.09 Metrolinx—Regional Transportation Planning 205 Section 3.10 Ontario Provincial Police 230 Section 3.11 Tax Collection 258 Section 3.12 University Undergraduate Teaching Quality 273 Section 3.13 Youth Justice Services Program 292 Chapter 4 Follow-up on 2010 Value-for-money Audits 318 Section 4.01 Casino Gaming Regulation 319 Section 4.02 Discharge of Hospital Patients 325 Section 4.03 Family Responsibility Office 333 Section 4.04 Home Care Services 343 Section 4.05 Hospital Emergency Departments 349 Section 4.06 Infrastructure Asset Management at Colleges 358 Section 4.07 Infrastructure Stimulus Spending 365 Section 4.08 Municipal Property Assessment Corporation 372 Section 4.09 Non-hazardous Waste Disposal and Diversion 382 Section 4.10 Organ and Tissue Donation and Transplantation 392 Section 4.11 School Safety 400 Chapter 5 Review of Government Advertising 407 Chapter 6 The Standing Committee on Public Accounts 420 Chapter 7 The Office of the Auditor General of Ontario 423 4 2012 Annual Report of the Office of the Auditor General of Ontario

Exhibit 1 Agencies of the Crown 449 Exhibit 2 Crown-controlled Corporations 450 Exhibit 3 Treasury Board Orders 452 number ofCrown employed attorneys hasmore charges laid by The policeforces across Ontario. Attorney prosecute General criminal ofthe Ministry Law Criminal Division Crownof the in the attorneys following subsections. regard inthe are inthis summarized observations tion to lookingfor efficienciesandcost savings. Our value-for-money audits,we atten- paidparticular easy matter. Consequently, year’s inmany ofthis the deficitwillbeno significant inroads in reducing renew itsfocus ondeficit reduction.” “To overcome government must challenge, the this Minister of Finance 2011as the saidinthe Budget, and, government recognizes this, long. TheOntario money, canonly practice becontinued for so this extra the by borrowing than itearns nificantly more revenues. spends sig- Andjust like ahouseholdthat ithasbeencollectingin spending farmore than like hasbeen many jurisdictions, Ontario, other recentimpacts global recession isthat ofthe lenges to get itsfiscalhouseinorder.the Oneof isnot- aloneinfacingsometough chal Ontario Criminal Prosecutions CHOICES FISCAL MAKING TOUGH Overview Chapter 1 Chapter Given the challenging fiscaltimes,making challenging Given the and Reviews of Value-for-money Audits andSummaries Overview ciency andeffectiveness ofitsCrown attorneys. relative ithasonthe workloads,information effi- betterlocal officeandshouldmakethe much useof how many shouldbeateach Crown there attorneys legal Divisionneedsto environment, determine the Factoring increasing complexity oftoday’s inthe year hasremainedof each relatively constant. charges disposed numberofcriminal the though doubledover last twothan the decades,even tion initiatives. diabetes, only 3%beingallocated with for preven- was earmarked for treating peoplewhoalready had cated. Aswell, we funding noted 97% ofthe that were and,insomecases,dupli- underused services diabetespeople with hasdefinitely improved, many availability ofcare forbeen mixed, whilethe inthat strategy. We to results have found date, that, the implemented afour-year, $741-million diabetes andLong-Term ofHealth Ministry In 2008,the Care eatingandexercise. includehealthier changes that 90%,islargely other lifestylethe preventable with able; however, Type 2diabetes, accountsfor which accounts for 10% ofdiabetes cases,isnot prevent- to 1.9 to 1.2 doubledfrom 546,000 in2000 has more than diabetesThe numberofpeoplewith inOntario Diabetes Management Strategy Diabetes

million by 2020.Type 1diabetes, which million in2010 andisprojected to grow 5

Chapter 1 Chapter 1 6 procurement andcost control. its processes relating investment, to infrastructure Metrolinx indicated mustdollars strengthen that significant projects costing hundreds ofmillions mated cost of$50billion.Ourreview ofseveral Hamilton area over 25 years next atanesti- the Toronto greater system inthe transportation and isresponsible for implementingthat anintegrated governmentMetrolinx Ontario is anagency ofthe home may have alongwait list. selected home,eventheir long-term-care though for bedhadappliedto along-term-care only one needs. However, 19% ofpeoplewaiting inhospital more complex health-care for patientswith other availability beds ofhospital home, andlimitsthe as homecare orplacementinalong-term-care community-based expensive alternatives than such ismore health, to isdetrimental their this though waiting for bed,even along-term-care necessary Many medically longer peopleare inhospital than facilityfee billingsare likelyhealth inappropriate. own estimateabout20%ofindependent was that andLong-Term ofHealth ate. TheMinistry Care’s orare inappropri nousefulinformation contribute imagingprocedures acrossscans andother Canada asmany as 30%ofCT that observed of Radiologists CanadianAssociation also noted in2009,the that We actualcosts ofproviding services. to the the facility fees inrelation beingpaidare appropriate andreview the unusual billingpatterns whether review facilitybillingsforsuggested Ministry the asx-rays, andCTscans.We ultrasounds ces, such - diagnostic servi These facilitiesprovide primarily 50% beingowned orcontrolled by physicians. independently owned about andoperated, with are facilitiesthat hasabout800health Ontario 2012 oftheOffice ofthe Annual Report Auditor General of Ontario Planning Transportation Metrolinx—Regional Long-Term-Care Home Placement Process Independent Health Facilities - practices andbetterpractices controlling overtime. costsoperating by improving deployment itsstaff acted upon.We OPPcouldreduce its noted the that we previously identifiedhadnot beenadequately exceeded inflationandcost-savings opportunities have beenhired, compensation increases have 27% over last five the becausemore officers years provided to municipalities)haveservices increased (excludingexpenditures recoveries for police since our last auditin2005.However,stable OPP Provincial PoliceOntario (OPP)have remained down to the inrecent years andcallsfor service motorserious vehicle accidentshasbeentrending trend. Aswell, numberof ofthis the has beenpart ada have 40%,andOntario declinedby more than last twoOver rates across Can- the decades,crime 2010/11 fiscal expenditures years, total program charge. From2005/06toof acriminal the the awaiting or have trial beenfound guiltyby acourt to young people aged 12 to 17 whoare either primarily ity andcustodialservices programs of Children andYouth provides commun- Services Ministry The Youth ofthe program Justice Services unpaid taxes it collects. strategies amountof atitsdisposal to maximizethe action onatimely basisandwas not usingallthe not appropriate was taking often Branch the that last fewcollections processes inthe years, we found havethere been someimprovements Branch’s to the itis responsible foring taxes collecting.While that about $1.4- $2.46billioninoutstand billionofthe 31, asofMarch that, 2012, itmay needto off write estimates taxes. TheBranch lecting most ofthese isresponsible forFinance’s col- CollectionsBranch voluntarily, of Ministry butsomeare not, andthe Most taxes owing province to the are remitted Ontario Provincial Police Provincial Ontario Youth Justice Services Program Youth Services Justice Tax Collection contributors to smog in Ontario. While this isdue Whilethis tocontributors smoginOntario. andare major nolongerthen amongthe domestic Vehicle emissionshave declinedsignificantly since hasbeeninplace since1999.testing program Cleanvehicle-emissions- Drive The mandatory typesofcancer. for certain high risk populationdeemedto beata segmentsofthe those meeting targets, itsparticipation-rate especially for number ofgoodprocesses butwas having difficulty Overall, we CCO hadimplemented found that a rates. highscreeningwas participation inachieving andhow screening programs effectiveformal CCO evidence to decidewhattypesofcancerwarranted clinical CCO usedestablished assessed whether Ouraudit provision screening services. the ofsuch agency responsible for co-ordinating andoverseeing provincial (CCO) isthe rates. CancerCare Ontario early stage canhave amajor impact onmortality typesofcanceratan detectsScreening that certain followingthe subsections. regard in are inthis summarized Our observations key focus ofsomeouraudits. services—a those provided processes followed andthe inoffering level ofservice delivery—the also madeservice being provided isequally We important. therefore funding possiblefor the best levelting the ofservice situation. However, publicis get the - that ensuring fiscal current to managingOntario’s sible iscritical ascost-effectivelyProviding aspos- publicservices facilitiesincreased by 50%. at these officers services youth 37%, numberofministry the secure facilitiesdecreasedin Ministry-operated by last fivethe numberof youths though evenyears, increased by only served 4%.Andinthe of youths number increased by 25%–30%,eventhe though Drive Clean Program Drive Clean Program Cancer Screening Programs THE PUBLIC TO ENHANCING SERVICES Overview and Summaries ofValue-for-moneyOverview Audits and Reviews charged to drivers. continuesto provide program fees valuefor the the incial andfederal government initiatives, to ensure larger citiesandinrelation to provOntario’s other impact onreducing smog,especially outsideof cost-effectiveness andits program’s tion onthe - informa policymakers are provided current with future to ensure that Ministry encouraged the impact onreducing overall vehicle emissions.We and cleanerfuelrequirements have hadalarger improvedwith vehicle-emission-control systems newer Cleanprogram, Drive to vehiclespartly the substantive hasbeenmadeto progress date. wegap. Information suggests little examined that academicachievement has beenmadeinclosingthe and hasnot any formallyassessedwhether progress framework implementation boards’ ofthe school hasnot adequately Ministry overseen the that local gap. However, inreducing the its progress we found andto track programs developmentthe ofsupport anddeveloped agoodframeworkpriority to guide this gap as aministry of Educationidentifiedclosing Ministry hasdoneso.Sixyears ago,the adults that percentage ofaboriginal andthe from highschool hasgraduated populationthat general Ontario is asignificant gap between the the percentage of academicachievement. There their affect ity—that minimal employment commun- prospects intheir poverty, and substandard housing,poornutrition as Many students facechallenges—such aboriginal periodically assess and report on the performance performance onthe assessandreport periodically universities hadinplace to three processes that public would Accordingly, agree. we reviewed the We parents andthe believestudents, their that future workforce. for the them dents andpreparing role istoimportant itsstu doagoodjobteaching Colleges andUniversities, auniversity’s most ofTraining, Ministry From perspective ofthe the Education of Aboriginal StudentsEducation of Aboriginal University Undergraduate Teaching Quality Undergraduate University - - 7

Chapter 1 Chapter 1 8 largest audited entityinOntario—were fairly the province—the dated financial statements of consoli - the that assurance the publicwith and the year, Office was able to provide the Legislature workthis on ourbehalf. Trustee, we engage private-sector auditors to do PublicGuardian and andthe Cancer Care Ontario financial statements of the aswith some cases,such Placeandmany In others. Commission, Ontario Securities Ontario LCBO, the agencies, asthe such provincestatements ofthe andmany ofitsCrown results are, infact,fairly stated. reported the publicare assured that lature andthe Legis- loopisnot accountability closeduntilthe the But financialaccountability to Ontarians. ment’s considered govern anessentialcomponent- ofthe Crown haslongbeen agencies andcorporations, province’s many province ofthe the orthose those of offinancial results, whether reporting responsibility ofourOffice.Annual also acrucial media,financialaudit publicandthe work is the Legislature, most attentiondraws from the the While ourvalue-for-money auditwork usually courses. selecttheir help them evaluations availableresults to ofthese students to make indicated they any our survey summarized universities responding toabout one-quarter ofthe student evaluations was beingdone,andonlythese universities we visited, littleaggregate analysis of took. However, they course ate each atleast atthe place to students to enableundergraduate evalu- processes in hadformal they visited, that reported three universities, includingthe tices, allOntario prac ofthese area. Aspart inthis practices of their 17 other Ontario’s surveyed universities onsome students. We undergraduate teaching of those also 2012 oftheOffice ofthe Annual Report Auditor General of Ontario Annual Reporting of Financial Results Annual Reporting ACCOUNTABILITY OVERSIGHT AND FINANCIAL I am pleased to report that for the 19th for the that I ampleasedto straight report Accordingly,the financial ourOfficeaudits - this year werethis alsofairly presented. many Crownments ofthe we agencies that audited financial state- we the that concludedthat report adian Institute Accountants. Icanalso ofChartered Can- by asestablished the accounting principles presented generally inaccordance with accepted tures ofhisorherministry. questioning policiesandexpendi aMinister on the for outsideofquestion period real opportunity only itwas the inthat mechanism accountability weeveryone spoke itwas still to agood saidthat government plans.However, expenditure almost proposed effective asameansofscrutinizing process was still not the that MPPs was that Legislature. in Ontario’s acombined total of80yearsserved aselected MPPs together parties—who had ofthe one from each We MPPs— retired three long-serving alsomet with have Standing Committee satonthe onEstimates. major ofthe politicalparties—who from each three MPPs— serving and interviewed ninecurrently United asthe systems, KingdomandAustralia, such similarparliamentary with jurisdictions of other ago, we decidedto area in2012. revisit this estimates review 15the process more than years we andthat last revieweda year onpublicservices government spendsover the Given that $120 billion spending on Internet advertising exceededspending onInternet advertising spending found inChapter 5.Thisyear, the time, first for the year. Theresults ofourwork area canbe inthis We reviewed items 565individualadvertising this in nature. it isnot toadvertising partisan ensure that ourOffice review most proposedthat government The Legislative Estimates Review Process Legislative Estimates Advertising Review and Approval Advertising WORK OTHER The consensus of both the current and former andformer current the The consensusofboth We estimates review the researched practices requires Act,2004, Advertising Government - thoughts andsomegoodnews. thoughts Aswell, dollars. tax afew Ioffer forward-looking on how well government care ofour istaking the objective information public with andthe members provides that elected independentwatchdog as the havethat work hadanimpact Office on the ofthe more significant eventsoverview ofsomethe peopleofOntario. Legislature andthe of the spective ofmythe Officeandwork we doonbehalf has happenedoverper past decadefrom the the 10 these describe years is“how timeflies.” Much most fittingadage the to Ithink In lookingback, Legislative Assembly. the with honour oftabling 10thThis isthe Ihave AnnualReport hadthe Standingbefore Committee the onPublicAccounts. subjectof17 hasbeenthe auditreport this hearings on completion. We 2012. inMarch didthis To date, auditin2011began this andwere asked to it table We AirAmbulanceandRelatedOrnge Services. This year we completed onespecialauditreport, inourAnnualReport. includingthem than rather oncompletion auditreports these table normally Public Accounts, orby Crown. aminister ofthe We Standingby Committee aresolution ofthe on ments asmay berequired Legislature, by the Under Auditor General the assign- Act , we perform Act. the costs)—yet isnot Internet advertising covered by on television (excluding advertising adproduction of Annual Reports of Annual Reports Looking Back Overa Decade Special Audits I thought that itwould that beusefulto provideI thought an Overview and Summaries ofValue-for-moneyOverview Audits and Reviews - improvements canbemade. being delivered andto pointoutwhere we believe are onhowuseful information well services these workThe focus ofthis hasalways beento provide provision care.and the ofhomecare andlong-term discharge ofpatientsfrom hospitals, departments, ship between emergency patientcare athospital - interrelation to hydro probing the the corporations boards, colleges atschool and of goodsandservices at universities; purchasing andfrom reviewing the students ofundergraduate teaching reviewing the homesto andlong-term-care diseases inhospitals detectionassessing the andprevention ofinfectious public sector. from Thishas includedeverything broader oforganizations inthe whole spectrum mandate by the conductingauditsacross virtually broader-public-sectorto these entities. government orabout$47 expenditures, public.Intotal, to about40%ofOntario the services provide organizations and hundreds that ofother agencies asChildren’s AidSocieties such service leges anduniversities, homes,social long-term-care boards, col- organizations school includehospitals, broader publicsector.organizations inthe These allow usto conductvalue-for-money auditsof decision to ourmandate expand inlate to 2004 the Legislature’s the past decadehasbeen Office in Undoubtedly, most significant event the the for months before aprovincialmonths election,and ourOffice six government produce afiscaloutlook report and Accountability, requires the Act,2004 that taxpayerfor TheFiscal with dollars. Transparency ads are not paid partisan toadvertising ensure that requires Office to review the proposed government , Act,2004 Advertising TheGovernment ities in2004. The Legislature alsogave ustwo more responsibil- BROADER PUBLIC SECTOR VALUE-FOR-MONEY AUDITING INTHE PRE-ELECTION FISCAL REVIEW PRE-ELECTION FISCAL ANDTHE GOVERNMENT ADVERTISING We have expanded beendiligent this inpursuing

billion, goes billion, goes 9

Chapter 1 Chapter 1 10 instance, has there beenasignificant increase in hasthere instance, work. cost ofdoingthis For isthe consideration workthe Office.Anequally doneby the important prove taxpayers are getting that valuefor money for However,2004. initselfdoesnot necessarily this responsibilities have significantly expanded since value for money. Asdiscussed earlier, ourOffice’s operates cost-effectively dueregard for andwith sonable question to ourown askis whether Office Given ourfocus onvalue-for-money auditing,area year-endCitizenship andImmigration’s grants. PowerBruce of andour2007 Ministry auditofthe cost to refurbishination ofthe nuclearreactors at our2007 our2009auditofeHealth, exam March, last audittabled Ambulance andRelated Services come to mindover Air Ornge years are the the Committee onPublicAccounts. Afewthat reports Standing public,andfrom the mediaandthe the Legislature, from dealofattention inthe great how we ourwork. approach good work andbeingobjective andindependentin because we have for areputation established doing frequent is inrecent years, andIlike this to think Our specialassignmentshave becomemore publicitserves. Legislature andthe the valueofourOffice to process enhancesthe the we to objective that scrutiny bring believe the that these two Officeinareas and continuing role ofthe forpursuits anauditor. However, the Ifully support forecasts offuture events are somewhat unusual andgivinganopinionon proposed advertisements Admittedly, peopleofOntario. ity to the reviewing initiatives have enhancedgovernment- accountabil nature, Ibelievemented legislationofthis these an opiniononitsreasonableness. is required to promptly forecast review andgive this 2012 oftheOffice ofthe Annual Report Auditor General of Ontario OUR VALUE-FOR-MONEY TRACK RECORD SPECIAL AUDITS Some of these specialauditshave aSome ofthese garnered have few jurisdictions other Although imple- - - of the other larger other Canadianprovinces.of the reasonable auditoffices compared ofthe to those levels ourstaffing does indicate are that certainly apples-to-applesstrictly comparison. However, it inoneoffice isnotto another a number ofstaff mandates andresponsibilities, comparing the offices audit offices.Asdifferent have different ColumbiaandAlberta Quebec,British the than count of102 people,we fewer operated with staff larger provinces. 31, AsofMarch 2012, astaff with Auditor ofother to officesinCanada’s General that want to know how sizeofourOfficecompares the nificantly more funding. requesting inourmandateexpansion without sig- work we the that required couldundertake by the earlier. We have atightfiscalshipso to run tried areas different discussed three inthe expanded whenourmandate 2004 was after increase shortly only noticeable the bumpwith beingasmall has increased only slightly over past decade, the AsFiguregoing to salaries. 1shows, ourstaffing amount payers bulkofthat $15.2 the million,with ofitsmandate? expansion ing dueto the Officeisspend amount ofmoney the orinthe staff 120 ofOntario the AuditorGeneral Officeof Source ofdata: 2002/03–2011/12General, attheOfficeofAuditor ofStaff Figure 1:Number 10 20 40 60 80

2002/03 0 0 When peopleaskaboutourbudget, often they 2011/12In the fiscal year, tax- ourOfficecost

2003/04

2004/05

2005/06

2006/07

2007/08

2008/09

2009/10

2010/11

2011/12 - and the land and air ambulance programs. landandairambulanceprograms. and the agencies, sibility Office,funding for social-service Family ofthe operation taxes, the property Respon anisms for process for physicians, assessing the of renewable energy initiatives,- new funding mech cometo mindrelatethat costs to ourauditsofthe examples more significant being made.Someofthe not beingproperly decisions the usedto support cases, however, butjust was there information the needed.Inother they information makers the weresystems that not designedto give decision- sometimes resulted from management information area inthis and timelyOurconcerns information. would have benefited from more relevant, reliable has highlighted where instances decision-makers good information.” itwould“gooddecisionsrequireforward, bethat believe publicsector must keep the inmindgoing I above that onetheme if Ihadto pick allothers faceustoday.” which those demanding than But more different, complex, andmore will bevery decade next challenges ofthe the Productivity “that 1970s Committee onGovernment by Ontario’s to “déjàvu,” where Iquoted astatement from the was your own” “needfor to better the oversight” have ranged from “spendtaxpayer money like it These atheme. contains AnnualReport year’s each to improving ofgovernment. operations Often, the are directedsee amyriad ofrecommendations that Ihavethat over tabled past decade,onewould the If onewerethe through to flip 10 Annual Reports also affect the day-to-day the also affect lives ofallOntarians. publicsector,the not only they cost more butthey wrong decisionsusually just cost more money; in sector, private may evenwrong ones.Inthe bethe best ones—and decisions madewillnot bethe the that risk the mation significantly increases LOOKING FORWARD LOOKING As I have said in the past, a lack ofgoodinforAs Ihave past, alack saidinthe Time work andtimeagain,Office the ofthe Overview and Summaries ofValue-for-moneyOverview Audits and Reviews - - generally accepted accountingprinciples. financial statements are not fairly presented under legislation,the accountingcomplies with the although Auditor positionofconcludingthat, inthe willnot infuture putthe years, this it. Ihopethat, butIdohave about accounting principles, concerns any from significantdeparture generally accepted to beaccounted for. Thishasnot so farresulted in to legislate arehas chosen transactions how certain credit for accomplishingment, andalldeserve this. years, govern have these the formed during another major atonetimeor All three politicalparties, solidated financial statements acleanbillofhealth. and Ihave beenableto con government’s give the media. and the public Legislature, the attentioncapture ofthe the itisourvalue-for-moneythough work tends to that its Crown agencies shouldnot beoverlooked, even the provinceannual financial statements of and andaddingcredibilityassurance by auditingthe chapter,earlier inthis role we the play inproviding least somereference to accounting.AsImentioned to 2003wouldback not becomplete at without look this andtherefore numbers, Auditors dealwith ernment business enterprises adopt International adopt International businessenterprises ernment gov that Institute Accountants (CICA) ofChartered year.for the respect to financialpositionand anentity’s results with whatisbeingreported and easily understand average the that reader may not beableto quickly statements are becoming solongandcomplicated pointwhere the the ment disclosures. Itisreaching accompanying andthe principles financial state- as atrend toward growing complexity inaccounting whatIsee I must confess towith somediscomfort STATEMENTS THE GOVERNMENT’S FINANCIAL PLAIN-LANGUAGE ACCOUNTING However, last few government inthe years, the For past 19 the years, my Peters predecessor Erik The recent recommendation of the Canadian Canadian The recent recommendation ofthe - - - 11

Chapter 1 Chapter 1 12 Report, IhaveReport, always includeda“goodnews” com- release ofourAnnual year onthe isheldeach that pressIn my conference atthe openingremarks “Sometimes, just plainEnglish isOK.” discussing how toitem: word report aparticular following Committeethe was comment whenthe Standing Committee onPublicAccounts whomade memberofOntario’s wordsthe ofalong-serving clearly.” Closerto home,Ihave always remembered words communicate complex that information past must giveand jargon ofthe way to everyday Exchange legalese Commission,whosaid:“The U.S. and ofthe Securities Chairman of aformer viewspronouncements, Ihopeitkeeps inmindthe tor development to the offuture IFRSaccounting isregarded- asarespected CICA contribu as the world globalization ofthe the economy. Andinsofar isprogress. this concluding that LCBO’s financial statements, Ihavethe ahard time actually is. key most lookfor— readers pieceofinformation LCBO’s bottom-line what the profit orloss—the the reader making itmore difficult to figurefor out As well, cannow there betwo incomestatements, statements increasing from 14 pages to 45pages. LCBO’stion ofIFRShasresulted financial inthe province. adop retail stores the Butthe throughout afinishedproduct andsellsitin business. Itbuys LCBO isafairlying perspective, the straightforward LCBO for many years. From abusinessandaccount them. having more difficultyinunderstanding reader offinancial statements prepared underIFRS lay ithasresulted inthe standards, Isuspectthat international to Canadianstandards after pattern CICA’s behindthe rationale decision the understand situationany better.opinion, madethis While Ican Financial Standards Reporting (IFRS)hasnot, inmy 2012 oftheOffice ofthe Annual Report Auditor General of Ontario SOME GOOD NEWS But Idohave given someempathy CICA for the typicalreader of From perspective ofthe the For we instance, have auditor ofthe beenthe - - be the following.be the overback past 10 the would they AnnualReports, cometo mindinlooking that positive things riding initiatives IfIwere over undertaken. three to pick havethat gonewell orwhere we saw somegood ponent where Ioutlinesomeexamples ofthings this area since2003. this improvementbeen afactor inthe we have seenin discussedabove twothe parties have undoubtedly impact.encouraging However, roles played the by Annualresults inevery Report hasinitselfhadan the auditandreport up two each years after taken recommendations. Auditor’s to address the outliningwhatactionsithas hearing entity atthe Clerk formally requests response from the awritten Committeemendations. Before any the hearing, broader publicsector tothe implement ourrecom- Crownon ministries, agencies andorganizations in Accounts hasconsistently brought pressure to bear deputyministers. responsibility ofthe our recommendations isconsidered animportant Office since2003havethat implementing indicated ofCabinet secretaries various the discussions with recommendations. Auditor’s My being taken onthe “centre” orCabinetthat actionis Office to ensure ismore pressure from the there First, Isensethat our recommendations. continuestoprogress bemadeinimplementing results ofourfollow-ups),(where we the report in Chapter 4ofourmore recent AnnualReports menting ourrecommendations, and,ascanbeseen In 2007, afavourable Ireported trend inimple has beenadefinite improvementthis area. in time,however,recommendations. Sincethat there littleactionhadbeentaken tooften, address our alltoo that, concern bly the in2003,Iexpressed the LegislativeIn my Assem Annual to first Report Implementation of Our Recommendations Implementation Admittedly, we follow- factthat doaformal the Standing CommitteeSecond, the onPublic Two factors have inthis. beeninstrumental - - - Committee Chairliked feet to put it,“holdtheir to long-serving former to, asthe hearing another oragency andhold ministry to re-call aparticular For example, over years, ithas not hesitated the recommendations implemented, asjust discussed. us clout,especially when itcomesto getting our From gives perspective, apragmatic itssupport the Office andwork we docannot be overestimated. for the Committee since2003, anditssupport the Accounts Committee. Ihave worked closely with diligent andactive hasavery Public Ontario time. first visiting for the wesector that have we audited, most ofwhich were broader public oforganizationsco-operation inthe regard, IdowantIn this excellent to recognize the have almost always received goodco-operation. auditors,while noonelikes we avisitfrom the andrecommendations. Aswell, observations respect toreasonable our andopen-mindedwith have over years dealtwith hasgenerally the been tainly beenexceptions, seniormanagement I the place.”have Andwhilethere right cer are inthe hearts saying goes,“their asthe always beenthat, and first foremost. However, my sensehasalmost decisionismadeputtingtaxpayers’ interestsevery always best decisionsorensures makes that the have beenrare. peopleheorsheserves for the best possibleservice one whodidn’t seemcommitted to the delivering ThetimeswhenIhaveothers. comeacross some- agencies, homesandmany long-term-care service universities, ashospitals, social- organizations such of seniormanagement inbroader-public-sector agency CEOsandCFOs, aswell asacross-section ministers, deputyministers, assistant andCrown years,Over Ihave dozensofdeputy the dealtwith Standing Committee on Public Accounts Committee Standing Senior Management Public-sector This doesn’t mean that seniormanagementThis doesn’t meanthat Overview and Summaries ofValue-for-moneyOverview Audits and Reviews - right. They deserve ourthanks. They deserve right. veres andultimately gets jobdone—anddone the timelines. However,- Officeperse ofthe staff the we frequently challenges andtight faceobstacles, well andontime.Thework we dois not easy, and money sideandingetting ourfinancialauditsdone to produce top-notch value-for- work, onthe both Office. the year,Year after hascontinued my staff ate to have acohesive such anddedicated team in Ihavethat Ihave tabled, - indeedbeenmost fortun overIn lookingback past 10 the AnnualReports not doneenoughto make improvements. oragency had fire” ifitbelievedtheministry the of which are presentedof which inChapter 3. onereviewmoney we auditsandthe conducted, all 12 ofthe value-for-following summaries are the public,goingbeyond numbers.” The “justthe the to ofservices delivery These auditsfocus onthe devoted to conductingvalue-for-money audits. resources are Office’s two-thirdsAbout ofthe manner, different.” mightbemuch things same Legislature couldoperatemittee inthe inthe Accounts Committee hasworked. com- Ifonly every best: “Ithasbeenapleasure to seehow Public the 2011 Legislature putit Speaker ofthe election,the October for the Houseadjourned just before the provided public.Last fall, level to the ofservice cost-effectivenessin operational andimprove the worked collegially to encourage improvements ated inarelatively mannerandhas non-partisan THANKS TO A HARD-WORKING GROUP A HARD-WORKING THANKS TO money and Reviews Audits Summaries of Value-for- For the most part, the Committee hasalsooper the For most part, the - 13

Chapter 1 Chapter 1 14 this regard, Ontario’s mortality rates are similarto mortality regard, Ontario’s this have over past two falleninOntario the decades.In typesofcancer three rates from these mortality earlycancer through detection andtreatment. The key from objectivenumberofdeaths reducing the formal decide whattypesofcancerwarranted CCO usedrecognized clinicalevidence tothat We cancers. breast, andcervical noted colorectal 2011/12 Inthe ing andtreatment services. fiscal Cancer Centres are responsible for cancerscreen- screening programs includethe screening programs Canadianaveragesthe typesof for these screening screening lion, $92 year, CCO hadtotal of$887 expenditures screening sible for implementing cancerprevention and province. the Itisalsorespon throughout services quality aimofdelivering andtimelythe cancer careing to providers, andother hospitals with fund- CCO directs health-care inOntario. services responsible for coordinating andoverseeing cancer (CCO) isaprovincialCancer Care agency Ontario 2012 oftheOffice ofthe Annual Report Auditor General of Ontario 3.01 SCREENING PROGRAMS CANCER • Our observations with respect to the three three respect to the with Our observations hasasits screening programs three ofthe Each CCO hasimplemented for screening programs CCO has13 Regional regional cancerprograms. th the Ministry of Health andLong-Term ofHealth Ministry the th In total, from 2008to 2010, only 27% of targeted populationremained unscreened. andCCO’s targets, andalmost halfthe istry’s Min- the ofboth cancer screening fell short ofCCO’s ownbut fell short targets. Colorectal targetscancer screening achieved Ministry inbreast cancerandcervical participation for cancer.increased risk Asoffiscal2009/10, especially for peopleconsidered to beat rates,to increase screening participation recent initiatives, have need recognized the ing commitmentin2010, andCCO, through a$45-millionfund- through Care (Ministry), Bo

million of which was spentoncancer million ofwhich

programs. programs. programs.

following:

cancer.

mil - - cer, we noted following: the screeningtypesofcan- processes for allthree of the • • • • • With respect to the wait times at various stages wait respect timesatvarious toWith the vel ofquality measures assurance or cervical cancerscreening, arecent CCO or cervical screening,or colorectal almost 30%ofcases raphy screening wait timesfor and the CCO isactivelyand the lookingatways to to appears haveprograms aplateau, reached screening Aswell, inthe group. participation screening tests age recommended for their eligible women completed cancer- allthree for the colorectal cancer screening program, cancerscreening colorectal program, for the had somequality processes assurance inplace were not requirements. subjectto the CCO and ings took placeoutsideCCO’s program However,screening program. 20%ofscreen- breast cancer for the program ity assurance considerably. CCO hasacomprehensive qual- varied screening programs ofthe for each lowera much younger rate than cancer,dying ofcervical were they screened at address grade abnormalities was aboutthree abnormalities grade Pap test result) for cervical high- abnormal wait time for acolposcopy (afollow-up on an review showed median preliminary the that positive Fecal OccultBlood Test family and17 histories with weeks for those were aslong72 weeks for peoplewith records found where instances wait times byestablished CCO. Our review ofhospital benchmarktime the colonoscopies within did not have recommended follow-up the 84 assessments for screening eligibilityaveraged for breast cancer, wait timefor genetic the targetsthat women considered athighrisk initsprogram CCO found that, months. ranged from just over two weeks to 10½ women average with for breast cancer risk cancer cervical but nonefor the The le Though olderw F F Mammog

days.

this. omen were of risk atgreater

results. program. women.

months. cators, at the level localoffices andindivid- cators, atthe ofboth to resolve indi- key charges, andother performance averageoutcomes ofprosecutions, the timeittakes workloads, onprosecutors’ reliable data with the that providefit from having systems it information 54 Crownoversight ofthe ateach by seniorstaff Crown Instead, attorneys. itrelies more oninformal relative workload, efficiencyandeffectiveness ofits Divisionmakes to littleuseofdata analyzethe the workloads onprosecutors’ impact ofthis because assigned to casesinvolving gunsandgangs and Also,more Crown havethem. attorneys been used to be,somore timeisneededto prosecute Freedoms, many casesare they more complex than changed—572,000 in1992, compared to 576,000 disposeofperyearattorneys hasnot substantially Crown charges that numberofcriminal the doubledsinceourlast auditin1993.than Yet overallthe Divisionhave costs staffing for more the province.offices across TheDivision’s operat Toronto, sixregional officesand 54 Crown attorney attorney offices. attorney We notedthe sameissuein dangerousother andhigh-risk in 950 Crown Division employs about1,500 includingabout staff, fiscal year, The was spenton staffing. 84% ofwhich totalleding expenses $256 prospect of public interest isareasonable to dosoandthere is to charge prosecuteonly ifitisinthe acriminal ACrown 60policeforces attorney inOntario. than year charges from each more 600,000 new criminal TheDivisionreceivesprovincial about courts. Crown charges onbehalfofthe beforecriminal prosecutes Attorney (Ministry) General ofthe istry Min- Law Division(Division)ofthe The Criminal 3.02 CRIMINAL PROSECUTIONS 3.02 CRIMINAL

2011. We continueto believeDivisionwould the bene- However, itisdifficult to the actual gauge ofRightsand Charter Partly asaresult ofthe The numberofCrown andthe attorneys The Divisionoperates from itsheadofficein

attorneys. conviction.

million in the 2011/12million inthe

offenders. Overview and Summaries ofValue-for-moneyOverview Audits and Reviews

1993. - already available, untilitcompletes develop the - is activitiesthat better oncourt useofinformation ual Crown TheDivisioncouldalsomake attorneys. the ment ofitsown information • • • • Our other major observations include majorOur other observations

following:

onic case-management system, Torontohe Division’s sixregions, the modelhasbeenestablished o staffing to attorney offices attorney went to trial were up to 20 someCrown ratesexample, atwhich the outcomesofcases.For results; andthe their numberofbailrelease applicationsand the charges are usedappropriately; criminal for diversion resolving programs minor court to prepare cases;whether andstaff attorneys is prosecuted; how longittakes Crown screened by Crown before attorneys acase onhowinformation charges efficiently are Itdoesnot gather performance. prosecutorial the insufficient resources have been dedicated to management oversight, factthat andthe significantly delayed becauseof weak project and becompleted by2010, March hasbeen projectedoriginally to cost $7.9 average ofother fewer charges perCrown the than attorney Region alsodisposedof anaverage of40% regions.age TheToronto other of$268for the cost percharge—$437, compared aver to the in fiscal2011/12,the highest butitdidsoat most charges intotalRegion disposedofthe one officeand 1,726 atanother, for regions—572 charges perCrown at attorney significantly amonglocalofficesand between Crown shouldbe.Workloads attorney varied mark for whatareasonable workload for each - there is nobench localoffice,and be ateach determine how many Crown should attorneys rates ofother the times higherthan The Divisiondoesno An electr Of t N

project.

regions. t formallyassessits

systems.

million

example. offices. - 15

Chapter 1 Chapter 1 16 most often preventable with lifestyle preventable most often with changes that 90%ofcases,is other betes, accountsfor the which has more than doubledfrom 546,000 in2000tohas more than 3% for prevention to treat peoplewhoalready haddiabetes, only with We funding was noted 97% earmarked ofthe that fundingshouldgotowardtheir preventive services. andmanyfunding are told underused, usmore of providers Strategymost set diabetes upwith service of diabetes care hasdefinitely improved. However, results haveshort-term beenmixed. Theavailability Diabetes StrategyOntario (Strategy). TheStrategy’s afour-year established Care (Ministry) $741 $4.9 system isexpected tohealth-care grow from cost generalto Ontario’s population,andthe of the rate system atabouttwicethe health-care use the grow to 1.9 1.2 eatingand include healthier its causeremains unknown. However, Type accounts for 10% ofcases,isnot preventable and untreated.managed Type orleft 1diabetes, which stroke, amputation andblindnessifpoorlyattack, Itcanleadto kidney failure, heart in Ontario. diseases most commonchronic insulin, isoneofthe complete inabilityto produce and/orproperly use or partial body’s Diabetes, results from the which 2012 oftheOffice ofthe Annual Report Auditor General of Ontario STRATEGY 3.03 DIABETES MANAGEMENT •

Some of our other observations were as observations Some ofourother andLong-Term ofHealth Ministry In 2008,the diabetesThe numberofpeoplewith inOntario million in2010, numberisexpected to andthat

h Ontario’s efforts to produce anelec efforts h Ontario’s billion in2010 to $7 to our audit fieldwork, the contract with the to ourauditfieldwork, with the contract date was extended many times.Subsequent 2009 was not proposed release met, andthe completion original deadlineofApril but the aprivate-sector Registry, vendorwith onthe hadbeenworkingbeen problematic. eHealth to monitor have patientdata Ministry and the tronic Diabetes to allow Registry physicians vendor was terminated inSeptember eHealt

million by diabetes 2020.People with

initiatives.

billion in

exercise.

2020.

2012.

follows: 2 dia

million - - • • • • e has been considerable duplicationand e hasbeenconsiderable vendor signeda$46 h andthe try has significantly increased the hassignificantly increased try needsto significantly enhanceits try since 2008/09. costs relatedlion ininternal Registry to the other sources, leading to service overlaps sources, leadingtoother service and own, fundingfrom oftheir with programs and physicians’clinicshave set upeducation professionals.other However, many hospitals a registered aregistered nurse, dietician and more Diabetes EducationTeams consisting of oneor with (DEPs),each Education Programs The provincial Strategy 152 runs Diabetes overlap ondiabetes. in educationprograms have about$24.4and eHealth incurred cancelled inSeptember 2012, Ministry the was Registry vendorbeen madeto andthe the certainty.price nopayment had Although timeinexchange for anddelivery progress province’s design, control over project’s the away traded contract this ofthe much that hasacknowledged eHealth was launched. Diabetes would the Registry bepaidonly after in2010contract vendor stipulated the that surgeries donein2009/10. demand andisactually lower 2,900 the than However, still doesnot current meet this the combat Type 2diabetes inobesepeople. from 245 in2007/08 to 2,500in2011/12—to surgeries— number ofin-province bariatric government line with claimswere mealexpense where not staff in ical decision-makers.” We also found instances “developing relevant polit- relationships with as “adviceonelectionstrategizing” and $105,000firm since2009/10 services for such Theorganizationviders. haspaidaconsulting pro- andanumberofdiabetes service Ontario year to manage andfund47 DEPsinnorthern organization itgives to which $20 offundsusedbymonitoring anot-for-profit under-utilization of90% Ther eHealt The Minis The Minis

policy.

the DEPs. the

million a

million

mil - technologies, federaltechnologies, requirements for cleanerfuel standards onemission-control manufacturing includingtighter program, the than factors other inception isactually Clean program’s Drive dueto 75% reduction invehicle ofthe emissionssince the emissionsestimates show more than Ministry that United inthe originate States.) that lutants Aswell, smogcomesfrom pol (HalfofOntario’s Ontario. major tothe domestic contributors smogin from 1998 to 2010, are nolonger andthey among during our during oldervehicles are factthat being and the identified for province’semissions exceed limitsare being the vehicles are getting tested vehicles whose andthat has effective procedures inplaceto ensure that tion ortransfer ownership. emissions test owner for the to registra- renew the allvehicles must passan but,otherwise, program, werethat builtbefore 1988 are exempt from the ownership isto Light-dutyvehicles betransferred. oneyearare sevenif olderthan years old,orthose repaired. vehicles owners ofsuch haverequires the that them whose emissioncontrols are malfunctioningandit identifies Theprogram vehicles smog inOntario. in1999program ofitsstrategy to aspart reduce Cleanvehicle Drive emissions mandatory duced the intro - Environment (Ministry) ofthe The Ministry 3.04 DRIVE CLEAN 3.04 • Some of the other significantissues other we notedSome ofthe On-road vehicle emissionsdeclinedsignificantly Overall, we found that the Drive Clean program Overall, Clean program Drive we the found that tests currently vehicles The program oncethey only the two-speed idle method, which uses two-speedonly which the idlemethod, adynamometer,with willbe tested using ure rate of11% to 31% in2010 when tested from 1988 to 1997, afail- experienced which built from 1998 on.Asaresult, vehicles built canonlytesting test which method, vehicles to beginusingan on-board diagnostic (OBD) Beginning Januar

audit were asfollows:

repair. y

PR 1, 2013, is program the OGRAM

retired. Overview and Summaries ofValue-for-moneyOverview Audits and Reviews - • • • try outsources six program services, services, outsources sixprogram try ehicle owners are not required to hough one of the program’s stated goalsis program’s hough oneofthe less stringent emissions limits than either the the either less stringent emissionslimitsthan lutants inhalfofthe lutants vehicles, andworse pol- for atleast oneofthe actually worse for in25%ofthe allpollutants repair were the emissionreadings after the in2011, repairs hadonlyvehicles partial that was repaired ofcases.For inonly one-third sions in2010—a faultycatalytic converter— commonly diagnosedcauseofexcessive emis- a conditionalpasswas only $255.Themost bill paidby owners received ofvehicles that not fully repaired in2011. Theaverage repair exceeds $450, about18,000 vehicles were incur any repair estimate repair costs ifthe dynamometer OBDtesting orthe 12 years was done Cleanprogram Drive for the support toa decade.Theonly measure survey public goalhasbeenachieved inmorenot than this targets orattempted to or measure whether hasnot performance established Ministry the to ahighlevel maintain ofpublicacceptance, provider to conduct. service to ofwhatitpreviously afraction required the numberofcovert auditsin2012reduced the compliance in2013, program Ministry the plannedintroduction ofanew to the prior non-compliance issuesannually. However, have identifiedabout3,000 auditefforts other covert auditsayear. and Inrecent years, these provider to conductupwardsservice of1,400 hasbeendiligent its inrequiring Ministry reduction inannualcosts. Until recently, the intocontracts a40% andexpects onecontract delivery sixseparate private-sector service the sector. private to the Itrecently consolidated ties for non-compliant activities, orfraudulent Cleanfacili- ofDrive monitoring including the The Minis Because v Alt

ago.

vehicles.

methods. 17

Chapter 1 Chapter 1 18 has beenmadesince2006inimproving achieve- framework to demonstrate what,ifany, progress more actively oversee implementation ofthis the However,academic progress. needsto Ministry the assesstheir andperiodically programs support work students, helpdevelop to identifyAboriginal educationinitiatives.ate Since2006,the Aboriginal federal government, andthe toministries coordin- boards,munities andorganizations,other school com- Aboriginal collaborates with (AEO), which by 2016. EducationOffice Aboriginal Itcreated the studentsbetween andnon-Aboriginal Aboriginal gap inacademicachievementfocus onclosingthe educationasapriority, a Aboriginal identified with take getting aneducation. how canaffect they seriously that ment, something Many livelittleprospect ofemploy inareas with poverty, substandard housingandpoor nutrition. academicachievement, including their affect the ment amongAboriginal for Aboriginal programs support hasprovided $170Ministry from high hadgraduated reserves Firstonly Nations 39%ofthe peoplelivingon for young adultsaged 20to age 24. group, Inthis 16%. Theacademicachievement gap isup to 50% pared to 78% general population—agap ofthe of com from highschool, hadgraduated in Ontario adults onlycensus indicates 62%ofAboriginal that 74,000 2006 from the Métisand2,000Inuit.Data including158,000in Ontario, First Nations people, peopleliving There are about242,000 Aboriginal 2012 oftheOffice ofthe Annual Report Auditor General of Ontario ABORIGINAL 3.05 EDUCATION OF • Among our significant observations are Among oursignificantobservations In2007,- designedapolicyframe Ministry the ofEducation(Ministry) Ministry In 2006,the studentsMany facechallenges that Aboriginal

following: ive years after the release of the release ofthe ive the years after progress against 10 anyprogress ofthe performance had notFramework assessedits Ministry , the First Nation, MétisandInuitEducation Policy F

S TUDENTS

students.

million infundingto

students.

school. Ontario Ontario - - • • • try hadnot abaseline established try hasaguideto helpschool try hough education on reserves is the finan isthe hough education onreserves and report on the measures. None of the three three measures. None ofthe onthe and report it hadnot required boards to school evaluate frameworkmeasures includedinthe because ondary School Literacy Test School ondary 2010/11 inthe 10 Grade passed the Sec- schools Ontario students attending provincialon-reserve that only halfof ability Officedata found analysis ofEducationQualityandAccount- transfer publicsystem. Our intowhen they the students may beseveral levels grade behind provincial andstudies suggest schools, these provide quality ofeducationfound the in haveschools generally not beenableto of limited per-student funding,on-reserve provincialinto the system. Partlybecause studentsmany eventually ofthese transition federal government,cial responsibility ofthe Ministry’s abilityto meet itsgoalofclosingthe Ministry’s question ofthe the population. Thisraises compared to 74% general 10 Grade ofthe were to from highschool, graduate ontrack students Only 45%ofthese as Aboriginal. themselvestion for students whoidentified forrecent data 10 Grade credit accumula- mostsuccess. We for the Ministry asked the indicator ofstudent isaprimary graduation students. Accumulation ofcredits toward ment between andnon-Aboriginal Aboriginal gap to inachieve- from measure which the academic anddetermine their port, inal students to better target fundingandsup andboards needto identify Aborig Ministry hadbeenidentified.The students inOntario estimated numberofAboriginal half ofthe However, timeofouraudit,fewer atthe than formally identifythemselves asAboriginal. boards develop policiesfor students to boards we visited haddone school achievement gap by The Minis The Minis Alt

year.

2016.

so.

progress. - - - the expenses. and staffing results faster inlower often be donemuch overhead tests allows tobecause new that certain technology vices. Thesecosts may have changed significantly overhead providing ser costs associated with the current hasnot the researched well, Ministry the As andultrasound. includingradiology services, diagnostic for certain continue to beunderserved municipalities patientsinabouthalfofOntario that indicates data areas, evenMinistry served though erally doesnot allow facilitiesto relocate to under gen still remain. For- concern Ministry example, the oversightthe offacilities.However, several areas of specificprofessional total ofthese the fees. not track does provided facilities.TheMinistry inthe service physicians astandard “professional fee” for each pays increased. Aswell, Ministry the of services 2006/07 to 2010/11, volume becausethe primarily payments increased by about4%peryear from paid$408millioninfacilityfees. Totalistry facility 2010/11and equipment. Inthe fiscal year,the Min- for overhead asrent,supplies staffing, costs such interpret, for example, x-rays. by physicians, many ofwhomare radiologists who are owned abouthalfofthem orcontrolled that facilities,estimates assessmentsofthese assurance for licensing,fundingandcoordinating quality Long-Term isresponsible which Care (Ministry), and ofHealth TheMinistry profit corporations. are 97% for-operated, andmore ofthem than andtest results physician. are sentto this services, requisition physician signedby to their receive the dialysis. Patients generallysurgery—or needa andplastic ascataract 25 provide surgery—such x-rays, andsleepstudies—and ultrasounds about as diagnostic services—such provide primarily facilities about800independenthealth In Ontario, 3.06 INDEPENDENT HEALTH Our other significant observations include significantobservations Our other hasimproved Ministry audit,the Since our2004 pays facilityowners a“facilityfee” The Ministry The facilitiesare independently owned and following:

FACILITIES Overview and Summaries ofValue-for-moneyOverview Audits and Reviews - - • • • • • ch 2012,ch X-Ray Ministry’s the hough the Ministry estimates that about estimates that Ministry hough the sameamountfor each h facilityispaidthe nlike facilitiesare assessedby hospitals, the he CanadianAssociationofRadiolo likely inappropriate. about20%offacility-feewas that tests are own estimate TheMinistry’s inappropriate. orare nousefulinformation contribute imagingprocedures acrossand other Canada gists noted asmany as30%ofCTscans that their physician’s form. referral their facilityon must go topatients assumethey the facilities.Further,or related persons’ many own patientstoof physicians their referring physicians, ithasnot patterns analyzed the 50% offacilitiesare owned orcontrolled by Ministry.cost to the additional areas without vices inunderserved in high-densitylocationsmightencourage ser smallerpopulationsandlowertions with fees Payingperformed. slightly higherfees inloca- numberofservices the proportionately with donot increase salaries and reception staff from economiesofscale,sincecosts like rent larger benefit facilitiesinurbanareas often Consequently, itperforms. number ofservices provided, regardlesstype ofservice the radiation appropriately shieldedto prevent excessive equipment, includingx-ray equipment, was required radiation-producing to ensure that almost 60%offacilitiesasfrequently as Unit hadnot inspectedInspection/Services working facilities. atthose didnot worksors review the ofallphysicians Even College for asses- assessedfacilities,the previous five the years. been assessedwithin 2012,March about12% offacilitieshadnot physicians. facilities’ However,by the asof diagnostic images read are beingcorrectly things, amongother to helpensure that, College ofPhysicians andSurgeons ofOntario Alt Eac U In 2009,t As ofMar

exposure. exposure. - - 19

Chapter 1 Chapter 1 20 but it must seek the approval Legislature but itmust seekthe ofthe delivery, andmanages service spending priorities Thegovernment setslion ayear onpublicservices. pletes itsreview, Legislature. to the back itreports they constitute the legal spending authority for legal spendingauthority constitutethey the Legislature approves the Legislature. After them, estimates Committeebefore to their the explain appear ministries ofthe ministers andseniorstaff reviewhave process, the elected During members. that politicalparties three ment (MPPs)from the provincialrently ofthe Parlia comprises members government year. officeseach TheCommittee cur 12,at least six,butnot or more than ministries Estimates (Committee) estimates reviews of the request formal Legislature for to the ernment’s budget’s gov release, represents the the after government 12 nolaterthe than sessionaldays by must betabled mates. Thisdocument,which inadocumenttitledEsti- summarized are then area andspendingactivity,program andthese expected budget,the rev alsoreflectsthe which Management basisfor Board ofCabinet, the form reviewed andapproved Treasury by the Board and plans,once manage. they Ministry public services proposed spendingplansfor the their put forward then Theministries and fiscaloutlook. its priorities reflect broad spendingguidelines that with istries earlier. months eral Thegovernment provides min- However,spring. sev process starts budgetary the presented Legislative to Assembly the inearly the answer and spending proposed the enues to support year foreach itsspending government spendsabout$120The Ontario 2012 oftheOffice ofthe Annual Report Auditor General of Ontario OCESS REVIEW 3.07 LEGISLATIVE ESTIMATES MPPs may Estimates alsodebate inthe the Standing Committee the on In Ontario, spendingplansare sub-dividedby Ministry budget Ontario hasbeen In recent years, the

government.

approval.

q uestions. When the Committee com When the uestions.

plans.

expenditures.

- bil ------question ministers outside of the Legislature’s question Legislature’s ministers outsideofthe to provide only directly realthey opportunity the because Committee are worthwhile ofthe hearings plans. However, the madeitclearthat members ofgovernmenting meaningfulscrutiny spending effective process isstill inprovid not very the - that madein1995—that commentsmembers the is, MPPswe of almost allofthe interviewed echoed legislative estimates review process. Theconsensus tomembers, get perspective historical onthe their as elected 80years ofexperience had more than former, MPPs,who,intotal,three long-serving MPPsby meeting current with with our interviews Estimates Committee.on the We supplemented whohadserved viewed party MPPsfrom each three process in1995. Aswe didin1995, we again inter have available: limitedive timecommittee useofthe members and New weakest Zealandashaving the budgetary Unitedments ofCanada,Australia, the Kingdom oversight,limited budgetary parlia andranked- the Westminster-style legislatures ingeneral have very Fund study noted Monetary 2005 International that MPPs.Indeed, a by expressed Ontario concerns review same processes. Many the are wrestling with effectivestyle estimates hadtruly parliaments that oversight processes ofany Western question • • • We last reviewed legislative estimates the review We ideasfor- makingmore effect these offered didnotOur research identifyany Westminster- briefing ministry equesting that officials inattendance. Committee, only perhaps senior ministry with for review amore in-depth programs by the planned for proposed expenditures. assess whatresults are being achieved and to provide abetter basisfor legislators to reporting performance past andcurrent with onspendingplans combineinformation books ministry.ings for each hear formal for inadvanceofthe members Consider selectingaf Consider r Consider ashor

period. t in camera estimates briefing estimatest incamera briefing ew specific ministry ew specific ministry

countries. - - alternatives bedeveloped to care. CCACs long-term that aging population,itiscritical Ontario’s in, andLTC homes may reject applications. LTCselect the are willingto homesthey beplaced beds are available. Aswell, peopleare allowed to isresponsible for how Ministry manythe LTC home control ofCCACs.ment are outofthe For instance, demand forcare. long-term 75, to turn likely start boomers creating additional increase beginningin2021 baby further whenthe to grow by almost 30%from 2012 to 2021 andto aged 75population ofthose andupisexpected Act took Ontario’s inthe effect, eligibility criteria decreased somewhat sinceJuly 2010, whentighter to 98days in2011/12. wait timeshave Although fiscal 2004/05 year 36days inthe tripled—from of timepeoplewait for anLTC homebedhasalmost undoubtedly onereason whymedian amount the and over is hasincreased20%,which by more than placement provisions. LTC CCACs Act’s complying that the with home isresponsible LHINsare for accountable, ensur the andLong-Termof Health to which Care (Ministry), Networks Integration Health (LHINs).TheMinistry province’s 14CCAC to oneofthe reports Local homesare 97% care bedsinthese occupied. Each LTC 640 Ontario’s homes.The76,000 long-term- 25,000 people,85%ofwhomwere 75 orolder, in able. Infiscal2011/12, CCACsthan placedmore placementwhenabedbecomesavailand arrange - eligiblepeopleonwait lists admission, prioritize (CCACs) to determine eligibilityfor LTC home province’s 14the CommunityCare Access Centres ities ofdaily living. in asecure setting activ orfrequent with assistance to have 24-hour care available, nursing supervision andaccommodationto peoplewhoneed services homes(LTCLong-term-care homes)provide care, PLACEMENT 3.08 LONG-TERM-CARE HOME The Ministry has recognized that, given hasrecognized that, The Ministry waitMany timesfor affect place- factors that aged 75 numberofOntarians Since 2005,the The Long-Term Care HomesAct(Act) authorizes

PR OCESS Overview and Summaries ofValue-for-moneyOverview Audits and Reviews - - ment. Our observations included the following: includedthe ment. Ourobservations process well, neededimprove butallhadareas that - LTC areas oftheir aging various homeplacement consistently. casesare prioritized crisis ensure that care. However,long-term more needsto bedoneto eligibility, alternatives to includingconsidering use astandardized process to determine client • • • • • • Overall, the three CCACs three weOverall, the visited were man- 9% of people waiting in hospital for9% ofpeoplewaiting inhospital anLTC ACs we visited didnot periodically 2012ch LTC homewait-list indicated data

needed by patients. other arecare alternatives andtakes upbedsthat community-based ismore costly than health, to isdetrimental aperson’s ally necessary ismedic longerremaining inhospital than a longwaiting list. Ithasbeenshown that selected homemayeven the have though home bedhadappliedto only oneLTC home, were offered the first were first available beds. the offered clients highest priority review the whether apply for them. rooms, butonly 40% ofpeople beds insuch because homescanhave upto 60%oftheir rooms are generally placedmore quickly 1,100 317within days, whereas atanother, ittook At oneCCAC, 90%ofclientswere placed into LTC others. homesmore quickly than the more popularhomes.it difficult to access clients get priority, clientsmay find non-crisis home.Becausecrisis home resided inanother 40% ofpeopleonwait lists for aparticular 2012, wait list. InMarch preferred home’s alternative LTC homebutstayed ontheir acceptedchoice ofhomes, an others first receiving LTC homeaccommodation. 2011/12 fiscal year, 15% ofclientsdiedbefore eligible clientshadwaited the years. During had waited 10 to 14 andmost other months; days point; upto moderate-needs that clients clientshadwaited amedianof94 crisis that Applicants insomear 1 The CC Clients ablet While 36%ofclientsw Mar

days. o pay for orsemi-private private eas of the provinceeas ofthe get ere placed in their ere placedintheir - 21

Chapter 1 Chapter 1 22 linx ifitisto best including practices, meet these must beaddressed by Metronumber ofissuesthat - earlyprojects inthe RTP stages ofthe identifieda are managed andprojects are deliveredrisks cost- that project hasasoundcost-benefit analysis; and network; that each less andefficienttransportation to individualprojects contribute aseam- sure that stakeholders andmake consultwith things, other problem have agencies usedsimilarstand-alone to have that facedthis jurisdictions GTHA. Other the networkmenting anintegrated transportation in as Metrolinx isareasonable strategy toward imple- population. Thecreation ofaCrown agency such ingly unableto needsofagrowing meet the highways, systems andpublictransit are increas- system overmenting aGTHA transportation 25 for imple policiesandprograms priorities, out the its Regional Transportation Plan(RTP), sets which the effectively andon coordinate regional transportation years atanestimated cost of$50 Greater Toronto andHamilton Area networkof anintegrated transportation inthe coordination, planning,financing anddevelopment was to established provide inthe leadership government,Metrolinx, Ontario anagency ofthe 2012 oftheOffice ofthe Annual Report Auditor General of Ontario TRANSPORTATION 3.09 METROLINX—REGIONAL • However, ourreview ofseveral significant To deliver onitsgoals, Metrolinx must, among The GTHAfrom congested suffers roads and In November 2008,Metrolinx formallyadopted

following: e believeMetrolinx’s initialassump that assessment that suggested mayassessment that ARLridership feasible. Metrolinx itselfconducted amarket objective, may isindeedthe notif that be tion costs, it onabreak-even operating basis, - costsoperating andany construc ofitscapital ARLmust recover the on whether itsannual afinaldecisionhasnot beenmade Although may beoptimistic. tion andPearson Airport Link(ARL)between AirRail the Union Sta- on tions aboutprojected annualridership W

time.

PLANNING

billion.

planning.

(GTHA). - - key to implementing aregion-wide integrated Presto Metrolinx fare-card system hasdeemed that transit faretransit system. • • • • As well, we the found anumberofissueswith h Ottawa’s body, transit Toronto the o major projects related revital to the esto has been in service for abouttwoesto years, hasbeen inservice mated fare level. not meet itsinitialassumptions esti- given the estimates. The cost of restoring the train train estimates. Thecost ofrestoring the significant cost increases their initial over ization ofUnion Station have experienced not themselves integrated, the Prestonot themselves integrated, the card has $38 twicethe couldbemore than Corridor Rail Union inthe Station switches replacing the Metrolinx’s initialestimate.than Thecost of $270shed couldreach because of the card’s because ofthe oldfare systems infavourtheir ofPresto agencies6%. Thesetransit cannot eliminate but itsoverall systems usage isonly onthose agenciesof eightmunicipaltransit usePresto, of March GTHA systems transit was only about18% as but itsoverall usage amongparticipating of more cost-effective technology linx ofpotential new developers and,possibly, believe would tendering have Metro informed - We contract. vendor’s existing under the develop open-endedchange orders itthrough development ofPNG, Metrolinx decidedto world. Instead ofcompetitively the tendering more expensive fare-cardthe systems inthe $700 be more than Presto original system andPNGthe may well $498 being developed atananticipated cost of Ottawa, Presto Next (PNG) Generation is to requirements meet the ofToronto and ally approved adoption ofPresto. the However, Transit Commissionhasonly now condition- not yet facilitated fare Because far The tw Pr Along wit

million on the original purchase original million onthe

million. Thetotal cost ofdeveloping

31, 2012. 905region, seven Inthe es for GTHA systems transit are

Specifically:

million, placing it among million, placingitamong

limitations.

integration.

million—25% more

solutions.

order. - a cost-recovery basis,aswell asto 19 First Nations to 322municipalitieson municipal policingservices making up87% amount.TheOPPprovides ofthat 2011/12lion inthe fiscal year, costs staffing with issues we onin2005continueto reported exist. motor andserious decliningrates ofcrime the increases,similar expenditure notwithstanding many large other police forces inCanadahave had have received highercompensation. We found that havebecause more officers been hired and staff last fivethe increase hasoccurred Most of years. from municipalitieshave increased by 27% over the OPPhasresponded to orinitiatedthe hasremained last fivethe numberofcalls for service the years, decreasing. Over andinjuries fatalities both with vehicle accidentshave alsobeen trending down, OPPhaveby the decreased 10%, motor andserious rates reported crime OPPin2004/05, audit ofthe trend. Sinceourlast ofthis hasbeenpart Ontario Canada have 40%,and declinedby more than Our major observations includethe Our major observations vehicle relatively communities, andwas $362 reimbursed ments and87 satellite policestations. and 2,300 civilian employees. It operates 78 detach province.to allcommunitiesinthe and provides services emergency support andother request tounder contract, municipalitiesthat them policingservices, Italsooffers and organized crime. conducts investigations into cases complex criminal own policeforces, patrols provincial highways, and donot have inareas that their line policeservices Provincial PoliceThe Ontario (OPP)provides front- 3.10 ONTARIO PROVINCIAL We many ofthe auditthat found inourcurrent However, net OPPexpenditures ofrecoveries last twoOver rates across the decades,crime totalled expenditures OPP operating $979 The OPPemploys about6,300policeofficers

accidents.

stable.

following: POLICE

million. Overview and Summaries ofValue-for-moneyOverview Audits and Reviews

mil - - • • • • • •

hough the OPPhadloweredhough the itsovertime he OPPtold usitwas working facesignificantly officers differ e found that verall cost of OPP services for municipal verall cost ofOPPservices about these cost increases. about these received, concern expressed they they services the OPP were satisfiedwith they very us that rate.inflation Whilemunicipalofficials told annual timesthe to three without—up those and 19% contracts with of 29%for those for ities from 2007 to 2011 increased anaverage sion benefits and other allowances.sion benefitsand other entitled, includingsignificantly better pen are not publicservice Ontario ofthe members benefits ians receiveother to which certain andcivil sated inCanada.Officers officers front-line officers. used to deploy 2,800 only about45%ofthe is isalmost 30years oldandthat model that mayments. Thereason for beastaffing this 137%- detach in officers other more callsthan handling 54% someofficers to assigned, with ent workloads are dependingonwhere they 50 fewer new modelcalculatedwhereas itneeded the force needed500more front-line officers, the 2012, model calculated existing OPP’s that the new model,butitdoesnot. InMarch uses this computer model.TheOPPhassinceclaimedit RCMP the onanewwith officer-deployment $53 overtime costs have increased by 60%to costs for by 2004/05 10% to $33 range of$5 in the couldresult insavings addressing this hours; in overstaffing slow during early-morning to work choose cers Thisresults 12-hour shifts. andalmost alloffi- atdetachments, scheduling In 2005,t W The o OPP officer OPP manag Alt

million since

officers. s are among the highest compens are amongthe ement hadlittlecontrol over shift

then. million to $10

million.

million, - - - - - 23

Chapter 1 Chapter 1 24 behalf of other provincial behalf ofother ministries. would revenueBranch alsocollectnon-tax on of Finance. Collections Under proposal, the this Ministry revenue the non-tax within ernment collectionofallgovproposed the centralizing accounts of the province.accounts ofthe havefor years andthat inthe beenexpensed havemostly from olderaccountsthat accumulated wouldresponsible for be collecting.Thewrite-off $2.46 of the expected itmay up to $1.4 needto off write ted voluntarily. AsofMarch remaining amountwas generallywhile the remit - revenuein taxation generated province, by the about $6 collecting these 2014 itwillhave wound down most ofitswork on by that expects TheMinistry transfer CRA. to the and Retail SalesTax amountsowing to their prior Taxis still responsible for collectingCorporations 2012. CRAinMarch to the However, Ministry the were members transferred 400staff Branch’s the Sales Tax inJuly 2010. Asaresult, about75% of Sales Tax, provincial replaced the Retail which Similarly, Harmonized CRAadministers the the Tax 2009. province onbehalfofthe inJanuary province, alsobegan Corporations administering onbehalfof the incometax collecting personal Revenue isresponsible Agency for (CRA),which TaxCorporations andRetail SalesTax. TheCanada was responsible forBranch collectingrelated to sale oftaxpayers’ seizure and forregister the warrants liens,orobtain remain unpaid, collectors canusegarnishments, by phoneandsometimes Iftaxes visitsinperson. sendsnotices byBranch taxpayers mail,contacts owed province. to the To collectunpaid taxes, the the unpaidtaxes of lection ofasignificantportion col isresponsible for the (Branch), lections Branch itsCol- through ofFinance (Ministry), The Ministry 2012 oftheOffice ofthe Annual Report Auditor General of Ontario 3.11 TAX COLLECTION Over the last five years, the Branch collected last fiveOverthe Branch the years, 2011In the Budget, government Ontario the As ofMarch

billion of the approximatelybillion ofthe $330

billion intaxes owing itwas that

31, 2012, the tax about90%ofthe

amounts.

property.

31, 2012, Branch the

billion

billion - - collectors were not fullextent. Some usedto their enforcement toolsenough, andthat available to collectionactionswere notthat taken often soon ever, inanumberofcaseswe reviewed, we found work. outanddocumenttheir How collectors carry tizes accountsandby developing guidelinesto help process inrecent years by improving- how itpriori of our significant observations are as of oursignificantobservations • • • • The Branch has strengthened itscollections hasstrengthened The Branch esearch has shown that the probability of hasshown the esearch that ith the Branch losing75% Branch ofitsworkforce the ith appropriately registeredanch liens premises increasesisiting ataxpayer’s the for six months ormore. for sixmonths whereinstance nocollectionactionwas taken cases inoursample, was atleast one there two-thirds ofthe also noted inmore than that attempt taxpayer to the by reach phone.We an average ofsevenfor collectors to months once anaccountentered collections,ittook dramatically astimepasses.We found that full collectiononadelinquent accountdrops additional staff hadbeen additional staff its post-transfer needsand,asaresult, no had not fully Branch evaluatedour audit,the timeof eventual butatthe lossofpersonnel, employeestemporary to compensate for the received approval infiscal2009/10 to hire are expected. TheBranch than write-offs or even Thiscouldresult in even tripled. more many caseloadsdoubled CRA, collectors’ the transfer ofresponsibilities tobecause ofthe and saleofthose seizure for the liensand warrants enforce the weof casesthat reviewed, failedto itthen but,inanumber onproperties, and warrants but itmadenofieldvisit. andhadbrokentax apayment phone ataxpayer whoowed $100,000 insales unsuccessfully for nearly two years to by reach Forwarranted. tried Branch example, the we reviewed, we although were felt they visits were not madeinanumberofaccounts likelihood ofcollecting whatisowed. Field R W The Br V

properties.

hired.

arrangement, arrangement,

follows: - and giving consideration to teaching performance toand givingconsideration performance teaching quality, centres teaching includingestablishing put someprocesses in placeto improve teaching tions available to choose students to helpthem evalua - results ofthese summarized makethey the universities indicated that a quarter ofOntario’s universities wewas doneatthe visited. Only about student evaluations little aggregate analysis ofthe take. However, they course of each we found that encourage students to complete evaluations formal used to doso.For universities allOntario instance, wascouldbe found available information that onaregular basis,we performance on teaching ties we visited were formallyassessingor reporting measures couldbedevelopedthat to insight offer quality. teaching graduate most felt Nevertheless, itisnot easytous that quantify andassessunder universities wespoke three to visited atthe told publicwould and the agree. future workforce. We believe parents students, their for the them itsstudents andpreparing of teaching itdoesagoodjob mandatemost important isthat level,school anditsgoalisto have 70%ofOntar require high beyond educationandtraining the 70%ofallnew jobswill that expects ties (Ministry) their into teaching credentials by post-secondary ians attain under instructors sessional responsibilities andpart-time no research with responsibilities,staff teaching and research ulty, teaching with includingtenure-stream staff universities employed about15,000 full-timefac- students eligiblefor provincial funding.These equivalent ofabout390,000full-time ties hadthe In 2010/11, 20publicly assisted universi Ontario’s - TEACHING UNDERGRADUATE3.12 UNIVERSITY All three of the universities we ofthe visitedAll three had universi- northe Ministry the neither Although headsweThe deansandfacultyordepartment perspective, auniversity’s From Ministry’s the ofTraining,The Ministry Colleges andUniversi-

courses.

quality.

QUALITY

contract.

2020. Overview and Summaries ofValue-for-moneyOverview Audits and Reviews - - that teaching quality teaching isvalued,encouragedthat and However, we feel universities needto better ensure when makingdecisionsonpromotions andtenure. rewarded. are as Ourkey observations • • •

try is making progress toward ismakingprogress try universities ingeneralario donot require 10% to 24% offull-time-equivalent peopleaccounted for eventhese ors, though beprovidedappraisals - to sessionalinstruct we visited performance required written that universities skills. None teaching ofthe their to hadsoughtassistance members improve specific guidance was providedthat faculty or was noevidence that butthere performance, ofteaching evaluations hadbeencritical Weformance. noted examples where student per onteaching feedback appropriate with didnot appraisals provide performance them program ofstudy.program their university and choosing beneficial in employmentmation ongraduate outcomes indicated students would to usthat findinfor Undergraduate StudentThe Ontario Alliance related studies. skillsacquired to intheir the were employedistry was full-time inajobthat Min- only by surveyed 65%ofgraduates the 2020. However, two graduation, years after credentials by population holdpost-secondary itsgoalofhavingachieving 70%ofOntario’s inteaching training facultyhadformal ally ofthis allmembers Interestingly,ratings. we- were virtu told that facultiesinoverallother effectiveness teacher education facultyconsistently outperformed evaluationstudent results course showed the peryear.hour perinstructor At oneuniversity, workshopsteaching averaged one lessthan we visited showed facultyattendance at that centres oftwouniversities learning ofthe and teaching Recordsin teaching. atthe tofaculty members have training formal A numberoffacultyt The Minis Ont

methods. old us their annual old ustheir

follows:

staff. - - 25

Chapter 1 Chapter 1 26 rehabilitation. to community-basedprovide shift becauseofthe they and services numberofprograms increase the agenciesagencies, have eventhe though hadto faster fundingto ratemuch transfer-payment than ata costs operating grew 5%. Aswell, ministry increased by only served numberofyouths the increased by program 25%–30%,while Services Youthtotal inthe expenditures program Justice community-based approach. rehabilitation todecade, from anincarceration-based approach a inphilosophy overhas undergone ashift last the agreements. cost-sharing under various $67 contributed ernment 200 community-based agencies. Thefederal gov $168 Youth including on the program, Justice Services open facilitiesand400insecure facilities). custody/detention600 inyouth facilities(200in and 9,200—8,600 undercommunitysupervision justice system youth was about tion inOntario’s tive programming. ute to communitysafety, largely- rehabilita through - andto incidenceofreoffending contrib reduce the TheYouth aimsto program court. Justice Services orwhohave acrime, beenfoundwith guiltyby a being charged awaiting after are trial primarily aged youths 12 to Ontario to 17and services who provides communityandcustodialistry) programs ofChildren andYouth (Min- The Ministry Services 2012 oftheOffice ofthe Annual Report Auditor General of Ontario SERVICES 3.13 YOUTH JUSTICE • In 2011/12, spent$370 Ministry the Our other observations included the following: includedthe observations Our other 2010/11From 2005/06to the the fiscal years, program Like the many jurisdictions, other fiscal2011/12,During the average daily popula- rowth indirect costs operating isprimar

60% of all full-time ministry staff in the Youth inthe staff 60% ofallfull-timeministry exception than the ofprobation offices.More areas increased substantially,program with ber offull-timeemployees justice inallyouth five-year ending2010/11,the period the num- ily dueto anincrease inemployee costs. Over The g million intransfer payments to approximately

PR OGRAM

million toward costs these

million - - • • • • verage signifi daily varies cost peryouth 10/11, onaverage, about50%of try recidivism rates (reoffending) weretry

ces officers increased ces officers by 50%. 2010/11,- servi numberoffull-timeyouth the facilities decreased by 37% from 2006/07 to averagethe daily populationinthese youth secure facilities.Although Ministry-operated were program workingJustice Services in from $1,001 to $1,483 for Ministry-operated for agency-operated secure facilities,and operated openfacilities,from $475 to $1,642 rangedyouth from $331 to $3,012 for agency- example, in2011, average the daily cost per cantly amongcustody/detention facilities.For still bejust 58%in overall the utilizationrateprojects that will facilities.However,in existing Ministry the by closingfacilitiesorby fundingfewer beds system,number ofbedsavailableeither inthe improve utilizationrate by the reducing the to hastried Ministry years,Over the the bedsincustody facilitieswerethe occupied. come into contact with the program. the come into with contact whohaveexclude 80%of youths more than sentences. However, recidivism statistics these custody and 59%for who hadserved youths communitysentences 35% for with youths conditions were unverifiable. was not orthe there enoughdocumentation we couldnot determine compliance because conditions were not or beingcomplied with, being documented. Also, many court-ordered needs and identified rehabilitation were not ever, required many assessments ofthe risk to gooduseto needs.How manage youths’ ofprobationwas put officers and experience we noted many knowledge timeswhere the system. Aswell, inour review ofcasefiles, the isin probationthe officeranyyouth time is to have same caseassignedthe ayouth’s model hasbeenapositive initiative. Theaim “single-case management” The Ministry’s secure The a In fiscal20 Minis

facilities.

2012/13. - - regarding the province’sregarding financialconditionand the provides section that additional information aFinancialtains Statement Discussion andAnalysis province’s annualreport. are included in the Report, my statements, alongwith Independent Auditor’s oromissions. Theconsolidated financial errors is,free ofsignificant misstatement—that material statements are free of the reasonable that assurance statements. Theobjective ofourauditisto obtain and proper records are maintained. assets are safeguarded, aretransactions authorized, procedures, isinplaceto ensuresupporting that aneffective system ofcontrol, with that ensuring sented fairly. Thegovernment isalsoresponsible for amounts basedonestimates andjudgment,ispre- many includingthe information, the that ensuring consolidated financial statements involvesthe financial information. volumes ofadditional supplementary and three province’s consolidated financial statements,the including province’s annualreport, comprise the cial Administration Act(Act). ThePublicAccounts Minister ofFinance,of the asrequired Finan by- the 31ing onMarch direction are prepared underthe PublicAccounts fiscal foryear end- each Ontario’s Introduction Chapter 2 Chapter The province’s 2011/12 alsocon- annualreport these consolidated financial My Officeaudits responsibility forThe government’s preparing of theProvince Public Accounts Accounts consist of the following:Accounts consist ofthe public. and the Legislative government toAssembly the the both of fiscal accountability enhancesthe information fiscal accomplishedyear. inthe Providing such governmentincluding somedetails ofwhatthe the fiscal results year ended March for 31, 180 fiscal year. end ofthe days ofthe three The Lieutenant to Governor the report inCouncilwithin government deliver the circumstances, itsannual cial statements. province’stion presentedconsolidated finan inthe informa Public Accounts for the consistency with andinVolumes annualreport ince’s 1and2ofthe • • • My Office reviews the information in the prov in the information My Office reviews The three supplementary volumes of the Public volumes ofthe supplementary The three The Act requires except that, inextraordinary olume 3—detailed schedules of ministry ofministry olume 3—detailed schedules olume 2—audited financial statements of and olume 1—statements from allministries recipients. payments to vendors andtransfer-payment audited financial statements; and cial statements, aswell miscellaneous asother province’s consolidated finan- included inthe and commissionswhoseactivitiesare boards,significant provincial corporations, financialinformation; and other liabilities,itsloansandinvestments,and other province’s revenues itsdebts and expenses, providing a numberofschedules details ofthe V V V

2012, - - - 27

Chapter 2 Chapter 2 28 and constructive. working relationship hasalways beenprofessional issues,our always onfinancial reporting agree Provincialof the Controller. Whilewe mightnot Office the andparticularly with Finance (Ministry) Accounts of we work Ministry the closely with reads asfollows: 31,ended onMarch 2012, It isfree ofreservations. consolidatedthe financial statements year for Legislative province’s to AssemblyReport the onthe my that IndependentAuditor’s pleased to report province’s consolidated financial statements. Iam results ofmyannually ofthe onthe examination The Auditor General Actrequires Ireport that ber volumes, onSeptemAccounts- supplementary Public Financial three Statements, the alongwith 2011/12ince’s andConsolidated AnnualReport 10within timeit resumes sitting. days ofthe lay Assembly itbeforetion publicandthen the Assembly- isnot informa if the insession,make the Legislative beforemust lay Assembly the them or, Lieutenant Governordocuments, the inCouncil fiscal year. endofthe of the these Upon receiving Lieutenant 240 Governor days inCouncilwithin volumes must besubmitted to the supplementary 2012 oftheOffice ofthe Annual Report Auditor General of Ontario Statements Consolidated Financial 2011/12The Province’s In conducting our annual audit of the Public In conductingourannualauditofthe This year, prov government released the the

13, 2012, 180-day meeting the deadline. which comprise the consolidated statement of comprise the which the Provincefinancial statements of ofOntario, I have accompanying audited the consolidated Ontario To Legislative Province Assembly the ofthe of Report Independent Auditor’s - that are appropriate in the circumstances, but circumstances, inthe are appropriate that statements inorder to designauditprocedures consolidated financial ofthe fair presentation control relevantpreparation entity’s and to the auditor internal considers assessments, the due to orerror. fraud risk Inmakingthose consolidated financial statements, whether misstatement ofthe ofmaterial risks of the assessment judgment, including the auditor’s ments. Theprocedures selected dependonthe consolidated financial statedisclosures- inthe amountsand auditevidence aboutthe obtain An auditinvolves procedures to performing misstatement. financial statements are free from material consolidated the aboutwhether assurance auditto reasonable obtain the and perform I comply requirements ethical andplan with iting standards. Thosestandards require that Canadiangenerallyance with accepted aud- on my audit.Iconducted my auditinaccord- consolidated financial statements based these My responsibility isto anopinionon express Responsibility Auditor’s duetoment, whether orerror. fraud are misstate free from- material ments that preparation ofconsolidated financial statethe - to enable Government determines isnecessary control asthe internal standards, andfor such Canadianpublicsector accounting ance with consolidated financial statements inaccord- preparationofthese andfairpresentation the isresponsible forThe Government ofOntario dated Financial Statements Management’s the Responsibilityfor Consoli- information. explanatory other ofsignificantaccountingpoliciesand summary a and cashflowthen endedand the year for in net debt,change inaccumulated deficit, consolidated statements change ofoperations, financial positionasatMarch 31, 2012, the and statements donot have any orsignificant material province’s consolidated financial the to that users words,statements. Inother Iamcommunicating financial for governments their whenpreparing recommends tute Accountants (CICA) ofChartered CanadianInsti- accountingstandards the ance with financial statements have beenprepared inaccord- province’s consolidated the ably concludethat basedonourauditwork,means that Ican reason 31,at March 2012. This“clean”auditopinion 2011/12for the fiscal year andits financialposition statements fairly province’s fiscal results present the consolidated financial the indicates that tion, which The above- any reserva auditopinioniswithout September 6,2012 Toronto, Ontario public sector accountingstandards. Canadian endedinaccordanceyear with then accumulated deficit,anditscashflows the for ations, change initsnet debt,change inits 2012 consolidated results ofitsoper andthe asatMarch Provinceof the ofOntario consolidated financialposition respects, the statements present fairly, inallmaterial In myconsolidated financial opinion,these Opinion vide abasisfor my opinion. issufficientandappropriate obtained to pro- auditevidence IhaveI believethe that consolidated financial statements. overall as evaluating the ofthe presentation estimates Government, madeby aswell the reasonableness ofaccounting used andthe ofaccountingpolicies appropriateness the control. Anauditalsoincludesevaluating internal entity’s effectiveness ofthe on the anopinion ofexpressing not purpose for the

A Jim McCar [signed] Licensed PublicAccountant uditor General ter, FCA

31, - - standards in the 1993/94standards inthe fiscal year. CICA’sadopt the Public Sector Accounting Board the provincefinancial statements since moved to province’s consolidated audit opinionsonthe predecessor andIhave beenableto issue“clean” standards. Accordingly, CICA the respects with my inpower,ical party have complied inallmaterial governments, polit- regardlessall Ontario ofthe past 19 inthe is anotable achievement that years, was required year. this reservation would berequired. normally However, nosuch inmy auditopinion adjustments, areservation management was not willingto required make the vidually orcollectively and threshold, exceed the revenues year. for the misstated items indi- Ifthe at around 0.5%oftotal government or expenses Historically,threshold. hasbeenset threshold this oromissionmaterial. error the answerstatements?” Iconsider Ifthe isyes, then province’s consolidatedby financial ofthe users decisionsmade that itcouldaffect nificant enough ormisstatement sig- error question “Isthis I askthe onmy professionalprimarily judgment.Essentially, (insignificant)isbased (significant) andimmaterial Anassessmentofwhatismaterial statement users. ifitisconsidered significant is material to financial overall consolidated financial statements. Anitem unrecorded ormisstated itemthe inrelation to the orsignificanceof materiality auditors considerthe statements. properly province’s consolidated financial inthe have not beenrecorded either ornot beenrecorded transactions meanssignificantfinancial reservation a Anauditopinionwith areservation. opinion with standards, Iwould berequired to issueanaudit accounting CICA the compliancegovernment’s with year. the actually transpired during andprovide afairreflectionerrors ofwhathas As a final comment, I wish to point out that it As afinalcomment,Iwish to pointout To assessmentIcalculate make amateriality this isneeded, areservation In determining whether If Iwerethe to have with significantconcerns Public Accounts oftheProvince 29

Chapter 2 Chapter 2 30 and Fiscal Review, ofOntario and Fiscal the AuditorGeneral Officeof andthe 2012Source ofdata: Budget,2011/12 Ontario 2012 Statements, andConsolidatedFinancial EconomicOutlook Ontario AnnualReport Provinceof Ontario Figure 1:Total Deficit,2007/08–2014/15 Debt, andAccumulated Debt,Net ($million) the sumofallpast represents deficit, which likely $320 surpass total liabilitiesanditsfinancialassets,ment’s will between difference govern isthe - the debt, which endof2007/08. debtwasoutstanding atthe Net still total $340 billionordoublewhatthe more than public debt,T-bills andU.S. commercial paper, will markets, publiccapital bonds issuedinthe non- government owesthe of form to inthe outsiders total represents amountofmoney the debt, which as itprojects in2017/18,its books total Ontario’s year, government we balances ifthe estimate that on itsprojected 2014/15 debtbeyond the fiscal tinues to increase, asillustrated inFigure 1. by total debt,net debt,oraccumulated deficitcon- measured province’s debt,whether the that observe alarge debtload. carrying consequences provincehighlighted to the the of and jurisdictions, Canadianandinternational other ofnet ratio compared debt-to-GDP to Ontario’s spending. deficits andincreased infrastructure We to government borrowing to finance recent large province’s growing debtburden was attributable debt, andaccumulated deficit. the We notedthat ent measures ofgovernment debt—total debt,net In our2011, we differ AnnualReport discussedthe 2012 oftheOffice ofthe Annual Report Auditor General of Ontario Total debt 12 Ontario Economic Outlook and Fiscal Review EconomicOutlook and Fiscal 12 Ontario 11/12 Statements ConsolidatedFinancial ProvinceofOntario c. Budget 12 Ontario b. a. Accumulated deficit Net debt Burden DebtUpdate on Ontario’s

20 20 20 While the government hasnotWhile the provided details year,In updatingourdebtanalysis this we

2007/08 billion, and the accumulated billion, andthe 156,616 105,617 162,217 a 2008/09 169,585 113,238 176,915 a 2009/10 Actual 193,589 130,957 212,122 - a 2010/11 236,629 144,573 214,511 annual deficits and surpluses, will stand ataround will stand annual deficitsandsurpluses, ing at a faster rate than the provincialing atafaster the economy. rate than net government’s debtisgrow itmeansthe rising, ability to is ratio manage itsdebtload.Whenthis considered to beagoodindicator ofagovernment’s domesticdebt to product (GDP)—isgenerally gross economy—thatis,netrelative sizeofthe to the increasing ordecreasing over level time,the ofdebt net to whether debtis While itisimportant examine best fiscalsituation. measure ofagovernment’s the We noted considered net last year debtisoften that $210 ment, Ontario has the highest net debt-to-GDP has the ratio ment, Ontario province federal governof the ofQuebecandthe - diction. AsofMarch amount ofdebtowed- juris resident by ofthe each respective ofnettheir ratios debtto GDP, andthe federal provinces, government andthe alongwith Figure netjurisdictions. 3illustrates debtofthe the debt loadisto Canadian compare other itwith provincial economy. ratedebt willfallbelow expected ofthe growth the rate expected ofgovernment growth when the provincial the economyrate than until2015/16, government debtwillcontinueto grow atafaster indicates ratio that inthis The projected growth 2011/12the fiscal year, asillustrated in Figure 2. ily from alow of26.8%in2007/08 to 36.9%in ONTARIO’S NETDEBT Ontario’s net stead debt-to-GDP hasrisen ratio Ontario’s Another useful tool for assessing Ontario’s usefultoolAnother for assessingOntario’s

billion by 2017/18. a 2011/12 235,582 158,410 257,278 b 2012/13

31, 2012, exception the with 172,800 278,000 257,600 c Estimate 2013/14 185,600 297,500 277,600 a c c 2014/15 293,300 195,700 314,100 - - a c c of developmentsregard sinceour2011 inthis However, risk. this have there beenanumber ing reliance onforeign would lenders, increase large borrowing requirements, itsgrow alongwith somenoted Ontario’s downgraded, although that credit needed to rating be to suggest Ontario’s that Source of data: 2012Source ofdata: Review EconomicOutlookandFiscal Ontario 2005/06–2017/18 (%) Ratio, Debt-to-GDP Net Figure 2:Ontario respect to specifieddebtobli- with creditworthiness A credit isanassessmentofaborrower’s rating Annual negative consequences. Theseinclude: province’s escalatingindebtedness hasanumberof In our2011 , we the AnnualReport highlighted that of ajurisdiction. highest amountofdebtowedand the by residents 45 20 25 30 35 40 10 15 ONTARIO’S CREDITRATING INDEBTEDNESS CONSEQUENCES OF HIGH

2005/06 0 5 • • • At that time,someanalysts sawAt that littleevidence tential credit-rating downgrades andchan awayvicing costs taking funding reater to vulnerability increases ininterest ges ininvestor sentiment. rates; and needed for programs; other 2006/07 po g debt-ser

Report 2007/08

2008/09 . 2009/10

2010/11

2011/12

2012/13

2013/14 Actual 2014/15 Projected

2015/16

2016/17

2017/18 - - ditions. Anoutlookisnot of aprecursor necessarily business con- changes ineconomic orfundamental agency any considers the outlook, ing arating totypically two sixmonths years. Whendetermin - direction over ofarating intermediate term, the potential indicates the issue acredit outlookthat assign similarcredit from ranging ratings AAAto D. orinterest.for recovery ofprincipal S&PandDBRS littleprospect indefault,with rated andare often whereas lowest obligations rated Care the risk, est quality lowest andsubjectto the level ofcredit Obligations are ratedjudged Aaa high- to beofthe andNR(notCaa, Ca,C,WR(withdrawn) rated). Baa,Ba,B, A, Aa, assigns credit ofAaa, ratings to debt.For rate ajurisdiction’s example, Moody’s unwilling to rating. holddebtbelow acertain (due to orinstitutional contractual constraints) or debt, assomeinvestorsa government’s are unable potentialalso result inareduction ofthe market for downgrade Arating can jurisdiction. lend to that ofahigherinterest form rate beforewill the they will accordingly premium in risk demandagreater ment defaultingonitsdebtishigher, andinvestors govern- agencyofthe believesrisk the cates the that government borrowing becausealower indi- rating A credit cost hasanimpact rating offuture onthe availability ofcredit. cost andthe the both affecting credit whenmakinginvestment rating decisions. and susceptibility toInvestors major risks. usethis financialhealth, prospects, institutional strengths, economicresources government’s and as that factors manage considersuch itsdebts,andthey largelycreditworthiness basedonitscapacityto Standard(Moody’s), (S&P),andDBRS. andPoor’s credit-rating agencies are Investors Moody’s Service cies to markets. issuedebtincapital main Thethree requires from ratings recognized credit-rating agen- obligations inatimely manner.these Theprovince of aborrower to intereston pay andprincipal the capacityandwillingness gations. Itindicates the In addition to a credit rating, the agencyIn additionto acredit may the rating, Credit-rating agencies useletter designations Credit influenceborrowing ratings conditions by Credit-rating agencies assessagovernment’s Public Accounts oftheProvince 31

Chapter 2 Chapter 2 32 Figure 4. to Canadian seniorgovernments other isshown in relative rating downgrades inlate 2009.Ontario’s S&PandDBRS todowngrade the isacatch-up fallof2009.Itmay well Moody’s in the bethat sincedowngrading rating itto AA(low)Ontario’s had downgraded in2009.DBRShasnot changed it which AA–rating, province’s current the tained to “negative”on Ontario andmain- from “stable” 2012 hand,inApril other S&Prevised itsoutlook Onthe to credit from rating Aa2. Aa1 Ontario’s December 2011, 2012 andinApril itdowngraded province in hadputthe Moody’s onacredit watch government released its2012 the after budget. province’s credit shortly rating assessment ofthe term. short in the isnot arating likelyoutlook meansthat to change mightbelowered, arating means that andastable Anegative mightberaised. arating outlook that upordown.ing—either Apositive outlookmeans agency’s potential viewthe ofthe evolution- ofarat investors informs changea rating butrather about ofOntario the AuditorGeneral Office of 2011/12 provinces,2012 ofother Statements Canada,andthe andConsolidatedFinancial provincialbudgets,Statistics AnnualReport 2011/12Source ofdata: 2012 Statements, andConsolidatedFinancial Budgetandbudgetupdates, Federal AnnualReport ProvinceofOntario 2010/11 ($)ofCanadianJurisdictions,2011/12 DebtPer Capita GDP(%)andNet Debtto Debt($),Net Figure 3:Net and 2012 oftheOffice ofthe Annual Report Auditor General of Ontario ON SK BC QC MB AB NB PEI NS Federal NL All three credit-rating agencies All three updated their (Net Asset) 235,582 ($ million) 650,135 170,887 (18,991) 13,243 10,046 35,973 Net Debt 14,511 3,560 1,737 7,769 2011/12 (Net Asset) to GDP(%) to Net Debt 36.9 25.5 35.1 32.6 23.5 35.0 51.2 (6.6) 7 0 17. 7 8 37. 6.3 Per Capita ($) Per Capita (Net Asset) 15,094 18,796 14,049 21,436 12,037 Net Debt 13,311 11,814 (4,901) 17,647 3,343 ,815 7, in the nearfuture. in the nomore adjustments rating expect indicated they financial position sohavethe medium overterm, fiscalflexibility to improve necessary has the its The credit-rating agencies alsobelieveOntario that to reluctant therefore invest jurisdictions. inthese downturn to extent, agreater andinvestors are have 2008global financial beenaffected by the world around the because many jurisdictions other debt. Ontario indicatesrevised, investors which still want to hold relatively were ratings unchanged sincethe bondinterest costs havecosts. Ontario’s remained borrowing had asignificantimpact onOntario’s evidence yet suggesting latest have ratings these isno future borrowinggovernment’s costs, there provinces’ credit theoretically ratings increase a for the While downgrades andpoorer outlooks REVISED OUTLOOK IMPACT OF LOWER CREDIT RATING/ Ontario bondsremain relativelyOntario attractive (Net Asset) ($ million) 158,955 616,900 214,511 (21,653) 30,637 12,837 12,837 Net Debt 9,480 3,783 1,695 8,129 2010/11 (Net Asset) to GDP(%) to Net Debt 35.0 24.0 50.1 33.2 34.9 38.0 28.8 35.7 15.2 (7.4) 6.2

Per Capita ($) Per Capita (Net Asset) 12,565 13,549 15,866 10,322 20,021 11,723 16,134 Net Debt (5,784) 17,987 3,600 6,728 provinces have higher ratings. federal Western Only government the Aa2. andthe provinces,credit aMoody’s of rating Maritime with samecredit asQuebecandthe rating has the in federal government debt. the relative ofinvestorbenefits from strength faith federal government, ofthe Ontario worthiness perceived credit the - debtwith associate Ontario to pushdown interest rates. And,because investors relatively risk-free investments. Thisdemandworks cial, ishigh,especially amonginvestors lookingfor adian government federal debt,both andprovin - canbeassigned.Thismeansdemandfor Can- that highest creditretained rating—the itsAaa/AAA worldfew has inthe remaining jurisdictions that paying down debtwould lineon any be to holdthe oncedeficitshave one strategythat tackled, beenfor andfutureexisting debt. We time indicated atthe strategy hadbeenpresented for paying down its the 2017/18its annualdeficits by fiscal year,no government hadpresentedthe aplanto eliminate In our2011 we while AnnualReport concludedthat budgetdocuments, and LaurentianProvincialMonitor, Provincialandfederal Source ofdata: asofApril26,2012 Target andBalancedBudget Figure 4:CreditRatingby Province Date Federal NL PEI NS NB QC ON MB SK AB BC CONCLUSION Even with its most recent downgrade, Ontario itsmost recentEven downgrade, Ontario with The government ofCanadaisalsoonethe Aaa Aa2 Aa2 Aa2 Aa2 Aa2 Aa2 Aa1 Aa1 Aaa Aaa Service Moody’s Investors AAA A A (low) A A (high) A (high) (low) AA A (high) AA AAA (high) AA DBRS These commentscontinueto beappropriate. andprojected debtburden. current address the targets long-term andastrategywith for how to should considerproviding public legislators andthe debt.Wethe went government on to the say that generated by agrowing economy reducing to start future debtincreases anduseadditionalrevenues AAA A+ A A+ AA– A+ AA– AA AAA AAA AAA &Poor’s Standard expected to peakat27.5% in2014/15. The accumulated deficit-to-GDPratio is below 41.3% the 2012 forecast inthe Budget. expected to peakin2014/15 at41.2%, is which is eliminated. Thenet debt-to-GDP is ratio deficit andbegintowill stabilize decline asthe deficits andinvestmentsratios The incapital. expected to projected increase dueto the 2012$0.4 billionfrom the Budget forecast. projected to be$14.4 billion, animprovement of the 2012/13Province’s deficit for fiscal yearis ahead ofitstargets inlowering deficit.The the year inarow,decade. For fourth the is Ontario second-lowest ina the inOntario rate ofgrowth 1%— spendingwas lessthan inprogram growth government ismanagingspending.Last year, In order totargets, Province’s fiscal meetthe the The Province’s debt-to-GDP are ratios MINISTRY RESPONSE MINISTRY Public Accounts oftheProvince 2015/16 2014/15 2014/15 2013/14 2014/15 2013/14 2017/18 2014/15 n/a (insurplus) 2013/14 2013/14 BalancedBudget to Target Return Datefor 33

Chapter 2 Chapter 2 34 would have showed in surplus itsfirst 10 years. liability. WSIB change, impact the ofthis the Absent rate introduced in2011 added$2billionto the that pallychange duetothe discount asignificant to unfunded liabilityhascontinuedto grow,- princi WSIB’s fund,the insurance ofthe operations the fiscalperformance. government’s on the financial would havereporting asignificant impact WSIB government’s Includingthe inthe trust. true WSIBquestioned the was whether like operating a factors, we other unfunded liabilityandvarious however,sification asa“trust”; given itssignificant the financial results isbasedon clas- WSIB’s these province’s financial statements. Excludingthe WSIB’s exclusion ofthe financial the results from to provide worker benefits. unable to meet andfuture itsexisting commitments ity andultimately WSIB couldresult inthe being system’s financialviabil- to the liability posedarisk unfunded andmagnitudeofthe growth the that liability. sectionwe Inthat ourconcern expressed WSIB’s discussedourreview ofthe unfunded that includedaseparate section 2009 AnnualReport obligations to pay benefits to injured workers. Our WSIB’sof the assets anditsestimated financial liability, between difference value isthe the which the growth in unfunded WSIB’s significant the made anumberofreferences about to ourconcerns premiums onemployerthrough payrolls. receives nofundingfrom government; itisfinanced toassistance workers job.TheWSIB injured onthe isto andmedical providepurpose incomesupport Safety andInsurance Act,1997 (Act). Itsprimary created Workplace by corporation the is astatutory The Workplace Safety Board andInsurance (WSIB) 2012 oftheOffice ofthe Annual Report Auditor General of Ontario Safety and Insurance Board and Insurance Safety Liability of the Workplace Update on theUnfunded Despite efforts taken by managementDespite efforts to improve We alsourgedgovernment to the reconsider have past decade,ourannualreports Over the ficiency ratios ficiency by specified dates: WSIB to following ensure itmeets the fundingsuf- 1, effective January 2013,which, willrequire the government filed Regulation 141/12the Act, under inJune2012 report, Arthurs the mendations ofthe ing key elements: funding strategy WSIB, follow for the includingthe financially andsufficiently stable funded. of recommendations WSIB to ensure the remains was comprehensive aseries andcontained Arthurs system. TheMay 2012 by Professor report Harry workplaceity ofOntario’s safety andinsurance how financialviabil- tolong-term best ensure the independent fundingreview to provide adviceon of December liabilities—was 52.2%asofDecember WSIB’s percentage ofassets fundingratio—the to its December totalled $14.2 As ofDecember ition. For 2011, example, during WSIB the reported improved results itsoperating andfinancialpos- WSIB have continuesto take actionsthat recently in our2011 follow-up AnnualReport section,the dates. As reported prescribed goalsby the these measures itwilltake tooutlining the achieve Ministera planto ofLabourby the June • • • • • • In response to our concerns and to the recom- andtoIn response to the ourconcerns In particular, the report recommended anewIn particular, report the In September 2010, WSIB the announcedan The regulation alsorequires WSIB the to submit 00% onorbefore December31, 2027 he WSIB to oncourse achieve a WSIBving the asquickly asfeasible beyond ealistic assumptions, includingadiscount 90%–110% 20years. within fundingratio sufficient funds to pay workers’ benefits);and andbyframe reasonable measures generate WSIB areasonable time the couldnot within inwhich ping pointbeingdefinedasacrisis (tip- a “tippingpoint”of60%fundingratio advice; best actuarial rate basedonthe 1 80% onorbef 60% onorbef putting t mo r

31, 2010, ithadbeen54.7%.) 31, 2010, balanceof$12.4

billion, anincrease of$2

31, 2011, unfundedliability the ore December31, 2022 ore December31, 2017

31, 2011. (As

billion from

30, 2013, billion. The - standards. assumptions andproposed new accounting will beaffectedactuarial by changes to certain the regulation in ratios prescribed ing sufficiency quickly aspossible. focus ongetting injured workers to work back as improvements buthave WSIB’s demonstrated the ment have not financial only to contributed these and fewer impair workers permanent suffering claims, more workers to returning work sooner, Increased premium revenue, fewer new lost-time discountrate discussedbelow). change inthe the in surplus operational its first 10 (excludingyears • • However, WSIB’s fund- the abilityto achieve the SIB reports itsfinancial results basedon SIB reports The new standard also proposes that anew The new standard alsoproposes that benefit obligations by anestimated $3 estimate ofWSIB’s actuarial increase the marketon current interest could rates, this uses to calculate itsbenefitsobligation. Based discountrate WSIB reduction inthe a further liability significantly because it would require unfunded itcouldincrease the form, current endoffiscal2016.as ofthe Ifapproved inits liabilitiesisexpectedinsurance to take effect (IFRS). Aproposed new standard for valuing Financial StandardsInternational Reporting obligation. add approximately $2.1 discount rate by onepercentage pointwould 6.0% thereafter. For example, reducing the 2012–2015 period for the andincrease itto to 5.5%discountrateappropriate retain this standards (asdiscussedbelow), itwould be subjectto anythat changes inaccounting liability. WSIB management have concluded an almost $2 rate reduction from 7.0% to 5.5%resulted in on itsinvestments. 2011 During adiscount companyinsurance orpensionfundcanearn expectedperceived rate an ofreturn asthe Inlayman’srent dollars. itisoften terms, calculate future benefitobligations incur The W The discountr

billion increase in the unfunded billion increase inthe ate is the interestate rate isthe usedto

billion to the benefits billion to the

billion. - - evaluate ourposition. isnotenough progress beingachieved, we willre- regulation. Shouldwe by the prescribed ratios feel being madetoward fundingsufficiency meeting the However, we willcontinueto progress monitor the province’s liabilities. unfunded liability from the the 2011/12exclusionthe therefore fiscal of year and the the for tinued classificationof WSIB asatrust address itsunfundedliability, con- we the support with the government on the merits ofadopting merits government onthe the with WSIB’s. Consequently, WSIB the willconsult asthe fundsuch insurance a goingconcern IFRS standards are not well suited to funding by regulation, prescribed ratios we believe that fundingsufficiency poses ofcomplying the with ing itsfinancial statements. However, for pur or falling. interestdepending onwhether rates are rising large positive magnitude,both andnegative, volatility aswell asone-timechanges ofavery at discrete pointsintime.Thisintroduces marketing liabilitiesusingthe rates prevailing liabilitieswillhaveinsurance ofvalu- effect the The proposed new IFRSstandard for valuing As a result of the government’s commitmentsto government’s As aresult ofthe WSIB isfollowing IFRSstandards inprepar WSIB RESPONSE WSIB tion that could increase the benefits obligation could increase the tion that benefitsobliga- margin beincludedinthe risk funding ratio of60%. funding ratio WSIB’sthe abilityto meet its2017 legislated proposed standard couldsignificantly impair combined$4 costs, the success inreducing lost-time claims to reduce rate, itsinvestment orfurther performance WSIB’spensating increases inthe premiums best estimate assumptions.) any Without com- result using corresponding calculation andthe betweendifference actualresult ofa the likely for the isareserve deviations, which provision actuarial foris similarto adverse the $1 by afurther

Public Accounts oftheProvince billion. (This new risk margin billion. (Thisnew risk

billion impact of this billion impact ofthis - - 35

Chapter 2 Chapter 2 36 dividing itinto two components: retire $20.9 the by assets. ing $20.9 System Operator, IndependentElectricity the leav assets ofHydro One, OPGand valueofthe by the $38.1 ofthe Only aportion markettricity onApril was restructured elec Hydro whenthe liabilitiesfrom Ontario other liabilities not to transferred Hydro OneandOPG. other Hydro andcertain oldOntario debt ofthe was responsibility givento legacy manage the the (OEFC).OEFC FinancialElectricity Corporation Power Ontario Ontario (OPG)andthe Generation mainsuccessorcompanies:into three Hydro One, Hydro oldOntario ofthe breakup including the industry, electricity ofthe in amajor restructuring Energy Competition Act,1998the resulted, which bill. electricity component ratepayer’s ofnearly Ontario every DebtRetirementsector andthe Charge (DRC), a electricity stranded debtofthe commented onthe ofour2011In Section3.04 , we AnnualReport 2012 oftheOffice ofthe Annual Report Auditor General of Ontario Sector StrandedSector Debt Update on theElectricity rely onit. workers system andthe must fundthe whomust consequent for instability employersthe who valuationofliabilitiesfrom year tothe year and potentially large multibillion-dollarswingsin to willavoid valueliabilities.Thismethod which expected rate-of-investmentterm with return poses. Thiswillhave ofusingalong- effect the fund,for insurance fundingpur liabilities ofthe benefits basis”to valuethe a “going-concern The government’s long-term plan to service and planto long-term service The government’s $38.1OEFC inherited billionintotal debtand passage of the The stranded debtarose with

billion ofstranded debtnot supported

billion instranded debt involved

billion was supported

1, 1999. - - - consumers and the electricity sector. electricity andthe consumers debtrepaymenting the burden between electricity - mannerwhiledistribut stranded debtinaprudent Specifically,the Minister section85 that requires andform. spirit inboth had beencomplied with provisions Act these andassessingwhether of the provisions ofthe ofsection85 our interpretations tions from our2011 were AnnualReport basedon - section specifieswhenitis to end.The key observa to DRC, implement same the authority andthis Debt Retirement Charge” gave government the the Residual Strandedentitled “The Debtandthe DRC tomight expect seethe eliminated. ratepayersstranded debt,andwhenelectricity residual ineliminatingthe progress collected, the details abouthow DRC much revenue hasbeen revenue hadbeencollected. March $940 than same today. Currently, OEFCcollects more the andremains the watt hour(kWh)ofelectricity The rate was at0.7centsperkilo- established • • This structure was intendedThis structure to eliminate the Collection of the DRCCollection ofthe began onMay Electricity Act,1998 Electricity Section 85ofthe (Act) Our 2011 focused onproviding AnnualReport emaining $7.8 timated $13.1 debt was retired. ratepayers, residual strandedtricity untilthe Retirement Charge (DRC), to bepaidby elec- Act,1998Electricity anew Debt authorized companies. electricity Thestreams from the expected dedicated by revenueported the couldnot besup- stranded debtthat of the estimated ual stranded debt,wasportion the its investment two inthe companies. $520-millionannualinterest cost of ernment’s govof OPGandHydro Oneinexcess ofthe cumulative annualcombinedprofitsfrom the utilities),and municipalelectrical and the electricity-sector companies (OPG,Hydro One from payments inlieuoftaxes madeby the future revenue through streamsported The r An es

31, 2012, approximately $9.7

million ayear in DRC revenue. Asof

billion was to besup

billion, called the resid billion, calledthe

billion inDRC

1, 2002. - - - amounts goingupto March stranded to debtestimates April back illustrated inFigure 5,reflecting annual residual Outlook andFiscal chart, Review the alsocontained 1999. 2012 Theupdate inthe Economic Ontario $7.8debt sincethe update provided estimated residual stranded onthe 2012 to the Prior budget, hadbeenno public there estimate of$4.5 of $5.8 aresidual stranded debtestimate contained which transparency wasthe 2012 alsoreflected in budget, Gazette. amountinTheOntario the is to becalculated, andrequires of annualreporting how residualformally establishes stranded debt the of residual stranded debt.Thenew regulation amount requirements outstanding reporting onthe on May ment introduced Regulation 89/12 Act underthe anupdate. with suggested Minister shouldprovide the ratepayers We adecadewas longenough,and concludedthat having onpaying down residual stranded debt. the ratepayers payments were their onwhatprogress had anobligation to provide update to aperiodic Ministers intent ofsection85was that the was that residual stranded debt.Ourview amount ofthe outstanding publicdetermination ofthe no such a decadeago,we noted Minister hadmade the Act passage more than determine. ofthe Sincethe day solely government upto to ofthe the be left time to time”was not formallydefined,andcould “from term calculation. We the that alsoobserved regulation, to whatisto establish beincludedinits was to bedone,itdoesallow government, by the residual stranded debt determination ofthe the DRCof the must cease. residual stranded debthasbeenretired, collection the Minister determines that tions public.Whenthe “from timeto time”andto determina- make these of Finance residual stranded debt determine the We were increased level pleasedto see this of govern- the observations, In response to these Act didnotWhile the specifyprecisely how

billion asofMarch 15, 2012, to provide transparency andmeet

billion asofMarch

billion estimate onApril

31, 2011, andaprojected

31, 2012.

1, 1999, and

1, Source of data: 2012Source ofdata: Review EconomicOutlookandFiscal Ontario FiscalYear Each Liability for Since1999 ($billion) Figure 5:Residual DebtandOEFCUnfunded Stranded a material departure from PSAB standards, the risk from risk PSAB departure standards, the a material consolidated financial statementsince’s containing developments have not yet prov resulted inthe andsubsequent this Although Accountants (CICA). CanadianInstitute(PSAB) ofChartered ofthe followed PublicSector by Accounting the Board asthat an independent,consultative process such through than legislationrather through principles was atroubling precedent to accounting establish it , where we that our 2008AnnualReport warned lic sector entities. specificaccountingpolicies tofor itspub- prescribe authority givelegislation orregulationsitthe that by government passing hasdone this Ontario the be recorded. Onseveral occasionsinrecent years how to prescribe ing principles would transactions relying generally onestablished accepted account- than would rather circumstances apply incertain that legislated specificaccountingtreatments somegovernments havepublic accountingisthat One significant recent development inCanadian 20 25 10 15 Accounting Standards Accounting Standards Use Overrideof Legislation to 1-Apr-99 1999/2000 0 5 We in this practice about raised concerns first

2000/01

2001/02

2002/03

2003/04

2004/05 Public Accounts oftheProvince

2005/06

2006/07

2007/08 Residual strandeddebt OEFC unfundedliability 2008/09 Initial

2009/10 stra nded debt 2010/11

- 2011/12 37

Chapter 2 Chapter 2 38 these developmentsthese since2008: synopsisof has increased. Here isachronological infuture financial statements departures of such 2012 oftheOffice ofthe Annual Report Auditor General of Ontario • • • • he 2009/10 fiscal year,the ario government regulationsario now require 11, aregulation underthe accounting. not provide useofrate-regulated for the Financial Standards, do Reporting which planned Canadianadoption ofInternational follow U.S. cameinanticipationofthe rules direction to government’s future years; the rate-regulated entitiesto defer to expenses allow for accountingrules to OPG.American ment hassinceprovided samedirection the with U.S.with generally accepted accountingprin prepare itsfinancial statements inaccordance Administration Actdirected Hydro Oneto affected by both these legislative affected amendments. by these both government other organizations are certain dated boards and financial statements. School province’s consoli- ments are includedinthe or non-publicentitywhosefinancial state- accounting standards to beusedby any public tion Actallow government to the specifythe Financial AdministraAmendments underthe - financial statements. boards useinpreparing school accountingstandards that the scribe Act Education amended. Amendmentsunderthe Financial Administrationand the Actwere irrespective ofPSAB accountingstandards. government fiscal foryear ofthe that expense Act wouldmade underthe berecorded asan dated financial statements. Any transfers inits consoli- reported unplanned surpluses additional transfers to eligiblerecipients from to reduce itsaccumulated deficit andprovide related regulations allowed government the recipients as deferred capital contributions. contributions. recipients capital asdeferred assetscapital to beaccounted for by transfer transfers andtransfers oftangible capital ciples, effective January Ont In 20 In t The specified that the government canpre- that specified Investing Act,2008(Act) and inOntario

1, 2012. Thegovern Education Act Financial

- - tion willnot arise. situa this “clean” auditopinions,Iamhopefulthat However, past 19 basedonthe consecutive years of inmy auditopinion.need to includea reservation generally accepted accounting standards, Iwill than whatit significantly would beunder different underlegislated accountingstandards is surplus deficit or reported government’s the conclude that generally accepted IfI accountingprinciples.” appropriate inaccordance with information PublicAccounts, present fairly inthe as reported consolidated financial statements ofOntario, the General Actto provide an opinionon“whether financial statements, Iam requiredthe by generally accepted PSAB standards. prepare itsfinancial statements inaccordance with • As the auditor of the province’s consolidated auditor ofthe As the We continueto Ontario believe that itiscritical ection taken by the provinceection taken by the beginning this treatment. this to introduce aregulation requiring necessary governmentconsistency considered it the respect, sototo ensure inthis interpretation However,principles. PSAB standards are open generallycomplies with accepted accounting ing aswe it believeinmost instances that We have account- this historically supported related assets. isrecognized onthe expense into revenue samerate asamortization atthe are toThe amountstransferred bebrought financial statements. usedtoand practices prepare itsconsolidated regulations accountingpolicies regarding the to make fullauthority government with the istration Act.Theseamendmentsprovide latest Financial amendments to- Admin the Act(Budget Ontario for Measures), 2012 , the Strong inthe Action supported was further in 2008to legislate accountingstandards The dir Auditor - follows five steps: these PSAB generallying anaccountingstandard, the standard-setting process. Indevelopingthe orrevis - objectivityof the considered, maintaining thereby views ofallinterested are heardthe and parties public-sector accounting. source ofguidancefor primary Canada andare the accepted for governments accountingprinciples in public sector. PSAB standards represent generally sible for accountingstandards for establishing the adian Institute Accountants isrespon ofChartered - open andtransparent standard-setting process. nized professional bodyusingacomprehensive, byshould beestablished anindependent,recog- To beobjective andcredible, accountingstandards measured anddisclosedinfinancial statements. events andother transactions are to berecognized, Accounting standards specifyhow andwhen to accountability taxpayers.strengthen ship over manage, resources they andthereby the help governments publicly demonstrate steward- PSAB standards jurisdictions. are intendedother to against results of actualresults andagainst the expected financialperformance government’s the the publiccan evaluate key financial reports, same set ofaccountingstandards to prepare their financial the statements. Whengovernments use economic updates andyear-end consolidated annualbudgets, their estimates, printed preparing Most Canadiangovernments usePSAB standards in GOVERNMENTS ACCOUNTING FOR STANDARDS Issues Future Public Accounts • • The PSAB emphasizes dueprocess to ensure that The PublicSector Can- Accounting Board ofthe oval of aproject proposal; esearch; appr basic r financial reporting, budgetingfinancial reporting, andfiscalpolicy. governments operate whenmakingdecisionson adequatelythe unique environment reflect inwhich havethey questioned proposed standards whether andrate-regulatedinstruments accounting—where section,we next discusstwothe areas—financial proposals. In accounting andfinancial reporting PSAB’s severalhave ofthe with becomeconcerned years, Canadiangovernments, includingOntario, challenges. For past fewwithout inthe instance, ever, role willnot leadership be this maintaining levelsstandards atallthree ofgovernment. How application ofgenerally accepted accounting tor world consistent respect to the leaderwith ernment financial instruments, and is similar to the andissimilar to financialinstruments, ernment measurement, anddisclosure presentation ofgov ments”—provides recognition, guidance onthe accounting standard—PS 3450 “Financial- Instru April on orafter standard beginning effective for fiscalperiods ordeficit. annual surplus government’s the changes shouldaffect such financial statements and,inparticular, whether held by agovernment shouldbe reflected inits contracts fairvalueofderivative changes inthe force task hadto addressissues the was whether keyforeign-exchange Oneofthe contracts. forward swaps and ascurrency such debt, andderivatives force. include Examples offinancialinstruments creation ofaPSAB the task 2005with January began in offinancialinstruments reporting the The PSAB’s project to develop anew standard for Financial Instruments • • • Canada isgenerally regarded asapublicsec- In March 2011,In March PSAB approved the anew oving andpublishingafinal standard. to adesig tatement ofprinciples ants for feedback; andnon-account- ofaccountants nated group individuals andorganizations; and and solicitingcommentsfrom allinterested appr issuing oneormor issuing as

1, 2015. Thenew public-sector Public Accounts oftheProvince e public exposure drafts drafts e publicexposure - - - 39

Chapter 2 Chapter 2 40 financial instrument standard. financial instrument the needto beaddressed within identify issuesthat Underlying Financial project might Performance December ted to standards onorbefore reviewing these on government finances. needsfor public’s meet transparent information the of government anddoesnot financingtransactions the economicsubstance doesnotoperations, reflect statement of even outsidethe reports, ifreported financial value gains andlossesinagovernment’s avoid place. first inthe were derivatives volatility the acquired very tothe yearpaper gains would andlosseseach reintroduce outflows.that Thegovernmentrecording argues impact onannualgovernment resource inflows or haveity andtherefore economic nolong-term would net outto to zero over matur period the Accordingly, remeasurement gains andlosses associated debtmatures. untilthe its derivatives intention andabilityto the hold ithasboth that related to holdingsand itslong-term-debt risks solely to mitigate andinterest-rate foreign currency are derivatives used view isthat ings. Ontario’s unrealized gains hold- andlossesonitsderivative fair-value recognition of remeasurements andthe introductiongovernments, of didnot the endorse remeasurement gains andlosses. must berecorded annually inanew statement of at fairvalue,andany unrealized gains orlosses arements, includingderivatives, to berecorded - financialinstru main requirements certain isthat sector. private standards applicableto the Oneofits 2012 oftheOffice ofthe Annual Report Auditor General of Ontario decisions. Concerns with the introduction of introduction of the with decisions. Concerns sound fiscal-policy support are spent,and that onhowability inreporting taxpayer dollars promotedards that transparency andaccount - public-sector- accountingstan supports Ontario It should be noted that the PSAB iscommit- It shouldbenoted the that inclusionoffair-Therefore, the itsview isthat Canadian other certain alongwith Ontario, MINISTRY RESPONSE MINISTRY

31, 2013, andhasnoted itsConcepts that - principles, would be expensed in the year incurred. year would incurred. inthe principles, beexpensed generally undernormal acceptedthat, accounting costs are typically Such set upasassetsperiods. costs forentities to recovery infuture defer certain may allows charge customers, andoften regulated aregulatedBoard, entity that approves prices the Energy Ontario asthe lished underlegislation,such Under rate-regulated accounting,aregulator- estab generators, tricity transmitters, anddistributors. unique nature ofregulatedaselec- the entitiessuch wereaccounting practices developed to recognize consolidatedRate-regulated financial statements. regulated government’s assets andliabilitiesinthe ofrecognizing rate- appropriateness about the past fourOver years, we the have concerns raised their accountingpoliciesbeing adjusted totheir remove consolidated financial statementsince’s without prov to be included inthe business enterprises, are definedasgovernmentand Hydro One,which near future. PSAB allow standards currently OPG couldbedisallowedaccounting principles, inthe still allowed generally underCanada’s accepted rate-regulatedaccounting by entities,while certain financial statements.rate-regulated Theuseof consolidated government’s are includedinthe and whosefinancialpositionoperating results OPA—thatand the userate-regulated accounting incially owned organizations—OPG, Hydro One Rate-regulated Accounting Rate-regulated ing the financial health of the province. of financialhealth ing the andcredit-rating agenciesors rely whenassess- net debt,akey invest both measure uponwhich - the province’s difficult-to-explain fluctuations in results would,introduce rates. Such inturn, a result ofmovements ininterest andexchange onayear-over-yearvalue ofderivatives basisas the recorded will causesignificantfluctuationsin they that the fact are basedon financial reports fair-value accountingconcepts into government Ontario’s electricity sector has three major sector hasthree electricity prov Ontario’s - - September 2012, Accounting International the time.Most recently,statements untilthat in late province’s consolidated financial inclusion inthe and Hydro Oneto userate-regulated accountingfor accounting standards willcontinueto allow OPG extension, to January September 2012, anadditionalone-year itgranted qualifying rate-regulatedentities with activities.In 2013, IFRSchangeover mandatory to date the for Board aone-year granted extension, to January 2012in March Accounting Canada’s Standards generally IFRS, accepted with accounting principles to reconcile U.S.accounting. Partlyinaneffort continuestoand therefore allow rate-regulated Accounting Standards Board, hasnot adopted IFRS standard-setter United inthe States, Financial the landscape haschanged again. Theaccounting accounting standards. would nolonger beallowed underCanadian accountingfor rate-regulatedmean that balances 1, January 2012,beginning onorafter would which IFRSfor fiscal ments inaccordance with years financial state- shouldprepare their enterprises that allgovernment business Board reaffirmed 2012,In January Accounting Canada’s Standards to Financial Standards. International Reporting were asCanadathat converting such jurisdictions regulated accountingappeared to beendingfor audit opinion). results didnot andtherefore our reported affect did not have province’s impact amaterial onthe consolidated statements.was (This includedinthe results shouldhave beenremoved OPA before the impact ofrate-regulatedso the accountingonits ofbeingagovernment businessenterprise, criteria consolidatedgovernment’s financial statements. ofthe bona fideassets purposes orliabilitiesfor the regulated assets definitionof andliabilities met the ment evenwe though questioned rate- whether PSAB’s position,we accepted accountingtreat- this impact ofrate-regulatedthe accounting.Given the However, rate-regulated time,the sincethat ofrate- era Last year the we that reported However, OPA the doesnot PSAB meet the

1, 2014. Therefore, PSAB

1, Standards Board. As the Ministry isaware,Standards we Ministry Board. do Asthe rate-regulated provisions from Accounting Canada’s meet reference PSAB standards without to any ofthe province’s rate-regulated assets andliabilitiesmight underIFRS. accountingmethod use ofthis ject onrate-regulated accountingto considerthe Standards itspro Board- (IASB)decided to restart accounting policiesmighthave province’s onthe 1, issueto January 2014,of the their effect it is the ual financial statements. However, the deferral with individ- regulation impact ontheir is not ofthis the auditors ofHydro OneorOPG,andsoourconcern future ofrate-regulated accounting.We are not the IASB hasnot yet completed onthe itsdeliberations view, regulations are required these becausethe rate-regulated government’s accounting.Inthe allowaccepted which for accountingprinciples, U.S.statements inaccordance with generally One andOPGto future financial prepare their allowing for andsubsequently Hydro directing both rate-regulated accounting,noting that: government commentedments, the specifically on andConsolidated FinancialAnnual Report State- position.InitsMarch this with not agree and subsequent statements, that is our concern. isourconcern. and subsequent statements, that consolidated financial statement ofMarch approach for rate-regulatedapproach accounting.Given requiredment to achieve consensuson the accounting-standards-setting environin the - ongoingchallenges recognizes the The Ministry Ontario’s Ministry of Finance contends that the ofFinance the contends that Ministry Ontario’s The government recently passedaregulation MINISTRY RESPONSE MINISTRY tions in consumer electricity rates. tions inconsumerelectricity couldresult insignificant fluctua- which yearwere they inthe expensed incurred, costs wouldlated accounting,these be on itsbalancesheet. rate-regu Without - to berecovered infuture, creating anasset are expected costsmay that defer certain Under aregulated practices, entity these Public Accounts oftheProvince

31, 2012,

31, 2014, 41

Chapter 2 Chapter 2 42 Standing Committee onEstimates (Committee) spendingproposals. The ministry’s basis, each tive Assembly outlining,onaprogram-by-program detailed Legisla- Expendituretables Estimates inthe presenting itsbudget, government after Shortly the the Assembly. Officeof mates ofthe between samevote Esti- the items inthe within onanyrequires Ireport transfers ofmoney that addition, section91 Legislative Act Assembly ofthe Treasury year. the Board Ordersissuedduring In required onany to report SpecialWarrants and Under section12 Auditor General ofthe Act , Iam 2012 oftheOffice ofthe Annual Report Auditor General of Ontario EXPENDITURES LEGISLATIVE APPROVAL OF Matters Statutory the future ofrate-regulatedthe accounting. regarding certainty outstanding resolution ofthe review and,ultimately, this ters undertaking to standards-set to- forward the government looks its project onrate-regulated accounting.The Accounting Standards Board decidedto restart issuesareuntil outstanding resolved. province’s rate-regulatedamong the entities also helpedto ensure consistency inreporting statements onaU.S. GAAP basisuntil2014. This rate-regulated financial utilitiesto submittheir have Commissionthat enabled Securities Ontario Administrators andthe Securities Canadian actionsby the consistent both with CanadianGAAP.historical Thisdecisionwas regulated assets andliabilities consistent with entities to continueto rate- accountfor their One andOPGto follow U.S. GAAP to allow the government haddirected issue,the Hydro this standards-setters by deferral to the resolvethe In late September 2012, International the lion inlegislative acts expenditures. Both offices with sufficient temporary appropriations to allow appropriations sufficient temporary with to beVoted Act,2012 TheSupply Appropriations. , are expenditures considered individual program isapproved, Act Supply estimates. Once the the set typically outinthe those programs, ministry canbespentby amountsthat stipulates the which by approving Act, aSupply spending authority Legislaturelegal provides governmentthe with the priation Act, 2011priation . Thesetwo actsreceived Royal Act, 2010 Appro- Interim ; andtheSupplementary 2011-2012 for Appropriation Interim the priations: ments by passingtwo actsallowing appro- interim these payfiscal year,the Legislature authorized to itspassage. prior For 2011/2012authority the require spending programs interim ministry relatedsometimes fiscal even the year—but after fiscal year—and ofthe start the Assent untilafter discussion of the legislative estimatesdiscussion ofthe review pro- votedof two andthen hours on.For amore in-depth mittee Legislatureare fordebated a maximum in the Com- estimates onby reported the ofthe for each Legislature. Ordersfor to Concurrence reported the deemed to Committee bepassedby andare so the are not selected that for ministries review are those Legislature. Theestimates to of the onthem report estimatesreviews andpresents a selected ministry and the authority to provided incurexpenditures authority and the ary, were Act,2012 Supply they repealed underthe , actswas intended interim tounder the betempor on April Act,2012 Supply when the received Royal Assent it to from incurexpenditures April were madeeffective asofApril tures, $3.5 incur upto $114.5 respectively, government to the andauthorized Assent onDecember cess, seesection 2012, received Royal AssentonApril fiscal toyear endedMarch the pertained which The two acts provided government interim the Once the Orders for Concurrence are approved, Ordersfor Concurrence Once the upply ActdoesnotThe Supply typically receive Royal

24, 2012. legal spendingauthority As the

billion ininvestments, and$193.5

3.07 AnnualReport. ofthis

billion in public service expendi billion inpublicservice

8, 2010, andMay

1, 2011.

1, 2011, to

24, 2012.

12, 2011,

mil

- 31, - - - 2006 any power, duty, Treasury orfunctionofthe Act, ofOntario employed PublicService underthe Executiveof the Councilorto any publicservant Treasurythe Board to delegate to any member repealed actwas retained5(4) ofthe andallows Administration ActinDecember2009,subsection Financialwas the repealed andre-enacted within Legislature. inthe been tabled audit have PublicAccounts have beenmadeandthe closed whenany from our finaladjustments arising to be books closed. Thegovernment the considers government fiscal foryear are ofthe the books the year. Theorder may bemadeatany timebefore voted not fiscal fully appropriations spentinthe reduction from other to ofexpenditures beincurred by increase acorresponding isoffset amount ofthe it was made.Theorder may bemadeonly ifthe for which the purpose out insufficient to carry any votedisexpected to be that appropriation amountof toizing expenditures supplementthe allows Treasury the Board to make anorder author Section 1.0.8 Financial Administration ofthe Act year endedMarch government. tenant recommendation ofthe Governor onthe by Orders-in-Councilandapproved Lieu- by the is insufficient.Special are authorized Warrants appropriation the Legislature orforby which the isnoappropriation there forof expenditures which incurring the ance ofSpecialWarrants authorizing Financial Administrationthe Actallows issu- for the Legislature isnot insession,section If the from $193.5 millionto $197 legislativeexpenditures offices total authorized Act,2012 Supply vided underthe increased , which pro- authority was subsumedinto the under them TREASURY BOARD ORDERS SPECIAL WARRANTS Even though the Treasury the Even though Board Act,1991 No SpecialWarrants were fiscal issuedfor the

31, 2012.

million.

1.0.7 of - as Exhibit 3 of this report. as Exhibit3ofthis isincluded andexpended, amountsauthorized the listing of2011/12 Treasury Board Orders,showing nearfuture. Adetailed Gazette inthe The Ontario 2011/12 fiscal expectedyear are to bepublishedin Ordersissuedfor the information. explanatory Gazette, together inTheOntario be printed with lative Assembly, Treasury Board Ordersare to will continueto untilreplaced by beineffect anew Treasurythe Board may specify. Thisdelegation Board, subjectto andrequirements limitations that Board Orders issued for the pastBoard five Ordersissuedfor fiscal the years. controlled contingency fund. centrally government’s amount available underthe by areduction isoffset to the of anappropriation are Treasury amount the Board Ordersinwhich appropriations contingency funds.Supplementary from appropriations and makingsupplementary betweenmaking transfers inprograms ministries Treasury Chairofthe andto Board for the istries, min- make transfers betweentheir within programs to ministers for issuingTreasury Board Ordersto 2012, Treasury the Board delegated itsauthority delegation. For fiscal year endedMarch the Source of data: TreasuryBoard Source ofdata: 2007/08–2011/12 ($million) Figure 6:Total ValueOrders, Board ofTreasury of fiscal year ended March the 12,000 10 14 2,000 4,000 6,000 8,000

, , issue. 000 000 According Legis- Standing to Ordersofthe the Figure Treasury 7summarizes Board Ordersfor totalFigure valueofTreasury the 6summarizes 0

070 080 091 001 2011/12 2010/11 2009/10 2008/09 2007/08 Public Accounts oftheProvince

31, 2012, by month

31, 43

Chapter 2 Chapter 2 44 Estimates. Votemade within 201 2011/12 respect to the with transfer(s) inourAnnualReport. of the ActrequiresAssembly we that make specialmention year related following: to the $432.1 (Thecomparable amountin2010/11off. was and non-government organizations were written $816.4 fiscal the Public year areAccounts. in to be reported anyThe amountsdeleted accountsduring from the subjectofasettlement ordeemeduncollectible. the accounts any Crown are amountsdueto the that anOrder-in-Councilauthorize to delete from the Minister ofFinance,recommendation ofthe may Lieutenant GovernorAct, the inCouncil,on the Under Financial Administration section5ofthe samevote, the section within the Assembly item Officeof to another mates ofthe transfer Esti ofmoney- from oneitemthe ofthe Economy Board ofInternal authorizes When the Treasury Board Source ofdata: Month Relating the2011/12 to FiscalYear Figure 7: by Total ValueOrders Board ofTreasury 2012 oftheOffice ofthe Annual Report Auditor General of Ontario Month ofIssue March 2012 April 2011–February 2012 April 2012 Total 2012August UNCOLLECTIBLE ACCOUNTS ECONOMY BOARD OF INTERNAL TRANSFERS AUTHORIZED BY THE Accordingly, Figure 8shows transfers the In the 2011/12In the fiscal year, receivables of

million due to the Crownmillion dueto the from individuals million.) The writeoffs in the 2011/12 inthe million.) Thewriteoffs fiscal

91 ofthe 121 81 31 # 8 1 Legislative ($million) Authorized 2,544 3,117 190 329 54 Source of data: Board of Internal Economy BoardofInternal Source ofdata: the Assembly, 2011/12 FiscalYear ($) Relating theOfficeof Figure 8:AuthorizedTransfers to required Order-in-Council approval. ments. However, accounts inthe actualwriteoff the consolidated government’s financial state- in the hadalready beenexpensed writeoffs most ofthe against accountsreceivable balances.Accordingly, ince, aprovision for doubtful accountsisrecorded the prov consolidated financial statements of the accounting policiesfollowed preparation of inthe by ministry. writeoffs Under the the summarizes Item 12 Item 10 Item 5 Item 4 To: Item 9 Item 8 Item 6 Item 3 Item 2 From: • • • • • • Volume 2011/12 2ofthe PublicAccounts

1 million for other tax and non-tax receiv andnon-tax 1 millionfor tax other 14.1 millionfor uncollectible receiv tax 55.8 millionfor uncollectiblecorporate ables under the Student Support Program Program Studentables underthe Support ($71.9 ables ($24.7 ($6.4 tax health ($118.8 Program DisabilitySupport Ontario under the ($145.2 ($65.1 $382.2 millionf $29. $48.9 millionf $86.3 millionf $1 $1 Lieutenant Governor’s Suite Lieutenant Governor’s Services OfficeSupport Members’ Services Administrative andTechnologyInformation Services Members’ Compensation Members’ Services Caucus Support Properties andPrecinct atArms Sergeant Legislative Services the Clerk Office of

million in2010/11); million in2010/11);

million in2010/11). or uncollectibleemployer or uncollectiblereceivables or uncollectibleretail sales tax

million in2010/11); and and Travel (1,366,100) 1,476,000 (109,900) - (41,800) (9,200) (9,900) 51,100 2,500 7,300 - - work. results ofpast auditsandrelatedthe follow-up related issues ofpublicsensitivityandsafety, and perceived sig ororganization’s financialimpact,a program’s its as such criteria, basisofvarious agement onthe year werethis seniorman- selected Office’s by the broader publicsectororganizations audited inthe past audityear. VFMauditsandonereview for conducted the inthe andrecommendations conclusions, observations publicisreceiving. the Thischapter contains the level service cost-effectiveness andthe their both ororganization beingauditedprogram to assess delve underlying into ministry the ofthe operations pass compliance issues.Essentially, VFMaudits delivery. auditsalsoencom- Where relevant, such effectiveness ofservice onthe measure andreport procedureswhere appropriate were not inplaceto dueregard forwithout economy andefficiencyor whereon any money was spent casesobserved Office report General requires the Act,which that conducted undersubsection12(2) Auditor ofthe andactivities.Theseauditsare programs their Crown andCrown-controlledmanage corporations broader publicsector,tions inthe agencies ofthe howexamine well governmentorganiza ministries, Our value-for-money (VFM)auditsare intended to Chapter 3 Chapter The ministry programs and activities and the andactivitiesthe programs The ministry nificance t o the Legislative Assembly,o the and Reviews Value-for-money Audits on Reports - by the CanadianInstitute Accountants, by ofChartered the standards for engagements assurance established money work professional the inaccordance with which encompasswhich Annual Report. chapter ofthe assectionsofthis are incorporated the VFMaudit reports are finalized, audit report fundingministry.ment ofthe tor, seniormanage discussionsare- alsoheldwith broader publicsec- caseoforganizations inthe the management responses to ourrecommendations. In andthe report auditee draft tofrom the discussthe senior management meet with senior Office staff isprepared. Then auditreport auditee andadraft year,the significantissuesare discussedwith completednormally audit by ofthat late spring auditfield conclusionofthe At the auditandensure openlinesofcom the auditee of to progress the review the dialogue with anongoing maintain audit,staff the During teria. - audit,includingourauditobjectives andcri of the auditee representativeswith focus to discussthe area to into research beaudited the depth andmeet whenappropriate. experts advice from external weures that considernecessary, includingobtaining tests proced- andother conductingthe They entail We onourvalue-for- andreport plan,perform Once the content VFM andresponses forOnce the each conductin- Before beginning anaudit,ourstaff

valuef or money andcompliance. w ork, which is ork, which munication. 45

Chapter 3 Chapter 3 Cancer Care Ontario Section 3.01 Cancer Screening Programs

2004, these centres were stand-alone organizations Background managed by CCO. On January 1, 2004, the centres formally integrated with their partner hospitals to provide more comprehensive care under Cancer Care Ontario (CCO), formerly known as the one location. Ontario Cancer Treatment and Research Founda-

3.01 In 2011/12, CCO had total expenditures of tion, was created in 1943 under the Ontario Cancer $887 million, $92 million of which was spent on Treatment and Research Foundation Act. In 1997, the

Section cancer screening programs. agency officially changed its name to Cancer Care VFM Ontario and became governed by the Cancer Act. •

3 CCO is responsible for co-ordinating and overseeing cancer services in Ontario. It works with each of Audit Objective and Scope

Chapter the province’s Local Health Integration Networks (LHINs) to address local and regional needs and The objective of our audit was to assess whether advises government on cancer matters. Cancer Care Ontario (CCO) had adequate policies CCO directs health-care funding to hospitals and procedures in place to: and other care providers, with the aim of delivering monitor and assess whether cancer screening quality and timely cancer services throughout the • programs were provided in accordance with province. It is also responsible for implementing legislation, agreements and applicable direc- cancer prevention and screening programs. tives/policies; and CCO has 13 regional cancer programs across the measure and report periodically on achieve- province. These programs bring together health- • ments of cancer-screening-program objectives. care professionals and organizations involved in Our audit objectives and criteria were reviewed cancer prevention and care. Each regional cancer and agreed to by CCO senior management. program is led by a CCO regional vice-president. In conducting our audit, we reviewed relevant The regional programs are required to ensure that legislation and administrative policies and proced- service providers meet the requirements and targets ures, and we interviewed appropriate CCO head- set out in their partnership agreements with CCO. office staff and Ministry of Health and Long-Term Regional Cancer Centres are responsible for Care (Ministry) staff. We also visited sites in the cancer screening and treatment services. Prior to

46 Cancer Screening Programs 47 following regions: Greater Toronto Area, Hamilton ing commitment in 2010, and CCO, through Niagara Haldimand Brant, South West (London), recent initiatives, have clearly recognized Champlain (Ottawa) and North East (Sudbury) the need to increase screening participation to review files and other summary information rates, especially for people considered to be at available for the Ontario Breast Screening Program increased risk for cancer. and the colorectal cancer screening program. We • Participation in breast cancer and cervical also contacted these sites to obtain information on cancer screening achieved ministry targets the cervical cancer screening program. To obtain a but fell short of CCO’s own targets. As of better understanding of and perspective on cancer 2009/10, colorectal cancer screening in screening programs, we spoke to various stakehold- Ontario fell short of both the Ministry’s and ers, such as the Canadian Cancer Society, the Insti- CCO’s targets, and almost half of the targeted tute of Clinical Evaluative Sciences, and the Cancer population remained unscreened. In total, Quality Council of Ontario. We also contacted between 2008 and 2010 only 27% of eligible similar cancer agencies in Manitoba, Alberta and women completed all three cancer screening British Columbia. tests recommended for their age group. As Our audit included a review of related activities well, participation in the screening programs of CCO’s Internal Audit Department. We reviewed appears to have reached a plateau, and CCO is its recent reports and, when we planned our work, looking at ways to address this. considered its audit work and any relevant issues it • There were wait times for screening services had identified. for all three types of cancer: 3.01 • In visits to regional offices, we found mammography screening wait times for women Section with an average risk for breast cancer but

Summary VFM no symptoms ranged from just over two •

3 weeks to 10½ months. CCO found that, in Similarly to agencies in other jurisdictions such as its program that targets women considered

Australia, New Zealand, the United Kingdom and at high risk for breast cancer, the wait time Chapter other Canadian provinces, CCO has implemented for genetic assessments of screening eligi- cancer screening programs for breast, colorectal bility averaged 84 days. and cervical cancers. The key objective for each of • For colorectal screening, almost 30% of the three cancer screening programs is to reduce cases did not have a follow-up colonoscopy the number of deaths from cancer through early within the benchmark time established by detection and treatment. The mortality rates from CCO. The data showed that, in 2011/12, these three types of cancer have fallen in Ontario the median wait times for a colonoscopy over the past two decades. In this regard, Ontario’s were 12 weeks for individuals with family mortality rates are similar to the Canadian averages histories of colon cancer and six weeks for for these types of cancer. those needing to be followed up after a Our major observations with respect to these positive Fecal Occult Blood Test (FOBT). three screening programs were as follows: However, we found instances in hospital • We noted that CCO appropriately used rec- records we reviewed where the wait times ognized clinical evidence in deciding what were as long as 72 weeks for individuals types of cancer warranted formal screening with family histories of colon cancer and 17 programs. Both the Ministry of Health and weeks for those whose FOBT was positive. Long-Term Care, through a $45-million fund- 48 2012 Annual Report of the Office of the Auditor General of Ontario

• For cervical cancer screening, a recent that reported normal results. These follow- CCO preliminary review showed that the up reviews are done to see if the cancer was median wait time for a colposcopy (a diag- missed at the previous screening or whether nostic procedure following up on abnormal the cancer actually developed after the cervical Pap test results) for high-grade previous screening (this is referred to as an abnormalities was about three months. interval cancer). In 2009, 225 reviews were • Even though they are at greater risk of completed, of which 81 warranted further dying of cervical cancer, older women were investigation. Of these 81 cases, about half screened at a much lower rate than younger were subsequently classified as missed-at- women. CCO has said that physicians too screening. No interval cancer reviews were often link Pap testing for cervical cancer to done from the end of 2009 to July 2011, annual health exams, contraception counsel- because CCO’s Ontario Breast Screening ling and screening for sexually transmitted Program radiologist-in-chief had retired and infections. Because older women often have a new radiologist-in-chief was not hired until fewer contraceptive and lifestyle reasons July 2011. At the time of our audit, a backlog to see their doctor, they often do not get of almost 900 interval cancer cases needed tested. Meanwhile, many low-risk younger to be reviewed. CCO informed us that it women were being screened more often expected to complete its follow-up review of than necessary. these cases by December 2012. • The level of quality assurance measures for • CCO measured and reported on its achieve- 3.01 each of the screening programs varied con- ment of cancer-screening-program objectives, siderably. CCO had developed a comprehen- including annually publishing its program Section sive quality assurance program to assess and indicators for its three screening programs VFM monitor the breast cancer screening program, through the Cancer Quality Council of •

3 but 20% of screenings took place outside Ontario’s Cancer System Quality Index and CCO’s program and therefore were not subject periodically issuing performance evaluations

Chapter to the requirements. CCO had set up some of its three cancer screening programs. quality assurance processes for the colorectal cancer screening program, but none for the OVERALL CCO RESPONSE cervical cancer screening program. Cancer Care Ontario (CCO) welcomes the rec- CCO did not analyze and monitor whether • ommendations in this audit. The audit acknow- individual endoscopists (specialists who look ledges the sound processes CCO has in place inside a body cavity or organ using an endo- to assess whether a cancer screening program scope) met performance requirements. For is needed and recognizes the Ontario-wide instance, endoscopists are required to perform commitment to establish high-quality, evidence- at least 200 colonoscopies annually to achieve based screening programs for breast, cervical or maintain competency. From data for the and colorectal cancer. years 2008/09 to 2010/11, we found that Screening is most effective when offered more than 20% of endoscopists had not met through a high-quality organized program this competency requirement. that promotes participation; identifies and There was a significant backlog for follow-up • follows the target population through the reviews of mammography images in cases in screening journey; sends eligible people invita- which a woman was diagnosed with cancer tions, results letters and recalls to screening; after having had a breast cancer screening test Cancer Screening Programs 49

incorporates follow-up processes for those across Ontario and detected more than 22,000 with abnormal test results; ensures access to cancers, the majority in the early stages. In high-quality diagnostic services; and includes March 2011, the OBSP was expanded to include program evaluation and reporting. women aged 30–69 years who are at high risk The audit identifies areas that CCO is already for breast cancer due to genetic factors, medical working to address, reinforcing the value of its or family history. current work and its future directions. CCO will As for colorectal cancer screening, the 2012 continue to work closely with the Ontario Min- Cancer System Quality Index (CSQI) reported istry of Health and Long-Term Care to ensure that in 2010, just over half of Ontarians aged that Ontarians have access to high-quality 50–74 were up-to-date with Fecal Occult Blood cancer screening services. Test, flexible sigmoidoscopy and/or colonoscopy. The Ministry and CCO are committed to OVERALL MINISTRY RESPONSE further increasing participation in colorectal cancer screening, including evaluating new The Ministry acknowledges the recommenda- screening technology for use in Ontario. tions contained in the Auditor General of Ontario’s cancer screening report and thanks him for conducting this timely audit. The Ministry is committed to the development and implementation of innovative initiatives and Detailed Audit Observations 3.01 solutions that address the impact of cancer and other chronic diseases on Ontarians. We wel- Section

STRATEGIC INITIATIVES come any insights and recommendations from VFM the Auditor General that may help to further Over the years, Cancer Care Ontario (CCO) has •

3 inform our ongoing planning and implementa- developed a number of strategic initiatives to tion of cancer screening programs and services. reduce the incidence and mortality from cancers. In

Breast, cervical and colorectal cancer screen- 2003, CCO in collaboration with the Canadian Can- Chapter ing saves lives when these cancers are detected cer Society issued Cancer 2020, Targeting Cancer: in early stages. The Integrated Cancer Screening An Action Plan for Cancer Prevention and Detection, (ICS) program, delivered by the Ministry in a report that provided a long-term provincial plan partnership with CCO, is integrating existing for reducing the number of people diagnosed with, breast, cervical and colorectal cancer screening and dying from, cancer by 2020. In addition to this programs and services into one co-ordinated long-term plan, CCO developed and released three provincial program to support the public, pro- separate three-year Ontario Cancer Plans to provide viders and health-system planners in improving a more detailed road map for cancer care. the quality and uptake of screening. In 2010, the Ministry of Health and Long-Term CCO reports that between 1990 and 2007, Care (Ministry) initiated an Integrated Cancer breast cancer mortality rates declined by 35% Screening (ICS) strategy. The Ministry committed for women aged 50–69 and by 29% for all ages. $45 million to CCO from the 2010/11 through Since the program was launched in 1990, the 2012/13 fiscal years to implement the strategy, Ontario Breast Screening Program (OBSP) has which aims to, among other things, increase patient provided more than 4.1 million screens to more participation in screening, make primary-care than 1.2 million women aged 50 and older providers aware of their role in the process, expand information systems to better identify eligible and 50 2012 Annual Report of the Office of the Auditor General of Ontario

high-risk people, and better monitor and report on A review of the mortality rates from 1992 to the screening process. 2012 showed a decrease in the rates for each of the CCO has implemented three cancer screening three types of cancers over the period. However, programs—the Ontario Breast Screening Program, the rates of decline have slowed in recent years, as ColonCancerCheck and a cervical cancer screen- shown in Figure 1. According to Canadian Cancer ing program. Our review showed that CCO had Society data, Ontario’s mortality rates for these sound processes in place to assess whether a cancer three cancer types are comparable to those of most screening program should be established and Canadian provinces, with British Columbia, Alberta that its decisions were based on clinical evidence and New Brunswick having slightly lower rates. that demonstrated that screening was effective in CCO told us that its efforts under the Integrated reducing mortality. According to a national body’s Cancer Screening strategy to improve screening expert panel, Canadian Partnership Against Cancer participation rates and the quality of screening ser- (CPAC), a “reduction in cancer mortality is the vices would help accelerate the reduction of cancer definitive requirement to confirm that the screening mortality rates. test is effective.” We noted that the criteria used to determine Participation in and Access to Cancer which types of cancer warranted screening pro- Screening Programs grams were in accordance with principles established by the World Health Organization (WHO). Screening programs are effective if they reach a These criteria included, for example, an assessment sizeable percentage of the target population. Over 3.01 of whether the condition is an important health the years, many different target participation rates problem, whether a suitable screening test or have been established to guide cancer screening. Section examination exists, and whether the overall benefit The Cancer 2020 report in 2003 set participation VFM of the screening program outweighs potential harm rate targets of 90% to 95%, but CCO subsequently •

3 from its application. For instance, in the case of established what it deemed were more realistic prostate cancer, the balance of evidence has sug- target rates in its three-year Ontario Cancer Plans.

Chapter gested that the harm of implementing an organized Then, the Ministry, seeing that participation rates prostate cancer screening program may outweigh for all three programs had been levelling off, the benefits. CCO has indicated that it is continuing worked with CCO to develop an Integrated Cancer to monitor the current research work in this and Screening (ICS) strategy to improve participation other types of cancer, such as lung cancers. in screening for breast, cervical and colorectal cancers. The strategy included targets for gradually increasing participation rates from 2009/10 to CANCER SCREENING PROGRAMS 2013/14. For example, the breast cancer screening There is substantial evidence to demonstrate that target increases from 66% in 2009/10 to 73% in early screening and detection is critical in helping 2013/14. to reduce deaths from cancer. Early detection of Participation target rates apply only to people cancers can lead to less invasive treatments and deemed to be eligible for the screening. Eligibility improved health outcomes. criteria are based on such things as an age range or As agencies in other jurisdictions such as a person’s other risk factors in developing a particu- Australia, New Zealand, the United Kingdom and lar cancer, such as family history. At the time of our other Canadian provinces have done, CCO has audit, the most recent CCO participation rate data implemented cancer screening programs for breast, available was for 2009/10, as shown in Figure 2. colorectal and cervical cancers. Cancer Screening Programs 51

Figure 1: Mortality Rates for Breast, Colorectal and Cervical Cancer in Ontario (per 100,000 Ontarians) Source of data: Cancer Care Ontario 35

15 Breast* Colorectal 10 Cervical*

5 3.01

0 a bbb b Section

VFM

•

* Breas t and cervical cancer mortality rates are per 100,000 women 3 in Ontario. Figure 2: Target and Actual Participation Rates in a. Cervical cancer mortality rates for 2008 to 2012 are not available. b. Breast and colorectal cancer mortality rates for 2009 to 2012 Cancer Screening Programs (%) are estimated. Source of data: Cancer Care Ontario Chapter

To promote public awareness of the cancer CCO Plan ICS Target Actual Cancer Target Participation Participation screening programs, the Ministry spent a total of Screening Participation Rate for Rate for $13 million from 2007/08 to 2011/12 for com- Program Rates 2009/10 2009/10 munication and promotion; CCO spent $5.9 mil- Breast 70 66 66.8 lion over the same period to promote the screening Colorectal 40 32 27. 4 programs to various health-care providers and Cervical 85 72 72.0 stakeholders. The Ministry and CCO were jointly responsible for the promotion of the cancer screening pro- Breast Cancer Screening grams. An overall framework was developed to ensure a consistent and focused message, with the The Ontario Breast Screening Program (OBSP) was Ministry leading public communications and CCO implemented in 1990, and the eligible population leading provider education. The Ministry and CCO was women aged 50 to 69 years (changed to 50 to worked together on certain specific initiatives. 74 years in November 2011) with average risk of Our specific observations with respect to partici- developing breast cancer. Average risk means there pation in and access to each of the cancer screening is no risk factor other than being a woman and programs are as follows. being older. 52 2012 Annual Report of the Office of the Auditor General of Ontario

In July 2011, Ontario became the only jurisdic- To access genetic assessment services, a woman tion in Canada to integrate screening for women at must be referred to the OBSP by her doctor. Women high risk for breast cancer into an organized breast confirmed as being at high risk for breast cancer cancer screening program. The eligible population will be booked for both a breast MRI and mammog- is women aged 30 to 69 who are at high risk of raphy. A May 2011 announcement indicated that developing breast cancer. High-risk factors include screening these women with annual breast MRI having a specific genetic mutation, a family history and mammography will detect approximately 17 that suggests hereditary breast cancer, a 25% or cancers per year in every 1,000 women screened. greater lifetime risk confirmed through genetic Program data for the first nine months of the assessment, and having had radiation therapy to program, from July 1, 2011, to March 31, 2012, the chest before age 30 or more than eight years showed that about 5,000 women aged 29 to 69 ago as treatment for another cancer or condition. were referred to the OBSP high-risk program. The average-risk program has tended to meet its Of this total, about 600 women were screened. participation targets, while the high-risk program This number is well below the 20,000 that the was well below its target, although it should be Ministry expected for the program. CCO stated acknowledged that the program is relatively new. that the low number could be because the program Breast cancer screenings and assessments are started in the summer, or because women first provided by more than 150 OBSP affiliate sites in had to get a referral from their family doctor to hospitals and independent health facilities. About the program and many of them then had to have a 523,000 women were screened through the OBSP genetic assessment. Of the $4.7 million allocated 3.01 in 2011/12. (Women can be screened outside of for high-risk screenings and related services, CCO this program. In the 2010/11 fiscal year, the most identified $3.3 million that needed to be returned Section recent year for which figures are available, about to the Ministry after the end of the fiscal year. CCO VFM 112,800 women received mammogram services told us it has lowered its projection for 2012/13 to •

3 outside of the OBSP in independent health facili- 5,000 screens. ties.) About $78 million was allocated to OBSP for

Chapter breast cancer screenings. Wait Times for Breast Cancer Screening Services For women with average risk of breast cancer, Each OBSP affiliate site or regional office manages guidelines recommend cancer screening with mam- its own mammography screening bookings. The mography every two years. For women at high risk, OBSP does not have a standard wait time for such the guidelines recommend annual screening with services. At the three regional offices we visited, we magnetic resonance imaging (MRI) or ultrasound, found variations in the wait times ranging from just as well as mammogram. over two weeks to 10½ months. One reason that In 2009/10, the most recent year for which women in the program wait could be due to the statistics are provided, the participation rate for the high number of screenings and assessments done at eligible population at average risk of breast cancer these sites for women who are not eligible for the was 66.8%, which meets the Integrated Cancer breast screening program. One site we visited did Screening strategy target, but is short of the CCO’s 20,500 mammograms in the 2010/11 fiscal year, target of 70% in its Ontario Cancer Plans. of which only 9,400 were for those women defined In 2011/12, the Ministry allocated about as being eligible under the OBSP criteria. CCO has $11.6 million for screening of high-risk women. data on the provincial totals of non-OBSP screens Of this, approximately $6.5 million was provided conducted, but it did not collect such data on a to CCO, including $4.7 million for an expected site-by-site basis to help it assess the capacity of its 20,000 screening exams and associated costs, and more-than-150 breast cancer screening facilities. $1.8 million for genetic assessments. Cancer Screening Programs 53

In addition, data from 2008 (the most recent CCO is conducting a one-year evaluation to year for which data is available) shows that 56% of identify areas where improvements can be made abnormalities identified in mammograms as requir- to the Ontario Breast Screening Program’s new ing biopsy were followed up within seven weeks. high-risk program. This is to be completed by This is below the national target established by the March 2013. This evaluation will include wait Public Health Agency of Canada of 90% within that times for MRIs and genetic assessments (i.e., time frame. CCO’s comparison with other jurisdic- counselling and testing) for women in the high- tions found that no other province had achieved this risk screening program. Based on the outcomes national target. Public Health Agency of Canada data of the one-year evaluation, CCO will implement from 2005/06 shows that only Nova Scotia did better improvements to the program, with a special than Ontario on this measure, with 58% of instances focus on reducing wait times for MRIs and followed up on time, as compared to 57% in Ontario. breast assessments, where needed, beginning in On our regional visits, we found there were 2013/14. wait times associated with various stages of the screening for those considered at high risk for MINISTRY RESPONSE breast cancer—up to six months at some sites. CCO conducted a survey at the end of 2011 of all genetic The Ministry supports the evaluation of wait assessment clinics that determined eligibility for times for breast screening at each of its screen- high-risk breast cancer screening. The survey found ing sites. The Ministry will work with CCO to that wait times from when a woman received her evaluate and improve wait times for screening 3.01 referral to her first appointment averaged 84 days. and follow-up of abnormal screens at all screen- Most clinics in the survey indicated that the waits ing sites. Section were primarily a result of their not having enough The Ministry has provided CCO with resour- VFM staff to do the genetic assessments more quickly. ces to conduct an evaluation of the Ontario •

3 Breast Screening Program to identify improve- RECOMMENDATION 1 ments, with a special focus on reducing wait

times for MRIs and assessments as appropriate. Chapter To improve breast cancer screening services to eligible participants, especially those considered to be at high risk of breast cancer, Cancer Care Ontario (CCO) should periodically evaluate Colorectal Cancer Screening the wait times at each of its screening facilities. In 2007, the Ministry committed $195 million As well, CCO should take measures to increase for colorectal cancer screening over the five-year its capacity to expedite genetic assessments for period from April 1, 2007, to March 31, 2012. CCO women who have been referred to the high-risk then launched the ColonCancerCheck program program by their doctors. with total committed funding of $72 million for the same five-year period. The eligible population of CCO RESPONSE the program was men and women aged 50 to 74. CCO agrees that a reduction in wait times at all The program chose the guaiac Fecal Occult screening sites is important in order to improve Blood Test (FOBT) kit as the primary colorectal breast cancer screening and assessment servi- cancer screening tool for Ontarians considered to ces. CCO will work with the regions to evaluate be at average risk. This would include those with and improve wait times for screening and no family history of colorectal cancer or symptoms. follow-up of abnormal screens. FOBT testing is recommended every two years. 54 2012 Annual Report of the Office of the Auditor General of Ontario

When an FOBT is positive, the primary-care pro- As well, because the participation rate figure vider should refer the patient for a colonoscopy did not include the direct colonoscopy screening of follow-up. people considered at increased risk for colorectal The program also includes people considered to cancer, the overall participation in colorectal cancer be at increased risk—these are people with one or screening in Ontario was understated. more immediate relatives who have, or have had, In December 2011, a Joint Steering Committee colorectal cancer—but the protocol is different. of the Ministry and CCO approved a method of People at increased risk can bypass the FOBT and calculating the colorectal cancer screening rate that are fast-tracked to have a colonoscopy. If no cancer includes people screened with FOBT, flexible sig- is detected during the colonoscopy, a person is moidoscopy or colonoscopy, and decided that this encouraged to be tested again every five to 10 years. calculation would be reported in addition to the In 2011/12, 63 hospitals signed agreements FOBT participation rate. However, this method of with CCO and received incentive funding of calculation will result in overstating the screening $4.6 million to participate in the ColonCancer- participation rate because it also includes all indi- Check program. The hospitals performed a total of viduals receiving treatment for colorectal cancer 14,300 colonoscopies in that year. or precancerous lesions, and such treatment is not considered to be screening. Under this new calculation method, the colo- Participation in Colorectal Cancer Screening rectal testing rate was reported in 2012 as 53% ColonCancerCheck reports that the participation for 2010. It included everyone who had completed

3.01 rate among the eligible population (men and women an FOBT in the previous two years, or who had aged 50 to 74) in 2009/10 was 27.4%, short of the had a flexible sigmoidoscopy in the previous five

Section ICS target of 32% and CCO’s Ontario Cancer Plan years, or colonoscopy in the previous 10 years, target of 40%. However, the program measured VFM from all sources, including private colonoscopy •

only the rate of participation in the FOBT. In our 3 clinics. Thus, about half of the eligible population visits to regional offices, and to some hospitals in remained unscreened. these regions, we were told that the low rates might Chapter be partly due to the fact that many doctors think the FOBT is not reliable enough as a screening tool, Colonoscopy Wait Times at Hospitals and they were instead referring many average-risk When the colorectal cancer screening program people directly for colonoscopy screening. Indeed, commenced in 2007/08, 57 hospitals participated a Ministry-commissioned survey in 2010 found in the program. This increased to 74 in the 2008/09 that 37% of physicians believed the FOBT was not fiscal year and then dropped to 64 in 2010/11 and reliable enough to be used as a population-based 62 in 2011/12. screening tool. Accordingly, there were likely many The Canadian Association of Gastroenterology more average-risk people being screened than were set a goal of two months to complete a follow- participating in the FOBT. (CCO told us at the time of up procedure for a positive FOBT, a benchmark our audit that it was conducting a pilot project using adapted by CCO as eight weeks. For people with a another stool-based test kit, the Fecal Immunochem- family history of colorectal cancer, the benchmark ical Test, or FIT, which has a higher sensitivity than for the time from referral to colonoscopy was set the approved FOBT kit being used to that point; the by the Association as six months, adapted by CCO different kit may address physicians’ concerns about as 26 weeks. The ColonCancerCheck program using the FOBT to screen for colorectal cancer.) set a provincial target of 75% for the eight-week benchmark and a provincial target of 80% for the Cancer Screening Programs 55

26-week benchmark for those with a family history explore approaches for reducing the wait of colorectal cancer. • times for colonoscopy procedures, especially The Cancer System Quality Index (CSQI), those for increased-risk patients. recently published by the advisory group Cancer Quality Council of Ontario, reported that in 2011 CCO RESPONSE 73% of positive FOBT cases met the eight-week wait-time benchmark, an improvement over 60% CCO agrees with this recommendation and is in 2009. As stated in CCO reports, 80% of family working to increase participation and improve history cases met the 26-week benchmark in the colorectal cancer screening program. It will 2010/11, an improvement over 71% in 2008/09. continue to educate primary-care providers Our review of records at hospitals found that wait about the highest-quality evidence supporting times exceeded the benchmarks of eight weeks for FOBT screening. In addition, it will continue positive FOBT follow-up and 26 weeks for family to evaluate the feasibility of introducing other history cases. Our review identified instances where screening tests that have recently been shown individuals with positive FOBTs waited as long as by highest-quality evidence to reduce colorectal 17 weeks for a follow-up colonoscopy and those cancer mortality—namely, a more sensitive with family history of colon cancer waited as long stool-based test (the Fecal Immunochemical as 72 weeks for a colonoscopy. Test, or FIT) and flexible sigmoidoscopy. Both While there was improvement in the two wait- methods are appropriate for average-risk time rates, almost 30% of participants did not screening and may be more acceptable to 3.01 receive a follow-up colonoscopy within eight weeks primary-care providers, including doctors. of a positive FOBT result and within 26 weeks of CCO will continue to work with the regions Section a referral for increased risk. Some people were to improve wait times for colonoscopy at sites VFM screened at private clinics, but CCO did not have that are not meeting wait-time targets, espe- •

3 access to the referral dates to private clinics to cially for increased-risk patients (those with a assess the wait times of participants. first-degree family history of colorectal cancer),

CCO attributed the shortfall in meeting its through regular quarterly performance reviews Chapter targets to a number of things, including physicians and contract management. who did not follow up with participants, reluctance of some people to have a colonoscopy, and phys- MINISTRY RESPONSE icians who told patients with positive FOBT results The Ministry has provided CCO with funding to to repeat the test when they should have been conduct provider education and a pilot evalua- referred at that point for a colonoscopy. tion of newer Fecal Immunochemical Test technology for use in Ontario. RECOMMENDATION 2 The Ministry has also provided CCO with To increase participation and improve its funding to: colon cancer screening efforts, Cancer Care • expand colonoscopy capacity to reduce wait Ontario should: times for individuals who are at increased • examine and work to address the concerns risk of colorectal cancer; and doctors have with the effectiveness of the • conduct a pilot to leverage non-hospital Fecal Occult Blood Test as a screening colonoscopy clinics to improve capacity for tool; and colonoscopy services, as part of the Colon- CancerCheck program. 56 2012 Annual Report of the Office of the Auditor General of Ontario

Cervical Cancer Screening cervical cancer did not have cervical cancer screening in the three years prior to being diagnosed, The cervical cancer screening program was compared to only one-third of women aged 21 to launched on June 15, 2000. In 2010/11, the Min- 34 years that were not screened. In fact, half of istry paid $54.8 million to doctors and labs for Pap women aged 55 to 74 years were not screened in tests. Screening is primarily performed with Pap the 10 years prior to being diagnosed with invasive tests done by physicians in their offices as part of cervical cancer. routine checkups. When a low-grade abnormality shows up on a Pap test, the woman usually receives a repeat Pap test in six months. When a high-grade Frequency of Cervical Cancer Screening abnormality shows up on a Pap test, the woman is In general, CCO recommends cervical cancer usually referred for a colposcopy—a visual exam- screening every three years for all women aged 21 ination of the cervix using an instrument called a to 69 who are or have ever been sexually active. colposcope. In some cases, tissue is removed in a However, if a woman receives an abnormal test biopsy, and a pathologist makes a diagnosis. Col- result, CCO recommends that she be tested annu- poscopies are performed in hospital-based clinics or ally until she has three successive normal results. in private clinics. The overall provincial participation rate for cer- CCO has recently issued updated guide- vical cancer screening in 2009/10 was 72%, which lines that recommend the use of the Human fell short of the Ontario Cancer Plan target of 85%, Papillomavirus(HPV)-DNA test as a cervical cancer but met the Integrated Cancer Screening target.

3.01 screening tool for women aged 30 years and older. There was a significant difference in participation

The agency is working with the Ministry to explore rates among age groups. CCO’s program evaluation

Section how the test can be incorporated into the screen- reported that the highest rates of screening par-

ing program. ticipation were among women aged 20 to 29 years VFM

•

(74%), and the lowest rates were among women 3 aged 60 to 69 years (66%.) Accordingly, younger Participation Rates of Women with Invasive women, who have a lower risk of cervical cancer,

Chapter Cervical Cancer have the highest rates of annual Pap test screen- Cervical cancer is largely preventable with HPV ing. Our review showed that 16% of women aged immunization, regular screening and appropriate, 20 to 29 who had normal Pap test results in 2009 timely follow-up of abnormal results. Cervical were screened again within 12 months. This only cancer mortality increases steeply from age 45. The occurred in 7% of women aged 70 and older. most recent Ontario Cancer Registry data, from Discussions with CCO management indicated 2009 to 2011, shows that 83% of deaths from cer- that younger women (20 to 34 years) are more vical cancer occurred in women over 45. frequently screened and rescreened because Our review showed that in spite of the fact physicians often link Pap testing to annual health that older women were at increased risk of dying exams, contraception counselling and screening from cervical cancer, they were not appropriately for sexually transmitted infections. The Ministry targeted for screening and were inadequately had identified this as an issue in 1996. Because screened. For instance, between 2009 and 2011, older women often have fewer contraceptive and older women were twice as likely not to have cer- lifestyle reasons to see their doctor, they do not vical cancer screening in the three years prior to get tested often enough, if at all, even though they being diagnosed with invasive cervical cancer than are at a greater risk of developing and dying from younger women. Specifically, two-thirds of women cervical cancer. aged 45 to 74 years diagnosed with invasive Cancer Screening Programs 57

CCO officials said the issue of too much or too aged 20 to 69 did not have a follow-up colposcopy little screening can also be attributed to lack of an within six months of a high-grade abnormal Pap organized system for telling people when they are test. The 2008 colposcopy guidelines specify that due to have the test. For the cervical cancer screen- less severe cytological findings should be followed ing program to have an organized call and recall up with colposcopy within eight to 12 weeks, while system, it must be allowed to collect health infor- more severe findings should be followed up within mation about individuals without their consent. a shorter time. As a minimum standard, time from This requires being approved for Prescribed Regis- referral to colposcopy should not exceed six months. try status by the Ministry and the Information and During our audit, we found that CCO had not Privacy Commissioner. CCO obtained Prescribed sent out any correspondence to the affected individ- Registry status for the Ontario Cancer Screening uals, but CCO informed us that now that it has Pre- Registry in May 2011. The Information and Privacy scribed Registry status, it planned, as of fall 2012, to Commissioner approved CCO’s information prac- send result letters to all women aged 21 to 69 years tices and procedures in respect of the Registry in with abnormal or unsatisfactory Pap test results. October 2011. As a result, the program can now col- In February 2012, CCO did a preliminary review lect data to identify eligible women and send them of colposcopy data for the years 2008 to 2010 to directly all appropriate correspondence about test determine the median time for receiving a col- results and to invite or recall them for screening. poscopy. This preliminary review showed that the According to CCO, it would be a better practice median wait time for high-grade abnormalities was to encourage doctors to view the Pap test as a generally about three months. 3.01 separate service, and not tie it to appointments for contraception counselling, annual health exams RECOMMENDATION 3 Section and testing for sexually transmitted infections. To improve the effectiveness of its cervical VFM CCO recommended that the Ministry, through its •

cancer screening services, Cancer Care 3 negotiations with the Ontario Medical Association, Ontario should: align its incentive payments for physicians with target promotional and educational efforts

its updated cervical cancer screening guidelines, • Chapter to increase participation and rescreening which were released to the public and health-care rates among older women; providers in May 2012. The current physician educate the public and health-care providers incentive bonus has been based on the percentage • on appropriate cervical cancer screening of the target population who received a Pap test intervals; and in the 30 months prior to March 31 of the fiscal monitor wait times for colposcopy proced- year for which the bonus is being claimed. This is a • ures for timely follow-up of women with six-month shorter interval than the 36 months that abnormal Pap test results. the CCO recommends between Pap tests, and may encourage over-screening. In addition, there were CCO RESPONSE no financial disincentives to screening women more frequently than at the recommended intervals. CCO agrees with this recommendation and is actively working to strengthen the cervical cancer screening program. Providers who Wait Times for Colposcopy Services perform cervical screening are key to ensuring For screening to be effective, timely follow-up of that screening and follow-up are done according abnormal Pap test results is critical. The 2012 Can- to guidelines. In May 2012, CCO disseminated cer System Quality Index noted that 17% of women its new cervical cancer screening guidelines to 58 2012 Annual Report of the Office of the Auditor General of Ontario

face challenges relating to low incomes, immigrant providers across Ontario, along with information backgrounds, functional difficulties or sexual they can distribute to women. orientation. CCO indicated that “current research In addition, efforts to increase awareness suggests that local, customized interventions best of cervical cancer screening guidelines among address the specific barriers to screening experi- providers and women of all ages will continue enced by this group.” As a result, CCO received a through targeted media outreach and corres- commitment from the Ministry of $4.5 million for pondence campaigns beginning in July 2012. 2010/11 to 2012/13 to develop initiatives directed This includes the development of awareness to the under- or never-screened population. campaigns with regional partners to promote In 2010/11, CCO selected five of its 13 regions appropriate screening for under-/never- for projects for under- and never-screened people, screened populations and older women. and it is providing funding to these five regions CCO will work with hospitals and colposcopy for two years, after which time it will evaluate the clinics to monitor and improve colposcopy wait initiatives to see if they should be expanded. CCO times, where needed, and will work with primary- informed us that there are plans to perform more care providers to improve follow-up rates for all detailed analysis on a LHIN level to expand on the women with abnormal Pap test results. current regional projects in the 2013/14 fiscal year. In addition, under the Integrated Cancer Screen- MINISTRY RESPONSE ing strategy, CCO has various proposals or pilot The Ministry will continue to work with and projects to help improve participation in screening 3.01 fund CCO to identify effective strategies for programs among people who do not have family delivery of promotion and education aimed at physicians. Such initiatives include sending colo- Section increasing knowledge and awareness among the rectal cancer screening kits directly to a sample VFM public and health-care providers on appropriate of eligible people, providing screening invitations •

3 cervical cancer screening and clinically recom- and reminders to eligible people, and developing mended guidelines. community-based education/recruitment to engage

Chapter under-screened populations.

RECOMMENDATION 4 Cancer Screening for People with No Family Physicians The Ministry of Health and Long-Term Care should monitor and assess current Cancer Care According to the Ministry, about 6% of the adult Ontario initiatives designed to improve partici- population does not have a family physician. Cur- pation in screening programs among people rently, when anyone without a primary-health-care who do not have family physicians to gauge provider participates in any of the three cancer their effectiveness. screening programs, CCO is responsible for ensuring that abnormal test results are followed up. CCO MINISTRY RESPONSE then encourages these people to contact Health Care Connect to find a physician for ongoing pri- Improving participation in screening programs is mary care. However, this process does not actively a key objective of the Integrated Cancer Screen- seek out people who do not have family physicians ing program. The Ministry provides CCO with and get them into screening programs. the appropriate resources, mandate and public CCO has recognized that people who never, support to carry out its initiatives and activities. or rarely, participate in screening programs often Cancer Screening Programs 59

grams live up to established minimum standards The Ministry and CCO are finalizing the cur- and that a process is in place to assess their reliabil- rent Transfer Payment Accountability Agreement, ity on an ongoing basis. Patients and their doctors which includes appropriate mechanisms for mon- need to be able to trust the results. Our review of itoring, oversight and reporting of CCO’s activities the quality assurance programs in place for each against clearly defined objectives and targets. of the screening programs found significant varia- In addition, the Ministry’s Health Care Con- tions. CCO had developed a comprehensive quality nect program continues to connect unattached assurance program for monitoring the breast cancer people with primary-care providers. Since its screening program, but it had established only lim- inception, Health Care Connect has matched ited monitoring for the colorectal cancer screening and referred more than 100,000 people. program, and none for the cervical cancer screening program. As well, our research indicated that CCO RESPONSE other jurisdictions, such as the United Kingdom, CCO agrees with this recommendation have well-defined quality assurance processes that and will work with the Ministry to ensure may warrant consideration here in Ontario. appropriate monitoring of CCO’s activities to improve participation among people who do Breast Cancer Screening not have primary-care providers. CCO and the Ministry are working to determine how to CCO had developed a comprehensive quality assur- increase screening access for all eligible people, ance program for monitoring the Ontario Breast 3.01 including those without providers, such as by Screening Program (OBSP). allowing self-referral for screening, having other Sites must meet specific minimum requirements Section health-care professionals provide screening before they can participate in the OBSP. Among VFM tests, mailing FOBT kits directly, and providing other things, they must receive and maintain •

3 screening in mobile settings. CCO and the accreditation from the Canadian Association of Ministry are collaborating to make necessary Radiologists Mammography Accreditation Program

regulatory changes to expand access to screening (CAR-MAP). The accreditation covers radiologist Chapter through these channels, such as by permitting and medical radiation technologist qualifications, laboratories to process screening tests dispensed equipment, quality control, quality assurance, by non-physicians. image quality and radiation dose. Increasing screening participation for people However, in 2009/10, 20% of breast cancer without primary-care providers must be coupled screenings were done outside of the OBSP and were with ensuring timely follow-up for those who performed at non-OBSP sites that were not subject have abnormal test results. CCO is working to the monitoring requirements. In discussions at with the Ministry to implement an organized regional offices we visited, we were told that the follow-up model for all those who have been use of non-OBSP sites could be because doctors screened, particularly people without primary- were referring women to sites that were close by care providers. or even in the same building as the doctor’s office, or because doctors and patients are not necessarily aware that there is a difference between OBSP and non-OBSP services. As a consequence, these MONITORING FOR QUALITY OF SERVICES women did not have access to CCO’s follow-up of Monitoring the quality of cancer screening pro- abnormal test results, reminders or recalls for the grams is important to help ensure that the pro- next appointment, or to the CCO’s quality assurance 60 2012 Annual Report of the Office of the Auditor General of Ontario

processes as did women screened at OBSP sites, radiologist-in-chief was not hired until July 2011. although their results were still read by a radiologist At the time of our audit, a backlog of almost 900 and sent to their doctor for follow-up. interval cancer cases needed to be reviewed. CCO We were informed by CCO during the audit informed us that it expected to complete its follow- that the College of Physicians and Surgeons of up review of these cases by December 2012. Ontario (College) would be requiring CAR-MAP CCO arranges for independent inspection of accreditation for non-OBSP independent health mammography machines every six months to make facilities by early 2013. The College has since sure their radiation levels are within the acceptable included a requirement for CAR-MAP accreditation range. We reviewed a sample of inspection reports by 2014 in the updated Clinical Practice Parameters at three regions we visited. One region had not and Facility Standards for all independent health received all the reports from its sites on how they facilities. Hospitals that provide mammography followed up on any issues that arose in the inspec- services are not required to have CAR-MAP tions. Some of these reports were due in August, accreditation, but CCO has recommended that the September or October of 2011. At another region, all Ministry require it. The Ministry has agreed that issues that were reported had been addressed, and CCO should strike an expert panel and work with at the third, no significant issues were identified. CAR-MAP and the College to further develop this Regional offices are required to conduct audits recommendation. Subsequent to our audit, we were of patient records (chart audits) to ensure that informed that the Ministry was reviewing options data entered in the screening system is accurate, in to ensure that all sites providing mammography accordance with OBSP standards and policies, and 3.01 services are CAR-MAP–accredited by 2014. consistent with data entered in the provincial infor- In addition to requiring accreditation, the OBSP mation system. We found that the chart audit policy Section has established several quality assurance processes. did not specify the sample sizes to be reviewed, the VFM These include regular reviews of the work of the frequency of the reviews, and, when concerns are •

3 radiologists who assess the screens, inspections of identified, what the subsequent review frequency mammography machines every six months, reviews must be. At the regions we visited, we found signifi-

Chapter of the work of medical radiation technologists, and cant variations: chart audits to ensure information on participants • Only three of the four regions we visited is complete and up to date. conducted chart audits. The region that did For quality assurance purposes, the OBSP also not perform chart audits chose to conduct conducts what are called “interval cancer reviews” sample reviews for only cases with abnormal of cases in which a woman has been diagnosed screen results. with cancer after having had a previous screen- • For the three regions that conducted chart ing test that reported normal results. This is to audits, the frequency and types of reviews determine if the cancer was missed at the previous varied. One region conducted chart audits screening or whether the cancer developed subse- every three years with less extensive chart quent to the screening, which provides feedback to reviews between the audit years. The other OBSP radiologists. two regions conducted annual chart audits. In 2009, 225 such cases had been reviewed and • The three regions each reviewed a sample of 81 were further investigated. Of these 81 cases, 20 files for each site they audited, regardless of 42 were subsequently classified as missed-at- the number of yearly screens performed at the screening. No interval cancer reviews were done site. Therefore, a site that performed 10,000 from the end of 2009 to July 2011, because CCO’s screens annually had the same sample size as a OBSP radiologist-in-chief had retired and a new site that performed 500 screens annually. Cancer Screening Programs 61

Colorectal Cancer Screening Quality of Hospital Services CCO uses colonoscopy as a primary screening tool CCO and the Ministry maintain some monitoring for people considered at increased risk of colon can- mechanisms for colon cancer screening. We noted cer, and as a follow-up test to positive FOBTs. Only the following observations. colonoscopies performed at participating hospitals are eligible for incentive funding from the colo- Quality of Laboratory Services rectal cancer screening program. In 2011/12, these In the ColonCancerCheck program, the completed hospitals performed about 14,300 colonoscopies FOBT kits are analyzed at participating community and received incentive funding of $4.6 million. Par- laboratories for the presence of blood in the sam- ticipating hospitals must meet specific quality stan- ples. Test results are then sent to the participant’s dards outlined by CCO, including the following: physician as well as to CCO. The physician informs • endoscopists must perform at least 200 colon- the participant of his or her screening result. CCO oscopies annually to achieve or maintain also notifies the participant of his or her result. If competency; a test result is positive, a colonoscopy is usually • the hospital’s rate of bowel perforation must recommended as a follow-up test. be no higher than one in 2,000 for screening Labs that want to participate in ColonCancer- procedures and one in 1,000 for all proced- Check must sign an agreement with the Ministry ures; and that outlines specific requirements, such as that • the examination of the bowel must meet a the lab must be accredited by the Quality Manage- particular standard so that the thoroughness 3.01 ment Program Laboratory Services of the Ontario of the procedure can be assessed. Medical Association, and that it must conform to We found that CCO collected the necessary data, Section the Canadian External Quality Assurance Program but it did not analyze and monitor the data with VFM established by the Ontario Association of Medical respect to whether individual endoscopists com- •

3 Laboratories (Association). Under an agreement plied with the requirements. Specifically, we found between the Association and the Ministry, total the following:

Ministry-approved funding is up to $45 million • From colonoscopy data from 2008/09 to Chapter over five years. A committee of the Association, 2010/11, we asked CCO to identify the comprising members from all labs that are partici- percentage of endoscopists in the colorectal pating in ColonCancerCheck, monitors the quality cancer screening program who did not do the of laboratory performance through monthly profi- minimum 200 colonoscopies annually. The ciency testing. review showed that more than 20% of endos- Not only are there no Ministry or CCO represent- copists did not meet the requirement. atives on this quality committee, but neither the • Our review of perforation data from 2009/10 Ministry nor CCO received reports on the quality to the third quarter of 2011/12 for three assurance process and related results. The agree- regions we visited showed that the participat- ment between the Ministry and the Association ing hospitals generally met the perforation states that the Ministry will only be informed of rate standards. However, the rates were concerns that are not satisfactorily resolved by the determined based only on perforations that appropriate laboratory, and must then be referred occurred on the procedure date. Our discus- to the Ministry for action. The Association told us sion with an expert in the field, as well as a that there have been no incidents that needed to be review of research articles, indicated that referred to the Ministry since the quality assurance complications may arise up to 14 days after program began in 2008. the procedure. CCO said it did not track 62 2012 Annual Report of the Office of the Auditor General of Ontario

this information beyond the procedure date and proficiency testing of labs by the Quality Man- because it was very challenging to do so. agement Program—Laboratory Services (QMP-LS), Other than the professional requirements of the operated by the Ontario Medical Association. In specialists who conduct colonoscopy procedures, as addition, the Ministry licenses laboratories to per- well as the hospitals’ requirements, there is no com- form a defined set of tests. prehensive quality assurance process. CCO told us CCO is responsible for quality assurance for the that it is considering developing a quality assurance cervical cancer screening program. A quality assur- process that is similar to that of the breast cancer ance program would include test quality standards; screening program. the collection of data and monitoring of compliance to those standards; performance indicators; and the development of laboratory-related targets. However, Quality of Services at Private Clinics CCO had not yet developed a quality assurance pro- There are approximately 50 private clinics in gram for cervical cancer screening. CCO indicated Ontario that offer colonoscopy services but are not that it did not have the authority to collect all of the eligible to receive incentive funding for colonoscop- data required for performance management and ies. Until 2010/11, these private clinics were not reporting by providers until it received Prescribed subject to any specific program quality standards Registry status in October 2011. and data collection requirements, as hospitals We also reviewed how quickly cytology test- are under the colon cancer screening program. ing should be completed. From 2007 to 2010, the In 2010/11, however, the College of Physicians provincial median turn-around time decreased to

3.01 and Surgeons of Ontario began monitoring and 15 days from 21. However, turn-around time varied inspecting these clinics. CCO is conducting a pilot widely among laboratories. For instance, in the

Section project to explore how to incorporate these clinics 2010 fiscal year, the median turn-around time at into the screening program. VFM individual labs ranged from seven to 33 days. •

Cervical Cancer Screening Quality of Colposcopy Services Chapter As mentioned earlier, CCO received Prescribed Colposcopy is performed to investigate cervical Registry status for the Ontario Cancer Screening abnormalities, such as pre-cancerous lesions. Registry in May 2011. The Information and Privacy In 2010/11, the Ministry reported that 125,400 Commissioner approved CCO’s information prac- colposcopy procedures were performed, a 28% tices and procedures in respect of the Registry in increase from the 98,000 completed in 2004/05. October 2011. Receipt of Prescribed Registry status Colposcopies are done in hospital-based clinics or enables the establishment of a comprehensive qual- in physicians’ offices. ity assurance program. Prior to receiving this Pre- In 2008, CCO established colposcopy guidelines, scribed Registry status, CCO was not able to obtain including guidelines on qualifications and train- enough data to establish such a program. We made ing for those who perform the procedure, and on the following observations. quality assurance measures. However, CCO has not assessed and monitored the quality of colposcopy services to make sure they are provided in accord- Quality of Laboratory Services ance with these guidelines. For instance, we found Cytology samples are generally obtained through the following: Pap tests done in doctors’ offices and are then According to the guidelines, those performing analyzed by laboratories. CCO relies on the quality • colposcopy should complete approximately assurance processes that govern the accreditation Cancer Screening Programs 63

100 colposcopies per year to maintain their CCO RESPONSE competency level, with at least 25% of cases being new patients. As discussed earlier, CCO CCO strongly agrees that screening is most did not have the authority to collect all of the effective when offered through an organized data required for performance management program that incorporates all service providers and reporting by providers until it received and uses robust quality assurance mechan- Prescribed Registry status in October 2011. isms to maximize the benefits of screening • The guidelines call for colposcopy clinics to and minimize the harms. CCO also agrees that undergo annual reviews for quality assurance, other jurisdictions, particularly the United and for clinical audits to be done at regional Kingdom, offer excellent models for quality and provincial levels to ensure consistent assurance programs. Building on models such results and provide appropriate feedback to as these, CCO will establish regular monitoring clinicians. CCO has not implemented a qual- procedures to assess performance against quality assurance program or conducted clinical ity assurance requirements, such as by tracking audits since the program began in 2000. screening frequency, cancer detection rates and After CCO was granted authority to collect competency of providers. CCO will work with and review the data to assess quality of services the Ministry to obtain the data and the man- in October 2011, it also signed a data-sharing date required to ensure that quality assurance agreement with the Ministry in January 2012 to requirements for screening programs are met access the necessary health information. With access by all service providers, regardless of whether 3.01 to the Ministry’s claims payment data, CCO will screening services are provided under programs now be able to gather information on the number of established by CCO. This will include primary- Section colposcopies performed, whether a biopsy was done, care providers, radiologists, colonoscopists, col- VFM the number of physicians performing colposcopies poscopists, colonoscopy sites, mammography •

3 and the physicians’ specialties. However, CCO still sites and laboratories. will not have data on the results of colposcopies and

biopsies from all sources, including hospitals, clinics MINISTRY RESPONSE Chapter and other facilities. The Ministry will continue to work with and provide CCO with the appropriate resources, man- RECOMMENDATION 5 date and support to carry out effective quality To ensure that Ontarians are receiving quality assurance and monitoring of service providers. cancer screening services, Cancer Care Ontario Under the Laboratory and Specimen Collec- should work with the Ministry to: tion Centre Licensing Act, all medical laborator- • establish monitoring procedures to ensure ies must, as a condition of licensing, meet that quality assurance requirements are met the requirements of the quality management for screening of breast, colorectal and cervical program carried out by the Ontario Medical cancers, regardless of whether they are pro- Association (Quality Management Program— vided under programs established by Cancer Laboratory Services, or QMP-LS). The quality Care Ontario or other service providers; and management program carried out by the QMP- • obtain screening data so it can review and LS is for all laboratory testing and includes assess the work performed by all service colorectal and cervical cancer screening tests. providers and measure the results against appropriate quality assurance standards. 64 2012 Annual Report of the Office of the Auditor General of Ontario

PERFORMANCE MEASURES public reports on its assessments. The Ontario AND REPORTING Cervical Screening Program Report, issued in 2011, covered 2003 to 2008; Ontario Breast Screening Public Reporting of Performance Indicators Program, 20th Anniversary Report, issued in 2010, CCO publishes annually its program indicators for covered 1990 to 2010; and ColonCancerCheck 2008 its three screening programs through the Cancer Program Report was issued in 2010. Quality Council of Ontario’s Cancer System Quality The three cancer screening programs adopted Index, a web-based public reporting tool that tracks some of the performance indicators that were the quality and consistency of all key cancer servi- developed by a national body, either the Public ces in the province, from prevention and screening Health Agency of Canada or Canadian Partner- through to end-of-life care. The Council makes rec- ship Against Cancer. The indicators used for the ommendations for improvements to cancer services breast cancer screening program were in line with to the Ministry, via CCO’s board of directors. In May these key national indicators. However, both the 2012, the Council said CCO needed to continue its colorectal and cervical cancer screening programs efforts to improve the participation in its screening lacked indicators to assess the programs’ follow-up programs. For instance, for 2008 to 2010, only 27% and detection activities and outcomes. Under the of eligible women completed all the cancer screen- Integrated Cancer Screening strategy, CCO is working tests recommended for their age. ing with the Ministry to set up and report on 13 key CCO has also conducted formal evaluations of performance measures for all three cancer screen- the three cancer screening programs and issued ing programs. 3.01

Section

VFM

•

Chapter Chapter 3 Ministry of the Attorney General Section 3.02 Criminal Prosecutions

The Division’s Crown attorneys also represent Background the Crown in criminal appeals; provide legal advice to the police, the Attorney General and other law enforcement officials; provide special services such The Criminal Law Division (Division) of the Min- as applications to a court for electronic-surveillance istry of the Attorney General (Ministry) prosecutes

authorizations, extraditions and search warrants; 3.02 criminal charges on behalf of the Crown before and develop criminal law policy recommenda- provincial courts in Ontario. It consists of Crown

tions for both provincial and federal applications. Section Attorneys, Deputy Crown Attorneys and Assistant In addition, the Division participates with other VFM Crown Attorneys, who are appointed under the •

stakeholders in major initiatives targeting criminal 3 Crown Attorneys Act (Act) and Crown Counsel, who activity related to guns and gangs, as well as the are appointed under the Ministry of the Attorney Ministry’s Justice on Target initiative to reduce the

General Act (collectively referred to as Crown Chapter average number of court appearances and days attorneys or prosecutors). The Act outlines the dut- needed to dispose of a criminal charge. ies of Crown attorneys in prosecuting charges laid The Division receives about 600,000 new crim- by police forces, such as summoning witnesses to inal charges each year from more than 60 police attend court, providing disclosure to defence attor- forces in Ontario. A Crown attorney is to prosecute neys, presenting evidence in court and dealing with a criminal charge only if there is a reasonable pros- bail applications of accused offenders. pect of conviction and if it is in the public interest It is the role of the Ontario Provincial Police and to prosecute. If at any stage of the case changed municipal police forces to lay criminal charges, circumstances make the prospect of conviction under the federal Criminal Code of Canada and no longer reasonable, the Crown attorney is duty the Youth Criminal Justice Act and other provincial bound to discontinue the prosecution. statutes, for such crimes as assault, uttering threats, Criminal charges are prosecuted in either the impaired driving, robbery and homicide. Certain Ontario Court of Justice or the Superior Court charges fall under the jurisdiction of federal pros- of Justice. The vast majority of charges are dealt ecutors, and Ontario Crown attorneys generally do with by the Ontario Court of Justice, which typ- not deal with crimes involving drugs, terrorism, tax ically tries less serious offences presided over by a law and money laundering. judge alone; trials for more serious Criminal Code

65 66 2012 Annual Report of the Office of the Auditor General of Ontario

offences take place in the Superior Court of Justice three other provinces’ prosecutorial services. We and are heard either by a judge alone or by a judge also engaged an independent expert who has senior and jury. management experience in delivering criminal The Division operates from its head office in prosecution programs. Toronto, six regional offices and 54 Crown attorney We considered recommendations we made in offices across the province. Operating expenses our previous audits of the Ministry, including Legal totalled $256 million in the 2011/2012 fiscal year, Aid Ontario (2011), the Court Services Division 84% of which was spent on staffing. The Division (2008), and our last audit of the Division (1993). employs approximately 1,500 staff, including about We also considered several major public reviews of 950 Crown attorneys and 550 support and admin- the criminal justice system over the last decade. istrative staff. In addition, the Division spends The Division’s internal auditor conducted approximately $3.2 million annually on contract several reviews that were helpful in our audit, lawyers who work on a part-time per diem basis. including those about travel card use and employee expenses; controls over the administration of proceeds of crime; and the Division’s project to implement a new electronic Crown Management Audit Objective and Scope Information System (CMIS).

Our audit objective was to assess whether the Division had established adequate policies, sys- 3.02 tems and procedures for the timely and efficient Summary prosecution of criminal matters on behalf of Section the Crown, and for measuring and reporting on The number of Crown attorneys and the overall VFM program effectiveness. staffing costs for the Criminal Law Division (Div- •

3 Senior management reviewed and agreed to our ision) have more than doubled since our last audit audit objective and associated audit criteria. in 1993. Yet the number of criminal charges that

Chapter We conducted our fieldwork at the Division’s Crown attorneys dispose of per year has not sub- head office in Toronto and visited five of the six stantially changed—572,000 in 1992, compared to regional offices and 11 of the 54 Crown attorney 576,000 in 2011. Partly as a result of the Charter offices. Our work included interviewing staff, of Rights and Freedoms, many cases are more including prosecutors assigned to the Guns and complex than they used to be, so that more time Gangs initiative; reviewing recent reports and stud- and more court appearances are needed to pros- ies; and examining policies, records, case files and ecute them. Also, additional Crown attorneys have systems. We also met with staff from the Ministry’s been assigned to deal with certain crimes, such as Justice on Target initiative. those involving gangs and other dangerous and We also held interviews with representatives of high-risk offenders. five police forces in the province, Legal Aid Ontario, However, it is difficult to gauge the actual the Criminal Lawyers’ Association, and the Ontario impact of this on prosecutor workload, especially Victim Services Secretariat to discuss their per- because the Division makes little use of numerical spectives on prosecutions and the criminal justice and statistical information to analyze the relative system in Ontario. workload, efficiency and effectiveness of its Crown We researched criminal prosecution programs attorneys, and relies more on informal oversight in other Canadian and foreign jurisdictions and by senior staff at each of the 54 Crown attorney met with senior management of the federal and offices. When we last audited the Division in 1993, Criminal Prosecutions 67 we noted “a systemic emphasis on prosecutorial benchmark for what a reasonable workload discretion,” and that monitoring was done by for each Crown attorney should be. Work- “more subjective means, such as informal feedback loads per Crown attorney varied significantly and personal knowledge about the individuals among local offices and between regions. involved.” This observation remains valid today. For example, at two similarly sized Crown We continue to believe the Division would bene- attorney offices, the average workload during fit from having information systems that would the 12-month period ending March 31, 2012, provide it with reliable summary data on prosecu- was 572 charges per attorney at one and 1,726 tor workloads, the outcome of prosecutions, the charges per attorney at the other. The Division average time taken to resolve charges and other key does no periodic analysis to assess: performance indicators, both at a local office and • the reasons for the significant decrease over an individual Crown attorney level. The Division the last two decades in the average number can also make better use of the information on of charges a Crown attorney disposes of per court activities currently available from within the year; or Ministry until it completes the development of its • whether Crown attorneys need to be own information systems. reassigned among Crown attorney offices Our other major observations were as follows: to balance workloads and ensure similar • The Division does not formally assess its pros- charges can be handled consistently regard- ecutorial performance—for example, it does less of where in Ontario they are laid. not gather information on how efficiently • Of the Division’s six regions, the Toronto 3.02 charges are screened; how long it takes to Region disposed of the most charges in prepare cases; whether court diversion pro- total, but it did so at the highest cost per Section grams for resolving minor criminal charges charge—$437, compared to the average of the VFM are used appropriately; the number of bail other regions of $268. The Toronto Region •

3 release applications, and what their condi- also disposed of an average of about 40% tions and results are; and what the outcomes fewer charges per Crown attorney than the

of cases are. Furthermore, the rates at which average of other regions. We also noted that Chapter certain Crown attorney offices went to trial the use of court diversion programs for per- were up to 20 times higher than the rates of sons accused of minor criminal charges varied other offices, significantly increasing justice widely between Crown attorney offices—for system costs. We noted that Statistics Canada example, one office reported that it resolved reported that Ontario had Canada’s highest 11% of its eligible charges using diversion pro- rate of adult criminal charges withdrawn or grams while a similarly sized office resolved stayed (suspended by a court) in 2010/11 75%. Reasons for these significant differences (43% for Ontario versus 26% for the rest of had not been analyzed. Canada) and the lowest rate of guilty verdicts • The Division does not have a systematic (56% for Ontario versus 69% for the rest of process in place to ensure that services at its Canada)—but the Division does not have the 54 Crown attorney offices are consistently information needed to determine the reasons meeting minimum professional and Division for this or whether this relates more to certain standards. In our review of case files at 11 regions or Crown attorney offices. Crown attorney offices we noted no standards • No staffing model has been established for recording decisions and events, forms to determine how many Crown attorneys were either missing or not used, and case files should be at each local office, and there is no were missing. 68 2012 Annual Report of the Office of the Auditor General of Ontario

A much-needed electronic case management • resulted in significant reductions in guilty system originally projected to cost $7.9 million pleas and significant changes in their timing, and to be completed by March 2010 has been which translate to more time being spent on a significantly delayed because of weak man- file. A typical impaired-driving case illustrates agement, oversight and financial reporting, the point. Today one case takes two days in and insufficient resources being dedicated to court, whereas in 1992, two to four cases were the project. Other provinces, such as Mani- prosecuted in one day. In part because penalties toba, already have such systems in place and have increased, such charges are vigorously we noted that, rather than develop a new defended and defence applications to exclude system, Alberta recently paid $1 for the rights key evidence that the defence alleges the police to use and further develop Manitoba’s system. obtained in violation of the accused’s rights Because the Division does not measure its • under the Canadian Charter of Rights and Free- performance, the Ministry makes no mention doms are common. Nevertheless, the Division of the Division in its annual reporting. In this agrees with the Auditor General that it needs respect, the Ministry differs from some other to find a way of measuring the impact that jurisdictions, which do measure and report on increased complexity has on workload. their criminal prosecution operations. We agree that information collection and We did note that the Division has contributed to analysis are essential decision-making tools. We some recent progress in improving court efficiency, recognize that a multi-faceted electronic case as reported by the Justice on Target initiative,

3.02 management system is key, not only to informa- including the reversal of a decades-long trend of an tion collection and analysis, but also to moving increasing number of appearances and days needed

Section some of the paper-based manual processes into to complete a criminal case in court. an electronic approach. We are disappointed VFM

•

with the progress made to date on our project 3 OVERALL MINISTRY RESPONSE to implement such a system. Nevertheless, our The Criminal Law Division is committed to commitment to implement such a system, or Chapter continuing to provide the citizens of Ontario group of systems, remains steadfast, and we are with the highest-quality, efficient and effective taking steps to get back on track. prosecution services in support of public safety. In the meantime, the Division is taking The Division finds great value in and is actively action to ensure the appropriate and necessary incorporating the observations and recom- measurement of our workforce and workload mendations of the Auditor General’s review as through information already available to us. The it continues to improve how it delivers prosecu- Division will identify the gaps in meaningful tion services in these changing times. data collection and will research similar metrics The audit correctly reports that cases are and systems that are being used to measure more complex than 20 years ago. In addition resourcing in the other jurisdictions referred to to the Charter of Rights and Freedoms, other in the report. factors add to the complexity of cases, resulting Through the actions that we are taking to in an increased demand on Crown attorneys’ increase our effectiveness and to continuously time and hence an increase in the number of improve, we will deliver on the commitment to Crown attorneys. For instance, the introduc- public safety for the people of Ontario. tion of mandatory minimum sentences has Criminal Prosecutions 69

of information as a means to gauge the relative Detailed Audit Observations workload and effectiveness of the Division’s Crown attorneys. Specifically, the Division lacks information systems at many levels to provide information MANAGING OPERATIONS to allow it to assess workloads and effectiveness. Since our last audit in 1993, the number We understand that there are reasons why dis- of criminal charges disposed of is virtually posing of criminal charges consumes more resources unchanged—572,000 in 1992 compared to 576,000 now than it did in the past. For example, additional in 2011. Yet the number of Crown attorneys and Crown attorneys have been assigned to focus on support staff, as well as staffing costs—even after domestic and sexual violence, guns and gangs, taking inflation into account—have doubled over dangerous and long-term offenders, and Internet the same period, as shown in Figure 1. In contrast child exploitation. Changes in legislation and case to some other provinces we visited, management law that have occurred since 1982 as a result of the oversight tends to be more informal in nature, so Charter of Rights and Freedoms have increased as not to be perceived to affect the independence the complexity of many cases. The average time of each Crown attorney. As well, less use is made and number of appearances required to dispose of

Figure 1: Annual Percentage Change in Number of Crown Attorneys, Charges Disposed and Criminal Law Division Expenditures, 1992–2011 Source of data: Ministry of the Attorney General, Public Accounts 3.02 140 Section

120 VFM

Crown attorneys •

Criminal Law Division expenditures* 3

100 Charges disposed Chapter

1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011

–20

–40 * Annual percentage change for expenditures were based on amounts adjusted for inflation to 2011 levels using Statistics Canada’s Consumer Price Index. 70 2012 Annual Report of the Office of the Auditor General of Ontario

a criminal charge have both doubled since our last of this information to the Division is limited for the audit, and these increases have been steadily climb- following reasons: ing over the last 20 years except for the most recent • ICON reports information on a criminal year. Mandatory minimum sentences, disclosure charge basis rather than by case or person. requirements, and more complex evidence, such as Since most cases involve multiple charges DNA evidence, cell phone activities and computer and not all the charges proceed, manage- forensics, have also contributed to an increase in the ment has not been able to determine whether demand on Crown attorneys’ time. whole cases have been lost, or whether minor Nevertheless, given a doubling of staff relative charges were simply withdrawn but the case to essentially no caseload increase, it is all the proceeded on more serious charges. more critical to objectively assess the efficiency • Although ICON reporting does separate the and effectiveness of the Division’s resource number of charges that are stayed by the management. This requires timely, relevant and court (proceedings against an accused are accurate information and analysis. Specifically, stopped before an acquittal or conviction) the Division needs information on how efficiently from the number of charges withdrawn by charges are screened; how long it takes to prepare prosecutors, it does not identify the reasons cases; whether court diversion programs are used for either occurrence; knowing this could help appropriately for minor charges; the number of bail the Division reduce their frequency. Stays and release applications, and what their conditions and withdrawals can occur for many reasons. For results are; and what the outcomes of cases are. example, the accused may have been denied 3.02 This information can help management assign pros- complete disclosure, a right to counsel or a ecutors to local Crown attorney offices to balance timely trial; evidence may have been deemed Section workloads across the province and monitor trends inadmissible; and witnesses may have refused VFM in charge resolution to identify situations that cause to testify. In some of these cases, preventing •

3 inefficiencies and delays. However, the Division has the stay and withdrawal is in the control of neither a manual nor a computerized system for the prosecutor.

Chapter collecting the needed information. It also has not Each Crown attorney office’s management established benchmarks against which it can assess responsibilities are assigned to a senior Crown aspects of its performance. attorney, who we noted also continues to carry Information that is available to the Division has his or her own caseload. Senior Crown attorneys come primarily from outside sources, most notably told us that they rely extensively on feedback from the Ministry’s Integrated Court Offences Network prosecutorial staff about their own workloads. (ICON), which reports on charges processed by the They also rely on anecdotal information, such as Ontario Court of Justice for each courthouse. Most comments from the judiciary, defence attorneys Crown attorney offices serve only one courthouse, and court staff, to make them aware of concerns making the ICON reports useful for assessing cer- with prosecutors’ performance. The management tain aspects of each office’s performance. However, framework tries to strike a balance between pros- even though this information is distributed monthly ecutors having independence in the day-to-day to regional and local Crown attorney management, decision-making on cases assigned to them and the we found no indication that the Division routinely need for Division management to hold prosecutors used the information to analyze the performance accountable for efficient and effective prosecutions of its regional and Crown attorney offices and that meet expectations and standards. individual Crown attorneys. As well, the usefulness We noted that local Crown attorney offices have developed their own management and operational Criminal Prosecutions 71 practices. In some cases, this is because local police MINISTRY RESPONSE forces or courts are administered slightly differently from one area to another or because the 54 local The Criminal Law Division recognizes the Crown attorney offices vary greatly in size—the importance of having reliable management smallest have only one or two Crown attorneys, information system(s) to make informed deci- while the downtown Toronto office has more than sions in support of the effective leadership of 125. In smaller Crown attorney offices, one person its operations. might handle all incoming cases from beginning The Division is reviewing existing systems to end, whereas in larger offices it is common for and the information available within the Min- several prosecutors to work on a case at different istry that relates to its work with a view to iden- stages and make court appearances. However, there tifying the gaps in current information analysis, is no overriding provincial management model, and reporting and report usage. The Division’s head office had not done a formal analysis on the longer-term objective is to ensure that future variation in practices among offices that could be information systems accurately capture and used to identify best practices in the various Crown support the analysis required to enable meas- attorney offices where it might be beneficial to urement of the cost-effectiveness and optimal standardize practices to reduce costs. For example, use of its resources, including measurement and we found no consistency in how case files were assessment of workload. handled by Crown attorneys, including ownership and custody assigned to files between police forces 3.02 and Crown attorney offices, standards for docu- OVERSIGHT OF PROSECUTORS mentation on case files, and notation standards for Section key decisions, such as screening charges or bail and Management of cases and their timely progression VFM sentencing recommendations. through the justice system has been a particularly •

3 crucial issue since October of 1990, when the RECOMMENDATION 1 Supreme Court of Canada released its ruling in the

case of R. v. Askov. In general, at the time, the Askov Chapter To ensure that decisions on the use of legal ruling and related rulings established that the and support staff resources and results of acceptable time to trial was generally eight to 10 prosecutions are supported by timely, relevant months. As a result, thousands of backlogged char- and accurate information, the Criminal Law ges across the country were dismissed on grounds Division of the Ministry of the Attorney General of unreasonable delay in the prosecutions. Since should identify what information is needed and then, there have been further court rulings that develop systems as soon as possible to deliver have narrowed the circumstances under which a this information to its regional and local Crown- judge can dismiss a charge on the basis of unreason- attorney-office management. The Ministry able delay caused by prosecutors. The Ministry has should also use this information to hold the Div- been examining its processes to resolve charges at ision accountable for demonstrating the cost- the earliest opportunity to reduce both costs and effective use of its resources. Until such time as the risk of delays. Its Justice on Target initiative is the Division can gather its own information on one of these strategies. its activities, it should make better use of the Justice on Target aimed to achieve, over a four- available ministry information on courthouse year period ending in June 2012, a 30% reduction activities to more effectively oversee operations in the number of court appearances and days to and report on its use of resources. dispose of a charge. Justice on Target reported that 72 2012 Annual Report of the Office of the Auditor General of Ontario

as of March 31, 2012, it had achieved success in ments was either incomplete or not used by reversing a decades-long trend of increases, but it the Division. was not close to meeting these reduction targets. We were surprised that the Division does not Court appearances and the number of days to formally track the number of motions made by dispose of a charge had decreased by 7% and 2%, defence attorneys to the court requesting that respectively. The Division is a key stakeholder in the charges be stayed due to delay, or the reasons for initiative, contributing to the reductions achieved the successful motions. Although the ICON system by helping to implement opportunities for charges identifies pending charges and the extent of the to be resolved earlier. delays, there is not enough data in ICON to analyze The management staff at Crown attorney offices what types of charges make up the backlog and the told us that they informally monitor charges that reasons for the delays. As a result, it is up to the reach the eight-month, 10-month and 12-month Division to gather information on the reasons for marks out of concern that charges will be stayed stays. Another example of an area where inconsis- due to the delay. As Figure 2 indicates, the backlog tencies should be probed by the Division to under- of charges in the courts taking longer than eight, 10 stand and reduce delays is the setting of trial dates. or 12 months still exists and has not changed sig- While courts control the progression of cases, we nificantly, although there has been some improve- noted at several Crown attorney offices we visited ment in the most recent fiscal year. that some courts had rules for setting trial dates The Division does not track whether specific within, say, 90 days, while other courts had no actions taken by Crown attorneys have any effect such rules. Prosecutors also have a significant role 3.02 on the progress of a case. For instance, Crown to play in ensuring that cases progress through the attorneys we interviewed told us that prosecutors court in a timely manner by bringing any unneces- Section initiate only a small percentage of court adjourn- sary delays to the attention of the court for action. VFM ments—one of the leading causes of delays—but A senior Crown attorney also has the opportunity •

3 there is little data to support this given that the to bring up more systematic causes for delays at a Division does not track the delays caused by particular courthouse at regular meetings that are

Chapter adjournments and the reasons for them. As well, held with the judiciary on the administration of information in ICON on the causes of adjourn- the courthouse. Statistics Canada reports that in 2010/11, Figure 2: Ontario Court of Justice Percentage of Ontario had the lowest proportion of guilty ver- Criminal Charges Pending, 2007/08–2011/12 dicts in adult criminal cases among all Canadian Source of data: Ministry of the Attorney General jurisdictions, at 56%. The national figure, exclud- 40 ing Ontario, was 69%. The Division has not analyzed its prosecution results to determine why this might be the case. 30 The Division has said it is aiming to reduce the number of trials that collapse—usually through 20 last-minute guilty pleas or the withdrawal of charges—on the day the trial is set to begin. Trials Over 8 months that collapse on the first scheduled court date incur 10 Over 10 months Over 12 months costs that could have been avoided because they unnecessarily tie up courtrooms, court staff, the 0 judiciary, witnesses and police, all arranged months 2007/08 2008/09 2009/102010/11 2011/12 in advance. Trial collapse rates vary widely among Criminal Prosecutions 73

Crown attorney offices, from 4% to 22%. The A stay of proceedings occurs when charges Division obtains trial collapse rates from ICON, but are temporarily or permanently suspended by ICON does not provide enough detail to allow the the court, such as when the rights of an accused causes of the collapses to be analyzed or compared. person have been violated or the Crown attorney For instance, ICON captures only collapsed charges, requests that the accused person participate in a not collapsed cases. Minor charges are regularly diversion program; in contrast, withdrawn charges withdrawn at trial, particularly when multiple are initiated by the Crown attorney when there charges have been laid, so statistics on collapsed are no reasonable prospects of conviction or it is charges do not isolate the real problem, which is not in the public interest to prosecute. Statistics the collapse of entire cases. Canada, which receives data from the Ministry’s In addition, the Division did not analyze why cer- Court Services Division, reports that 43% of adult tain regions and Crown attorney offices had higher criminal cases in Ontario in 2010/11 were resolved trial rates—local offices ranged from about 1% to by staying or withdrawing charges laid, the highest 20% of total annual charges, and the region with the proportion of such cases in Canada. The average highest rate was Toronto. The cost implications of of other provincial and territorial jurisdictions in a Crown attorney office proceeding to trial at a rate Canada was 26%. There could be many reasons for of 20 times more than another office warrants more withdrawing charges, including the quality of the formal attention, particularly since the Toronto Crown briefs that police send to Crown attorneys Region has the largest total caseload in the province. for charge screening, plea negotiations, witnesses not co-operating and the success of diversion 3.02 programs. Such a significant difference could Disclosure and Screening of Charges also indicate that Ontario as a whole or certain Section After police lay charges, they provide a report Crown attorney offices are incurring unnecessary VFM on those charges to the Crown attorney’s office costs because weaker charges are not being suf- •

3 in what is known as a Crown brief. Police forces ficiently screened out before court proceedings want to move to an electronic disclosure system to begin. However, the Division does not collect data

improve efficiency, but the Division has been slow so that charge withdrawal rates can be analyzed Chapter to implement its system for accepting Crown briefs to determine if there are any systemic issues that electronically. At the time of our audit, only five of warrant attention. 54 Crown attorney offices were accepting electronic disclosure from police forces. Diversion Programs After receiving a Crown brief from police, Crown attorneys assess, or screen, the charge(s) to deter- The Ministry’s voluntary diversion program, called mine whether to prosecute—essentially, whether the Direct Accountability Program, is an alterna- there is a reasonable prospect of conviction and, if tive to formal prosecution for people who have so, whether the prosecution is in the public interest. been charged with minor criminal offences. The During this screening, Crown attorneys can decide program benefits the accused, the courts and the not to prosecute charges by withdrawing some or community. The program involves accused people all of the charges or otherwise resolving charges being held accountable through community-based without going to trial by recommending to the sanctions such as restitution, community service court the use of diversion programs or alternative work, charitable donations or participation in pro- sentencing options. Alternatively, additional char- gramming such as anger management or alcohol ges could be laid by the police on recommendation and drug awareness. The charge(s) against the of the Crown attorney. accused can be withdrawn or stayed if the person 74 2012 Annual Report of the Office of the Auditor General of Ontario

successfully completes the sanctions imposed. across the province for dealing with minor offences Diversion strategies—used for charges that Crown and the resulting criminal records of those who are attorneys deem eligible, such as theft-of-property not diverted. offences—are less costly than formal court proceedings and they allow for quicker resolution of RECOMMENDATION 2 charges, which frees up court resources. In order for the Criminal Law Division to Diversion programs were in place in almost adequately oversee its prosecutions, monitor all Crown attorney offices. However, as Figure 3 its costs and assess its performance, it should illustrates, referral rates by Crown attorneys as regularly analyze the trends, rates and reasons reported by the Justice on Target initiative differed for stays and withdrawals, adjournments, trial widely among the six regions. For example, the rates, bail release violations, guilty pleas and Toronto Region had a referral rate of 57% and the guilty verdicts, and use of diversion programs. Eastern Region 35%. The referral rates varied even In addition, the Division should compare its more significantly among Crown attorney offices. performance to other provinces and, where For example, in the 2011/12 fiscal year, two Crown Ontario’s overall trends differ from those of attorney offices in the same region that are roughly other large provinces, determine the reasons for the same size had referral rates of 11% versus 75%. such differences. The Division’s process for tracking and analyzing its use of the programs and the results of the referrals MINISTRY RESPONSE did not sufficiently address the varying referral 3.02 rates or improve their consistency. As a result, the The Criminal Law Division recognizes the Ministry cannot determine whether the diversion importance of using information to evaluate the Section programs are being appropriately used to the quality and results of its prosecution service. VFM extent possible. In fact, the varying use of diversion Analyzing and seeking to understand the •

3 programs by Crown attorney offices and Crown trends, rates and reasons for outcomes in crim- attorneys may result in an inconsistent approach inal proceedings will be helpful in overseeing

Chapter our prosecutions. Further analysis and research Figure 3: Referral Rates to the Direct Accountability into how this will assist with the monitoring Program (Diversion) by Region, of costs and performance of the Crown in con- May 1, 2011–April 30, 2012 (%) ducting prosecutions is required. The Division Source of data: Ministry of the Attorney General recognizes the need for consistent practices and approaches within its regions and local offices, 70 taking into consideration the variables that 60 exist, such as size of office, geographical/demographic information, size of police force and 50 charge volume. There are lessons to be learned 40 from each office, with perhaps the most valu-

30 able coming from the comparison of like offices. The Division will also benefit from the experi- 20 ence of other large prosecution services across 10 Canada and will seek opportunities to make meaningful multi-jurisdictional comparisons. 0 Central Central EastNorth TorontoWest East West Criminal Prosecutions 75

MANAGING WORKLOADS Figure 4: Average Number of Charges Disposed of per As we noted earlier, the number of Crown attorneys Crown Attorney by Region, 2011/12 has doubled since 1992 despite the fact that the Source of data: Ministry of the Attorney General number of criminal charges has remained essen- 1,000 tially constant. The Division is unable to assess 900 whether the increases in the number of Crown 800 attorneys is reasonable because it does not gather 700 the necessary information to analyze whether the 600 complexity of the cases or the time needed to dis- 500 pose of the charges has increased to such an extent 400 as to require twice as many staff. The Division does 300 not use a staffing model to determine the appropri- 200 ate number of Crown attorneys, and there is no 100 benchmark for what a reasonable workload for 0 Central Central EastNorth TorontoWest each Crown attorney should be. East West Using data from the Ministry’s ICON system, we calculated the average number of charges disposed attorneys be hired for initiatives such as efforts of per Crown attorney and found a wide variance to address guns and gangs, high-risk offenders among Crown attorney offices. For example, at half and domestic violence, but these plans have not

of the 54 Crown attorney offices, the rate of charges included any expected outcomes or workload 3.02 disposed varied by more than 25% of the overall measures. As a result, it is difficult for the Division

average of 700 charges. Furthermore, the average to demonstrate what payback has resulted from the Section workload per Crown attorney at two similarly sized incremental costs that were incurred or why the VFM

•

Crown attorney offices differed significantly for additional staff resources were necessary, especially 3 the 12-month period ending March 31, 2012—572 since it has no measures in place to assess the work- charges per attorney at one office versus 1,726 load of its existing prosecutors. charges per attorney at the other. There were also We noted that both Manitoba and the Public Chapter two regions that disposed of significantly fewer Prosecution Service of Canada (federal prosecu- charges per Crown attorney, compared to the other tors) track each prosecutor’s workload. In addition, four regions, as shown in Figure 4. British Columbia and Manitoba assign each case to One of the key reasons for ensuring that Crown a specific prosecutor, which allows their manage- attorney workloads are reasonably comparable is ment to monitor and assess workloads; this is not that they should be able to devote a similar amount the usual practice in Ontario. These other provinces of time to charges of a similar nature and complex- and federal prosecutors also use electronic case- ity, regardless of where in Ontario the charge is laid. management systems to track workloads. Crown attorney offices also had different ways In addition, we calculated the average cost the of organizing staff and assigning cases to Crown Division incurred to prosecute charges and noted attorneys. For example, one office we visited significant differences between regions, as indicated had case management co-ordinators to carry out in Figure 5. For instance, the Toronto Region had administrative tasks for incoming charges, while the highest average cost at $437 per charge, versus in another office Crown attorneys performed the average of other regions of $268 per charge. As these functions. Figure 4 already illustrated, the Toronto Region also Over the past few years, the Division has drawn disposed of an average of about 40% fewer charges up business plans requesting that additional Crown per Crown attorney than the average of other 76 2012 Annual Report of the Office of the Auditor General of Ontario

Figure 5: Average Cost per Charge Disposed of by • collect and analyze information on work- Region, 2011/12 ($) loads and cost variances between regions Source of data: Ministry of the Attorney General and Crown attorney offices to identify opportunities to use resources as efficiently 450 Average per region as possible and address inconsistencies; and 400 Overall Ontario average, excluding Toronto Region ensure that management has the ability 350 • and flexibility to address temporary and 300 permanent workload pressures by, for 250 example, relocating prosecutors and sup- 200 port staff between Crown attorney offices, 150 and using contract lawyers where and 100 when appropriate. 50 0 MINISTRY RESPONSE Central Central East NorthToronto West East West The Criminal Law Division recognizes the need and the increasing operational requirement to regions—478 compared to 811. The Division had develop measures of workload. not done a formal analysis to determine the cause of Defining workload is challenging and must these variances. go beyond the number of cases per Crown attor-

3.02 Our discussions with the Division’s senior man- ney. There are a number of factors that impact agement also revealed that the Division’s collective workload. For instance, a significant murder

Section agreement with its prosecutors might be interpreted trial could take two Crown attorneys three in such a way as to restrict the Division’s ability to VFM years, equating to a fraction of a case per year. •

address workload pressures, particularly in its abil- 3 Also, in a small office, two prosecutors might ity to relocate prosecutors between Crown attorney carry a workload of 1,500 charges per year and offices. In addition, the collective agreement limits

Chapter have to fulfill all obligations on these charges the Division’s use of contract lawyers to address from screening and vetting right through to workload pressures to a maximum of 30 days in any disposition. They also might need to travel three quarter of a year. This could also account for manage- hours to deal with some of the charges. It is ment’s reluctance to deal with some of the differ- challenging to determine the relative workload ences in costs and workloads per office. We did note weight of these two examples. during our fieldwork that the Division had completed The Division will research other jurisdictions’ an assessment of its head office support services to attempts to measure workload and will develop Crown attorney offices and as a result had been able an approach that meets its needs. Using avail- to reduce its head office staff by 11 positions. able data, the Division will then review workload and develop processes for making comparisons. RECOMMENDATION 3 The Division will also use the information to To ensure that Crown attorneys have the work- analyze the differences between regions and load flexibility to devote a similar amount of local offices to determine how resources can time to charges of a similar nature, the Criminal be used as effectively as possible. This will also Law Division should: promote staff wellness in the Division by provid- • establish benchmarks for what a reasonable ing additional information on which workload workload for each Crown attorney should be; distribution decisions can be made. Criminal Prosecutions 77

QUALITY ASSURANCE justice, child abuse, impaired driving and matters under the Youth Criminal Justice Act. The Division’s management style provides regular Our review of case files at the 11 Crown attorney feedback and support from senior Crown attorneys offices we visited showed a number of inconsistent to prosecutors through ongoing collaboration. practices. For instance, ownership and custody However, the Division does not periodically review of closed case files was handled inconsistently, a a sample of the work done by each of its Crown number of files we requested could not be located, attorneys, particularly its more than 750 assistant and standardized forms, such as for charge screen- Crown attorneys, to assess whether they generally ing, were either missing or not used by the Crown meet expectations and professional and divisional attorney office. There were also no standards standards. We were advised that the criminal for recording decisions and events, which were justice system itself acts as an external quality therefore inconsistently recorded, both in different control measure, in that the work of prosecutors offices and within the same office. is reviewed by police, judges and defence counsel In addition, Crown attorney offices have their during each case they prosecute. However, senior own management-oversight processes, supple- Crown attorneys who oversee other prosecutors mented by employee performance evaluations done generally get information on the quality of the work semi-annually by the local senior Crown attorney. done in court by their staff prosecutors only when However, these evaluations did not include a they are told by court staff, or, periodically, when review of any recent files the prosecutor had com- prosecutors observe colleagues’ court proceedings pleted. Including a spot check of a sample of files

when they happen to be in the same courtroom. 3.02 as part of the employee evaluation process would Without such periodic spot checks, divisional also communicate the importance of documenting

management faces a challenge in its bid to ensure Section compliance with Division standards. that high-quality prosecutorial services are con- VFM We did note certain circumstances where case • sistently delivered across the province. Periodic 3 files were reviewed by senior Crown attorneys, reviews would help to identify whether case files but these processes were ad hoc and were not are acted on in a timely manner, whether profes-

based on any comprehensive or formal checklist Chapter sional standards are met, whether policies for to assess compliance to professional and divisional bail and charge screening are complied with, and standards. For instance, quality of case work will be whether efficiency-increasing initiatives such assessed if an appeal is granted by a court, which as diversion programs are used appropriately occurs in less than one-quarter of 1% of all closed and consistently. cases. In addition, we noted that the Division had Crown attorneys have prosecutorial independ- initiated a good practice of periodically reviewing ence in their decision-making, but they are account- some high-profile cases processed by the Major able for carrying out the policy direction set out in Case Management and Guns and Gangs units of the Divisional Crown Policy Manual. The manual the Division to assess processes or practices that is a compilation of prosecution policies, detailed worked well and what could be improved. legal advice, and practice memoranda and guide- From our research, we learned that one other lines intended to provide a consistent approach to province does reviews of prosecutors’ case files prosecutions across the province. For instance, the and that the federal prosecutors recently initiated manual addresses procedural policies on charge a pilot project to do so. In addition, we noted that screening, providing disclosure, dealing with vic- the Crown Prosecution Service of England and tims and sentencing, as well as accepted practice for Wales (CPS) had a robust, publicly reported quality specific types of prosecutions, including Aboriginal assurance program for assessing its work. The CPS’s 78 2012 Annual Report of the Office of the Auditor General of Ontario

program included assessing case work on closed CROWN MANAGEMENT files against the CPS’s core quality standards and INFORMATION SYSTEM the code setting out what their prosecutors do, how Many Crown prosecution services—for example, they make decisions, and the level of service they Manitoba in 1999, British Columbia in 2002 and commit to in key aspects of their work. In its most the Public Prosecution Service of Canada in 2002— recent assessment of more than 10,000 closed case have moved to electronic systems that essentially files, the CPS found that 78% of commitments in manage and track cases and how resources are used the case files were fully met, 11.5% partially met to prosecute those cases. and 10.5% not met. Ontario Crown attorney offices still rely largely on paper-based manual processes to manage RECOMMENDATION 4 their workload. As of the end of our fieldwork in To ensure that regional and division manage- July 2012, the Division’s project to implement an ment have adequate assurance that cases are electronic case management system, known as the prosecuted in a consistent, timely and effective Crown Management Information System (CMIS), manner that meets expected standards, the was significantly delayed and projected to be Criminal Law Division should perform a per- over budget. iodic, objective review of a sample of files from The original business case for CMIS was submit- each Crown attorney relating to the prosecuted to Treasury Board for approval in December tions each one handled during the year. 2006 and estimated that the project would cost

3.02 $7.9 million and be completed by March 2010.

MINISTRY RESPONSE CMIS would allow the Division’s 54 Crown attorney

Section offices to receive, track, store, modify and share The Criminal Law Division will compile a list of electronic documents; and to automate several VFM practices to advance the leadership’s commit- •

processes, including scheduling, criminal case 3 ment to reinvigorate the performance planning management and business intelligence. In addition, and feedback process, including more meaning- the new system would allow the Division to receive

Chapter ful feedback that differentiates performance, electronic documents from police forces, including attendance by the supervising Crown attorney Crown briefs and pre-trial disclosure documents. during proceedings and spot-checking files. This The Ministry’s Justice Technology Service unit was information will supplement existing quality to work with the Division to provide the informa- control practices with respect to periodic review tion technology expertise and manage the project. of files. Consultations with the supervising In 2010, the Division engaged a consultant to Crown attorneys will occur prior to implementa- assess why the CMIS project had been delayed. tion of items on the list. In doing so, the Division The consultant recommended that the Division put anticipates that it will be able to ensure the together a dedicated project team, including special- continued development of its workforce and to ists in change management, communications and allow decision-makers to enhance consistency implementation management. It also recommended in how files are prosecuted on a local, regional that the software application that was originally and provincial level. Although these practices chosen for the project be replaced with more robust, have existed in local offices for some time, the vendor-supported, user-friendly software. The pro- Division will benefit from a more consistent ject completion date was extended to March 2012, approach across all offices. and the Ministry established a new project team. Criminal Prosecutions 79

By September 2011, the new project team leader As of July 2012, CMIS was still underway, and a reported that the project was delayed again. In Nov- total of approximately $5.2 million had been spent. ember 2011, the Ministry developed a new tentative The Ministry will need to seek Treasury Board’s strategy under which an additional three years approval should it revise its budget and timeline to would be required to complete the project, making complete the project. the projected completion date March 2015, at a revised projected cost of $11.5 million. As a result, RECOMMENDATION 5 it was decided in May 2012 that the project would To ensure that the paper-intensive processes be reviewed by the Ministry’s internal auditors. currently used by the Criminal Law Division are In its June 2012 report, the Ministry’s internal replaced with an electronic case-management auditors raised a number of concerns with respect system to better manage and track prosecutions to the management of this project. The internal and staff resources, the Ministry of the Attorney auditors also noted that IT consultants hired to General should significantly strengthen project develop CMIS, at a cost so far of $1.3 million, were management to mitigate the challenges posed not managed effectively and that billings were by its Crown Management Information Sys- based on time spent rather than meeting project tem (CMIS). In addition, the Ministry should deliverables and outcomes. formally evaluate existing case-management In addition to the internal auditor’s findings on systems in other jurisdictions to identify any project delays, we questioned why a totally new potential for achieving savings and shortening project was initiated and developed without more

the time to get the required system in place. 3.02 carefully considering the systems already in use in other provinces and the cost and time savings

MINISTRY RESPONSE Section that could be achieved by using an already proven VFM system. We noted that, although the Ministry did The Criminal Law Division agrees with the •

3 research existing systems in 2005, it did not pursue, Auditor’s recommendation that having a robust for example, working with Manitoba’s system, information management system in place

which we observed has useful features for tracking will enhance the Division’s capacity to more Chapter cases and workloads, something that would address effectively and efficiently prosecute cases and many of the concerns we have raised in this report use resources. The Division has already taken regarding the Division’s lacking information on its steps to enhance its project management and operations (although Manitoba’s system does not oversight of the CMIS project. Rather than electronically store case documents, which is a fea- implement the tentative $11.5 million strategy, ture the Division has specified for its new system). the Division is now evaluating existing case We were advised that Alberta’s new case manage- management systems from other jurisdictions, ment system will be modelled on Manitoba’s as well as systems used by police forces, and will system, which was provided to Alberta at a cost of continue to do so. Specifically, systems will be $1. Alberta plans to enhance the system to electron- re-examined with a view to establishing specific ically store case documents. Manitoba’s agreement components of a system rather than continue to with Alberta requires that Alberta provide Mani- seek one ideal system for Ontario’s prosecution toba with any modifications and enhancements to service. For example, the Division is currently the case management system, which we consider re-examining Manitoba’s case management to be a beneficial collaborative arrangement for system to determine if it could meet some of both parties. Ontario’s needs. 80 2012 Annual Report of the Office of the Auditor General of Ontario

PERFORMANCE MEASUREMENT AND dictions told us that the idea of measuring “success” PUBLIC REPORTING by computing and emphasizing conviction rates was not consistent with what has been established We found the Ministry does not, in any substantive as the Crown attorney’s role: to simply lay out the way, measure or report on the performance of the facts and let a judge and/or jury decide innocence Criminal Law Division. In fact, the Ministry makes or guilt. However, this view does not take into no mention of the Division in its annual reporting. account the fact that Crown attorneys make deci- As we have noted throughout this report, the sions on whether or not to prosecute based on Division collects little data of its own. For example, what they consider the likelihood of conviction. the Division does not currently have performance For instance, dramatically high conviction rates indicators for Crown attorneys’ workloads, charges might indicate undue conservatism in proceeding disposed of per Crown attorney, cost per case, to trial with charges, while low rates might indi- average cost of trial resolution, and prosecution cate the need for increased senior level oversight outcomes such as conviction rates and use of court and guidance. diversion programs. It also does not track certain As noted earlier, Statistics Canada data shows key prosecutorial outcomes, such as trial collapse that Ontario achieved the lowest proportion of rates, rates of withdrawal of charges, the number guilty verdicts in adult criminal cases among of stays and adjournments, crimes committed Canadian jurisdictions in 2010/11, as well as the by accused persons on bail release, and witness highest proportion of cases in which charges were attendance rates. stayed or withdrawn. Without further analysis,

3.02 A number of other jurisdictions we researched, these statistics could mean that, overall, Ontario including British Columbia, Manitoba, the Public Crown attorneys are not as successful as prosecu-

Section Prosecution Service of Canada, and the Crown tors in other provinces. On the other hand, they Prosecution Service of England and Wales (CPS) did VFM could indicate that Ontario successfully re-directs •

measure and report on their results. For instance, 3 a higher proportion of cases away from courts British Columbia’s Prosecution Service publicly and to its diversion programs and achieves lower- reports a strategic plan, which includes its priorities

Chapter cost solutions for cases involving relatively minor and major projects, and timelines for their imple- charges. Without analyzing the reasons for these mentation. CPS also reports extensively on its busi- variances, the Division and the Ministry cannot ness planning and performance management, and make an informed judgment on issues such as analyzes its efforts to meet strategic goals. It reports these. We were advised that the Ministry also does on criminal case outcomes for each of its districts, not compare key performance measures with other witness attendance rates and average costs per case. provinces because there have not been successful In its annual report, the Alberta justice ministry collaborative efforts among the various prosecution reports on the public perception of fairness in its services to identify appropriate, consistent and prosecution service—that is, the percentage of meaningful performance measures. Albertans who agree that the Alberta justice ministry provides fair and impartial service in prosecut- RECOMMENDATION 6 ing people charged with crimes. Jurisdictions such as the United Kingdom and Particularly given the importance of the Crim- the United States report on prosecutors’ success inal Law Division to the mandate of the Ministry rates in obtaining convictions, but we acknowledge of the Attorney General, the Ministry should that this is not common practice in Canada. Div- develop performance indicators specifically for ision staff and prosecutors in other Canadian juris- the Division, and should publicly report on the Criminal Prosecutions 81

Division’s progress toward those indicators. It formance indicators and of measuring results in should also consider liaising with other prov- support of its core mandate of contributing to inces’ prosecution services to develop common public safety for the citizens of Ontario. The Div- performance measures that would allow for ision will continue to explore additional qualita- comparison, benchmarking and the identifica- tive and quantitative performance indicators, tion of best practices. both within the Ministry and with other prosecution services. The audit accurately outlines MINISTRY RESPONSE our concerns about defining success through the measurement of conviction rates. Once these Given that what gets measured gets done, indicators are identified, the Division will move the Criminal Law Division acknowledges the forward to publicly report on the results relating importance of continuing to evolve its key per- to these performance indicators. 3.02

Section

VFM

•

Chapter Chapter 3 Ministry of Health and Long-Term Care Section 3.03 Diabetes Management Strategy

• The number of people with diabetes in Background Ontario is expected to almost quadruple, from 546,000 in 2000 to 1.9 million by 2020, as illustrated in Figure 1. Ontario’s diabetes Chronic diseases are the leading cause of death growth rate is among the highest of all prov- and disability in Ontario. One of the most common

3.03 inces, as illustrated in Figure 2. chronic diseases is diabetes, which results from the People with diabetes incur medical costs that body’s partial or complete inability to produce and/ •

Section are roughly twice as high as those without or properly use insulin, a hormone that regulates the disease. A diabetes patient costs Ontario’s VFM blood sugar. Diabetes can lead to kidney failure, •

3 heart attack, stroke, amputation and blindness Figure 1: Diabetes Prevalence and Costs in Ontario, if poorly managed or left untreated. In Type 1 2000–2020*

Chapter diabetes, which accounts for 10% of cases, the Source of data: Canadian Diabetes Association pancreas produces no insulin. Type 1 is not prevent- Direct costs (hospitalization, physicians, able and its cause remains unknown. The remain- specialists and medications) ing 90% of people with the disease have Type 2 Indirect costs (loss of economic 2.0 output from illness, disability 7. 0 diabetes, in which the pancreas does not produce 1.8 and mortality) enough insulin, or the body cannot properly use 6.0 1.6 Diabetes population the insulin it does produce. Type 2 is most often 1.4 5.0 preventable with lifestyle changes that include 1.2 4.0 healthier eating and exercise. 1.0 In recent years, diabetes, especially Type 2, has 0.8 3.0 Cost ($ billion)

grown significantly as a health problem in Ontario, Population (million) 0.6 2.0 affecting the quality of life of people who have 0.4 1.0 it and straining the health-care system. Factors 0.2 driving this growth include high obesity rates, 0.0 0.0 200020102020 sedentary lifestyles, unhealthy diets and an aging population. Statistics from the Ministry of Health * Based on estimates by the Canadian Diabetes Association using the Canadian Diabetes Cost Model. The two main sources of data used for and Long-Term Care (Ministry) and the Canadian the estimates and forecasts came from the National Diabetes Surveillance System and Health Canada’s study “The Economic Burden of Illness in Diabetes Association suggest some alarming trends: Canada.”

82 Diabetes Management Strategy 83

ewan, Manitoba and Nova Scotia). In the four years Figure 2: Percentage Growth in Diabetes Prevalence, up to the 2011/12 fiscal year, the Ministry allocated 2000–2020* (%) about $648 million of the Strategy’s original fund- Source of data: Canadian Diabetes Association ing announcement of $741 million to various initia- 300 tives, as illustrated in Figure 3. The Strategy was 250 subsequently extended for another four years with a new funding approval of $152 million, which is in 200 addition to funding for diabetes services delivered through other program areas of the Ministry. 150

100

50 Audit Objective and Scope

0 AB BC MB NB NS NL ON PEI QC SK Our audit objective was to assess whether the * Based on estimates by the Canadian Diabetes Association using the Ministry of Health and Long-Term Care (Ministry) Canadian Diabetes Cost Model. The two main sources of data used for the estimates and forecasts came from the National Diabetes Surveillance had adequate systems, policies and procedures in System and Health Canada’s study “The Economic Burden of Illness in Canada.” place to: • monitor and assess whether service providers health-care system more than $3,000 a are meeting the needs of people with diabetes 3.03 year, but this can rise to more than $5,000 if the patient experiences complications. Figure 3: Allocation of Ontario Diabetes Strategy Section Diabetes complications account for 69% of Funding by Key Initiatives, 2008/09–2011/12 VFM limb amputations, 53% of kidney dialysis and ($ million) •

Source of data: Ministry of Health and Long-Term Care 3 transplants, 39% of heart attacks and 35% of strokes. Chronic Kidney Disease Program, $220 (34%)

In 2008, the Ministry announced a new four- Chapter Prevention initiatives, $19 (3%) year, $741-million plan, called the Ontario Diabetes Regional Coordination Centres Strategy (Strategy), to expand services and improve (RCCs), $19 (3%) the health of Ontarians with diabetes. The Ministry Other, $41 (6%) said that the Strategy’s goals included raising awareness of diabetes risk factors through prevention programs, creating more Diabetes Education Teams to help patients better manage the disease, and developing an online Diabetes Registry to track individual patients and the overall prevalence of the disease across Ontario. The Ministry has identi- Diabetes Registry and fied the Diabetes Registry as a “top clinical priority Baseline Diabetes Dataset Initiative (BDDI), $150 (23%) for eHealth Ontario,” which has been working in Diabetes Education Programs partnership with a private-sector vendor to develop (DEPs), $58 (9%) and implement it. Insulin Pump and Supplies According to the Canadian Diabetes Associa- Program, $63 (10%) tion, Ontario is one of five provinces with a formal Bariatric surgery, $78 (12%) diabetes strategy (the others are Alberta, Saskatch- 84 2012 Annual Report of the Office of the Auditor General of Ontario

by providing them with timely access to appropriate and quality care; Summary • ensure funding and resources provided for the Ontario Diabetes Strategy (Strategy) are used The number of people with diabetes in Ontario cost-effectively; and has more than doubled from 546,000 in 2000 to measure and report periodically on the results • 1.2 million in 2010, and that number is projected and the effectiveness of the Strategy. to grow to 1.9 million by 2020—about one in In conducting our audit, we reviewed relevant every eight Ontarians. People with diabetes use policies and files, analyzed data, interviewed the health-care system at about twice the rate of appropriate ministry staff and reviewed relevant the general population, and the cost of diabetes to studies from Ontario and other jurisdictions. We Ontario’s health-care system is estimated to grow also interviewed staff and reviewed documents from $4.9 billion in 2010 to $7 billion by 2020. The related to the Diabetes Registry at eHealth Ontario Ministry recognized the long-term implications of and Infrastructure Ontario. As well, we conducted this and in 2008 established a four-year $741 mil- surveys of about 10,500 physicians with assistance lion Ontario Diabetes Strategy (Strategy). Although from the Ontario College of Family Physicians it is still too early to gauge the Strategy’s mid- and (580 responded) and of all 152 Diabetes Education long-term impact, in the short term the results have Programs (DEPs), with the assistance of an external been mixed. survey company (103 DEPs responded). In addi- On the one hand, there undoubtedly has been tion, we contacted and visited stakeholders and dia-

3.03 an improvement in the availability of diabetes betes-care providers across the province, including care, giving people more options and knowledge to six DEPs, five bariatric surgical sites, four Regional

Section enable them to manage the impact of diabetes. On Coordination Centres, and the Canadian Diabetes the other hand, most of the diabetes service pro- VFM Association. We also engaged an independent •

viders that were set up with Strategy funding are 3 consultant with expert knowledge in the study of under-utilized, and many of those who responded diabetes on an advisory basis. to our surveys felt that more of their funding should

Chapter Our audit did not cover any initiatives under be directed toward preventive services. the Strategy that had just started, had recently According to the Diabetes Expert Panel estab- changed, and/or had been audited by our Office in lished by the Ministry in 2006, “keeping people well recent years. These include the Centres for Complex and preventing disease is the most cost-effective, Diabetes Care (providing people with complex dia- affordable and sustainable strategy for coping with betes a single point of access to specialized care), chronic disease.” We noted, however, that 97% of the Chronic Kidney Disease Program (providing the $741 million funding was earmarked to treat dialysis services to diabetes patients who have kid- people who already had diabetes, with only 3% for ney failure, a common diabetes complication), and preventive initiatives. Given that 90% of people the Insulin Pumps and Supplies Program (providing with diabetes have Type 2, which can often be funding assistance to people with Type 1 diabetes). prevented or postponed with good nutrition and As well, we did not rely on the Ministry’s internal exercise to limit weight gain along with other pre- audit service to reduce the extent of our audit work ventive measures, we believe an increased focus on because it had not recently conducted any audit prevention warrants consideration by the Ministry. work on diabetes initiatives in Ontario. Some of our other observations were as follows: • eHealth Ontario (eHealth) has been working in partnership with a private-sector vendor to Diabetes Management Strategy 85

Health Teams (FHTs) to educate diabetes develop and implement a new electronic Dia- patients. The Ministry funded 101 new DETs betes Registry, a key initiative of the Strategy, under the Strategy, increasing the total num- to give physicians and the Ministry real-time ber of DETs in Ontario to 322. However, many patient data and comprehensive online mon- hospitals and FHTs have also set up education itoring of the disease. The Registry’s original programs of their own with funding from delivery date was April 2009, but this dead- other sources, including another branch of the line was not met and the proposed release Ministry, and this has led to service overlaps date has been extended a number of times. and under-utilization of about 90% of DEPs. Subsequent to our audit fieldwork, we were The DEPs are required to conduct audits regu- advised that the contract with the vendor was • larly on the quality of care provided by their terminated in September 2012. staff. However, the Ministry has never verified In August 2010, eHealth and the vendor, • whether they actually do so. Our survey of through a request-for-proposal process, DEPs found that about 25% said they “have signed a six-year, $46-million contract that not done” any audits or “do not know” they stipulated that the vendor would be paid only are required. after successful completion of the Diabetes The Ministry provides an organization with Registry. This was designed to help protect the • $20 million annually to manage and fund public’s interest and to motivate the vendor diabetes service providers, including 47 DEPs to deliver a system that meets performance in Northern Ontario, on its behalf. While

requirements and timelines. Although no pay- 3.03 the Ministry has an accountability agreements had been made to the vendor as of mid- ment with the organization and a reporting

2012, the Ministry and eHealth had already Section process in place, it needs to significantly incurred about $24.4 million in internal costs VFM enhance its oversight of this organization •

directly related to the Diabetes Registry since 3 to ensure that the organization and the 2008/09. They also spent another $50 mil- service providers funded by it have used lion on other supporting electronic health

the Ministry’s funding appropriately and Chapter records initiatives, such as electronic portals in compliance with applicable policies. For to other systems and the Ontario Laboratory instance, the organization has paid a consult- Information System, which have already been ing firm $105,000 since the 2009/10 fiscal deployed. eHealth acknowledged that the year for “advice on election strategizing” and arrangement to pay the vendor only after suc- “developing relationships with relevant pol- cessful completion of the contract has traded itical decision-makers.” We also noted that away much of the province’s control over the the organization could produce no original project’s design, progress and delivery time in itemized receipts to support its staff meal exchange for price certainty. expenses, contrary to its own policies and to The province’s 152 Diabetes Education Pro- • policies set out by the government for provin- grams (DEPs) help teach people with diabetes cially funded public-sector organizations. We about the disease and how to manage it. Every found instances of staff claiming unreason- DEP runs one or more Diabetes Education able amounts for meals and claiming alcohol, Teams (DETs), each consisting of a regis- neither of which is in line with the rules of tered nurse, a registered dietician and other the Ontario Public Service. professionals. DETs operate in hospitals, in The Ministry has significantly expanded community health centres and within Family • Ontario’s capacity for performing bariatric 86 2012 Annual Report of the Office of the Auditor General of Ontario

surgery, a procedure that combats Type 2 will be more opportunities to prevent diabetes diabetes in obese people by removing part of for those at risk and, for people with diabetes, a the stomach and/or the small intestine. This more positive experience with the health system has led to savings because far fewer out-of- and a better quality of life. country surgeries are being done. The number Investments in the Strategy will help build of in-province surgeries has risen significantly, system capacity and an infrastructure for a com- from 245 in the 2007/08 fiscal year to 2,500 prehensive chronic disease prevention and man- in 2011/12. However, this still does not meet agement system that may be readily expanded the current demand and is actually lower than to address other chronic conditions. the 2009/10 total of 2,900, when more out-of- country surgeries were performed.

OVERALL MINISTRY RESPONSE

The Ministry acknowledges the recommenda- Detailed Audit Observations tions made by the Auditor General of Ontario and thanks him for conducting this timely audit. KEY INITIATIVES AND PROGRESS OF The Ministry and this government are commit- ONTARIO DIABETES STRATEGY ted to the development and implementation of In response to Ontario’s growing diabetic popula- innovative initiatives and solutions that address tion and the costs associated with it, the govern-

3.03 the impact of diabetes and other chronic dis- ment launched the Ontario Diabetes Strategy eases on Ontarians. (Strategy) in the 2008/09 fiscal year with funding

Section The Ministry welcomes the insights and of $741 million over four years. The Ministry said at recommendations provided by the Auditor Gen- VFM the time that the Strategy aimed to “prevent, man- •

eral. The audit identifies areas of consideration 3 age and treat diabetes across the province.” that the Ministry is already taking measures The Ministry has used the modified Kaiser to address and reinforces the Ministry’s com-

Chapter Chronic Disease Management model to graph- mitment to addressing the complex challenge ically illustrate diabetes services along the of diabetes. continuum of care, as illustrated in Figure 4. In June 2008, Cabinet approved $741 mil- The model divides Ontarians into three levels, lion in funding for the first four-year phase depending on the severity of their diabetes, and a (2008–12) of the Ontario Diabetes Strategy, fourth for people who do not have the disease but a comprehensive strategy to enhance preven- are at risk of contracting it. The Ministry defined tion and management of diabetes and improve specific initiatives for dealing with each level, as health outcomes for Ontarians impacted by well as “system enablers” to assist in executing diabetes. In April 2012, the Ministry confirmed the overall Strategy. Since the Strategy’s introduc- continued funding for the Strategy of $152 mil- tion, the Ministry has improved the availability lion from 2012 to 2016. of diabetes services by funding 101 new Diabetes Through the Strategy, the government has Education Programs (DEPs) and establishing 14 built on existing investments in prevention and Regional Coordination Centres (RCCs) across care across the health system to build capacity, the province. make it easier for people to get services they The Ministry also developed performance need, and improve the overall quality of dia- measures to assess the Strategy’s progress at vari- betes service and care in Ontario. The result ous stages. As Figure 5 illustrates, the short-term Diabetes Management Strategy 87

Figure 4: The Modified Kaiser Chronic Disease Management Model Source of data: Ministry of Health and Long-Term Care

Key Initiatives or Investments under Strategy • Bariatric surgery Level 3 • Centres for Complex Diabetes Care (CCDCs) Complex diabetes • Chronic Kidney Disease Program (~175,000*) • Insulin Pump and Supplies Program

Level 2 Moderate diabetes • Diabetes Education Programs (DEPs) (~409,000*) • Self-management Initiative Level 1 Early diabetes • Tools for individuals and health-care providers (~584,000*) • Prevention programs for high-risk groups (e.g., Aboriginal people, Asians) At-risk & general populations • EatRight Ontario • Improved access to primary care for prevention and early identification

• Diabetes Registry and Baseline Diabetes Dataset Initiative (BDDI) System enablers 3.03

• Regional Coordination Centres (RCCs) Section

* Based on estimates of diabetes prevalence by the Canadian Diabetes Association. VFM

•

3 performance measures have shown mixed results DIABETES REGISTRY AND BASELINE and the intermediate or long-term measures have DIABETES DATASET INITIATIVE not yet shown improvement. In particular, hospi- Chapter A key initiative of the Strategy was to have been the talization rates for heart attacks, infections, ulcers electronic Diabetes Registry, a database containing or amputations among people with diabetes have information about every Ontarian with diabetes increased and did not meet targets, but the Ministry intended to facilitate the delivery of care by clin- indicated that it has not established specific time- icians. The Registry, in conjunction with other lines for achieving these targets. Overall, rates have foundational systems developed by eHealth Ontario stabilized or decreased for some diabetes-related (eHealth), an agency of the Ministry, would also complications, hospitalizations and deaths, but the be a first step toward an eventual province-wide actual numbers remain high and can be expected to Electronic Health Record for every Ontarian by continue to rise significantly due to the growth of 2015. Pending delivery of the Registry, the Ministry the diabetic population. implemented the Baseline Diabetes Dataset Initia- Based on the data available to date, we noted tive (BDDI) as an interim measure to provide phys- that it is still too early to gauge the Strategy’s icians with paper reports containing their patients’ impact over the medium and long term. Clearly, most recent dates for the three key diabetes tests. however, much remains to be done to reduce the The Ministry has continued to use the BDDI in the growing burden of diabetes in Ontario. absence of a complete Diabetes Registry. 88 2012 Annual Report of the Office of the Auditor General of Ontario

Figure 5: Summary of Key Performance Measures for the Ontario Diabetes Strategy Source of data: Ministry of Health and Long-Term Care

Key Performance Measures for the Strategy Target Baseline* Oct 2010 Oct 2011 Short-term Measures % of Ontarians who are physically inactive 45a 52.6 51.2 51.0 % of Ontarians who are overweight or obese —b 51.8 51.7 53.0 % of Ontarians with diabetes who have a regular family doctor —b 97. 6 96.4 96.9 Number of Ontarians with diabetes registered with Health Care Connect —b 3,744 4,768 10,335 % of Ontarians with diabetes referred to family health-care provider by —b 60.5 59.8 63.0 Health Care Connect % of Ontarians with diabetes for whom a Diabetes Management —b 25.5 26.3 29.2 Incentive (Q040) was billed % of Ontarians with diabetes for whom a Diabetes Management —b 27. 5 28.5 32.0 Assessment (K030) was billed % of Ontarians with diabetes who received all three key tests within the 80 37. 6 37. 8 39.6 guideline periods Intermediate or Long-term Measures % of diabetes prevalence in Ontario population —b 8.7 9.3 9.4 Rate of emergency visits for hyperglycemia or hypoglycemia (abnormal —b 1,185 1,115 1,063 blood-sugar levels) per 100,000 people with diabetes 3.03

Rate of kidney dialysis and transplant per 100,000 people with Maintain 712 842 845 diabetes original rate Rate of hospitalization for infections, ulcers or amputations per Reduce last Section 100,000 people with diabetes reported rate 2,294 2,932 3,347 VFM by 10% •

Rate of hospitalization for heart attacks per 100,000 people with Reduce last diabetes reported rate 877 1,022 1,082 by 10% Chapter Rate of eye surgeries per 100,000 people with diabetes —b 3,500 3,612 3,365

* Baseline data were derived between April 1, 2008, and March 31, 2010, depending on the performance measure. a. Although the Strategy did not set a target for this measure, the ACTIVE2010 Strategy, launched in 2004, set a target of increasing Ontario’s physically active population to 55% by 2010 (or reducing its physically inactive population to 45% by 2010). b. The Strategy did not set a target for this measure.

Diabetes Registry partnership with a private-sector vendor to develop and implement the Registry, which the Ministry The Diabetes Registry is an interactive, real-time identified as “a top clinical priority for eHealth.” information system that health-care providers can use to quickly identify and manage Ontarians with diabetes, check patient records, access diagnostic Timeline of Diabetes Registry Development information, send patient alerts and track the care The original target date for a first release of the provided to patients against evidence-based guide- Diabetes Registry was April 2009, but this deadline lines. The Registry is intended to be used to per- was not met. According to our review of documenta- form continuous and comprehensive province-wide tion—which included reports to the Management surveillance of diabetes to support planning and Board of Cabinet and the Treasury Board, the monitoring of care. eHealth has been working in project charter and plan, original and amended Diabetes Management Strategy 89 agreements, and meeting minutes between eHealth • When bidding for the Diabetes Registry con- and the vendor—the proposed release date was tract, the vendor who won the contract may moved forward a number of times. The Registry still have underestimated both the time required had not been delivered at the end of our fieldwork for the project and the project’s complexity. in June 2012. • The vendor’s project-management team may Management at eHealth told us it could com- have failed to identify the project’s critical path ment only on the timing changes that had taken timelines and effectively manage completion place after April 2010, which was when the new of the project components and deliverables. management took office (eHealth changed its • The project-design blueprint developed by management after our 2009 audit of Ontario’s the vendor appeared to contain many errors Electronic Health Records Initiative). eHealth and omissions, which led to rejections and also informed us that the official and contractual reworking of the design. targeted release date under its current management • The vendor spent a great deal of time fix- was June 2011. ing numerous quality issues in the Diabetes On completion, the Diabetes Registry initially Registry since the first test in September 2011. was to have been phased in at several pilot sites There were still hundreds of defects remain- within two of the province’s 14 Local Health ing in March 2012. Integration Networks (LHINs), the not-for-profit corporations that plan, integrate and fund local Development Costs of Diabetes Registry health services. However, the two pilot LHINs

In August 2010, eHealth signed a six-year, 3.03 expressed concerns about the repeated delays and $46-million contract with a vendor to design, about the fact that eHealth no longer meets with

build, implement and manage the Diabetes Regis- Section them weekly to discuss the status of the Registry try. The contract included a lump-sum payment VFM project. For its part, eHealth advised us that it and •

of about $12 million when the Registry is ready 3 the LHINs decided in fall 2011 to suspend the meet- for use and a monthly payment of $575,000 for 60 ings because of the lack of a reliable schedule for months during a period of operation and mainten-

delivery of the Registry. Given that the first release Chapter ance. Payments to the vendor would not begin of the Registry at the pilot sites will have only until after the Registry was successfully completed limited functionality, one of the two LHINs told and deployed. us that it did not see a clear road map for future Since no payments had been made to the vendor enhancements; the other questioned the value of of the Registry as of June 2012, eHealth has not the Registry altogether. spent all of the $150 million funding approved eHealth cited the following reasons for the under the Strategy for the Registry and other repeated delays: related eHealth projects. Since the 2008/09 fiscal The Treasury Board approved the Diabetes • year, the Ministry and eHealth have incurred sig- Registry budget in August 2008, but procure- nificant internal costs to develop the Registry and ment was put on hold until May 2009 because other related projects. Specifically: of the creation of eHealth and a government Since the 2009/10 fiscal year, eHealth has decision to involve Infrastructure Ontario in • incurred about $20 million in costs for pro- selecting a vendor to develop and implement jects directly related to the Diabetes Registry the Registry. Through a request-for-proposal and another $50 million on other key elec- process, eHealth signed the contract with the tronic health records initiatives. These include vendor in August 2010. the electronic portals to other systems, the 90 2012 Annual Report of the Office of the Auditor General of Ontario

Ontario Laboratory Information System, the and expenditures throughout the life cycle of a Patient Registry and the Provider Registry, project to give management a snapshot perspective which have been deployed and play a role in of a project’s progress at any time. However, this supporting improvements in diabetes care and methodology did not apply to the Registry, which management. was developed using the Alternative Financing and • In the 2008/09 fiscal year, the Ministry Procurement (AFP) model, the first time that the engaged consultants to carry out Diabetes government used AFP for an information technol- Registry–related projects at a total cost of ogy project. eHealth advised us that it applied an about $4.4 million. The Ministry informed alternative project oversight methodology specific us that some of the projects were suspended to an AFP technology-related project to the Dia- due to the transition of the Registry from the betes Registry. Ministry to the newly formed eHealth, and to Under AFP, the government as client establishes changes to eHealth’s strategic direction fol- the scope and purpose of the project, and the work lowing recommendations of our 2009 audit of is financed and carried out by a private-sector Ontario’s Electronic Health Records Initiative. vendor. Only after a project is complete does the According to the contract with the vendor, it private-sector vendor get paid. According to Infra- appeared that eHealth could terminate the contract structure Ontario, which provided eHealth with without any obligation to make any payments if the procurement-management services for the Registry vendor did not achieve the project’s independently project, AFP allows for projects to be delivered certified completion by a specified date. While the faster and more efficiently, on time and on budget. 3.03 Diabetes Registry was regarded as a clinical priority AFP also differs from traditional procurement pro- for the implementation of certain key aspects of cesses in that it is designed to avoid cost overruns Section the Strategy, we were advised that with many more and to transfer risks from the government to the VFM physicians migrating to Electronic Medical Records, vendor, who has the expertise and experience to •

3 the Registry is no longer seen as an essential com- handle them. ponent. Subsequent to completing our audit field- eHealth acknowledged that there have been

Chapter work, we were advised that the contract with the trade-offs in using AFP for the Registry; as the vendor had been terminated in September 2012. vendor gets paid only upon successful completion and delivery of the project, the province gets price certainty in exchange for only minimal influence on Procurement Process and Contract Management the design, progress and delivery time of the pro- Our review of eHealth’s procurement documents ject. Thus, while the province does not have to pay related to the Diabetes Registry in the 2009/10 fis- the vendor for non-delivery of the Registry, it still cal year indicated that most procurements did not ultimately bears other risks and costs associated follow an open competitive process but, instead, with project delays. were sole-sourced without the approvals necessary Infrastructure Ontario engaged an external firm to bypass the competitive process. However, sub- to perform a risk analysis before eHealth signed the sequent to our 2009 audit of Ontario’s Electronic contract with the vendor, and the analysis identi- Health Records Initiative, we noted that eHealth fied the risk of delays, which could increase costs improved its procurement processes: all procure- and discourage user adoption. However, Infrastruc- ments we examined in the 2010/11 and 2011/12 ture Ontario concluded that the stipulation that the fiscal years followed an open competitive process. vendor would not be paid until after the system was eHealth also informed us that it instituted a pro- certified as complete would be a sufficient incentive ject management methodology to track deliverables to ensure that the vendor completed the project. Diabetes Management Strategy 91

Infrastructure Ontario engaged an Independent Baseline Diabetes Dataset Initiative Certifier on behalf of eHealth and the vendor to The Ministry launched the Baseline Diabetes Data- monitor the Registry’s progress and to issue a final set Initiative (BDDI) in spring 2009 as an interim acceptance. In its monthly reports, the Certifier measure pending delivery of the Registry. The BDDI repeatedly raised concerns about impediments to provides physicians with paper reports containing its ability to assess, measure or control the project’s information on their diabetes patients’ most recent progress. However, while eHealth had similar dates for three key tests. This information helps concerns, it informed us that under AFP, it had less physicians to better manage their diabetes patients leverage to require the vendor to address the defi- by determining the dates on which patients should ciencies identified by the Certifier, including: receive their next tests. The recommended time serious delays with the overall project; • frame for each test is as follows: no visibility into the project’s schedule, scope • blood-sugar test at least once every six months; and progress, and none into the causes for • cholesterol test at least once a year; and the delays; • retinal eye exam at least once every two years. absence of an approved project plan and no • • The Ministry and eHealth spent $5.6 million targets for milestones; between 2009 and 2012 to implement three waves tight timeline with minimum buffer; • of the BDDI and another $2.6 million in incen- quality concerns, as evidenced by the large • tive payments to physicians to encourage them to number of defects identified; and participate in it. Although the Canadian Diabetes • fragmented testing approach, with multiple Association has cited the BDDI as a best practice 3.03 plans and ambiguous criteria. that “facilitates identification of patients for testing In April 2011, in an attempt to improve the

on the three tests,” we noted the following: Section timeliness and certainty of delivery of the Diabetes The Ministry set a long-term goal that 80% VFM Registry, the project was split into two releases: one • •

of adult Ontarians with diabetes should 3 to be delivered in September 2011 and the other receive all three key tests within the recom- in May 2012. The vendor was to be paid 75% of mended times. According to BDDI results as of

the acceptance payment after the first release and Chapter December 31, 2011, 45.6% of adult Ontarians the remaining 25% after the second, as opposed to with diabetes had all three tests within the 100% on successful completion of the entire Regis- recommended periods—still well below the try as per the original contract. long-term target of 80% and only slightly eHealth informed us that the purpose of amend- higher than on December 31, 2009 (43.1%). ing the contract was to ensure that the first release, The Ministry informed us there is no pre- a core clinical module, was delivered as early as determined timeline for achieving this target. possible. However, at the end of our audit fieldwork The BDDI does not include test results from in June 2012, neither release was ready and the • hospital labs, even though hospitals accounted delivery date remained uncertain. Infrastructure for about one-third of all labs in Ontario and Ontario advised us that, in its opinion, making first performed 9% of blood-sugar tests and 10% of payment to the vendor only after project comple- cholesterol tests. According to our physician tion constitutes an incentive for the vendor to com- survey and our review of correspondence plete and deliver the project as soon as possible. between the physicians and the Ministry, many As well, the Ministry and eHealth informed us that physicians indicated that hospital labs are they have taken steps to mitigate risks associated the only major lab services in some rural and with delays by continuing to develop other comple- northern regions. The Ministry acknowledged mentary initiatives that support diabetes care and management. 92 2012 Annual Report of the Office of the Auditor General of Ontario

this limitation but noted that the BDDI is significant advances in meeting its obligations an interim measure pending delivery of the to develop, integrate and deliver the comple- Registry. mentary foundational systems that serve as the Many of the physicians who responded to our • backbone of Electronic Health Records. survey and the four Regional Coordination In applying lessons learned from the Dia- Centres we visited expressed other concerns betes Registry procurement model, eHealth about the BDDI, and a number questioned its has already taken steps to improve the procure- usefulness. They noted that the information ment process for future projects. In particular, provided by the BDDI was not timely and was eHealth has applied the AFP model’s milestone- already accessible through currently avail- based payment structure to procurement of the able Electronic Medical Records now being Drug Information System and has combined this used by 43% of Ontario physicians. They also with a rigorous procurement process with con- noted that the BDDI was a time-consuming tractual provisions allowing eHealth to closely manual process of sharing and reviewing monitor progress, approve the vendor’s delivery patient information. strategy and hold the vendor more accountable Participation in the BDDI is voluntary. As of • for delays. April 2012, only about half of Ontario physicians were taking part in it.

RECOMMENDATION 1 DIABETES PREVENTION AND 3.03 HEALTH PROMOTION To allow for efficient and effective diabetes sur-

Section veillance at the provincial level and to gauge the Type 1 diabetes is not preventable. But the opposite

progress of the Ontario Diabetes Strategy, the is true for Type 2, which accounts for 90% of the VFM

•

Ministry of Health and Long-Term Care (Min- diabetic population. The World Health Organiza- 3 istry) should work closely with eHealth Ontario tion (WHO) says a healthy high-fibre and low-fat (eHealth) and Infrastructure Ontario to: diet, along with at least 30 minutes a day of Chapter • ensure that eHealth’s initiatives for chronic- physical activity, can reduce the risk of Type 2 dia- disease prevention and management are betes by 50%. The WHO also notes that more than implemented with an appropriate quality 90% of Type 2 cases can be prevented or postponed assurance process so that they meet the with good nutrition, regular physical activity, smok- needs of physicians and other users; and ing cessation and effective stress management. • implement measures based on lessons According to the Diabetes Expert Panel estab- learned from using the total outsourcing lished by the Ministry in 2006, “keeping people well system development model for the Diabetes and preventing disease is the most cost-effective, Registry if this procurement process is used affordable, and sustainable strategy for coping with for future information technology projects. chronic disease.” The Canadian Diabetes Associa- tion 2008 Clinical Practice Guidelines also noted MINISTRY RESPONSE that “preventing Type 2 diabetes would result in significant public health benefits, including lower Quality assurance with respect to electronic rates of cardiovascular disease, renal failure, blind- health tools has been, and will continue to be, ness, and premature mortality.” a priority of both the Ministry and eHealth. Despite the challenges the vendor has faced with the Diabetes Registry, eHealth has made Diabetes Management Strategy 93

Coverage of Diabetes Prevention Initiatives set priorities. Such evaluations involve ongoing quantitative analysis of the impact of preven- The Ministry has earmarked only 3% of the tion initiatives on mitigating the risk factors for $648 million funding allocation of the Ontario Dia- diabetes, which include unhealthy diet, physical betes Strategy (Strategy)—about $19 million—for inactivity, and being overweight or obese. prevention (see Figure 3). The Ministry informed The Ministry informed us that it developed no us that this funding for diabetes prevention initia- performance measure under the Strategy to mon- tives under the Strategy focused on reducing the itor the extent of unhealthy eating among Ontar- risks of new diabetes cases. The funding targeted ians because scientific literature has not established prevention initiatives at 24 local agencies in selected unhealthy diet as an independent causal risk factor communities with high prevalence of diabetes and for diabetes. It has monitored two other risk fac- large concentrations of ethnic groups at high risk tors, body weight and physical inactivity. Although of developing diabetes (for example, Asians and the Ministry has not developed targets for these Aboriginal people). We noted that: two measures as part of the Strategy, it said that Prevention initiatives in the Strategy targeted • its ACTIVE2010 Strategy in 2004 set a target of the regions of Toronto, Peel, North East and reducing the proportion of the population that is North West, even though statistics from the physically inactive to 45% from 52% by 2010. Institute for Clinical Evaluation Sciences, a However, as shown in Figure 5, these two meas- leading independent research organization ures have remained relatively constant in recent in Canada, noted that other Local Health years. More specifically, the October 2011 Report

Integration Network (LHIN) regions, such as 3.03 of the Strategy Key Performance Measures said Central East and Erie St. Clair, also have an that “the proportion of adults who are overweight

equally high prevalence of diabetes. Section or obese has increased significantly since 2003/04 The Ministry achieved its target of reach- VFM • (from 49.6% to 53.0%)” and “the proportion of •

ing 40,000 at-risk people in communities 3 physically inactive adults has not improved since with a high prevalence of diabetes through 2003/04, fluctuating from 50% to 53%.” community-based primary prevention initia-

In May 2011, the Ministry commissioned an Chapter tives but acknowledged that it has little reliable external evaluation of its diabetes prevention information about the size and distribution of initiatives. The evaluation concluded that the the entire at-risk population in Ontario. initiatives had limited impact on increasing public Almost two-thirds of Diabetes Education Pro- • awareness of diabetes and on changing behaviours. grams that responded to our survey said they In response, the Ministry conducted a diabetes thought more of their resources should go to forum in January 2012 with the local agencies that educating people at high risk of developing it funds to do diabetes prevention. The agencies diabetes. However, they have been unable had two major suggestions, which they say are to do this because their funding from the needed to promote the long-term success of the Ministry is intended to assist people already prevention initiatives: diagnosed with diabetes. • The Ministry needs to provide agencies with sustained multi-year funding commitments Effectiveness of Diabetes and avoid delays in transfer of time-limited Prevention Initiatives funds to minimize implementation delays and difficulties in hiring and retaining staff. Policy makers need reliable evaluations of the The Ministry needs to improve its co-ordina- effectiveness of prevention initiatives to help them • tion and communication with the agencies. 94 2012 Annual Report of the Office of the Auditor General of Ontario

The agencies indicated that they have been us that it has screened about 4,600 people for “working in silos.” In order to avoid duplicate diabetic risk factors through its community-based efforts, the agencies suggested that the prevention initiatives. It has also implemented a Ministry needs to take a more proactive role project called the Ottawa Model for Undiagnosed in developing “a centralized understanding Diabetes aimed at identifying undiagnosed diabetes of who’s doing what, who needs additional in patients hospitalized for other reasons and con- help and support” and facilitating knowledge necting them to diabetes care in their community. exchange on best practices. The Ministry informed us that because this project The Ministry informed us that it planned to covered only about 500 patients at four hospital conduct a further evaluation of its diabetes preven- sites, it would be expanded to other hospital sites in tion initiatives during the 2012/13 fiscal year, the 2012/13 fiscal year. and planned to use the results of this evaluation together with information from the diabetes forum Development of a Comprehensive Health- to make changes to the prevention initiatives. promotion Strategy

The World Health Organization says a healthy Screening for Undiagnosed Diabetes lifestyle that includes healthy eating and physical People with undiagnosed diabetes are those who activity is a first line of defence in the prevention have developed the disease but have not yet been of Type 2 diabetes. In 2005, the government estab- identified as such by health-care providers. Accord- lished the Ministry of Health Promotion and Sport 3.03 ing to the Canadian Diabetes Association (Associa- (which was merged into the Ministry of Health tion) 2008 Clinical Practice Guidelines (Guidelines), and Long-Term Care in October 2011). In 2006, Section undiagnosed Type 2 diabetes may occur in nearly the Ministry introduced Ontario’s Action Plan for VFM 3% of the adult population. So at least 284,000 Healthy Eating and Active Living (HEAL). However, •

3 Ontarians may not know that they have diabetes. as noted previously, there has been no recent Studies indicate that Type 2 diabetes can remain improvement in the percentages of Ontarians who

Chapter without symptoms for up to 10 years, and that at are overweight, obese or physically inactive. the time of diagnosis, 20% to 30% of patients will In March 2011, the government established have already developed complications. Undiagnosed the Commission on the Reform of Ontario’s Public diabetes, therefore, will place more costly burdens Services to provide advice on the way government on the health-care system in the long run. delivers services. In February 2012, the Commission Since 2008, the Association has recommended released a report suggesting the government should: screening individuals as early as age 40 because • do more to promote healthy lifestyles; this has proved useful in detecting undiagnosed • establish a province-wide chronic-disease diabetes. As yet, there has been no specific strategy prevention strategy; for identifying undiagnosed cases across Ontario • take a more comprehensive approach to popu- through targeted screening for Type 2 diabetes and lation health; pre-diabetes (a blood-sugar level that is higher than • explore regulatory options for the food normal, but not yet high enough to be diagnosed as industry; Type 2 diabetes). • work with the federal government on nutri- The Ministry indicated that, to date, its focus has tion regulation; and been on screening for people with risk factors for • replicate British Columbia’s ActNow initiative. developing diabetes rather than identifying undiag- We noted that several well-respected organiza- nosed diabetes. In that regard, the Ministry advised tions, including the WHO, the Conference Board of Diabetes Management Strategy 95

Canada, and the Health Council of Canada, have RECOMMENDATION 2 also credited British Columbia’s ActNow initiative, with its cross-government and multi-sectoral To enhance the focus on prevention and early approach, as a best practice in health promotion detection of diabetes as long-term, cost-effective and prevention of chronic disease. strategies, the Ministry of Health and Long- We also noted that in April 2012, Cancer Care Term Care should: Ontario and Public Health Ontario published • re-assess whether allocating only 3% of total a report indicating that Ontario’s health-care dedicated diabetes funding to prevention spending focused on treating people after they initiatives is the most cost-effective long- became ill rather than on keeping them healthy term strategy; in the first place. The report noted that there are • devise ways to identify, on a more timely four key risk factors—unhealthy eating, physical basis, people with undiagnosed diabetes; and inactivity, tobacco use and alcohol consumption— • develop comprehensive health-promotion that are strongly related to people developing a strategies that focus on all Ontarians and chronic disease such as diabetes and that Ontario consider similar strategies used in other has developed a comprehensive strategy only to jurisdictions. reduce tobacco use. The report recommended a comprehensive chronic-disease-prevention strategy MINISTRY RESPONSE targeting the entire Ontario population using a In addition to the allocation under the Ontario whole-of-government approach that engages all

Diabetes Strategy, the Ministry has invested 3.03 sectors and levels of government, community more than $335 million annually in health- groups, businesses, educational institutions and

promotion programs to promote healthy eating, Section media. The Diabetes Education Programs and many prevent chronic diseases like diabetes, and VFM of the physicians who responded to our surveys, as •

reduce injury and addiction. However, the 3 well as the diabetes experts we interviewed, also Ministry will undertake a review of the funding mentioned that Ontario lacks a multi-faceted strat- allocation for diabetes prevention initiatives

egy to prevent chronic disease and obesity. Chapter under the Strategy. The Ministry informed us that the government The Ministry is also currently developing has introduced initiatives to support healthy eating, a provincial framework that will inform a co- including the Healthy Communities Fund Grant ordinated approach to diabetes screening in Program for the delivery of health-promotion community-based primary health care organiza- initiatives; the Student Nutrition Program, funded tions across Ontario. by the Ministry of Children and Youth Services; and The Ministry will continue to assess the com- the Northern Fruit and Vegetable Program, target- prehensive health-promotion and prevention ing elementary-school children in selected areas in strategies used in other jurisdictions. The Min- Northern Ontario. Also, in May 2012, the govern- istry currently participates in several federal/ ment assembled a Healthy Kids Panel to advise it provincial/territorial committees dealing with on the development of a childhood obesity strategy. issues such as healthy weights, sodium reduc- We noted, however, that the adult population could tion and tobacco control, to share information be helped by similar health-promotion initiatives on best practices. The integration of health pro- and a specific obesity strategy. motion within the Ministry will allow a renewed focus on broader health-promotion and diabetes prevention strategies. 96 2012 Annual Report of the Office of the Auditor General of Ontario

DIABETES EDUCATION PROGRAMS As well, the benchmark was developed on the assumptions that each patient would get five hours In recognition of the fact that providing appropri- of services a year, and that 60% of services would ate information to people with diabetes is essential be delivered through individual counselling and to the management of the disease, the government 40% in group sessions. However, the majority of initiated the Diabetes Education Programs (DEPs) the DEPs that responded to our survey indicated in 1992 and placed them in three settings: hospi- that these assumptions did not reflect their actual tals, community health centres and Family Health activities and needed to be reviewed. The Ministry Teams. Each DEP consists of one or more Diabetes informed us that it has initiated a benchmark Education Teams (Teams), and each Team consists review during our audit and was in the process of of a registered nurse and a registered dietician. finalizing an analysis report in August 2012. Teams may also have other health-care profession- We noted that about 90% of DEPs did not meet als, such as social workers, psychologists, foot-care the 1,000-patient caseload benchmark in each specialists, pharmacists and physiotherapists. fiscal year since 2008/09, and more than one- DEPs use counselling in groups and one-on-one to third of them failed to achieve even 50% of the promote self-care and help patients improve their benchmark in 2010/11. The Ministry informed us quality of life, minimize their symptoms, and pre- that in June 2011 it required any DEP that failed vent complications. to meet at least 50% of the benchmark to submit Given the extra funding available with the Strat- a Performance Improvement Plan. After our audit egy, the Ministry has expanded the DEPs by adding fieldwork, the Ministry indicated that it was in

3.03 101 new Teams since the 2009/10 fiscal year. Cur- the process of reviewing the results of those DEPs rently, the Ministry funds 322 Teams in 152 DEPs implementing the plans as part of its ongoing

Section across the province. Of the 152 DEPs, 47 in North- monitoring of the DEPs. ern Ontario were funded and managed on behalf VFM

•

of the Ministry by a not-for-profit organization that 3 has signed an accountability agreement with and Consistency and Quality of DEPs received funding from the Ministry. We noted that the Ministry has not adequately mon- Chapter itored the consistency and the quality of service provided by the DEPs across the province, other Monitoring of Diabetes than to evaluate them against the 1,000-patient Education Programs caseload benchmark. For example: Performance Evaluation of DEPs: • According to the Policies and Procedures Caseload Benchmark Manual for DEPs issued by the Ministry, DEPs The Ministry has monitored the DEPs using a case- are required to conduct routine audits of the load benchmark: each Team (one full-time regis- quality of care they provide on a regular basis tered nurse and one full-time registered dietician) and hand over all information about such in a DEP should have an active caseload of 1,000 audits to the Ministry on request. The Ministry patients or more per fiscal year. We noted, however, has given the DEPs no guidance on what the that this benchmark may no longer be representa- audits should cover or how often they should tive because it was developed in 2001, when there be conducted; nor has the Ministry requested were only 71 DEPs, compared to the current total any audit results from DEPs to determine if of 152. The program’s scope has also grown and they were in compliance with the require- evolved since 2001. ment. According to our survey of DEPs, the frequency of such audits varied, with about Diabetes Management Strategy 97

one-quarter of the DEPs saying they “have not with the Ministry, the organization is responsible done” any audits or “do not know” they are for administering the funds in a prudent and required to conduct such audits. effective way. • Physicians who responded to our survey indi- The organization transfers most of its ministry cated that high staff turnover and inadequate funding to the community-based diabetes service training for new staff at DEPs has affected providers, including the DEPs in the north, to quality of care. In addition, they were uncer- deliver diabetes care. The organization has given tain about the DEPs’ ability to give patients about $66 million to service providers between the appropriate advice. They also noted that DEPs 2006/07 and 2010/11 fiscal years, and transferred often promoted new drugs that had no proven the majority of these funds, about $44.5 million, to track record, and provided education of vary- adult DEPs. The Ministry requires the organization ing quality to patients. to have an accountability agreement with each • There is a certification program for the desig- northern DEP, and to collect quarterly and annual nation of Certified Diabetes Educator (CDE). reports summarizing caseloads, activities and However, since this program is voluntary, not financial information from each DEP. The Ministry every DEP has staff with the CDE designa- informed us that the organization has accounted tion. The Ministry informed us that it has for funding granted to the DEPs through an annual encouraged the DEPs to recruit staff with settlement and reconciliation process. However, our this certification and/or help staff obtain this review of the organization’s audited financial state- certification. However, it has not tracked or ments indicated that, since 2007, its auditors have 3.03 monitored the competence, skill levels and been unable to provide a normal “clean” audit opin- qualifications of DEP staff. ion of its financial statements because the auditors Section did not examine the records of the service providers VFM funded by the organization. The organization • Oversight of Northern Diabetes 3 informed us that it assumed the funds it gave the Education Programs service providers went to providing diabetes care.

The Ministry has an accountability agreement with Based on our review of the organization’s Chapter and provides transfer payments to a not-for-profit expenditures, we also noted some questionable corporation to fund and manage 47 adult DEPs in practices, some of which ran contrary to its own Northern Ontario, 35 pediatric diabetes programs, policies and/or to those in the Broader Public Sec- Northern Ontario aboriginal diabetes services, and tor (BPS) Expense Directive that became effective Regional Coordination Centres in the northeast April 1, 2011. Examples included: and northwest regions. The Ministry informed us • Since the 2009/10 fiscal year, the organ- that the organization is responsible for selecting ization has paid a consulting firm a total diabetes service–providers to meet its funded of $105,000, including a $5,000 monthly objectives as outlined in its accountability agree- “retainer fee” and reimbursements for ment with the Ministry. telephone, fax, meal and taxi expenses. In The organization is small, with about 10 staff October 2010, it signed a formal agreement located in three offices. The same senior manage- with the firm for such services as “providing ment has been in place since its creation in 1992. strategic advice on election strategizing and Over the past decade, annual ministry funding to membership mobilization,” and “developing the organization has increased from $7 million to relationships with relevant political and $20 million for expanding services across Northern bureaucratic decision-makers.” The organiza- Ontario. According to its accountability agreement tion informed us that its board of directors 98 2012 Annual Report of the Office of the Auditor General of Ontario

approved the engagement of the consulting 2011 due to difficulties in securing qualified firm for high-level strategic advice to ensure staff. This resulted in a lease termination that the government was aware of the organ- charge of about $12,000. The Ministry ization. However, the Ministry informed us informed us that although the organiza- that it never approved a budget for consulting tion had discussed with it the consolidation services of this nature and that the organiza- of its offices in Kenora and Thunder Bay, tion never sought ministry approval to reallo- the Ministry was unaware that the Kenora cate funding to these services. office had been closed and that the organ- • The organization had inadequate record- ization had incurred a relatively significant keeping practices relating to staff travel and termination charge. meal expenses. Almost all of the meal expenses • The organization paid about $2,000 to send we reviewed were supported only by credit- an executive assistant to Las Vegas for a card slips or handwritten notes rather than by management course designed for managers, the original itemized and detailed receipts, as supervisors and others with management required by the organization’s own policies and responsibilities. the BPS Expense Directive. Often, these records While we suggested that an enhanced oversight did not specify the names of the people who of this organization was required, the Ministry indi- attended or the purpose of the meals. cated that, as with any broader-public-sector organ- • The organization’s policies did not align ization it funds, its oversight role does not include with the rules in the Ontario Public Service monitoring expenses incurred by the organization’s 3.03 (OPS) regarding consumption of alcohol and staff. The Ministry said it expected organizations to reasonableness of meal expenses. Although have proper policies and procedures in place, but Section the organization informed us that it did not indicated that ensuring adherence to the applicable VFM encourage the claiming of alcohol on expense policies and directives is the responsibility of the •

3 charges, we found instances where staff did organization’s board of directors. so. As well, the organization’s policies did Subsequent to our audit, the Ministry informed

Chapter not specify the maximum amounts per meal. us that it had taken some actions, including the We also noted instances where staff claimed recovery of about $40,000 in unapproved expendi- meals costing up to $80 per person. tures from the organization in June 2012. The • Travel expenses claimed by the organization’s Ministry also followed up in areas of the organiza- CEO amounted to about $40,000 each year. tion’s non-compliance with requirements of the When travelling to Toronto, the CEO received Broader Public Sector Accountability Agreement $100 per day for accommodation in a private in August 2012. home. This was four times more than the per diem rate of $25 specified in its policies. RECOMMENDATION 3 Although the BPS Expense Directive does not To ensure that Diabetes Education Programs specifically address this per diem, it does say (DEPs) provide diabetes patients with consistent that “due to mandatory requirements, and the and quality care, and in compliance with applic- principles of transparency and accountability, able policies, the Ministry of Health and Long- it is clear that per diems would no longer be Term Care should strengthen its oversight of allowed” and “in the OPS, per diems are no DEPs and other recipients of diabetes funding by: longer used.” developing appropriate service-delivery and The organization leased an office in Kenora • • cost-effectiveness measures and requiring in October 2010 but decided to close it in July Diabetes Management Strategy 99

CO-ORDINATION OF AND ACCESS TO DEPs to periodically report on these meas- DIABETES-CARE PROVIDERS ures; and • conducting periodic site visits to selected Co-ordination among DEPs regional, community and broader- As part of the Strategy, the Ministry’s Provincial public-sector organizations that receive Program Branch (PPB) has funded 101 new DEP diabetes funding. teams in hospitals, community health centres and Family Health Teams (FHTs) since the 2009/10 MINISTRY RESPONSE fiscal year. The Ministry informed us that the loca- The Ministry’s current agreements with transfer- tions of the new teams were based on proposals payment recipients require that local diabetes from the LHINs. At the same time, we noted that programs comply with and provide minimum the Ministry also funded other diabetes programs. service components. The Ministry, together with For example, FHTs have received funding from the the LHINs, will conduct a follow-up review of Ministry’s Primary Care Branch to set up chronic- the agreement framework to identify further disease management programs, which include opportunities to strengthen accountability. diabetes programs, and hospitals have allocated Apart from the measurement framework portions of their global funding to set up diabetes developed by the Ministry in partnership programs. However, the Ministry informed us that with Health Quality Ontario and the Regional the PPB maintained records only for PPB-funded Coordination Centres in 2011/12, the Ministry DEPs and was unable to confirm which other FHTs 3.03 will develop additional DEP performance and hospitals have also funded diabetes services. measures and reporting requirements that are The DEPs we visited, and those that responded Section aligned with the attributes defined in the meas- to our survey, indicated that they have been unable VFM urement framework. to meet the Ministry’s 1,000-patient caseload •

3 The Ministry will review how it can apply benchmark because their catchment areas over- greater oversight of broader public sector organ- lapped with those of other diabetes programs. This

izations receiving funding for diabetes services, situation has led to under-utilization of many DEPs Chapter and will continue to take timely, appropriate and even competition among DEPs for diabetes action when non-compliance with agreements patients as they attempt to meet the benchmark. In is identified. The Ministry is developing further fact, all of the DEPs we visited were located close to monitoring activities such as refinements to other diabetes programs. For example: its annual reconciliation process, developing a • One DEP was located within a five-minute protocol for periodic site visits, and examining walk of two FHTs with diabetes programs and the potential of obtaining attestations from the within a 10-minute drive of another DEP. organizations’ boards to ensure their compli- • A DEP was located in the same hospital as a ance with the requirements of the Transfer Pay- registered dietician who also delivered educa- ment Accountability Agreements. tion about diabetes management in a hospital- based clinic. • One significantly under-utilized DEP was located in a rural area that also had four other DEPs covering the same catchment area. 100 2012 Annual Report of the Office of the Auditor General of Ontario

Co-ordination between DEPs and Diabetes Management Incentive Physicians for Physicians

The Canadian Diabetes Association 2008 Clinical In April 2006, the Ministry introduced a Diabetes Practice Guidelines recommended diabetes self- Management Incentive (DMI) to promote qual- management education for all people with diabetes. ity diabetes care. DMI is a $75 annual payment In 2011, a team of researchers from two Ontario to physicians for co-ordinating, providing and universities studied physicians’ patterns of refer- documenting all required elements of care for each ring their patients to DEPs in a suburban region diabetes patient consistent with the Canadian Dia- of southern Ontario. With the assistance of the betes Association 2008 Clinical Practice Guidelines. Ontario College of Family Physicians, we extended Ontario Health Insurance Plan (OHIP) data showed this study by surveying all family physicians in the that the Ministry made about $95 million in DMI province. Both the 2011 study and our survey found payments to physicians since 2006. However, we that many physicians did not refer all of their dia- noted that the impact of the DMI in encouraging betes patients to DEPs. The most common reasons physicians to provide continuous and co-ordinated given for not referring were “patients unwilling diabetes management was unclear. Specifically: to go” and “physicians able to provide education • Forty-six per cent of physicians who in-house.” The main reasons that patients were responded to our survey indicated that the unwilling to go were “times unsuitable, with no DMI had no impact on the way they managed evening or weekend services” and “language bar- their patients. Of these, about 14% did not

3.03 riers.” These reasons were in line with our survey even realize the DMI existed.

of DEPs, which found that almost all of them were • The October 2011 report on the Ontario

Section closed evenings and weekends, and about half did Diabetes Strategy Key Performance Measures

not offer language-specific services. showed that the DMI was claimed for less VFM

•

As well, our DEP and physician surveys than 30% of diabetes patients in each fiscal 3 suggested that lack of communication and co- year since 2008/09, with only a slight increase ordination between the DEPs and physicians has during the period, as illustrated in Figure 5. Chapter been a barrier to provision of diabetes care. About This suggested a lack of awareness or usage of 60% of the DEPs that responded to our survey the DMI. noted that co-ordination and communication with • In April 2012, researchers from the University other health-care providers, especially physicians, of Toronto released a study reporting little has been a major challenge. Even though lab test difference in overall performance before results on blood sugar and cholesterol could help and after the introduction of the DMI. They the DEPs monitor their patients, 43% of the DEPs found that those physicians already providing in our survey said they received lab test results higher-quality care simply continued to do so only half the time or less from physicians and other while claiming the DMI, and they concluded health-care providers. Physicians, for their part, that the DMI “led to minimal improvement in noted they often did not receive information from quality of diabetes care at the population and local DEPs regarding the progress of patients the patient level.” DEPs were seeing. Diabetes Management Strategy 101

Access to Specialized Diabetes-care Diabetes Regional Co-ordination Centres Providers After Cabinet approved the Regional Coordination A multidisciplinary team approach involving Centre (RCC) program in October 2009, the Min- different health-care providers has been proven istry established a diabetes RCC in each of Ontario’s effective in delivering diabetes care. According to 14 Local Health Integrated Networks (LHINs) to the Canadian Diabetes Association 2008 Clinical promote access to co-ordinated diabetes services Practices Guidelines, “a team of health-care profes- regardless of where in Ontario people live. The sionals—including physicians, nurses, diabetes Ministry and each LHIN selected a host agency (a educators, pharmacists and other health-care hospital, community health centre or Community experts who work together with the individual liv- Care Access Centre) to manage the RCC. The RCCs ing with diabetes—is the recommended approach do not deliver diabetes services directly, but work to achieve optimal care.” The Diabetes Expert Panel with their respective LHINs and health-care provid- appointed by the Ministry in 2006 also noted that ers to co-ordinate diabetes care in a region. the “interdisciplinary team is critical to the success- The RCCs have been in place only since the ful diabetes management.” 2009/10 fiscal year, so it is not yet possible to assess Our survey of the DEPs indicated that the their long-term impact. We did note that the RCCs levels of care and access to specialists varied from have made some progress in improving regional one DEP to another. Only 25% of the DEPs that co-ordination of diabetes services. However, the responded to our survey said they were funded for four RCCs we visited raised the following concerns,

specialists other than registered nurses and regis- which they said must be addressed to enhance their 3.03 tered dieticians. Although it is unlikely that there is effectiveness. Among them:

sufficient funding to staff every DEP with an array There is a need for clarity and standardization Section • of specialists, certain specialists are critical to dia- across regions regarding the interpretation VFM

• betes patient care. For instance, 60% of DEPs in our of RCC deliverables, such as depth of support 3 survey identified foot-care specialists as the most- to service providers. However, the Ministry needed professionals, but 40% said their patients informed us that variation across the RCCs is had no ready access to these specialists. About half needed to reflect local and regional needs. Chapter of the physicians who responded to our survey also • Communication of RCC roles at the provincial identified “lack of timely access to foot-care special- level has been limited or ineffective. Although ist” as one of the most common problems in caring the RCCs have also made an effort to promote for patients with diabetes, for whom foot infections themselves, they acknowledged that many are a major reason for hospitalization. According to stakeholders, especially physicians, were still a report published in April 2012 by the Institute for not aware of their role and mandate. The Clinical Evaluative Sciences, a leading independ- Ministry indicated that it provided formal ent Canadian research organization, about 74 per communication about the RCCs in quarterly 10,000 Ontarians with diabetes have had lower newsletters, at conferences and events, and extremity amputations that were mainly triggered through a refreshed diabetes website. by foot infections resulting from poor circulation • The RCCs have been leading regional diabetes and nerve damage caused by diabetes. planning and co-ordination by working closely with the DEPs, but the RCCs and the DEPs report to different branches of the Ministry, hindering efforts to evaluate their relative effectiveness. 102 2012 Annual Report of the Office of the Auditor General of Ontario

Despite monthly teleconferences and quar- • munity health centres, hospitals and Family terly face-to-face meetings, there has not been Health Teams when allocating diabetes fund- enough dialogue between the Ministry and ing and other resources to avoid duplication the RCCs. For example, we noted cases where or under-utilization of services; one branch of the Ministry did not consult evaluate the need for the Diabetes Man- and inform the RCCs when making changes • agement Incentive, given the evidence at the DEPs. As well, although one branch indicating its lack of impact on encouraging of the Ministry asked the RCCs to do patient physicians to provide continuous and co- consultations, the RCCs found that another ordinated diabetes management; and branch had also initiated a similar survey. The monitor whether people have timely and RCCs also informed us that there is a need to • equitable access to diabetes-care specialists increase the Ministry’s participation in RCC in high demand, such as foot-care specialists, stakeholder meetings to address the issues especially where there is evidence that a and barriers faced by the DEPs. lack of timely treatment is likely to result in The RCCs have been developing their own • hospitalization. tools and processes, such as patient surveys, performance indicators and guidelines for MINISTRY RESPONSE leading practices, concurrently, resulting in duplication of effort and inconsistencies in The Ministry will undertake a review of its reporting and measurement of effectiveness. current operating framework to identify oppor- 3.03 The Ministry informed us that since varia- tunities to further reduce potential duplication tions exist across the RCCS, there is a need and/or under-utilization. The current RCC Section for flexibility to address local and regional program leverages information on the demand VFM circumstances. for, and availability of, diabetes services within •

3 The Ministry acknowledged these issues and local regions to drive effective and efficient co- indicated that they are being addressed. In April ordination, integration and delivery of services.

Chapter 2012, for example, the Ministry in partnership with The Ministry continues to benefit from the RCCs and Health Quality Ontario finalized a expert panels in strengthening primary care, Measurement Framework that included measures which will be extended to include advice from to track and monitor regional performance. The the expert group on all physician incentives and Ministry advised us that it is working with the RCCs their effectiveness, including the Diabetes Man- to identify common indicators for these measures in agement Incentive. the 2012/13 fiscal year, and it has formally clarified The Ministry will, as appropriate, include expectations and respective roles of the RCCs and equitable access to diabetes care specialists as their host agencies. part of its regular review and refinement of the Ontario Diabetes Strategy. RECOMMENDATION 4

To improve co-ordination among diabetes-care providers and access to specialized diabetes BARIATRIC SURGERY care, the Ministry of Health and Long-Term About 90% of people with Type 2 diabetes are Care should: overweight or obese, according to the World Health take into account the demand for and avail- • Organization and the Canadian Diabetes Associa- ability of diabetes services offered in com- tion. The International Diabetes Federation also Diabetes Management Strategy 103 noted that the risk of developing Type 2 diabetes Since the expansion of the capacity to perform is 93 times higher for obese women and 42 times them in-province, about 5,200 bariatric surgeries higher for obese men than for those at a healthy were performed in Ontario between 2009/10 and weight. 2011/12. Annually, the number of bariatric surger- Although bariatric surgery by itself does not ies done in Ontario increased by 180%, from about “cure” obesity, it does result in substantial, sus- 890 in the 2009/10 fiscal year to about 2,500 in tained and long-term weight loss by removing 2011/12. However, the total number of people get- part of the stomach and/or the small intestine. As ting the surgery actually dropped 14%, from about one would expect, post-surgical follow-ups and 2,900 in the 2009/10 fiscal year to about 2,500 in ongoing diabetes management are critical to main- 2011/12, as illustrated in Figure 6. This is because tain weight loss. Studies have consistently dem- significantly fewer out-of-country bariatric surger- onstrated that bariatric surgery is cost-effective ies are being approved. for severely obese diabetes patients and stabilizes In February 2011, the Ministry announced Type 2 diabetes in 60% to 80% of patients within a that the expansion of bariatric services in Ontario year or less. would save the province about $45 million in out- of-country OHIP costs in the 2010/11 fiscal year. We noted that a more accurate estimate of savings Access to and Cost of Bariatric Surgery would be about $35 million after including the Prior to the Strategy, most Ontarians requiring bari- cost of additional bariatric surgeries performed in atric surgery received it in the United States, with Ontario. However, this figure does not reflect any 3.03 OHIP covering the costs. As part of the Strategy, potential costs of patients who could develop other the Ministry announced it would spend $75 mil- health complications as a result of waiting longer Section lion to increase Ontario’s capacity to do bariatric for bariatric surgeries in Ontario, as indicated in the VFM surgeries. The Strategy called for the establishment following sections. •

3 of the Ontario Bariatric Network, consisting of six Regional Assessment and Treatment Centres and Figure 6: Number and Cost of Bariatric Surgeries,

four Bariatric Centres of Excellence in hospitals 2005/06–2011/12 Chapter across Ontario. Additional funding brought the Source of data: Ministry of Health and Long-Term Care total to about $108 million by the 2011/12 fiscal Number of in-province Number of out-of-country 80 year. The number of in-province bariatric surgeries 3,000 Total cost has increased from 245 in the 2007/08 fiscal year 70 2,500 to 2,500 in 2011/12. 60 Since February 2010, the Ministry has required 2,000 geries 50 Ontarians seeking bariatric surgery to be assessed 1,500 40 at a Regional Assessment and Treatment Centre. 30 Patients would be approved for out-of-country 1,000 Cost ($ million) Number of Sur 20 bariatric surgery only if their referring physicians 500 applied for them and they met the regulatory 10 requirements for out-of-country health services. 0 0 The Ministry’s goal was to have more bariatric surgeries performed in Ontario to save money and 2005/06 2006/07 2007/08 2008/09 2009/10 2010/11 2011/12 to ensure that adequate assessments and follow-ups are being done. 104 2012 Annual Report of the Office of the Auditor General of Ontario

Capacity and Demand for Bariatric Surgery bariatric surgeries and to treat 1,650 potential bariatric surgery patients in medical or behavioural In May 2009, the Ministry estimated that 342,000 programs as an alternative to surgery. Ontarians would be eligible for bariatric surgery and assumed that 2% (about 6,800) of those would proceed to the surgery by 2013. The Ministry also Referral Process and Wait Times of projected about 3,000 new referrals per year, based Bariatric Surgery on data from 2005, then the most recently avail- Alberta, British Columbia, Quebec and Saskatch- able. Ministry data showed that in the 2011/12 ewan also fund bariatric surgery in their provinces. fiscal year, about 2,500 bariatric surgeries were Cross-jurisdictional comparison of the processes performed in Ontario, as illustrated in Figure 6. and wait times is not possible because policies and This exceeded the Ministry’s current target of 2,400 practices vary from one province to another. For but is not yet meeting the projected demand. instance, Ontarians seeking bariatric surgery follow Our research also indicated that the actual a standard, centralized referral process to better demand for bariatric surgery has significantly ensure consistent assessment and prioritization of exceeded the forecasts. In the 2011/12 fiscal year patients, as illustrated in Figure 7. The referral pro- alone, there were about 8,000 new referrals, or cess in other provinces has not been centralized and 2.6 times more than the ministry forecast. We each surgeon manages his or her own wait list. also noted that wait lists for bariatric surgery in The Ministry collected wait-time data for Ontario suggest demand has far outpaced capacity. bariatric surgery from the hospitals. However, we

3.03 According to hospital data for the fiscal years noted that two hospitals each reported identical between 2009/10 and 2011/12, there were about wait times for every month from December 2010

Section 22,000 referrals. The Ministry informed us that to November 2011, but neither was able to provide about 70% of these referrals, or about 15,400 VFM us with any documentation to support the wait •

cases, would proceed to surgery, but we noted that 3 times they reported to the Ministry. The Ministry only about 5,200 bariatric surgeries were actually indicated that it was likely due to the transition performed. The remaining referrals were either

Chapter from a paper-based referral process to a centralized awaiting or undergoing pre-surgical assessment electronic system. Nevertheless, this made us ques- and preparation procedures. This gap will likely tion whether the wait-time data being collected widen if demand continues to grow more quickly by the Ministry was reliable, and accordingly, we than capacity. We noted that the average overall measured wait time by reviewing patient files at wait time was 12 months if the surgery was done in the hospitals. Ontario, compared to six months if the surgery was As illustrated in Figure 7, the Ministry measured approved to be done outside Ontario, mainly in the only the wait time from surgeon’s approval to actual United States. surgery, and did not include the period prior to The Ministry informed us that it has been dif- surgeon’s approval. According to the Ministry, this ficult to make accurate demand forecasts due to the is the standard for tracking wait times for surgeries elective nature of bariatric surgery. To address this in Ontario. Using the Ministry’s wait-time meas- challenge, the Ministry informed us that it has cre- ure, the average surgical wait time was only two ated a clinical registry to monitor quality and wait months. However, our review found that patients times. It also said that it will continue to allocate waited an average of 12 months for surgery if the more resources to bariatric surgery. In the 2012/13 wait time was measured from the time of their fiscal year, for example, there will be a one-time physician’s original referral to the actual surgery, as funding allocation of about $48 million, and illustrated in Figure 7. Ontario hospitals will be expected to perform 2,580 Diabetes Management Strategy 105

Figure 7: Referral Process for Bariatric Surgery in Ontario Prepared by the Office of the Auditor General of Ontario

12 months

4 months 6 months 2 months

Physican’s Central Hospital Orientation Assessments of patient Surgeon’s Bariatric referral system received by health-care referral class at approval surgery (BariatricRegistry) referral hospital professionals Ministry’s Measure: Surgical Wait Time

Realistic Measure: Overall Wait Time

Our review of patient files and hospital data referrals from the 2009/10 to 2011/12 fiscal showed that the significant backlog for bariatric years) were waiting for or undergoing assess- surgery was due not to a shortage of surgeons and/ ments as of March 31, 2012. 3.03 or operating rooms but rather to the long time that While the Ministry’s approach to reporting wait patients waited between the original physician’s times for bariatric surgery is the same as that used Section referral to when the surgeon approved the surgery. for all surgeries in Ontario, we believe additional VFM Specifically: information regarding times that patients have to •

3 • Patients waited between two and seven wait for orientation and assessment would provide months for a hospital orientation class, which the public with a better understanding of the

bariatric surgery requires, possibly because referral process and what to expect. However, we Chapter classes were small and/or infrequent, or also noted a McGill University Health Centre study because of confusion among physicians about published by the Canadian Medical Association in the referral process; physicians sometimes June 2009 which said that although most health mistakenly sent referral forms to hospitals authorities defined wait time as the period between rather than to the central referral system. a surgeon’s approval and either hospital admission Hospital data showed that about 2,600 or actual surgery, “this same definition is inappro- patients (representing about 12% of total priate for bariatric surgery because the enormous referrals from the 2009/10 to 2011/12 fiscal numbers of patients requesting this surgery would years) were waiting for orientation classes as lead to a clinically unacceptable period from declar- of March 31, 2012. ing a patient fit for surgery to the actual surgery. • Patients waited between five and eight The realistic wait time for bariatric surgery must be months for specialist assessments. The waits defined as the overall wait time.” depended on availability of hospital resources and level of patient needs, with some patients Quality of Bariatric Services requiring more assessments before surgery. Hospital data showed that about 10,500 We found that the Ministry did not have adequate patients (representing about 48% of total procedures in place to assess whether the referral 106 2012 Annual Report of the Office of the Auditor General of Ontario

process for bariatric surgery operated as intended a physician’s referral and completion of the and whether patients received quality care. We required pre-surgery assessments; and noted, for example, that since the 2009/10 fiscal periodically monitor surgical outcomes to year, 29 bariatric surgeries were performed at • determine whether hospitals offering this hospitals not designated as a Bariatric Centre of surgery need to go through an accreditation Excellence. The Ontario Bariatric Network also process as hospitals in the United States do. informed us that it raised questions with the Ministry about the quality of services provided by MINISTRY RESPONSE non-designated hospitals. Since bariatric surgery has become more com- The Ontario Bariatric Network (OBN) reviews mon, there have been growing concerns about the trends of demand and capacity on a regular uneven quality across hospitals. In the United basis and provides updates and recommenda- States, the American College of Surgeons Bariatric tions to the Ministry. In 2009/10, the Ministry Surgery Center Network (College) and the Amer- established the Bariatric Registry to provide reli- ican Society for Metabolic and Bariatric Surgery able data and to track both care-path timelines (Society) accredit hospitals that perform bariatric and performance measures, and will continue surgery. In contrast, no professional medical bodies to refine the use of this data. In partnership with in Canada offer similar accreditation for bariatric hospitals and the OBN, adjustments are made surgery. Bariatric surgical sites in Ontario could on a continuous basis, such as transferring also obtain accreditation from the United States, patients between centres to reduce wait times, 3.03 but of this province’s eight sites, only two have modifying referral zones and allocating addi- obtained such an accreditation, both from the tional resources, where appropriate, in order to Section College. However, the Ministry informed us that it increase surgical volumes. VFM does not support U.S.-based accreditation, as the Patients are advised of the referral process •

3 U.S. accreditation bodies require mandatory data (including full wait-time information) and what submission, which would impose costs and admin- they can expect during the information sessions

Chapter istrative burdens on Ontario’s hospitals. The Min- that all patients are required to attend. The Min- istry also indicated that in the absence of Canadian istry will consider options for advising the pub- accreditation, the Ontario Bariatric Network has set lic on full wait times from the original date of provincial standards and protocols for continuous referral. The Ministry currently publishes wait quality improvement and monitoring of bariatric times for bariatric surgery through the Ontario services in Ontario. Wait Times Information System. Between April 2012 and June 2012, the wait time for bariatric RECOMMENDATION 5 surgery once a surgeon had approved it was 85 days, which is well within the general surgery To ensure that people receive adequate, timely target of 182 days. and quality bariatric surgical services across the The Ministry will continue to regularly province, the Ministry of Health and Long-Term monitor surgical outcomes of this program, Care should: including reviewing quality benchmarks and review trends of demand and capacity for • monitoring improvement plans as required for bariatric surgery to identify gaps and needs, any bariatric centre. especially on a regional basis; • consider providing the public with information on the average elapsed time between Chapter 3 Ministry of the Environment Section 3.04 Drive Clean Program

ents—one for light-duty vehicles (passenger cars Background and sport utility vehicles) and one for heavy-duty vehicles (buses and trucks). The program currently tests vehicles once they are seven years old, Smog is a form of air pollution that poses a serious or those older than one year if ownership is to be health threat to Ontarians. According to the Min-

transferred. Light-duty vehicles that were built 3.04 istry of the Environment (Ministry), approximately before 1988 are exempt from the program, but, half of Ontario’s smog comes from pollutants that

otherwise, all vehicles must pass an emissions test Section originate in the United States and are transported for the owner to renew the registration or transfer VFM here by winds. The other half, however, comes from •

ownership. When a vehicle passes its emissions 3 domestic sources, including utilities (for example, test, the testing facility issues a uniquely numbered power plants), industries (for example, metal emissions test certificate, which the vehicle owner

smelters and petroleum refineries), on-road motor Chapter must take to a ServiceOntario kiosk or office (that vehicles and other forms of transportation (for is, a vehicle-licensing office) when applying to example, trains and aircraft). Vehicles also contrib- renew their licence plates or to transfer ownership. ute to greenhouse gas emissions (such as carbon As of December 2011, approximately 7.6 million dioxide) and toxic contaminants (such as carbon light-duty vehicles and almost 300,000 heavy-duty monoxide and benzene), which also adversely vehicles were registered in Ontario. About 90% affect air quality. For more than a decade, the of these vehicles are registered in the geographic Ministry has been implementing a number of initia- area covered by the program. Annually, more tives aimed at helping to reduce smog. One of these than 2.5 million light-duty vehicles and more than initiatives has been the Drive Clean program. 100,000 heavy-duty vehicles are subject to the Drive Clean, which was introduced in 1999, is Drive Clean test. Ontario’s mandatory vehicle emissions inspection More than 30 jurisdictions in North America and maintenance program. The purpose of the pro- have a vehicle inspection and maintenance program is to identify high-pollution-emitting vehicles gram for emissions. In Canada, British Columbia with missing or malfunctioning emission controls has been testing light-duty vehicles since 1992, and require the owners of such vehicles to have and recently announced that it will phase out that them repaired, with the ultimate goal of reducing program by the end of 2014 and start testing heavy- on-road emissions. The program has two compon-

107 108 2012 Annual Report of the Office of the Auditor General of Ontario

duty diesel vehicles only. Quebec, on the other In 2012, the Ontario government commissioned a hand, introduced a bill in December 2011 to begin review on reforming public services in the province. emissions testing for light-duty vehicles by the end The resulting report, commonly referred to as the of 2013. Drummond Report, recommended that the govern- Ontario’s Drive Clean program operates under ment consider delivering the Drive Clean program the authority of regulations made under the via a “delegated administrative authority” model. A Environmental Protection Act and the Highway Traf- delegated administrative authority is a private, not- fic Act. The program is administered jointly by the for-profit corporation that administers legislation on Ministry of the Environment, the Ministry of Trans- behalf of the government under an accountability portation and ServiceOntario, with the Ministry of and governance agreement. This entity would the Environment being ultimately accountable for assume responsibility for all aspects of the program’s its performance. day-to-day decision-making and service delivery that The Ministry of the Environment’s Drive Clean are now being handled by the Drive Clean Office and Office, with a staff of approximately 25 people, its private-sector service provider. develops regulations, sets standards, establishes policies and procedures, and contracts with service providers to deliver various aspects of the program. Emissions tests and/or repairs are performed at Audit Objective and Scope more than 2,000 Drive Clean facilities, which are private auto shops accredited by the Ministry. All The objective of our audit was to assess whether 3.04 testing facilities are electronically linked to the the Ministry has adequate systems and procedures Ministry’s Drive Clean database, which maintains a in place to ensure compliance with legislation Section record of all tests and any related repairs made. and regulations related to Drive Clean, and to VFM The Ministry has contracted with a private- determine and report on whether the program is •

3 sector service provider to administer all operational effective in reducing vehicle emissions and thereby Drive Clean program activities. This involves contributing to improved air quality. Senior Min-

Chapter supplying and servicing emissions testing equip- istry management reviewed and agreed to our audit ment, training inspectors and repair technicians, objective and associated audit criteria. ensuring quality control practices at Drive Clean Our audit work was conducted primarily at the facilities, operating a call centre to provide tech- Ministry’s Drive Clean Office and at the private- nical support to Drive Clean facilities and handle sector service provider. In conducting our audit, public comments, and developing and operating an we interviewed appropriate ministry and service- information system to support all of the program’s provider staff, reviewed relevant documents, ana- functions, including linking to the Ministry of lyzed information and reviewed specific controls of Transportation’s licensing system. new testing equipment that is slated for rollout in Vehicle owners pay a fee to the Drive Clean January 2013. In addition, we employed a number facility that conducts their emissions test. A portion of computer-assisted audit techniques to analyze of this fee is remitted to the Ministry as revenue. two sets of data: the results of emissions tests and In the 2011/12 fiscal year, the Ministry collected the records on repair costs. $30 million in test revenue and spent approxi- We met with the Canadian Vehicle Manufactur- mately $19 million to deliver the Drive Clean ers’ Association and the Environmental Commis- program, of which $12 million was paid to the sioner of Ontario to obtain their perspectives on private-sector service provider. the Drive Clean program. To gain insight on how similar programs operate in other jurisdictions, we Drive Clean Program 109 reviewed studies and reports on vehicle inspection these factors, the Ministry further estimated that, and maintenance programs elsewhere in Canada since 2007, the Drive Clean Program has been and in the United States. Because the Ministry’s responsible for reducing smog-causing vehicle internal audit service team had recently performed emissions by about 36% annually. Initiatives in audits on Drive Clean revenue, we took its work other sectors (for example, changes in industrial into consideration in determining the scope and processes) have also contributed to the reduction in extent of our work in this area. smog-causing emissions. It is therefore critical that, on a go-forward basis, policymakers have relevant and up-to-date information on the actual impact of the Drive Clean Summary program in reducing smog compared to the impact of other smog-reducing initiatives. The Drive Clean program has implemented effect- There are a number of issues that we noted dur- ive procedures to ensure that vehicles that should ing our audit: be tested are getting tested, and that vehicles • The worst polluting vehicles either are exempt whose emissions systems have deteriorated to the from emissions testing or will be tested using point where their emissions exceed the province’s a less stringent method. The program’s light- limits are being identified for repair. The Ministry duty component does not require vehicles has made some headway in refining the program’s built before 1988 to be tested, even though features—for example, increasing the age at which they would likely have about a 30% failure 3.04 vehicles must begin emissions testing, extending rate. In addition, the on-board diagnostic testing to include vehicles that in 2012 were as old (OBD) testing method that is slated to begin Section as 24 years, changing the vehicle emissions testing January 1, 2013, cannot be used to test VFM method to the on-board diagnostic testing method vehicles built before 1998 because these •

3 that is now used in all other North American juris- vehicles were built without OBD systems. As dictions with similar programs, and consolidating a result, vehicles built from 1988 through

six alternative service delivery contracts into one 1997, which experienced a failure rate from Chapter at an expected annualized cost savings of 40%. 11% to 31% in 2010 when tested with a The Ministry, together with the Ministry of Trans- dynamometer, will be tested using the two- portation, has also put in place procedures aimed speed idle method only—a method that uses at preventing the use of duplicate certificates, a less stringent emissions limits than either the fraudulent practice that was identified as a signifi- dynamometer or the OBD testing method. cant problem in our 2004 audit on air quality. • Since 2002, all gasoline-powered vehicles that On-road vehicle emissions have declined so meet the program’s age criteria and that are significantly from 1998 to 2010 that they are no located in the Windsor–Quebec City corridor longer among the major domestic contributors of are required to have emissions tests every two smog in Ontario. However, ministry emissions esti- years. There are 10 large municipalities in this mates show that more than 75% of the reduction in geographical area that account for about two- vehicle emissions since the program’s inception is thirds of all vehicles in Ontario. These muni- actually due to factors other than the Drive Clean cipalities generally also have a higher than program, such as tighter manufacturing standards average number of smog days. However, the on emission-control technologies, federal require- Ministry has never formally assessed whether ments for cleaner fuels and ongoing retirement there would be any significant impact on the of old vehicles. For emissions not eliminated by environment if vehicles not located in these 10 municipalities were not required to be tested. 110 2012 Annual Report of the Office of the Auditor General of Ontario

• Because vehicle owners are not required to ance, the Ministry has not established incur any repair costs if the repair estimate performance targets or attempted to measure exceeds $450, about 18,000 vehicles avoided whether or not this goal has been achieved in being fully repaired in 2011. In fact, the over a decade. The only survey undertaken to average amount paid for repairs in 2011 measure public support for the Drive Clean by the owners of vehicles that were given a program was conducted 12 years ago. conditional pass was only $255. Furthermore, in 2010 the most commonly diagnosed cause OVERALL MINISTRY RESPONSE of excessive emissions—problems with the The Ministry appreciates the Auditor General’s catalytic converter—was actually repaired in observations and recommendations regarding only one-third of cases. Without full repairs, the Drive Clean program and will continue to a vehicle’s emission control system will take actions to improve the program. continue to malfunction, and emissions will Poor air quality is a public health concern, fluctuate. We noted that, for vehicles that had particularly to children, the elderly and people partial repairs done to their emission systems who have respiratory challenges. Ontario in 2011, the emission readings after the repair protects air quality through its comprehensive were actually worse for all pollutants in 25% approach, which includes a combination of of the vehicles and for at least one of the pol- regulations, compliance and enforcement lutants measured in 50% of the vehicles. activities, monitoring, and Drive Clean. The • To uncover unscrupulous practices at Drive 3.04 approach tackles emissions from the electricity Clean facilities, the Ministry has, until sector, commercial and industrial facilities, recently, been diligent in requiring its service

Section vehicles, and cross-border sources. Our regula- provider to conduct upwards of 1,400 covert tions and targeted programs and partnerships VFM audits annually. In recent years, these and •

with other jurisdictions have all helped to 3 other audit efforts have been effective in iden- reduce air pollutants. To put the Drive Clean tifying approximately 3,000 non-compliance program’s mandate into context, on-road

Chapter issues annually. However, with the planned vehicles account for 27% of all emissions of introduction of a new compliance program to nitrogen oxides in the province and 13% of all coincide with the new OBD testing method emissions of volatile organic compounds. Since in 2013, the Ministry reduced the number of 2007, the program has consistently reduced covert audits in 2012 to a very small fraction vehicle emissions that cause smog and poor air of what the Ministry previously required quality by approximately 36% a year. the service provider to conduct. While a The Ministry notes the Auditor General’s significant decrease may well be justified observation that on-road emissions have been once the new compliance program is in place, continuously declining. Still, parts of Ontario we believe the Ministry should reconsider experience elevated smog levels. Since the whether such a dramatic reduction is cur- beginning of the program, Drive Clean has rently warranted, especially given that there reduced smog-causing pollutants by 335,000 is a deterrent effect when Drive Clean facility tonnes; carbon monoxide by about 3.18 million operators know that covert or “secret shop- tonnes; and carbon dioxide, a key greenhouse per” audits are being done. gas, by more than 296,000 tonnes. In 2003, Although one of the program’s key goals is • when operating at their peak, coal-fired gen- maintaining a high degree of public accept- erating plants emitted over 58,000 tonnes of Drive Clean Program 111

smog-causing pollutants. In 2010 alone, Drive Clean prevented 34,600 tonnes of smog-causing Detailed Audit Observations pollutants. A review of the program’s costs, benefits PROGRAM EFFECTIVENESS and effectiveness was an important driver in Overall Change in Ontario’s Air Quality the Ministry’s 2010 decision to modernize the Drive Clean program. The Ministry consolidated Smog is a form of air pollution that is composed multiple service contracts into one contract to primarily of ground-level ozone and fine particulate improve efficiency and effectiveness of delivery. matter. Ground-level ozone is created when nitro- The new vehicle emissions testing technol- gen oxides (NOx) and volatile organic compounds ogy will allow problems to be sourced faster (VOCs) react together in the presence of sunlight. and more effectively. We anticipate further Fine particulate matter (PM2.5) is microscopic solid decreases in smog-causing pollutants with our particles and liquid droplets in the air. new modernized program, and once the tech- Wind-blown pollution from the Midwestern nology is implemented in 2013, we will continue United States was and remains the largest source to monitor the program’s effectiveness. of smog-causing pollutants found in Ontario. How- The Ministry is taking action to address the ever, as Figure 1 shows, between 1998 and 2010, concerns raised by the Auditor General. The Ontario-based emissions from smog-causing pol- new approach to testing vehicle emissions is lutants (NOx, VOCs, and PM2.5) and carbon mon-

used in similar programs across North America. oxide have decreased by about 30% to 40%. The 3.04

It is expected to: Ministry’s report on air quality for 2010 credited the

provide better information to vehicle owners decrease in emissions to a combination of federal Section • about needed repairs; and provincial air quality initiatives: VFM

•

reduce the number of conditional passes; the phase-out of coal-fired generating stations 3

• • • ensure high-quality customer service thanks (between 2005 and 2014); to online real-time quality control and assur- • emission trading regulations (effective 2001 ance tools; and for the electricity sector and 2005 for the Chapter • continue to improve Ontario’s air quality by industrial sector); reducing emissions an additional 20%. • emission controls at Ontario smelters (effective 2005); • Drive Clean emissions testing (beginning 1999);

Figure 1: Changes in Province-wide Domestic Emissions and Contribution by Road Vehicles to Those Changes, 1998–2010 Source of data: Ministry of the Environment

Change in Emissions % of Total Domestic Emissions Source of Most from Domestic Contributed by Road Vehicles Domestic Emissions Pollutant Sources (%) 1998 2010 (2010) Nitrogen oxides 41 i 34 25 Non-road transportation Volatile organic compounds 38 i 21 12 General solvent use Particulate matter 39 i 4 3 Residential Carbon monoxide 31 i 62 44 Road vehicles 112 2012 Annual Report of the Office of the Auditor General of Ontario

• federal vehicle manufacturing standards for sion levels, ambient air quality measures provided emissions (phased in from 2004 through mixed results. That is, for calendar years 2008 2010); and through 2010, Ontario did not exceed the Canada- • the federal requirement for lower sulphur con- wide standard for PM2.5, but did exceed the tent in transportation fuels (effective 2005 for Canada-wide standard for ozone in each of those gasoline fuel and effective 2006 for diesel fuel). years, although ozone concentrations have been Just before the Drive Clean program started, decreasing for some time. vehicles were the number one domestic source When smog levels are (or are expected to of emissions for carbon monoxide and nitrogen become) elevated for several hours, the Ministry oxides, and the number two domestic source of issues a smog advisory. The total hours spent under emissions for volatile organic compounds. As of a smog advisory during a particular time period can 2010, vehicles were still the primary domestic be expressed as “smog days”: every 24 hours under source of carbon monoxide emissions but were no a smog advisory equals one smog day. We analyzed longer the primary source of domestic emissions the number of smog days experienced province- for any of the smog-causing pollutants listed in wide each year since the Drive Clean program’s Figure 1. For each year from 1998 through 2010, inception and noted that smog days varied con- the vehicle sector experienced either the largest or siderably from year to year but have been generally the second-largest decrease in emissions for each trending down. of these pollutants, and total vehicle emissions decreased more than 50% in that time. Other 3.04

Impact of the Drive Clean Program sources—such as non-road transportation (which includes air, rail and marine transportation)— To determine the effectiveness of the Drive Clean Section demonstrated the least reduction in smog-causing program, the Ministry has engaged an external VFM emissions, as shown in Figures 2 and 3. consultant to calculate emissions reductions. Gen- •

3 We also looked at ambient air quality measures, erally, emissions reductions are calculated based on which monitor the concentration of contaminants the difference between a vehicle’s actual emissions

Chapter in the air for a select period of time. Unlike emis- readings before and after repairs are conducted

Figure 2: Nitrogen Oxide Emissions, by Domestic Figure 3: Volatile Organic Compound Emissions, by Emission Source, 1998–2010 (Kilotonnes) Domestic Emission Source, 1998–2010 (Kilotonnes) Source of data: Ministry of the Environment Source of data: Ministry of the Environment Miscellanous/residential Residential Electricity utilities Non-road transportation 700 Industrial sources 700 Industrial processes Non-road transportation Road vehicles 600 Road vehicles 600 Miscellaneous/solvent use

500 500

400 400

300 300

200 200

100 100

0 0

1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 Drive Clean Program 113 after the vehicle fails its initial Drive Clean test, Figure 4: Reduction in Emissions of Smog-causing factoring in the distance typically travelled by the Pollutants (Nitrogen Oxide and Hydrocarbons) repaired vehicle in the year. The total is then fur- Attributable to the Drive Clean Program for Light-duty ther extrapolated to account for vehicles that were Vehicles, 1999–2010 (Tonnes) not actually tested in a given year (because vehicles Source of data: Ministry of the Environment are tested only every two years) and to estimate kilometres travelled by vehicles tested for the first 45,000 time (for which the system therefore contains no 40,000 previous odometer reading). 35,000 We reviewed the consultant’s 2010 Emissions 30,000 Benefit Analysis reports for the three vehicle types 25,000 covered by the program, and noted that the pro- 20,000 gram is believed to have had the following impact 15,000 on emissions: 10,000 For heavy-duty diesel vehicles, particulate • 5,000 matter emissions (the only pollutant that 0 can be measured using the available testing 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 technology for such vehicles) have been reduced by an average of 250 tonnes each Figure 5: Vehicle Emissions of Smog-causing Pollutants year throughout the program’s existence. 3.04

(Nitrogen Oxide and Hydrocarbons) with and without For heavy-duty non-diesel vehicles, the • the Drive Clean Program, 1999–2010 (Tonnes) consultant concluded that the impact of the Source of data: Ministry of the Environment Section program was negligible because only a few VFM such vehicles are on Ontario’s roads. 250,000 •

3 • For light-duty vehicles, emissions were reduced by steadily increasing amounts from 1999 200,000

through 2007. Since then, although emissions Chapter Effect of factors have still been reduced annually, the amount 150,000 unrelated to Drive Clean (54% reduction as of 2010) by which they’ve been reduced has been declining, as shown in Figure 4. According to 100,000 Effect of Drive Clean the consultant’s estimates, the Drive Clean (16% reduction program has been responsible for reducing as of 2010) 50,000 Emissions when Drive Clean started smog-causing vehicle emissions by about 36% Emissions without Drive Clean a year since 2007. The consultant noted a simi- Emissions with Drive Clean lar trend for carbon monoxide emissions. 0 However, as Figure 5 indicates, the consultant 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 estimated that had the Drive Clean program not were estimated to have reduced emissions by a existed, smog-causing emissions from light-duty further 16%, for a total reduction in emissions of vehicles would still have decreased by 54% from 70%. In other words, more than 75% of the total 1999 to 2010 because of other factors—such as the emissions reductions for light-duty vehicles from retirement of older vehicles, the introduction of 1999 to 2010 can be attributed to factors other than vehicles with cleaner emissions control technolo- the Drive Clean program. gies, and fuel improvements. The vehicle repairs required as a result of the Drive Clean program 114 2012 Annual Report of the Office of the Auditor General of Ontario

The Ministry indicated to us that it believes the • attaining a high degree of public acceptance; Drive Clean program has also led to changes in • achieving revenue neutrality over the pro- vehicle owners’ behaviours that cannot be quanti- gram’s lifespan, with full-cost recovery via test fied: for example, vehicle owners make an increased fees; and effort to maintain their vehicles in order to be able • maintaining business integrity (that is, zero to pass the Drive Clean test, and vehicle owners may tolerance for fraud). decide to replace vehicles sooner than they other- We found that the Ministry does not have wise might have if the program did not exist. quantifiable targets and measures for most of these Given the declining impact of the program on goals. The Ministry has set a published target for air quality, we reviewed practices in other jurisdic- only one performance measure—emission reductions and noted that British Columbia announced tions. In Ontario’s Anti-Smog Action Plan for 2000, in May 2012 that it will terminate its program for the Ministry stated that by the fiscal year 2005/06, light-duty vehicles by the end of 2014, because it the Drive Clean program was expected to achieve determined that vehicles were no longer one of the a 22% reduction in vehicle emissions, for nitrogen primary contributors of pollutants in that province. oxides and volatile organic compounds (VOCs) Five U.S. jurisdictions have also ended their pro- combined. But the Ministry has not reported grams, and one other has announced plans to do against this target either in its annual Air Quality so in 2012, either because emissions have returned Report or in the annual Emissions Benefit Analysis to acceptable levels or because other methods reports prepared by external consultants. In fact, are expected to have a more significant impact on the Emissions Benefits Analysis reports do not 3.04 reducing air pollutants. report reductions of VOCs but rather reductions of hydrocarbons, which are a component of VOCs. As Section previously noted, the reported cumulative emis- Emissions Test Results VFM sions reduction of nitrogen oxides and hydrocar- •

3 Over the Drive Clean program’s life, initial pass bons attributable to the Drive Clean program from rates—that is, the rates at which vehicles of a par- its inception to the end of 2010 was 16%.

Chapter ticular type passed their emissions tests on the first try—have improved. As shown in Figure 6, initial Figure 6: Initial Pass Rates for Light-duty Vehicles, pass rates are currently 90% or more for all vehicle Heavy-duty Non-diesel Vehicles and Heavy-duty Diesel types tested. For light-duty vehicles, the initial pass Vehicles, 1999–2010 (%) rate increased from 84% in 1999 to 95% in 2010, Source of data: Ministry of the Environment

and has exceeded 90% every year since 2004. 100 Heavy-duty non-diesel vehicles have shown the 90 greatest improvement, with an initial pass rate that 80 increased from 72% in 1999 to 90% in 2010. Heavy- 70 duty diesel vehicles have continuously had the high- 60 est initial pass rates throughout the program’s life. 50 Since 2001, their initial pass rate has exceeded 95%. 40 Heavy-duty diesel vehicles Light-duty vehicles 30 Heavy-duty non-diesel vehicles Reporting Other Performance Achievements 20 10 The Drive Clean program has four key goals: 0 reducing vehicle-related emissions of smog- • 2010 causing pollutants; 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 Drive Clean Program 115

We noted that the Ministry had also set internal RECOMMENDATION 1 targets that appeared to be easily attainable. To illustrate, the Ministry establishes an annual inter- To ensure that policy-makers are provided with nal target for emissions reductions for non-diesel- current and relevant information, the Ministry powered vehicles. We were informed that this of the Environment should formally evaluate the target was established by projection based on previ- extent to which the Drive Clean program con- ous years’ actual results, yet the new target was tinues to be an effective initiative in reducing generally lower than the actual emissions reduc- smog relative to the cost and impact of any other tions achieved in the previous year. For example, initiatives for reducing smog and improving since emissions reductions from 2007 through overall air quality. In addition, the Ministry 2009 declined by 8% annually, we expected the should periodically evaluate its progress against 2010 emissions reduction target to be about 32,000 all stated program goals and report the results tonnes, yet the Ministry set a 2010 target of only of its assessments publicly on a timely basis. 28,500 tonnes. With regard to the program’s other goals, we MINISTRY RESPONSE found the following: The Ministry agrees that it is critical for policy- Although the Ministry informed us that it had • makers to have the best decision-making infor- conducted a public survey in 2000 to measure mation available. the level of public acceptance for the program, As new data from the modernized program no additional public surveys had been con-

services and emission tests becomes available, 3.04 ducted in the ensuing 12 years. the Ministry will continue to evaluate the • For the goal of business integrity, the Ministry

progress and success of the Drive Clean pro- Section discloses on its website a list of individuals gram relative to its goals and other initiatives VFM and Drive Clean facilities convicted of fraud- •

in Ontario. 3 related offences. It also discloses a list of Drive Clean facilities that have been terminated

or suspended in the last three years. The Chapter Ministry could build on these good disclosure VEHICLES SUBJECT TO TESTING measures by reporting on the number or Vehicle Age percentage of facilities that are in compliance with contract requirements and standard For both light-duty and heavy-duty vehicles, the operating procedures. Drive Clean program exempts from testing any Since 2005, the Ministry has been publishing vehicle built in the last seven years. Vehicles exempt reports on emissions reductions resulting from from testing and other program details are shown the Drive Clean program. However, we found in Figure 7. Over the program’s life, the Ministry has that the Ministry did not report on a timely basis. several times increased the age at which vehicles For example, consultants’ reports on Drive Clean must begin being tested: when the program started, emissions reductions for 2009 and 2010 were not that age was 3; in 2006, it was changed to age 5; publicly released on the ministry website until June and in September 2011, it was changed again, to 2012, and Ontario’s latest Air Quality Report for age 7. In comparison, British Columbia also exempts 2010 was not publicly released until April 2012. vehicles built in the last seven years; Quebec plans to exempt vehicles built in the last eight years; and two U.S. jurisdictions, including California, exempt vehicles built in the last six years. In October 2012, 116 2012 Annual Report of the Office of the Auditor General of Ontario

Figure 7: Drive Clean Program Details Prepared by the Office of the Auditor General of Ontario

Light-duty vehicles Heavy-duty vehicles Program Area Southern Ontario from Windsor to Ottawa Diesel: all of Ontario Non-diesel: Southern Ontario Types of Vehicles Vehicles weighing 4,500 kg or less (e.g., passenger cars, Vehicles weighing more than 4,500 kg (e.g., Tested vans, light trucks and sport utility vehicles), beginning large trucks and buses) that are registered with 1988 models that are registered in the program area in the program area Vehicles Exempt Vehicles manufactured before 1988, hybrid vehicles, Vehicles that are designated as “historic” from Testing designated “historic” vehicles, light–duty commercial farm according to the Highway Traffic Act vehicles, kit cars and motorcycles Testing Frequency • Every two years, beginning when vehicle is 7 years old, • Every year, beginning when the vehicle is to renew registration. Odd model years are tested in 7 years old, to renew registration. even calendar years. • Diesel vehicles may skip a year of testing • When transferring ownership of a used vehicle to if their opacity reading was 20% or less someone other than a family member or the lessee, in their most recent test. if the vehicle has not been tested in the previous 12 • When transferring ownership of a used months. vehicle older than the current year, if the vehicle has not been tested in the previous 12 months. Testing Methods (Effective until December 31, 2012) Diesel: Opacity smoke test • Accelerated Simulation Mode – for most vehicles. Non-diesel: Two-speed idle test 3.04 Testing is conducted on a dynamometer (i.e., vehicle treadmill) Section • Two-speed idle test – for vehicles that cannot be safely tested on a dynamometer VFM

• • Visible smoke test – for diesel vehicles only 3 (Effective January 1, 2013) • On-board diagnostic test – for post-1997 models Chapter • Two-speed idle test – for pre-1998 models • Visible smoke test – for diesel vehicles only Emissions Measured Carbon monoxide, hydrocarbons, nitrogen oxide Diesel: Particulate matter Non-diesel: Carbon monoxide and hydrocarbons Possible Test Results Pass / Conditional Pass / Fail Pass / Fail Conditional Passes Allowed only where the purpose for testing is to renew Not allowed. All vehicles must be repaired vehicle registration, not for ownership transfer. Estimates to the point where they pass an emissions and any actual repairs must be done by a certified repair test. technician at an accredited Drive Clean facility (DCF). Repair Cost Limit $450 None Who Can Perform Anyone and any auto shop, but only those repairs Anyone Repairs performed by a certified repair technician at a DCF are eligible for a conditional pass Program Program is delivered through a decentralized network of Program is delivered through a decentralized Infrastructure test–only, test–and–repair, and repair–only DCF locations network of test–only DCFs, some of which provide mobile testing for diesel vehicles Cost of Test to Up to $35 per test; re-tests cost $17.50, if done at the Market rate Vehicle Owner same DCF as the initial test Ministry Revenue $11.67 of fee charged by DCF for a pass or conditional $15 for each passed test pass Drive Clean Program 117

Ontario’s Environmental Commissioner supported Figure 8: Initial Failure Rates by Model Year for Light- the new seven-year exemption limit in his latest duty Vehicles Tested in 2010 (%) annual report. Source of data: Ministry of the Environment Any vehicle older than a year whose ownership 35 is being transferred to a third party is required to undergo an emissions test, as a consumer protec- 30 tion feature for the prospective new owner. 25 As of December 2011, half of all light-duty 20 vehicles registered in Ontario had been built in the last seven years. For those in this category that had 15 an emissions test in 2010 (likely due to an owner- 10 ship transfer), the initial failure rate was 1% or less, as shown in Figure 8. Since these newer vehicles 5 were built to stricter manufacturing standards for 0 emissions, it is reasonable to assume that with 19881989199019911992199319941995199619971998199920002001200220032004200520062007200820092010 proper maintenance, they will operate more cleanly as they age compared to previous model years. That The Ministry noted that this situation is mitigated is, when a newer vehicle is 10 years old, it will emit by the fact that, according to odometer data col- lower levels of pollutants than an older vehicle did lected by the Ministry of Transportation when a when it was 10 years old. In contrast, initial failure vehicle is registered or re-registered, older vehicles 3.04 rates for vehicles built in 1997 or earlier ranged are not driven as often or as far as newer vehicles. from 11% to 31% in 2010. The Ministry has not For example, in 2010, 2007 light-duty passenger Section compiled pass/fail rates by model year for heavy- vehicles were driven on average three times farther VFM duty vehicles. than 1987 light-duty passenger vehicles (17,000 •

3 Prior to 2007, the program also exempted light- km versus 5,000 km, respectively). However, we duty vehicles that were more than 20 years old. As noted a similar phenomenon with heavy-duty diesel

of 2007, the program instead exempts vehicles built vehicles—that is, in 2010, 2007 heavy-duty diesel Chapter before 1988 (that is, at least 24 years old in 2012). vehicles were driven on average six times farther This is because vehicles built before 1988 were not than 1987 heavy-duty diesel vehicles (52,000 km required by the federal government to have built-in versus 9,000 km, respectively)—yet the Ministry emissions-reduction controls (that is, a catalytic does not exempt pre-1988 heavy-duty diesel converter, which converts toxic gases from a vehicles from emissions testing. vehicle’s exhaust system into less toxic substances). While the Ministry analyzed the impact of As a result, vehicles that are potentially among the increasing the exemption age for new vehicles from worst polluters could legally be on the road. In com- five years to seven years and determined that the parison, about 80% of other North American juris- change will have little impact on total emissions, no dictions, including British Columbia and California, analysis was done on the impact of excluding older require emissions testing for vehicles older than 24 vehicles from the program. years. As of December 2011, more than 100,000 pre-1988 vehicles were registered in Ontario. Geographical Area Covered by the Program Because older vehicles have less stringent emissions limits than newer vehicles, an older vehicle All heavy-duty diesel vehicles registered anywhere that fails an emissions test will pollute more, per in Ontario that meet the age criteria (specified kilometre driven, than a newer vehicle that fails. in the previous section) are required to be tested 118 2012 Annual Report of the Office of the Auditor General of Ontario

under the Drive Clean program. In contrast, only the Ministry of Transportation increased by 90%, those light-duty vehicles and heavy-duty non-diesel while in the decade ending in 2006, the number vehicles registered in a defined geographical area of farmers and the number of farms decreased by called the “program area” are subject to testing, 15%. The Ministry informed us that it had discussed assuming they also meet the age criteria. The the issue of the rise in number of registered farm program area can generally be considered to be the vehicles with the Ministry of Transportation at the Windsor–Quebec City corridor. This corridor was fraud prevention working group in November 2009, selected as the program area because it had been but no corrective action had been taken by the time identified in the 1990s by the Canadian Council we completed our audit fieldwork. of Ministers of the Environment as one of three problem areas in Canada that experienced higher- RECOMMENDATION 2 than-acceptable levels of smog. About 90% of the To help assess the appropriateness of vehicles province’s passenger vehicles are located in this exempted from testing and the geographical part of the province. area covered by the Drive Clean program, the The program area for light-duty vehicles and Ministry of the Environment should: heavy-duty non-diesel vehicles has not been review initial pass/fail rates and evaluate reviewed since 2002, when it was last expanded. • estimated vehicle emissions by model year; Although the current program area contains more formally analyze the impact of excluding than 30 municipalities, 10 municipalities, or about • all light-duty vehicles except those in the 10 one-third, account for two-thirds of the province’s

3.04 larger municipalities in the Windsor–Quebec passenger vehicles and population. In addition City corridor; and to having the highest vehicle density, all but one

Section work with the Ministry of Transportation of these municipalities also experienced more • on a strategy for verifying the legitimacy of VFM smog days than the provincial average every year •

farmers’ vehicle registrations. 3 from 2005 through 2010. The Ministry has never formally assessed whether excluding those vehicles MINISTRY RESPONSE

Chapter not located in these 10 municipalities from the required biennial testing could be done with little The Ministry agrees that reviewing pass/fail or no adverse effect on the environment. rates and emissions by model year is important. The Ministry will continue to collect emissions test data to ensure a sufficient data set is Farm Vehicles assembled to conduct accurate analysis and will According to O. Regulation 628 of the Highway also continue to monitor the appropriateness of Traffic Act, vehicles registered to farmers are exempt the geographical area covered by the program. from the Drive Clean program. The Ministry In addition, the Ministry will work with the Min- informed us that this exemption was put in place istry of Transportation to evaluate its policies for to ease the economic burden of emissions testing registering farm vehicles. and the required repairs on farmers. However, we noted that when these vehicles are registered or re- registered, no verification is required to ensure that CONDITIONAL PASS their vehicle owner is indeed a farmer. In essence, applicants merely have to tick a box on a form iden- In principle, the Drive Clean program requires tifying themselves as such. We noted that from 1998 vehicles that fail the emissions test to undergo to 2010, the number of farm vehicles registered with repairs so that emissions are once again below the Drive Clean Program 119 ministry-prescribed limit. But to alleviate some repair costs in order to fraudulently obtain of the financial burden on vehicle owners, the conditional passes for their customers without Ministry has implemented a $450 repair cost limit. being detected. This problem will be mitigated That is, vehicle owners are allowed to defer emis- starting in 2013, when the program adopts sion system repairs entirely or partly. If a single a new testing method. Under the existing repair is required that would cost more than $450, testing method, repair technicians determine that repair need not be done. If multiple repairs what repairs are needed by a failed vehicle, are required that together total more than $450, whereas under the new test method, the only repairs that fall within the $450 limit need to vehicle’s on-board computer system will be done. Many other North American jurisdictions specify the problem. As well, the Ministry have a similar feature in their vehicle emissions plans to have the service provider monitor inspection and maintenance programs. Owners the estimated and actual repair costs entered who choose to defer repairs costing $450 or more by the technician against a standard industry are given a conditional pass, which allows them to price list; this approach will enable it to flag renew their vehicle registration but not to transfer unreasonable repair costs on a real-time basis. ownership. The vehicle can then be driven for two • In order to assess whether partial repairs are more years without the required repairs—in other of any benefit to the province’s air quality, we words, until the next required emissions test, when compared the emission readings before and they can again get a conditional pass assuming the after partial repairs were done on vehicles repair estimate has not changed. that were issued with conditional passes in 3.04 More emissions reduction benefits could be 2011. We noted that for 50% of these vehicles, achieved by eliminating or raising the repair emission readings for at least one of the pol- Section cost limit, so that all required repairs would be lutants measured got worse after the repairs, VFM performed. To that end, the Ministry raised the to the point where they exceeded emission •

3 repair cost limit from $200 in 1999 to $450 in 2001 limits. For 25% of vehicles, emission readings for part of the geographical area covered by the got worse for all pollutants measured. program, and in 2004, the $450 limit was extended • The appropriateness of the repair cost limit Chapter to the entire area. In order to receive a conditional has been in question since 2005, when an pass, the vehicle owner must have a ministry- external consultant recommended increasing accredited Drive Clean facility determine what it to $600. The consultant also noted that the repairs need to be made, and have obtained a repair current $450 cap represents the same amount estimate exceeding $450. recommended in 1992 by the U.S. Environ- In 2011, conditional passes were issued to 18,000 mental Protection Agency to vehicle inspec- vehicles. We noted the following with respect to tion and maintenance programs operating at conditional passes under the present system: that time in the United States. In our review • We found that 7% of the vehicles that received of practices in other jurisdictions, we found a conditional pass in 2011 had no repairs that almost half of North American jurisdic- done to their emissions control systems before tions either have no repair cost limits or have being issued the conditional pass. For vehicle a higher repair cost limit than Ontario’s. For owners in that year who got a conditional pass example, British Columbia sets different but did have partial repairs done, the average limits based on the vehicle’s age, starting repair cost was only $255. from $500 to unlimited. We also noted that • Under the existing system, there is a risk that from 2006 through 2010, the percentage of technicians could inflate actual or estimated vehicles that pass a retest after any repairs 120 2012 Annual Report of the Office of the Auditor General of Ontario

had been made dropped steadily each year, increasing or eliminating the repair cost limit; while the percentage of vehicles that received • requiring vehicles that receive a conditional a conditional pass increased steadily. In 2010, • pass to be retested annually rather than 56% of vehicles passed a retest (down 10% biennially; and from 2006), and 39% received a conditional limiting the number of conditional passes pass (up 13% from 2006). This pattern could • allowed over a vehicle’s lifetime. indicate either that any repairs that were done are increasingly not effective or that the $450 MINISTRY RESPONSE repair cost limit is too low. In addition, in the Emissions Benefit Analysis report for 2010, Less than 1% of all vehicles tested in the program the consultant noted that the most commonly use the repair cost limit. Starting in January recommended repair involved servicing the 2013, the new program will better identify emis- catalytic converter, which was repaired or sions problems and prioritize the repairs needed, replaced in only one-third of cases. The con- making more effective use of the $450 limit. sultant speculated that this could be because The Ministry scans other jurisdictions and the maximum $450 repair cost limit was too reviews its practices regularly. Once the new low to cover this type of repair. program is in place, we will continue to con- • Ontario requires vehicles that receive a duct jurisdictional reviews, which will include conditional pass to be retested only every reviewing repair costs and conditional passes, two years—in other words, on the same to support our analysis and evaluation of the 3.04 schedule as for vehicles that earn a regular program. pass. In comparison, 13 other North American Section jurisdictions require any vehicle that receives VFM a conditional pass to be retested annually • EMISSIONS TEST METHODS 3 until the problem has been fixed. In 2005, the Ministry’s external consultants recommended The methods used to test emissions depend primar-

Chapter annual testing for vehicles that failed their ily on the type of vehicle and how it is powered: initial emissions test, but this recommenda- • Light-duty gasoline-powered vehicles, which tion was never implemented. are by far the majority of the vehicles in the • Ontario allows an unlimited number of con- province, are tested on a dynamometer—a ditional passes to be issued for any particular kind of treadmill that simulates actual driving vehicle. In comparison, five other North Amer- conditions. The Drive Clean facility’s techni- ican jurisdictions allow only one conditional cian inserts a probe into the vehicle’s tailpipe; pass to be issued during a vehicle’s lifetime. the probe, which is connected to a computer, The Ministry informed us that unlimited condi- measures the concentrations of various gases tional passes are necessary to lessen the finan- in the tailpipe’s emissions. cial burden for those who cannot afford to fully • Heavy-duty non-diesel vehicles, and certain repair or replace a grossly polluting vehicle. light-duty vehicles that cannot be tested on a dynamometer for safety reasons, are tested RECOMMENDATION 3 by the two-speed idle method. This testing method measures fewer gases, and also has To help ensure that polluting vehicles are less stringent emissions limits than the dyna- repaired once emission problems are identified, mometer test. the Ministry of the Environment should consider: Drive Clean Program 121

• Heavy-duty diesel vehicles are tested using to 10% in 2013, which in its view brings Ontario the opacity test method, where smoke density in line with other jurisdictions. In other words, the is measured by a smoke sensor. Drive Clean program could fail up to 270,000 light- • Light-duty diesel vehicles are inspected vis- duty vehicles in 2013, compared with the 135,000 ually for emissions. that failed in 2010. When British Columbia adopted Starting on January 1, 2013, Ontario will adopt OBD testing, its initial failure rate went from about a new on-board diagnostic (OBD) testing method 12% to about 15%. for light-duty vehicles manufactured after 1997. All However, older vehicles, which typically are light-duty vehicles manufactured after 1997 have a the worst polluters, cannot be tested by the new built-in OBD system that continuously checks the testing method, because light-duty vehicles manu- condition and operation of key emissions control factured in Canada before 1998 were not equipped components and emissions-related systems in the with OBD technology. In December 2011, almost vehicle. If the built-in system finds an emissions- 600,000 vehicles (8%) registered in the geographic related problem, it illuminates the “check engine” area covered by the program were built between light on the vehicle’s dashboard and stores one or 1988 and 1997. The Ministry plans to have these more “diagnostic trouble codes” in the vehicle’s vehicles, which are now being tested on a dyna- computer. Thus, any vehicle with an illuminated mometer (equipment that will be phased out when “check engine” light may have an emissions prob- the OBD equipment is installed), tested by the two- lem, and the specific diagnostic trouble code(s) speed idle method. Since the two-speed idle test help to identify this as well as the cause or causes method has less stringent emission limits than the 3.04 of the problem. Once the vehicle is connected to old dynamometer test, the initial pass rate for these the new testing equipment it will retrieve the prob- older vehicles will likely improve, even though Section lems and produce a “fail” result. All other North there will be no real improvement in emission per- VFM American jurisdictions with vehicle inspection and formance—and fewer of these older vehicles that •

3 maintenance programs have been using this testing require repairs will be identified. We researched method for a number of years. the testing methods used in other North American

The OBD testing method will require the Drive jurisdictions for vehicles that are not equipped with Chapter Clean facilities to use new testing equipment. The an OBD system, and noted that more than half of test certificate issued to vehicle owners will also these jurisdictions use a testing method that is more have a different format. This certificate, unlike the stringent than the two-speed idle method Ontario ones currently being given out, will not include plans to use. The Ministry informed us that since all emission limits and actual emissions levels. Instead, emissions testing equipment requires regular main- it will show whether the “check engine” light is tenance, it will not require Drive Clean facilities illuminated and which vehicle diagnostic trouble to maintain both a dynamometer and OBD testing code or codes were detected. equipment. Consequently, older vehicles—which The emission limits set in the vehicle’s OBD currently have both lower initial pass rates and less component are more stringent than those in the stringent limits—will be held to even less stringent dynamometer. The Ministry informed us that OBD limits, whereas newer vehicles—which currently testing could cause a vehicle to fail when emis- have both high initial pass rates and more stringent sions exceed 1.5 times the federal limit, whereas a emission limits—will be held to even stricter emis- dynamometer will not cause a vehicle to fail until sion limits. emissions are much higher—an estimated five to eight times the federal limit. As a result, the Min- istry expects initial failure rates to double from 5% 122 2012 Annual Report of the Office of the Auditor General of Ontario

Implementation of On-board Diagnostic mation can be stored in the system, allowing Test Method compliance staff at the private-sector service provider to verify later on that the vehicle External consultants first recommended OBD test- being tested was in fact the one subject to the ing to the Ministry in the program’s operational test. We noted that uploading photographs via review in 2005 and again in 2007. One of these a dial-up Internet connection (one of four pos- consultants noted that the costs of the dynamom- sible ways to connect the testing equipment to eter test method will outweigh its emission benefits the program’s information system—and prob- by 2015. It further noted that the OBD test method ably the only way available to testing facilities would yield greater emission reductions from all in certain remote areas) would abort the test vehicles combined than the dynamometer test and crash the testing equipment. method. OBD testing has been used in British Col- During an emissions test, the Drive Clean umbia since 2007 and was in place in all U.S. juris- • facility can manually change the record for dictions with vehicle inspection and maintenance the engine type of a vehicle from non-diesel to programs by 2006. The improved testing method is diesel, thus allowing the vehicle to undergo a being implemented eight years after the first oper- simple visual test rather than the more strin- ational review that recommended this switch and gent OBD test. six or seven years behind all other North American The new testing equipment accepts unreason- jurisdictions that have a vehicle inspection and • able odometer readings, which in turn will maintenance program. The Ministry estimates that affect the accuracy of the emission benefit

3.04 by 2015, 96% of the vehicles on Ontario’s roads will calculated. For example, we observed that be equipped with OBD technology. the equipment accepted an odometer reading

Section One benefit of being the last jurisdiction to that was lower than that entered in a previ- adopt OBD testing is that the Ministry has been able VFM ous test, and nevertheless generated a “pass” •

to research the lessons learned in other jurisdic- 3 emissions report. tions that have already adopted the technology, particularly with regard to detecting possible fraud

Chapter RECOMMENDATION 4 by testing facilities. To this end, we noted that the Ministry considered other jurisdictions’ experi- To optimize the benefits of the new on-board ence in using devices (such as video cameras) and diagnostic testing technology, the Ministry of triggers (predetermined indicators of suspicious the Environment should ensure that appropriate activity that set off warnings within the information technical testing is completed and problems are system) to monitor program compliance by test and resolved before rolling it out to all Drive Clean repair facilities. testing facilities in January 2013. The Ministry should also monitor the potential impact of using the less reliable two-speed idle method New Emissions Testing Equipment for testing vehicles older than model-year 1998 We requested that certain tests be performed on once the new on-board testing technology has one of the new testing units that will be rolled out been introduced. to all testing facilities in January 2013, and noted the following problems: MINISTRY RESPONSE The new testing equipment has a camera that • The Ministry agrees that appropriate technical allows the Drive Clean facility to take photo- testing is important. In the last year, the Min- graphs of the vehicle identification number, istry instituted a formal defect-management licence plate and odometer so that this infor- Drive Clean Program 123

tors of suspicious activity (called triggers), so that it process to ensure that technical testing and can target those facilities that are most likely to be corrections on the new test equipment are participating in non-compliant or fraudulent activ- completed. As well, the Ministry initiated field ities. Facilities that set off these triggers are then testing earlier this year in 30 facilities. Issues audited using one of the following approaches: identified at the pilot sites were resolved. telephone audits—unconcealed audits where All of the issues identified in the Auditor’s • the service provider’s compliance staff call the report regarding uploading photos, manual facility to obtain certain documents for review; changes to the engine type and odometer read- overt audits—annual unconcealed audits ings will be resolved prior to the program roll- • where compliance staff check various admin- out in January 2013. istrative matters, such as whether the facility We note the Auditor General’s comments is using appropriate testing equipment; and that vehicles built from 1988 to 1997 can only covert audits—ad hoc, undercover “secret be tested using a tailpipe test. The Ministry has • shopper” audits where compliance staff pose determined that the two-speed idle tailpipe test as customers to try to uncover fraudulent is a cost-effective solution for an ever-decreasing test practices. proportion of older vehicles (that is, an esti- Audits are designed to uncover cases of non- mated 8% of all light-duty vehicles in 2013). It is compliance, such as charging test fees that are worth noting that maintaining the current dyna- higher than allowable, falsifying test results, mometer equipment would be cost-prohibitive and testing vehicles using a more lenient testing

for many Drive Clean facilities. 3.04 method than the required method. We noted that covert audits can identify some of the most serious Section program offences and generally detect more cases VFM MONITORING PROGRAM DELIVERY of non-compliance than any other type of audit. In •

3 each of the last three years, covert audits have dis- Compliance of Drive Clean Facilities covered non-compliance in about 60% of the audits

Fraudulent test results have negative impacts conducted, compared to about 40% with other Chapter on both the environment and vehicle owners: a types of audits. fraudulent pass will permit a high-polluting vehicle However, the use of covert audits has dramatic- to remain on the road, and a fraudulent fail will ally diminished. The latest contract required the cost the vehicle owner for unnecessary repairs. service provider to perform 1,400 covert audits As of December 2011, Ontario had almost 1,500 annually at Drive Clean facilities that test light-duty testing and repair facilities for light-duty vehicles vehicles until June 2011, none between July and and more than 500 testing and repair facilities for December 2011, and, going forward from 2012, heavy-duty vehicles. a very small fraction of the initial requirement of The Ministry outsources the monitoring of Drive 1,400. No covert audits are required for facilities Clean facilities for non-compliant or fraudulent that test heavy-duty vehicles. In its 2009 Treasury activities to the private-sector service provider. Board submission requesting approval to consoli- The service provider uses a number of compliance date all outsourced program services, the Ministry techniques to monitor facilities’ compliance with indicated that the number of covert audits was contract requirements and standard operating pro- expected to be at most 50% of what was done prior cedures. Generally, the service provider analyzes to July 2011. data captured in the Drive Clean information sys- To compensate for the reduction of covert tem to detect occurrences of predetermined indica- audits, the Ministry plans to implement a real-time 124 2012 Annual Report of the Office of the Auditor General of Ontario

monitoring function in its Drive Clean information imposed by the service provider. The various types system starting in 2013, to coincide with the new of remedial actions, in order of severity, as well as OBD testing method. This means that when trig- the frequency with which each type of action was gers for certain suspicious activities are set off, the taken by the Ministry in 2009 and 2010, were as information system will immediately lock out the follows: testing equipment, and compliance staff will have a • phone call (11%); face-to-face conversation with Drive Clean facility • note to file (27%); staff using a video camera attached to the testing • warning letter (47%); equipment. According to the Ministry, Ontario may • suspension (11%); and be the first North American jurisdiction to employ • termination (4%). this kind of cost-effective compliance technique. Because the service provider conducted far Triggers are primarily designed to ensure that fewer covert audits in 2011 than in previous years, the vehicle tested is the one subject to the test and the number of remedial actions applied in 2011 to allow compliance staff to monitor patterns that was about 60% lower than the average number of suggest inappropriate testing and repair practices. actions taken in 2009 and 2010. We noted that although the number of triggers will We noted that the Ministry does prosecute ser- increase in 2013 from about 30 to 50, only 1 in 4 ious offences. Between 2009 and 2011, the Ministry of these triggers will be monitored on a real-time convicted 29 individuals and seven Drive Clean basis; the rest will be recorded in the information facilities for offences relating to Drive Clean fraud, system, and compliance staff will follow up on them and collected fines totalling $446,000. 3.04 at a later date. Because these triggers were not Ministry staff informed us that they typically operational at the time of our audit, we could not apply more lenient penalties than those recom- Section assess whether they will be as effective as covert mended in the remedy schedule in the Ministry’s VFM audits done in the past. procedural manual. For example, when a Drive •

3 Clean facility is caught overcharging vehicle owners for Drive Clean tests, Ministry policy requires that Consequences of Non-compliance

Chapter the facility be suspended for a first offence and The private-sector service provider brings cases terminated for a repeat offence, but we noted that of non-compliance to the Ministry’s attention on between January 2005 and February 2012, only a weekly basis, so that the Ministry can determine one in every 34 facilities found to be overcharging the appropriate penalty depending on the type and were in fact suspended or terminated. We analyzed frequency of the offence. For calendar years 2009 the nature of the overcharges, and while many and 2010, the service provider’s audit activities cases of non-compliance related to misapplying identified about 3,000 cases of non-compliance sales tax on the test fees, a few facilities—some of annually, and the Ministry took remedial action in which were repeat offenders—had more question- about 15% of these cases. The Ministry explained able overcharges or engaged in inappropriate fee that it imposes penalties only on non-compliance collection practices but were not suspended or ter- cases found through covert audits, and that it minated. We could not assess the appropriateness imposed penalties in half of those cases. For non- of penalties for more serious non-compliance items, compliance identified through other types of audits, such as falsifying test results or testing vehicles the Ministry stated that it expects that the appropri- using an inappropriately lenient testing method, ate remedial action will be taken by the service pro- because the Ministry does not possess complete vider, since those cases are usually minor in nature. data for all non-compliance items identified and the However, the Ministry did not track remedial action resulting penalties applied. We also found almost Drive Clean Program 125

40 cases of non-compliance (where, for example, MINISTRY RESPONSE facilities were caught selling vehicle owners unnecessary services or repairs, or testing vehicles The Ministry agrees with the recommendation using an inappropriate test method) for which no of the Auditor General. penalty had yet been applied even after one to two The modernized Drive Clean program years had passed. includes the transformation of the current Our analysis of audit results showed that the reactive audit approach to a proactive real-time, compliance rate remained the same between 2009 risk-based audit process. Through the use of and 2011, which could indicate that the penalties state-of-the-art, web-based technology, Drive applied in previous years may not be having the Clean auditors will be able to identify potential desired deterrent effect. non-compliance issues and speak directly with Drive Clean technicians, review test information, and provide guidance as tests and repairs Duplicate Test Certificates are being conducted. Should a potential non- In our 2004 audit of the Ministry’s Air Quality Pro- compliant event be detected, the auditor will gram, we identified 3,200 uniquely numbered test interrupt the test and promptly address the certificates that had been presented at Ministry of issue. The Ministry will continue to use a variety Transportation (MTO) offices (now ServiceOntario) of enforcement activities, including education more than five times each. In 2006, the Ministry and outreach, suspensions, and revocations of introduced a security code on all certificates operating certifications. 3.04 designed to prevent fraudulent certificate use. Our new audit process, which includes a Unless both the test certificate number and a valid suite of audit tools, will begin in January 2013. Section security code match the vehicle identification num- The Ministry will continue to monitor the pro- VFM ber, ServiceOntario staff will not accept the pass test cess and collect compliance data to determine •

3 certificate for renewing the vehicle’s registration. the appropriateness, consistency and effective- While technical issues do arise from time to time, ness of both its compliance tools and penalties.

they are infrequent, and we concluded that the Chapter Ministry has taken appropriate steps to address the concerns we raised in 2004. CONTRACT MANAGEMENT RECOMMENDATION 5 Procurement of Service Provider

To maintain the integrity of the Drive Clean pro- Before June 2011, the Ministry had six contracts gram, the Ministry of the Environment should: with three different service providers to deliver six • use compliance rates to periodically evalu- distinct program services: ensuring quality control ate the appropriateness of the mix of audit at Drive Clean facilities for the light-duty and the compliance tools, especially given the heavy-duty vehicle program components, training planned substantial decrease in covert audit vehicle inspectors and repair technicians, operating activities; and a call centre to provide technical support to Drive • maintain complete data for all non-compliance Clean facilities and handle public comments, print- items identified and their related penalties, ing and distributing program materials to Drive and ensure that the penalties applied are Clean facilities, and developing and operating an appropriate, consistent and timely. information system to support all program functions, including linking to the Ministry of Transpor- tation’s licensing system. 126 2012 Annual Report of the Office of the Auditor General of Ontario

In 2011, after a competitive procurement pro- life of the program. However, we found that the cess, the Ministry began outsourcing all program Ministry compared Ontario pricing with that in only services to a single service provider—the one that one other jurisdiction, which coincidentally uses the had been providing four of the six services since the same service provider as Ontario. program’s inception. Under the current contract, this service provider will also supply and service Monitoring Service Provider Activity the new OBD emissions testing equipment. The contract, which expires in June 2018, includes an The private-sector service provider must meet option to renew annually for up to three additional certain service levels in order to receive full con- years. Consolidating all program services under one tract payment. The agreement indicates that if the contract with a single service provider was recom- service provider fails to meet deliverables by the mended by an external consultant in 2007. The required dates, the Ministry can withhold specified Ministry’s decision to do so was supported by Treas- amounts from subsequent payments to the service ury Board in November 2009, and this practice is provider as a penalty. In negotiating the contract consistent with that in several other jurisdictions with the new service provider, we felt the Ministry with similar programs. demonstrated good foresight by including more The Ministry hired a fairness commissioner to than 70 deliverables in various schedules, many oversee and evaluate the procurement process for of which were aimed at ensuring a successful the new contract. He concluded that “appropriate transition to the new testing method and related procurement practice was used throughout” the processes by January 1, 2013. 3.04 process. Perhaps most importantly, the new con- We assessed about 50 of the most significant tract’s expected annualized price will be 40% lower deliverables to ensure that the Ministry was mon- Section than the sum of what the previous six contracts cost itoring them and found that almost 80% of the VFM each year. According to the contract, the service pro- deliverables deadlines were met. For the remaining •

3 vider can receive up to $62 million from the prov- 20%, the deadlines were extended or there was a ince over the contract term, or cancel on six months’ lack of documented evidence that the service was

Chapter notice, and will also receive amounts directly from ultimately provided. For the sample of deliverables facility operators for training their inspectors and we tested, penalties were generally applied when repair technicians, facility accreditation, and supply deliverables were late, unless their delivery dates and maintenance of the new OBD testing equip- were extended, in which case no penalty was ment. In this regard, the Ministry awarded the applied. At the time of our audit, we could not service provider the exclusive right to subcontract assess how ready the Ministry will be to roll out the the design, building and testing of the new testing new testing method and related processes by Janu- equipment, and to be that equipment’s sole supplier ary 1, 2013. to Drive Clean facilities. Based on information avail- We also noted that the Ministry sometimes able at the time of our audit, the service provider relies on the service provider’s claim that it has plans to sell the testing equipment to facilities for met certain requirements but does not verify the up to $20,000 plus up to $350 per month for main- reported information. For example, service provider tenance. The Ministry indicated that, to assess the staff is required to be up to date on the program, reasonableness of the purchase price and mainten- but the Ministry does not review the results of ance costs that facility operators will have to pay, tests designed to verify that the service provider’s it had compared these prices to those offered by a employees have maintained current program know- supplier in another jurisdiction, and found that the ledge. In another example, the Ministry does not costs were generally comparable over the expected review summary reports of call centre data to verify Drive Clean Program 127 the service provider’s claim that calls were answered RECOMMENDATION 6 within the prescribed time. In both of these cases, the Ministry has made payments to the service pro- To help ensure that the private-sector service vider in full without checking whether the service provider meets contractual obligations in deliv- provider had indeed met the service requirements. ering the Drive Clean program, the Ministry of the Environment should adequately monitor the delivery of all services, including periodically Complaints verifying reported service levels achieved. Complaints are received and resolved by the call centre that is operated by the private-sector service MINISTRY RESPONSE provider. The call centre receives on average 100 The Ministry agrees with the recommendation complaint calls per month from the public. In a of the Auditor General. typical month, more than half of all callers report In order to monitor the service provider’s a smoking vehicle, 30% express dissatisfaction performance, the Ministry, along with the with a Drive Clean facility (for example, because of service provider, is developing a Service Level repair misconduct or because their vehicle failed its Tracking application to continuously monitor all emissions test at one testing facility but passed at service levels and apply penalties as appropri- another), and 12% voice general complaints about ate. If service-level standards are not met, the the program (regarding, for example, the cost of service provider must pay a monetary penalty. the emissions test or the required frequency of test-

The Ministry has also instituted a business 3.04 ing). We were informed that Drive Clean facilities process to ensure that all calls received by the that are the subject of complaints are considered for

Drive Clean Office are reviewed, followed up Section audit by the service provider. and closed. VFM Under the terms of the agreement, the Ministry •

3 can withhold payments to the service provider if call handling quality is substandard—that is, if the

call centre does not provide accurate, complete and PROGRAM REVENUE Chapter current program information to callers 95% of the Fee Collection time. To assess service quality, the Ministry must review a specified minimum number of calls each Owners of light-duty vehicles pay $35 for an initial month. However, we found no documented evi- test and $17.50 for a retest after any repairs have dence that the Ministry had monitored the required been made, while owners of heavy-duty vehicles number of calls. Without such oversight, the Min- pay market rates for the test. The Ministry is entitled istry can’t reliably assess performance and withhold to receive $11.67 for light-duty vehicles and $15 payment when warranted. for heavy-duty vehicles, but only if the vehicle We also noted that between July 2011 and receives a pass or conditional pass. (The govern- February 2012, almost 160 calls were logged as ment receives no revenue from failed tests or retests, unresolved. The Ministry advised us that these call- because it does not want to collect multiple test fees ers’ complaints were indeed resolved, but because from vehicle owners. Vehicle owners still have to pay call centre staff were unfamiliar with the call sys- the test fee imposed by the testing facility.) tem, the handling of the calls had been erroneously At the time of our audit, Drive Clean facilities recorded as unresolved. were forwarding the Ministry’s portion of test revenue to the service provider, who in turn submitted all revenues received to the province. Starting in 128 2012 Annual Report of the Office of the Auditor General of Ontario

January 2013, the facilities will directly remit test should be based on consideration of the full costs of revenues to the government without the involve- delivering the service. ment of the service provider. One of the Drive Clean program’s goals is to To ensure that the Ministry receives all test achieve revenue neutrality over the program’s revenues owing to it, each month the Ministry rec- life. By the end of the 2010/11 fiscal year the onciles payments received from the service provider present value of cumulative revenues collected with the number of pass or conditional pass results and expenses incurred since 1999 had reached indicated by the Drive Clean information system. the break-even point. However, revenues are now However, due to instances of data corruption exceeding program expenses. In December 2011, and system communications errors, the information the Ministry determined that (based on a net system may not contain all test results. Therefore, present value calculation) the program is expected to identify omissions in the system, the Ministry to realize an accumulated surplus of $11 million runs a monthly exception report that reconciles by the end of the 2011/12 fiscal year and $50 mil- Drive Clean test certificate numbers processed by lion by the time the current consolidated contract ServiceOntario using the Ministry of Transportation with the service provider ends in 2018. At the time licensing system with those recorded in the Drive of our audit, the Ministry informed us that it had Clean information system, and any extra tests are received advice and was considering options for validated and added to the system. This is an effect- becoming compliant with both the 2006 Costing ive detective control process, and we noted that and Pricing Policy and the 1998 Supreme Court of in 2011, this process detected about 340 tests that Canada decision. 3.04 were registered at ServiceOntario but not initially recorded in the Drive Clean information system. Section

OTHER MATTER VFM Vehicle Retirement Program • Revenue Neutrality 3 The Ministry provides in-kind contributions Test revenues collected by the Drive Clean program (through promoting the program to increase public

Chapter are considered a user fee, not a tax. However, awareness of its existence) to a national vehicle according to a 1998 Supreme Court of Canada deci- scrappage program under which a vehicle owner sion, user fees must have a reasonable relationship who donates an older vehicle that meets certain to the cost of the services provided. In other words, conditions is given a charitable donation receipt a user fee cannot exceed the cost to the governof at least $100. Other jurisdictions have similar ment of providing the service. Otherwise, a court programs, but with more generous incentives. For could determine that the amount of the excess fee instance, California’s car scrappage program offers is really an unlawful tax and therefore repayable. a minimum of $1,000, and up to $1,500 for low- The Ministry of Finance’s 2006 Costing and Pricing income earners, for unwanted vehicles that meet Policy indicates that where fees or other charges are certain eligibility guidelines. collected for services offered to the public, pricing Chapter 3 Ministry of Education Section 3.05 Education of Aboriginal Students

high as 50%—for the youngest adult age group Background (aged 20–24). Only 39% of the First Nation people living on reserve in this age group had graduated from high school. This issue is all the more import- According to the most recent Statistics Canada ant considering that 46% of First Nation people census data (2006), there are 242,000 Aboriginal

are under the age of 25, as compared to the overall 3.05 people living in Ontario, primarily comprising population where 32% are less than 25 years old. three distinct groups: First Nation (158,000),

The Ministry of Education (Ministry) adminis- Section Métis (74,000) and Inuit (2,000). The remaining ters provincially funded elementary and second- VFM 8,000 Aboriginal people classified themselves as •

ary education in Ontario and is responsible for 3 “other.” Aboriginal peoples in Ontario have diverse developing the curriculum, setting requirements for languages, cultures and traditions. The census also student diplomas and providing funding to school

identified that approximately 47,000 First Nations Chapter boards. In 2006, the Ministry identified Aborig- people live on reserves in Ontario; these are lands inal education as one of its key priorities, with a set aside for the use and benefit of a specific band focus on closing the gap in academic achievement or First Nation. There are a total of 133 First Nation between Aboriginal and non-Aboriginal students by communities in Ontario, each of which has its own the year 2016. To meet this challenge, the Ministry government or tribal council. The delivery of edu- released various policy documents and funded a cation through schools on reserve is the responsibil- number of initiatives intended to improve outcomes ity of the First Nation and the federal government. for Aboriginal students. Since 2006, the Ministry The federal government is financially responsible has provided $170 million in funding to improve for the education of First Nation students living on academic achievement levels for Aboriginal stu- reserve, whether these students attend First Nation dents and increase awareness of Aboriginal cultures or provincially operated schools. for all students. As shown in Figure 1, the 2006 census (the most In January 2006, the Aboriginal Education recent definitive data on Aboriginal education) Office (AEO) was created to provide co-ordination identified that only 62% of Aboriginal adults had and leadership on Aboriginal education issues and graduated from high school as compared to 78% of initiatives. The AEO works in collaboration with the general population, a gap of 16%. The academic Aboriginal communities and organizations, school achievement gap is even more pronounced—as

129 130 2012 Annual Report of the Office of the Auditor General of Ontario

Figure 1: The Gap in Aboriginal High School Graduation Rates for Adults and Young Adults Source of data: 2006 Statistics Canada census and Ministry of Education

All Adults (2006) Age 20–24 (2006) # Enrolled in Provincial Population Group % Graduated Gap % Graduated Gap Schools (2010/11) All Ontario 78 — 89 — — First Nations—on reserve 44 34 39 50 5,690 First Nations—off reserve 64 14 66 23 27,765 Inuit 59 19 66 23 700 Métis 70 8 81 8 18,245 All Aboriginal 62 16 66 23 52,400

Note: The percentage graduated is the percentage of the population group with a high school diploma. The gap in graduation rates equals the percentage of all people graduated in Ontario minus the percentage graduated in the Aboriginal group.

boards, other ministries and the federal govern- • ensure that transfer payments intended for ment. Currently, the AEO employs 12 staff, includ- Aboriginal education initiatives are spent for ing five regional education officers who work with the purposes intended and allocated on the Ontario’s 72 school boards. basis of student need. Research indicates that many Aboriginal stu- Senior management reviewed and agreed to our dents are faced with significant challenges that audit objectives and associated audit criteria.

3.05 impact their achievement levels in school, such Our audit work was conducted at the Ministry’s

as high rates of poverty, substandard housing and head office, primarily in the Aboriginal Education

Section poor nutrition. Equally important, many of these Office. We also conducted audit work at selected

students live in areas with little prospect of future school boards and at a sample of elementary and VFM

•

employment, which is often a demoralizing factor secondary schools in these boards. The three 3 in taking their education seriously. In 2008, Health boards we visited were Algoma District School Canada reported that there is an extraordinarily Board (Sault Ste. Marie), Kawartha Pine Ridge Chapter high and increasing rate of suicide among Aborig- District School Board (Peterborough) and Lakehead inal youth. Although the Ministry of Education District School Board (Thunder Bay). cannot resolve all of these issues, it can promote Our audit work included reviewing and analyz- achievement in school to help Aboriginal students ing ministry files, administrative directives, policies overcome these challenges. and procedures, and interviewing ministry staff. We also met with school board staff, including supervisory officers, principals and teachers. We researched Aboriginal education practices in other Audit Objective and Scope jurisdictions and solicited the opinions of First Nation education managers, First Nation Elders, the The objectives of our audit were to assess whether Chiefs of Ontario and the Métis Nation of Ontario. the Ministry and selected school boards have Our audit also included a review of activities of adequate procedures in place to: the Ministry’s Internal Audit Services Branch • identify and implement initiatives to improve relating to Aboriginal education. We reviewed the Aboriginal student achievement and to meas- branch’s recent reports and considered its work and ure and report on the effectiveness of those any relevant issues identified when planning our initiatives; and audit work. Education of Aboriginal Students 131

• Since the Ministry did not have an accurate Summary and reliable mechanism to identify Aboriginal students in the Ontario school system, in 2007 it released Building Bridges to Success for First In 2006, the Ministry identified Aboriginal educa- Nation, Métis and Inuit Students. This policy tion as one of its key priorities with a focus on meet- guide assists school boards in developing ing two primary challenges by 2016: to significantly Aboriginal student self-identification policies improve achievement among First Nation, Métis whereby students are given the option to and Inuit students, and to close the significant formally identify themselves as First Nation, student achievement gap between Aboriginal and Métis or Inuit. At the time of our audit, less non-Aboriginal students. The Ministry also wanted than half of the estimated Aboriginal student all students to have an appreciation of Aboriginal population in Ontario had self-identified. cultures. In order to help achieve these objectives, Improved progress on this self-identification the Ministry designed an overarching policy strategy would enable the Ministry and framework that endeavoured to identify Aboriginal boards to better target funding and periodic- students in Ontario, guide the development of ally benchmark the academic progress of appropriate support programs, periodically assess Aboriginal students. academic achievement and report on the progress Five years after announcing its intention to made in implementing the framework. However, • significantly close the gap in achievement the Ministry needs to more actively oversee the between Aboriginal and non-Aboriginal stu-

implementation of this framework to demonstrate 3.05 dents, the Ministry has still not established a what, if any, progress has been made since 2006 in baseline or even a starting point from which

improving achievement among Aboriginal students Section to measure the gap. Therefore, we requested and closing the achievement gap between them and VFM that the Ministry provide us with the most • other students in Ontario. 3 recent student data, including Grade 10 credit Some of our more significant observations accumulation for self-identified Aboriginal regarding Aboriginal education in Ontario were:

students. Accumulation of credits toward Chapter In 2007, the Ministry released the Ontario • graduation is one of the primary indicators First Nation, Métis and Inuit Education Policy of student success in high school. We noted Framework (Framework), which includes 10 that only 45% of the self-identified Aboriginal performance measures intended to gauge the Grade 10 students were on track to graduate success of ministry, school board and school from high school, compared to 74% for all stu- efforts to improve the academic achievement dents in the Grade 10 population. This brings of Aboriginal students. Although the Ministry into question the Ministry’s ability to meet its has reported some success on several of these goal of significantly closing the achievement measures, five years after the release of the gap between Aboriginal and non-Aboriginal Framework the Ministry has not quantified students by 2016. any of the 10 performance indicators so that Even though an Aboriginal organization progress could be objectively measured, nor • engaged by the Ministry in 2009 reported has it required that the school boards individ- that consistent implementation of the ually evaluate and report on these perform- Framework is necessary to make systemic ance measures. None of the three boards we change, we found that it was largely left visited had done so. up to each board to decide how it would 132 2012 Annual Report of the Office of the Auditor General of Ontario

implement the Framework. The emphasis through progressive collaboration and specific placed on this varied at the three boards we supports. Some boards have made progress in visited. For example, one of the school boards implementing supports for Aboriginal students we visited had successfully implemented the and recent evidence reveals specific instances self-identification policy as virtually all of its where the achievement gap has been reduced. Aboriginal students had self-identified. This The Ministry agrees with the spirit of the board was using student achievement data to Auditor General’s recommendations and will identify challenges unique to its Aboriginal continue to develop and implement strategies population and was implementing specific targeted to improve Aboriginal student success strategies to address these challenges. within the context, philosophy and approach Although the early education of many First • outlined in the Ontario First Nation, Métis and Nation students is provided by reserve Inuit Education Policy Framework. schools, which are the financial responsibility The Ministry’s Framework is based on of the federal government, many of these stu- Ontario’s New Approach to Aboriginal Affairs, dents eventually transfer into the provincial 2005, which charts a new course for a construct- system. Given limited per-student funding, ive, co-operative relationship with the Aborig- on-reserve schools have generally not been inal peoples of Ontario to develop productive able to provide the quality of education found partnerships, collaborate on key initiatives and in provincial schools; studies suggest that achieve real progress on shared goals. these students may be several grade levels

3.05 The Ministry, with its partners, has put in behind when they transfer into the public sys- place the necessary foundation for success in tem. Although the Ministry does not measure

Section keeping with our overall student achievement this, our analysis of Education Quality and strategy: building relationships and capacity VFM Accountability Office (EQAO) data found that •

within the sector; designing and developing 3 only half of on-reserve students attending data collection mechanisms; and analyzing and provincially funded schools passed the Grade implementing evidence-driven strategies to sup-

Chapter 10 Ontario Secondary School Literacy Test port advancements in student achievement. in the 2010/11 school year, compared to the When considering evidence-based strategies provincial average of 83%. and reporting, it is important to understand Aboriginal peoples’ perspectives on the school OVERALL MINISTRY RESPONSE system, including the intergenerational mistrust Too many Aboriginal students do not achieve resulting from residential school experiences. the academic success we want for all students. Ontario respects Aboriginal and treaty rights Addressing this education attainment gap is a protected by Section 35 of the Constitution Act, top priority for the government of Ontario. We 1982. This is why the collection of Aboriginal are pleased that the Auditor General shares student data must continue to be voluntary our commitment to improving educational out- and confidential. comes for Aboriginal students. Moving forward, the Ministry will pursue a The Ministry of Education (Ministry)and path that builds on demonstrated successes, in boards, working with First Nation, Métis and partnership with communities, school boards, Inuit communities, share the view that signifi- the federal government and Aboriginal part- cant progress has been achieved, and that con- ners, to support First Nation, Métis and Inuit ditions for future success have been established students across the province. Education of Aboriginal Students 133

(Framework). The Ministry considers the Frame- Detailed Audit Observations work to be the foundation for delivering quality education to all Aboriginal students who attend provincially funded elementary and secondary POLICY GOALS, IMPLEMENTATION schools. Figure 2 shows the Ministry’s three goals PLANS AND PERFORMANCE MEASURES and the 10 performance measures it intends to use Policy Goals and Planning to gauge success, as outlined in the Framework. In order to be truly effective, the Framework In June 2005, the government of Ontario released recognized that it was necessary for the Ministry Ontario’s New Approach to Aboriginal Affairs, which to develop an implementation plan to guide its outlined a number of initiatives aimed at achiev- activities and assist school boards in meeting the ing real progress on shared goals with Aboriginal broad objectives in the Framework. Although the people. In line with this approach, the Ministry Ministry has engaged in some preliminary imple- identified Aboriginal education as one of its key mentation planning, five years after the release of priorities, with a focus on meeting two primary the Framework, the Ministry has still not developed challenges by the year 2016: to improve achieve- a formal implementation plan. In our opinion, such ment among First Nation, Métis and Inuit students; a plan should identify the key obstacles faced by and to close the gap between Aboriginal and Aboriginal students and outline specific activities non-Aboriginal students in literacy and numeracy, to overcome various obstacles. This was exactly retention of students in school, graduation rates the process we found the Ministry had used in our

and advancement to post-secondary studies. 3.05 recent audits of school safety programs and student In 2007 the Ministry released the Ontario First success initiatives, both of which had achieved

Nation, Métis and Inuit Education Policy Framework Section

VFM

• Figure 2: Ontario First Nation, Métis and Inuit Education Policy Framework Goals and Performance Measures 3 Source of data: Ministry of Education

Goal Performance Measure Chapter High levels of student 1. Significant increase in the percentage of First Nation, Métis and Inuit students meeting achievement provincial standards on province-wide assessments in reading, writing and mathematics 2. Significant increase in the number of First Nation, Métis and Inuit teaching and non-teaching staff in school boards across Ontario Reduce gaps in student 3. Significant increase in the graduation rate of First Nation, Métis and Inuit students achievement 4. Significant improvement in First Nation, Métis and Inuit student achievement 5. Significant improvement in First Nation, Métis and Inuit student self-esteem 6. Increased collaboration between First Nation education authorities and school boards to ensure that First Nation students in First Nation communities receive the preparation they need in order to succeed when they make the transition to provincially funded schools 7. Increased satisfaction among educators in provincially funded schools with respect to targeted professional development and resources designed to help them serve First Nation, Métis and Inuit students more effectively High levels of public 8. Increased participation of First Nation, Métis and Inuit parents in the education of their children confidence in public 9. Increased opportunities for knowledge sharing, collaboration and issue resolution among education Aboriginal communities, First Nation governments, and education authorities, schools, school boards and the Ministry of Education 10. Integration of educational opportunities to significantly improve the knowledge of all students and educators in Ontario about the rich cultures and histories of First Nation, Métis and Inuit peoples 134 2012 Annual Report of the Office of the Auditor General of Ontario

some success. For these programs, the Ministry An Aboriginal organization engaged by the Min- had formulated a strategy, devised a detailed plan, istry in 2009 to obtain feedback on the implementa- and measured results along the way to ensure that tion of the Framework had reported that, although progress was being made. In contrast, the Ministry consistent implementation is necessary to make could not demonstrate how it was systematically systemic change, implementation of the Framework implementing the Framework. Instead, we were appeared to be dependent on the individual in informed that most of the Ministry’s activities could charge of Aboriginal education initiatives at the be matched to at least one of the 10 performance school board. Similarly, based on our visits to the measures. However, these performance meas- three boards, we concluded that there was a need ures were developed to assess the success of the for greater ministry co-ordination and guidance. Aboriginal education strategy and do not contain the detail necessary to guide ministry or school Performance Measurement and Reporting board activities. According to the Framework, Aboriginal edu- In 2007, the Ministry committed itself to report cation must be integrated into the school board publicly every three years on the implementation planning process. School boards in Ontario must be of the Framework according to the 10 performance guided by the Board Improvement Plan for Student measures. It released the first progress report in Achievement, a document that helps teachers, the fall of 2009, titled Sound Foundations for the principals and school board staff plan and imple- Road Ahead, to provide an update on the progress ment strategies to improve student achievement. made since 2007 in implementing the Framework. 3.05 The Ministry acknowledged that, although 40 However, much of the discussion provided in this out of Ontario’s 72 school boards had included an progress report was of a very general nature and Section Aboriginal education component in their improve- did not quantify any of the 10 performance meas- VFM ment plans, it did not have a formal process in place ures or identify any specific targets to be used to •

3 to review these or other board plans. The Ministry assess progress against these measures. could not provide us with any documentation to For this first progress report, the Ministry

Chapter demonstrate that it had reviewed these board plans engaged two Aboriginal organizations to solicit or assessed the degree to which the Framework has community feedback on the Framework, and also been implemented at each school board. worked with an independent research team to To assess the Aboriginal education planning obtain feedback from students, parents and school process at the school board level, we reviewed the boards. Additionally, the Ministry sent out surveys improvement plans for the past three years at the to all school boards. three boards we visited. We found that one of the We reviewed the reports from the independ- boards made no reference to Aboriginal education, ent research team and the two organizations that while another board listed Aboriginal cultural the Ministry used to support its progress report. activities and events that it planned for that year. We noted that of the 1,200 students interviewed, The third board set specific targets that focused on only 111 had self-identified as Aboriginal students. Aboriginal student achievement, one of which was Additionally, almost half the students interviewed to increase by 10% the number of Aboriginal stu- were from the Ottawa region and 90% were from dents who were successful on the Ontario Second- Catholic school boards, which represent only 30% of ary School Literacy Test (OSSLT). However, none of Ontario’s elementary and secondary school student these boards specifically mentioned the Framework population. The research team noted that these or identified any of its 10 performance measures in and other sampling concerns limited the amount of their improvement plans. reliable data collected. Clearly, an unrepresentative Education of Aboriginal Students 135 or inadequate sample would compromise the integ- RECOMMENDATION 1 rity of any related conclusions or inferences in the progress report. To help Aboriginal students succeed in school A second progress report on the implementation and reduce the gap in student achievement as of the Framework was scheduled to be released outlined in the Ontario First Nation, Métis and in the fall of 2012. However, in May 2012 the Inuit Education Policy Framework (Framework), Ministry could not provide us with a draft of any the Ministry of Education (Ministry) and school progress it may have made on the report and stated boards should: that the report is now scheduled to be released in • develop specific implementation plans early 2013. that identify and address the key obstacles At the board level, we noted that two school faced by Aboriginal students and routinely boards we visited recently began to assess their review and update these plans to assess what status with regard to the Framework. This was the progress is being made; and first time both of these boards had undertaken • include in these plans specific goals and per- such an assessment since the Framework’s release formance measures as outlined in the Frame- in 2007. In October 2011, one of these boards work and objectively measure and report prepared a document that identified what it has aggregate results to determine whether any already done and is planning to do to implement progress is being made toward improving each of the 10 performance measures. In March Aboriginal student outcomes. 2012, the other board was asked by the Ministry 3.05 to perform a self-assessment to determine how MINISTRY RESPONSE well that board has implemented the 10 perform-

The Ministry agrees that implementation plans Section ance measures. We noted that this process was are necessary to support initiatives going for- VFM not consistent among all regions and, in general, •

ward to close the achievement gap. Initiatives 3 each board operated autonomously in the degree to support Aboriginal student achievement to which it implemented specific strategies to must be holistic and leverage evidence-based

address the Framework and assessed the success of Chapter strategies developed in supporting the student its strategies. achievement agenda. The Framework’s three goals and 10 perform- The Ministry will do the following: ance measures were developed to enable the Release a preliminary progress report and Ministry and school boards to assess progress made • draft implementation plan in fall 2012 toward improving achievement among Aboriginal to engage key partners on strategies for students and closing the gap between Aboriginal progress against the Framework’s perform- and non-Aboriginal students. Overall, we found ance measures, in order to close the achieve- that, five years after the release of the Framework, ment gap between First Nation, Métis and neither the Ministry nor the boards we visited had Inuit students and all students. measured progress in relation to the Framework’s Release the second progress report on the performance goals or set any measurable targets • implementation of the Framework in spring so that progress could be measured. As a result, 2013. The report will include an implemen- there is little province-wide information to indicate tation plan up to 2016. what, if any, progress has been made in improving Align the implementation plan with educational success among Aboriginal students and • other key government commitments and closing the achievement gap between them and investments that are designed to provide other students in Ontario. 136 2012 Annual Report of the Office of the Auditor General of Ontario

the Ministry and school boards have not been able targeted supports for Aboriginal youth, to identify these students in the Ontario school such as Ontario’s Mental Health and system. As a result, data could not be aggregated Addictions Strategy. to develop programs, report on results achieved, or identify areas for improvement. Consequently, in BOARD RESPONSE April 2007, the Ministry released Building Bridges All three boards agreed with this recommenda- to Success for First Nation, Métis and Inuit Students tion. One board noted that its process of setting (Building Bridges) as a guide provided to school specific plans, goals and performance measures boards to assist in developing policies for voluntary assists it in measuring student learning over Aboriginal student self-identification whereby stu- time, and that such efforts have resulted in dents would be given the option to formally identify improvements for Aboriginal students, par- themselves as First Nation, Métis, or Inuit. Board ticularly in primary reading and writing. The implementation of the self-identification policy board noted that it would continue to work on was not mandatory but the Ministry encouraged all further improvements, specifically in the area school boards to develop such a policy. of junior mathematics. A second board’s plans The Ministry did not have a formal process in included strategies to close the achievement place to oversee implementation of self-identification gap and provide specific supports for Aboriginal policies at the school board level. As of May 2012, students. This board stated that it has shared although 68 of Ontario’s 72 school boards had imple- student achievement results internally with mented a self-identification policy, less than half of 3.05 staff and with each band, and that beginning the estimated Aboriginal students had self-identified in 2012 it would share aggregate data publicly (23,000 out of 52,400). The Ministry stated that Section in its director’s annual report. The third board the recent implementation of the policy at many VFM cautioned that the number of students who had boards was one of the primary reasons why only •

3 self-identified was very low for some grades, 44% of the estimated number of Aboriginal students and measuring achievement over time with had self-identified.

Chapter such a small group may not result in reliable We reviewed data pertaining to self-identification data. The board noted, however, that such at all 72 school boards and identified 18 boards that information was still useful to support students had not shown a significant increase in the percent- by focusing on tiered intervention and differ- age of students self-identified over the previous three entiated instruction. This board also noted that years. In addition, another 30 boards had identified annual planning allows it to validate systemic less than 20% of their estimated Aboriginal student learning and, moving forward, it will more enrolment. In contrast, virtually all the estimated closely align planning with student learning Aboriginal students in 16 other school boards had and teacher needs. self-identified, indicating that it was possible for these school boards to successfully implement a policy that encourages their Aboriginal students to voluntarily identify themselves. VOLUNTARY, CONFIDENTIAL At two of the three school boards we visited SELF-IDENTIFICATION virtually all of the estimated Aboriginal student Student Self-identification population had self-identified. During the first year of policy implementation, one of these boards Historically, one of the obstacles related to focusing had identified 150% of the number of Aboriginal education supports on Aboriginal students is that students estimated by the Ministry. This board Education of Aboriginal Students 137 attributed its achievement to creating a success- as mentors and role models, share their Aboriginal ful awareness campaign prior to implementation. culture, and help foster an environment where Furthermore, this board has received only two Aboriginal students can reach a high level of minor complaints from parents pertaining to stu- achievement. In 2008, the Ministry met with the dent self-identification, which it addressed to the Ontario College of Teachers to develop a self- parents’ satisfaction. At the third board we visited identification process for teachers of Aboriginal only about 50% of the estimated Aboriginal student descent. In 2009, the College reported to the population had self-identified. This board stated Ministry that teachers need to understand what that it was unaware of the Ministry’s estimate of self-identification information would be used for the number of Aboriginal students in its schools and how any potential concerns would be miti- and the Ministry had not provided them with the gated. The College also noted that the successful estimate. We believe that boards would find such implementation of teacher self-identification may information useful to help them to assess progress take many years and both short- and long-term in identifying their Aboriginal students. implementation plans are needed. In July 2011, the Ministry surveyed school At the time of our audit, the Ministry still did not boards to assess progress to date in the implemen- have any plans to encourage school boards to imple- tation of the self-identification policy. More than ment an Aboriginal staff self-identification policy. 66 boards responded to the survey, and many Also, at the time of our audit the Ministry had not provided recommendations that they thought formally gathered any information on the number would be useful in increasing the success of this of boards that had implemented an Aboriginal staff 3.05 policy. The boards wanted to see more ministry self-identification policy or gathered any data on involvement in the process, with assistance such as the number of staff who have self-identified. Section a public awareness campaign, direct communica- One of the school boards we visited imple- VFM tion with Aboriginal communities, and standard mented a voluntary self-identification policy for •

3 materials for boards to use. Currently, each board its Aboriginal staff members in 2008. This board independently develops its own materials, such as has experienced success, as many of its staff who

brochures for parents outlining the policy. Some self-identified have become resources to reach out Chapter boards find this challenging, as they need to ensure to Aboriginal students. We noted that the trustees that information is correct and culturally appropri- at another school board we visited recently passed ate. Boards also recommended that the Ministry a motion to develop a similar policy. Following our provide more support in sharing best practices discussions with this board, it expressed interest province-wide. One of the school boards we visited in contacting the first board to obtain a copy of its mentioned that boards are struggling to create staff self-identification policy to help in its own effective strategies and approaches to implement policy development. the self-identification policy even though they have consulted with each other to obtain ideas on how RECOMMENDATION 2 to best implement a policy. To obtain the population data necessary to better develop specific support programs, report on Teaching and Non-teaching results, and identify opportunities to improve Staff Self-identification Aboriginal student achievement, the Ministry of Education (Ministry) should: The Framework also outlines the need for a sig- develop standard communication tools and nificant increase in teaching and non-teaching • disseminate best practices to assist boards Aboriginal school staff. These individuals can act 138 2012 Annual Report of the Office of the Auditor General of Ontario

in successfully implementing an effective use of the data. This board noted that the major- student self-identification process; and ity of its eligible students had voluntarily self- • develop a policy guide for self-identification identified, and this number continues to grow in by Aboriginal teaching and non-teaching step with the changing demographics of its com- staff and oversee the effective implementa- munity. A second board stated that it supported tion of this policy. the development of communication tools that Both the Ministry and school boards should can be used as exemplars to augment present exercise effective oversight to help ensure that self-identification processes. The third board the student self-identification policy is being stated that its self-identification policy had been successfully implemented in Ontario schools. in place since 2007, with almost 100% participation of its Aboriginal students. The board noted MINISTRY RESPONSE that it had positive results in implementing its staff self-identification policy and was willing to The Ministry agrees that it is necessary to sup- share its communication documents. port self-identification efforts and commends the significant progress that boards have made thus far in building trust with Aboriginal families choosing to participate in the initiative. A DATA COLLECTION AND ANALYSIS multi-pronged approach that further engages In order to assess progress toward the goals of boards and Aboriginal partners is required.

3.05 improving student achievement and closing the gap The Ministry will do the following: between Aboriginal and non-Aboriginal student • Implement a communication strategy to

Section achievement by the year 2016, the Framework support all boards in strengthening their noted that it will be important to have reliable and VFM Aboriginal student self-identification policy •

valid data. In addition, Aboriginal student-specific 3 implementation by spring 2013. data is needed to support performance measure- Provide support to all boards in the develop- • ment and reporting. To evaluate progress, the

Chapter ment of board-specific strategies regarding Framework included three performance measures the continued implementation of Aborig- that directly assess Aboriginal student achievement inal voluntary self-identification policies and require student-specific data to measure: throughout the 2012/13 school year. significant increase in the percentage of First Continue to work with education partners, • • Nation, Métis and Inuit students meeting including those representing teaching and provincial standards on province-wide assess- non-teaching staff, to discuss the feasibility ments in reading, writing and mathematics; of the development of a policy guideline for significant increase in the graduation rate of voluntary staff self-identification, by the end • First Nation, Métis and Inuit students; and of the 2013 calendar year. • significant improvement in First Nation, Métis and Inuit student achievement. BOARD RESPONSE The Ministry collects information from schools All three boards supported the recommenda- and school boards through a secure, web-enabled tion, with one board noting that sharing best system called the Ontario School Information Sys- practices regarding the implementation of vol- tem (OnSIS). The Ministry also acquires data from untary self-identification would be very helpful, third-party sources such as Statistics Canada and especially regarding appropriate and effective the Education Quality and Accountability Office Education of Aboriginal Students 139

(EQAO), the Ontario government agency that still not determined the number of self-identified performs standardized testing of elementary and students it considers to be enough before it will secondary school students. Once collected, the data begin measuring and reporting on Aboriginal is run through various quality assurance processes, student achievement. personal information is removed, and the data is The Ministry has also not analyzed Aboriginal stored as the authoritative source of education student achievement data for the province as a data. Our assessment showed that OnSIS is rela- whole, with the exception of some preliminary tively up to date, as data is submitted, verified and analysis in 2011 that focused on six school boards. finalized three times annually. However, the Min- Therefore, we requested that the Ministry provide istry does not currently use this data to periodically us with data including EQAO results as well as produce standard reports to assess the progress Grade 9 and 10 credit accumulation numbers made in achieving the goals of the Ontario First for all self-identified Aboriginal students in the Nation, Métis and Inuit Education Policy Framework. provincial school system. Accumulation of credits Five years after identifying the challenge of toward graduation is one of the primary indica- closing the achievement gap between Aboriginal tors of student success in high school. Using these and non-Aboriginal students, the Ministry has still two data sources as a means to analyze student not determined the size of the gap. Further, the achievement, we noted that, for 2011, a significant Ministry has not established a baseline or even a achievement gap continued to exist between self- starting point from which to measure the gap. The identified Aboriginal students and the rest of the Ministry has also not decided when it will begin student population. We also noted that this gap 3.05 measuring and reporting on Aboriginal student tends to increase as Aboriginal students make their achievement, nor has it decided what “significant” way through the educational system, as shown in Section means in its performance measures, such as to see Figure 3. VFM “significant improvement in … student achieve- The Ministry annually reports the graduation •

3 ment.” Finally, the Ministry has not developed rate for the entire student population, but it does specific targets in connection with the performance not calculate a graduation rate for Aboriginal

measures as required by the Framework. students. The Ministry advised us that it will begin Chapter The Ministry stated that it has not begun to to report on the graduation rate for Aboriginal assess the gap because, in its opinion, the 44% students following the 2013/14 school year. The of students who have self-identified are not suf- most recent information available, the 2006 census, ficiently representative to compare their achieve- identified that 65.5% of Aboriginal adults aged ment results with the results for the non-Aboriginal 20–24 had graduated from high school compared to student population. However, the Ministry has 88.4% of the non-Aboriginal population of the same

Figure 3: Comparison of Student Achievement Data, 2010/11 (%) Source of data: Ministry of Education

All Self-identified Gap in Aboriginal Measure Students Aboriginal Students Student Achievement Grade 3 EQAO: achieved provincial standard* 69 52 17 Grade 6 EQAO: achieved provincial standard* 68 49 19 Grade 9: 8 or more credits earned 83 63 20 Grade 10: 16 or more credits earned 74 45 29

* Overall average of the percentage of students who achieved the provincial standard on the reading, writing and mathematics tests. 140 2012 Annual Report of the Office of the Auditor General of Ontario

age—a gap of 23%, as shown in Figure 1. Our analy- incial and school district levels. Alberta Education sis of the credit accumulation data for 2011 shows reports on academic performance measures for that only 45% of the Grade 10 self-identified Aborig- Aboriginal students directly within its annual inal students were on track to graduate from high report. These measures include indicators similar to school, compared to 74% of all other students—a those contemplated in the Framework, such as the gap of 29%. These statistics, while not conclusive, level of achievement on province-wide standard- would certainly indicate that the Ministry faces sig- ized testing and high school graduation rates. nificant challenges in meeting its 2005 stated goal of closing the gap in achievement between Aboriginal RECOMMENDATION 3 and non-Aboriginal students by 2016. To help assess the progress being made toward One board we visited analyzed on a regular achieving the goals and performance measures basis indicators such as Grade 9 pass rates, and outlined in the Ontario First Nation, Métis and Grade 9 and 10 credit accumulation numbers. Inuit Education Policy Framework, the Ministry of From this type of analysis, the board identified Education (Ministry) and school boards should: that Aboriginal students were having difficulty establish a baseline with respect to the goals passing Grade 9 physical education. As a result, it • and performance measures identified in the introduced culturally relevant games and activities. Framework and set measurable, realistic This example shows that analyzing good data targets; and can uncover previously unsuspected barriers to periodically review progress made with achievement. We also noted that this board sets •

3.05 regard to closing the gap between Aboriginal measurable targets specific to Aboriginal student and non-Aboriginal student achievement so achievement. For example, in the 2011/12 school

Section that additional or alternative strategies can year the board aims to increase the percentage of be implemented where necessary. VFM self-identified Aboriginal students who achieve •

3 12 or more credits by the end of the first Grade 10 MINISTRY RESPONSE semester from 26% to 40%.

Chapter We also researched data collection and analysis The Ministry agrees that it is essential to measure at various other school boards in Ontario and noted and track progress being made toward achieving that one of the larger boards found that 80% of its the goals and performance measures outlined in Aboriginal students did not meet the EQAO prov- the Ontario First Nation, Métis and Inuit Educa- incial standard on the Grade 6 reading, writing or tion Policy Framework. The Ministry’s approach math assessments and, in general, Aboriginal stu- is consistent with lessons learned in its successful dents were highly at risk academically. This board student achievement agenda. The Ministry now reported that it used such information to assess the has a larger and more representative subset of gaps in student achievement and better target its Aboriginal student self-identification data. For support programs. the next phase of reporting in 2013 and moving Our review of public reporting on Aboriginal toward the target of closing the gap by 2016, the student results in other jurisdictions revealed that Ministry will engage with Aboriginal commun- the two provinces with the second and third largest ities to develop a reporting approach that best fits Aboriginal populations behind Ontario report on the current Ontario context. Aboriginal student achievement results annually. The Ministry will do the following: British Columbia’s Ministry of Education publishes • Engage with key First Nation, Métis and Inuit an annual report that provides information about partners in fall 2012 regarding its approach Aboriginal student performance at both the prov- to public reporting of the Aboriginal student Education of Aboriginal Students 141

Nation, Métis and Inuit Education Supplement, self-identification data in the second and the remaining $41 million was for Framework progress report. implementation funding, as detailed in Figure 4. • Release baseline voluntary Aboriginal student achievement data in spring 2013. • Review First Nation, Métis and Inuit student First Nation, Métis and Inuit achievement progress annually. Education Supplement Continue to support boards and engage • In 2007, the Ministry introduced the First Nation, Aboriginal partners in developing targeted Métis and Inuit Education Supplement to support strategies to support Aboriginal learners. the goal of improved Aboriginal student achieve- Share best practices and refine implemen- • ment as outlined in the Framework. This funding tation processes with boards in the 2013 (per-pupil amount) is intended to help boards progress report. offer and expand Native Language and Native Studies courses and support programs to assist BOARD RESPONSE Aboriginal students. The boards agreed that this recommendation The goal of Native Language courses is to was very important, with one board stating instill Native students with pride in their ancestral that data collection and analysis are critical to language and to motivate them to use it to com- improving achievement, life chances and life municate in their daily lives. Native Studies courses choices for Aboriginal students. This board went are aimed at providing all students with a broad 3.05 on to note that although individual boards can range of knowledge relating to Aboriginal history use their own data to compare their Aboriginal and cultures, and are offered at the secondary Section student results over time, they have limited school level to both Aboriginal and non-Aboriginal VFM access to provincial results for comparison pur- students. We found that the number of students •

3 poses. A second board stated that it had already enrolled in both Native Language and Native Stud- implemented additional, alternative strategies ies courses has increased since the supplement was

to support Aboriginal students and that a review introduced in 2007. Chapter of its data since 2008 demonstrated a continual We reviewed the per-pupil amount, which is increase in achievement for Aboriginal students. the largest component of the First Nation, Métis The third board stated that while data collection and analysis would be very important, the small Figure 4: Total Ministry Funding for Aboriginal sample size that it gathers would make the cor- Education Programs, 2006/07–2011/12 ($ million) relation between the impact of board initiatives Source of data: Ministry of Education and student achievement unreliable. Funding Education Supplement Per-pupil amount 60.0 FUNDING Native Studies 38.0 Native Languages 30.6 Since the inception of the Ontario First Nation, Total 128.6 Métis and Inuit Education Policy Framework in 2007, Framework Implementation the Ministry has allocated $170 million, primarily Implementation funding 35.2 to school boards, for Aboriginal programs over Alternative Secondary School Program 5.6 and above regular school board funding. Almost Total 40.8 $129 million has been provided through the First Total Funding 169.4 142 2012 Annual Report of the Office of the Auditor General of Ontario

and Inuit Education Supplement. The per-pupil in per-pupil funding to determine if this funding has amount is allocated to school boards based on the had any impact on Aboriginal student achievement. estimated Aboriginal student population as derived from the 2006 Statistics Canada census. One of the Framework Implementation Funding school boards we visited had identified, through the self-identification process, 50% more Aboriginal Framework implementation funding to school students than the number contained in the Min- boards is provided to support Aboriginal student istry’s estimate. This illustrates that the information achievement and help boards implement the the Ministry is using to estimate the number of Framework. Initiatives under this program include Aboriginal students at each board may not be up to self-identification policy implementation, student date and may not be reflective of the actual student success projects, professional development for population. Consequently, if funds had been allo- teachers and student transition programs. School cated on the basis of the actual Aboriginal student boards must submit proposals to the Ministry for population, this board would have received signifi- projects they plan to undertake and, upon project cantly more per-pupil funding than it received. completion, are required to submit a report detail- The Ministry intends that school boards will ing project outcomes. target this funding to local needs in support of We reviewed the proposal selection process for a activities linked to the Framework’s goals and per- sample of eight boards and found that there was lit- formance measures. However, the Ministry has not tle documented evidence of proposal prioritization placed any specific stipulations on this funding or by the Ministry or that a formal scoring system was 3.05 provided any formal guidance on how the funding used to rank proposals. Also, we were informed that is to be spent. According to the Ministry, this pro- ministry staff discussed proposals with each other, Section vides boards with the flexibility to determine how but these discussions were not documented and VFM best to allocate resources. In addition, although justification for rejecting proposals was not appar- •

3 the Ministry may have discussed program spending ent or communicated to the school boards we vis- with board representatives, there is no documented ited. In addition, there was no evidence that funds

Chapter evidence of such discussions and there is no formal were allocated on the basis of either the Aboriginal report-back process from the boards of any con- student population at each board or documented firmation that the funds have been used to support need. In fact, of the boards we reviewed, the board Aboriginal students. with the fewest Aboriginal students received more All three school boards we visited could pro- funding than the board with the greatest Aboriginal vide us with a brief explanation of how they had student population. On a per-student basis this spent the per-pupil funds. They informed us that worked out to $347 and $73, respectively. the funds were used for purposes such as paying We reviewed a number of project report-backs salaries for Aboriginal support workers, funding to the Ministry and found that the reports did not professional development for teachers, and provid- include indicators of success even though such ing resources for students making the transition indicators had been defined by the boards in their from on-reserve schools to provincially funded proposals. Some of the indicators of success noted schools. However, as is consistent with all ministry in proposals were to increase Aboriginal student grants for student needs, boards are not required to credit accumulation and participation in extra- formally track these expenditures to verify that the curricular activities. Although the project comple- per-pupil funds were used for the benefit of Aborig- tion reports discussed the perceived overall impact inal students. In addition, the Ministry has not of the projects, none of the reports reviewed directly performed any formal analysis on the $60 million Education of Aboriginal Students 143 addressed the result indicators noted in the related enrolment data, as this could result in a original project proposals. more equitable funding allocation; We also reviewed the Alternative Secondary where funding is allocated in response to School Program within Native Friendship Centres, • board proposals, document the underlying which provides individualized and culturally rationale for the funding and communicate appropriate supports to help Aboriginal students to boards the justification for accepting or complete their secondary school diploma if they rejecting their proposals; and have previously dropped out of school or have implement report-back processes not only fallen behind because of life circumstances. The • to demonstrate that funds are spent for the program is a working partnership between 11 purposes intended but also to obtain infor- friendship centres, which are not-for-profit corpora- mation on the success of different types of tions mandated to serve the needs of all Aboriginal support programs boards are undertaking. people, and their nearby district school boards. The district school board is required to provide teachers MINISTRY RESPONSE and course materials for the program. The Ontario Federation of Indian Friendship The Ministry agrees to consider the review of Centres (OFIFC), a provincial organization repre- existing funding mechanisms and reporting senting the collective interests of friendship centres, procedures. Other funding sources will con- provides administrative oversight of the program. tinue to be leveraged to support Aboriginal The OFIFC flows approximately $90,000 in ministry student achievement. 3.05 funding to each friendship centre annually. The The Ministry will do the following: OFIFC collects and sends to the Ministry financial • Examine the feasibility of remodelling the Section information and program outcome reports. These First Nation, Métis and Inuit Education VFM reports demonstrate that the funds were used by Supplement based on available Aboriginal •

3 the centres for the purposes intended and provide student self-identification data. information on the centres’ relevant successes and • Work more closely with boards to sup-

challenges including enrolment and exit informa- port their efforts to promote Aboriginal Chapter tion, diplomas and certificates awarded, and credits student self-identification. attempted and granted. We reviewed the report- • Ensure that all boards receive feedback on backs for the past three years and noted that the their proposals, that enhanced reporting OFIFC had submitted all the reports as required by processes are put in place by 2013, and that the Ministry. In addition, a 2009 ministry evalua- impacts on student achievement are con- tion of the program found that there were increases sistently evaluated as part of the proposal in student achievement, credit accumulation and evaluation and reporting processes. school retention rates.

RECOMMENDATION 4 FIRST NATION STUDENTS LIVING To better ensure that funding is allocated based ON RESERVES on the needs of Aboriginal students, the Min- First Nation Schools istry of Education (Ministry) should: • consider basing per-pupil funding on more The education of First Nation students living on current and reliable Aboriginal student reserve is the financial responsibility of the federal government. Through Aboriginal Affairs and 144 2012 Annual Report of the Office of the Auditor General of Ontario

Northern Development Canada, the federal gov- into the provincial education system. We inquired ernment provides funding directly to First Nation if the Ministry has worked with the federal govern- bands to deliver education programs and services. ment to promote early learning for First Nation stu- There are 118 band-operated schools on reserves dents and were informed that, due to jurisdictional in Ontario (80 elementary, 7 secondary and 31 matters, the province does not have a mandate alternative schools). The Ministry informed us that to implement programs or undertake research or approximately 12,700 Aboriginal students attend assessments with regard to students in on-reserve First Nation schools. schools. Nevertheless, in the 2012 budget, the The Ministry’s 2009 progress report, Sound government stated that Ontario is prepared to work Foundations for the Road Ahead, identified that together with First Nation communities and the there was a significant disparity between the level federal government to share the province’s exper- of funding targeted for the on-reserve schools by tise in delivering education, provided the federal the federal government and ministry funding for government supplies the necessary funding for on- provincial schools. In 2011, the Chiefs of Ontario reserve First Nation education. reported that federal funding was approximately 65% of per-student provincial funding. Research Tuition Agreements suggests that this difference in funding has resulted in schools on reserves being unable to pay competi- According to the Ministry, there are approximately tive salaries to teachers or purchase adequate, up- 5,700 First Nation students who live on reserves but to-date resources for students. This has resulted in attend provincially funded schools. Many of these 3.05 on-reserve schools often not being able to provide students attend provincially funded schools because the quality of education found in the provincial the reserve does not have a secondary school. Since Section school system. funding for these students is paid by the federal VFM The Education Quality and Accountability Office government to the band council, these students are •

3 (EQAO) may enter into agreements with First not considered to be pupils of the school boards and Nations to test students enrolled in band schools. In therefore the Ministry does not pay the cost of their

Chapter the 2010/11 school year, 39 out of 118 First Nation education. Therefore, boards are required to charge schools participated in EQAO provincial standard- band councils tuition fees through negotiated ized testing. These tests help First Nation commun- tuition agreements. Although the Ministry releases ities assess whether students in their schools are annual fee regulations that outline a base tuition meeting provincial standards. We analyzed the most fee, school boards may charge a fee in excess of recent EQAO results and found that, for participat- this amount for services such as special education ing First Nation schools, there was a gap on average programs or specific supports. Amounts charged in of 40% from the provincial average for Grade 3 and excess of the base tuition fee must be agreed upon Grade 6 students in reading, writing and math. by the school boards and First Nation communities These results reaffirm a 2004 report by the Auditor as part of the tuition agreement. General of Canada and other studies that suggest The Ministry does not keep track of the tuition that these students may be performing at a level agreements in place between the school boards and several grades below that which is appropriate for First Nation communities in Ontario. However, the their age. Ministry estimates that there are approximately 166 The Ministry has identified early learning as cru- tuition agreements and that school boards collect cial to student success. Even though education for about $60 million per year from First Nation com- students who live on reserves is a federal respon- munities to educate reserve students in provincially sibility, many of these students eventually transition funded schools. Education of Aboriginal Students 145

We identified that there should have been a to a provincially funded school—typically moving separate tuition agreement in place with each of from a primary school on the reserve to a provincial the 25 First Nation communities sending students secondary school. For example, some students to the three school boards we visited. However, we living in remote northern communities leave their noted that there were only 13 agreements in place; families to stay with host families in urban areas four agreements had expired, and there were 12 to attend secondary school. This may cause emo- instances where no tuition agreements existed. tional, physical, spiritual and psychological chal- Nevertheless, we tested a sample of invoices billed lenges for the student. As part of the Framework by school boards to First Nation communities and implementation funding, the Ministry provides found that the appropriate base tuition fee was funding for transition projects intended to assist charged and paid where no agreements existed school boards in providing supports to First Nation and, in general, First Nation communities were up students who live on reserve but attend provincially to date with payments. However, in one case we funded schools. found that the school board was charging in excess In the 2011/12 school year, the Ministry proof the base fee where it had no authority to do so, vided each board with up to $50,000 for Aboriginal since a negotiated tuition agreement did not exist. student success projects, including transition pro- The First Nation community paid the school board jects. We reviewed the transition projects approved the base fee, but the school board continued to bill over the last three years for the boards we visited excess fees that totalled $1.3 million over the last and noted that they often include assigning a lead three years. The representative responsible for edu- teacher for First Nation students and setting aside a 3.05 cation at this First Nation community indicated that room for these students to meet. One of the school the inability to agree on the amount being charged boards we visited had a formalized Aboriginal Section in excess of the base fee was the primary reason a transition program that included welcome kits, VFM tuition agreement had not been signed. assigning a special teacher to transitioning students, •

3 There are currently no standard ministry policies and offering Grade 9 and 10 transition courses. or procedures in place pertaining to tuition agree- Overall, we noted that the implementation of transi-

ments. The Ministry, the federal government, the tion programs was at the discretion of the boards. Chapter Chiefs of Ontario and four school board associations We reviewed provincial supports for transi- have been working together since 2009 to develop tions in other jurisdictions and found that Alberta tuition agreement guidelines. Rough drafts of these and Manitoba had created a number of resources guidelines had been distributed to the school boards to help educators with student transitions. For we visited. However, two of these boards were hav- example, Manitoba Education has released a policy ing problems negotiating tuition agreements. Both guide to help school boards identify specific student these boards suggested that if standardized template needs and develop transition plans. We noted that agreements could be developed and agreed upon one of the Ontario school boards we visited was by the Ministry, the federal government and First drafting a secondary school entrance form for First Nation communities, this would greatly facilitate the Nation students that includes sections pertaining process for local boards. to educational background (such as attendance, report card marks, standardized testing results and strengths in school), as well as any health issues or Transition of Students to the Provincial behavioural concerns. School System

First Nation students may experience many challenges when they transition from a school on a reserve 146 2012 Annual Report of the Office of the Auditor General of Ontario

First Nation Student Self-identification funded high schools. For example, in the 2010/11 school year only half of on-reserve First Nation stu- In the 2007 Building Bridges policy guide for dents attending provincial schools passed the Grade student self-identification, the Ministry identified 10 Ontario Secondary School Literacy Test, com- four groups under which Aboriginal students could pared to the provincial average of 83%. This indi- identify: First Nation (off-reserve), First Nation– cates that all stakeholders, including the Ministry, Tuition Paying (on-reserve), Métis and Inuit. How- school boards, the federal government and First ever, in October 2011, the Ministry eliminated the Nation communities, need to work more effectively distinction between on- and off-reserve First Nation to ensure that the needs of these students are students. The Ministry explained that it had elimin- identified, as more targeted support programs may ated this distinction because some students were be required to ensure that they are given an equal experiencing self-esteem issues in self-identifying, opportunity to succeed. some parents did not understand the distinction, and school staff found it difficult to have students change their identification when they moved Tripartite Agreements on and off the reserve. The Ministry stated that Since 2009, the Ministry has been engaged in these concerns had been informally reported to it. discussions with the federal government and First However, the Ministry did not assess more formally Nation communities to formalize education agree- whether these concerns were representative of the ments designed to promote collaboration between First Nation community as a whole or attempt to all parties on initiatives to improve First Nation

3.05 alleviate potential concerns before eliminating this student outcomes. One of these agreements, the distinction. The Ministry advised us that the data is Education Partnership Program, is a federally led

Section still maintained internally and therefore its ability initiative where First Nation communities submit to target programs to meet the needs of all Aborig- VFM proposals to the federal government to help •

inal students would not be impaired. 3 advance First Nation achievement in both First On-reserve First Nation students experience Nation and provincially funded schools. The pro- unique challenges as they transition to schools

Chapter gram supports partnerships between First Nation in the provincial system. With a significantly and provincial officials who share expertise and greater achievement gap, we believe students services, and co-ordinate learning initiatives. living on reserve continue to need to be identi- Joint initiatives with the federal government fied separately so the Ministry, school boards, and First Nation communities to improve the schools and First Nation communities can target academic achievement of students living in First specific supports to meet their needs. One of the Nation communities will not be implemented until school boards we visited during our audit was the agreements have been finalized. However, as reviewing its self-identification policy and planned of April 2012, the education agreements that the to include the First Nation on-reserve category. Ministry has been discussing since 2009 with the This board had determined that student supports federal government and First Nation communities can be better targeted if the board collects and had not been finalized. analyzes student achievement data for on-reserve Eight other provinces, including British Colum- students separately. bia, have signed tripartite education agreements Even though the Ministry does not analyze stu- with the federal government and their First Nation dent achievement data with respect to on-reserve communities. British Columbia has had an agree- students, we found that these students continue ment in place since 1999 and reports annually on to have difficulty transitioning into provincially Aboriginal education results. It reports that over the Education of Aboriginal Students 147 last five years (2006/07–2010/11) it has reduced 2012. Included in the guide are sug- the gap in high school graduation rates between gested examples of key provisions to be Aboriginal and non-Aboriginal students by 6%. included in any agreement. The Ministry will continue to work with First Nation RECOMMENDATION 5 and district school boards to support their In order to improve educational outcomes for negotiation processes. First Nation students living on reserves, the • Identify and share best practices in the Ministry of Education (Ministry) and, where development and implementation of tar- applicable, school boards, should: geted initiatives to support First Nation, • develop standardized template tuition agree- Métis and Inuit student achievement, includ- ments and guidelines that can be used by ing transition programs, in spring 2013. all boards and periodically monitor whether • Create and implement successful evidence- valid tuition agreements are in place with based targeted programs, based on all bands; proven success. • take a more proactive role to encourage • Develop and enhance successful transi- boards to share best practices to assist with tion programs for students moving from the transition of students from on-reserve schools on reserve to provincially funded schools to the provincial education system; schools in partnership with boards and First • separately measure the effectiveness of Nation communities.

initiatives implemented to address the Report on provincially funded activities 3.05 • unique challenges faced by on-reserve undertaken by boards and the Ministry to

students attending provincially funded support student transitions. Section

schools; and Continue to work with First Nations VFM

• •

continue to participate in and more pro- and the federal government to finalize 3

• actively engage in tripartite agreement and implement tripartite agreements as discussions with the federal government and negotiations proceed. First Nation organizations. Chapter BOARD RESPONSE MINISTRY RESPONSE All three boards recognized the importance of The Ministry agrees that support for First Nation helping students with the transition from reserve students is essential to close the achievement schools to the provincial school system. One gap. The Ministry recognizes that the treaty board noted that it would support any practice and Aboriginal rights of First Nations people that encourages integration and the fair treat- are recognized and affirmed in Section 35 of ment of all students within a strong, accountable the Constitution Act, 1982, and the treaties are a system. A second board noted that ongoing fundamental part of the relationship among the dialogue as well as newsletters and orientation First Nations, Canada and Ontario. Therefore, sessions helped smooth the transition for its the Ministry will develop its approach to any students. The third board noted that many of its reporting that may reflect on on-reserve schools First Nation students leave home, live in a board- in partnership with First Nation communities. ing home and move to a large secondary school The Ministry will do the following: all at the same time. The board stated that, in • Release the tuition agreement resource addition to the supports currently in place, it has guide by the end of the calendar year implemented a new program designed to provide 148 2012 Annual Report of the Office of the Auditor General of Ontario

academic supports, a safe space with a caring agreements and formats. The third board stated adult and a way for students to contact home. that it agrees with the concept of standardized In regard to tripartite and tuition agree- templates for tuition agreements but has found ments, one board noted that expediting efforts that First Nations are reluctant to sign new to finalize the tripartite agreement with the fed- agreements until there is a consensus among eral government would be very helpful to school all First Nations. The board noted that it would boards. A second board stated that the develop- continue to meet with its First Nation partners ment of a common tuition template agreement with the goal of finding mutually acceptable may be problematic, since this board serves agreements that serve their communities and several First Nations with very different tuition students in the best possible manner. 3.05

Section

VFM

•

Chapter Chapter 3 Ministry of Health and Long-Term Care Section 3.06 Independent Health Facilities

overhead costs, such as rent, staffing, supplies Background and equipment. In the 2010/11 fiscal year, the Ministry paid $408 million in facility fees. Total facility-fee Independent health facilities are located in com- payments increased by about 4% per year between munities throughout Ontario and provide certain

the 2006/07 and 2010/11 fiscal years, primarily 3.06 health services at no charge to patients insured because of increased volume of services (because under the provincially funded Ontario Health

the fees paid to facilities for each type of service Section Insurance Plan (OHIP). About 800 facilities provide remained largely unchanged). The total facility-fee VFM primarily diagnostic services (such as x-rays, ultra- •

payments are broken down by type of service in 3 sounds and sleep studies), and about 25 provide Figure 1. surgery (such as cataract and plastic surgery) or As well, the Ministry pays physicians working in

dialysis. Patients generally require a requisition Chapter these facilities—often radiologists—a standard fee signed by their physician in order to receive the for each service provided, known as a “professional services, and test results, where applicable, are sent fee.” The Ministry does not track the total profes- to this physician. Under the Independent Health sional fees paid to physicians for the specific servi- Facilities Act, the Ministry of Health and Long-Term ces they provide in independent health facilities. Care (Ministry) is responsible for licensing, funding The Ministry contracts with the College of Phys- and co-ordinating quality assurance assessments of icians and Surgeons of Ontario (the professional these facilities. oversight organization for physicians in Ontario) to The facilities are independently owned and obtain assurance on the quality of the services pro- operated, with most being for-profit corporations; vided by facilities licensed under the Independent less than 3% are non-profit organizations. The Health Facilities Act. As well, the Ministry conducts Ministry estimates that about half are fully owned inspections of facilities with x-ray equipment, or controlled by physicians, many of whom are including independent health facilities, under the radiologists who interpret, for example, x-rays and Healing Arts Radiation Protection Act. ultrasounds. The Ministry pays facility owners a “facility fee”—an amount for each type of service provided and/or a contracted amount—for

149 150 2012 Annual Report of the Office of the Auditor General of Ontario

Figure 1: Total Facility Fees Paid, by Type of Service, 2006/07–2010/11 ($ million) Source of data: Ministry of Health and Long-Term Care

2006/07 2007/08 2008/09 2009/10 2010/11 Diagnostic Services Ultrasound 129.9 138.0 150.7 166.2 173.0 Radiology (includes x-rays) 116.1 114.6 110.7 113.6 111.0 Nuclear medicine1 39.5 41.6 44.0 42.5 40.8 Sleep studies 29.3 32.8 36.5 39.1 39.6 MRI/CT 5.7 5.7 8.4 8.9 10.0 Pulmonary function studies2 2.1 2.0 2.1 2.3 2.4 Subtotal 322.6 334.7 352.4 372.6 376.8 Surgical/Therapeutic Services Dialysis 9.9 9.8 10.2 11.0 15.2 Abortion 6.4 6.2 6.5 7. 0 7. 1 Ophthalmology 6.4 6.5 6.4 6.4 6.4 Plastic 1.0 1.0 1.0 1.0 1.1 Vascular3 0.9 0.7 0.7 0.8 0.8 Laser4 0.4 0.3 0.4 0.5 0.4 Subtotal 25.0 24.5 25.2 26.7 31.0 3.06 Total 347.6 359.2 377.6 399.3 407.8

1. Patient tests using radioactive material (e.g., a small amount is administered and images are taken of where it goes in the body to observe blood flow

Section through the heart or lungs).

2. A test to measure a patient’s lung function. VFM

3. Surgery for patients who have problems with their arteries or veins. • 4. Surgery using a laser beam to cut tissue or remove a surface abnormality or lesion. 3

analyzed information, interviewed appropriate Chapter Audit Objective and Scope ministry staff, and reviewed relevant research from Ontario and other jurisdictions. As well, we surveyed independent health facilities to determine Our audit objective was to assess whether the whether certain services were available. We also Ministry had implemented systems and processes had discussions with the College of Physicians and to determine whether independent health facilities Surgeons of Ontario and the Canadian Nuclear were providing Ontarians with insured services in Safety Commission related to the operation of a timely and cost-effective manner, in accordance independent health facilities in Ontario. Further, with legislated requirements. Ministry senior man- we obtained information and had discussions with agement reviewed and agreed to our objectives and senior management about similar facilities in other associated audit criteria. Canadian jurisdictions. Our audit work was primarily conducted at We did not rely on the Ministry’s internal audit the Kingston offices of the Ministry’s Independent service team to reduce the extent of our audit work, Health Facilities Program. As well, we obtained because it had not recently conducted any audit information from the Toronto offices of the Min- work on independent health facilities. istry’s X-ray Inspection Services Unit. In conducting our audit, we reviewed relevant documents, Independent Health Facilities 151

medical procedures may be moved from hospitals Summary into community clinics, such as independent health facilities. Some of our other more significant observations include the following: Our last audit in 2004 found that the Ministry gen- Every facility is paid the same amount for erally had adequate procedures in place for ensur- • each type of service provided, regardless ing that independent health facilities complied of the total number of services the facility with legislation and ministry policies but noted a provides. Consequently, larger facilities in number of areas requiring ministry action. These urban areas often benefit from economies of included determining the service levels necessary scale, because certain costs (such as rent and to meet patient demand, the reasonableness of fees salaries for reception staff) generally do not paid for facilities’ overhead costs, the wait times increase proportionately with the number of for services and which community-based services services performed. Paying slightly higher should be subject to a quality assurance process. fees in locations with smaller populations Since our 2004 audit, the Ministry has taken and lower fees in high-density locations, for action to enhance the oversight of services offered example, might encourage more facilities in independent health facilities. For example, it to service areas that are currently under- has arranged for the College of Physicians and served without affecting the Ministry’s total Surgeons of Ontario (College) to conduct more facility-fee payments. frequent assessments of facilities with previously • The Ministry generally does not allow facilities identified issues; it has also amended the Medicine 3.06 to easily relocate to more underserved areas. Act, 1991 to enable the College to conduct service- • The Ministry estimated that certain servi-

quality inspections of community health clinics that Section ces—such as MRIs, dialysis and colonoscop- use anaesthesia (such as those providing plastic VFM ies—were about 20% to 40% less expensive • surgery and colonoscopies) but are not covered by 3 if delivered in community clinics, including the Independent Health Facilities Act. independent health facilities, rather than Nevertheless, in several other areas the concerns

in hospitals. Chapter we raised in 2004 still remain. For example, min- Although the Ministry has some information istry data indicates that patients in about 50% of • on facility ownership and estimates that about Ontario municipalities continue to be underserved 50% of facilities are owned or controlled by for certain diagnostic services (including radiology, physicians, many of whom are radiologists, ultrasound and pulmonary function studies). As it has not analyzed the patterns of physicians well, even though the Ministry has not increased referring patients to their own facilities or the rates paid to facilities for overhead costs since related persons’ facilities. Further, many 2006 (in fact, it decreased the rates by 2.5% in patients assume they must go to a facility spring 2012), the Ministry has not researched the listed on their physician’s referral form, when current actual overhead costs of providing the ser- in fact they can choose a hospital or any facil- vices. These costs may have changed significantly ity that offers the required service. In 2009, because new technology allowing certain tests to the Canadian Association of Radiologists be performed much faster often results in lower noted that as many as 30% of CT scans and overhead and staffing expenses. other imaging procedures across Canada con- Ensuring timely availability of services and rea- tribute no useful information or are inappro- sonable facility fees are both especially important priate. The Ministry estimated that about 20% because the Ministry’s 2012 Action Plan for Health of facility-fee tests are likely inappropriate. Care indicated that a number of less complex 152 2012 Annual Report of the Office of the Auditor General of Ontario

• Unlike hospitals, facilities are assessed by the College to help ensure, among other things, Detailed Audit Observations that diagnostic images are correctly “read” by the physician, who is often a radiologist. How- ACTIONS SINCE OUR 2004 AUDIT ever, as of March 2012, about 12% of facilities had not been assessed within the last five Since our last audit in 2004, the Ministry has years. Even for the assessed facilities, the Col- undertaken several initiatives related to independ- lege assessors did not review the work of all ent health facilities, including the following: physicians working at each assessed facility. • In 2006, the Ministry, in conjunction with • As of March 2012, the Ministry’s X-ray the College of Physicians and Surgeons of Inspection Services Unit (Unit) had not Ontario (College), initiated unannounced inspected almost 60% of facilities as fre- reassessments of some facilities and in 2009 quently as required to ensure that radiation- initiated more frequent facility assessments producing equipment—for example, x-ray for facilities with previously identified equipment—was appropriately shielded significant issues. to prevent staff and patients from being • In 2008, the Ministry began receiving from exposed to excessive radiation levels. the College the names of suspended phys- • The Unit and the Independent Health icians who had worked in facilities. Facilities Program (Program) areas did not • In 2011, the Ministry commenced a review of regularly share information. For example, the independent health facility billings to identify 3.06 Unit did not have the current location of 12 questionable billing practices. This review was facilities that had moved, so the Unit had not ongoing at the time of our audit. Section inspected whether the radiation-producing VFM equipment at the new locations for these • ACCESS TO SERVICES 3 facilities was safely installed. Further, the Unit’s inspection reports on facilities were not Distribution of Facility Services

Chapter routinely forwarded to the Program. Before the introduction of the Independent Health Although the Ministry has attempted to • Facilities Act (Act) in 1990, organizations could improve patient service by introducing two offer health services outside of hospitals and charge websites listing, among other things, certain patients a fee to cover their overhead costs. When locations where patients can obtain diagnostic the Act became effective, any organizations that services such as x-rays and ultrasounds, were already providing the health services covered neither site lists all locations offering these under the Act were “grandparented” if they sent services. One of the websites, which lists all a facility application to the Ministry and passed a independent health facility locations and ser- quality assurance assessment by the College. As a vices, could be made more user friendly: result, these organizations were permitted to be if it had search capability (for example, • licensed for the services they were providing, in the by postal code or service) to help patients location where they were providing them at that locate facilities; and time. Once licensed, they could bill the Ministry for if it included information on facility wait • their facility-fee costs, but were no longer permitted times for those services that historically do to bill patients for their overhead costs. not have same-day access (such as MRIs and CTs) to help patients who want their tests as soon as possible. Independent Health Facilities 153

Assessing Service Levels To assist in determining whether an area is Since grandparenting those facilities in existence underserved or overserved, the Ministry calculates at the time the Act became applicable to them, the the total number of services billed per capita by Ministry has approved only six new licences for hospitals for outpatients and by independent health facilities that bill the Ministry on a fee-for-service facilities in the Ministry-defined areas, and com- basis. However, facilities with licences may apply at pares these to the provincial average. This calcula- any time to provide services they are not currently tion is performed for the five services as well as for licensed to provide, in five service categories (radi- more than 15 subspecialties within those services. ology, nuclear medicine, ultrasound, pulmonary As shown in Figure 2, both hospitals and independ- function studies and sleep studies) or their sub- ent health facilities perform a significant number of specialties, including mammography (a radiology these services. subspecialty) and obstetrical and gynecological The Ministry does not have a benchmark for ultrasound (an ultrasound subspecialty). The Min- what constitutes a reasonable level of per capita istry will approve the request only if the area where diagnostic services. Therefore, the Ministry has the facility is currently operating is determined defined an underserved area as any area providing to be underserved for those diagnostic tests or less than 70% of the provincial average per capita procedures. The Ministry has divided the province service level. An overserved area is defined as any into a total of 105 areas, primarily municipalities of area providing over 150% of the average per capita varying sizes. service level. The Ministry determines all other In response to a recommendation in our 2004 areas (that is, those providing between 70% and 3.06 Annual Report, the Ministry indicated that its Diag- 150% of the average per capita service level) to be nostic Services Committee (with members from the adequately served. The Ministry indicated that, Section Ministry, the Ontario Medical Association and the based on its analysis, overall there is an adequate VFM Ontario Hospital Association) was expected, among supply of the five main services, with about 1,300 •

3 other things, to make recommendations concerning services per 1,000 people in Ontario. patient access to diagnostic services in underserved But the Ministry has not, for the most part,

areas. This Committee was discontinued in 2008, analyzed the distribution of underserved and Chapter and according to the Ministry, no such recommen- overserved areas across the province or over time. dations were ever made. We analyzed the Ministry’s data and noted that,

Figure 2: Number of Selected Diagnostic Services Performed by Hospitals and Facilities, 2010/11 Source of data: Ministry of Health and Long-Term Care

Hospital Outpatient Facility Type of Service Services Services Total Radiology (includes x-rays) 4,515,000 3,878,000 8,393,000 Ultrasound 2,152,000 4,267,000 6,419,000 Nuclear medicine1 752,000 432,000 1,184,000 Pulmonary function studies2 593,000 152,000 745,000 Sleep studies 43,000 106,000 149,000 Total 8,055,000 8,835,000 16,890,000

1. Patient tests using radioactive material (e.g., a small amount is administered and images are taken of where it goes in the body to observe blood flow through the heart or lungs). 2. A test to measure a patient’s lung function. 154 2012 Annual Report of the Office of the Auditor General of Ontario

according to the Ministry’s definition, about 50% of population had, for example, a higher percentage of Ontario municipalities (both rural and urban areas) women or older adults. had been consistently underserved and about 7% The Ministry does not encourage facility owners had been consistently overserved from the 2007/08 to relocate to areas within the province that are fiscal year to the 2010/11 fiscal year. We also noted underserved. Consequently, even though popula- that in some cases, an underserved area was next to tions in certain areas of the province may have an overserved one. For example, one underserved longer wait times for access to these diagnostic area with 632 services per 1,000 people in the services, the Ministry’s current method of approv- 2010/11 fiscal year was next to an overserved area ing new services to be offered by existing facilities in which 3,299 services per 1,000 people were may not be addressing the underservicing issue. provided. The Ministry had not analyzed which The Ministry believes that community hospitals areas it defined as underserved might in fact have may be better able to meet local service demands in adequate access to services that could be as close sparsely populated areas. as a few kilometres away in neighbouring areas. The Ministry indicated that, because independ- Furthermore, the Ministry’s most recent analysis ent health facilities provide only a small portion of of service availability per Local Health Integration other services such as Magnetic Resonance Imaging Network (LHIN, the corporation mandated to work (MRI) and Computed Tomography (CT), its analy- with local health providers to determine the health sis of the per capita levels of these other services by service priorities for its region, with the province LHIN generally excludes those provided by facili- being divided into 14 LHIN regions) was conducted ties. However, we noted that the per capita counts 3.06 in 2007. At that time, the number of services ranged of CT and MRI machines would rise by at least 10% from about 1,100 services per 1,000 people in one to 50% if machines performing the services at the Section LHIN to almost 3,400 services per 1,000 people facilities were included. VFM in another LHIN. Currently LHINs are responsible •

3 for planning and funding diagnostic services only Changing Facility Locations in hospitals, but the Ministry indicated that it is In order to ensure service availability across

Chapter considering the extent to which LHINs should be Ontario, especially in areas that the Ministry has involved with planning and funding diagnostic ser- determined are underserved, independent health vices at independent health facilities in the future. facilities are required to obtain ministry approval Furthermore, the Ministry does not age- or before relocating their operations. The Ministry gender-adjust the population statistics used for its permits facilities to relocate only within their per capita analyses. Because diagnostic procedures current Ministry-defined area, or to within five tend to be required more frequently as individuals kilometres of the current location if crossing into age, and some services are more likely to be pro- another area. Further, the Ministry will deny any vided to women rather than men, factoring in these move if a hospital within one kilometre of the pro- demographics would result in more meaningful posed new location objects. As well, the Ministry results. For example, obstetrical and gynecological will not approve moves from an overserved area ultrasounds are performed only on women, and to an underserved area if the move violates any bone mineral analysis is much more commonly of these rules. In 2011, the Ministry approved 47 performed on older adults, because bones become facility moves. less dense with age. However, all per capita analy- We noted that some Ministry-defined areas ses conducted by the Ministry were based on total are quite large, whereas other areas are small. populations, and did not consider whether an area’s For example, facilities within the city of Ottawa, Independent Health Facilities 155 which is almost 2,800 square kilometres, have Ontario’s Action Plan for Health Care (Plan), many more options for where to deliver services released by the Ministry in January 2012, notes as compared to facilities located within the city of that more routine procedures may be moved in the St. Thomas, which is about 40 square kilometres. future out of hospitals and into community-based Facilities located in the city of Toronto can move clinics, such as independent health facilities. The anywhere within it. We also noted that there were Plan specifically notes that more cataract proced- several donut-shaped areas (generally made up of ures will be conducted by an existing independent the areas surrounding a city) that facilities could health facility because the facility can do them at a move around in, but they generally could not move lower cost than a hospital can. Although not men- to the city within the centre of the area, and facili- tioned in the Plan, the Ministry indicated to us that, ties in the centre generally could not move to the in the 2013/14 fiscal year, it expects to increase the donut-shaped area. number of MRIs offered in one existing independ- The Ministry indicated that many facility owners ent health facility, also at a lower cost than in would like to relocate from less populated areas to hospitals. At the time of our audit, the Ministry had more population-dense locations. Consequently, no other specific plans to move any other services to certain owners have attempted to “leapfrog” from community facilities. their current location to a more populated location. For example, in one case, over a four-year period, a RECOMMENDATION 1 facility applied to move seven times, two of which To help ensure that Ontarians have timely and were approved and five of which were denied due

convenient access to required tests and pro- 3.06 to exceeding both the five-kilometre rule and cross- cedures, the Ministry of Health and Long-Term ing an area boundary. While these relocation con-

Care should: Section trols are undoubtedly necessary, we expected that better identify areas within the province VFM the Ministry would have considered what changes • •

where the combined levels of services 3 could be made to encourage facilities to move from offered by hospitals and independent health overserved to underserved areas. facilities indicate that the area is underserved (for example, by analyzing popula- Chapter Future Independent Health Facility Services tion and gender distribution within each area and determining the resulting needs for The Ministry has analyzed the cost of providing services); and certain services conducted in hospitals compared develop ways to help address patient needs with the cost of providing those services in the • in regions identified as underserved, such community and has determined that money could as offering incentives to encourage facilities be saved by moving various services, particularly to provide services in underserved areas or less medically complex ones, out of hospitals and reviewing policies that restrict a facility’s into community-based clinics, such as independent ability to move into underserved areas. health facilities. Specifically, the Ministry estimated that MRIs and dialysis, as well as colonoscopies MINISTRY RESPONSE and echocardiograms (ultrasounds of the heart), are less expensive—by a range of about 20% to The Ministry is pleased that the Auditor has 40%—if delivered by a community provider rather referenced the Minister’s Action Plan for Health than in a hospital. At the time of our audit, most of Care, which communicates our commitment these services were more frequently delivered in a to ensuring access for all Ontario residents to hospital setting than in the community. appropriate health-care services, including 156 2012 Annual Report of the Office of the Auditor General of Ontario

physicians in Ontario) and are intended to reflect timely and convenient access to required tests the approximate overhead cost (including rent, and procedures. Through the Action Plan, more staffing, supplies and equipment) of performing services will be provided in the community and each service. The rates are set out in the Schedule under a variety of service delivery models suited of Facility Fees for Independent Health Facilities. to the service type and community health-care Each facility fee has a corresponding professional needs. The Independent Health Facility (IHF) fee that the physician can charge for performing model of service delivery provides a strong the test or interpreting the test results. Professional foundation for moving more diagnostic tests fees, which are also negotiated between the Min- and procedures into the community. istry and the OMA, are set out in the Ontario Health To support the move of more services into Insurance Plan’s Schedule of Benefits. Figure 3 lists the community and to ensure that services are the top five facility fees billed, the volume of each planned and delivered in accordance with local service, and the corresponding professional and and regional population needs, the Ministry is: facility fees per service. The Ministry did not have actively exploring opportunities for joint • information available on the total professional fees planning between the Local Health Integra- billed for these services. tion Networks and IHFs to ensure the right Between 1992 and 2005, the Ministry made mix and distribution of diagnostic tests and some across-the-board decreases and one increase procedures to meet local health needs and to the Schedule of Facility Fees for Independ- to better identify areas within the province ent Health Facilities. From 2005 on, no changes

3.06 where the combined levels of services were made until May 2012, when the Ministry offered by hospitals and IHFs indicate that implemented a 2.5% across-the-board reduction in

Section the area is underserved; and facility fees and a 5% reduction in their associated enhancing the planning relating to the VFM • professional fees “in recognition of the latest evi- •

volume of services delivered in the com- 3 dence, improvements in technology, and changes munity and closer to home for patients, in standards of care.” Reductions were also imple- with consideration to incentive options to

Chapter mented for certain other service fees, such as those attract IHF service providers to underserved for colonoscopies. areas, as well as reviewing its IHF reloca- All facilities are paid the same amount for each tion policies to better facilitate the move- service, no matter where the facility is located ment of diagnostic tests and procedures to or how many services it provides. Consequently, underserved areas. larger facilities can benefit from economies of scale because certain costs, such as rent and the salaries of reception staff, do not increase proportionately BILLINGS with the number of services performed. A 2000 report produced by the Committee on Funding Arrangements Technical Fees (members of which included the Fee-for-service Facilities Ministry, the OMA and the Ontario Hospital Asso- Fee-for-service independent health facilities (for ciation) noted that “cost reimbursement should be example, x-ray and ultrasound facilities) are paid used as the underlying principle for the funding of a standard facility fee for each type of service technical components of diagnostic services” and performed. These fees were negotiated between acknowledged that most of the facility fees had not the Ministry and the Ontario Medical Association been set through a rigorous costing process. The (OMA, the bargaining organization that represents Committee suspected that with the introduction of Independent Health Facilities 157

Figure 3: Top Five Total Facility Fees Paid, 2010/11, and Associated Fee Levels Source of data: Ministry of Health and Long-Term Care

2010/11 As of April 1, 2011 ($) Total Facility Fees Facility Fee Professional Fee Type of Service Paid ($ million) Volume Paid per Service Paid per Service Chest radiology (x-rays—2 views) 20.0 826,000 24.18 11.30 Pelvic ultrasound—complete (physician present at 17. 7 355,000 50.00 32.30 time of service) Pelvic ultrasound—complete (physician not present 16.4 328,000 50.00 24.20 at time of service) Sleep study (initial diagnostic) 15.5 41,000 380.25 128.30 Ultrasound of vagina or rectum 14.6 291,000 50.00 24.20 new technology and equipment (which make some tions’ fee levels. We noted that both British Colum- services less time-consuming to perform), some bia and Alberta provided a range of services, both fees did not accurately reflect the costs, and noted surgical and diagnostic, in community clinics but that there was a lack of information on the extent they did not have separate professional and facility- to which fees deviated from actual costs. Therefore, cost fees; rather, the entire payment was combined the Committee recommended that the fee schedule in one fee. We compared the combined facility and

be reviewed as soon as possible, and suggested that professional fees paid in Ontario for three services 3.06 an appropriate costing methodology would ideally to the fees paid for the same services in British Col-

incorporate factors such as the economies of scale umbia and noted that the Ontario fees were within Section available to high-volume facilities. 6%, more or less, of the fees paid there. VFM

•

In our 2004 Annual Report, we recommended In April 2011, the Ministry requested that 3 that the Ministry “objectively determine the current the OMA (representing physicians), the Ontario cost of providing each type of service and examine Hospital Association (representing hospitals) and the relationship between the volume of services the Coalition of Independent Health Facilities Chapter provided and the costs of providing services.” At (representing facilities) submit facility-fee funding that time, the Ministry indicated that the Diagnos- options to it by October 2011. The Coalition’s tic Services Committee would be reviewing this response did not suggest any funding options; area. But this Committee was discontinued in 2008, it focused on structuring future funding discus- without completing any such analyses, making any sions. Neither the OMA nor the Ontario Hospital related changes to the fee schedule or doing any Association submitted options, although both were work to assess the current overhead costs of run- given an extension until March 2012 to do so. The ning a facility. Ministry indicated that it did not expect any further The Ministry did, however, provide us with a submissions relating to facility fees from the OMA 2008 Jurisdictional Comparison of Medical Imaging or the Ontario Hospital Association, because the Systems prepared by another province involving renegotiation of the Physicians Services Agreement, five Canadian jurisdictions, using data from the which sets out compensation for physicians, was 2006/07 fiscal year. The comparison found that, at the key priority in 2012. that time, Ontario’s facility fees were often lower As previously noted, the Ministry will not than fees paid by other provinces. However, the approve new services for a facility if the area is Ministry has not investigated whether this is still overserved for the services in question. But facilities the case, using updated data on the other jurisdic- can increase the number of services they currently 158 2012 Annual Report of the Office of the Auditor General of Ontario

provide at their existing locations by purchasing, facilities. The Panel’s 2005 report indicated that without ministry approval, additional diagnostic these services were not being delivered efficiently. equipment—the Ministry’s licence with facilities In the case of MRIs, this was primarily due to the requires only that they inform the Ministry within Ministry’s having allocated a longer-than-necessary 30 days of new equipment being purchased. More time for certain types of MRI procedures and pay- equipment leading to more services being provided ing facilities for rerunning MRIs when the facility results in the Ministry paying more facility and made an error. In the case of CT scans, the report professional fees. But if the facilities providing the indicated that the tests should take 20 minutes services are in an underserved area, the area bene- regardless of which body part is being scanned, fits from the increased service availability. whereas the contract rates were based on estimated In contrast to the unlimited number of services times per service that were, on average, more than that facilities can receive payment for, laboratories, 60% longer. The Ministry told us that it did not which are also funded on a fee-for-service basis, are at that time consider the rate reductions for CT subject to a provincial maximum funding level for scans suggested by the Panel to be appropriate for all labs across the province as well as a maximum facilities, although it had no analysis supporting its that applies to each laboratory. conclusions. When renegotiating five-year contracts between 2007 and 2009 for the five independent health facilities performing MRIs, the Ministry Negotiated Contract Facilities again did not consider the Panel’s recommenda- The Ministry funds 35 facilities, including facilities tions. On the other hand, a couple of years after

3.06 performing MRIs and cataract surgery, through these contracts were negotiated, the Ministry did negotiated contracts for providing an established approach these MRI facilities regarding a rate

Section volume or number of hours of services. If a facil- reduction. While some of them agreed to reduced ity performs fewer services than the contracted VFM rates, the others refused. The Ministry continued to •

amount, the Ministry reduces the facility’s fund- 3 pay these facilities at the 2007 contract rate while ing. Facilities performing more than the contract paying the other facilities the reduced rate. The amount do not receive additional ministry funding.

Chapter Ministry indicated that it expected to negotiate a The contracted dollar amount (based on volume reduced rate with the other facilities when the con- and hours) for facilities that were “grandparented” tract is renewed in 2012. when the legislation was introduced was deter- With most negotiated contracts, the Ministry mined through negotiations between the Ministry does not periodically confirm that the rates paid are and the facilities. Subsequently, in seven cases, the reasonable. Although it might not be practical to Ministry approved MRI and CT services, which had always conduct a competitive process, the Ministry not previously been covered under the Act, through could, for example, periodically obtain information a competitive process. For other new services, from other jurisdictions or otherwise periodically such as cataract surgeries, the Ministry negotiated review costs to help obtain assurance on the rea- directly with known providers, as permitted under sonableness of the rates. Because facility fees are the Independent Health Facilities Act. In all cases, not being periodically reviewed, there is a risk that subsequent contracts are negotiated between the some services are increasingly expensive to provide, Ministry and each facility. but more of a risk that technological improvements In 2004, the Ministry established an Expert have resulted in facility fees significantly exceeding Panel to help improve, among other things, access the actual cost of performing the services. to, and the quality and efficiency of, MRI and CT services in hospitals and independent health Independent Health Facilities 159

Referrals for Service One condition for maintaining a licence for a fee-for-service facility is ministry approval of any Patients generally require a physician’s referral changes in the facility’s controlling ownership; in in order to obtain services from an independent the calendar years 2008 through 2011, more than health facility. The Health Council of Canada indi- 175 such changes were approved. Facilities are cated in its 2010 report Decisions, Decisions: Family required to confirm ownership every five years as Doctors as Gatekeepers to Prescription Drugs and part of their licence renewal. In addition, some Diagnostic Imaging in Canada that increased access owners forward documents indicating the date to diagnostic imaging has “allowed for increased on which changes were made in share ownership. use of diagnostic imaging, some of which may be However, the Ministry does not otherwise peri- considered over-use.” The report noted research odically ask facilities to confirm ownership by, for indicating the overuse of diagnostic imaging, but example, having owners tick a box on a form and concluded that it is difficult to say why and by how sign if ownership has not changed or state chan- much. In this regard, the Canadian Association of ges in writing and sign. As a result, the Ministry Radiologists’ website notes that as many as 30% spends extensive administrative effort in tracking of CT scans and other imaging procedures across ownership structures. Canada contribute no useful information or are Although the Ministry estimates that about half inappropriate. Studies from other jurisdictions also of Ontario’s facilities are fully owned or controlled indicate the incidence of potentially inappropriate by physicians, it has not analyzed patterns of refer- imaging. On the basis of studies in two other juris- rals by these physicians to their own facilities. The

dictions, the Ministry estimated that about 20% of 3.06 Ministry also does not have information on whether facility-fee billings in Ontario are likely inappropri- the physicians who refer patients to a given facility

ate—for example, due to unnecessary testing. Section have a spouse or other direct relative who owns In some cases, the physician referring the VFM part or all of the facility. The Ministry could use a • patient is the physician who performs the service, 3 form confirming ownership to also periodically con- or otherwise works at or owns the facility to which firm whether the owner(s) or any of their immedi- the patient is being referred. This is called “self-

ate family are physicians. Chapter referral,” because the physician is referring the In May 2012, the Ministry proposed to pay only patient to him- or herself. Physician self-referrals 50% of the facility and professional fees for services can improve patient care in certain situations: for where a physician referred a patient to the clinic example, ultrasounds of the eye may be ordered the physician worked at. The Ministry had not, and completed by an ophthalmologist to help however, estimated the impact of this 50% reduc- diagnose and treat suspected eye disease earlier. tion on independent health facilities, although However, various studies, including a Journal of ministry documents indicate that some facilities the American Medical Association article published might be forced to close operations and that there in 2012, have indicated that “evidence continues was a potential reduction in the quality and safety of to mount showing that physicians with ownership imaging or testing services because clinics might use stakes in imaging equipment are more likely to refer “shortcuts” to reduce costs in order to remain viable. their patients for imaging tests than physicians who Furthermore, the Ministry generally did not have send their patients to radiologists for independent any information on what might be a reasonable imaging.” The article further noted that “when proportion of self-referral tests and what proportion physicians can refer to scanners they own, [and] appeared excessive and thereby warranted follow- there is no third-party oversight, they might be up. In June 2012, in response to concerns raised making subconscious decisions to image.” by the Ontario Medical Association, the Ministry 160 2012 Annual Report of the Office of the Auditor General of Ontario

announced that it was establishing a panel to review that about 25% of operating facilities had some physician self-referrals and would wait for the unusual billing patterns. For example, five facilities panel’s recommendations before changing the fees were billing more than 500 and up to 2,200 com- related to self-referrals. bined head and pelvis ultrasounds on the same visit for patients, whereas most facilities billed fewer than 50 of this ultrasound combination. Accord- Verifying Billings ing to the Ministry, this diagnostic combination is As shown earlier in Figure 1, total payments to expected to occur only rarely, because few patient facilities have increased by about 4% a year, from conditions can be diagnosed by using it. Despite $348 million in the 2006/07 fiscal year to $408 mil- this being a good analysis, the Ministry believed lion in 2010/11. In particular, facility-fee payments that many of the unusual billings could be resolved for diagnostic services increased by about 4% a through educating facilities on appropriate bill- year, from $323 million to $377 million. However, ing practices and indicated that it was developing ultrasound services increased by 7% a year, from educational materials that would be shared with $130 million to $173 million, over this time period. facilities starting in fall 2012. In addition, although More specifically, with 12 ultrasound services, the Ministry provided us with a list of potential both numbers of services performed and payments additional data analyses that might identify other increased by more than 50% between 2006/07 and inappropriate billing practices, no plans were in 2010/11. For example, the number of ultrasound place to conduct such analyses. billings under the code “miscellaneous extremities The Ministry provides the College of Physicians 3.06 charge per limb” increased 130% from 2006/07 and Surgeons of Ontario with detailed information to 2010/11 or over 20% per year on average, from on the services billed by each facility. However, Section a total of $2.9 million to $7.5 million. Facility fees the College informed us that although it does VFM have changed very little, so most of this change was receive patient-care data from the Ministry, this •

3 due to increases in the volume of services provided. information is not provided to assessors and that Total payments for contracted services (such as the assessors’ role does not include any claims

Chapter dialysis and abortions) increased from $31 million verification processes. Rather, the College indicated in 2006/07 to $41 million in 2010/11. The biggest that assessors are provided with only the number increase was due to the Ministry providing one-time of patients who received a particular service within funding in 2010/11 for the replacement of about 70 a particular time—for example, the previous few dialysis machines at the facilities offering dialysis ser- months. The College’s assessors select a sample of vices. Although the Ministry reviews service volumes procedures performed on patients based on infor- and their effect on total costs to some extent for con- mation provided by the facility visited to ensure, tracted services, it has not reviewed the reasons for among other things, that each has an appropriately changes in the volume of fee-for-service payments. authorized requisition from a physician. However, The Ministry indicated that historically it has the assessors do not test whether the procedures informally identified (for example, through com- billed to the Ministry were actually performed. We plaints and assessments of facility expansion appli- also noted that the Memorandum of Understanding cations) questionable facility billing practices a few between the Ministry and College did not state that times a year, resulting in about one referral a year assessors were expected to check facility billings to to the Ontario Provincial Police. In November 2011, the Ministry, either by using data from the Ministry the Ministry began a claims integrity project, which or by any other method. involved reviewing the data on facility-fee claims Facilities are to send the Ministry information on for the 2010/11 fiscal year. This review identified when physicians start or stop working at the facility. Independent Health Facilities 161

The Ministry indicated that it will pay only those for selected services, periodically verify that billings from a facility that relate to physicians who • facilities have billed the Ministry only for are listed as working at that facility. Physicians services provided to patients—for example, sign the form that facilities submit that states the through matching facility billings to phys- physician’s start date, but when facilities submit the ician requisitions or to the associated phys- form that states a physician has left, the physician ician’s professional fees for the same service. generally does not sign the form, nor is a departure date indicated. As well, every facility fee billed to MINISTRY RESPONSE the Ministry generally should have a corresponding professional fee billed by the facility physician The Ministry will consider the feasibility of who interpreted the diagnostic test or, in the case conducting an inter-jurisdictional review and of surgery, performed the procedure. The Ministry a relative-value-to-cost analysis of the tech- indicated that it does not periodically reconcile or nical fees that it pays to Independent Health spot-check the facility fees and the professional fees Facilities (IHFs). billed by facility owners and physicians, respect- The Ministry agrees that information about ively, to identify discrepancies. equipment that is purchased by IHF operators is of interest. In this regard, the Ministry estab- RECOMMENDATION 2 lished a grant program in the 2011/12 fiscal year to assist IHFs in the purchase of digital To enhance the cost-effective management of equipment and is considering a second year of

the Independent Health Facilities Program, the 3.06 grant funding for this purpose. Furthermore, Ministry of Health and Long-Term Care should: the Ministry is considering capacity planning at

periodically review the fee it pays to Section • the local level involving the Local Health Inte- independent health facilities (to cover staff- VFM gration Networks, as well as possibly requiring •

ing, equipment and other overhead costs) 3 approval prior to capacity expansion by IHFs by assessing the actual costs of the services operating in overserved areas. and by making periodic comparisons to

The Ministry is concerned about self-referral Chapter other jurisdictions; and ensuring appropriate utilization of diagnos- consider alternatives for better managing • tic and imaging services in Ontario, including the volume of fees chargeable by facili- services rendered in physician-owned facilities. ties in overserved areas, such as requiring The Ministry’s Expert Panel on Appropriate these facilities to obtain ministry approval Utilization of Diagnostic and Imaging Studies is before increasing capacity by buying expected to make recommendations in fall 2012 more equipment; regarding the appropriate utilization of diag- consider requiring facility owners to declare • nostic and imaging services rendered in vari- all potential conflicts of interest to the Min- ous professional settings, including IHFs. The istry, and periodically review billing data recommendations, when received, will be fully to identify facilities with unusual billing reviewed with the Ontario Medical Association patterns, including billings resulting from and actions will be formulated to address issues unexpectedly high levels of self-referrals of relating to self-referral and appropriate utiliza- patients by physicians who own or work at tion in Ontario, including in the IHF sector. As that facility, or who are related to someone well, the Ministry agrees that the level of phys- who owns the facility—and follow up with ician ownership or material interest in an IHF these facilities; and business operation is of interest and is actively 162 2012 Annual Report of the Office of the Auditor General of Ontario

If the facility does not meet the Standards, the exploring mechanisms to require that physicians report is reviewed by the Ministry’s medical con- report this information. sultant, who advises the Ministry on any additional In addition, the Ministry is taking further actions that should be taken. When requested to action to implement processes to monitor and do so by the Ministry, the College will conduct a manage utilization through proactive claims- reassessment, and occasionally focused assess- data analyses, provider outreach and education, ments to follow up on certain issues. The Ministry clinic audits, and payment recovery processes. may act to suspend or revoke licences based on the The clinic audit process would include verifica- assessment report’s recommendations. tion processes to ensure that facilities have The Ministry paid the College about $1.5 million billed the Ministry only for services they have for conducting the quality assurance processes at provided to patients. facilities, which included more than 200 facility assessments in the 2010/11 fiscal year. At the time of our audit, the Ministry and the College were PERFORMANCE MONITORING negotiating a revised MOU, which was expected to require the facilities to pay the College for College of Physicians and Surgeons of conducting assessments, rather than the Ministry Ontario Quality Assurance Assessments paying the College. To help ensure the quality of services, all facilities are required under the Act to have a quality adviser, 3.06

Determining Assessor Independence who is responsible for advising the owner about the Because assessors must have experience in the area facility’s quality and standards of services. Facility

Section that they are reviewing, most of the physicians and owners who are physicians may appoint themselves technologists who perform facility assessments VFM as the facility’s quality adviser. To obtain independ- •

for the College regularly work in hospitals and/ 3 ent assurance on the quality of the services pro- or independent health facilities. To help prevent vided by facilities, the Ministry has entered into a potential conflicts of interest between an asses-

Chapter Memorandum of Understanding (MOU) with the sor and the facility that he or she is assessing, the College of Physicians and Surgeons of Ontario (Col- College verbally asks assessors if they have any lege), the professional oversight organization for potential conflicts (after providing examples of physicians in Ontario. Under this agreement, the what would constitute a conflict, such as being College is to conduct quality assurance assessments related to someone or having previously worked of the services provided by facilities using “Clinical with someone at the facility to be assessed). Asses- Practice Parameters and Facility Standards” (Stan- sors are generally not permitted to assess a facility dards) developed by the College. with which they have a conflict. However, the Col- The assessments are conducted by College- lege does not require assessors to sign a document appointed assessors—usually a team of one phys- declaring that they have no conflicts. ician and one technologist who have experience Although the College is responsible for in the facility service(s) being assessed. Assessors appointing facility assessors, because of the com- generally spend about a day at a facility and then petitiveness of certain facilities, facilities have the submit a report to the College. This report indicates option to reject an assessor and have an alternate whether the facility met the Standards and provides appointed in certain situations (such as potential detailed observations to support the conclusion, conflicts of interest). The College informed us that as well as recommendations for improvement if facilities exercise this option about 2% of the time. needed. Once reviewed by the College, the report is forwarded by the College to the Ministry. Independent Health Facilities 163

Scheduling Assessments the same minimum sample sizes (between 10 and The Ministry uses its database, which includes the 15 per type of service provided, depending on the dates of all facility assessments and reassessments, type of facility) for all facilities providing a particu- to identify which facilities it will request the Col- lar service, regardless of the number of procedures lege to assess during the next year. The Ministry’s provided by the facility. We also noted that the general policy is to have each facility assessed every assessors’ approach to choosing this sample did not five years. It provides that year’s list to the College, ensure that they reviewed the work of all profes- which is responsible for scheduling assessments sionals at the facility. with each facility. After it receives the assessors’ report, the Col- We found that, as of January 2012, the College lege is to report assessment results to the Ministry had not assessed within the previous five years within three to 20 business days, depending on about 12% of the facilities that should have been the seriousness of the assessment results (these assessed. The Ministry indicated that assessments time frames were implemented in 2005). We noted could be delayed for various reasons, including a that the College generally met these deadlines in lack of assessors with service-specific knowledge the 2010/11 fiscal year. But there is no established (for example, the College indicated that assessors deadline by which assessors must report results to for sleep study clinics are difficult to find because the College. We noted that the time between the fewer people specialize in this area) and facilities assessment date and the date that the Ministry postponing assessments because of proposed moves received the assessment results from the College to other locations. was a median of 47 days in the 2010/11 fiscal 3.06 In our 2004 Annual Report, we recommended year, with 90% of reports being received within 84 that the Ministry consider having the College days. Most of this longer time frame was the time Section perform at least some assessments without between the assessment date and the College’s VFM advance notice. At the time of our current audit, receipt of the report from the assessor. •

3 facilities were always provided with at least six The assessment results provided to the Ministry to eight weeks’ notice so that they could prepare rate facilities on a scale of one (the highest rating:

for the visit. (Reassessments may be unan- the facility is following the required standards, and Chapter nounced, as discussed further under Following Up there are no recommendations) to five (the lowest on Assessments.) rating: patient care is at risk, and recommendations may include the immediate closure of the facility). Between January 2007 and January 2012, more Conducting Assessments and Reporting than 80% of the approximately 1,100 facilities that Assessment Results were assessed were rated as meeting the standards The College’s assessors use a checklist based on the with few or no significant recommendations (that College’s “Clinical Practice Parameters and Facility is, the facilities received one of the top two ratings). Standards” (Standards) to conduct their assessments. The Ministry has a copy of the Standards but generally does not receive a copy of the checklists Following Up on Assessments used. To complete the checklist, assessors select The College generally determines whether facilities and review a sample of test images (in the case of require any additional follow-up after an assess- diagnostic facilities) and/or patient charts, surgical ment, based on feedback from its assessors. Follow- notes, and so on (in the case of dialysis and surgical up can range from requiring the facility to forward clinics); where possible, they also observe tests documents, such as an action plan detailing how it being performed on patients. We noted that the will address identified deficiencies, to the Ministry College’s sampling guideline recommended using requesting the College to conduct a reassessment. 164 2012 Annual Report of the Office of the Auditor General of Ontario

Reassessments are generally conducted three producing equipment, including primarily the months to a year after the original assessment, equipment used to perform x-rays but also the based on the College’s judgment. Six facilities were equipment used for CT scans and fluoroscopy reassessed in the 2010/11 fiscal year. (which involves a series of x-rays that produce a In our 1996 and 2004 Annual Reports, we continuous moving image on a monitor, giving, noted that the Ministry and the College had not for example, a picture of the movement of contrast agreed on timeframes for the College’s follow-up dye through a patient’s body). The Ministry’s X-ray activities, and recommended that the Ministry Inspection Services Unit (Unit) is responsible update its Memorandum of Understanding with the for these inspections, as well as for ensuring College to incorporate such timeframes. However, the appropriate initial installation of radiation- no timeframes were in place at the time of our producing equipment (for example, sufficient lead current audit. in the walls for shielding). The Unit also ensures The Ministry can ask the College to conduct its that x-ray equipment does not produce radiation planned reassessments on an unannounced basis. in excess of standards set out in the HARP Act. In these cases, the College either sends assessors Radiation produced by CT equipment is not in completely unannounced or notifies the facility reviewed by the Unit because there are no similar that an assessment will be conducted within the standards for CT equipment. next month, without providing a specific date. We The Unit is responsible for inspecting just over noted one case where the facility denied the asses- 7,600 locations—including hospitals, dental sites sors entry on two “unannounced” occasions in 2010 and more than 450 independent health facili- 3.06 before providing access in March 2011, even though ties—with radiation-producing equipment. In the Independent Health Facilities Act states that all the 2011/12 fiscal year, the Unit inspected a total Section facility owners are required to co-operate fully with of nearly 1,700 locations, including about 70 VFM assessors. However, the Act has limited penalties for independent health facilities. •

3 refusal to co-operate, and the College told us that it The Unit has four inspectors who perform does not have any authority to impose penalties in announced visits that are scheduled using

Chapter this situation. risk-rankings for the various types of facilities. In the 2009/10 fiscal year, to obtain more For example, independent health facilities are assurance that facilities with previously identi- required to be assessed within one year of opening fied and rectified problems were continuing to a new location and every two to three years after meet standards, the Ministry asked the College to that, unless they receive a ranking of “bad” as assess certain facilities every two to three years, a result of an inspection, in which case they are rather than the standard five years. In the 2010/11 inspected annually until the situation is rectified. fiscal year, the College conducted 26 of these Ministry policy is to inspect all radiation-producing mid-cycle assessments. equipment at new locations and about 25% of the equipment at established locations. The Unit cannot easily determine the percentage of facility Inspections Conducted by the X-ray equipment tested because it did not document all Inspection Services Unit equipment in use at the inspected facilities. We While the College’s standards require facilities also noted that ministry policy does not indicate to check radiation-producing equipment every that inspectors should ensure that all equipment is six months, the Ministry is responsible under the tested over time; rather, the policy indicates that Healing Arts Radiation Protection Act (HARP Act) higher-risk equipment, such as fluoroscopy equip- for periodically inspecting a facility’s radiation- ment, should be tested. Independent Health Facilities 165

We reviewed the data from the Unit’s inspec- Ministry Monitoring tion database and found that as of March 2012, Suspensions and Revocations of Licences almost 60% of the independent health facilities The Independent Health Facilities Act (Act) states had not been assessed within the Unit’s prescribed that the Ministry can revoke, suspend or refuse to time frames. Furthermore, the Ministry could renew a facility’s licence for a variety of reasons. not determine how many of these facilities were Problems can include quality assurance issues new or how many had been rated as “bad” in their (such as equipment requiring maintenance) and last inspection. operational issues (such as facilities operating The Unit does not rely on the work of other over- out of unapproved locations; ceasing to operate sight entities, including the Canadian Association for at least six months without taking reasonable of Radiologists, the Ontario Association of Radiolo- steps to prepare to reopen; or transferring over gists and the Ontario Breast Screening Program, 50% of a facility’s ownership without obtaining all of which review certain types of radiation- ministry approval). producing diagnostic imaging equipment, including The Act allows the Ministry to immediately equipment in independent health facilities. The suspend a facility’s licence if there is an immediate work of these organizations might enable the Unit risk to a patient’s health or safety. We noted that to reduce its time at some facilities and focus its the last suspension occurred in 2011. The Ministry efforts on facilities and/or equipment that are not indicated that in most cases, there is no immediate otherwise being tested. risk to patients. In these cases, under the Act, the The Ministry had no policies or procedures on

Ministry issues a “proposal to suspend” a facility’s 3.06 what type of information should be exchanged licence, an approach that provides the facility with between the Unit and the Ministry’s Independent

time to correct any identified quality assurance Section Health Facilities Program; rather, communications problems in order to avert a licence suspension. VFM occur at the discretion of staff in each ministry •

Between January 2007 and January 2012, the 3 area. We noted that minimal communication or Ministry issued such notices to 32 facilities. The exchange of information took place between the Ministry ultimately suspended seven for not taking

Unit and the Program. For instance, the Unit’s Chapter the required corrective action on a timely basis. inspection system and the Program’s database were The Ministry can revoke the licence of sus- not linked, and although the Ministry indicated pended facilities that do not correct identified that data comparisons to identify discrepancies problems, as well as facilities with operational could be conducted manually, this has not been issues. The Ministry’s records indicated that six done. As a result, for example, the Unit incorrectly licences were revoked from January 2007 through categorized almost 40 independent health facilities January 2012. Most of these revocations occurred as dental offices, which are generally inspected because the facility had ceased operations for more every five years rather than every three years for than six months and did not make sufficient efforts independent health facilities. The Unit also had two to reopen. facilities that were actually open incorrectly listed When the Ministry proposes to suspend or as “closed” and did not have current information on revoke a licence, the Act allows the facility owner 12 facilities that had moved more than a year previ- 15 days to request a hearing with the Health ously. Furthermore, the Unit’s inspection reports Services Appeal and Review Board. Facilities that were not forwarded to the Program—although the request a hearing can continue to provide services inspection staff told us that they would inform the to patients and bill the Ministry until the Board Program of any significant issues. makes a decision. We noted that between January 166 2012 Annual Report of the Office of the Auditor General of Ontario

2007 and January 2012, facilities requested hear- conduct a focused facility assessment to review ings regarding seven proposals to revoke. Of the the issues. 21 hearing requests that were resolved between The Ministry also conducts other administrative January 2007 and January 2012, 17 were settled activities to ensure compliance with the Act, some between the Ministry and the facility owner before of which may involve time-consuming processes. the Board reached a decision, three were not For example, as previously noted, ministry staff pursued by the facility owner, and the Board made make extensive efforts to track information on facil- a decision on the remaining one. Most of the 17 ity owners. And every month, ministry staff manu- that were settled involved three facility owners. ally review a report on facility billings, track which We also noted that the average time from request facilities have not billed in the previous six months for hearing until resolution was about five years and send a letter to those facilities. The letter asks for quality assurance issues and two years for the facility to either confirm that it has ceased oper- operational issues. ations or provide the steps it is taking to reopen. Even though the Act gives the Ministry signifi- A follow-up letter is issued after an additional six cant discretion regarding whether or not to license months if necessary, and if billings do not begin facilities if it has reasonable grounds to believe that again, more progressive actions, such as a proposal a facility is not being operated in accordance with to revoke the facility’s licence, are undertaken. the law or with honesty and integrity, this provision Between January 2007 and January 2012, the Min- is rarely used. For example, the Ministry licensed istry sent about 200 letters regarding 100 facilities an owner to provide a new service even though that did not bill for at least six months. 3.06 ministry staff had identified significant concerns While these may be worthwhile efforts, we about this owner’s billings the year before (a full expected that the Ministry would have assessed Section investigation of the billings irregularities was still whether its time might be more effectively used VFM under way when the owner was licensed for the following up on those facilities with unusual •

3 new service). or possibly inappropriate billing practices, as discussed previously. Chapter Other Monitoring Activities RECOMMENDATION 3 The Ministry is made aware of activities or circumstances that constitute a contravention of the Act in To better ensure that independent health facili- a number of ways, including complaints by the pub- ties are providing services according to quality lic and other facility owners. However, the Ministry medical standards established by the College of has no information about complaints made directly Physicians and Surgeons of Ontario (College) to facilities. The Ministry tracks the complaints and are meeting other legislated requirements, it receives (mostly from the public) that indicate the Ministry should: quality-of-care deficiencies or other violations of • work with the College to ensure that every the Act. About 35 such complaints were tracked in facility is inspected at least once between each 2011, with over half pertaining to quality assurance five-year licence renewal for that facility; issues such as equipment not being disinfected • consider including additional expectations in between patients and facility staff not behaving its Memorandum of Understanding with the professionally. The Ministry’s complaint follow-up College, such as: activities varied depending on the nature of the • requiring assessors to review the quality of complaint. For example, in two cases the Ministry each physician’s work at the facility; and asked the College of Physicians and Surgeons to • requiring that assessment results for facilities with significant issues be more Independent Health Facilities 167

promptly reported to the Ministry after frequency as other physician specialties. As well, the assessment; with respect to the College’s prompt reporting of • consider, when next reviewing the facilities with significant issues to the Ministry, Independent Health Facilities Act, adding a working group has been established to review penalties for facility owners who refuse reporting policies and to suggest appropriate access to the College’s assessors when they standards for the turnaround time from the arrive unannounced; date of assessment to notifying the Director of • develop policies and procedures to improve significant issues. information-sharing between the Ministry’s The Ministry will consider implementing Independent Health Facilities Program and penalties or other appropriate provisions in its X-ray Inspection Services Unit, includ- situations where facility operators deny access ing information on the location of facilities to the College’s assessors. offering x-ray services as well as information Patient safety is a ministry priority. The X-ray on inspection results, so that each has the Inspection Unit is embarking on a review of its most current information available on the inspection delivery model and is collaborating facilities they oversee; and with other branches of the Ministry in a broader • consider options for streamlining the review of the Healing Arts Radiation Protection monitoring of facilities’ activities, including Act (HARP Act) to identify options to enhance determining whether the Ministry’s X-ray the oversight of x-ray machines in Ontario.

Inspection Services Unit can rely on the work The Ministry has made a specific commitment 3.06

of other professional or federal oversight to review the current HARP Act to identify

entities to enable it to focus its activities on opportunities that would help promote continu- Section

the newer or higher-risk facilities. ous improvement in the safe use of radiation- VFM

•

emitting medical imaging devices. 3 MINISTRY RESPONSE

The Ministry is working with the College to Chapter ensure that every independent health facility COMMUNITY HEALTH-CARE CLINICS is inspected at least once within the five-year NOT COVERED BY THE ACT renewal cycle. Although there may be excep- Community health-care clinics fall into tional circumstances (for example, where servi- two categories: ces are not currently being provided or when a those that are covered by the Independent facility is in the process of relocating, changing • Health Facilities Act (Act), which we refer to owners or expanding the scope of services being throughout this report as “independent health provided), the Ministry will work closely with facilities” (facilities), and the College to minimize such exceptions. those that are not covered by the Act, which The Ministry agrees with the recommen- • we refer to as “community health-care clinics” dation with respect to requiring assessors to (or community clinics). review the quality of each physician’s work. At In our 2004 audit of independent health facili- this time, the College is in the process of align- ties, we noted that some diagnostic procedures, ing the quality assurance assessment program such as colonoscopies, were not licensed services for independent health facilities with its peer under the Act and were therefore performed at review program for physicians, to ensure that community health-care clinics not covered by radiologists are peer-reviewed at the same this legislation. We also observed that a licensed 168 2012 Annual Report of the Office of the Auditor General of Ontario

service, abortion, was being performed both in fees. The Ministry’s Independent Health Facility independent health facilities and in community Program was not aware of these community x-ray clinics. At that time, neither community clinics pro- clinics, and, unlike the case with independent viding colonoscopies nor community clinics provid- health facilities, the x-rays read by radiologists ing abortions were subject to the College’s quality in these clinics are not periodically reviewed by assurance assessments. the College. In 2007, the Ministry licensed one independent health facility in Northern Ontario to provide, RECOMMENDATION 4 among other things, colonoscopies. The Ministry To ensure that all community clinics providing had no documented rationale for why only this insured services—even those that do not use community clinic was licensed as a facility to anaesthesia—offer quality medical services, the provide colonoscopies. At the time of our cur- Ministry of Health and Long-Term Care should rent audit, the Ministry had no information on consider engaging the College of Physicians the number of non-facility community clinics and Surgeons to oversee those clinics that operating in Ontario that were performing colon- offer services that would be subject to College oscopies. However, information at the Ministry oversight if they were classified as independent indicated that in the 2010/11 fiscal year, about health facilities. 36% of all colonoscopies were performed in community clinics. MINISTRY RESPONSE In 2010, legislative changes to the Medicine Act, 3.06 1991 made the College of Physicians and Surgeons The Ministry agrees that the provision of qual- of Ontario (College) responsible for inspecting all ity health care that includes quality medical Section community clinics that use anaesthesia to provide services is a priority and established Health VFM insured services (such as cosmetic surgery and Quality Ontario (HQO) to lead the Ministry’s •

3 colonoscopies) and uninsured services (such as quality and evidence-based agenda. Among hair transplants). However, the Ministry does not other things, HQO makes recommendations to

Chapter obtain information on the frequency or outcome the Ministry regarding the quality of health- of these inspections. We contacted the College, and medical-care services provided to patients, which informed us that since commencing such including services provided by community and inspections in fall 2010, it had inspected about specialized clinics, and also provides guidance 50 community clinics performing colonoscopies and makes recommendations to health-care and four community clinics performing abortions. providers and relevant organizations on stan- The College indicated that most of the community dards for patient care based on evidence, and colonoscopy clinics had passed their inspections clinical best practice guidelines and protocols. with some conditions. No information was pro- Furthermore, Cancer Care Ontario and vided by the College regarding the community the College are working in partnership to plan abortion clinics. and implement a model for quality improve- We also noted that about 50 of the x-ray sites ment focused on selected health services inspected by the Ministry’s X-ray Inspection Servi- that are delivered in settings other than an ces Unit were community clinics, not independent independent health facility, including pathol- health facilities. These clinics cannot bill the Min- ogy, colonoscopy and colposcopy. The Ministry istry or their patients for facility fees for insured expects to receive an implementation plan by services, but physicians working in these clinics March 2013. can bill the Ministry for the related professional Independent Health Facilities 169

PUBLIC INFORMATION number, as well as all the services each facility is licensed to provide. The Ministry told us that this Patients have the right to choose which independ- list is updated monthly and that patients interested ent health facility they go to. To make these in identifying alternative facilities can review choices, patients need information on which facili- this list. ties provide the service(s) they require and where However, we noted that the list was not as help- these facilities are located. ful as it could be. For example, unlike the Health Care Options Directory website, this website had Referral Forms no tools allowing users to search by postal code or by service area. More significantly, the list did not Patients requiring services provided by an always offer an accurate description of the services independent health facility often get a referral form offered at a given facility: a clinic listed as licensed from their physician that provides contact informa- to offer a specific service might not actually offer tion for one facility or for several facilities owned that service. When we contacted a sample of facili- by the same owners. Many patients assume that ties that the website indicated were licensed to they must go to a facility on the referral form; but provide fluoroscopy services, 35% said that they in fact they are free to choose a hospital or any facil- did not provide such services at all, and another ity that offers the required service(s), including a 20% said that they had temporarily suspended the facility that may be closer to home. As noted earlier, service because of machine-maintenance or other some physicians have a financial interest in certain issues. One facility’s phone number was no longer

facilities, which, as several studies have indicated, 3.06 in service, and a staff person at another directed may influence both the rate at which they prescribe us to a different location, which we noted was not

certain tests and which facilities they refer patients Section licensed to provide the service. (The Ministry was to. In contrast, referral forms for laboratory ser- VFM following up with this facility.) The website also • vices (such as blood tests) are standardized and 3 did not list the specific services provided at a given do not readily lend themselves to listing specific facility. For example, not all facilities offering ultra- laboratories. At the time of our audit, the Ministry

sound services perform knee ultrasounds. Chapter did not have any plans to develop a standardized We noted that as of January 2012, the list referral form for services provided by independent included addresses for almost 950 facilities; how- health facilities. ever, information at the Ministry indicated that there were about 800 unique facility locations. We Ministry Website were informed that some facility owners operate with multiple licences out of one location. As well, The Ministry has a website (the Health Care we noted that more than 20 listed facilities had not Options Directory) that allows the public to search been open for at least five years. for locations providing services such as x-rays and In our 2004 Annual Report, we recommended ultrasounds. But this site does not list all of the that the Ministry consider publicly disclosing any independent health facilities and hospitals provid- serious quality assurance problems at independ- ing these services. Rather, it is primarily a listing ent health facilities. We noted during our current of these services provided outside of facilities and audit that the Ministry’s website included the dates hospitals, such as in after-hours clinics. In 2010, the and results of each facility’s last quality assurance Ministry introduced another website that lists only assessment. However, it did not indicate which the independent health facilities in Ontario. The facilities had been suspended for failing to meet list includes each facility’s address and telephone required standards; instead, suspended facilities 170 2012 Annual Report of the Office of the Auditor General of Ontario

were just removed from the online list. Because sus- RECOMMENDATION 5 pended facilities may continue to offer patient services (they cannot bill facility fees, but can still bill To ensure that patients have access to relevant professional fees), the Ministry requires suspended information about independent health facilities facilities to remove their licences from their walls that can help them obtain required services, the so that patients know that they are suspended. But Ministry of Health and Long-Term Care should: the Ministry has not analyzed whether this is an • consider the costs and benefits of introdu- effective way of informing patients that the facility’s cing a standardized referral form, similar to licence is suspended. that used in the laboratory program, that The website also provides no information restricts physicians from recommending a about the process for filing a complaint about an preferred facility and that contains informa- independent health facility. tion about how to locate an independent health facility using the Ministry’s website; combine existing website information into one Wait Times • website with search functionality that speci- The Ministry’s Wait Time Strategy website reports fies all locations where patients can access wait times for MRI and CT examinations at community services, such as x-rays and ultra- hospitals, and for cataract surgeries at hospitals sounds, as well as available services and wait and at one of the two independent health facili- times for services that do not have same-day ties providing these surgeries. The Ministry does access (for example, MRIs and CT scans); and 3.06 not report the wait times for an MRI or CT at the • provide information on its website regarding seven independent health facilities offering these how to register a complaint about an Section services. As a result, patients cannot “shop” for the independent health facility. VFM shortest wait time for these services. As well, the •

3 Ministry does not collect or report wait times for MINISTRY RESPONSE other diagnostic services provided by independent The Ministry agrees that public information-

Chapter health facilities, such as x-rays and ultrasounds, sharing accomplished through accessible and because the Ministry does not expect significant user-friendly formats, tools and mechanisms is waits for these services. Although we would important. In this regard, the Ministry will: not expect the Ministry to track services with explore options for standardizing diagnostic same-day access (such as many types of x-rays), • referral forms, including the potential for public information on wait times for services electronic referral formats; that have historically involved a significant wait explore options to combine existing website (such as MRIs, CTs and cataract surgeries) might • information to ensure patients have access be beneficial. to comprehensive information (the Ministry is in the process of having independenthealth-facility MRI and CT service wait times reported through the Provincial Wait Times Strategy); and • include information on its website on how to register a complaint about an independent health facility. Chapter 3 Section 3.07 Legislative Estimates Review Process

a fiscal accountability perspective, the budget’s Background “bottom line” is the projected surplus or deficit for the year. The government of Ontario currently spends GOVERNMENT SPENDING AND THE ROLE about $120 billion a year on public services ranging OF THE LEGISLATURE

from health care to education to social assistance. 3.07

In the Canadian parliamentary system, the party While the government sets spending priorities and

with the most elected members after a federal or manages service delivery, it must seek the Legisla- Section provincial election normally forms the government. ture’s approval each year for its spending plans. VFM

•

If that party has a majority of elected members, it In jurisdictions that follow the Westminster-style 3 can govern without the support of the opposition parliamentary system, such as Canada, the United parties. However, if it has only a minority of elected Kingdom, Australia and New Zealand, parliament- members, it usually requires the support of at least ary oversight and approval of a government’s Chapter one other party to pass any legislation, including proposed spending plans has long been considered the annual budget. a key element to ensure government fiscal account- Legislative oversight of government spending, ability. Legislative scrutiny of proposed government including the annual budget, is fundamental to spending, especially by opposition members, is a any democracy. In Canada, such oversight typically key component of this accountability. falls to the opposition parties, although all elected officials are ultimately responsible for ensuring THE BUDGET, ESTIMATES AND THE that revenues are spent prudently on behalf of LEGISLATIVE SPENDING REVIEW the public. PROCESS The annual budget is generally regarded as a government’s most important piece of legislation In recent years, the Ontario budget has usually been each year given that it outlines the government’s presented to the Legislative Assembly in late March fiscal direction and its policy priorities. It also sets or early April. However, the budgetary process out how much the government proposes to spend starts several months earlier, when the government on the services it provides to citizens, and how it provides ministries with broad spending guidelines will fund the cost of providing these services. From that reflect its policy priorities and fiscal outlook.

171 172 2012 Annual Report of the Office of the Auditor General of Ontario

The ministries then put forward their proposed approves them, through passage of the Supply Act, spending plans for each of the public services that they become voted appropriations and constitute they manage. Ministry plans, once reviewed and the legal spending authority for the government. approved by the Treasury Board and Management The vote-and-item details provide a mechanism Board of Cabinet, form the basis for the budget for legislative control of this proposed spending, as and for expense projections of future years. The spending of the funds must be consistent with the budget and accompanying fiscal plan also reflect stated purpose of each approved vote and item, and the expected revenues available to support these the amounts spent cannot exceed voted amounts proposed expenditures. without further legislative authority. Ministry spending plans are subdivided by program area and spending activity. Each program PAST EXAMINATIONS OF ONTARIO’S area, such as post-secondary education or adult ESTIMATES REVIEW PROCESS social services, is allocated a vote number, and each spending activity within a particular vote, In 1987, we asked a number of MPPs for their views such as grants to colleges and universities or adult on the estimates review process. We then reported employment-support assistance, is allocated an in our 1987 Annual Report that the MPPs generally item number. These proposed votes and items are felt that the process was “maligned and ineffect- summarized in a document entitled Estimates, and ive.” We recommended at the time that considera- represent the government’s formal request to the tion be given to creation of a dedicated committee Legislature for spending approval. The government mandated to review the annual estimates of 3.07 must table its Estimates no later than 12 sessional selected ministries. days after the budget’s release. The following year, the Standing Committee Section In Ontario, a separate legislative committee on Public Accounts issued a Special Report on the VFM called the Standing Committee on Estimates (Com- Estimates Process, and in 1989, the Legislature •

3 mittee) is responsible for reviewing the estimates established the current Standing Committee on of at least six but not more than 12 ministries or Estimates (Committee).

Chapter government offices each year. The Committee is After this Committee had operated for about currently composed of members from Ontario’s five years, we decided in 1995 to revisit this area three major political parties, with representation to determine whether there had been progress proportionate to the percentage of members each in improving the estimates review process. The party has in the Legislature. During the review pro- consensus among the nine MPPs we interviewed— cess, ministers and senior staff of those ministries three from each party—was that the process was selected for review appear before the Committee to still not very effective in ensuring that ministry explain their estimates and answer questions. spending plans were appropriately scrutinized. When the Committee completes its review, it In our 1995 Annual Report, we quoted a comment reports back to the Legislature and the estimates from one MPP as indicative of the views we heard: are then deemed to have been approved by the We do not serve the public very well in Committee, even if the Committee did not formally assessing whether the ministry is plan- endorse them. Estimates of those ministries or ning to spend or has spent the money offices not selected for review are deemed to be wisely. This disturbs me. In theory, we are automatically approved. supposed to be looking after the finances Members of the Provincial Parliament (MPPs) of the province. However, by the end of also have the opportunity to debate the tabled the day we are no more enlightened. Estimates in the Legislature. After the Legislature Legislative Estimates Review Process 173

on Estimates are still a worthwhile exercise because Objective and Scope of they provide the only real opportunity to directly Current Review question a minister outside of question period. We asked the members a point-blank question: Was the whole exercise little more than a “rubber We decided to revisit the legislative estimates review stamp” with respect to overseeing proposed govern- process this year because it has been more than 15 ment expenditures? Most acknowledged that, to years since our last review. In addition, our research some extent, it probably was. We heard comments found that other Westminster-style parliaments along the following lines: have acknowledged, just as Ontario’s has, that there have been challenges in implementing effective The estimates review process can be a legislative oversight of this area, and a number have very frustrating exercise when you sit carried out changes to try to address them. It was there for up to 80 hours and not look in our view that Ontario’s elected members might find any detail at the expenditures reported some of the practices in use elsewhere of interest. in the Estimates. In reality, the process is Aside from our review of practices elsewhere, a tool for political purposes—it has noth- the main focus of our study was to obtain the views ing to do with review of expenditures for of MPPs with several years of experience on the reasonability. Standing Committee on Estimates. Accordingly, I do not feel the Estimates Committee is we interviewed nine current members—again, meeting the objective of financial scrutiny,

three from each party—who had served or are 3.07 and it doesn’t provide for a better Ontario. currently serving on the Committee. We also took I give it a scale of two out of 10.

the opportunity to obtain the views of three retired Section

MPPs from different parties, each of whom served But on the other hand, members were unani- VFM

• a significant number of years as an elected member mous in acknowledging the process’s value as an 3 and whose collective experience in the Legislature opportunity for questioning ministers about gov- totalled more than 80 years. ernment policy and related proposed expenditures. We understand that the Legislature’s Standing As several MPPs put it: Chapter Orders—which govern the workings of the House The estimates review process is a great and its Committees, including the Standing Com- opportunity to get questions that are mittee on Estimates—are currently under review, so important to you answered and is a good the observations from our review may prove timely. learning opportunity.

You learn a lot from the Estimates. Sometimes you will see that members Overall Observations will ask questions that the minister is not expecting, and it’s good to see how they The consensus of almost all of the members we respond. interviewed echoed the comments members made While it’s not perfect, in aggregate it’s a in 1995 when we last examined the estimates good thing. review process—the process is still not very effective in providing meaningful scrutiny of government Our research did not identify any Westminster- spending plans. However, members also made it style parliaments that appeared to have devised a clear that the hearings of the Standing Committee truly effective estimates review process, and many 174 2012 Annual Report of the Office of the Auditor General of Ontario

are wrestling with the same concerns and issues the Legislature is sitting, and so would not expressed by Ontario MPPs. Indeed, the Inter- welcome any changes requiring them to add national Monetary Fund noted in a 2005 study that to the already significant time they spend on Westminster-style legislatures in general have very estimates. However, most MPPs we spoke with limited oversight influence, and ranked the parlia- said it should be possible to identify changes ments of Canada, Australia, the United Kingdom that would make the time they currently and New Zealand as having the weakest budgetary spend on estimates more effective. Specific- influence of those in any western countries. The ally, members said: legislatures in the United States and Sweden, both We do not get good value for all non-Westminster systems, ranked the highest in the time spent in Estimates. Surely, budget oversight powers. there are some changes that could A common observation elsewhere was that be made which, while not adding since Committee members cannot realistically more committee time, would result amend spending proposals, they see little point in in a better process. giving the numbers more than a superficial look, regardless of whether they review the estimates in a Whatever you recommend, do separate estimates committee or in a series of policy not include anything that adds committees. The research consensus was that more time to the estimates review elected members must first believe their scrutiny process because we just do not have can influence government spending decisions, pro- the time. 3.07 grams and services before the review process can Ministry briefing materials to support the fully engage them. A 2006 report prepared by the •

Section estimates review process are quite detailed but United Kingdom’s Hansard Society, an independ- are still perceived as not providing, in an easily VFM ent, non-partisan educational charity that promotes •

understandable manner, adequate information 3 effective parliamentary scrutiny, summed up the on how a program is being delivered and what perceived effectiveness of the United Kingdom’s results are being achieved. This makes it dif-

Chapter process as follows: ficult for Committee members to evaluate ser- In the view of many commentators, vice delivery relative to funds being requested. Parliament’s influence over government The following two comments reflect some of proposals for taxation and expenditure, the feedback we heard on this issue: and priorities within that expenditure, is Unless someone knows the ministry virtually non-existent. The essential rela- in depth, you really cannot make tionship between Parliament and govern- heads or tails of the Estimates. ment is that the latter proposes and the former simply agrees. To draw an analogy, Members should read the Estimates the government decides the value of the in detail, but no one really reads cheque, to whom it should be paid and them. Estimates are complicated to when, and Parliament simply signs it. read. It is very difficult for a non- financial person to read and sort Ontario MPPs raised the following specific issues through them. or concerns, which our research suggested were also common elsewhere: • Often, a significant amount of the requested • Elected members have numerous and compet- funding may have already been spent before ing demands on their time, especially when the Estimates are reviewed. MPPs said this Legislative Estimates Review Process 175

raises questions about whether the purpose There are a lot of talented people, of the exercise is to have members scrutinize but the politics remove the ability the government’s proposed spending plans of the Committee to undertake any or, instead, to give MPPs access to ministers. meaningful scrutiny. Members observed that: The adversarial dynamics in any The chamber is here to represent legislative committee comprised of the views of Ontarians. Therefore, both opposition and government the estimates review process is members will almost always foster needed, but how it should work an atmosphere of partisanship. As is the question. There should be a result, government members ask some ability for the Committee to questions that are intended to make identify problems and report on the government look good, while recommendations, but this does not opposition members ask questions happen. So the estimates review intended to do the opposite. essentially becomes what I want to In general, the three long-serving retired MPPs use it for—taking up the concerns we spoke with echoed the sentiments of the nine of my constituents. Essentially, the current members interviewed that the legislative Estimates becomes a place to get estimates review process was not seen as being that information on those constituency effective as a means of scrutinizing government

issues you need to address. 3.07 spending. While they also acknowledged the bene- The roles of the Committee should fit of having the opportunity to directly question Section be to review, quantify and qualify a minister, one retired MPP made the following VFM government expenditures and to be observation: •

3 in a position to make amendments I like Public Accounts because the Min- to the government’s spending plan, ister is not there. One change I would

where appropriate. The Committee, Chapter make: take the Minister out of the pro- along with the Legislature, should cess, as this is the only way committee hold the government to account on members will get legitimate and straight- the spending of tax dollars. The pro- forward answers. cess should be about transparency. But in reality the process is a bal- The retired MPPs, who have witnessed the ance between politics and numbers. evolution of the estimates review process over Therefore, the Committee needs time and have been both committee members and an upgrade—a review of what it is ministers, felt that the process has become less doing and why it is doing it. effective as the size of government has grown. They also noted that any reform had to go beyond In answer to our question about partisan- • administrative changes. Specifically, the govern- ship, the MPPs said that the estimates review ment needed to be more open in responding to process would likely be more effective if the questions, and committee members needed to partisanship could be dialed down. However, focus both on financial scrutiny of the Estimates as all MPPs recognized that this is easier said well as on policy-related issues. One of the retired than done since legislative committees are MPPs expressed the following opinion: composed of both opposition and government members. As two MPPs commented: 176 2012 Annual Report of the Office of the Auditor General of Ontario

The concentration of power without Figure 1: International Monetary Fund Index of a counterbalance is the problem with Legislative Budgetary Powers* government today—that is why you need Source of data: IMF (2005) the Standing Committee [on Estimates]. United States If you want to get accountability, it is Sweden Italy important for the Committee to decide, Japan where do we want to go with this issue Austria Finland and how do we get there. Netherlands Norway France Germany Spain Australia Canada Detailed Observations United Kingdom New Zealand 0246810

THE WESTMINSTER SYSTEM OF * This figure measures the relative power of legislatures over national budgets GOVERNMENT on a scale of one to 10, with 10 representing significant power. The IMF considered whether legislatures: (a) approve an annual budget strategy; (b) have powers to amend draft budgets; (c) allot time for discussion of the The ability of a legislature to influence a govern- annual budget; and (d) receive technical support for scrutinizing the budget. ment’s budget depends on the balance of power between the legislative and executive branches. (Canada) entitled Meaningful Scrutiny: Ontario follows the Westminster-style system, Practical Improvements to the Estimates Pro- 3.07 which emphasizes the executive’s right to govern. cess] lamented that “committees continue Budgetary power is accordingly concentrated to provide relatively cursory attention to Section within the government, and the parliament as a the main spending estimates and explana- VFM whole has only minimal influence over it, especially tory reports,” and MPs interviewed for •

3 in majority situations. Noting this, the International a Commission research study confirmed Monetary Fund (IMF) ranked the legislatures of this finding.

Chapter Canada, Australia, the United Kingdom and New In contrast to Westminster-style democracies, Zealand in a 2005 study as having the weakest the IMF ranked the legislatures of the United States budgetary oversight powers among major western and Sweden as having the strongest budgetary countries, as shown in Figure 1. oversight powers. However, this power balance The federal Gomery Royal Commission, which also has its disadvantages. In the United States, for in 2004 launched an investigation into the so- example, budget debates can lead to paralyzing called Sponsorship Scandal in the federal govern- political gridlock, as when Congress threatened in ment, also touched on the issue of Parliament’s 2011 to defeat the Administration-backed budget ability to effectively oversee a government’s spend- bill and effectively shut down the government ing plans, observing: unless certain of its demands were met. Concerns about the effectiveness of Parliament in examining government DEDICATED ESTIMATES COMMITTEE programs and discharging its financial VERSUS POLICY-FIELD COMMITTEES accountability role have been expressed for decades. Most recently, a 2003 com- Ontario is the only Canadian jurisdiction that mittee report [Report of the Government has established a dedicated Standing Committee Operations and Estimates Committee on Estimates, as illustrated in Figure 2. All other Legislative Estimates Review Process 177

Figure 2: Provincial Comparison—Estimates Review Time and Committee Type Prepared by the Office of the Auditor General of Ontario

Province Time Given for Estimates Review Committee Type British Columbia 125 to 200 hours (on average) Committee of the Whole1 Alberta 53 hours (minimum) Policy-field committees2 Saskatchewan 75 hours (minimum) Policy-field committees Manitoba 100 hours (maximum) Committee of the Whole1 Ontario 90 hours (maximum) Standing Committee on Estimates Quebec 200 hours (maximum) Policy-field committees New Brunswick 80 hours (maximum) Committee of the Whole1, 3 Nova Scotia 80 hours (maximum) Committee of the Whole1, 4 Newfoundland and Labrador 75 hours (maximum) Policy-field committees Prince Edward Island None specified Committee of the Whole1, 5 Yukon None specified Committee of the Whole1, 5 Nunavut None specified Standing Committee on Oversight of Government Operations and Public Accounts5

1. This term is used to describe the Legislative Assembly, which, in essence, is a committee of all elected members. 2. Three hours for each department and two hours for the Executive Council. 3. The Legislative Assembly may refer Estimates to a dedicated Standing Committee on Estimates, but this occurs only rarely. 4. Forty hours for each of the Committee of the Whole on Supply and the Subcommittee on Supply. 5. Information for Prince Edward Island, Yukon and Nunavut was not readily available. 3.07

provinces have assigned this oversight role to their mittees, as illustrated in Figure 3. We noted from Section policy-field legislative committees or to a commit- our research that, irrespective of the approach VFM tee of the whole Legislature. Typical policy fields taken to review estimates, all jurisdictions indicated •

3 often comprise those ministries that share a com- similar challenges in legislative scrutiny of their mon theme such as justice, and which cover, for government’s proposed spending plans.

example, such areas as police, correctional facili- Ontario’s 90-hour maximum time allocation for Chapter ties, probation and parole. estimates review is fairly consistent with that of the One benefit of having a distinct estimates com- other provinces, as illustrated in Figure 2. How- mittee is that parliamentarians get dedicated time to ever, our research found that, like Ontario, other review and debate the proposed spending of at least Canadian jurisdictions do not use all of the allotted some ministries. On the other hand, the policy-field time. In Ontario, the Estimates Committee typically committee approach often allows members to gain spends about 50 hours in hearings. MPPs told us more detailed knowledge about certain ministries there were several reasons why the full 90-hour and programs, making them better informed about allocation was not used: both the issues and related program-expenditure • There are competing demands on members’ trends. However, a drawback of the policy-field time that include attending debates in the approach is that these committees are often also Legislature, conducting other committee required to review draft legislation or perform other work and addressing party-caucus and con- tasks. Only part of their time may be spent actually stituency issues. reviewing ministry estimates. • In recent years, the Estimates have not been The estimates review in the major Westminster- tabled until mid-April, leaving limited time for style countries also tends to be done by policy-field committee meetings before the summer recess committees rather than dedicated estimates com- in mid-to-late June. 178 2012 Annual Report of the Office of the Auditor General of Ontario

Figure 3: National Comparison—Estimates Review Time and Committee Type Prepared by the Office of the Auditor General of Ontario

Country Time Given for Estimates Review Committee Type Canada Three months Policy-field committees United Kingdom 14 days (recently increased from eight days) Policy-field committees Australia Four days per committee Policy-field committees New Zealand Two months Policy-field committees

• The Estimates Committee normally meets and an explanation of the programs and funding only when the Legislature is sitting and by a particular item.” The Treasury Board provides when its day’s routine proceedings have been guidelines to help ministries comply with these completed. requirements. Despite this, however, we found Given that devoting more time to estimates that the information in briefing books often did review is not an option for most committee mem- not clearly identify or discuss expenditure trends bers, any improvements must focus on making the or address significant changes to programs beyond best use of what time is available. one year. As well, briefing books tend to focus on government initiatives but provide little quantifiable information on program results. QUALITY OF BRIEFING INFORMATION Interviews with MPPs led us to conclude that

3.07 Ministries selected to appear before the Estimates most MPPs did not spend much time reviewing the

Committee provide MPPs with supplementary briefing books in advance of committee meetings

Section material in the form of briefing books. Since many because they did not find the information all that

MPPs lack an accounting or financial background, it useful. Several MPPs advised us that they would like VFM

•

is essential that this material be easy to understand, more performance and future-oriented information 3 identify major program changes and spending for programs with significant changes to spending trends and, perhaps most importantly, be reason- or service delivery so as to better assess the impact Chapter ably concise. Several MPPs commented that it takes of the proposed changes. We believe this suggestion several years sitting on the Estimates Committee has considerable merit. In particular, MPPs told us before they are able to fully grasp the ministries’ that information on program reductions is particu- briefing materials. As one of the retired MPPs with larly relevant in this era of cutbacks and austerity. whom we met stated: As a recent Ontario ministry publication dealing with performance measurement at the municipal The way the material is presented is so level put it: important because it frames the issues. If presented well, it will allow both the Government today is very complex, so it is experienced member and the average important that elected officials and public member to gain the necessary knowledge servants inform taxpayers what govern- to understand the program and then to ments plan to achieve, what it is actually ask the right questions. accomplishing and what public services cost. With this information, taxpayers Legislature rules provide for some consistency can make informed decisions about the in the preparation of briefing books, requiring level of services they desire. This notion of that they include “information on growth rates, accountability is fundamental to our form interim expenditures for the previous fiscal year, of government. Legislative Estimates Review Process 179

Program Performance Information It is not good enough to dump data into the public domain. It must be analysed to Clearly, MPPs need accurate, transparent and be relevant, robust and fit for purpose. understandable information to help them assess whether government spending is meeting its stated The MPPs we interviewed who commented objectives. In this regard, we believe Ontario’s on this issue generally were of the opinion that Standing Committee on Estimates would find brief- they required “more relevant” rather than “more” ing materials more useful if they contained: information. • measurable targets for program objectives; and Future-oriented Information • a comparison of actual performance with forecasts. Although the Ontario budget provides projected Many other jurisdictions have recognized this future expense information, the Estimates Com- need since our last review in 1995. We reviewed mittee has a mandate to review only current-year a number of publicly available annual plans and spending plans. Consequently, ministry briefing performance reports published by departments and books do not provide information or projections ministries in Alberta, British Columbia, Canada, regarding the next few years. Australia and the United Kingdom. Most jurisdic- A 2006 study by the Parliamentary Informa- tions reviewed provided more comprehensive plan- tion and Research Service of Ottawa’s Library of ning reports that laid out strategy, activities and Parliament suggested that federal parliamentary

performance measures with stated targets. Subse- policy-field committees reviewing estimates should 3.07 quent annual performance reports explicitly com- expand their focus to include future spending

pared actual performance with prior targets and plans. They argue that the government likely Section discussed variances. The majority of jurisdictions would be more receptive to recommendations that VFM

• reviewed also provided plans, expense projections touched on future spending because criticism of 3 and performance targets for three to four outlook the more detailed current-year plans could create a years, as illustrated in Figure 4. non-confidence issue. We believe this observation It is notable that the United Kingdom has has some pragmatic merit; committee members Chapter recently reformed its performance reporting prac- would likely be more motivated to focus on spend- tices. As part of major reforms to the public service ing and the level of services actually being provided following the 2010 election, the government abol- if ministries and the government considered their ished the previous reporting regimen of centrally input more seriously. defined Public Service Agreements. In its place, At the time of our 1995 review, only a few juris- a five-year strategy was developed, and depart- dictions were providing future-oriented estimates ments annually develop business plans that set out information. Today, however, much more of this proposed actions to implement the government’s information is available, as illustrated in Figure 4. strategy. Subsequent performance reporting tracks The results of these initiatives have been mixed: these actions. Interestingly, targets are no longer • On the positive side, the U.K. Hansard Society formally set. Departments also report on several noted in 2006 that scrutiny of departmental indicators that are meant to help monitor perform- annual reports has enabled committees to ance. The development of these indicators was still conduct more comprehensive analyses of gov- ongoing at the time of our review. ernment spending priorities both within and As the chair of the U.K. Public Accounts Commit- across departments. tee recently said: 180 2012 Annual Report of the Office of the Auditor General of Ontario

Figure 4: Estimates Briefing Information: Other Jurisdictions Prepared by the Office of the Auditor General of Ontario

Jurisdiction Program Performance and Future-oriented Information British Columbia • The government releases an annual strategic plan of priorities, objectives and expected results over the next three years. • Annual departmental service plans provide three-year business plans consistent with the government’s strategic plan and outline specific performance measures with targets for all outlook years. • Annual departmental service plan reports compare prior year’s actual performance against targets set in the previous service plan.

Alberta • Ministries provide a three-year business plan with goals, priority initiatives and performance measures with targets for all outlook years. • Annual ministry reports track key performance measures and discuss the prior year’s performance against targets set in the prior year’s business plan.

Canada • An annual ministerial Report on Plans and Priorities outlines forecast spending and expected results over three years, with stated performance indicators and overall targets. • Departmental performance reports compare the prior year’s actual performance against plan. • In the fall, the targets in the ministerial Report on Plans and Priorities are compared to actual departmental reports, which allow for feedback and adjustments before the next Estimates are tabled in the spring. 3.07

Australia – Queensland • Departmental Strategic Plans outline a department’s agenda over the next four years. • Departmental annual reports provide an account of the operational and financial performance Section as well as an assessment of performance against the Strategic Plan. VFM

• United Kingdom • In November 2010, the coalition government directed all departments to develop five-year 3 business plans that detail the department’s strategy for implementing the government’s core priorities.

Chapter • Departmental business plans are updated annually and describe plans and actions; in 2012, expenditures were forecast out to 2015. • Departmental annual reports detail the progress of actions outlined in the prior year’s business plan and report on input and impact indicators that are used to monitor performance.

• Several Canadian and U.K. studies have noted While some MPPs we interviewed supported that the provision of future-oriented informa- the inclusion of future spending plans in estimates tion has resulted in a more effective scrutiny briefing materials, others opposed it. One argument process. But other studies also found that against including future fiscal outlook data is that estimates documentation, including supple- such scrutiny may not be meaningful since govern- mental planning and performance reports, ment plans may change significantly in subsequent continues to be of poor quality. budgets. On the other hand, as one member noted, • In 2002, the Alberta Financial Management suggestions on future spending plans might find Commission, an expert panel set up by the some resonance with a minister or senior ministry province to examine its financial and budget- officials even though any public acknowledgement ing practices, noted that ministries tend to of this is unlikely to be forthcoming. As one of the treat future years merely as mathematical retired MPPs we spoke to put it: exercises. Legislative Estimates Review Process 181

If you are participating in estimates This information could usefully include 10-year his- review as a Minister, it may not change torical and future-year expenditure trends, data on things, but it is very useful for oversight service levels being provided, outcomes achieved because as Minister I may begin to rethink and the fiscal impact of any recent policy changes some of the program decisions based on and future expenditure trends on the expected level the questions I am getting. of service demand. Several MPPs told us that it might not be necessary for the minister to be present during NARROWING THE FOCUS FOR PART OF this portion of the estimates review. Some cited THE ESTIMATES REVIEW as an example the Standing Committee on Public Accounts, where only senior ministry and broader- Another way to enhance legislative spending public-sector officials are questioned. Those control is to narrow the Estimates Committee’s familiar with this committee said such an approach oversight focus by selecting only a few ministry pro- seemed to work well and tended to de-emphasize grams for more detailed scrutiny. This idea was put the partisan nature of committee hearings. This forward by the Library of Parliament study referred approach would focus the second track of estimates to above. The premise behind this idea is that the hearings on the fiscal and administrative delivery Committee just does not have the time to conduct a aspects of the program rather than on the merits of detailed scrutiny of all programs administered by a government policy, a matter best left for the minis- large and complex government ministry. ter’s appearance.

However, adopting such a targeted strategy is 3.07 Two MPPs who felt this two-track approach not without risk, as important issues might well might warrant consideration said:

go unaddressed. It would also mean that the Com- Section mittee’s ultimate approval or rejection of ministry I believe each of the parties should be VFM

• estimates as a whole would be based on a review asked what to focus on. This way, more in- 3 of only a portion of its activities. Despite these depth analysis on policy and expenditures concerns, several of the MPPs we interviewed said will occur rather than have the estimates this approach might have some merit for part of the review be wide open. This would likely Chapter time allocated to a particular ministry’s estimates. lead to a better understanding of govern- In essence, this method would employ a two- ment spending than is being obtained track approach. The first part of the review would from the current estimates review process. be a general session with the minister in which I’d like to see it. It is better to zero-in on committee members could ask questions about specific areas of concern …. any ministry programs and activities, while the second would focus on specific programs. Selecting One MPP who supported the two-track option programs for review in more detail could be done in did offer a caution, however: one of two ways: I am not in favour of limiting what can be a subcommittee composed of one member • reviewed with the minister or ministry from each party would choose from one to staff during the estimates review process. three programs; or What is currently open to scrutiny should each of the three parties would select a pro- • be left open. gram on which to focus. These choices would be made in advance so that One of the retired MPPs noted: the ministry briefing books could provide more There is no way the committee can detailed information about the programs selected. effectively scrutinize the Estimates today 182 2012 Annual Report of the Office of the Auditor General of Ontario

because the programs are just too big ing, “Here is what I noted …” would go and complex for committee members to a long way in helping members do their understand and know what questions to job … by pointing out trends, it would ask. Having served as both a member of trigger questions we should be asking the committee and as a former Minister about what is happening and, more who was on the receiving end of ques- importantly, why is this happening. tions, I am a big believer in narrowing the If someone can brief us in a non-partisan focus of the review. way, then yes this would be a great idea … We need non-partisan and factual briefings. Perhaps the deputy minister or assist- ADVANCE BRIEFINGS ant deputy minister can give the briefing. Another option we raised with MPPs was the value Yes. This would be great. Currently, too of a 30-minute to one-hour in camera briefing much time and emphasis is spent on the prior to the public committee hearing to provide an minister’s response. If the Auditor General overview of the selected ministry’s estimates. For gave a briefing, maybe this would not be instance, this briefing could cover any major chan- in the best interest for him, but it would ges in program expenditures or service delivery be better for the committee, given his levels, per capita cost comparisons with other prov- independence. inces that had similar programs, or other issues

3.07 such as the service and fiscal impact of recent

policy changes. We suggested the briefing could SPECIALIZED SUPPORT

Section be done either by the Legislature’s research staff

or by senior ministry officials—or possibly even Ministries spend billions of dollars across dozens VFM

•

by our Office, if we had done a recent audit in that of individual programs and activities each year. 3 program area. The Standing Committee on Public However, while the Legislature’s Public Accounts Accounts has effectively utilized such private in Committee, for example, has a dedicated legislative Chapter camera briefings in advance of the public hearing research officer available to assist committee mem- for most of the past decade. bers, the Estimates Committee does not. Without The 2003 report of the federal Government such support, and considering the time demands Operations and Estimates Committee, entitled on MPPs, it is more difficult for members to fully Meaningful Scrutiny: Practical Improvements to the understand a ministry’s estimates or obtain answers Estimates Process, made a recommendation along to fundamental questions such as: “Is public money these lines. It suggested that the committee could being spent effectively? Are desired outcomes being leverage the work of the Auditor General’s office by achieved? Should government spending priorities obtaining its advice when reviewing the estimates be revised in light of new events or circumstances?” of recently audited departments and agencies. It is equally difficult to identify those areas The majority of MPPs we spoke to felt an advance where proposed spending may not be appropriate briefing would be useful. Among their comments: relative to the level of service being delivered, or where significant variances in spending over time It would be very helpful to have a one- warrant additional scrutiny. hour briefing from a non-partisan official Our research indicated that most legislatures in or even a ministry official that might Westminster-style parliamentary democracies have explain why spending is up by 6% in one minimal, if any, dedicated research support. By area and down 4% in another. Even a contrast, the U.S. Congressional Budget Office has briefing from the Auditor General indicat- Legislative Estimates Review Process 183 a large research arm of accountants and economists The briefing would cover such areas as any available to help legislators analyze and assess the significant changes in program expenditures government’s proposed budget. where relevant, expenditure trends over time The MPPs we spoke to generally agree that and expenditures relative to other provinces, having research assistance would help them better given that most provinces deliver similar ser- understand the estimates briefing materials. In vices to their citizens. It could be conducted fact, the federal governments of Canada, the United by legislative research staff, by senior ministry Kingdom and Northern Ireland, Scotland and Wales officials or perhaps even by our Office. have moved to address this need in recent years, as 2. Requesting that ministry briefing books illustrated in Figure 5. combine information on spending plans In its 2006 report, entitled The Fiscal Maze, the with past and current performance U.K. Hansard Society pointed to the success of the reporting to provide a better basis for United Kingdom’s House of Commons Scrutiny legislators to assess what results are being Unit, established to provide specialized support for achieved for proposed expenditures. select parliamentary committees, as evidence that Specifically, the briefing materials would offering resources and research can contribute to provide historical trend information over a more effective financial review. The society advo- five- to 10-year period and include program cated expanding the unit’s role to provide compre- and service delivery indicators as well as hensive support for all committees. future-oriented outlook data, at least for the most significant ministry programs. How- 3.07 ever, this should not result in any significant increase in page count, and the readability Section Three Things To Consider and understandability of briefing materials VFM should be a priority. •

3 We offer the following ideas for consideration. 3. Selection of a few specific ministry pro- 1. A short in camera estimates briefing for grams for more in-depth review by the

members in advance of formal committee committee, perhaps with only senior min- Chapter sessions for each selected ministry. istry officials in attendance.

Figure 5: Specialized Support: Other Jurisdictions Prepared by the Office of the Auditor General of Ontario

Jurisdiction Specialized Support Available Canada The Library of Parliament’s Parliamentary Information and Research Service provides research support. In the past, it has hired analysts to provide additional support to committees doing estimates work. Such support includes: • briefings on the Estimates and the estimates process; • background research on programs and activities; and • analysis of plans and briefing notes to help Committee members prepare for meetings and pose appropriate questions. The Parliamentary Budget Office also produces additional reports that can be used by committees. United Kingdom Recent reforms include the formation of a Scrutiny Unit that has a staff of 15, including economists, lawyers and accountants. The unit provides select committees with financial and legislative expertise, identifies areas of interest and suggests lines of inquiry. Northern Ireland, Specialist researchers within the research unit of the governing body are available to provide guidance Wales and Scotland on financial and technical matters. 184 2012 Annual Report of the Office of the Auditor General of Ontario

Given that the larger ministries spend bil- When asked about the estimates briefing materi- lions of dollars in dozens of different areas, a als provided to members, the former MPPs agreed more focused review of several key programs that more relevant and future-oriented perform- for at least part of those ministries’ time allo- ance information was needed. One said: cations might have merit. Either a subcommit- In my view, ministries spend a tremen- tee composed of one member from each party dous amount of time preparing material would select the ministry programs for more for briefing books that members do not detailed examination, or each party would really use—but if properly oriented, esti- select one program for review. The minister mates review would have more meaning would continue to attend a significant portion if members had good information on proof estimates hearings so that policy issues in gram performance and if the information any area could still be raised. also focused on looking forward rather In regard to our proposals, the former MPPs we than just looking at the past. met with responded positively to having advance briefings. One suggested that to reduce partisan- On the issue of focusing on only a few specific ship, only ministry staff should be present during programs for part of the review, two of the retired meetings, to focus questions away from policy and MPPs supported this proposal, while one member toward program administration. However, another was concerned this could limit the scope of mem- former MPP disagreed, believing that attendance by ber questions. the minister would be useful. 3.07

Section

VFM

•

Chapter Chapter 3 Ministry of Health and Long-Term Care Section 3.08 Long-term-care Home Placement Process

Each CCAC reports to one of 14 Local Health Background Integration Networks (LHINs). In the 2011/12 fiscal year, the Ministry of Health and Long-Term Care (Ministry) provided $2.1 billion of funding to the Long-term-care homes (LTC homes) provide care, CCACs through the LHINs. This funding covered services and accommodation to people who require

the CCACs’ LTC home placement services, as well 3.08 the availability of 24-hour nursing care, super- as their other activities, including the provision of vision in a secure setting, or frequent assistance

home care and community support services. Infor- Section with activities of daily living such as dressing and mation was not available on the cost of LTC home VFM bathing. LTC homes are sometimes referred to as •

placement alone. The Ministry, to which the LHINs 3 nursing homes or homes for the aged. They may be are accountable, is responsible for ensuring that for-profit, not-for-profit, or municipally run organ- CCACs comply with provisions for LTC home place-

izations, and often have waiting lists for their beds. Chapter ment under the Act. In July 2010, the Long-Term Care Homes Act, 2007 (Act) came into effect upon finalization of related regulations. This Act replaced the Nursing Homes Act, the Charitable Institutions Act and the Audit Objective and Scope Homes for Aged and Rest Homes Act. It governs the process for placing people in LTC homes, and The objective of our audit was to assess whether authorizes the province’s 14 Community Care the processes in place at selected CCACs were Access Centres (CCACs) to manage this process effective for placing individuals in LTC homes (see Figure 1 for CCAC boundaries). Accordingly, in a consistent and timely manner, based on the CCACs determine eligibility for admission, pri- their needs and in accordance with ministry and oritize eligible individuals on LTC homes’ wait lists legislative requirements. and arrange placement when a bed becomes avail- We conducted our audit work at three Commun- able. In the 2011/12 fiscal year, the CCACs placed ity Care Access Centres of different sizes: Central more than 25,000 people in Ontario’s 640 LTC East CCAC (responsible for 9,700 LTC home beds, homes, which have a total of 76,000 long-term-care with head office in Whitby); North East CCAC beds that are over 97% occupied. About 85% of the (responsible for 5,000 LTC home beds, with head people placed were aged 75 and older.

185 186 2012 Annual Report of the Office of the Auditor General of Ontario

Figure 1: Community Care Access Centre Boundaries Source of data: Ontario Association of Community Care Access Centres

1. Erie St. Clair 2. South West 3. Waterloo Wellington 4. Hamilton Niagara Haldimand Brant 5. Central West 6. Mississauga Halton 7. Toronto Central 8. Central 9. Central East 10. South East 11. Champlain 12. North Simcoe Muskoka 13. North East 14. North West

office in Sudbury); and Waterloo Wellington CCAC We did not rely on reports from the Ministry’s 3.08 (responsible for 3,800 LTC home beds, with head internal audit service team because it had not office in Kitchener). Senior Ministry and CCAC conducted any work on the CCACs’ LTC home Section management reviewed and generally agreed to our placement processes. VFM audit objective and associated audit criteria. •

3 The scope of our audit included a review and analysis of relevant files and administrative policies Summary

Chapter and procedures, as well as interviews with appropriate CCAC and ministry staff. We also reviewed relevant research, including best practices for the Since 2005 the number of Ontarians aged 75 and LTC home placement process in other jurisdic- over has increased by more than 20%, which has tions. In addition, we obtained the perspective undoubtedly been a key reason why the median of the Ontario Association of Community Care time that people wait for accommodation in an Access Centres, which represents the 14 CCACs; LTC home has almost tripled—from 36 days in the Ontario Long-term Care Association and the the 2004/05 fiscal year to 98 days in the 2011/12 Ontario Association of Non-profit Homes and Ser- fiscal year. Although wait times have decreased vices for Seniors, which between them represent somewhat since July 2010, when tighter eligibility the majority of LTC homes in the province; and the criteria in the Long-Term Care Homes Act took effect, Advocacy Centre for the Elderly, which represents Ontario’s population of people aged 75 and up is low-income seniors. We also used computerized expected to grow by almost 30% between 2012 data extraction techniques to analyze data from the and 2021, creating additional pressures to meet Ministry’s Client Profile Database, which includes the needs of people who require long-term care. LTC home placement information received from As well, beginning in 2021, the first of the baby the CCACs. boomer generation—those born between 1946 and 1964—will start to turn 75, at which point the Long-term-care Home Placement Process 187 demand for long-term care is expected to become process also helps determine each client’s wait-list even greater. priority; however, more needs to be done to ensure While CCACs are responsible for the process that crisis cases are prioritized consistently. of placing individuals in LTC homes, numerous All three of the CCACs that we visited were factors outside their control affect wait times for managing various areas of their LTC home place- placement. In particular, the Ministry of Health and ment process well. However, all also had areas Long-Term Care (Ministry) is responsible for the where improvements could be made, although number of available LTC home beds; individuals any changes made in these areas would likely not are allowed to select the LTC home(s) that they are significantly improve LTC home wait times. Some of willing to be placed in; and LTC homes may reject our more significant observations are as follows: applications if they believe their home lacks the • The provincial agency Health Quality Ontario nursing expertise or physical facilities needed to indicated that nearly 20% of the CCACs’ meet the applicant’s care requirements. home-care clients who were subsequently Numerous studies have shown that remaining placed in LTC homes could have remained in in hospital longer than medically necessary is detri- the community, and a Ministry-commissioned mental to a patient’s health, yet many people wait study noted that 37% of clients waiting in hos- in hospital for an LTC home bed to become avail- pital for an LTC home bed had care needs that able, which occupies a hospital bed that is often were no more urgent or complex than those of needed by other patients who have more complex people being cared for at home. health-care needs. As well, occupying a hospital bed • Not all people eligible for an LTC home 3.08 is more expensive than community-based alterna- require such care; for example, all veterans tives. This situation is exacerbated because people and spouses of current residents are eligible Section can wait in hospital for the LTC home(s) of their regardless of their health-care needs. VFM choice, even if the chosen home(s) have a lengthy In the 2011/12 fiscal year, CCACs province-

• •

3 wait list. We noted that during the 2011/12 fiscal wide completed a total of about 36,000 formal year, 19% of clients waiting in hospital had applied client reassessments, which are required to be

to only one LTC home. Our research indicated that, completed in six-month intervals and within Chapter to minimize the time such patients spend waiting in three months of LTC home placement. How- hospital, other provinces have stricter policies: five ever, conducting a quick “touch-base” with provinces require patients to go to the first vacant clients and their families might more quickly bed in any LTC home; and two require patients to and cost effectively provide information on go to any home with an available bed within 60 and whether a client’s condition has changed 100 kilometres, respectively. enough to warrant a formal reassessment Given our aging population, developing alterna- rather than conducting reassessments every tives to long-term care and implementing more six months. efficient processes for placing people in an LTC • March 2012 LTC home wait-list data indicated home in a consistent and timely manner is critical. crisis clients had waited a median of 94 days The Ministry has recognized this and has supported up to that point; moderate-needs clients a number of initiatives to help reduce or delay the had waited 10–14 months; and most other need for long-term care, and improve the place- eligible clients had been on the wait list for ment process when a bed in an LTC home is needed. years. Further, during the 2011/12 fiscal For example, all CCACs use a provincially standard- year, 15% of clients died before receiving LTC ized process to determine client eligibility, includ- home accommodation. ing considering alternatives to long-term care. This 188 2012 Annual Report of the Office of the Auditor General of Ontario

While 36% of clients were placed in their first • OVERALL MINISTRY RESPONSE choice of homes, others generally accepted the offered home but remained on the wait The Ministry of Health and Long-Term Care list for their preferred home(s). In fact, in (Ministry) welcomes the advice contained in March 2012, 40% of people on the wait list this value-for-money audit. The audit recognizes already resided in long-term care. At least a 20% increase since 2005 of Ontarians aged half of admissions to more than 70 LTC homes 75 and over, which has impacted wait times for during the 2011/12 fiscal year were for crisis admission to long-term-care (LTC) homes. clients, who typically get priority for the The Long-Term Care Homes Act, 2007 was home of their choice. Consequently, non-crisis proclaimed into force in July 2010 and recog- clients may find it difficult to access accommo- nizes the principle of access based on assessed dation in the newer or more popular homes. need and LTC homes as places where residents • The CCACs visited did not periodically review live with dignity, security, safety and comfort. client placement decisions to ensure that the The Ministry is pleased that the Auditor has highest-priority person meeting an available recognized that the Community Care Access bed’s criteria was offered the bed. Nor did the Centres (CCACs), the designated placement CCAC systems retain wait-list information so coordinator for LTC home admission, are man- that these decisions could be reviewed after aging various areas of the LTC home placement the fact. process well. • Applicants living in some areas of the prov- Ontario’s Action Plan for Health Care is a 3.08 ince get into LTC homes more quickly. At one reflection of the government’s commitment to CCAC, 90% of clients were placed within a better patient care. At the heart of the plan is Section low of 317 days, whereas at another CCAC, a commitment to ensure that patients receive VFM it took about 1,100 days until 90% of clients timely access to the most appropriate care in the •

3 were placed. most appropriate place. It is about getting the • While LTC homes can designate up to 60% greatest value for patients from the system and

Chapter of their beds as preferred accommodation ensuring seniors receive the care they need as (that is, private or semi-private), only 40% close to home as possible. of clients apply for these more costly beds. The Ministry is committed to supporting sen- Therefore, regardless of care needs, clients iors to remain in their community. For example: who can afford to pay for preferred accommo- • The Ontario government’s Aging at Home dation tend to get placed more quickly than Strategy, announced in August 2007, other clients. invested close to $1.1 billion over four years • Although information on LTC home wait in the delivery of an integrated continuum times by priority level or accommodation of community-based services so seniors can type (that is, private, semi-private and basic) stay healthy and live more independently in would help people consider where to apply, their homes. only one CCAC we visited made some of this • Increased investments were made in CCACs information available publicly. to support the transition of patients from hospital to home or community settings. • In January 2011 the Assisted Living Services for High-Risk Seniors policy was introduced to address the needs of high-risk seniors who reside at home and require the availability Long-term-care Home Placement Process 189

services designed to relieve pressures on hos- of personal support and homemaking ser- pitals and LTC homes by helping to find the vices on a 24-hour basis to avoid premature appropriate health-care setting for clients. admission to an LTC home. In 2008, LHINs working with CCACs intro- The The Ministry is working on launching a • duced the Wait at Home approach to provide Seniors Strategy with a focus on supporting sen- CCAC-organized homemaking and personal iors to stay healthy and to stay at home longer, support services to higher-needs clients who reducing the strain on hospitals and LTC homes. required more help than that provided in The Ministry will continue to work with the regular home-care hours. This enabled clients Local Health Integration Networks, CCACs and to wait in their homes for a long-term-care the LTC home sector to identify opportunities to vacancy rather than waiting in hospital. improve the LTC home placement process. • In July 2010, the Long-Term Care Homes Act took effect. Among other things, it introduced stricter eligibility criteria for LTC home placement. For example, it no longer permitted Detailed Audit Observations people access based solely on whether they would be financially, emotionally or physically harmed if they stayed in their current OVERVIEW OF PLACEMENT PROCESS residence. It also increased the number of In general, the key steps followed to place a client LTC homes to which an individual may apply 3.08 in an LTC home involve: the CCAC determining a from three to five, and decreased the wait to client’s eligibility and priority for LTC home accom- reapply from six months to 12 weeks in cases Section modation through a formal assessment process; where the client refuses a bed at an LTC home VFM clients applying to one or more LTC homes; clients to which they applied. •

3 accepted by the LTC homes being put on a wait list • The provincially standardized Resident if a bed is not available; and clients at the top of a Assessment Instrument for Home Care, which

wait list being offered a bed. Figure 2 illustrates is used to consistently determine clients’ Chapter these key steps. eligibility for long-term care and prioritize clients on the basis of urgency, is continuing to be refined. INITIATIVES • The piloting of Resource Matching and The Ministry has supported a number of initia- Referral systems, which help match hospital tives to help the LTC home placement process patients to the earliest available appropriate work consistently, fairly and in the most timely LTC home bed, is expected to shorten the manner possible: placement process. At the time of our audit, • From 2007 to 2012, the Aging at Home two LHINs were testing their own systems in program helped seniors stay in their homes conjunction with their associated CCACs; the longer through home-care assistance and remaining LHINs were expected to pilot simi- thereby postponed or reduced the need for lar systems during the 2013/14 fiscal year. long-term-care accommodation. Although this • A document management system supported initiative has ended, the Ministry indicated by the Ontario Association of Community that the LHINs are continuing to implement Care Access Centres has been implemented community-based programs and services by six CCACs, and the Association indicated that support seniors, including programs and that the remaining CCACs would also be 190 2012 Annual Report of the Office of the Auditor General of Ontario

Figure 2: Key Steps in Long-term-care Home Placement Prepared by the Office of the Auditor General of Ontario

KEY STEPS CLIENT’S ROLE CCAC’S ROLE LTC HOME’S ROLE

Assesses whether client is I. Assessment of Requests eligibility eligible for LTC home placement Client Eligibility assessment. and informs client of decision.

II. Prioritization Determines priority level of of Clients eligible clients.

Completes application, III. Application Sends applications to homes usually choosing 1–5 homes Accepts/rejects application. to Home chosen by client. in order of preference.

Puts accepted client on homes’ wait lists based on priority level 3.08 and usually by date of Informs CCAC that bed is application to home. available. Periodically reassesses client Section lV. Wait for Bed eligibility and priority level. VFM

•

Selects client highest on 3 wait list who matches Accepts/rejects application. criteria for the bed. Chapter

bed offer and moves Accepts Offers bed to accepted client. V. Offer of Bed into home, or rejects* offer.

* Clients who reject a bed offer are generally removed from all LTC home wait lists, but may reapply after 12 weeks. Hospitalized clients who reject a bed offer stay on the LTC home wait lists, but hospitals have the option of charging these patients a hospital-determined fee to continue waiting in hospital for an LTC home bed.

implementing this system. This system as well as improved access, including greater enables CCACs and LTC homes to send wheelchair access for residents, and therefore and receive clients’ medical and placement make these homes a viable option for many information electronically. more people. • Over the next 10 years, older LTC homes • A toll-free Long-term Care Action Line has containing 35,000 beds will be renovated. The been established to allow citizens to phone the Ministry indicated that the renovations will Ministry with concerns and complaints about provide more modern and comfortable living, LTC homes and the placement process. Long-term-care Home Placement Process 191

WAIT-LIST MANAGEMENT Figure 3: People Waiting for a Long-term-care Home The number of people across the province waiting Bed, 2005–2012 for an LTC home bed increased by almost 85% Source of data: Ministry of Health and Long-Term Care between March 2005 and March 2012, as shown 40,000 in Figure 3, while the number of LTC home beds 35,000 increased by about 3%. However, the number of people waiting decreased by almost 15% between 30,000 March 2010 and March 2012. This was primarily 25,000 due to the stricter eligibility criteria in the new Act. 20,000 Of the 32,000 people on the wait list as of 15,000 March 31, 2012, about 19,000 (or about 60%) were 10,000 waiting for placement in an LTC home. The remaining 13,000 (or about 40%) already resided in 5,000 long-term care, but were waiting for another, more 0 preferred, home. 2007 2010 2005 2006 2008 2009 2011* 2012* On average, because residents tend to be older Note: Data as of March 31 each year. and often in poor health, they live in LTC homes for * Reduction due primarily to tightened LTC home eligibility rules. about three years. Therefore, although the numbers may vary among homes, about one-third of the completed. If the preliminary assessment indicates

76,000 LTC home beds in Ontario become available that the client is likely to require home care for 3.08 each year. more than 60 days or admission to an LTC home,

ministry policy requires the CCAC to complete Section

an eligibility assessment within the next 14 days. VFM Determining Client Eligibility •

However, if the client seeking LTC home admission 3 To be eligible for an LTC home, individuals must be is already receiving CCAC services, there is no simi- at least 18 years old and insured under the Ontario lar requirement. To help reduce the time patients Health Insurance Plan. In addition, they generally spend waiting in a hospital bed, two of the CCACs Chapter must require the availability of 24-hour nursing we visited had established stricter internal policies care, supervision in a secure setting, or frequent on the timeframe to assess them—three days at one assistance with activities of daily living such as and five days at the other—regardless of whether dressing and bathing. they were already receiving CCAC services. According to ministry information for the 2011/12 fiscal year, province-wide the CCACs Timing until Assessment completed almost 80% of the assessments for hos- Potentially eligible candidates come to the attention pitalized individuals within 14 days of receiving the of CCACs in a number of ways: people may apply request for LTC home accommodation; for people for long-term care on their own or on someone’s applying from home, 60% were assessed within 14 behalf; the physician of a hospitalized individual days and 90% were assessed within 54 days. may refer him or her to a CCAC; or CCAC staff may refer an existing client. If the individual is not an existing client, the Reviewing Alternatives CCAC generally conducts a preliminary assessment To enable people to live at home as long as possible, within a few days to determine how urgently a full the Act requires that CCACs review all community- eligibility assessment for long-term care should be based alternatives before determining that a client 192 2012 Annual Report of the Office of the Auditor General of Ontario

is eligible for long-term care. For people who do not an LTC home using the provincially standardized need the full range of LTC home services, alterna- Resident Assessment Instrument for Home Care tives might include more day or respite programs, (RAI-HC). This assesses, among other things, the or supportive-housing and assisted-living options— client’s level of functioning, behaviour patterns for example, people live in their own apartment in a and requirements for personal care. It includes building that has on-site care available. a Method for Assigning Priority Levels (MAPLe), Almost all cases we reviewed at the CCACs vis- which helps determine the urgency of the client’s ited indicated that at least some alternatives to an need for long-term care. A rating score is generated LTC home were investigated. However, satisfaction based on the results of the assessments. surveys conducted by those CCACs indicated that A client whose RAI-HC score is 7 or less is usu- between 30% and 44% of clients did not feel they ally not considered eligible for an LTC home, while were informed of all the available alternatives (the a client with a score of 11 or more is considered eli- CCACs each surveyed a relatively small number of gible. Determining eligibility of clients with scores LTC home placement clients). One CCAC indicated of 8 to 10 is generally based on the MAPLe score that many clients received CCAC care at home over and the case manager’s professional judgment of an extended period, and that alternatives were the extent of caregiver burden present in the client’s explored during that time. situation. At two of the three CCACs we visited, a According to a 2012 report by Health Qual- senior manager was required to review the decision ity Ontario, a provincial agency that monitors if a client had a score of 8 to 10, and eligibility was and reports on health care, in the 2010/11 fiscal often based largely on caregiver burden. The third 3.08 year nearly 20% of the CCACs’ home-care clients CCAC had no formal secondary review process, but who were subsequently placed in long-term care indicated that it was implementing one. Section could have stayed in their homes or been placed As part of the eligibility determination process, VFM elsewhere in the community. In 2011, a Ministry- the Act requires that a physician or registered nurse •

3 commissioned report, Caring for Our Aging Popula- complete a health assessment. This five-page assess- tion and Addressing Alternate Level of Care, indicated ment provides information on, among other things,

Chapter that 37% of clients waiting in hospital for an LTC the client’s condition, including any medications. home bed have care needs that are no more urgent The three CCACs told us that these assessments or complex than those of many people being cared do not add much value to the eligibility process, as for successfully at home. This report also suggested they are often not fully completed by the client’s that LTC homes should focus more of their capacity physician and duplicate information obtained on restorative and transitional care programs through the RAI-HC. The Ministry indicated that (which promote, for example, the recovery of the health assessment is an intentional secondary strength, endurance and functioning) that might review to better ensure that all client information is assist clients in moving out of hospital more quickly accurate. We were informed that, in 2010, the Prov- and potentially returning to their own home rather incial Placement Committee (a group comprising than residing in an LTC home. However, these pro- representatives from each CCAC and the Ontario grams represented only about 2% of the LTC home Association of CCACs) recommended discontinuing system capacity at the time of the report. these health assessments. However, they were still required at the time of our audit. According to information we received from the Assessing Client Needs Ministry, in the 2011/12 fiscal year the CCACs con- CCAC staff—generally a case manager or a place- ducted more than 40,000 eligibility assessments of ment co-ordinator—determine clients’ eligibility for more than 36,000 individuals province-wide; some Long-term-care Home Placement Process 193 people are assessed more than once, for instance if be as high as others’. Therefore, in many of these they change their minds about long-term care. The cases, clients with lesser care needs may be ranked CCACs determined that more than 33,000 appli- ahead of clients with higher care needs. cants were eligible for an LTC home. At the three If a person’s condition deteriorates, he or she CCACs we visited, the percentage of applicants who may be re-prioritized to a higher level. However, cli- met the placement eligibility criteria ranged from ents will be placed on the 3A and 4A wait list using 88% to 93% that year. Rejected applicants may their original 3B or 4B wait-list date—and therefore appeal their case to the Health Services Appeal and possibly ahead of other 3A or 4A clients who have Review Board, an independent quasi-judicial tribu- been waiting at a higher-needs priority level for a nal established by the Ministry. The three CCACs longer time. informed us that appeals rarely occur. All three of the CCACs used judgment in determining which clients were a crisis priority, including a determination of the caregiver’s burden. Determining Initial Priority Level of Clients Caregiver burden considerations can potentially In the event that accommodation is not available give one client priority over another whose needs immediately, names are added to the LTC home’s are more urgent and whose caregiver burden is at wait list based on each client’s priority level. The least as arduous but whose caregivers are less insist- Act stipulates the priority levels and the eligibil- ent. To reduce the risk of inappropriately designat- ity for each level. The CCACs are responsible for ing clients as crisis, one CCAC required a senior determining at which priority level clients should manager to review each crisis designation and 3.08 be placed. The Ministry and CCACs told us that the sign a crisis approval form. In most of the files we highest priority levels are seldom used—only 33 reviewed, the senior manager had signed this form. Section such clients were on the wait list as of March 31, Another CCAC required a second case manager to VFM 2012. These priority levels include a category review each crisis designation. However, 50% of •

3 for veterans (who are eligible for veterans’ beds, the files we reviewed did not indicate whether this which are less than 1% of all LTC home beds), and review had been completed. The third CCAC did

a category for clients requiring readmission to an not require crisis designations to be reviewed by a Chapter LTC home after being discharged involuntarily, second person, but said that more borderline cases such as following an extended hospitalization. could be discussed between case managers. Figure 4 outlines the other, more commonly used We noted that two of the CCACs used additional priority levels. factors, which were not specifically based on needs, Not all the priority levels are based exclusively for designating clients as crisis. For example, one on medical need—for example, the federally sup- CCAC’s policy included designating clients as crisis ported veterans’ beds in LTC homes are available if they had waited three years in an LTC home that to veterans, even if they have no health-care needs was not their first choice. Another CCAC, which had or would not otherwise be eligible. These beds a program to enable hospitalized clients to wait at therefore may not be available for clients with care home for an LTC placement, designated such clients needs. Similarly, under priority levels 3B and 4B, as crisis once they had waited at home for 30 days, spouses of current residents wishing to live in the mostly because of difficulties the clients had with same LTC home do not require any care needs to coping at home. be eligible. And, while category 2 clients must have Patients waiting in hospital for an LTC home care needs, these clients are prioritized within level bed are generally prioritized as a 3A or 4A, with 2 based on their spouse/partner’s date of admission no priority over people waiting in the community. to the home, even though their care needs may not We noted that one other province gives a higher 194 2012 Annual Report of the Office of the Auditor General of Ontario

Figure 4: Commonly Used Long-term-care Home Priority Levels Source of data: Long-Term Care Homes Act and Ministry of Health and Long-Term Care

Priority Ranking within Description Level Eligibility Criteria Priority Level Crisis 1 • Client requires immediate placement, such as a client with By urgency of client’s dementia whose primary caregiver dies need for placement • Client waiting in a hospital that the LHIN has declared “in crisis” to free up beds when the hospital is experiencing severe capacity pressures Reunification with 2 Client is eligible based on care needs and wishes to reside in By date of spouse/ spouse/partner who the same LTC home as spouse/partner partner’s admission to already resides in the the home LTC home Clients who are of, 3A Client or spouse/partner has applicable background, and By date of application to or whose spouse/ client: LTC home partner is of, the • has higher care needs*; same religion, ethnic • is waiting in hospital; or origin or linguistic • is residing in another LTC home, but this is their first choice origin that the LTC of LTC homes home specializes in 3B Client or spouse/partner has applicable background, and By date of application to client: LTC home

3.08 does not meet eligibility for 3A, but is otherwise eligible • based on care needs; • is residing in another LTC home and has applied for a bed in Section this home, but this is not their first choice of homes; or VFM • does not have care needs, but wishes to reside with •

3 spouse/partner who is already in the home

Other clients 4A Client not eligible for any other higher priority level who: By date of application to • has higher care needs*; LTC home Chapter • is waiting in hospital; or • is residing in another LTC home, but this is their first choice of LTC homes 4B Client not eligible for any other higher priority level who: By date of application to • is eligible based on care needs; LTC home • is residing in another LTC home and has applied for a bed in this home, but this is not their first choice of homes; or • does not have care needs, but wishes to reside with spouse/partner who is already in the home

* The Provincial Placement Committee’s guideline indicates that clients with higher care needs are generally those with a RAI‑HC score of 16 or higher, or those with both a RAI-HC score of 11 to 15 and a MAPLe score of 4 or higher.

wait-list priority to people who are waiting in the patients waiting for an LTC home in this particular hospital, in order to more quickly free up hospital hospital are generally given crisis priority. When beds for other patients. In Ontario, in order to free these patients move up to the crisis priority level, it up hospital beds more quickly when a hospital is causes other 3A/4A patients, both in other hospitals experiencing severe capacity pressures, the LHIN as well as in the community, to wait longer for an can declare the hospital to be “in crisis,” and all LTC home. In 2011, two of the CCACs we visited Long-term-care Home Placement Process 195 had a combined total of nine hospitals declared in Placing Clients on the Wait List crisis; and one of the hospitals was designated as Client Application to Homes being in crisis for most of that year. The Act requires CCACs to provide clients with The Act requires that crisis clients be prioritized information on the implications of different LTC on the basis of urgency of need. However, the home choices, and, if the client wishes, assistance CCACs we visited told us that all crisis clients have in selecting homes. Clients eligible for an LTC high needs, so it is often hard to distinguish whose home generally select a maximum of five homes, needs are more urgent. with crisis clients permitted to select an unlimited One of the CCACs prioritized crisis clients on number of homes. Clients complete an application the basis of their total wait time for an LTC home. for the home(s) to which they wish to apply; if Another CCAC prioritized most crisis clients on more than one is selected, they rank them in order their total wait time as well, but also maintained of preference. This application is provided to the a “high-crisis” list that gave top priority to clients CCAC. However, there is no deadline for completing with the most urgent needs waiting in the com- the application. This gives clients and their family munity. The third CCAC generally gave priority to time to consider their options carefully. However, crisis clients according to their wait time in just the for hospital patients it can also extend the time that crisis category. they occupy a hospital bed. After crisis clients, spouses and partners— Under the Act, clients have the right to voluntar- including relatives or friends—of current LTC home ily choose which LTC home(s) they want to apply clients are the next-highest-level priority. During

to; in February 2011, and again in May 2012, the 3.08 our audit, the Ministry clarified its definition of Ministry clarified with the LHINs that clients can- partner to include only those individuals who had

not be required to choose from a pre-selected list of Section lived with the client during the year preceding homes. In essence, clients can only be placed in a VFM the client’s application to an LTC home. All three •

home that is acceptable to them. However, we noted 3 CCACs visited had adopted this clarified definition. that one CCAC had a policy of asking crisis clients to The Act gives clients seeking LTC homes that select homes with current vacancies or short waiting

serve their religion, ethnic origin or linguistic origin Chapter lists if the applicant’s selected home(s) could not a category 3 priority level. Although no documenta- accommodate immediate admission. If the client did tion is required to be placed in this priority level, not agree to do this—perhaps because of distance it is important for CCACs to accurately identify from family or because the homes were older facili- clients who are entitled to this priority. However, ties—he or she might lose the crisis designation, and for three of the specialty homes at one CCAC, 26 be moved to a lower priority level by the CCAC. We clients, accounting for 75% of all the clients ranked noted that another CCAC had a policy in place until at the priority level 4A/4B, actually qualified for the October 2011 that required crisis clients to choose higher 3A/3B priority level. We brought these cases all homes within 70 kilometres of their residence, to the attention of the CCAC, which reclassified or similarly risk losing their crisis designation. them to priority level 3A/3B. While all the clients However, the Ministry required this CCAC to change moved up the wait list, nine of the clients moved up the policy, stating that the crisis designation is based the wait list by more than 600 people. This CCAC on the clients’ condition or circumstances, not on indicated that, because no one moved to the top their willingness to consider alternative LTC home position on the wait list, none of these clients had choices. This CCAC indicated that it focuses on cli- missed a bed offer. ents requiring immediate placement when designating clients as crisis. 196 2012 Annual Report of the Office of the Auditor General of Ontario

Numerous studies have shown that remaining If the home requires more information from the in hospital longer than medically necessary, includ- CCAC, the home has three additional days after ing waiting in hospital for an LTC home, can be it receives the information to make its decision. detrimental to a person’s health for various reasons, According to ministry data, in the 2011/12 fiscal among them the potential for a hospital-acquired year LTC homes province-wide made a decision infection such as C. difficile, and, for older patients, on 65% of the applications within five business a decline in physical and mental abilities due to days. LTC homes responded to 90% of applications lack of activity. As well, it is much more costly for a within 28 days. person to wait in hospital than in an LTC home or We were informed that LTC homes rarely reject at home with appropriate home-care support, and clients unless they have very high care needs; in the it might prevent another person requiring hospital 2011/12 fiscal year, only about 1% of clients’ appli- care from occupying that bed. We determined using cations were rejected. An accepted client can move ministry data that, during the 2011/12 fiscal year, into the LTC home immediately if a bed is available. 19% of clients waiting in hospital had applied to However, in most cases the client is added to the only one home. In fact, at one CCAC we visited, home’s wait list based on the client’s priority level 35% of clients waiting in hospital had applied to because no bed is currently available. only one LTC home. While this practice would tend to increase the time clients wait in hospital for a bed Reassessing Clients in their preferred home, we noted that the median wait time province-wide for 3A and 4A hospitalized In some cases a client’s condition can deteriorate 3.08 clients was about half that of people waiting in the significantly while waiting for an LTC home bed. community. This may be due to hospitals encour- These changes might merit adjusting the client’s Section aging clients to apply to homes with vacant beds. priority level to a higher level, such as a crisis VFM Rather than allowing patients to wait in priority. As well, when beds become available, LTC •

3 hospital for their preferred home, our research homes require up-to-date information about the indicated that many other provinces have stricter care needs of the clients who are moving in.

Chapter policies: five provinces require the patient to go CCACs may be made aware of changes in a to the first vacant bed in any LTC home; and two client’s condition by various means, for instance provinces require patients to go to any LTC home when the client’s family contacts the CCAC or within 60 and 100 kilometres, generally of their when the CCAC conducts a reassessment. The Act home, respectively. requires that clients have an assessment or reassessment within three months of their placement in long-term care, which helps ensure LTC homes Acceptance/Rejection by LTC Homes have up-to-date information to prepare for the The CCAC forwards client applications, including client’s needs. This involves completing all aspects information on the client’s care needs, to the applic- of the initial eligibility assessment again, including able LTC homes. At the three CCACs visited, this the RAI-HC assessment and the health assessment information was usually faxed. At the time of our (generally obtained from the client’s physician). As audit, the Ontario Association for Community Care well, ministry policy states that a reassessment of Access Centres was piloting a system at six CCACs the RAI-HC should be completed every six months. for the electronic transmission of documents to and This applies to clients requiring care at home, to from LTC homes. ensure their care plans meet their needs, as well as Under the Act, LTC homes generally have five to clients not receiving care at home and hospital- business days to accept or reject an application. ized clients. CCACs province-wide completed a Long-term-care Home Placement Process 197 combined total of about 36,000 reassessments visited did a one-time check of clients already in of clients on LTC home wait lists in the 2011/12 an LTC home who were on the wait list for another fiscal year. home, resulting in about 10% of those clients being The CCACs visited confirmed that it is chal- removed from the wait lists. lenging to complete all required reassessments, and therefore they used their resources as follows: Placing Clients in LTC Homes • Two CCACs ensured that assessments/ reassessments for the 10 highest-priority When a bed becomes available, the LTC home noti- clients on each home’s wait list had been fies the CCAC and provides information on the type completed in the past three months. As well, of bed. This information includes whether the bed one CCAC indicated that clients with condi- is basic (varies from one or two people per room tions that were likely to change would be in newer homes, to three or four people in older reassessed every six months, while the other homes) or preferred (that is, semi-private or pri- CCAC indicated that it conducted as many vate) to match the client’s request; whether a bed in six-month reassessments as possible. a shared room is appropriate for a male or female; • The third CCAC’s approach was to reassess and whether the bed is in a locked area and appro- 3A/4A clients every three months and 3B/4B priate for clients requiring secure accommodation. clients every six months. (Clients ranked Once notified, the CCAC selects the client at 3A/4A at each home are eligible for beds the top of that home’s wait list who matches the before 3B/4B clients.) specifications of the available bed. To better match 3.08 For hospitalized clients and clients not receiving hospital patients to the first available appropriate CCAC care at home, conducting a quick “touch- LTC home bed, Resource Matching and Referral Section base” with clients and other appropriate persons, systems were being piloted in two LHINs, and the VFM including families, could more quickly provide remaining LHINs were expected to pilot similar sys- •

3 the CCAC with information on whether a client’s tems during the 2013/14 fiscal year. The CCAC then condition has changed, and therefore whether a sends current information on the selected client to

more formal reassessment is needed to determine, the LTC home. The LTC home reviews the informa- Chapter for example, whether a client’s priority level might tion and may accept or reject the client. Rejections have also changed. Furthermore, identifying and from LTC homes at this stage are generally because reassessing clients who are likely to be placed the client’s needs have changed significantly since within the next three months, such as those nearing the home accepted the initial application. If the the top of the wait lists, might provide LTC homes applicant is rejected, the client is notified and the with the information needed to prepare for clients CCAC generally removes this client from the home’s without requiring that the CCAC repeatedly con- wait list and proceeds to the next person. Rejected duct a formal reassessment of all individuals every clients may apply to another home. six months. If the client is accepted, the CCAC contacts the As well, a quick “touch-base” might identify individual and offers him or her the bed. Under clients already in an LTC home who stayed on wait the Act, the client has 24 hours to respond. If the lists for other more preferred homes, but who have client accepts the bed offer, the Act requires that subsequently decided to remain where they are. he or she move into the LTC home within five days. One CCAC indicated that it followed up with clients Our review of ministry data indicated that 83% of after six weeks, and all three CCACs indicated beds offered to clients were accepted province-wide that they followed up annually, for example, as in the 2011/12 fiscal year. Furthermore, 36% of part of client reassessments. However, one CCAC the clients who were placed got their first choice 198 2012 Annual Report of the Office of the Auditor General of Ontario

of homes. Other people who had not selected the fore require a reassessment of their needs, home as their first choice nonetheless were offered rather than conducting formal reassessments the bed, and, after accepting, generally remained of all clients every six months as is currently on the wait list for their preferred home(s). required; and If a client applying from the community rejects conduct periodic independent reviews of a bed, he or she is generally removed from all LTC • placement decisions to ensure that the home wait lists for a period of 12 weeks. If the highest-priority client matching the bed client was waiting in hospital, he or she may remain specifications (such as male versus female, on the wait list for LTC homes, but under the Public and private versus semi-private and basic Hospitals Act the hospital has the option of charging accommodation) is offered the first available the client a hospital-determined fee to continue LTC home bed. waiting in a hospital bed. In the latter half of the 2010/11 fiscal year, two CCAC RESPONSE of the CCACs visited incorporated a process into their information system to assist in tracking the All of the CCACs visited agreed with this recom- status of available LTC home beds, including when mendation and indicated that they would: the bed became available, to whom the bed was • work with the Provincial Placement Working offered, the date the bed was offered, and whether Group (a group comprising representatives the client accepted or rejected the bed. The third from each CCAC and the Ontario Association CCAC used spreadsheets to track this information of CCACs) to develop a consistent province- 3.08 but indicated that it was implementing an inte- wide process for ranking clients within the grated system similar to that of the other CCACs. crisis priority level; Section We noted that CCACs did not periodically review • work with the Ministry of Health and VFM client placement decisions in order to ensure the Long-Term Care to create a province-wide •

3 highest-priority person was offered the available standardization of the current CCAC probed. Furthermore, the information system used by cesses to include a periodic “touch-base”

Chapter the CCACs did not have the capability to retrieve to determine whether wait-listed clients’ what an LTC home’s wait list looked like on a conditions or circumstances have changed specific date, and therefore CCACs could not review and whether a formal reassessment is these decisions after the fact. required; and • in conjunction with the Ontario Association RECOMMENDATION 1 of CCACs, develop the necessary reports to conduct periodic independent reviews To better ensure that higher-needs clients are of placement decisions to ensure that the identified and placed in long-term-care homes highest-priority client matching the bed (LTC homes) as soon as possible, Community specifications is offered the available LTC Care Access Centres (CCACs) should: home bed. • develop a consistent province-wide process for ranking clients within the crisis priority level; RECOMMENDATION 2 in consultation with the Ministry, consider • To help clients move out of hospital more conducting a periodic “touch-base” to deter- quickly and to help manage growing wait lists, mine whether wait-listed clients’ condition the Ministry of Health and Long-Term Care or circumstances have changed and there- Long-term-care Home Placement Process 199

(Ministry) should consider options employed care, and avoidable hospital readmissions. by other jurisdictions, as well as making more The government also signalled its intent to community alternatives to long-term-care increase financial support to the community (LTC) homes available and having LTC homes sector for three fiscal years. provide more restorative and transitional care • Regulatory amendments made in June 2008 programs to improve, among other things, and September 2009 enabled innovation clients’ functioning. and flexibility in the delivery of Community As well, to better ensure that clients assessed Care Access Centre (CCAC) home-care and as eligible for an LTC home are placed as soon community services. Service maximums for as possible, the Ministry should streamline the personal support/homemaking and nursing client health assessment (to avoid duplicating were increased, and new services and service information that is already obtained as part of locations were introduced. the eligibility assessment and to avoid poten- The current health assessment requirements tially delaying the process). were a response to concerns relating to the content and timing of assessments during the MINISTRY RESPONSE placement process that were identified as a result of a Coroner’s Inquest into the deaths of The Ministry values the Auditor General’s rec- two residents at the hands of another resident ommendation and will continue to demonstrate on his first day of admission (Casa Verde Nurs- its commitment to supporting seniors to remain

ing Home). In some instances duplication is a 3.08 in their community through more community necessary part of verifying information about alternatives. For example:

a client’s status and is a critical component of a Section As part of Ontario’s Action Plan for Health • comprehensive assessment. The Ministry will VFM Care the Ministry has announced a Seniors •

work with the CCACs and LTC homes to review 3 Strategy focusing on supporting seniors to the health assessment process and will look for stay healthy and live at home longer through opportunities to reduce unnecessary duplication

enhanced preventative care and home-care Chapter in the process. services, thereby reducing pressures on LTC homes and hospitals. The strategy will help inform decisions regarding the role of restorative and short-stay programs in LTC WAITES TIM homes and the future development of com- According to ministry data, 50% of clients munity alternatives closer to home. province-wide (excluding crisis, spousal/partner The 2012/13 Community Sector Invest- • reunifications and persons waiting for a transfer ment in home care and community services to another home) were placed in an LTC home announced in the 2012 Ontario Budget within 98 days in the 2011/12 fiscal year, with 75% was allocated to the Local Health Integra- of clients placed within 10 months and 90% of tion Networks (LHINs) in August 2012 to clients placed within about two years. This reflects increase investments in home care and the wait from the time a CCAC received a client’s community services to support seniors and request to be assessed for an LTC home until the other Ontarians at home and to reduce the client was placed in a home. The median wait for all numbers of emergency room visits, patients clients not yet in an LTC home, including crisis and waiting in hospital for an alternative level of spousal/partner reunification, drops to 85 days, 200 2012 Annual Report of the Office of the Auditor General of Ontario

primarily due to 90% of crisis clients being placed in 2021, the first of the baby boomer generation— within three months of being designated as crisis. those born between 1946 and 1964—will start to During the 2011/12 fiscal year, 15% of clients died turn 75, at which point the demand for long-term before receiving LTC home accommodation. care is expected to become even greater.

Wait Time Trends Wait Times by CCAC and Client Priority

As Figure 5 shows, the median wait times have Ministry data indicates that applicants who live almost tripled from 36 days in the 2004/05 fis- in some areas of the province get into LTC homes cal year to 98 days in the 2011/12 fiscal year. An more quickly. In particular, median wait times for increase in the number of LTC home beds of 3% LTC homes in the 2011/12 fiscal year ranged from a during that period has not kept pace with the rising low of 50 days at the Erie St. Clair CCAC to a high of demand from an aging population. However, the 187 days at the North West CCAC. (Overall, 90% of wait time has decreased since 2009/10, due in part clients were placed within a low of 317 days at the to tighter eligibility requirements under the new Central West CCAC to within a high of about 1,100 Act that took effect in July 2010. days at the Champlain CCAC, as shown in Figure 6.) About 85% of LTC home residents are aged 75 Part of this variance reflects differences among and over, and between 2005 and 2012 the number CCACs in the demand for long-term care due to of Ontarians aged 75 and older increased by more population health and age characteristics in that than 20%. According to Statistics Canada, between region of the province, as well as the number, age 3.08 2012 and 2021, Ontario’s population aged 75 and and location of LTC home beds. older is expected to increase by almost 30%. This According to the Ontario Hospital Association, Section trend will likely increase the demand for long-term as of March 31, 2012, about 1,000 people were VFM care, although enhanced community alternatives waiting province-wide in an acute-care hospital •

3 could meet some of these needs. As well, beginning bed for a bed in an LTC home; another 1,000 were waiting in other types of hospital beds, such as

Chapter Figure 5: Long-term-care Home Median Wait Times* rehabilitation or mental health beds. We noted that and the Number of Ontarians Aged 75 and Older, the number of people waiting in hospital for an LTC 2004/05–2011/12 home bed had decreased by about 25% since March Source of data: Ministry of Health and Long-Term Care and Statistics Canada 2010. However, for those still waiting, the wait time

1,000 120 until placement had significantly increased. Min- 900 istry information indicated that, in the 2011/12 fis- 100 800 cal year, about half of the acute-care patients were placed within about two months (within one month 700 80 600 in the 2009/10 fiscal year), with 90% placed within 500 60 me (Days) 495 days (within 128 days in 2009/10). Wait times

400 # of Ontarians aged it Ti in hospital tend to be longer for harder-to-care-for

40 Wa

Population (000) 75 years or more 300 patients, for example, people who have dementia, Median wait time 200 20 are significantly overweight, or require frequent 100 medical treatments such as dialysis. 0 0 The Act requires CCACs to provide an “estimated wait time” to clients who request information on 2004/052005/062006/072007/082008/092009/102010/11 2011/12 their expected wait. The CCACs visited said that * Excludes crisis, spousal–partner reunifications and transfers between clients have become upset if their actual wait time LTC homes. Long-term-care Home Placement Process 201

Figure 6: Number of Days Within Which 90% of Each CCAC’s Clients Were Placed, 2011 Source of data: Ministry of Health and Long-Term Care

1,200

1,000

800

600

400

200

0 3.08

Central South East North East North West Champlain Central West Erie St. Clair South West Central East Section Toronto Central Hamilton Niagara Haldimand Brant Waterloo WellingtonMississauga Halton VFM North Simcoe Muskoka •

3 exceeded the estimate. Therefore, the CCACs gener- people in the lower priority levels could often be ally tell clients how long the top person on the wait many years. list, at the same priority level as the client, has been As well, the high priority given to crisis clients Chapter waiting so far. We reviewed ministry wait-list data as may make it difficult for non-crisis clients to get of March 31, 2012, to see how long clients had been into some homes. For example, ministry data shows waiting. For those in the crisis category, including that at more than 70 LTC homes, at least half of those who previously waited at a lower priority level, admissions during the 2011/12 fiscal year were their median total wait time up to that point was crisis clients. In fact, at one CCAC visited, crisis cli- 94 days. People in categories 3A and 4A had been ents made up two-thirds of placements to the four waiting a median of 423 and 309 days, respectively, most popular LTC homes—that is, the homes with with people in categories 3B and 4B (considered to the longest wait lists. Furthermore, ministry data have lower needs than 3A and 4A clients) waiting a showed that more than 40% of crisis placements in median of 712 and 587 days, respectively. the 2011/12 fiscal year were people whose prior- In the 2011/12 fiscal year, about 5,600 (or ity level was escalated to crisis (for example, due 22%) of the LTC home placements were for clients to their condition deteriorating or circumstances in the crisis priority level and another 71% were changing, such as increased caregiver burden) in from the generally higher-need categories 3A and order to place them quickly after they had waited at 4A. As shown in Figure 7, based on the number of a lower priority level, in some cases for an extended people waiting and the number actually placed in period of time. the prior year, the expected time to placement for 202 2012 Annual Report of the Office of the Auditor General of Ontario

Figure 7: Number of Clients Waiting for Long-term Care, Number of Placements and Expected Average Time to Placement for Commonly Used Priority Levels Prepared by the Office of the Auditor General of Ontario

# of People # of People Waiting as of Placed During the Expected Time Priority Level March 31, 2012 2011/12 Fiscal Year to Placement 1 — Crisis 1,400 5,600 3 months 2 — Spousal/partner reunification 200 900 2.5 months 3A — Religious, ethnic, linguistic home (higher needs) 3,100 900 3.5 years 3B — Religious, ethnic, linguistic home 1,400 100 14 years 4A — Other (higher needs) 19,700 17,000 1 year 4B — Other 5,800 800 7 years Total 31,600 25,300

Impact of Client Choices on Wait Times or two people per room, compared with up to four people per room in an older home. Partially as a Although the Ministry provides funding to LTC result of this, newer homes tend to have longer wait homes, residents must make a co-payment for their lists (many over 1,000 people), while less desirable

3.08 accommodation costs. As of July 2012, the monthly homes may have empty beds for lengthier periods co-payment ranged from a low of almost $1,700 per of time. Therefore, clients selecting less desirable

Section month for basic accommodation to a high of over LTC homes that have available beds or short wait $2,200 for private accommodation. The co-payment VFM lists can get placed more quickly. •

for semi-private and private accommodation in 3 newer homes (that is, the approximately 55% of long-term-care beds that generally meet or exceed Public Reporting of Wait Times Chapter the Ministry’s 1999 LTC home design standards) is Health Quality Ontario publishes the overall prov- about $30 to $50 more respectively per month than incial LTC home wait time annually. However, no for older homes. Financial assistance is available information was reported on regional wait times or, from the Ministry if a resident is unable to pay, but more specifically, the wait times for each LTC home. only for basic accommodation. Reporting wait times for particular homes is some- The Act allows LTC homes to designate up to what complex because wait times vary based on a 60% of their beds as preferred—private or semi- number of factors, including the priority level of the private—accommodation, which means a minimum client and the type of accommodation chosen. of 40% of their beds must be basic accommodation. In August 2012, one of the CCACs we visited However, almost 60% of clients applying for an LTC began publicly disclosing on its website informa- home bed requested basic accommodation. There- tion on wait times for each LTC home in its region, fore, clients who can afford to pay for preferred including the number of clients waiting for each accommodation may get placed faster than clients type of accommodation, the average number of applying for basic accommodation, regardless of days that clients have waited so far, and the average their medical needs. number of beds that became available monthly. We Furthermore, people pay the same rate for basic believe this is a good initiative for providing public accommodation in homes of any age, even though wait time information. basic accommodation in a newer home has only one Long-term-care Home Placement Process 203

RECOMMENDATION 3 OVERSIGHT The Ministry is responsible for ensuring compliance To better ensure that clients have sufficient with the Long-Term Care Homes Act (Act), which information on the long-term-care (LTC) home includes the LTC home placement process. The placement process and wait times for LTC home Ministry indicated it commenced monitoring the admission, the Ministry of Health and Long-Term placement process in fall 2011, primarily by follow- Care (Ministry), in conjunction with the Com- ing up on complaints it received directly. We noted munity Care Access Centres (CCACs), should: that the three CCACs visited all had processes in provide the public with detailed information • place for handling complaints. They indicated that on the LTC home admission process and very few complaints—on average, fewer than 20 the policies in place to ensure the process is per year—had been received about their LTC home administered equitably; placement process. Although one CCAC provided examine options for encouraging greater • some general complaint statistics to its LHIN, no utilization of basic accommodation in less other information on complaints was regularly desirable homes; and provided by the CCACs to either the LHINs or promote the public disclosure of informa- • the Ministry. tion that would help people choose which Otherwise, the Ministry has delegated the LTC homes to apply to, such as wait times oversight of the CCACs to the LHINs. The Local by home, by type of accommodation—pri- Health System Integration Act restricts LHINs’ access vate, semi-private and basic—as provided

to information on individual clients. However, 3.08 on one CCAC’s website, and wait time by through the LHINs’ performance agreements with priority level.

the CCACs, the LHINs receive information on the Section

percentage of higher-needs clients who are placed VFM MINISTRY RESPONSE •

in LTC homes. Furthermore, the CCACs visited all 3 The Ministry agrees with the principles identi- provided additional information to their LHINs fied in the recommendation and will work with regarding the LTC home placement process, such the CCACs, through the Ontario Association of as the number of crisis placements by location in Chapter Community Care Access Centres, to continue which the client was waiting and the number of to promote ongoing and timely communication placements by hospital. of appropriate information regarding the LTC Although the Ministry agreed with the recom- home placement process. The Ministry will con- mendation in our 2010 Discharge of Hospital tinue to work in partnership with CCACs in the Patients report, regarding the need to establish ongoing review of these policies and processes. benchmark standards for completing each stage in Further, the Ministry regularly reviews the poli- the LTC home placement process (such as times to cies and processes to ensure maximum utiliza- determine client eligibility, for hospital clients to tion of all levels of accommodation in all homes. complete applications, to get clients onto a wait list, and to place clients), there are still few benchmark CCAC RESPONSE standards for this process. (See the Discharge of Hospital Patients follow-up section in Chapter 4 for Although this recommendation was not directed additional information on the current status of this toward the CCACs, one CCAC highlighted its sup- recommendation.) Other performance measures port for the disclosure of information that would that would provide the Ministry or LHINs with help people choose which long-term-care homes information about the effectiveness of the LTC to apply to, such as average wait times by home. 204 2012 Annual Report of the Office of the Auditor General of Ontario

placement process could include wait times for at home, wait times for clients requesting clients requesting preferred versus basic accom- preferred (that is, private or semi-private) modation; wait times for clients waiting in hospital versus basic accommodation, and the versus at home; percentage of clients who die while percentage of clients who receive their awaiting placement; and percentage of clients requested transfer to another LTC home; and who receive their requested transfer to another develop target guidelines for completing LTC home. • each stage of the LTC home placement process, such as the times to determine client RECOMMENDATION 4 eligibility, for hospital clients to complete To enhance the oversight of the long-term-care placement applications, and for clients to get (LTC) home placement process, the Ministry onto a wait list. of Health and Long-Term Care (Ministry), in conjunction with the Local Health Integration MINISTRY RESPONSE Networks (LHINs) and Community Care Access The Ministry will continue to review the Centres (CCACs), should: data requirements as necessary to provide develop consistent performance measures • regular reporting on and monitoring of for monitoring the process, such as wait performance measures. times for clients waiting in hospital versus 3.08

Section

VFM

•

Chapter Chapter 3 Section 3.09 Metrolinx—Regional Transportation Planning

relates only to upgrading and expanding the regional Background transportation network but does not include the estimated maintenance that is expected to be required to keep the additional transportation infrastructure Metrolinx, an agency of the government of Ontario, in a state of good repair over its useful life. was created by the Greater Toronto Transportation

In the first 15 years, Metrolinx plans to imple- 3.09 Authority Act, 2006, now the Metrolinx Act, 2006 ment the priority transit projects listed in Figure 1. (Act). According to the Act, one of Metrolinx’s

Metrolinx’s estimate of the cost of these projects is Section key objectives is to provide leadership in the co- approximately $33 billion, of which approximately VFM ordination, planning, financing and development of •

$3 billion had been spent by the province as of 3 an integrated, multi-modal transportation network March 31, 2012. For about half of these projects, in the Greater Toronto and Hamilton Area (GTHA). the majority of the funding comes from a 2007

The GTHA consists of two single-tier municipalities Chapter provincial commitment of $11.5 billion, along with (Toronto and Hamilton), four regional munici- previously announced project funding. The remain- palities (Durham, Halton, Peel and York) and 24 ing priority projects that are funded—such as the local municipalities. Air Rail Link between Union Station and Pearson In November 2008, Metrolinx formally adopted International Airport and projects to revitalize a Regional Transportation Plan (RTP)—also known Union Station—are being funded from the prov- as “The Big Move”—that sets out the priorities, ince’s capital budget for GO Transit (the commuter policies and programs for implementing a trans- rail and bus system serving the GTHA, a division of portation system within the GTHA. The RTP, which Metrolinx). At the time it made the 2007 commit- was the result of two years of public consultation, ment, the province asked the federal government to was adopted by Metrolinx’s Board of Directors contribute $6 billion toward the RTP’s implementa- (Board), which at that time included representation. To date, the federal government has com- tives from the GTHA municipalities. mitted $1.93 billion on a project-by-project basis. Among the RTP’s more significant proposals is The combined funding is expected to sustain the to build more than 1,200 km of rapid transit with RTP’s implementation until about 2018. By 2013, the aim of getting 80% of GTHA residents within Metrolinx must provide the province with recom- 2 km of rapid transit. The timeline for implementing mendations for funding the implementation of the the RTP is 25 years. Its estimated cost of $50 billion

205 206 2012 Annual Report of the Office of the Auditor General of Ontario

Figure 1: List of Priority Transit Projects in the Regional Transportation Plan’s First 15 Years Source of data: Metrolinx

Provincial Estimated1 Spending as of Capital Cost March 31, 2012 Transit Priorities ($ million) Funded ($ million) Express rail service from Hamilton to Oshawa 5,970 No2 — Rapid transit line in downtown Hamilton 830 No2 — Rapid transit on Dundas Street in Halton and Peel 650 No2 — 403 transitway from Mississauga City Centre to Renforth Gateway 259 Yes3 89 Hurontario rapid transit from Port Credit to downtown Brampton 1,350 No2 — Brampton’s Queen Street AcceleRide (now Züm) 259 Yes3 95 Air Rail Link (ARL) between Union Station and Pearson Airport4 456 Yes3 40 Upgrades to Georgetown South line 1,501 Yes3 407 VIVA bus rapid transit on Highway 7 and Yonge Street through York Region 1,755 Yes3 295 Yonge Street subway capacity improvements and extension to Richmond Hill 2,380 No2 — Spadina subway extension to Vaughan Metropolitan Centre 2,600 Yes3 891 Rapid transit on Eglinton Avenue 4,600 Yes3 Finch/Sheppard rapid transit 2,150 Yes3 471 Upgrade and extension of Scarborough rapid transit line 1,400 Yes3 3.09 Rapid transit service along Highway 2 in Durham 500 No2 — Improvements to existing GO rail services and extension of GO rail service to 4,300 No2 — Section

Bowmanville

VFM Other Projects

• Presto fare card 701 Yes3 364 3

Union Station revitalization (a combination of projects) 1,393 Yes3 355 Total 33,054 3,007 Chapter 1. Estimated capital costs were established in 2008, 2009 or 2010 depending on the project. 2. Funding is pending, so work has not yet been started on these projects. 3. Funding is in place for these projects, so in most cases work has begun. 4. The ARL will benefit from the upgrades to the Georgetown South line (see following row, beneath dotted line).

remaining projects contemplated under the RTP’s • regularly report on activities and progress first 15 years as well as the projects contemplated in toward achieving the RTP. years 16 through 25. Senior management of Metrolinx reviewed and agreed to our objective and associated audit criteria. We looked at the delivery to date of three major Audit Objective and Scope capital projects contemplated within the RTP’s first 15 years, whose construction or development The objective of our audit was to assess whether was under way at the time of our audit—the Air Metrolinx had adequate systems and procedures in Rail Link, the Presto fare card and two significant place to: projects that form part of the Union Station revital- • cost-effectively implement the initial stages of ization (restoring the train shed and replacing the the Regional Transportation Plan (RTP); and switches in the Union Station Rail Corridor). Metrolinx—Regional Transportation Planning 207

Our audit also assessed the processes followed “seamless, coordinated, efficient, equitable and progress made in implementing the RTP since and user-centred”; its adoption in 2008. • a credible analysis of costs and benefits, The audit was primarily conducted at Metro- based on objective and sound data, exists for linx’s head office in Toronto, where we interviewed each project; staff and reviewed pertinent documents. We also • the project management process ensures that interviewed representatives from many of the risks are managed and that projects are deliv- regions and municipalities within the GTHA, along ered cost-effectively and on time; with representatives of their respective transit agen- • key stakeholders are sufficiently consulted cies, to obtain their perspective on various aspects with; and of the RTP, its overall implementation and the indi- • clear targets are in place for achieving the vidual projects currently being implemented within transportation plan, and there is regu- the plan. As well, we researched transportation lar reporting on progress in relation to planning in other jurisdictions similar to the GTHA these targets. to identify best practices and lessons learned that Our review of the more significant projects in could be applied to implementing the RTP within the early stages of the Regional Transportation Plan the GTHA. (RTP) identified a number of issues that must be Our audit also included a review of the relevant addressed by Metrolinx, if it is to follow the best audit reports issued by Metrolinx’s internal audit practices outlined above. Specifically: department and the province’s internal audit div- • We believe that Metrolinx’s initial assump- 3.09 ision, which were helpful in determining the scope tions about projected annual ridership on and extent of our audit work. the Air Rail Link (ARL) may well be overly Section optimistic. Although a final decision has not VFM been made on whether the ARL must recover •

3 its annual operating costs and any of its cap- Summary ital construction costs, if operating the ARL

on a break-even basis is indeed the objective, Chapter In the Greater Toronto and Hamilton Area (GTHA), achieving that objective may not be feasible. congested roads and highways and public transit Ministry of Transportation documentation systems that are increasingly unable to meet the indicated that a private-sector group that transportation needs of a growing population previously was the successful proponent for support Metrolinx’s mandate of expanding and designing, building and operating the ARL improving regional transportation across the was unable to secure financing for the venture area. We noted that other jurisdictions facing this because prospective lenders felt that despite problem have used similar stand-alone agencies all reasonable efforts to attract riders, the to co-ordinate regional transportation planning. service might not generate enough revenues Accordingly, creating a co-ordinating agency like to be a viable business. A market assess- Metrolinx is a reasonable strategy toward implement conducted by Metrolinx also suggests menting an effective, integrated and sustainable that ARL ridership may not meet the initial transportation network in a large urban centre. assumptions about ridership growth. To successfully deliver on its challenging man- • A region-wide integrated transit fare system date, Metrolinx must ensure that: is one of the RTP’s key strategies. The Presto • individual projects under a regional trans- fare card now sponsored by Metrolinx is portation plan deliver transportation that is regarded as a key component in implementing 208 2012 Annual Report of the Office of the Auditor General of Ontario

this strategy. Metrolinx’s view is that the participating GTHA transit systems as Presto fare-card system creates the underlying of March 31, 2012, was only about 18%. technology platform needed for fare integra- Although seven of the eight municipal tion. But to date the card has not facilitated transit agencies in the 905 area code have fare integration within GTHA transit systems implemented Presto, overall Presto usage because the fares across these systems are on those systems was even lower, at only themselves not integrated. We noted the fol- 6%. These transit agencies currently cannot lowing additional issues with respect to the completely eliminate their old fare systems Presto fare-card system: in favour of Presto because of some of the • When the Presto fare card was initially fare card’s limitations. developed, the Toronto Transit Commission • The contract for the Presto base system (TTC), which has over 80% of the GTHA’s contains 22 measures designed to gauge transit ridership, had not agreed to imple- the contractor’s performance in such ment Presto on its system. Along with the areas as system availability and customer city of Ottawa, the TTC has now condition- management. In 2011, the contractor ally approved Presto’s adoption subject to failed to meet the set standard in nearly satisfactory resolution of some key issues. a third of the measures, but Metrolinx However, to meet the requirements of did not seek any of the related penalties Ottawa and Toronto, Presto Next Genera- stipulated in the contract. The contract tion (PNG) is currently being developed at also contains reliability measures for the 3.09 an anticipated cost of $498 million. In total, devices used by the Presto base system, but more than $700 million could be paid to neither the contractor nor Metrolinx tracks Section the contractor for developing the original this information. VFM Presto system and PNG. We acknowledge The two major projects related to the revital-

• •

3 that Presto is intended to be the primary ization of Union Station have experienced fare collection system on GO Transit and significant cost increases over their initial cost

Chapter municipal transit agencies in the GTHA and estimates. For instance, the cost of restoring therefore must be flexible enough to meet the train shed could now reach $270 mil- the needs of agencies and to adjust to new lion—25% over Metrolinx’s initial estimate. technologies as they become available; Similarly, the cost of replacing the switches in however, it will be among the more expen- the Union Station Rail Corridor could be more sive fare-card systems in the world. than twice the amount of the original pur- • Rather than competitively tendering the chase order, which totalled about $38 million. development of PNG, Metrolinx decided Although those GTHA municipalities and transit to develop it by way of open-ended change agencies we talked to questioned the priority given orders under the existing vendor’s contract. to some of the RTP’s projects, they generally sup- We believe that tendering would, at the ported the plan as currently conceived. However, very least, have informed Metrolinx of some GTHA municipalities indicated that Metrolinx potential new developers and whether needs to provide more regular updates on the major other vendors might have had more cost- projects under the RTP and on the RTP’s overall effective technology solutions. status, including the strategies being considered to • Since going into service approximately two fund projects contemplated under the plan that are years ago, Presto’s overall usage within not yet funded. Metrolinx—Regional Transportation Planning 209

OVERALL METROLINX RESPONSE With nine transit systems in the GTHA, all with various payment methods, Presto intro- Metrolinx welcomes the Auditor General’s duced a new fare-payment system that gives observations and recommendations. We have riders the convenience of being able to travel already taken action to address many of the within the GTHA seamlessly and conveniently. Auditor General’s recommendations, and As a regional fare card serving many transit we will continue our efforts to improve on providers, Presto is a unique product and one our processes. of the world’s most advanced fare-card systems The audit acknowledges that there is a press- (similar to London’s Oyster card, The Nether- ing need to deal with congestion and improve lands’ OV-chipkaart and the Chicago Card). our transportation system. Metrolinx appreci- Today, throughout the GTHA, Presto is deployed ates the Auditor General’s observation that among eight transit agencies and on GO Transit creating Metrolinx was a reasonable strategy systems. As of October 2012, more than 350,000 toward implementing an effective, integrated people were using the Presto card to travel and sustainable transportation network. throughout the multiple transit systems, and we The Greater Toronto and Hamilton Area have added an average of 22,000 customers per (GTHA) is Canada’s largest and fastest-growing month over the last six months. When Presto is urban region. With the GTHA generating 25% fully deployed on the TTC, its usage will grow to of Canada’s GDP, the productivity impacts of 2.5 million customers. congestion are significant. Today, GTHA com- 3.09 mute times are among North America’s longest. With population in the area growing by 100,000 Section people per year, the GTHA is at risk of seeing VFM commute times continue to increase. Detailed Audit Observations •

3 Metrolinx was created less than six years ago with a mandate to transform the way the region The Greater Toronto and Hamilton Area (GTHA) is

moves and a mission to champion and deliver Chapter Ontario’s most populous region, with over 6 million solutions for the GTHA. Our Regional Trans- people—a total that is expected to grow to 8.6 mil- portation Plan (“The Big Move”) is intended lion by 2031. Currently, regional transportation to lead integrated region-wide transit and within the GTHA primarily is served by several transportation planning. major expressways and by 10 different public tran- We are committed to bringing forward sit agencies, each with its own separate fare and new solutions to boost economic growth and infrastructure systems. help people and businesses move more easily Symptoms such as congested roads and highways throughout the region. This transformation has and public transit systems that are increasingly chal- begun through a number of projects, such as the lenged to meet the needs of ever-growing population Mississauga bus rapid transit/403 transitway, levels suggest that there is undoubtedly a pressing the Toronto–York Spadina subway extension, need within the GTHA to expand and improve the York VIVA rapidways, and Toronto light-rail transportation system across the region, because transit projects, as well as improvements on GO the existing system may no longer be meeting Transit’s Kitchener line and the construction of the needs of the area’s residents and businesses. the Air Rail Link between Pearson International Creating a co-ordinating agency like Metrolinx is Airport and Union Station. a reasonable first step toward implementing an 210 2012 Annual Report of the Office of the Auditor General of Ontario

effective, integrated and sustainable transportation tion hubs: Union Station in downtown Toronto and network within the GTHA. Our research indicated Toronto Pearson International Airport. The ARL’s that other major urban centres around the world target completion date is spring 2015, in time for have used similar agencies to co-ordinate regional the Pan/Parapan American Games to be held in transportation planning. Toronto in summer 2015. There are a number of best practices that such The 25 km line will primarily use GO Tran- agencies must follow to ensure the successful imple- sit’s existing Georgetown South rail corridor. A mentation of effective transportation within their new 3.3 km branch line (“spur”) connecting the jurisdiction. Some of the key principles contained Georgetown South line with the airport is also in Metrolinx’s Regional Transportation Plan (RTP) being constructed. The ARL will have four stops: that guide the delivery of the individual projects Union Station, the Bloor GO station, the Weston within the plan and the delivery of the overall plan GO station, and Terminal 1 at the airport. Trains itself include: are expected to run every 15 minutes, seven days • the individual projects should deliver trans- a week. A one-way trip is expected to take 25 min- portation that is “seamless, coordinated, utes. Metrolinx expects the ARL to be a premium efficient, equitable and user-centred”; rail service: some of the features being considered • the projects should be subject to a fair, clear include on-board refreshments, Wi-Fi, power and rigorous benefits case analysis process outlets for laptops, screens with flight information, that considers financial, economic, environ- self-service airline check-in machines and luggage mental and social needs and impacts to ensure facilities. Metrolinx has not yet determined the fare 3.09 that the most optimal investment decisions range for this service. (Fares will probably vary are made; according to how far along the ARL a rider travels— Section • the project delivery process should ensure that is, one, two, or three stops—but the specific VFM that risks are managed and that projects are fares have not yet been determined.) •

3 delivered cost-effectively and on time; As Figure 3 shows, the ARL’s estimated cost • there should be sufficient consultation with is about $456 million. A significant number of

Chapter key stakeholders; and enhancements are also under way on the George- • there should be clear targets for achieving the town South rail corridor, primarily to support an RTP and regular reporting on progress in rela- increase in the level of service for GO Transit on tion to these targets. that line; but the ARL will also benefit from these In reviewing several of the major priority transit enhancements. The estimated total cost of the projects contemplated within the RTP’s first 15 enhancements on the Georgetown South rail cor- years and in discussion with GTHA municipalities ridor is about $1.5 billion. and transit agencies, we noted that Metrolinx has encountered challenges in successfully imple- Cost Recovery menting some of these practices. The following are our specific observations. At the time of our audit, the province had not specifically required that Metrolinx recover the cost of operating the ARL from revenues that the AIR RAIL LINK service generates. The Ministry of Transportation One of the more significant RTP projects currently (Ministry) informed us that Metrolinx would set the under construction is the Air Rail Link (ARL). As ARL’s fare in consultation with the province. Figure 2 shows, the completed ARL will provide rail If operating the ARL on a break-even basis service between Canada’s two busiest transporta- is indeed the objective, this may prove to be Metrolinx—Regional Transportation Planning 211

Figure 2: Map of the Air Rail Link Source of data: Metrolinx 3.09

Section

VFM

•

Chapter

financing for the venture because its lenders did Figure 3: Estimated Cost of the Air Rail Link ($ million) not feel that they had sufficient protection from Source of data: Metrolinx “no market” risk (that is, from a situation where, Estimated despite all reasonable efforts to attract riders, the Description Cost service does not generate enough revenues to be a “Spur” and station in Terminal 1 at 168 viable business). They perceived this project to be Pearson Airport riskier than other infrastructure projects because Trains 98 Other (stations, tracks and signals, etc.) 190 there was no “pre-existing demonstrated revenue stream.” The group proposed that the province Total 456 assume the lenders’ risk by purchasing ARL assets if the “no market” scenario arose. The province a challenge for Metrolinx. In 2003, Transport rejected this proposal, so the group walked away Canada announced a private-sector group as from the project. In 2010, the government decided the successful Public–Private Partnership (P3) that the province, through Metrolinx, would build proponent that would design, build and operate and operate the ARL itself. the ARL. However, the group was unable to secure 212 2012 Annual Report of the Office of the Auditor General of Ontario

Metrolinx’s preliminary estimate of the ARL’s Their one-way ticket prices range from only annual operating cost is approximately $30 million. $1.60 to $13.00, compared to a ticket price However, according to Metrolinx, the cost could for the ARL that may well cost $20 to $30. We well be higher, because the service’s exact nature believe that the ARL’s high fare will negatively has not been finalized, so some relevant costs may affect the projected ridership capture rate. not have been identified yet. For example, the • The results of a market assessment of GTHA estimate does not include the annual access fee of residents conducted in November 2011 by approximately $5 million that GO Transit was going Metrolinx revealed the following: to charge the private-sector group for using the • More than 90% of GTHA residents leave GO-owned Georgetown South rail corridor. As well, from and return to their home when travel- if the fare was to recover the capital cost of the pro- ling, so the added cost and inconvenience ject over time, we estimate this would approximate of getting to and from one of the three ARL $20 million annually over a period of 20 years. If stations with their luggage would prob- that amount is included as part of the ARL’s operat- ably discourage some residents from using ing cost, the total cost to be recovered from fares the ARL. each year would rise to about $50 million. • The ARL’s likely price point may also be a Metrolinx’s projection of annual ridership for concern. Although nearly 70% of potential the nine-month period of April 2015 to December riders currently using Union Station as an 2015 is 1.35 million (based on the assumption airport access or egress point indicated that that the one-way fare for riding the full distance they would probably use the ARL, nearly 3.09 would be $20), and its estimate for the full first 75% of those respondents who were GTHA year is 1.8 million riders. The agency expects that residents also indicated that they would Section ridership will increase by more than 65% to nearly not be willing to take the ARL at a cost of VFM 3 million by year 3, capturing 10.3% of the surface $22.50 or more. As well, 60% of visitors •

3 access market—primarily travellers using taxis and 90% of airport employees would not or those travelling by car who either park at the use the ARL at a cost of $22.50 or more. As

Chapter airport or are dropped off and/or picked up. If the would be expected, the percentages who aim was for the ARL to break even in its first year (a would not use the ARL increased as the goal that has not yet been decided on), Metrolinx proposed price increased. would have to charge about $28 for the full dis- Metrolinx advised us that it did take these fac- tance (based on current ridership projections and tors into consideration but still concluded that its estimated annual operating costs, including capital ridership projections at these premium fare levels amortization). Under the private-sector group’s would be achieved. proposal, the fare for a full one-way ARL trip was expected to be $27. If ARL ridership increases as RECOMMENDATION 1 projected by Metrolinx, the break-even fares over Metrolinx should work with the Ministry of the longer term would be lower. But the following Transportation to clearly define the business factors lead us to question whether ridership will model under which the Air Rail Link (ARL) actually grow as currently projected: should operate to ensure that the ARL will be Although the projected capture rate of 10.3% • a viable and sustainable operation. Given the is comparable to that of other North American importance of having a reliable estimate of airport rail services, these services differ signifi- projected ridership at the various possible fare cantly from the premium downtown-to-airport levels, Metrolinx should periodically update its rail service that Metrolinx anticipates offering. ridership forecast. Metrolinx—Regional Transportation Planning 213

responsibilities and business risks. A Crown Agency, METROLINX RESPONSE Infrastructure Ontario (I/O), oversees the delivery The Air Rail Link (ARL) will provide direct, of all AFP projects in the province. reliable express service connecting Canada’s Before deciding on the delivery model for a par- two busiest transportation hubs: Union Sta- ticular project, I/O assesses which delivery model tion and Pearson International Airport. It is a will provide the most value for money (VFM). This priority project of the Regional Transportation VFM assessment compares the total project costs of Plan (“The Big Move”) and is scheduled to open two different delivery models (that is, AFP versus a in 2015. traditional delivery method). Four basic categories Metrolinx agrees with the Auditor Gen- of cost make up the total project costs under each eral on the importance of reliable ridership delivery model: base project costs (for example, forecasts, and independent analysis has been construction costs), financing costs, the monetary obtained to create ridership projections. value of the risks that will be retained under each As the ARL launch approaches, a number of delivery model, and any ancillary costs (such as significant decisions need to be made. Metrolinx legal, project management or engineering advisory will continue to use best-in-class ridership infor- fees). Any positive difference between the AFP and mation to guide our internal decision-making the traditional delivery model represents the esti- and to inform our business model, and we will mated monetary benefit from using the AFP. continue working with the Ministry of Transpor- On the basis of a positive VFM assessment, I/O tation to finalize the business model. decided to use the AFP model in the delivery of the 3.09 As we would with any new service, Metro- “spur”; a $128.6 million contract was subsequently linx will closely monitor the ARL over its first awarded to a private-sector consortium. With Section years of operation and make adjustments based respect to the procurement of the AFP contractor, VFM on customer feedback we receive. we found that the process was competitive and fair •

3 to all respondents. In evaluating the VFM of procuring assets either

in the traditional manner or by way of the AFP Chapter The “Spur” Line model, it is often the value of the risks retained As noted earlier, the ARL requires the construction under each delivery model that tends to tip the of a new 3.3 km branch line, commonly referred scale in favour of the AFP model. The VFM assess- to as the “spur,” off of GO Transit’s Georgetown ment concluded that using the AFP model for South rail corridor connecting to a new passenger delivery of the “spur” would result in a net savings station in Pearson International Airport’s Terminal of about $20 million. While the total of the base 1. In July 2010, when Metrolinx became responsible project costs and ancillary costs under the AFP for ARL development, the government directed it approach was estimated to be about $22 million to evaluate options for the delivery of the “spur” higher, this was offset by an estimated $42 million line and any related station work, including pos- in savings related to the transfer of risks under sibly using the Public–Private Partnership (P3) the AFP model. As Figure 4 shows, the two largest model—which in Ontario is called the Alternative risks retained under the traditional delivery model Financing and Procurement (AFP) model. Gener- are construction risk (the cost associated with ally, AFPs are contractual agreements between the construction delays) and design and tender risk government and the private sector under which (the cost incurred because of omissions in the the private-sector businesses provide assets and original design and changes that are required after deliver services, and the various partners share the construction has started). These two risks account 214 2012 Annual Report of the Office of the Auditor General of Ontario

Figure 4: Valuation of the Retained Risks Source of data: Infrastructure Ontario

Traditional Delivery 1 AFP Delivery2 Retained Risks ($ 000) (%) ($ 000) (%) Construction 21,160 42 1,327 15 Design and tender 12,618 25 1,525 18 Policy/strategy 6,518 13 1,688 20 Site conditions/environmental 5,428 10 1,107 12 Project agreement 871 2 581 7 Permits and approvals 354 1 64 1 Other 3,722 7 2,310 27 Total 50,671 100 8,602 100

1. Under the traditional delivery model, the province bears all the risks. 2. Under the AFP delivery model, risks are shared between the province and the contractor.

for two-thirds of total risk retained under the trad- alternatives, which was the deciding factor in itional delivery model. going with the AFP approach. Of concern to us is the process used to assign • Because Metrolinx would be locked in very values to the various risks seen as being retained early on the specifications of the project

3.09 under the two delivery models. Specifically: under the AFP model, the additional cost that

• The values assigned to the risks seen as could be incurred as planning and design

Section retained under both delivery models were progress because of subsequent changes iden-

derived based on the judgment of I/O staff, tified and considered necessary could also VFM

•

Metrolinx staff and a consulting firm that have been considered in the allocation and 3 devised the probabilities and impacts associ- valuation of the risk retained under the AFP ated with the various risks. While we acknow- delivery model. Chapter ledge that I/O has significant experience in • I/O’s procedures allowed the consulting firm capital projects such as hospitals, courthouses that devised the project’s risk allocation matrix and other buildings, we saw no evidence that to later bid on a contract to provide engineer- the estimates of the risks of delivering the ing and technical advisory services to support “spur” under traditional procurement were the planning and procurement of the “spur” based on actual experience of similar, trad- line under the AFP delivery model. The con- itionally procured transportation projects. For tract was subsequently awarded to this firm. instance, over the past eight years GO Transit has completed a number of large and complex RECOMMENDATION 2 rail and grade-separation projects. The actual When assigning values to transferable risks in experience from these could have been used the evaluation of value for money between pro- to assess the reasonableness of the values curing assets by way of the traditional method assigned to the risks that are seen as being or by way of the Alternative Financing and Pro- retained under the traditional delivery model, curement (AFP) model, actual experience from especially given the significant $42 million recent traditional infrastructure procurements risk differential between the two procurement and AFPs should be thoroughly assessed. Metrolinx—Regional Transportation Planning 215

In 2002, the Ontario Ministry of Transporta- INFRASTRUCTURE tion, in conjunction with GO Transit and the GTHA ONTARIO RESPONSE municipalities, began researching the development The process for developing the value-for-money of a regional fare card, now called Presto. Presto, analysis was robust and based on current best which is now one of Metrolinx’s priority transit practices. An experienced transportation con- projects, allows transit riders to load amounts sulting firm created a standard transportation onto a reloadable plastic card (the size of a credit risk matrix based on the firm’s analysis of indus- card) and pay their fares by tapping the card on try data and on its own in-house experience. electronic card readers. Amounts ranging from $10 The matrix values were then further reviewed to $1,000 can be loaded onto the card online or in and revised by Metrolinx and Infrastructure person at participating customer service outlets. Ontario in consultation with the external advis- A number of guiding principles for Presto’s ers. Given the design and construction risks and development and implementation were identified the scheduling risk for this project, the Alterna- in 2002, including the following: tive Financing and Procurement (AFP) model • The fare system should, where possible, use was determined to deliver value for money off-the-shelf products whose components can compared to traditional delivery. be purchased from multiple sources. Infrastructure Ontario ensured that strict • The fare system needs to have the ability controls were in place to maintain objectivity to add new transit participants of any size of the firm conducting the value-for-money without major modifications to its core 3.09 analysis and the engineering advisory services. operational structure. For future projects contemplated under the Section Regional Transportation Plan, Infrastructure Project Cost VFM Ontario and Metrolinx will continue to use risk •

3 workshops to fully assess the actual experience In October 2006, the Ministry of Transportation of transportation projects procured under trad- signed a 10-year, $250-million contract with a

itional methods, including new developments in vendor to design, develop and operate Presto for Chapter procuring large transportation projects both in the GTHA. The $250 million is composed of about and outside of Canada. $150 million in capital development costs, $82 million in operating costs, and taxes of about $20 million. The province anticipated that all GTHA transit systems, including GO Transit, would use this card. PRESTO FARE SYSTEM To encourage this, the province has indicated that Currently, there are 10 public transit agencies in the transfer of gas-tax funding to municipalities the GTHA, each with its own fare structure and a would be contingent on their adopting and staying separate system for collecting fares. As a result, for with Presto. example, a person travelling from a local bus in However, when the agreement was signed the one GTHA municipality to the GO Train and then Toronto Transit Commission (TTC), which has over to the City of Toronto transit system must pay three 80% of the GTHA’s transit ridership, had not agreed different fares. One of the RTP’s key strategies is to to implement Presto on its system. Anticipating “implement a region-wide integrated transit fare that the TTC would eventually opt in, the agree- system by 2012 that allows users to pay a seam- ment with the vendor stipulated that the original less, integrated fare for all transit systems across base system would be built with the capability to the region.” expand to meet the needs of all Ontario transit 216 2012 Annual Report of the Office of the Auditor General of Ontario

providers without the need for significant modifica- after direct negotiation with the contractor. The tions to core systems. The agreement also stated Ministry and Metrolinx believe that developing that full rollout of the Presto base system was to be PNG on an open architecture framework, as had completed by October 2010. Although GO Transit been originally intended in 2002, will allow for started adopting Presto on its system in Novem- more procurement options should there be a need ber 2009, followed by the GTHA municipalities to add additional functionalities in the future. At in the 905 area code in May 2010, full rollout the time of our audit, PNG was initially expected to (meaning that Presto base was implemented on be rolled out in Ottawa in July 2012, but the rollout all intended transit systems, and the fare system was postponed to February 2013. Metrolinx was was functioning as planned) did not occur until also negotiating with the TTC to become a fully par- February 2012. ticipating PNG member in time for the 2015 Pan/ In 2007, the City of Ottawa approved the Parapan American games. implementation of the fare card on its transit As Figure 5 shows, PNG’s anticipated cost system based on an agreed functionality to be includes $498 million specifically for the system’s provided by Presto. In November 2009, the TTC development and $152 million to be paid to the also conditionally approved the adoption of the fare vendor for operating the system and running a card subject to the satisfactory resolution of some call centre until 2016, for an anticipated total of key issues (such as the system meeting the TTC’s $650 million. The total cost of developing Presto business needs and being affordable from both base and PNG could well reach $700 million. As of a capital and an operating perspective). Rather March 31, 2012, about $360 million of this amount 3.09 than expanding the Presto base system to meet the had been spent on system development costs, includ- requirements of Ottawa and Toronto, as had origin- ing about $40 million in internal charges incurred Section ally been planned, the Ministry of Transportation by the Presto office at Metrolinx, which has a staff VFM decided to develop a new system, Presto Next of about 60. Additionally, although one of the key •

3 Generation (PNG). reasons for the new PNG is to meet the TTC’s needs, The Presto base system, contrary to the guid- at the time of our audit the TTC had not yet formally

Chapter ing principles established for its development, signed on to using the fare card. The TTC indicated works on a closed proprietary model: that is, the to us that one of its main reasons for not yet signing contractor provides and controls the central system on was that Metrolinx and the TTC had not yet final- and other infrastructure for the fare-card operation. ized the TTC’s service-level requirements and how Changes can be made only through change orders the service levels will be achieved through PNG.

Figure 5: Estimated Presto Project Costs ($ million) Source of data: Metrolinx

Capital Operating Presto Base PNG Total Presto Base PNG Total Original contract (2006) 149.0 — 149.0 82.5 — 82.5 Nine separate requests to Metrolinx Board 54.0 154.0 208.0 19.0 — 19.0 (August 2009–February 2012) Additional request (April 2012)* — 344.0 344.0 — 152.0 152.0 Total 203.0 498.0 701.0 101.5 152.0 253.5

* At the time of our audit, the Board had approved only $48.5 million of this additional amount and had asked Metrolinx management to carry out further due diligence on whether value for money is being received with respect to this expenditure. Metrolinx—Regional Transportation Planning 217

the 905 area code, has not in itself facilitated the RECOMMENDATION 3 integration of fares across GTHA transit systems. Metrolinx should ensure that it formally consid- Currently, it is being used only as an “e-purse” ers the risks of continuing with the development that allows users to tap the card to the Presto card of Presto Next Generation (PNG), given that the reader and automatically be billed the individual specific business requirements of the Toronto fares of the participating GTHA transit systems. Transit Commission (TTC) for using PNG on its GTHA municipalities and transit systems indi- transit system and the costs for which the TTC cated to us that as long as transit funding remains would be responsible have not yet been formally a municipal responsibility, fare integration will be agreed to. difficult to achieve, because GTHA municipalities are not willing to absorb the cost of the subsidies METROLINX RESPONSE that an integrated fare system may entail. For example, the fare arrangement between GO Transit Metrolinx has been working with the TTC since and transit systems in the 905 area code costs GO 2009 to define the business requirements of Transit approximately $7 million annually, because Presto. The TTC approved the implementation GO Transit riders pay a reduced local transit fare to of Presto on November 23, 2011, and authorized encourage these riders to use local transit instead the execution of all necessary agreements. This of cars to arrive at their respective GO stations, was reaffirmed on May 1, 2012, and we antici- with GO Transit paying the difference to the pate completing a master agreement with the respective municipalities.

TTC in 2012. The discussions regarding the mas- 3.09

ter agreement have included both the operating

requirements and financial arrangements; thus, Section

Presto Usage these discussions have informed the develop- VFM As Figure 6 shows, at the time of our audit, the •

ment of Presto Next Generation. As well, Presto 3 Presto card was accepted within the GTHA as has been installed at 14 of the TTC’s highest- follows: on seven of the eight municipal transit volume subway stations and it is used daily by

systems within the 905 area code, as well as on Chapter more than 8,000 riders. It should be also noted GO Transit (both rail and bus) and at 14 of the 69 that the TTC has estimated that when Presto is TTC subway stations (but not on any TTC buses or fully operational, costs for fare collection could streetcars). As of March 31, 2012, despite the sub- be reduced by up to $10 million annually from stantial investment in the Presto base system and current levels. despite Metrolinx being six years into a 10-year contract for the system’s development and operation, Presto’s overall usage within participating GTHA Fare Integration transit systems was only about 18%. Overall Presto usage was even lower—only As noted earlier, a key success factor of the RTP was 6%—within the participating GTHA transit agen- to develop a seamless and integrated fare for all cies in the 905 area code. Several of these agencies transit systems across the GTHA that would allow indicated to us that a good portion of their rider- riders to cross regional and municipal boundaries ship (nearly a third in some regions and municipal- using different transit systems by paying just one ities) are considered low-income and either cannot fare rather than having to pay an individual fare for afford to load the minimum $10 currently required every system travelled on. The Presto base system, by the Presto card or do not have bank accounts or apart from facilitating fare arrangements between credit cards and therefore cannot load the cards GO Transit and bus systems in municipalities within 218 2012 Annual Report of the Office of the Auditor General of Ontario

Figure 6: Presto Usage Rate for March 2012 Source of data: Metrolinx and GTHA transit systems

System Presto Presto Transit Systems In-service Date Ridership Ridership Ridership (%) GO Transit System Rail Aug. 8, 2011 4,169,337 1,788,037 43 Bus Sept. 12, 2011 1,506,716 242,335 16 GO Transit System Subtotal 5,676,053 2,030,372 36 TTC (at select subway stations) Jan. 1, 2011 — 252,025 — 905 Transit Systems Mississauga Apr. 4, 2011 3,315,817 140,655 4 Brampton Apr. 4, 2011 1,593,637 231,770 15 York* Apr. 4, 2011 1,755,264 64,843 4 Hamilton Apr. 4, 2011 1,969,218 67,258 3 Durham Apr. 4, 2011 964,168 31,669 3 Burlington May 10, 2010 204,729 35,179 17 Oakville May 10, 2010 258,310 41,614 16 905 Transit Systems Subtotal 10,061,143 612,988 6 Overall Total 15,737,196 2,895,385 18

3.09 * Because of a transit strike and its effect on ridership for March 2012, we used April 2012 data for York Region.

online. The initial $6 charge for the card also acts as rent low demand and the nearly $5,000 cost Section a disincentive for these riders to migrate to Presto. of installing a single terminal, very few are VFM For these reasons, GTHA transit systems within the being installed. •

3 905 area code may need to maintain some form of • One region contracts out routes used by about disposable fare media (defined as media that are 30% of its ridership to the TTC, but the TTC

Chapter good for only a short term—either a single trip or currently does not accept the Presto card on multiple trips over the course of a day—such as its buses. tickets, tokens or day passes). These agencies cited • GTHA municipalities within the 905 area code the following additional reasons why they cannot that have a significant population of university completely eliminate their existing fare systems and students and offer students a special transit force their ridership to migrate to Presto: pass for the university term under their own • Not all Presto-related transactions can be fare system cannot currently do so on Presto. done online. For instance, to load monthly These transit agencies also raised concerns passes or if a student or senior wants to about the quality of the Presto equipment installed register for a card, the rider must physically on their vehicles and the repair costs for what are go to a Presto location. But many municipal deemed “out-of-warranty” damages. Because the transit systems have only one location where equipment is proprietary, if “out-of-warranty” in-person Presto transactions can be carried repairs are needed, municipal transit agencies out. GTHA municipalities and transit systems can turn to only one approved supplier under the within the 905 area code indicated to us that existing contract. The contract does not provide point-of-sale terminals installed in such loca- specific pricing for the different types of repairs. tions as convenience stores would provide In our discussions, municipal transit agencies riders with greater access, but given the cur- cited examples of quotes for repairs that they had Metrolinx—Regional Transportation Planning 219 received from the contractor that they felt were For instance, Metrolinx is working closely with unreasonably high. Brampton Transit to retire that municipality’s Lastly, the transit agencies indicated to us that current payment systems in 2013. In Durham, the system currently lacks back-end support for we are also working to convert student riders reporting and financial reconciliation of trans- to Presto in 2013. Similar strategies are being actions. To obtain ridership information, many developed for each municipality in the GTHA. transit agencies have had to design their own On GO Transit, Presto usage will increase programs for extracting information from a data further in 2013 with the retirement of the GO dump provided by Presto. The transit agencies monthly pass. also indicated that they have little assurance that the system is capturing all riders who use their respective systems. Project Procurement RECOMMENDATION 4 As noted earlier, in October 2006 the Ministry of To ensure that the Presto base system and Transportation signed a 10-year, $250 million con- the Presto Next Generation system meet the tract with a vendor to design, develop and operate objective of facilitating a seamless, integrated the Presto base system. This contract was procured fare for all transit systems across the GTHA, through a competitive process and subjected to a Metrolinx should: fairness review that concluded that the process

work with the provincial government and was conducted in a procedurally fair, open and 3.09 • GTHA municipalities to resolve the issue of transparent manner. However, with respect to the

subsidizing fare integration so that progress development of the PNG system, Metrolinx was Section

can be made on implementing an integrated unable to provide evidence supporting its 2009 VFM

•

fare system; and decision to develop this system through change 3 • work with GTHA municipalities and regions orders to the existing Presto contract rather than to resolve outstanding issues related to through a competitive tender. As noted in Figure 5, the operation of Presto that inhibit riders’ earlier, at the time of our audit, Metrolinx had Chapter use of the fare card within their respective Board approval to spend an additional $227 million transit systems. ($208 million capital plus $19 million operating); of this amount, $154 million was for PNG. In April METROLINX RESPONSE 2012, Metrolinx went to the Board for approval of an additional $496 million ($344 million capital The Presto system is a foundational step toward plus $152 million operating) for PNG. developing an integrated fare system across the After deciding to develop PNG using change GTHA. The system is currently being deployed orders, Metrolinx hired Ontario’s former Integrity across the GTHA and in Ottawa. A staged Commissioner to review the appropriateness of this deployment provides the least amount of risk as decision. In September 2011, Metrolinx also hired the system is implemented. a consulting firm to assess this additional invest- As the Presto system grows, Metrolinx will ment in PNG and to assess whether value for money continue to work with the province and with the (VFM) would be achieved. The commissioner’s municipalities involved to develop strategies for February 2012 letter concluded that there was no increasing customer usage as well as to enhance compelling reason to restart the procurement pro- the level of integration, up to and including cess on PNG if the results of the VFM review were the development of a common fare structure. positive. The VFM review compared the per capita 220 2012 Annual Report of the Office of the Auditor General of Ontario

cost of fare-card systems around the world with the funding for the purchase of new streetcars and the per capita cost of the Presto base system and PNG rapid transit system on Eglinton Avenue would be after considering the capital portion of the $227 mil- contingent on the TTC signing on to Presto. lion in the first set of change orders and concluded As noted earlier, at the time of our audit, a positive VFM on the basis that Presto’s per capita Metrolinx was unable to provide evidence that cost ranked in the middle. However, if the capital it had explored alternative procurement options portion of the additional $496 million expected to at the time the decision was made to develop be incurred had been included in the analysis, Presto PNG. We questioned whether tendering the base and PNG combined would turn out to be one of new system’s development would have, at the the more expensive fare-card systems in the world. very least, informed Metrolinx of the range of In April 2012, citing concerns about the request options and what a reasonable cost would be for for an additional $496 million in spending authority developing PNG. for PNG, the Board asked Metrolinx’s management to carry out further due diligence on PNG. In RECOMMENDATION 5 response, Metrolinx asked the same consulting firm To ensure that Metrolinx complies with the for a second VFM review of PNG. This second review intent of the government’s policy of open, concluded that, although reprocurement “may drive competitive procurement, all value-for-money pricing benefits,” it incurred a significant risk of considerations and an appropriate business-case not meeting the timelines for the development of a justification should be completed and approved new fare card, because procuring, developing and

3.09 by Metrolinx’s Board and the Ministry of Trans- implementing a new system would take more than portation before any decision on the procure- 24 to 48 months. Furthermore, having a new vendor

Section ment of significant transportation projects is build the system would introduce significant other finalized, especially if retendering the projects is VFM project and operational risks that could hinder the •

not considered to be a viable option. 3 efficient delivery of the entire system. We note with respect to these concerns about METROLINX RESPONSE

Chapter reprocurement that the existing plan for developing and implementing PNG, having begun in 2009, A comprehensive review of Presto Next Gen- will take about four years anyway because rollout eration (PNG) technology was undertaken by of the system in the City of Ottawa is not expected independent advisers, who confirmed that the until February 2013. We also note that there may development of PNG was fair and created value have been value in considering the possibility of for money (VFM). The VFM analysis identified alternative procurement options. In this regard, for concerns with retendering the work, including example, the TTC had an agreement in principle increased project costs and risks associated with a company in 2011 to develop an open-fare with the introduction of new vendors, increased payment system. Under the terms of the agreement, project timelines and the loss of efficiency. As the TTC would not have had to pay any capital we move forward, we are reducing the role of costs up front. Instead, the vendor was willing to the contractor, increasing the amount of work to take a percentage of the revenues collected by the be procured in separate competitive processes open-fare system. In its proposal to the TTC, the by about $200 million. As the technology vendor had also agreed to make its system compat- continues to advance, more functionality and ible with the existing Presto base system. The TTC conveniences will be added for Presto custom- abandoned this option after the province confirmed ers, including additional services and other that provincial gas-tax funding and provincial payment methods. Metrolinx—Regional Transportation Planning 221

in the system’s original development. In the case Metrolinx complies with all provincial of the above example, the documentation was not requirements for an open, competitive procure- clear enough to determine whether the payment ment process, and has ensured that VFM con- related to enhancements to the system or to the siderations as well as appropriate business-case correction of defects, which should have been done justification are part of the decision-making pro- at no cost. cess. However, Metrolinx does agree with the GTHA transit systems in the 905 area code Auditor General’s recommendation that VFM that we met with indicated that changes to the considerations and an appropriate business-case Presto base system often seemed too costly and justification should be completed and approved that change requests were not always completed before making any decision on a project’s pro- on what they felt was a reasonably timely basis. curement strategy. Metrolinx has implemented The consulting firm mentioned previously that was this recommendation with the expansion of hired to assess PNG noted in its December 2011 Presto to the TTC. report that Metrolinx’s change-order process lacks With respect to Metrolinx’s $700 million detailed cost breakdowns and pricing methodolo- investment in Presto, approximately $275 mil- gies and that no formal budget estimates had been lion is expected to be recovered from the TTC, prepared for requested changes that could be used OC Transpo and the GTHA transit agencies in to assess the reasonableness of the amount being the 905 area code for assets specifically used in billed for each change order. the provision of service to their customers. 3.09 RECOMMENDATION 6

In order to effectively manage the cost of change Section Change-order Management orders related to the Presto base and Presto Next VFM

•

The $250 million contract for the original Presto Generation systems, Metrolinx should: 3 base system is a 10-year fixed-price contract to • implement a process that distinguishes deliver an electronic fare-card system for GO between change orders that amend the Transit and the seven participating GTHA muni- systems from their original specifications in Chapter cipalities in the 905 area code. However, since the contract and those that correct identified the contract’s execution in 2006, a total of 330 defects in the systems’ original develop- change requests, adding $146 million to total costs, ment, and allow the contractor to charge have been made under the contract. Of these, 281 for only those change orders that pertain to change orders totalling $45 million relate to fixes or requested changes or enhancements to the enhancements to the Presto base system that were original design specifications; and requested by either Metrolinx or the participating • prepare internal cost estimates for each transit agencies, with the balance relating to PNG. change order to enable the reasonableness of For example, in one case, the contractor charged the amount charged by the contractor to be $7 million to make nine enhancements to the Presto knowledgeably assessed. base system and at the same time fix 40 defects that had been identified in the system’s original METROLINX RESPONSE development. It is reasonable for a contractor to Metrolinx agrees with the Auditor General on charge for change orders that enhance or alter the the importance of effectively managing the cost system from its original agreed-upon design speci- of change orders. fications, but the contractor should not be charging for change orders that correct identified defects 222 2012 Annual Report of the Office of the Auditor General of Ontario

it was finalizing the negotiation of a letter of A consulting firm recently completed a intent to secure ownership in Canada of intel- review of the change-order process. Although lectual property relating to these systems. this process was found to be efficient and effect- The contract for the Presto base system ive, opportunities for improvement were identi- • contains 22 measures designed to gauge the fied. Metrolinx has now implemented controls contractor’s performance in such areas as over the change-order process, including gener- system availability, customer management ating internal cost estimates before completing and the management of the Presto devices negotiations on change orders and increasing (for example, the card reader). In 2011, the documentation of each change order’s purpose contractor failed to meet the set standard and scope. in nearly a third of the measures. However, Metrolinx is in the process of implementing Metrolinx did not seek any of the remedies additional accountability measures, which will stipulated in the contract for these failures be independently reviewed to ensure that these and indicated to us that it will just continue additional measures have been implemented to monitor performance until system usage and to identify whether any further improve- reaches maturity. The contract also contains ments are necessary. reliability measures for the equipment With regard to the $7 million charges noted used by the Presto base system, but neither by the Auditor General, Metrolinx has reviewed the contractor nor Metrolinx tracks this these charges and has confirmed that they were information. Therefore, Metrolinx cannot

3.09 related to system enhancements rather than to determine whether the equipment is meet- the correction of defects. ing the reliability measures, which is all the Section more important because some municipalities VFM we talked to expressed concerns about •

3 Other Presto Issues equipment reliability. During the period May 1, 2007, to April 30, We noted several additional issues in our review •

Chapter 2012, independent contractors were used of the development of the Presto base and in senior positions with signing authority PNG systems: to supervise other consultants. The amount Ownership of certain key components of the • paid to these contractors was $4.2 million. Presto base and PNG systems is currently But the government’s procurement directive unclear. The contractor maintains that it owns specifies that “consultants must not perform the system and can therefore market it to other functions normally assumed by management, parties. Metrolinx has asked the contractor including supervising and hiring staff and to pay $25 million for the right to market the other consultants.” At the time of our audit, Presto base and PNG systems anywhere in the Metrolinx was in the process of terminating world (including to non-government entities in these relationships. Canada), while Metrolinx can market the systems only to government entities in Canada. RECOMMENDATION 7 If the ownership and marketing rights of the system are not resolved, Metrolinx risks losing To ensure that the Presto base and Presto Next key components of the Presto base and PNG Generation systems remain available for use systems at contract termination, which would after the end of the existing contract, Metrolinx render the rest of the systems inoperable. needs to finalize its current negotiations with Metrolinx advised us in mid-October 2012 that Metrolinx—Regional Transportation Planning 223

We reviewed these two projects and made the the contractor to ensure that it secures owner- following observations. ship of these two systems. If the contractor fails to meet the performance standards stipulated in the contract, Metrolinx should have a valid justi- Train Shed Restoration fication for not applying the available remedies Under the federal Heritage Railway Stations Protec- and penalties set out in the contract. tion Act, Parks Canada must approve all rehabilitation work planned for Union Station, including its METROLINX RESPONSE train shed (the structure that shelters the station’s Metrolinx is in advanced negotiations with the platforms and the tracks alongside them). For contractor to safeguard ownership rights of the instance, between 2005 and 2010, Parks Canada intellectual property created and expects these approved the replacement of approximately 20% negotiations to be successfully concluded in of the centre portion of the train shed roof with October 2012. The expected agreement provides a glass atrium. Approval was also received to for the use of the current and future system replace the existing roof with an eco-friendly green in perpetuity. roof on either side of the atrium. Although the As the system has matured, Metrolinx has initial estimate for the cost of restoring the train been engaged in an extensive internal review shed—including design, administration and con- of contractor performance and has developed a struction—was $215 million, when the bids for the plan that provides for a more rigorous monitor- work came in, the lowest bids totalled $242 million. 3.09 ing of key service measures, as well as appropri- The majority of the difference was in the construc- ate remedies and penalties for situations where tion work, for which the initial estimate had been Section these measures are not met. $165 million, but the lowest bid was $196 million. VFM The project was initially scheduled to be com- •

3 pleted in November 2014, but delays experienced during the design and construction phases could UNION STATION REVITALIZATION

push completion to December 2016. We noted that Chapter Union Station, federally designated as a National the construction contract contained no incentives Historic Site and a Heritage Railway Station, is for on-time completion or liquidating damages in Canada’s busiest rail transportation facility, serv- the case of delays. Metrolinx chose not to include ing more than 250,000 passengers daily who use such clauses because it felt that the contractor the services of the TTC, GO Transit, Via Rail and did not have full control of the site (that is, Union Amtrak. In August 2000, GO Transit and the City of Station would be fully functioning during construc- Toronto bought the facility from Toronto Terminals tion, and no more than two tracks or platforms Railway Company. Specifically, GO Transit bought would be taken out of service at any time). the three-mile rail corridor leading in to the sta- In the construction contract for the restoration, the platforms and the train shed, and Toronto tion of the train shed, Metrolinx retained, among bought the heritage building, including the GO other risks, the risk associated with concealed Transit concourse area. or unknown conditions that may arise during Revitalizing Union Station is one of the priorities construction. It was felt that if such risks were in the RTP’s first 15 years: it consists of a series transferred to the contractor, either contractors of projects. Two of the more significant projects would not bid on the work or the premium charged include restoring the train shed and replacing to cover unforeseen risks would be cost-prohibitive. switches in the Union Station Rail Corridor (USRC). Metrolinx included a $15 million contingency in the 224 2012 Annual Report of the Office of the Auditor General of Ontario

contract’s original price (8% of its total value) for weak monitoring of the project or a combination any unforeseen situations. Metrolinx then approved of these problems. In 2011, the province’s Internal an increase in this contingency to $36 million (18% Audit Division reviewed Metrolinx’s budgeting of the contract’s total value) by not requiring the and forecasting process and found that the capital contractor to carry out work originally stipulated budgeting and forecasting processes were not well in the contract totalling $21 million and moving established, and also that recent years’ budget-to- this amount to the contingency. As of May 2012, actual results suggest that Metrolinx may need to $30 million of this contingency had been allocated re-evaluate how project costs and/or contingencies to the contractor, leaving approximately $6 million. are determined. In April 2011, Metrolinx had also received Board approval to increase the construction contract’s Switch Replacement Project price by an additional $30 million by way of change orders. As of August 2012, change orders totalling Since June 2000, GO Transit (a division of Metro- $18.1 million had been issued to the contractor. linx) has had a single-sourced agreement with Most of this amount was requested by the con- Toronto Terminals Railway Company (TTR)— tractor to cover any additional overheads caused by which previously owned the Union Station Rail extending the project completion date by another Corridor (USRC)—to conduct routine USRC two years. The cost of restoring the train shed maintenance and rail traffic control services for an could now reach $270 million—25% more than annual fee. In June 2006, a new agreement with Metrolinx’s initial estimate. We note that nothing in TTR for a further six years, at approximately $7 mil- 3.09 the current agreement prevents the contractor from lion annually, was entered into. Under this agree- coming to Metrolinx for even more funds over the ment, in 2006, GO Transit issued a purchase order Section remaining term of the contract. totalling nearly $38 million to replace about 100 VFM Metrolinx informed us that as a hedge against switches within the USRC over a six-year period. •

3 unforeseen situations, construction contracts According to GO Transit, the new switches allow commonly provide for contingencies of 5% to 15% trains to run faster into and out of Union Station,

Chapter of the contract’s original value. This contract’s thereby providing additional train capacity and contingencies and the change orders totalled nearly more efficient train operation. As Figure 7 shows, $55 million, or 28% of the construction contract’s beyond the initial 2006 purchase order issued for original price—almost twice the high end of the switch replacement, three additional purchase norm. Significant price changes in contracts can orders totalling over $50 million were also issued occur because of poor planning, inadequate pro- (in 2008, 2010 and 2011, respectively). Metrolinx cesses for estimating the initial cost projections, advised us that the significant cost increases

Figure 7: Approved Purchase Orders for Switch Replacement Project ($ million) Source of data: Metrolinx

PO # Date Description Amount 1 June 9, 2006 Removal and installation of approximately 100 new switches 37. 6 2 Nov. 5, 2008 Cost increases for changes in regulations and standards, staff training and material costs 14.0 3 July 20, 2010 Cost increases for delays due to testing and need for redesign and staff costs 15.0 4 Mar. 14, 2011 Cost increases associated with the more complicated switches to be installed in 2011 and 2012 23.0 Total 89.6 Metrolinx—Regional Transportation Planning 225 resulted primarily from changes in regulations METROLINX RESPONSE and safety standards, delays due to the switches being replaced because of their age, and the instal- Union Station is Canada’s busiest passenger lation of the more complicated switches later in transportation hub. Balancing operations and the project. As of May 2012, TTR had installed the safety of the 250,000 passengers who rely approximately 90 switches at a total cost to date of on it daily while renovating a National Historic $76.1 million—more than twice the total amount of Site’s structure provides unique challenges. the original purchase order. For example, when renovating an 82-year-old, This project was managed by an external con- 8.6-acre train shed roof, it is difficult to antici- sulting firm under a contract that expired in 2010. pate all structural issues. Co-ordination with Although the firm handed over information on the the federal government and with the City of project to Metrolinx before ceasing to work on the Toronto, who are developing new concourses project, the information was not well organized, below the train shed, was also challenging. making the search for details on this project very When determining procurement options, difficult and time-consuming. As a result, other Metrolinx assesses the potential for risk transfer, than requests forwarded to Metrolinx’s Board whether the contractor is in a better position to for funds to pay for cost increases related to the manage risks, and the potential cost premium project, no other documentation was available to for that risk transfer. Metrolinx balances these support the reasonableness of such significant cost factors to determine the most appropriate increases. TTR also works on other projects within procurement option under the circumstances, 3.09 the USRC for Metrolinx, and we noted that the recognizing that no contract type is right for all contractor’s invoices did not always clearly specify projects. At the time of procuring the contract Section which project the work relates to, making monitor- for the restoration of the train shed, GO Transit VFM ing of project costs very difficult. determined that the best way to address the •

3 In our 2007 report on GO Transit, we expressed significant risks associated with this project concern over the fact that for work in the USRC, GO was to use a modified stipulated-price contract.

had not actively sought other qualified suppliers or GO Transit subsequently engaged an independ- Chapter considered the feasibility of developing in-house ent fairness officer to review the process and expertise to prevent becoming overly dependent on consider the change orders involved. The review the USRC’s previous owner. We continue to have confirmed that the process was fair. this concern. With regard to the Union Station Rail Corri- dor, Metrolinx continues to take additional steps RECOMMENDATION 8 to reduce its future reliance on existing suppliers, including obtaining in-house expertise to To ensure that projects under the Regional Trans- carry out similar work in the future. Metrolinx portation Plan are delivered cost-effectively and will apply a different model upon the comple- on time, Metrolinx should ensure that contracts tion of the Union Station revitalization, which is have firm ceiling prices, whenever possible. expected in 2016. Contracts should then be monitored for adher- Double slip switches are a complex section of ence to the original ceiling price. For work in the rigid and movable railway tracks more than 50 Union Station Rail Corridor, Metrolinx should metres long and weighing more than 55 tonnes. also consider seeking other qualified suppliers or They are very rare in the railway industry, obtaining in-house expertise. and the switches in use at Union Station today date back to the 1920s. The renewal program 226 2012 Annual Report of the Office of the Auditor General of Ontario

the right course of action for these projects. Spe- includes upgrading tracks and switches to cifically, when the RTP was adopted, the City of increase their reliability and allow for faster Toronto’s “Transit City” plan—a plan for developing train speeds, but replacing this type of switch public transportation in the city—included light- while continuing to operate GO trains was a rail transit (LRT) projects on three major arteries difficult process. With changed construction within the city: Sheppard Avenue East, Finch assumptions and the added complexity brought Avenue West and Eglinton Avenue. These three on by extensive signal-testing requirements, projects were adopted into the RTP, and Metrolinx Metrolinx found that switch replacement was prepared benefits case analyses (BCAs) to evaluate going to be more time-consuming and expensive the costs and benefits of all reasonable alterna- than we had first estimated. tives so that the best version of each project could be built, taking cost and service to riders into consideration. Metrolinx’s analyses concluded that REGIONAL TRANSPORTATION PLAN the most cost-effective strategy was a mix of light rail with traffic on two of the lines (Sheppard and Although those GTHA municipalities and transit Finch) and a fully grade-separated rail system on agencies we talked to questioned the priority given Eglinton Avenue. However, the Eglinton project to some of the RTP’s projects, they generally sup- was approved only as a partially grade-separated ported the plan as currently conceived. One transit project, because there wasn’t enough provincial agency indicated that the plan focuses only on funding for a fully grade-separated system.

3.09 new projects and that perhaps some consideration In March 2011, soon after the election of a new should have been given in the plan to maintaining mayor, the province and Metrolinx entered into

Section existing transit assets. a Memorandum of Understanding (MOU) with VFM the new mayor to revise these projects within the •

3 Role of Metrolinx RTP. The MOU called for a subway on Sheppard Avenue—a decision that would have resulted in As noted earlier, one of Metrolinx’s key objectives

Chapter sunk costs of $65 million—as well as bus rapid is to provide leadership in the co-ordination, plan- transit (BRT) on Finch Avenue and a fully grade- ning, financing and development of an integrated, separated LRT system on Eglinton Avenue. However, multi-modal transportation network within the Metrolinx’s analysis concluded that, for Sheppard GTHA. In order to effectively carry out this man- Avenue, the ridership projections did not warrant date, Metrolinx’s decisions regarding transportation using the higher-cost subway technology along and transit planning must be made on the basis the entire corridor. Similarly, forecast ridership for of a credible business case supported by objective Finch Avenue was found to be too large to be effect- and sound data. As well, some of the municipal ively served by BRT. But for Eglinton Avenue, the stakeholders we spoke to said that it is important decision reflected in the MOU supported the BCA’s for Metrolinx to remain objective and independent conclusion that transit/auto conflicts along any of any decisions of a political nature made by the at-grade sections would affect service reliability on governments of the municipalities within the GTHA the entire line, so that a fully grade-separated LRT and by the federal and provincial governments. system would serve the area best. In the recent debate over the City of Toronto’s Before the MOU was signed, the Sheppard transit projects within the RTP, Metrolinx could Avenue LRT was considered a top priority, and have been perceived as not being a strong enough construction had already begun. But after the MOU advocate of what its own analysis suggested was was signed, the majority of the provincial funding Metrolinx—Regional Transportation Planning 227 would now be taken up by the fully grade-separated Plan Funding transit line on Eglinton Avenue, so the City of Without long-term sustainable funding, the RTP as Toronto became responsible for funding the Shep- currently contemplated cannot be implemented. pard Avenue and Finch Avenue lines. Because the The RTP’s $50 billion cost estimate may well prove City of Toronto did not have sufficient funds at the low, because it is a high-level estimate derived for time to construct a subway on Sheppard Avenue, the most part using average costs per kilometre the City initially decided to cease work on the Shep- to construct various transit technologies based on pard Avenue line. The Sheppard Avenue project, Canadian and international historical data. The rec- previously considered a top priority, would now ord of cost overruns to date on the priority projects be delayed. we examined also suggests that fully implementing In February 2012, however, Toronto’s City the RTP will cost more than estimated. Council rejected the revised plans under the MOU When the RTP was adopted, the detailed plan- for Sheppard Avenue, Finch Avenue and Eglinton ning and design work that would yield a more Avenue and directed the City Manager to work with precise cost estimate had understandably not yet Metrolinx on developing these projects as previ- begun for the majority of the projects contemplated ously planned—that is, before the MOU was signed. under the plan. Detailed planning and design for On April 25, 2012, Metrolinx formally accepted this proposed transit projects can take at least two decision, which will result in approximately $4 mil- to four years before any construction can begin. lion in sunk costs. Although planning and design work is necessary Some GTHA municipalities and transit agencies

before decision makers can be advised on project 3.09 that we talked to used the debate over the City of costs and schedule, the costs of doing this necessary Toronto transit projects as an example to question

upfront work may not be fully realized if the project Section Metrolinx’s ability to objectively act as the GTHA’s is subsequently shelved for a long time due to lack VFM central transit planning authority to ensure that the •

of funding. 3 most cost-effective and value-added transit infra- Funding has been committed for more than half structure decisions are being undertaken. of the priority transit projects within the RTP’s first 15 years. By June 1, 2013, Metrolinx must report Chapter RECOMMENDATION 9 back to the province on an investment strategy to Metrolinx should ensure that all projects con- fund the remaining projects within the RTP’s first templated under the Regional Transportation 15 years, as well as the projects contemplated in Plan are subjected to a rigorous cost/benefit years 16 through 25 of the RTP. Some examples analysis that considers financial, economic, of revenue tools that Metrolinx is contemplating environmental and social needs and impacts using to raise funds for implementing the balance and that transit infrastructure investment deci- of the RTP include an increase in sales and payroll sions are made on the basis of that analysis. taxes specifically for GTHA residents, land value capture for GTHA residents whose property value METROLINX RESPONSE has increased or will increase when new transit is introduced, and a transit fare surcharge. To this Metrolinx has completed a cost/benefit analysis end, Metrolinx informed us that one of the key on all of the projects included in the first stage issues it faces in formulating a funding strategy is of the Regional Transportation Plan (“The Big identifying the beneficiaries within the region of Move”). This approach will also be applied the investment in public transit. when considering future projects. 228 2012 Annual Report of the Office of the Auditor General of Ontario

Some GTHA municipalities indicated to us In our discussions with GTHA municipalities, that Metrolinx has not consulted with them on some indicated that Metrolinx should more the options being considered. They felt that being regularly update their respective councils on the consulted while the strategy is being formulated— RTP’s overall status, including the status of initia- specifically on options that will affect local residents tives contemplated under the RTP that are not yet and businesses (for example, payroll and sales taxes funded. These updates would help municipalities specific to GTHA residents and businesses that could to better prioritize local projects. For instance, reduce GTHA competitiveness and job creation)— one municipality indicated that the impact of would be beneficial in encouraging timely adoption the uncertainty of funding on the timing of local of the investment strategy. Better co-ordination by projects under the RTP was making it difficult for the province and municipalities in formulating strat- the municipality to co-ordinate certain of its public egies for raising funds would also help avoid dupli- works infrastructure projects. cation. For instance, the City of Toronto’s January Metrolinx plans to release an update to the RTP 2012 report “Sheppard Subway Development and in June 2013. This update will be an addendum Financing Study” identified a number of revenue to the current plan, and will reflect any new infor- tools that may be available to the city to finance its mation that has come to light since the plan was proposed transit expansion. The options anticipated adopted. However, no changes are expected to the were similar to those contemplated by Metrolinx to RTP’s current vision, goals and objectives, policies fund the RTP. Metrolinx advised us that it expected and actions. to start the public consultation phase of developing 3.09 the funding strategy in fall 2012. RECOMMENDATION 10 Metrolinx has developed a project prioritization

Section To ensure that provincial, regional and munici- framework for ranking unfunded priority projects pal stakeholders are kept up to date on the fund- VFM using such criteria as the project’s contribution to •

ing requirements and progress of the Regional 3 quality of life, environmental health and economic Transportation Plan (RTP), Metrolinx should: prosperity. This framework was not used for regularly consult with GTHA municipalities

Chapter already funded projects, because those projects • and other key stakeholders as the funding were approved under previous funding agreements strategies are being formulated, especially and subsequently adopted into the RTP. on options that affect local residents; and • have clearly defined targets for the Plan Progress Reporting RTP’s more significant projects and regularly report on costs and progress Apart from the timelines covering the first 15 years toward completion. and years 16 through 25, Metrolinx has no other defined targets for the overall achievement of the METROLINX RESPONSE RTP. Although there is an urgency to complete certain funded projects—such as the Air Rail Link Metrolinx agrees on the importance of regu- and projects associated with the revitalization of lar input and consultation on the Regional Union Station—in time for the summer 2015 start Transportation Plan (“The Big Move”). Since of the Pan/Parapan American Games, the remain- the plan was launched, Metrolinx has been ing projects have no clearly defined timelines for regularly engaging municipal officials and key completion. As noted earlier, funding dictates the stakeholders on the RTP and related initiatives. completion of these projects for the most part. Recently, Metrolinx has increased its engagement with municipal officials (such as chief Metrolinx—Regional Transportation Planning 229

budget to actual results. But to facilitate time- accounting officers and treasurers) as well as lier project monitoring, users must be able transit managers. to make such comparisons for shorter terms We have also sought input from municipal- (that is, monthly or quarterly). ities’ chief planning officials through regular In addition, information on disbursements meetings as we move forward with our work on related to various projects is downloaded to the sys- the investment strategy. tem from Metrolinx’s financial system each night. Metrolinx is committed to working with our These disbursements are supposed to match the partners and stakeholders and will continue to commitments set up in the system for the respective engage in regular dialogue with municipal and projects. However, we noted that the system was transit-agency representatives. not properly distributing all costs incurred to the appropriate commitment. This impacts the project managers’ ability to reliably compare the projects’ OTHER MATTER actual costs to their respective budgets.

Project Management Information System RECOMMENDATION 11 In 2008, Metrolinx purchased a program manage- Metrolinx should ensure that its project man- ment system to plan, record and monitor capital agement information system provides the projects and report project information. The functionality needed to facilitate the effective system downloads information from Metrolinx’s

monitoring of individual projects. 3.09 procurement and financial systems and also relies on input from project managers for budget and

METROLINX RESPONSE Section actual information. VFM We noted that in order to effectively monitor Metrolinx will continue to introduce project •

3 projects, project managers often supplemented the management tools and training to support its information provided by the system with manual project managers in ensuring that projects are

spreadsheets maintained outside the system. This completed on time and on budget. For instance, Chapter approach was necessary because the system did some of the anticipated and implemented not have adequate functionality in areas such as tools include: scheduling and forecasting. Specifically, the system • project and program dashboards to allow has the following limitations: project managers and senior management to • Limited scheduling capability. The system can- track project performance on key indicators; not support the scheduling of tasks on mul- • new functionality to be added to the existing tiple projects, especially if the projects are on project management information system to the same corridor. For example, we noted that allow for monthly forecasting with associ- the system was unable to provide sufficient ated dashboards and reports; and support for the scheduling of interrelated • more scheduled training for user access and tasks on multiple projects on the Georgetown integration with additional project manage- South rail corridor. To compensate, project ment systems, where there is value added, to managers had to maintain spreadsheets and have greater overall functionality in schedul- other scheduling tools outside the system to ing, cost control and risk management. effectively manage their projects; and • Limited forecasting capability. The system allows users to compare only the year-to-date Chapter 3 Ministry of Community Safety and Correctional Services Section 3.10 Ontario Provincial Police

Figure 1: OPP Full-time Equivalent Staff Strength as of Background May 31, 2012 Source of data: Ontario Provincial Police

The Ontario Provincial Police (OPP) operates under Police Civilian the Police Services Act (Act), and primarily provides: Officers Employees 3.10 • front-line police services in areas of Ontario Office of the Commissioner 10 35 that do not have their own police force; Corporate Services 138 242 Section • patrols on provincial highways, waterways Field Operations 4,435 695 VFM and trail systems; Investigations/Organized Crime 819 284 •

3 municipal policing services under contract Traffic Safety and Operational • 869 1,058 where requested by municipalities; Support Total 6,271 2,314 Chapter • emergency and other support services to all communities in the province; and • investigations into complex criminal cases and as to 19 First Nation communities. It also provides organized crime. support as requested to 53 municipal and nine The OPP is one of the largest police forces in Aboriginal police forces of varying sizes in Ontario. North America, with about 6,300 police officers The Commissioner of the OPP reports to and is and 2,300 civilian employees, as illustrated in accountable to the Minister of Community Safety Figure 1. It has 78 police stations, called detach- and Correctional Services. ments, located throughout the province, and the OPP operating expenditures totalled $979 detachments operate an additional 87 satellite million in the 2011/12 fiscal year, as detailed in stations. Individual detachments report to one of Figure 2, with staffing costs representing 87% of five regional headquarters or to the Highway Safety expenditures. Municipalities receiving OPP policing Division, which in turn report to General Headquar- services reimbursed $362 million of that total. The ters in Orillia. The OPP also operates five regional OPP also spent $174 million for capital projects, communications centres that take 911 emergency such as new facilities for detachments, forensic and public calls, and dispatch police. identification units and regional headquarters. The OPP provides municipal policing services to 322 municipalities on a cost-recovery basis as well

230 Ontario Provincial Police 231

the OPP. As most police work is carried out at local Figure 2: OPP Expenditures, 2011/12 ($ million) detachments, we focused our work on the OPP’s Source of data: Public Accounts regional operations. We did not review areas such Fleet management ($61) as capital expenditures, investigations and organ- Other ($6) ized crime bureaus, and the force’s emergency preparedness activities. Corporate and strategic services We interviewed senior management of the ($145) Ministry of Community Safety and Correctional Services, staff and elected officials from nine municipalities for which the OPP provides policing services, and representatives from the Association Investigations and organized crime of Municipalities of Ontario and the Association ($102) of Municipal Managers, Clerks and Treasurers of Ontario. We researched police operations in sev- Field and traffic services ($665) eral other jurisdictions and visited the RCMP and the Sûreté du Québec (Quebec’s provincial police force) to discuss their perspectives on the administration of policing services. We also engaged on Audit Objective and Scope an advisory basis the services of an independent expert with senior management experience in the 3.10 policing sector. Our audit objective was to assess whether the Prior to our last audit in 2005, the OPP did not

Ontario Provincial Police had systems and proced- Section use the government’s Community Safety and Cor- ures in place to: VFM rectional Services Audit Service Team; however, the •

ensure its provincial and municipal police 3 • OPP now utilizes the Ministry’s internal auditors services are delivered cost-effectively, and on a regular basis. We reviewed a number of recent in compliance with key requirements of the

internal audit reports and found them helpful in Chapter Police Services Act and the OPP’s own oper- planning and conducting our audit, particularly ational policies and procedures, known as their reviews of front-line policing supervision, over- Police Orders; and time costs and municipal policing arrangements. measure and report on its effectiveness in • We also found helpful the reports prepared achieving its mandate and priorities. by the OPP’s Quality Assurance Unit, which Before beginning our fieldwork, we identified regularly inspects and reviews police practices at criteria we would use to address our audit object- detachments. ive. Senior management at the OPP reviewed these criteria and agreed to them. The scope of our audit included interviews with OPP management, civilian staff and officers; and Summary tests, reviews and analysis of relevant files, policies and procedures at the OPP’s General Headquar- Over the last two decades, crime rates across ters, three regional headquarters, one regional Canada have declined by more than 40%, and communications centre and 10 detachments. We Ontario has been part of this trend. For instance, also considered the recommendations we made since our last audit of the OPP in 2004/05, crime in our 2005 Annual Report and in earlier audits of rates reported by the OPP have decreased 10%, 232 2012 Annual Report of the Office of the Auditor General of Ontario

and serious motor vehicle accidents have also been since then claimed it uses this new model, but trending down, with fatalities and injuries lower by it does not. In March 2012, the OPP’s existing 45% and 32% respectively. Over the last five years, model calculated that the OPP requires 500 the average number of calls for service the OPP has more front-line officers overall, or 18% more, responded to or initiated has remained relatively whereas the new model calculated that the stable. OPP needs 50 fewer officers. We also found By contrast, OPP expenditures net of recoveries that the OPP incorrectly applied staffing from municipalities have increased by 27% over the requirements for officer safety in the existing same period. Most of the increase has resulted from model; when the error was corrected, the the hiring of additional officers and staff compen- model calculated that only an additional sation increases over the last six years. However, 156 officers were needed. The OPP also does our research did indicate that many of the other not analyze either its officers’ availability to larger police forces across Canada have had similar immediately respond to a call or the time it expenditure increases, notwithstanding the long- takes for officers to respond to a call, both term trend of declining rates of crime and serious key factors in determining the deployment of motor vehicle accidents. officers to detachments across Ontario. In our last audit report in 2005, we made a • The force also lacks a model or method for number of recommendations on issues relating ensuring that 44% of the total detachment to staff deployment and controlling costs such as workforce, comprising the 1,600 officers who overtime. We nevertheless found that many of do not perform front-line duties and 640 3.10 these issues continue to exist today. Among our civilian staff, are efficiently and effectively major observations: deployed. Section • The force has 2,800 officers assigned to • Shift scheduling practices since our last audit VFM front-line duties in detachments, but it uses a have shown no improvement whatsoever in •

3 computer-based model developed in the 1980s matching the demand for officers, especially to calculate how to deploy only about 1,250, during peak workload periods. OPP manage-

Chapter or 45%, of these officers, and only to those ment had little control over shift scheduling detachments that provide policing services at detachments, and almost all officers choose under contract to municipalities. It has been to work 12-hour shifts. Current shift arrange- more than 10 years since the model was used ments result in overstaffing during slow to assess the deployment of some 1,550 other early-morning hours compared to the busy officers to detachments that provide other afternoon hours. Similarly, Sundays are over- front-line policing services, such as patrolling staffed by an estimated 20%. We estimated provincial highways, or to municipalities that that improved shift scheduling could allow the use the OPP to provide basic police service OPP to save millions of dollars per year or, at without contracts. Accordingly, officers face the very least, make its officers more available significantly different workloads depending to respond to calls for service. on where they are assigned. We found that • The OPP could assign more corporate-services officers in some detachments handled an aver- duties and other non-policing work to lower- age of 54% to 137% more calls than officers in paid civilian staff to free up officers’ time for other detachments. policing duties and to save money. The OPP • In 2005, the OPP advised us that it was has recognized this and initiated a review of working with the RCMP on a new officer- some staffing positions, which identified more deployment computer model. The OPP has than $6 million a year in potential savings. Ontario Provincial Police 233

• OPP officers are among the highest compen- • While the OPP reports extensive information sated police officers in Canada. Officers and on its policing activities, crime and motor- civilians receive certain benefits to which vehicle accidents in its annual report, it does other members of the Ontario public service, not report information on its own efficiency, such as correctional officers, are not entitled, such as per capita operating costs and including significantly better pension benefits. response times. Since 2003, OPP officers have been receiving an annual incentive payment worth up to $7,500 to make their salaries comparable to another large police force that had negotiated Detailed Audit Observations a retention incentive at that time. • In our 2005 report, we noted that the OPP FUNDING LEVELS FOR POLICING had lowered its overtime costs for 2004/05 Over the past two decades, the crime rate across by 10% to $33 million. However, no further Canada has shown a steady decline. In total, progress has been made to decrease overtime criminal offences are down more than 40%. In hours, and overtime costs have increased by Ontario, since our last audit in 2005, the number 60% to $53 million for 2011/12. of offences is down about 20%. As well, there has The overall cost of OPP services for municipal- • been a decline in the number and seriousness of ities from 2007 to 2011 increased an average crimes reported to police across Canada and in of 29% for those with contracts and 19% for

Ontario. Statistics Canada’s report on the Canadian 3.10 those without, or up to three times the annual crime severity index, which tracks changes in the inflation rate. Municipal officials told us that

relative seriousness of police-reported crime from Section the force offered no adequate explanations year to year, indicates that the severity of crime has VFM or accountability for these increases, which •

declined every year since 1991; it was 6% lower in 3 are due to changes in OPP billing and costing 2011 than in the previous year and 26% lower than methods, and increases in officer compensa- in 2001. Across Canada, both the rate and severity

tion. On the other hand, all municipal officials Chapter of violent crime fell 4% in 2011 from the previous with whom we spoke said they were very year, and it was the fifth consecutive annual decline satisfied with the OPP services they received. in severity of violent crime. In Ontario, the violent The way the OPP charges municipalities for • crime severity index fell 5% from 2010 to 2011, and its services was complex and onerous for both the total crime severity index dropped 6%. the force and municipalities. In contrast, the As well, motor vehicle fatalities and injuries have RCMP and Sûreté du Québec billing processes declined by 45% and 32% respectively since 2005. for municipalities appeared much more A common detachment workload measure for straightforward. policing is “calls for service.” More than half of OPP processes were insufficient to ensure that • all the calls for service that police respond to are detachments consistently met the require- related to either Criminal Code offences or traffic ments of the Police Services Act and the force’s incidents, and they come either from the general own policies. In most of the eight detachments public or officers themselves. A call from the we inspected, we noted instances where public—a “citizen-generated” call—might involve, seized cash, weapons and drugs, as well as for example, an assault or a missing person. A call officers’ weapons, were either improperly initiated by an officer, typically while out on patrol, stored or accounted for, or were missing. could involve the apprehension of someone driving 234 2012 Annual Report of the Office of the Auditor General of Ontario

under the influence of alcohol, although the count tures at other large police forces in Canada have of officer-initiated calls does not include routine increased by similar percentages. And while the vehicle stops for traffic violations such as speeding. number of criminal offences reported by the OPP Over the last five years, the average number from 2005 to 2011 declined by 10%, the clearance of all calls for service to the OPP was relatively rate (where charges are laid, or the case is closed stable at approximately 716,000 a year, with about with no charges) improved by 19%. This clearance 463,000 of them initiated by police themselves and rate compares favourably to that of other Canadian 253,000 being citizen-generated. and Ontario police forces. In total, OPP expenditures have increased 34% We are not aware of any recent independent since our 2005 audit (27% after recoveries from studies or assessments that analyze the long-term municipalities), while the number of calls for ser- trend of OPP funding and staffing increases in the vice remained unchanged. Increased funding has face of stable or declining demands for police servi- been needed primarily to cover the cost of more ces. However, we understand that the OPP has been officers and increases in staff compensation. In con- asked to reduce its expenditures in 2012 to help trast, for the six years prior to our 2005 audit, total address the province’s budget deficit. OPP expenditures increased 37% compared to total calls for service rising 31%. RECOMMENDATION 1 Statistics Canada data indicates the overall To support future decisions on funding for the increase in total OPP expenditures in recent years Ontario Provincial Police, given the long-term is consistent with the increase in total expenditures

3.10 trend of decreasing crime rates and fewer for all policing in Canada. The number of OPP serious motor-vehicle accidents in Ontario and officers rose by nearly 13% from 2004 to 2011,

Section across Canada, the OPP, in conjunction with the comparable to the national increase of 14% over Ministry of Community Safety and Correctional VFM the same period. The per capita cost of municipal •

Services, should formally assess the correlation 3 and provincial policing in Canada for 2008 (the last of its funding and staffing levels with the actual year for which the information was published) was demand for policing services, based on such fac-

Chapter $268, up from $205 in 2003. Ontario had the high- tors as calls for service, motor vehicle fatalities est such cost at $294, up from $225 in 2003. Since and injuries, number of reported offences, clear- 2005, the OPP has added about 700 officers, 350 of ance rates for crimes and crime severity levels. whom were hired to fulfill new municipal policing requirements for which the cost was recovered. OPP RESPONSE Most of the other 350 additional officers were approved in 2008, when the province provided We agree with the Auditor General that it is funding for 200 additional positions for highway important to formally assess the correlation of safety and investigative units, and the federal our funding and staffing levels with the actual government provided temporary funding for 125 demand for policing services. Crime rates are officers until March 31, 2013. Also, 500 more civil- not always an accurate barometer of the need ian employees were added, raising their number by for police resources, given that proactive work 28% since 2005. and the implementation of prevention programs In summary, OPP expenditures have clearly often lead to the reduction of crime. In addition, increased at a much faster pace than inflation, trends can quickly change. The total number of even though crime rates and serious motor-vehicle deaths on OPP-patrolled roads as of the end of accidents are in a trend of long-term decline. We September 2012 was 215, compared to 201 at do recognize in this regard, however, that expendi- the same time last year. Ontario Provincial Police 235

MANAGING COSTS The OPP is currently in the process of updating the Deployment Model to complete an Detachment Staffing Levels assessment of staffing requirements for front- The OPP has approximately 4,500 of its 6,300 offi- line services. The model includes calls for service cers and 640 of its 2,300 civilian staff working in 78 for most criminal offences and all motor-vehicle community-based detachments. The rest of its staff collisions, as well as other parameters used to work primarily at the OPP’s regional headquarters, determine front-line policing staffing levels. communications centres, and at General Headquar- As the province’s police service, the OPP ters in Orillia. Individual detachments can have up also provides a wide variety of services not to three distinct policing responsibilities: municipal identified in the Deployment Model. For policing for municipalities that have a contract with example, the level of provincial highway staff- the OPP, basic municipal policing for non-contract ing cannot be determined solely on the basis municipalities, and patrol of provincial highways of motor-vehicle collision rates; factors such and waterways. as kilometres of highways, traffic volumes As we noted in our 2005 audit, the OPP still and patrol standards must also be taken into does not have a consistent method for deploying account. Investigations into major crimes and officers to detachments. The computer-based model organized criminal activity are also influenced that the OPP developed in the 1980s—called the by a number of factors, including the growing Deployment Model—is programmed to calculate complexity of crimes; sophisticated and time- the number of officers required for front-line duties

consuming investigative techniques; and court 3.10 in detachments. The Deployment Model takes into and case law requirements. Further, assistance account six key parameters:

to municipal police agencies is covered under Section detachment-area characteristics; Section 9 of the Police Services Act. Operational • VFM number and type of citizen-generated calls for • support and many specialized services (such • 3 service; as tactical and emergency response, major percentage of time an officer is available to case management, and search and rescue) are •

respond to a call for service; Chapter staffed based on the need to have these resour- patrol standards; ces available to the province in case of major • total hours that officers are available for front- incidents—their size cannot be determined • line policing; and solely on the number of occurrences. minimum staffing levels for officer safety. The OPP will continue to review its oper- • While the OPP uses the Deployment Model to ations to find efficiencies and will analyze data calculate the number of front-line officers required and information to assess whether savings are in detachments that serve municipalities with con- available or funding can be reallocated across tracts (currently about 1,250 officers), it has been programs. Through the Future of Policing Advis- more than 10 years since the model was used to ory Committee and other partnerships, the OPP assess the number of front-line officers assigned to will review statistics to ensure that the causal other provincial duties or to municipalities without factors of crime rates and other trends are contracts (currently about 1,550 officers). The understood and that decisions are made based lack of a consistent method for deploying officers on sound information. to detachments has contributed to significant variances in the workloads of officers depending on the detachment to which they are posted. Front-line officers in some detachments handle an average of 236 2012 Annual Report of the Office of the Auditor General of Ontario

more than twice as many calls for service annually Figure 3: Range in Annual Average Calls for Service as officers in other detachments in the same region, Per Officer Within Regions (by Detachments), 2011 as indicated in Figure 3. Source of data: Ontario Provincial Police We reviewed the information used by the OPP for the parameters of the Deployment Model and Average Calls/Officer Difference identified the following regarding its accuracy: Region Lowest Highest (%) • In early 2012, the OPP adjusted the model to Central 225 348 54 use more current actual workload information East 182 335 84 for officer availability and patrol standards. North East 122 282 131 For instance, information for total hours that North West 127 301 137 West 195 336 72 officers are available for front-line policing had not been updated since 1998. The annual the time. However, the OPP does not track its average available hours per officer has success in meeting this target, or whether the declined in recent years by 120 hours, from standard is even reasonable and appropriate, 1,467 to 1,347, due primarily to additional because it monitors neither officers’ availabil- sick time off (22 more hours), administrative ity to respond to calls nor the average time it requirements (53 more hours) and training takes to respond to a call. (34 more hours). As a result, the model will In our 2005 audit, we noted that the OPP in future calculate a need for more front-line was working on a new joint OPP–RCMP staff- officers at detachments. Similarly, the new

3.10 ing model—called the Policing Resource Model traffic volumes, which were last updated in (PRM)—to replace the Deployment Model. The the 1990s, will require more officers to meet

Section PRM uses information that is similar to but more the patrol standards. detailed than the information used by the Deploy- VFM We also determined that the parameter •

• ment Model to determine how many officers a 3 designed to ensure officer safety was detachment should have for front-line duties. When incorrectly applied. The OPP’s policy requires we did our 2007 follow-up report, OPP officials told

Chapter that officers work in pairs for the eight hours us they had completed testing of the PRM for six between 10 p.m. and 6 a.m. when practical. detachments and that further testing for another 25 However, the model used a 12-hour period detachments was scheduled for the fall of 2007. The for pairing. Our request for the OPP to re-run OPP told us that if testing was successful, the PRM the model based on the policy requirement would be implemented across the province. resulted in a calculation that the OPP required However, we found during our current audit only 156 more officers province-wide, or 344 that the OPP does not use the PRM for determining fewer than the 500 originally calculated, as the number of front-line officers required; instead discussed below. However, our interviews it continues to use the older Deployment Model. indicated that, in practice, very few officers Despite this, the OPP said in its 2008 business plan in detachments actually work in pairs as per to the Ministry of Community Safety and Cor- the policy. As a result, the model has been rectional Services (Ministry) that it was using the calculating for some time that more officers PRM, and in 2012 the Ministry informed Treasury are necessary than actually are required for Board that, based on the PRM, the OPP had a short- serving municipalities with contracts. fall of officers to support front-line service delivery. Under the model, there is supposed to be an • The OPP even received an international policing officer available to immediately respond to award in 2009 for its implementation of the PRM. any citizen-generated call for service 92% of Ontario Provincial Police 237

We noted that the RCMP is currently using the PRM RECOMMENDATION 2 and has been doing so for more than five years. In March 2012, the OPP conducted an analysis To help ensure that the number of front-line of its front-line officer requirements at all detach- officers at each detachment is based primarily ments using both models. The Deployment Model on need and that officers are cost-effectively calculated that the OPP required 500 more officers, deployed, the Ontario Provincial Police should whereas the PRM calculated that it actually needed reassess its two computer-based models to 50 fewer officers. In addition, both models identi- determine which one provides the best estimate fied that detachments required anywhere from of requirements based on up-to-date and accur- 30% fewer to over 100% more officers, although ate operational and workload standards, and, the results varied widely between the models. For once validated, use its staffing models to deploy example, at a detachment with 59 officers, the PRM and reassign officers. The OPP should also calculated that 28% fewer officers were needed establish formal staffing methodologies for the while the Deployment Model calculated 105% more other 44% of detachment staff not covered by its officers were required. We talked to a number of deployment model. regional and detachment senior staff, including staff in the 59-member detachment above, and they OPP RESPONSE indicated to us that, from an operational perspec- The OPP has conducted extensive analysis tive, they had no serious concerns with their cur- comparing the Policing Resource Model (PRM) rent officer staffing levels.

and the Deployment Model, and determined 3.10 Both staffing models calculate the number of that the Deployment Model is the most suitable front-line officers needed at detachments, but the

model for identifying municipal and provincial Section OPP has not established a staffing template for staffing requirements at detachments. The OPP VFM the other 44% of staff who also work in detach- •

is updating the Deployment Model parameters 3 ments, including 1,600 officers who do not perform and expects to complete this project in late front-line duties and 640 civilian staff. These other 2013. This will make it possible to compare

officers work in administrative and other policing Chapter staffing results from the model with the actual duties, including courts, community relations and staffing levels at each detachment. Given the crime investigations. We could find no basis for diverse nature of detachments, local input and determining the number of these other officers— analysis of operations must also be conducted for example, no detachment-workload indicators and taken into account to determine the were being used. As well, we were told that the approved staffing requirements. number of officers and civilians working at detach- The OPP agrees with the Auditor General ments was generally based on historical levels, and that formal staffing methodologies are neces- the numbers have been relatively stable over the sary for the detachment staff not covered by the last five years. Deployment Model, such as specialists, admin- At the time of our audit, the OPP was engaged in istration and supervisory positions. The OPP a project to develop templates for detachment staff- has been working on a set of consistent methods ing and to identify inconsistencies and inequities in for staffing the various types of non-front-line current detachment staffing levels. staff at detachments. The detachment structure review has led to the creation of a model for ideal staffing of non-front-line staff, which the OPP will implement as part of its current review of staffing levels in detachments. 238 2012 Annual Report of the Office of the Auditor General of Ontario

Officer Shift Scheduling detachments by assigning more of those duties to civilian staff. As with staff deployment, a key to operating cost- OPP senior management acknowledged that effectively is having the right number of officers moving from 12-hour shifts to a mixture of shift working at the right times. In 1998 and in 2005, lengths, using part-time officers to supplement we reported that officers were not scheduled in a regular officers and increasing shift overlaps dur- way that corresponded to peak workload periods, ing peak times would save money and improve the and this issue remains. We saw no evidence of any productivity of detachments. However, manage- changes to make more efficient use of staff time ment also advised us that the current collective in order to reduce staffing costs or improve officer agreement might not contain the flexibility needed availability during peak times. to make such changes. We estimate that adjusting We learned in interviews that OPP management the current 24-hour fixed scheduling from equally had little control over shift scheduling at detach- manned 12-hour shifts to a mixture of shifts and less ments. Instead, local officers generally determined staff from 3 a.m. to 7 a.m. would either result in sav- their own scheduling, and almost all chose to work ings in the range of $5–10 million a year or lead to 12-hour shifts, which offer more days off. Manage- improved service to the public and officer safety by ment in detachments whom we spoke to said they more effectively matching staffing to workload. believed they did not have the authority to change In addition, the OPP operated 77 of its 78 officers’ 12-hour shift schedules, and it was appar- detachments on a 24-hour basis in 2011. In 2008, ent that there were different interpretations and only 68 of the detachments provided policing

3.10 knowledge around OPP shift-scheduling policies 24 hours a day; the other 10 were open 20 hours and the collective agreement. a day, with officers working 10-hour shifts. Even

Section For the most part, the 12-hour shifts are sched- though the number of calls for service remained uled without overlap, so that the same number of VFM essentially unchanged from 2008, the OPP now •

officers is working at all times during the day and 3 operates virtually all its detachments on a 24-hour all through the week. Alternatively, overlapping basis, with almost all of its officers working 12-hour shifts can offer the benefit of having more officers

Chapter shifts. For instance, citizen-generated calls for ser- working during the busiest periods of the week and vice averaged 1,237 per year over the past five years day. Although overlapping shifts could pose a prob- for its only detachment open for only 20 hours a lem if, for example, there were not enough avail- day. Yet, we identified eight detachments that were able patrol vehicles, the issue has not been formally open 24 hours a day and had either the same or examined to assess its workability. even significantly fewer citizen-generated calls for At detachments, the total number of calls for service. In 2012, the OPP did change one detach- service and citizen-generated calls for service from ment’s hours so that it also operates only 20 hours 3 a.m. to 7 a.m. is only about one-third of the num- per day. ber from 3 p.m. to 7 p.m., when demand is greatest, The same issue exists with respect to weekday as shown in Figure 4. Because the scheduling of versus weekend shift scheduling. The schedules shifts is not based on workloads, detachments are are made up so that the same number of officers overstaffed during slow times in the early morning is assigned to each day of the week. We calculated hours compared to the busy afternoon hours. Front- that on Sundays, there was about 20% overstaffing line officers told us that after 3 a.m., they spend of officers, based on the average number of most of their time catching up on writing reports citizen-generated calls for service. By redeploying and court briefs, and other administrative duties. officers throughout the week based on the demand However, we also noted that the OPP has already for service, the OPP could either reduce costs by reduced the burden of report-writing at many Ontario Provincial Police 239

Figure 4: Total Number of Calls for Service by Hour of Day, 2011 Source of data: Ontario Provincial Police 45,000

Police-initiated calls 40,000 Citizen-generated calls

35,000

30,000

25,000

20,000

15,000

10,000 3.10

5,000 Section

VFM

•

0 01234567891011121314151617181920212223

several million dollars or make more effective use We raised this same concern in our 1990 and Chapter of those officers. 1998 reports. In 1990, front-line officers were Shift scheduling also becomes a significant issue taking 40% of their collective vacation during in the summer months, when the demand for OPP the summer months, and in 2011, that figure was service grows in many areas of Ontario with the about 35%. Some police forces have separate influx of tourists and cottagers. Figure 5 shows that summer and winter work schedules to make sure citizen-generated calls for service were 11% higher they have enough staff in the summer, but the OPP than the 12-month average during the summer has yet to implement this practice. We noted that, months of July and August in 2011; however, it also while the OPP has a policy that requires detach- shows that officers take significantly more vacation ment management—when they approve vacation in these months than in any other month. In fact, time—to ensure a minimum number of officers total officer hours worked during the months of be maintained during the summer months, the July and August were less than most other months expected calls for service are not formally taken that had fewer calls for service. For example, 8% into consideration. As well, since the population more staff hours were worked in March than in in many areas serviced by the OPP swells in the July, yet there were 23% more citizen-generated summer months and the number of calls for service calls for service in July. In addition, we estimated increases, consideration should be given to increas- that the OPP paid $2 million in overtime to cover ing staff availability rather than maintaining min- officers’ summer vacations. imum staffing levels at detachments. 240 2012 Annual Report of the Office of the Auditor General of Ontario

Figure 5: Monthly Percentage Change in Averages for load with staffing levels during all times of Four Variables, 2011 the year; Source of data: Ontario Provincial Police • reassess its current practice of having almost 120 all detachments operate on a 24-hour basis 100 to identify the savings potential of reducing 80 operating hours at some detachments; and monitor adherence to the existing policy 60 • requirement that shift scheduling practi- 40 ces at detachments be formally reviewed 20 annually to assess their efficiency and cost 0 effectiveness. JanFeb MarApr MayJun JulAug SepOct NovDec –20 –40 OPP RESPONSE Citizen-generated calls for service –60 Officer hours worked The OPP recognizes that shift scheduling is Vacation hours an important operational issue with a formal Overtime hours protocol outlined through the collective agreement. Significant research on this has In 2000, in response to our 1998 recommen- been undertaken by the OPP in the past year. dations, the OPP introduced a shift-scheduling The OPP considers operational issues such

3.10 manual. However, by 2008, only a few detachment as vehicle requirements, supervision, officer commanders had used the manual to change their health and morale, training, court appearances,

Section shift schedules. As a result, in 2009, the OPP issued administrative work and report-writing. Due to a new policy requiring that detachment and regional VFM operational considerations as well as the rural •

management annually review shift schedules. The 3 nature of OPP detachments, more formal assess- policy also required that a committee be established ments of shift scheduling at each detachment to conduct an annual review of shift scheduling and

Chapter may result in identifying savings that are less issue a report to senior OPP management by August than the figures estimated by the Office of the of every year. However, there have been no formal Auditor General and lower than those that an reviews of scheduling practices at any level, and the urban police service might generate. In addi- committee has not yet met. tion, the OPP is reviewing its policies regarding 24-hour detachments to assess the viability of RECOMMENDATION 3 this type of service in some locations. In order to reduce operating costs and/or make The OPP will continue to seek and imple- the best use of available officers’ time to more ment measures to better co-ordinate staffing effectively respond to calls for service, the with the demand for police services. Several Ontario Provincial Police should: tools and types of training have been developed • implement measures to give management and are available for detachment command- greater control over officers’ shift schedul- ers on the OPP intranet. The OPP’s goal is to ing and vacation entitlements to better develop additional tools, such as a software co-ordinate staffing with hourly, daily and tool in 2013, for detachment commanders to monthly demand for policing services; better analyze the implications of existing and • provide detachment management with proposed schedules, consistent with the criteria regular information that compares work- that must be considered as part of the process. Ontario Provincial Police 241

agement—that could just as well be filled by civil- When the comprehensive analytical tools are ians. The study said the OPP could save $760,000 finalized, it is anticipated that detachment man- by shifting some of these jobs to civilians. agement and staff will work together to develop We noted that the OPP has not done a full their own improved staffing schedules. Each review to identify the jobs where officers could be region will conduct annual reviews to ensure replaced by qualified civilian staff, apart from the optimum shift scheduling. positions already mentioned. In many cases, offi- The OPP has made efforts to include shift cers are assigned to administrative positions while scheduling in collective agreement negotiations. retaining their officer titles and salaries and bene- This was not successful in the last round of bar- fits, and they continue to receive the same police gaining; however, the OPP continues to press for training as front-line officers. The RCMP advised us its inclusion. Shift scheduling changes using the that they introduced a cost-saving initiative in 2012 existing manual will require a lengthy timeline to more rigorously ensure officers are used only for and a detachment-by-detachment review and policing activities, while civilians handle virtually approval process. Shift scheduling is included all other non-policing and administrative duties. in our proposed transformational plans and savings are expected by 2014/15. RECOMMENDATION 4

To help ensure that non-policing duties and responsibilities are handled as cost-effectively as Use of Civilians

possible, the Ontario Provincial Police should: 3.10

The OPP could make better use of more civilians, • conduct a review of all staffing positions and

who already make up 27% of its staff, to free up responsibilities at its detachments and its Section officers for policing duties and reduce operating regional headquarters and General Head- VFM

• costs. We noted this issue as far back as 1990. quarters to determine where opportunities 3 In early 2012, the OPP said it determined that exist to fill positions currently held by offi- having civilian employees take over more court and cers with civilians at a lower cost; community-service duties now performed by offi- • establish cost-saving targets and timelines Chapter cers would lower staffing costs. As of 2012, there for designating positions to either civilians were 103 officers working in such community- or officers, depending on the duties of the service duties as crime prevention programs in position; and schools and media briefings. A first-class officer • reassign officers who are currently in civilian earns a salary of $83,500 a year, while a civilian positions back to front-line policing duties community officer earns $57,250. The overall sav- where possible. ings if all these duties were transferred to civilians would be about $2.6 million a year. Similarly, there OPP RESPONSE are about 110 officers and 23 civilians performing The OPP is a participant in the Future of Poli- court duties. Transferring all court duties to civil- cing Committee (FPAC) review created by the ians would save another $2.8 million annually. Ministry of Community Safety and Correctional In its recent cost-management study, the OPP Services and involving municipal and police identified positions in Corporate Services now filled stakeholders to redefine core duties. This by officers—in areas such as business, finance, extensive review will provide service options human resources, training, records management, for consideration by all Ontario police agencies. policy development, and equipment and fleet man- 242 2012 Annual Report of the Office of the Auditor General of Ontario

our 1998 and 2005 audits, the DRU program is not The OPP has conducted a preliminary analysis yet consistently managed and executed. of potential savings opportunities with regard The DRU function is supposed to work as fol- to civilianization and will complete the analysis lows: When any of the five regional communica- with a more comprehensive review of the cost tions centres receives a citizen-generated call for savings and the operational impacts. Due to service, the detachment platoon sergeant decides provincial responsibilities and the number of whether an officer will be sent out or if the call can small, geographically dispersed detachments, be handled over the telephone at a later time. If the OPP must ensure it has adequate police offi- the sergeant decides that a front-line officer does cers available for local emergencies and major not need to be dispatched, the call is forwarded to provincial events such as Caledonia, G8/G20, the region’s DRU, which normally deals with the major investigations and others. call over the phone. (Under OPP policy, however, The OPP acknowledges that with the finan- if a caller or a municipality asks for an officer to cial constraints in today’s economy, the cost respond in person, one is dispatched regardless of savings of increased civilianization may be a the nature of the call.) component in the future of policing. The OPP is The OPP reviewed DRU operations in 2007 and committed to continuing its review of staffing found that all regions did not provide the same positions and will develop cost-savings targets kind of service and the units often lacked proper and timelines for changes. supervision, which could cause service delivery to The OPP is limited by the parameters of suffer. The OPP noted these inconsistencies and

3.10 the collective agreement in relation to the deficiencies again in 2010, and said it would launch establishment of and changes to job descrip- a project to ensure that the DRU program had a

Section tions and classifications. Some roles currently standardized structure to ensure consistent manage- being performed by uniformed officers that VFM ment and accountability. However, we found no evi- •

could potentially be civilianized (e.g., Com- 3 dence that this project was ever set up. OPP senior munity Service Officers) do not have specific job command requested another review of the DRU pro- descriptions associated with them because they

Chapter gram, to start in fall 2011. We were informed that as are not distinct positions. In addition, there are of July 2012, a draft report was being finalized. many operational benefits of police performing Our review of the number of calls handled by Community Service Officer and Court Services DRUs in 2011 noted significant differences in use of Officer roles: they are able to respond to other the program, as shown in Figure 6. One region, for calls for service, they have powers of arrest, example, used DRU to handle just 1% of its calls. and they generally have more influence over We believe that the DRU program, with proper students and the public. management, could help the OPP reduce operating costs and make more efficient use of the time of its front-line officers. However, only one region has a Differential Response Unit centralized DRU; in the others, local detachments cover DRU functions by assigning the duties to The OPP uses what it calls a Differential Response officers being accommodated for pregnancy, injury, Unit (DRU) in each of its five regions and its High- illness or any other cause making them unable to way Safety Division to handle low-priority calls for go out to respond to calls. It is likely more cost- service over the phone. The goal of the program is effective and efficient for a region to centralize its to make the most efficient and cost-effective use of DRU or for all communications centres to share one front-line officers’ time. However, as we noted in DRU for the entire province. Ontario Provincial Police 243

Figure 6: Percentage of Citizen-generated Calls for Response Unit (DRU) program. A comprehen- Service Handled by Differential Response Units, 2011 sive report, reaffirming these opportunities, was Source of data: Ontario Provincial Police completed and presented to senior management Region % of Calls in September 2012. Central 10 All regions have established a DRU program East 8 to some extent, but implementation provincially North East 1 has been inconsistent, as it relates to having an North West 22 effective operating environment and staffing West 6 requirements, including short-term accom- Highway Safety* Unknown modated officers, and appropriate management and training. The current report concludes that, * The Highway Safety Division operates a DRU, but does not maintain any statistics on its use. rather than centralization, the most decisive success factors are having clear direction from the In addition, in our 2005 Annual Report we noted regional commanders, oversight, accountability that one region’s DRU officers were able to each and well-defined reporting relationships among handle an average of 950 calls for service annually, those assigned to perform the DRU function. compared to about 200 calls for a front-line officer. The OPP is committed to moving forward In the current audit, the same region reported that on this initiative and will invest the resources its DRU officers each handled on average fewer required to implement improvements, includ-

than 600 calls per year, and consequently, if more ing setting performance targets and measuring 3.10 calls were forwarded to DRU, they could likely be against these targets.

handled with minimal staffing impact. Section

VFM

• RECOMMENDATION 5 3 OFFICER COMPENSATION To help achieve the significant cost and oper- Salary and Benefits

ational benefits of implementing a Differential Chapter Response Unit (DRU) program to free up front- According to a quarterly RCMP survey of salaries, line officers’ time for more serious matters, the benefits and pensions, the OPP ranked third among Ontario Provincial Police should: Canadian police forces at $66.45 an hour as of • establish a strategic plan for fully imple- March 31, 2012. Only the Toronto Police Service menting its DRU program throughout the ($70.90/hour) and the Vancouver police ($70.52/ province, with targets for measurable savings hour) were higher. The RCMP stood at $65.72 an and benefits, and associated timelines; and hour and the Sûreté du Québec at $57.84. • given the lack of widespread success in OPP members received an average annual implementing the DRU program over the salary increase of 2.9% for 2005 to 2010, a 5% last decade, consider centralizing the wage increase in 2011, and no increases for 2012 program to improve service levels, enhance and 2013. However, following a recent provincial consistency and help realize economies of government announcement that OPP officers would scale and cost savings. receive the “highest paid first-class constable base rate” in the province, an officer’s annual salary may OPP RESPONSE well increase by at least 8.5% effective January 1, 2014. In addition, officers and civilian staff are The OPP acknowledges that there are oppor- entitled to a number of compensation benefits that tunities to enhance and improve the Differential 244 2012 Annual Report of the Office of the Auditor General of Ontario

few other members of the Ontario public service, pay. From 2009 to 2011, 30% of staff received such as correctional officers, receive. For instance: this payment, and it represented about $4 mil- • In 2003, after the Toronto Police Service lion annually. began paying its officers a retention bonus to • The OPP provides an allowance of $1,250 encourage them not to leave the force in sig- annually to each officer whose policing duties nificant numbers, the provincial government require plain clothes in addition to uniforms. negotiated an incentive called the Provincial In 2006, the OPP extended that allowance Responsibility Incentive with the officers’ to all 200 commissioned officers, who work union—even though staff were not leaving primarily in an office environment, where the OPP for other employment in significant the OPP has a business-attire dress code. In numbers—that resulted in OPP officers being 2010, the allowance was expanded to include paid at compensation levels comparable dry cleaning expenses. The cost of providing to Toronto officers. Commissioned officers commissioned officers a clothing allowance is (generally senior OPP management staff) $250,000 annually. who are not part of the union also received the incentive, which was officially established Overtime to compensate officers for being expected to respond across the province to emergencies, In our 2005 Annual Report, we noted that the OPP for long-term investigations and for large local had achieved some success in lowering overtime events when extra resources are required. In expenditures, decreasing them by more than 10% 3.10 2012, the annual incentive was 3% of salary to approximately $33 million in 2004/05. Since after eight years of service; 6% of salary after then, overtime hours have been kept to a consistent Section 16 years of service; and 9% of salary for 23 rate per officer, but the impact of compensation VFM years or more, up to a maximum of $7,500. increases and increases in the number of officers has •

3 We noted that in 2002, before the incentive been a 60% increase in overtime costs to $53 mil- was provided, only 43 officers that year left lion for 2011/12. Approximately 63% of overtime

Chapter the force, which then stood at 5,300. In 2011, was incurred by officers in detachments, who on only 25 officers left the force. average received overtime payments of $7,500 each • OPP staff have better pension benefits, includ- in the 2011/12 fiscal year. The increase in overtime ing a lower retirement age, than other Ontario occurred even though over the last five years, the government employees who are members of average annual total calls for services handled by the same Public Service Pension Plan or other the OPP remained relatively stable, the number of pension plans. This in itself is a retention criminal offences reported by the OPP declined and incentive. In addition, as of January, 2012, the the OPP has added 700 officers since 2007. pensions of officers are calculated based on In addition, we noted that in 2011, about $7 mil- a 36-month average of their highest annual lion of overtime was logged for administrative dut- salary, rather than a 60-month average used ies, such as training, report-writing and meetings, for other provincial government employees. in apparent contradiction of OPP overtime policy. For civilian OPP staff, the pension is based on a 48-month average. Using fewer months to RECOMMENDATION 6 determine average salaries results in higher To help inform future decisions on compen- pension costs for the government. sation levels for officers and as part of the Each year, instead of taking five vacation days, • preparation for future collective bargaining staff may opt to receive an additional week’s Ontario Provincial Police 245

negotiations, the Ontario Provincial Police will review the current administrative overtime (OPP) should analyze the working condi- and will ensure it is reduced where appropri- tions and compensation levels of its officers in ate or coded to the correct activity. The results comparison to other major police forces across will be monitored and reviewed for 2012/13 Canada and in relation to current Ontario gov- and onward. ernment compensation policies. Over the past few years the OPP imple- The OPP should also increase its oversight mented additional overtime oversight and of overtime expenditures as well as identify and processes including detailed monthly overtime address the underlying reasons for the signifi- reports, detailed detachment analysis and cant increase in overtime costs in recent years. memos regarding overtime management. Over- time management is part of the OPP’s current OPP RESPONSE cost management strategies and savings targets are reflected in the Results-based Planning The OPP has committed in its 2012 trans- process, underlying the OPP’s commitment to formational plan proposal to continue to work overtime management. through the Police Bargaining Working Group to identify opportunities to bend the cost curve in police compensation. The OPP uses Statistics Canada and RCMP Police Sector Total Compen- MUNICIPAL POLICING sation reports and communicates with other

Under the Police Services Act (Act), municipalities 3.10 policing agencies regarding compensation. The may establish police forces either on their own or Employer (Ministry of Government Services),

with other municipalities, or they may purchase Section in consultation with OPP, has reviewed many policing services from other municipalities or from VFM aspects of OPP compensation and will continue •

the OPP under a five-year renewable contract with 3 to bring these forward to the negotiation table. the Ministry. If municipalities make no arrange- The OPP will continue to assess compensation ments, the OPP is required to provide the basic

trends and will annually review its compensa- Chapter policing services it thinks are necessary and those tion compared to other policing agencies and municipalities reimburse the OPP for the cost of the Ontario government. the policing provided. Municipalities that have The OPP agrees that overtime is an import- contracts with the OPP have more control over the ant cost to manage. After significant effort, accountability and level of service that they get overtime costs were brought to very low levels from the OPP, and in setting community priorities for the 2004/05 fiscal year, as noted by the Aud- for their own policing. itor General. Since that time, overtime has been During the 2011/12 fiscal year, the OPP serviced significantly impacted by major events including 149 municipalities with contracts, and 173 muni- protracted First Nations protests, several large- cipalities that pay for services without a contract. scale investigations and the G8/G20. The OPP received $362 million in revenues for Although overtime has increased in dol- providing these services, with 75% of that coming lar terms, a portion of the increase is directly from municipalities with contracts. Billings and attributable to compensation increases and the collections for both contract and non-contract increase in officers for new municipal contracts. municipalities are handled by the Shared Services Administrative overtime is generally not permit- Bureau of the Ministry of Government Services ted and in some cases may be a coding issue using information provided by the OPP. In addi- rather than administrative overtime. The OPP tion, the provincial government assists small and 246 2012 Annual Report of the Office of the Auditor General of Ontario

rural municipalities with their policing costs. In costly than maintaining their own municipal police 2011, municipalities policed by the OPP received forces. According to OPP estimates, municipalities $71.9 million from the Ministry of Finance under with and without contracts save on average any- the Police Services Grant Program. where from 35% to 60% by using the OPP instead of In addition, the OPP provides support to smaller having their own police forces, as shown in Figure 7. and rural municipal police forces that may need OPP senior staff told us that some municipalities assistance from time to time, and to help them choose not to sign contracts for OPP services, while meet policing standards for events such as emer- others with contracts decide to pay for services over gency response. and above the basic requirements. Consequently, municipalities can receive different levels of policing. As an example, two municipalities with simi- Relations with Municipalities lar numbers of calls for service had very different A number of municipal officials told us that while levels of service: One, a non-contract municipality, their annual costs for OPP services have increased paid $9 million a year for a minimum level of police significantly, the OPP offered no adequate explana- service; the other paid more than $15 million a year tions or accountability for these increases, and for a contract that provided for 30 staffing enhance- showed no flexibility to limit them. The OPP and ments, including a dedicated traffic patrol and Ministry have acknowledged these concerns and, greater officer presence in the community. Several during our audit, were working on ways to better larger municipalities have chosen a basic level of share information to improve the relationship with service by not entering into contracts with the OPP, 3.10 municipalities. likely due to the lower costs of such arrangements. Municipalities are paying more for OPP services Interestingly, the OPP advised us that three muni- Section largely because of changes over the last several cipalities that paid for enhanced OPP services were VFM years to the OPP’s billing and costing methods, as cited in 2010 by a national magazine as being in the •

3 well as increases in officers’ salaries. We calculated top five safest communities in Canada. that for municipalities with contracts, the overall The Ministry and OPP have been working

Chapter cost of the OPP’s services increased 29% from 2007 to improve relationships with municipalities by to 2011, an average of about 7% annually—or more providing more information on the costs of their than three times the rate of inflation. For municipal- services. In February 2012 , a project team of ities without contracts, the overall increases were representatives from the OPP, the Ministry, munici- 19% over the same period, or an average of about palities and their associations, and police services 4% annually. Over the same period, individual boards was formed to improve communication municipalities experienced a range of changes in and accountability about the OPP’s costing policies costs for OPP service, from a decrease of 60% to an and processes, and to identify opportunities for increase of 135%. Factors besides internal OPP cost municipalities to contain or reduce policing costs. increases that affect the chargeback include changes In March 2012, about 40 policing organizations in the number of calls for service to a municipality. and municipal stakeholders took part in a summit Notwithstanding the concerns expressed about on the future of policing hosted by the Ministry. As costs, all municipal officials we spoke with said well, the Ministry created the Future of Policing they were very satisfied with the OPP services they Advisory Committee with municipal involvement received and with the communication provided to oversee and make recommendations by 2015 for by local detachment commanders on community changes to policing standards. issues. As well, all the municipalities acknowledged that using the OPP’s services was generally less Ontario Provincial Police 247

Figure 7: Cost Comparison of Municipal Police Services, 2011 Source of data: Ontario Provincial Police

Estimated per Capita Cost Provider of Police Services # of Municipalities of Police Services ($) Population 100,000 and over Municipal police force 17 282 OPP – with contract 0 — OPP – no contract 0 — Population 50,000–99,999 Municipal police force 7 264 OPP – with contract 6 153 OPP – no contract 2 117 Population 15,000–49,999 Municipal police force 14 284 OPP – with contract 21 150 OPP – no contract 4 131 Population 5,000–14,999 Municipal police force 14 329 OPP – with contract 48 156

OPP – no contract 31 133 3.10

Population less than 5,000 Municipal police force 1 371 Section

OPP – with contract 74 238 VFM

•

OPP – no contract 136 142 3

Costing and Billing Methodology To determine the minimum number of officers Chapter a municipality requires, the OPP uses either its The OPP sets the minimum number of officers and Deployment Model or a workload analysis tool. hours required to service a municipality, regardless However, these different methods can produce of whether it has a contract in place. This deter- different results. In 2011, internal auditors noted mines the municipality’s expected costs, including that the OPP had not established clear criteria for direct costs for officers’ salaries plus an allocation using one or the other method. We were told that for other support and operating costs, such as a manual on the contract renewal and negotiation vehicles, training, communications, forensic iden- process was to be completed by late 2012, and tification units and facilities. Municipalities with shared with municipalities, to address the internal contracts can also opt to enhance the service level auditor’s recommendation that the OPP ensure that beyond the minimum recommended by the OPP, the process of establishing municipal contracts is and to pay extra for more officers. Municipalities transparent and managed effectively. are given a budget at the beginning of the year, As of our current audit, the agreements with billed monthly based on that budget, and receive municipalities did not clearly indicate whether a final invoice that reconciles the budget to actual the number of officers was the original minimum expenses at year-end. determined by the OPP or whether it was a higher 248 2012 Annual Report of the Office of the Auditor General of Ontario

number based on a municipal request. For example, costs in the year that the contracts were renewed. contracts for one municipality showed that the As of March 31, 2012, the OPP was still billing 32 number of officers increased from 65 in 2009 to 80 municipalities using the 2003 formula and four in 2013. However, there was no indication whether using the 2008. We noted that if all municipalities these additional officers were needed as a minimum were updated to the current 2010 formula, the OPP to meet increasing workloads, or whether they had would have recovered an additional $5.4 million of been requested as enhancements to the contract. its other support and operating costs annually. Without sufficient contractual or other docu- The OPP has received approval from Treasury mentation, the OPP risks not being able to clearly Board to update and apply the formula on an demonstrate that it is providing policing services to annual basis beginning in 2012. Internal auditors municipalities consistently and cost-effectively. also identified that costs in the formula are not con- The costing formula for the OPP’s contracts is sistently included, updated or validated, and that established by a regulation under the Police Ser- there is no clear documentation for the rationale for vices Act and approved by Treasury Board. Direct including or excluding a particular cost. Beginning costs represent about 80% of the formula and are in 2012 with the annual updates to the formula, updated annually based on actual salaries and the OPP was to have a formal process in place to benefits paid to officers, including all increases review all the items used in the costing formula, negotiated as part of collective agreements. Other and improve its documentation for determining support and operating costs make up the rest of the which costs should be included. Better documenta- formula and are based on provincial average costs tion should also help improve the OPP’s ability to 3.10 calculated periodically and applied based on the disclose its charging practices to municipalities. number of officers assigned to the municipality. The In addition, we found that in 2011, municipal- Section cost per officer to municipalities for both direct and ities covered by 49 of the 118 contracts in place VFM other support and operating costs increased from received more hours of service than their guaran- •

3 approximately $122,200 in 2007 to $144,000 in teed minimums, including three locations that have 2011, or 18%. each been over-serviced by more than 10,000 hours

Chapter Other support and operating costs embedded annually since 2007. However, the OPP is unable to in the formula were not updated annually, so the recover the additional costs of providing more ser- charges did not necessarily reflect current OPP vices than required in the contracts. As well, muni- costs. Beginning in 2012, all municipalities without cipalities covered by 21 contracts received fewer contracts were billed using the 2010 costing for- hours of service than the guaranteed minimum. mula, the most recent approved by Treasury Board. OPP officials told us that these discrepancies could However, for municipalities with contracts, the stem from officers not properly recording their time OPP is currently billing based on 2003, 2008 and or because the use of their time had not been prop- 2010 formulas, depending on when the five-year erly monitored by their supervising officer. contract was last negotiated. This is because as the contracts expired and were renewed, the most cur- Annual Credit Reimbursement to rent formula at the time was applied to the contract Municipalities for the entire five-year renewal without provision for the OPP to update the formula. Other support OPP detachments operate on an integrated service and operating costs increased from about $18,000 model whereby one detachment may provide poli- per officer under the 2003 formula to $29,000 in cing services to several municipalities—contract the 2010 formula. As a result, municipalities could and non-contract—as well as perform provincial have experienced increases of up to 60% in these Ontario Provincial Police 249 policing duties either within or outside of the Other Provinces detachment area. The charging practices for the OPP’s services are Officers who work in detachments are not complex and onerous both on municipalities and on designated as specifically municipal or provincial. the OPP, and, as already noted, not well understood Whatever their duties on a particular day and what- by municipalities. In contrast, our understanding of ever municipality they work at, officers record their the RCMP and Sûreté du Québec costing processes activities and location in the OPP’s Daily Activity suggests they appear much easier to administer. For Reporting (DAR) system. As part of the billing pro- example, the RCMP dedicated officers to its munici- cess, the OPP provides contract and non-contract pal contracts and charged municipalities 80% of the municipalities with a Provincial Service Usage cost of those officers. As a result, the RCMP did not (PSU) credit to make up for the time officers in the provide credits to municipalities for any police work detachment spend performing provincial policing they performed outside of the municipality, since duties or working outside of the detachment area co-operation between police forces is considered rather than performing municipal duties. The OPP a normal practice. In Quebec, all municipalities provided an average of $23 million per year over with populations up to 50,000 have been required the last five years in total PSU credits to municipal- since 2001 to use the Sûreté du Québec for policing. ities. These credits were in addition to the police Ten-year contracts are used, and regional muni- services grants managed by the Ministry of Finance. cipalities are invoiced based on property values For municipalities with contracts, we asked why and population. the OPP needed to provide PSU credits of about

The OPP prepared a briefing note to the Minister 3.10 $15.6 million annually when its contracts and of Community Safety and Correctional Services in annual budgets with these municipalities are based

2004 proposing to implement a provincial per capita Section on hours of service and municipalities are charged or per household methodology for invoicing muni- VFM only for the time spent on municipal policing, thus •

cipalities, but no action was taken. The OPP advised 3 making the credits unnecessary. (For municipalities us at the time of our audit that it was reconsidering without contracts, annual budgets are calculated a per capita costing methodology and expected to

in a different way that requires providing a PSU Chapter present a briefing note to the Ministry in late 2012. credit.) In addition, we noted that the PSU credit and billing method does not take into account whether the minimum required contract hours Assisting Municipal Police Services were exceeded by the OPP for municipalities with The OPP is sometimes called on to assist municipal contracts. For instance, five municipalities with police forces, but it has never charged municipalities contracts were each over-serviced by more than for this, even though it has the authority to do so 10,000 hours in 2011 and still received between 3% under the Act. In some cases, a formal support and 8% of their gross costs back in PSU credits at agreement with the OPP is required to allow the the end of year. municipal police force to meet the adequacy stan- The OPP notes that municipalities rely on PSU dards under the Act for a municipality to have its credits to reduce their police services costs, and own police force. The OPP has calculated that from that any proposal to eliminate the credit would 2007 to 2009, the average cost of providing assist- need Treasury Board approval, because it would ance to other police services was $11.3 million per place a greater financial burden on municipalities year. However, we were told that this calculation at the same time that the updated costing formula included costs that fall under the OPP’s provincial was being applied to include more of the OPP’s true responsibilities, such as complex cross-jurisdictional costs for providing services. 250 2012 Annual Report of the Office of the Auditor General of Ontario

investigations, which may not be recoverable, and The OPP, Ministry of Community Safety and that the OPP’s system does not track the strictly Correctional Services and Ministry of Finance municipal assistance costs separately. have been collaborating on identifying and As part of its 2012/13 planning, the Ministry has evaluating changes to the current complicated recommended to the Treasury Board that it start system of charges and grants. Proposals for a charging municipalities for its assistance. However, more equitable and simple billing process are the OPP noted that it was the smaller municipalities being developed for consideration as part of that most often required the most assistance with the government’s commitment to economically major incidents or investigations, and they might sustainable policing. not be able to afford to pay for such help. The recovery of costs for providing police service to municipalities that receive policing RECOMMENDATION 7 from the OPP is governed by the regulations To promote better relations with, and consist- under the Police Services Act. The OPP and ent services to, municipalities, and fairer Ministry report annually on costs in accordance and more transparent billing processes, the with these regulations. In addition, the OPP Ontario Provincial Police, in conjunction with brings forward options for additional cost cat- the Ministry of Community Safety and Cor- egories for inclusion in the cost-recovery model rectional Services, the Ministry of Finance and if they are found to be directly attributable and municipalities, should: material to the provision of services. AMO is

3.10 seek ways to simplify, and make more aware of this practice/policy and Police Service • transparent, its cost-recovery methods Boards are also made aware through their

Section and consider whether various grants and biannual conferences.

credits should be amalgamated into one all- OPP senior management is open to the con- VFM

•

encompassing costing formula; cept of charging municipalities for some services 3 • address the issues in its costing and billing that the OPP provides to municipal police forces, methods that result in municipalities paying depending on a variety of factors, particularly Chapter different rates and consider phasing in cost where core police services are provided that increases over time rather than when con- would normally be routine business for a force. tracts are renewed; and • consider establishing a policy that would require identifying all costs for providing USE OF VEHICLES services to support municipal police forces as well as the proportion to be recovered. The OPP operates a fleet of about 3,800 vehicles— 1,250 assigned to officers or civilian staff and the OPP RESPONSE rest designated as patrol and pool-usage vehicles. In 2008, to comply with Canada Revenue A comprehensive document explaining the Agency regulations, the OPP notified officers and cost-recovery formula was prepared in August civilian staff that they would incur a taxable benefit 2012 in collaboration with the Association of when a vehicle assigned to them was used for Municipalities of Ontario (AMO) and three personal reasons, such as routine commuting from municipal Chief Administrative Officers. This home to work. In March 2009, the OPP approved document was shared with every municipality a list of job descriptions and positions that were that receives policing services from the OPP. deemed exempt from a taxable benefit. Ontario Provincial Police 251

The OPP calculates the amount of taxable ensure that its processes result in compliance benefit to be reported on employee T4 income slips • with tax laws that require that any signifi- using its fleet bureau system, in which drivers are cant personal use of vehicles be reported as to record their personal mileage each month. Man- employee income. agers are responsible for reviewing the fleet bureau entries at the end of the year to ensure employees OPP RESPONSE are properly reporting their mileage. In late 2011, the OPP began a review to see if The OPP agrees that the vehicle management it could operate more efficiently by cutting down process implemented in 2008/09 is not meeting on the number of vehicles it assigned to officers current needs. A project has been established and civilian staff and to examine whether taxable to review the current system and improve the benefits were being properly administered. The management of vehicle taxable benefits. As well, review, completed in February 2012, found that all discrepancies between the fleet management poor record-keeping and problems with various database and the physical vehicles have now processes resulted in the following: been resolved. New processes and policies will • The whereabouts of 200 vehicles listed in the enable the OPP to effectively and efficiently fleet management database were not readily manage vehicle assignments and ensure compli- known, since the vehicles were not listed as ance with tax reporting requirements. assigned to a region or investigations bureau. • The list of vehicles assigned to staff could not 3.10 be located and had to be recreated. DETACHMENT INSPECTIONS • The approved list of job descriptions and pos- Section itions deemed exempt from a taxable benefit The Police Services Act stipulates that all police VFM had not been reviewed and updated since forces have quality assurance processes in place for •

3 it was released in March 2009. The review the delivery of adequate and effective policing. The also identified 40 staff listed as exempt, but OPP’s Quality Assurance Unit uses two processes for

whose current positions did not qualify for the conducting audits for this purpose: Chapter exemption. • It provides a Management Inspection Process • No personal mileage was reported by more (MIP) questionnaire to each detachment com- than half of the 640 non-exempt staff who mander three times a year to assess whether were assigned vehicles. the detachment is meeting the standards OPP staff informed us that, as of September of the Police Services Act and the OPP’s own 2012, they had accounted for all vehicles and issued policies. amended T4 tax slips to the 40 employees whose • It conducts an on-site physical inspection of positions did not qualify for an exemption. all detachments every two years (three years up until mid-2011) to verify the information RECOMMENDATION 8 provided in the MIP questionnaire; audit high-risk areas such as vaults that hold seized To help adequately manage and control the use firearms, drugs and cash, and storage of the of vehicles, the Ontario Provincial Police should: detachment’s armaments; and examine the improve its record-keeping and other pro- • detachment’s financial processes. The Unit cesses for tracking inventory and assigning sends its findings to the detachment com- vehicles and capturing personal-use mander and to management at the regional mileage; and headquarters and at General Headquarters. 252 2012 Annual Report of the Office of the Auditor General of Ontario

We expected that with these quality assurance bags, where documentation for items was processes in place, relatively few problems would missing, and where only one signature was be found at the detachments. However, inspection obtained for disposal of illicit drugs when two reports that we saw often noted that detachments signatures are required. At one detachment a were not complying in a number of areas and were civilian staff member was assigned to manage told to correct deficiencies. Similarly, we did our the storage vault, which is contrary to policy. own inspections at eight detachments and found a • In some cases, officers’ handguns were stored number of deficiencies, even though, just as with in locked firearms lockers that did not have inspections by the Unit, the detachments were noti- the required locking cable or an approved fied in advance of our inspections. The problems lock. As well, we found off-duty officers’ pep- we found at the detachments included missing per spray canisters were not in handgun stor- documentation, the use of improper signatures for age lockers as required by policy. authorizations, and poor controls over evidence and OPP weapons. For example: RECOMMENDATION 9 We noted several cases where physical counts • To ensure that detachments meet legislative and of seized items in the vaults did not match policy requirements for ensuring the security the inventory listings in the computerized and integrity of seized cash, drugs and firearms, Records Management System (RMS). At one and detachment weapons, the Ontario Provin- detachment, the staff could not provide an cial Police should: inventory listing from RMS for seized cash,

3.10 reassess its quality assurance processes and drugs and firearms, and instead were using • increase senior management oversight of a manually prepared ledger that lacked the

Section results to identify ways to make inspections controls in the RMS system. We found three more effective, including the periodic use of VFM cash court-evidence exhibits in the vault that •

surprise inspections to promote sustainable 3 were not recorded in the ledger. At two other compliance; and detachments, the physical count of seized fire- make detachment commanders more

Chapter arms did not match the RMS inventory listing; • accountable for ensuring that actions have in one of the cases, the vault contained five been taken to correct any deficiencies noted. fewer seized firearms than the RMS listing. In several other detachments, the required OPP RESPONSE records of items placed in and removed from the vaults were not always maintained or the The OPP has made a strong commitment to items were improperly documented. It was ensuring that Police Orders and legislative also often not clear who had access to court requirements for policing standards are consist- evidence in the vaults. ently followed. The OPP has investigated and • Several transfers of seized cash out of detach- resolved the discrepancies identified by the ments were made without acknowledgment Auditor General’s review. Through this review, from the recipients. At one detachment, the regions have identified accountability, train- $7,500 was transferred to another OPP inves- ing and policy clarity as areas that will improve tigations bureau without proper signatures, seized asset management. The OPP will review and an exhibit of $1,700 in seized cash could the suggestions and implement new processes not be located in the vault. where required. • We found cases in which seized items were The OPP’s Quality Assurance (QA) unit secured in non-approved exhibit storage works in partnership with the regions to Ontario Provincial Police 253

tell OPP staff about the new model and train them continually review and improve QA processes. accordingly, and to let communities know about it. Furthermore, substantial changes were made On our visits to eight detachments, we asked after the Auditor General’s recommendations in commanders, sergeants and constables whether, or 2005 and follow-up report in 2007. what, they knew about the new community-policing Currently, the Deputy Commissioner of Field model. Some detachment members knew about the Operations and each Regional Commander new model and had even received some training receives a summary of all findings. The QA unit or had attended awareness sessions; others were works with the detachments and the region to unaware of it. We also checked to see whether the ensure all the issues are resolved in a timely fash- community-policing committees still existed. In ion. If an issue is systemic or involves a risk that is three jurisdictions, the committees still met, and in not resolved, the QA unit will inform the Deputy another three, the committees existed in some form, Commissioner. In most cases, errors are related but had not met recently or were being redeveloped. to documentation or process discrepancies. In two detachments, there were no committees. The OPP will continue to reassess its quality In our 1998 and 2005 audits, we reported that assurance processes and ensure detachment the community-oriented policing program needed commanders are accountable. to have better management oversight and a process to measure the effectiveness of the program against established criteria. We found during our EFFECTIVE POLICING current audit that these observations had not been 3.10 addressed. Community-oriented Policing Section The Police Services Act requires the Minister of Com- Results-Driven Policing VFM munity Safety and Correctional Services to develop •

3 and promote programs for community-oriented In 2006, the OPP introduced the Results-Driven police services. As well, the Act establishes min- Policing Accountability Framework for improving

imum police service standards and requires every community safety through targeted crime and Chapter chief of police to establish specific procedures and traffic-enforcement initiatives. Using the framework, processes for community-oriented policing. detachments and regional headquarters collect and Historically, the OPP’s community-oriented analyze statistics on rates of certain crimes and the policing program involved regular meetings and numbers of road-related fatalities and injuries, and communications between a committee of local these statistics are discussed at quarterly meetings. citizens and members of the local detachment, dur- If, for instance, the statistics show an increase in ing which the group would deal with local concerns accidents on a certain highway, or a spike in certain about such issues as crime and traffic. The goal of types of crimes in a particular area, resources may be the program has been to improve community safety redirected to address the situation. Senior manage- and to involve the community in crime prevention. ment at General Headquarters reviews the Results- In 2010, the Ontario Association of Chiefs Driven Policing reports quarterly and, if necessary, of Police introduced a new community-policing requests corrective action. model, which, according to the 2011 OPP business This sort of response strategy is useful in that plan, was to be the foundation of the force’s crime- it helps detachments focus their resources on prevention efforts. The business plan indicates that improving tangible results, but the framework sets the OPP needed to make significant efforts through no targets against which detachments can meas- awareness sessions or other communications to ure their results and compare them with other 254 2012 Annual Report of the Office of the Auditor General of Ontario

detachments. Setting targets could allow the OPP and establish measures for assessing the to more effectively monitor and improve detach- effectiveness of these programs at individual ments’ performance. detachments; and • monitor average officer response times to Response Times calls for service for each detachment to ensure that adequate response times are Communication devices in OPP vehicles have a achieved, particularly for higher-priority button that officers use specifically to report to calls and during peak demand periods. the communications centre when they arrive at a scene, and officers are required to use this button. OPP RESPONSE However, unlike municipal police services and the RCMP, the OPP does not analyze information The OPP will develop a reporting mechanism for about how much time elapses between when a call community-oriented policing. The OPP Crime is received by one of its regional communications Prevention Section has begun communicating centres and when the officer arrives on the scene. with regions on how Ontario’s Mobilization and Consequently, the OPP does not know whether Engagement Model of Community Policing can average response times are reasonable, particularly be used most successfully. The OPP has also for more serious or higher-priority calls. OPP established a tactical priority-setting process officials told us it would be impractical to set a enabling officers in the OPP and other police response-time standard for all calls because of sig- agencies to focus on severe crime problems 3.10 nificant geographic differences across the province. affecting communities. Nevertheless, keeping track of average response The Results Driven Policing (RDP) program Section times on a detachment-by-detachment basis would is an accountability process meant to identify VFM allow the OPP to monitor officer response times to and respond to issues quickly, rather than com- •

3 determine whether there were any detachments paring benchmarks against other detachments that may need more officers during certain periods. and regions. Significant work is done at the

Chapter The OPP compiled an informal sample of aver- detachment and regional levels regarding RDP age response times from one regional communi- statistics and in response to trends and emerging cations centre in 2011 and found that an officer issues. Best practices are identified and shared. arrived on the scene within 10 minutes in 78% of All of those results are in turn rolled-up and calls for service and within 40 minutes for 95% of reviewed monthly at OPP Executive Council calls. No other analyses of response times have been meetings at General Headquarters. The OPP is carried out, whether of individual detachments, currently developing a better corporate analysis regions or the province as a whole. framework and reporting process on the benefits and the successes/failures of RDP regional initia- RECOMMENDATION 10 tives. In the new year, a streamlined and more efficient RDP process will be implemented at the To help ensure that police resources are focused detachment, regional and provincial levels. on the Ontario Provincial Police’s key objectives The OPP will investigate ways to efficiently for effective policing, the Ontario Provincial monitor response times and has set a target to Police should: provide reports and analysis to senior manage- improve the reporting to management on • ment by the end of 2013. the community-oriented policing program and the results-driven policing program, Ontario Provincial Police 255

INFORMATION SYSTEMS was only partially implemented, and a supervisor’s approval is required only when officers record over- Officers use two separate OPP systems on a daily time hours. Supervisors are also required to ensure basis to record their activities: weekly that officers have updated DAR, but they The Records Management System (RMS) • do not check or approve the officers’ data entries. tracks occurrences, such as those resulting By comparison, the Sûreté du Québec requires that from calls for service, and permits analysis supervisors approve officers’ activity reports on a of case-related information, such as types of daily basis. crime committed, location, people and prop- There is data overlap between the occurrences erty involved, witness statements and officers’ recorded in RMS and the calls for service recorded notes. Officers are required to update RMS in DAR, with between 50% and 80% of similar data after each occurrence. in both. Since there is no linkage between the two The Daily Activity Reporting system (DAR) • systems, data has to be manually entered twice. is, primarily, a time-accounting system that The OPP does not have any processes that regularly tracks an officer’s regular and overtime work compare or reconcile the information in RMS with hours, the number and types of calls for that in DAR. Our own analysis comparing the similar service to which an officer responds, and how data in the two systems for 2011 noted a significant much time each officer spends on activities difference between the 635,000 calls for service in such as traffic patrol, Criminal Code investiga- DAR and the approximately 815,000 occurrences tions and administration. Officers are required reported in RMS. For example, some of the differ-

to update DAR daily. 3.10 ences in the number of occurrences or calls for ser- Officers, administrators and local and provin- vice between RMS and DAR were approximately as

cial governments rely on the RMS, which holds Section follows: 4,000 thefts, 1,200 assaults, 13,000 alarm about 11 million records, to provide accurate data VFM responses, 26,000 police assistances, 130,000 traffic • quickly. For example, front-line officers use the 3 incidents and 6,000 incidents relating to compulsory RMS to look up records of known violent offenders, auto insurance. The general view among the officers vicious animals or high-risk residences, as well as

we spoke to at the OPP is that the occurrences data Chapter to access a suspect’s criminal background and other from RMS are more reliable than the calls-for-service vital data. Organizationally, the OPP uses RMS to data in DAR; however, it is the DAR data that is compile crime statistics for policing initiatives such published in annual reports and used for staffing as community-oriented and results-driven policing, deployment models. and uses DAR for reporting publicly on the number of calls for service to which it responds. RECOMMENDATION 11 The OPP has acknowledged concerns about the quality of data in RMS. Random internal audits To help ensure that its two key information sys- have revealed a number of data-quality issues, tems contain accurate information that can be including the fact that occurrence information is reliably used for managing and reporting on its often incompletely or poorly documented, and that policing activities and on crime and traffic occur- occurrences that are already recorded in the system rences, the Ontario Provincial Police should: do not show up during RMS searches by officers. • assess the extent to which the Records Data integrity problems are more prevalent in Management System and Daily Activity DAR. In 2005, we recommended that the OPP insti- Reporting systems do not reconcile with tute procedures to require supervisory officers to each other for critical data such as occur- oversee the completeness and accuracy of the infor- rences and calls for services; mation entered into the system by officers. This 256 2012 Annual Report of the Office of the Auditor General of Ontario

accidents, it does not report on its own efficiency. consider whether periodic supervisory • By comparison, municipalities and their police approval of officers’ daily or weekly data forces typically report on their operating costs for input would help minimize inconsistent and police services on a per capita or per household inaccurate data between the two systems; and basis. Other police forces set targets for achieving on a longer-term basis, assess the cost/ • efficiency levels, such as times for its communica- benefit of system changes that would enable tions centre to respond to an emergency call and officers to enter information such as occur- officer time lost to illness. In addition, the OPP does rences and calls for service only once to not include in its annual report the percentage of update both systems. time its officers spent on administrative duties. In its 2011–13 Strategic Plan, the OPP identified OPP RESPONSE four goals: As the Auditor General recommends, the OPP • public safety, with emphasis on excellence in has initiated work to identify where RMS and delivering core police services; DAR data do not reconcile. Particular attention is • strong, effective partnerships with commun- being paid to the areas of occurrences and calls ities, stakeholders and colleagues; for services. Simplification of system input and • a sustainable workforce that demonstrates the elimination of duplication is a specific deliv- expertise, pride and dedication; and erable of a project established to review the DAR • efficiency and effectiveness in an increas- system. The OPP is exploring opportunities for a ingly complex and challenging policing 3.10 customizable off-the-shelf product developed by environment. the private sector, and estimates that a product The force established performance indicators Section will be procured within the next 18 months. for each goal, but very few are measurable based VFM In addition, the OPP will develop policy and on actual performance, and no targets have been •

3 procedure around regular review and signoff set. For the efficiency and effectiveness goal, for for officer-entered data. The expansion of civil- instance, the performance indicators include having

Chapter ian data entry into RMS, which is to begin in “uniform workloads statistics,” “offering financial 2013/14, is expected to result in improved data training opportunities,” and “response to various entry because of direct supervision and oversight. inquests, inquiries and audits.” In its 2010 annual report, the OPP reported on results pertaining to effectiveness achieved on its 2008–10 Strategic Plan, including a list of activities completed, but PERFORMANCE MEASUREMENT no measurable results were provided on the cost- AND REPORTING effectiveness of its operations. The OPP uses its annual report to publicly give The OPP conducts a community satisfaction details about its programs, services and activities; survey every two years. In its last annual report, its three-year strategic plan; motor-vehicle fatalities the force said the 2009 survey found that 95.4% of and injuries; and crime levels and clearance rates. Ontarians felt safe or very safe in their communities. Crime statistics are reported in the same uniform The results were based on interviews with the gen- reporting format used by police forces across eral public in specific communities about their views Canada. As of August, 2012, the most recent OPP of the OPP. However there was no way to ascertain annual report covered the 2010 calendar year. whether respondents actually had any contact with While the OPP reports extensive information the OPP. We noted that the RCMP annually reports on its policing activities, crime and motor-vehicle on the percentage of clients it dealt with who are Ontario Provincial Police 257 satisfied that the force is providing high-quality ser- and operational efficiency within police agen- vices. In 2010/11, the RCMP’s target was 80% and cies. The OPP will ensure that the targets it sets the actual performance was 79%. for its goals are measurable. The annual OPP Community Survey includes RECOMMENDATION 12 questions on effectiveness as well as questions While the Ontario Provincial Police provides specific to residents who have had recent contact good information on crime rates and its activ- with the force. The results are analyzed inter- ities and services, additional information to nally. The OPP agrees with the Auditor General’s enable the public to assess its cost effectiveness recommendations on reporting the results and and operational efficiency is needed. It should will post its 2012 survey results on the public also periodically and independently survey com- Internet site. Also, detachment survey results are munity residents who have had recent contact available to Detachment Commanders to share with the force to determine their satisfaction with their communities. with the service they received.

OPP RESPONSE

The OPP will research best practices with regard to measuring and reporting on cost-effectiveness 3.10

Section

VFM

•

Chapter Chapter 3 Ministry of Finance Section 3.11 Tax Collection

behalf of the province, also began administering Background Corporations Tax on behalf of the province in Janu- ary 2009. Similarly, in July 2010 the Harmonized Sales Tax, also administered by the CRA, replaced Taxes are the province’s largest source of revenue. the provincial Retail Sales Tax. As a result, approxi- The Ontario Ministry of Finance (Ministry),

3.11 mately 75% of the Branch’s staff of almost 400 were through its Collections Branch (Branch), is transferred to the CRA effective March 2012. How- responsible for collecting a significant portion

Section ever, the Ministry remains responsible for collecting of the unpaid taxes owed to the province. The Corporations Tax and Retail Sales Tax amounts VFM Branch’s head office is in Oshawa, with regional • owing prior to the transfer of the administration of 3 and district offices in Toronto, London and Ottawa. these taxes to the CRA. The Ministry expects that by The Branch’s collections process involves initially 2014 it will have wound down most of its work on

Chapter sending notices by mail, contacting the taxpayer by collecting these amounts. phone, and sometimes visiting in person. If these In the 2011 Ontario Budget, the government actions fail to recover amounts owing, collectors proposed centralizing the collection of all govern- have other means at their disposal, such as garnish- ment non-tax revenue within the Ministry of ments or registering liens and warrants for seizure Finance. Under this proposal, the Ministry’s Collec- and sale of the taxpayer’s property. tions Branch would continue to collect taxes that Accounts are prioritized on the basis of risk, it administers, but would also become responsible using criteria such as the amount owing, number of for collecting non-tax revenue on behalf of other times or length of time in collections, whether there provincial ministries. At the time of our audit, the is a history of broken promises, and if any legal Ministry of Agriculture, Food and Rural Affairs action has already been taken. Once their priority was the first ministry scheduled to transfer non-tax has been set, the accounts are assigned according to receivables—in this case, overpayments to crop collectors’ seniority and experience. producers totalling approximately $29 million—to As of March 31, 2012, approximately 90% of the the Ministry of Finance in fall 2012. taxes owing that the Branch was responsible for collecting related to Corporations Tax and Retail Sales Tax. The Canada Revenue Agency (CRA), which is responsible for collecting personal income tax on

258 Tax Collection 259

accounting firm to assess and identify the structure Audit Objective and Scope of a centralized collection service as proposed in the 2011 budget. We also reviewed recent reports on effective Our audit objective was to assess whether the Min- collection practices from associations that represent istry of Finance (Ministry), through its Collections private collection agencies in North America. Branch (Branch), had adequate systems, policies and procedures in place to: • maximize the recovery of taxes owed to the province; and Summary • reliably measure and report on the effectiveness of these collection efforts. Over the last five years, the province has generated Ministry senior management reviewed and about $330 billion in taxation revenue, of which the agreed to our objective and associated criteria. Ministry of Finance’s Collections Branch (Branch) The Branch’s head office in Oshawa is respon- collected about $6 billion, with the majority of sible for accounts representing about 80% of the the remaining amount being remitted voluntarily. taxes that the Branch is responsible for collecting. Although some write-offs are to be expected in any Our audit, therefore, focused on a review and collection process, the Branch expects that it may analysis of relevant collection accounts at this need to write off up to $1.4 billion of the $2.46 bil- office, its administrative policies and procedures, lion in taxes owing to the province that the Branch

as well as discussions with appropriate staff. 3.11 was responsible for collecting as of March 31, However, we also reviewed a number of accounts 2012. The $1.4-billion amount is predominantly

assigned to collectors in the Branch’s regional and Section made up of older accounts that have accumulated district offices. VFM over a number of years, and it has been previously • The scope and extent of our audit work also 3 expensed in the government’s financial statements. took into account several recent initiatives by the In recent years, the Branch has taken some Ministry. The Branch hired a consulting firm in

initiatives to strengthen its collections process. In Chapter 2009 to review its operations and recommend best 2008, collection activity for all tax statutes except practices and potential collection tools from juris- Corporations Tax and Self-employed Health Tax dictions similar to Ontario. In 2009, the Branch also migrated to a new system called OntTax. With co-founded the Inter-Jurisdictional Tax Operations the implementation of OntTax, the Branch has Network (ITON), whose membership includes 11 improved how it prioritizes and assigns accounts to American states and seven Canadian provinces, to collectors. The Branch has also recently developed share knowledge about the collections process. As guidelines to assist collectors in carrying out and part of the audit, we reviewed and assessed best documenting collections activities, as well as imple- practices identified through ITON surveys and the menting a tool that analyzes an account’s collection external review commissioned by the Branch. In history to predict the likelihood of payment. August 2010, the Ministry’s Internal Audit Division To understand why the Branch needed to write conducted a review of the collection processes off a significant amount of taxes owing, we exam- of accounts deemed to be low-risk. This division ined the collection process both for active accounts also annually reviews the Ministry’s allowance for and for those that the Branch was considering writ- doubtful accounts and the write-off process. And ing off. We found that in most cases we reviewed, in 2011, in anticipation of an expanded role for its the collection actions taken were not timely and Collections Branch, the Ministry commissioned an the enforcement tools available were not utilized 260 2012 Annual Report of the Office of the Auditor General of Ontario

fully. Some of our more significant observations are to Branch guidelines, multiple interim pay- as follows: ment arrangements that covered only a small • Taking prompt action is vital in collecting portion of the total debt had been in place for debts. Research shows that the probability extended periods in many of these accounts. of full collection on a delinquent account • The Branch did not always make full use of drops dramatically as time passes. Our work partnership and information-sharing agree- indicated that once an account entered collec- ments with third parties. For example, the tions, it took an average of seven months for Branch may seek to have a delinquent tax- collectors to attempt to reach the taxpayer by payer’s motor vehicle dealer or liquor licence phone. We also noted that in more than two- suspended or revoked. We noted cases where thirds of the cases in our sample, there was at the Branch did not request such action on a least one instance where no collection action timely basis, or at all, after normal collection was taken for six months or more. efforts had been exhausted. • Visiting the taxpayer’s premises will often Corporations Tax and Retail Sales Tax make increase the likelihood of collecting the up 90% of the Branch’s tax-receivable inventory. amount owing. Field visits were warranted Responsibility for administering these taxes was but were not made in a number of accounts recently transferred to the Canada Revenue Agency that we reviewed. For example, the Branch (CRA), but responsibility for collecting the amounts tried unsuccessfully for nearly two years owed to the province pre-transfer remains with to reach by phone a taxpayer who owed the Branch. As a result of the transfer, the Branch 3.11 $100,000 in Retail Sales Tax and had broken recently lost three-quarters of its workforce, includ- a payment arrangement. Making prompt field ing managers, collectors and support staff. Of the Section visits to the taxpayer’s retail or business loca- remainder, more than one-third is management and VFM tion would be a more effective strategy, espe- support staff, and the caseloads of collectors have •

3 cially when significant amounts are involved. in many cases doubled and in some cases tripled. • The Branch had in most cases registered liens This may result in even more write-offs than cur-

Chapter and warrants on properties appropriately rently expected. The Branch received approval in when such action was considered necessary. the 2009/10 fiscal year to hire temporary employ- However, in a number of the cases that we ees to compensate for the loss of its personnel to the reviewed, the Branch did not enforce warrants CRA. At the time of our audit, the Branch had not for the seizure and sale of property because fully evaluated its post-transfer staffing needs and, of concerns for the viability of the taxpayer’s as a result, no additional staff had been brought business and the impact on the taxpayer’s abil- on board. ity to pay outstanding amounts. Even when In order to oversee collection activities effect- businesses had ceased to operate, we noted ively, managers should have access to sufficient and cases where the Branch still failed to enforce timely operational and performance information. existing warrants to determine whether any However, we found that reports produced by the assets remained to help pay the debt. Ministry’s information systems did not adequately • The Branch may arrange interim payment support the oversight of the collection function. The plans if the taxpayer has outstanding returns Branch’s performance measures are also not suf- to file or needs time to determine a permanent ficient to properly evaluate collection efforts at the payment arrangement. Payment arrange- branch level and at the individual collector level. ments were in place in nearly half of the accounts that we reviewed. However, contrary Tax Collection 261

OVERALL MINISTRY RESPONSE Detailed Audit Observations The Ministry appreciates the recommendations made by the Auditor General to improve the col- The province generated approximately $330 bil- lections function as we move toward the goal of lion in taxation revenue over the last five years, of centralized collections of tax and non-tax debt, which the Ministry of Finance’s Collections Branch as proposed in the 2011 Ontario Budget. (Branch) collected about $6 billion, with most of Of the $1.4 billion in write-offs referenced the remaining amount being remitted voluntarily. in the Auditor General’s report, $600 million As of March 31, 2012, the total amount of the has been approved for write-off in 2012. Of taxes owing that the Branch was responsible for the remaining $800 million, approximately recovering was $2.46 billion. However, of this total, $400 million is in bankruptcy/insolvency pro- about $680 million has been deemed uncollect- ceedings subject to federal legislation and from ible and therefore is to be written off. Staff are which Ontario may receive some future pro- focusing their collection efforts on the remaining ceeds; $200 million is still in active collection; $1.78 billion in active accounts, of which the Branch and the remaining $200 million is slated for estimates that another $720 million may need to be write-off because progressive collection action written off. has been exhausted. Furthermore, prior to Corporations Tax and Retail Sales Tax make writing off accounts, the Branch registers legal up most of the active amounts owing, as shown in actions (personal property liens and warrants

Figure 1. Because administration of these taxes has 3.11 of seizure and sale) to secure the Crown’s inter- been transferred to the Canada Revenue Agency, est should any undisclosed assets be identified.

the Branch’s current priority is to wind down the Section Historically, this has resulted in the recovery of existing pre-transfer inventory of accounts by 2015. VFM monies from written-off accounts. •

3 In recent years, the Collections Branch has Figure 1: Active Amounts Owing by Tax Statute as of worked to enhance its processes, tools and best March 31, 2012 practices to improve the collections process. For Chapter Source of data: Ministry of Finance example, the tax administration system, Ont- Tax, implemented in November 2008, provides Retail Sales Tax, $641 (36%) the functionality for an automated monthly Employer Health Tax1, $71 (4%) statement for all accounts with a balance, along Other2, $107 (6%) with automated letters detailing specific actions Corporations Tax, $964 (54%) on the account to encourage debtors to make contact with the Branch and resolve their tax obligations. The Branch has also introduced data analytics and performance measures, and initiated and led regular meetings of an inter- jurisdictional group of North American collection administrators to identify and leverage best practices specific to collections operations.

1. Includes Self-employed Health Tax (SEHT). 2. Includes Motor Fuels and Tobacco Tax, Land Transfer Tax, Mining Tax, Beer and Wine Tax. 262 2012 Annual Report of the Office of the Auditor General of Ontario

For the most part, it will assess the need to write Figure 2: Collections Process off accounts that are old and where the likelihood Source of data: Ministry of Finance of collecting is low, and then focus efforts on the remaining accounts. Account enters collections when payment is in default

COLLECTIONS PROCESS System-generated collection letter is sent Overview

The flowchart in Figure 2 illustrates the process that the Branch typically follows in collecting Attempts to contact taxpayer may be made amounts owed. The timeliness of collection actions is the most important factor in successful recovery of amounts Final notice is sent owing. A recent survey of its members conducted by an association that represents commercial collectors responsible for 80% of the claims in the United Personal property (e.g. vehicles, furniture, equipment) liens States found that the probability of full collection and warrants may be registered for future seizure and sale on a delinquent account drops dramatically as time passes. Its survey results indicated that after only

3.11 three months, the probability of collecting on an

Real property (e.g., land and buildings) liens account drops to less than 70%; after six months, may be registered and garnishments may be issued

Section collectibility drops to about 50%; and after a year, it

drops to a little over 20%. After two years, the prob- VFM

• ability of full collection on a delinquent account 3

Liquor or motor vehicle dealership licences is less than 10%. The association noted that other may be suspended or revoked where applicable keys to successful collections include keeping credit Chapter records, such as addresses and contact information, current; periodically reviewing collection policies

Warrants for seizure and sale of properties may be enforced and procedures, as well as rigid adherence to these policies; and ensuring that commitments of a final payment accompany all partial payments. As shown in Figure 3, as of March 31, 2012, only In cases where a corporation has insufficient assets or has filed for bankruptcy, the directors about $390 million or 22% of the active amounts of a corporation may be held liable for amounts owing were less than a year old, while approximately $925 million or more than half were two years old or more. Due to the lower success rate in collecting Account can be recommended for write-off older accounts, the Branch estimates that $720 million (mainly consisting of Retail Sales Tax and Corporations Tax) of these active amounts owing Note: At any point in the process, a collector is encouraged to attempt to negotiate a will also need to be written off. Combined with the payment arrangement, and can: • obtain security, e.g., in the form of a bank guarantee or a letter of credit; more than $680 million already scheduled to be • conduct a field visit to, e.g., verify taxpayer information, review books and written off, this means that the Branch will need records, or inspect assets; or • refer the account for an audit. If fraud is suspected, the account may be to write off $1.4 billion of the $2.46 billion in taxes referred for special investigation. owing to the province as of March 31, 2012. Tax Collection 263

initial contact, in more than two-thirds of Figure 3: Age of Active Accounts Receivable as of the accounts that we reviewed there was at March 31, 2012 least one instance where the account was not Source of data: Ministry of Finance touched for six months or more. The average Active A/R length of time where there was no activity on Age ($ million) % the account in these cases was 11 months. Less than 1 year 391 22 Visiting the taxpayer’s premises is often neces- 1 year to less than 2 years 361 20 • sary to verify information about the client 2 years to less than 4 years 343 19 and the state of the business and, in some 4 years to less than 10 years 451 25 instances, to determine whether there might 10 years and older 131 8 be assets that could be used to help recover Subtotal 1,677 amounts owing. The 2009 external review of Other* 107 6 Branch operations noted that field visits early Totals 1,784 100 in the collection process add the most value * Includes accounts whose age could not be determined. when businesses are still in operation and assets remain in place, but that staff tend to exhaust all options from the desk before pro- Collection Activities gressing to a field visit. Indeed, in a number The Branch has taken steps in recent years to better of cases that we reviewed we felt that field guide collectors through the collections process by, visits were warranted but were not under- 3.11 for example, developing procedures for activities taken. For instance, in one case the Branch such as carrying out searches and registering liens negotiated a payment arrangement of $2,400 Section and warrants on properties. The Branch has also a month in October 2009 with a client who VFM established standards to ensure that pertinent owed $100,000 in Retail Sales Tax. However, •

3 information from collection activities is docu- after paying only a little more than $5,300, mented fully and properly. the client made no further payments. Over

To understand why the Branch’s collection the next 22 months, the Branch attempted to Chapter efforts have not been successful in recovering a reach the taxpayer by phone without much significant amount of taxes owing, we examined success. An immediate field visit would have the collection process through a review of the active been a more effective strategy for attempting accounts as well as accounts that the branch has to establish contact with the taxpayer sooner. deemed uncollectible. In most cases, collection The taxpayer finally contacted the Branch in actions should have been initiated sooner and the February 2012 only because the renewal of available enforcement tools used more effectively. the establishment’s liquor licence was in jeop- • The sooner that collectors establish contact ardy due to the debt. At the time of our audit, with a delinquent taxpayer, the greater the this amount was still outstanding. chance of collecting on that liability. In the • The Branch may take legal actions such as accounts that we reviewed, after the system registering liens on properties and warrants sent collection notices automatically, it took for their future seizure and sale if the collector an average of seven months for collectors to is unable to contact the taxpayer or identifies attempt to reach the taxpayer by phone. In a risk of loss. The Branch may also take these several cases this initial contact was made legal actions when a payment arrangement two to five years after the account entered cannot be negotiated with the taxpayer or an collections. We also noted that after the existing agreement is broken. For the accounts 264 2012 Annual Report of the Office of the Auditor General of Ontario

that we reviewed, we noted that the Branch of time. Interim payment plans can also be had in most cases registered liens and war- arranged if a taxpayer has outstanding returns rants on properties appropriately when such to file or needs time to determine a permanent action was considered necessary. However, we arrangement. According to Branch guidelines, noted in a number of the cases we reviewed interim payment plans are only a short-term that the Branch had not enforced warrants for measure and do not satisfy the taxpayer’s seizure and sale of properties, for fear of put- outstanding liability to the province. Payment ting the taxpayer out of business and thereby arrangements were in place in nearly half reducing any opportunity to collect amounts of the accounts that we reviewed. However, owing. While we agree that collectors must multiple interim payment arrangements that exercise judgment in enforcing warrants, we covered only a small portion of the total debt noted that in several instances businesses had had been in place for extended periods, which ceased to operate but the Branch still failed was contrary to Branch guidelines. According to enforce existing warrants. For example, to the 2009 external review of the Branch, in one case a restaurant owner broke a pay- some U.S. jurisdictions use automated debit ment arrangement after paying only 5% of payments, which have been found to signifi- a $522,000 tax debt. After several attempts cantly reduce the default rate on payment in 2011 to contact the taxpayer, the Branch arrangements. At the time of our audit, the learned that the company had closed its bank Branch had not formally considered adopting accounts. While the Branch did revoke the this practice. 3.11 restaurant’s liquor licence, it did not enforce warrants that had already been registered for Section

Use of Third-party Information the seizure and sale of property even though VFM there may well have been assets available to The use of third-party information to corroborate •

3 help pay the debt. taxpayer information, locate an individual or busi- • Directors of companies can be held liable for ness, or obtain other data is a critical component

Chapter certain tax amounts owing if previously regis- for successful collection. tered warrants are enforced and no assets are Once an account enters collections, the Branch found, or if the company declares bankruptcy has access to the Ontario Business Information and the Branch files a proof of claim against System maintained by the Ministry of Government its assets within six months. In several cases, Services, to the Ministry of Transportation’s driver we noted that the Branch could not pursue and vehicle licensing and registration databases, this option because it had not enforced the and to the Municipal Property Assessment Corpora- warrants it had registered on properties. One tion’s database. Although we found that the Branch company that owed $384,000 in taxes filed uses these databases to search for individuals and for bankruptcy, but the Branch failed to file businesses and any related assets, it did not always a proof of claim in time and therefore lost commence the searches on a timely basis. In our any opportunity to recover taxes owing by review of a selection of accounts, the average this method. length of time between an account entering collec- • In certain cases where the taxpayer is will- tions and the first search was 12 months. ing to pay amounts owing but cannot do so For a number of years, the Branch has had immediately or in full, the Branch may negoti- partnership agreements with some provincial ate an agreement with the taxpayer for the agencies and the Canada Revenue Agency (CRA) payment of the amount owing over a period to assist its collection effort, as shown in Figure 4. Tax Collection 265

Figure 4: Partnership Agreements for Assisting the Collection Effort Source of data: Ministry of Finance

Agency Year Effective Purpose of Agreement Alcohol and Gaming 2003 Ensures that applicants will not be granted a liquor licence renewal or be Commission of Ontario (AGCO) able to transfer an existing licence until they resolve any tax arrears or tax returns in default. Ontario Motor Vehicle Industry 1996 Allows the Ministry to recommend that a motor vehicle dealer’s right to sell Council (OMVIC) in Ontario be revoked if they fail to resolve any tax arrears. Canada Revenue Agency (CRA) 1982 Revenue sharing: Enables sharing of proceeds where both the CRA and the Ministry have taken collection action against a common debtor. 1998 Revenue set-off: Allows for set-off of revenue from the CRA to recover debts owed to the Ministry, and vice versa.

However, we found the Branch did not make full taxpayers. Such information could be used to cor- use of the avenues available through its partnership roborate details that the Branch has on file or to agreements. In several cases where taxpayers held identify taxpayers’ other sources of income. The a vehicle dealer registration issued by the Ontario Branch acknowledged that the CRA exchange of Motor Vehicle Industry Council (OMVIC) or a information is a valuable tool that is available for liquor licence issued by the Alcohol and Gaming use during the collection process, but that it is not Commission of Ontario (AGCO), the Branch did used routinely.

not initiate having the licences revoked on a timely 3.11 basis, or at all, after normal collection efforts had Out-of-province Accounts

been exhausted. For example, the Branch warned Section a liquor licensee who owed $1.1 million in Retail At the time of our audit, a Branch analysis revealed VFM

• Sales Tax that it would apply to the AGCO to have that more than $320 million in taxes was owed by 3 the licence suspended or revoked if the taxpayer individuals and businesses whose mailing address did not contact the Branch within 30 days. For was outside Ontario. According to the Branch, approximately six months after receiving the while it can follow up with these debtors by send- Chapter warning letter, the taxpayer did not respond, yet ing notices or by telephoning, it cannot initiate the Branch did not begin proceedings to suspend any legal actions such as garnishing bank accounts or revoke the liquor licence. Subsequently, after or registering liens and warrants on properties repeated attempts by the Branch over a one-and-a- if the accounts and properties are also out of the half-year period, the taxpayer agreed to a payment province. The Branch indicated that the current arrangement. However, the taxpayer defaulted legislative framework may not allow the Ministry after making only three payments totalling approxi- or a private collection agency that may be acting mately 5% of the debt outstanding. Again at this on its behalf to enforce debt outside the province. point, the Branch did not take any action to have It is exploring options including requesting legisla- the liquor licence suspended or revoked. Two years tive changes that would permit it to sign inter- after breaking the payment arrangement, the tax- provincial agreements to allow it to initiate more payer voluntarily surrendered the liquor licence just effective collection activities to recover its tax debts as it was about to expire. At the time of our audit, outside Ontario. this amount was still outstanding. The Branch also needs to make better use of information that the CRA may have on delinquent 266 2012 Annual Report of the Office of the Auditor General of Ontario

RECOMMENDATION 1 It is anticipated that expansion of this program will begin in March 2013. To maximize the recovery of amounts Ontario legislation is currently not enforce- owing, the Ministry of Finance’s Collections able in other jurisdictions. The Branch will Branch should: continue to work with legal services and inter- make initial contact with delinquent taxpay- • jurisdictional partners to explore options for col- ers sooner and carry out follow-up efforts, lecting debt from taxpayers outside Ontario. This including field visits, in a more continuous issue is to be discussed at the Inter-Provincial and timely manner; Territorial Tax Conference in September 2012. make better use of all available collection • Viable options will be proposed for consideration and enforcement tools, including partner- by decision-makers. ship and information-sharing agreements with other parties; and • continue to consider options, including obtaining any legislative authority that may Staffing be needed to allow it to initiate legal actions On July 1, 2010, the provincial Retail Sales Tax to collect debts from businesses and individ- (RST) was replaced by the Harmonized Sales Tax uals residing outside the province. (HST), which the CRA administers. As part of the amalgamation and transfer of the administration MINISTRY RESPONSE

3.11 of RST, the Branch in March 2010 negotiated an

The Branch will continue to enhance and agreement with the CRA in which all Ministry

Section strengthen its collection processes—for employees affected by the wind-down of the prov-

example, by developing best practices to guide incial RST received job offers from the CRA. As a VFM

• collectors in carrying out and documenting result of this agreement, in March 2012 the Branch 3 collection activities executed or not executed, lost three-quarters of its staff, including managers, and by improving timeliness of actions and how collectors and support personnel. Prior to the trans- Chapter work is prioritized based on risk. The Branch fer, the Branch had a workforce of 389 people—264 will take into consideration the right balance collection and insolvency officers and 125 manage- of initiating actions based on third-party agreement and support staff. ments that result in compliance, and actions Since the transfer, only 62 collection and that limit a business’s ability to remain oper- insolvency officers remain and are now responsible ational. The Branch will also initiate a review for the core collection function. Management and of its payment arrangements in the fall of 2012, support staff number 34 and make up more than and recommendations based on the results will one-third of the Branch’s workforce. Account case- be implemented by March 31, 2013. loads have in many cases doubled for most levels The Branch is continuing to dialogue with of collection officers since the transfer, and in some the CRA to enhance our information-sharing cases they have tripled. Such a significant reduction ability. Process improvements from this dialogue in the Branch’s workforce, especially at the collector will be implemented immediately. The Branch level, and the consequent increase in caseloads will is also working to expand the tender contract further hinder the effectiveness of collection efforts compliance program to the broader public sec- on accounts where the probability of recovery still tor to help ensure that grants are not paid out to exists, and may well result in even more write-offs businesses that owe a tax debt to the province. than expected at present. Tax Collection 267

Prior to March 2012, the Branch had a dedicated all other means of collections had failed. Similarly, Field Services Unit of eight field officers who sup- a study conducted by the Inter-Jurisdictional Tax ported the desk collectors by visiting businesses Operations Network, co-founded by the Branch, and residences to review clients’ records, inspect found that six of nine North American jurisdictions and appraise assets, and in some cases negotiate that were surveyed had considered outsourcing payment arrangements. The field officers were collections on a limited basis as a way to share the also responsible for attending AGCO and OMVIC workload and increase efficiency—for example, hearings for the possible suspension of licences on low-value accounts, accounts located outside when taxes were in arrears or returns were not the jurisdiction, and accounts where internal col- filed. This unit was eliminated with the transfer lection had yielded minimal results. Some of these of staff to the CRA; the field functions are now jurisdictions have had some success in using private handled by senior RST and Corporations Tax (CT) collection agencies on tax debts, but have required collectors whose caseloads have in many instances legislative authority identifying the information doubled, as indicated above. This combination that the jurisdiction could give the agencies prior of significantly higher caseloads and additional to outsourcing. responsibilities may result in even fewer site visits We should also note that the Branch still has to and additional delays in the desk collection func- determine the resources needed to fully analyze tion, further decreasing the likelihood of collecting outstanding defaults—that is, corporations that amounts owing. have not filed returns. Specifically, as of March 31, The Branch received approval in the 2009/10 2012, there were more than 200,000 CT accounts 3.11 fiscal year to hire up to 74 temporary staff up to with approximately 500,000 defaults. At the time March 2014. It had hired some temporary staff to of our audit, the Branch had not yet analyzed Section work until March 2012 to compensate for its loss of these accounts to identify the age of the defaults VFM staff as a result of the transfer of the administration or the filing history of the corporations to deter- •

3 of CT to the CRA. However, at the time of our audit, mine the likelihood of these defaults resulting in the Branch had not fully evaluated its needs related taxes payable.

to the amalgamation and transfer of the administra- Chapter tion of RST and, as a result, no additional staff had RECOMMENDATION 2 been brought on board. The Branch informed us in To mitigate the impact of the significant loss of August 2012 that this analysis had been undertaken its staff to the Canada Revenue Agency, the Min- and actions such as hiring temporary staff were istry of Finance’s Collections Branch should: under way. ensure that temporary staff hired to com- While other Ontario ministries use private • pensate for the loss continue to have the agencies to collect non-tax debt, the Ministry of appropriate skill set and experience to carry Finance’s Collections Branch does not outsource out collection duties effectively; any part of its collection function. The 2009 exter- reassess whether senior collectors, in addi- nal review of the Branch’s operations suggested • tion to their regular responsibilities, will that the Ministry evaluate the use of private agen- be able to carry out required field visits cies for collections. The review scanned practices effectively and on a timely basis, and attend in seven jurisdictions that had a similar mandate to hearings for the possible suspension of liquor that of Ontario and found that all used third-party and motor vehicle dealer licences, especially agencies to some extent in their collection efforts. given their recent significant increase in Typically, these jurisdictions used private collection caseloads; and agencies for low-value accounts or accounts where 268 2012 Annual Report of the Office of the Auditor General of Ontario

istered through a relatively new system called Ont- evaluate the use of private-sector col- • Tax. Prior to 2008, ICS was the only system used lection agencies for certain aspects of its for the administration of tax collection. In 2008, collections function. past collection activity for all statutes except CT In the longer term, the Branch should assess and SEHT was migrated onto OntTax. The Branch whether its current permanent staff comple- expects to decommission ICS once it has resolved ment is sufficient to maximize the collection of CT amounts owing. OntTax will continue to support non-tax receivables should its mandate expand the collection and administration of the remaining to include all government non-tax receivables. statutes, and it will be considered in supporting the collection of the province’s non-tax amounts MINISTRY RESPONSE owing if the Branch’s role is expanded as proposed The Branch is proactively addressing staffing for in the budget. its accounts receivable function in addition to An effective method of prioritizing accounts the Retail Sales Tax wind-down. Seventy-four based on the collection risk is important in deter- temporary staff are being recruited to address mining the appropriate collections procedures. the remaining accounts receivable inventory For example, high-risk accounts should be subject to maximize collection efforts and recoveries. to more aggressive action, whereas a more mod- Work will be rebalanced by December 2012. erate approach could be used for accounts that The temporary staff will be in place until are likely to be paid. With the implementation March 2014. of OntTax and its risk-scoring methodology, the 3.11 The Branch had previously considered the use Branch has significantly improved how it prioritizes of private-sector collection agencies and deter- accounts held in this system and assigns them to Section mined that it was not a cost-effective or viable the appropriate level of collectors. The Branch has VFM option due to privacy issues and the requirement also recently implemented a tool called the Flex- •

3 for legislative amendments. As part of ongoing ible and Integrated Risk System (FAIRS), which discussions with its inter-jurisdictional col- analyzes an account’s collection history to predict

Chapter leagues, it will explore the option of strategic use the likelihood of payment. This helps determine of private-sector collection agencies. the extent of collection efforts that should be As the Branch continues to pursue the collec- committed to that account. For accounts that are tion of non-tax receivables, staff resourcing will closed, FAIRS provides an overview of which col- be a key consideration and a component of the lection activities were successful in recovering tax implementation plan. amounts owing. Currently, FAIRS is used only on RST accounts; the Branch plans to start using the tool for the remaining accounts housed in OntTax in 2013. Since ICS does not have formal risk-scoring Account Prioritization capability, accounts housed in ICS for the most The Ministry’s Collections Branch primarily uses part continue to be prioritized and assigned to col- two stand-alone information systems. Corporations lectors on the basis of dollar value and date of entry Tax (CT) and Self-employed Health Tax (SEHT) into collections. accounts reside in the Branch’s older legacy sys- Through its risk-scoring function, accounts in tem, the Integrated Collections System (ICS). The OntTax are prioritized using criteria such as the Branch’s collection responsibility for the remain- amount owing, number of times or the length of ing tax statutes—for example, Retail Sales Tax, time in collections, whether there is a history of Employer Health Tax and Tobacco Tax— is admin- broken promises, and if any legal actions have Tax Collection 269 been taken on the account. Once the priority of MINISTRY RESPONSE an account has been established, it is assigned to collectors in one of three tiers: Tier 1 (low- The Branch’s Business Enhancement Unit risk accounts) and Tiers 2 and 3 (medium- and and the staff in the unit who work with the high-risk accounts). However, we noted that once Ministry’s automated risk assessment system accounts are assigned, collectors often do not use (FAIRS) will work together to address the Aud- the risk ranking to determine the order in which itor General’s concerns. The Branch will review they work on the accounts. its current methodology and guidelines and Since the development of the risk-scoring make any changes to ensure that collectors pri- methodology in OntTax in 2008, the Branch has oritize their work in consideration of risks and not updated or formally evaluated it to determine in ways that will maximize the recovery of taxes whether this tool prioritizes accounts appropriately. owed to the province. The initial review is to be We noted several anomalies in the scoring that completed by March 31, 2013. indicate a need to review and update the criteria so that collectors can make better use of it to set priorities. For example, risk scoring assesses some OVERSIGHT OF COLLECTION ACTIVITIES of the criteria noted above over the entire history of the account while other criteria are assessed Sufficient oversight of collectors’ work is paramount only after 2008, when risk scoring was adopted in in ensuring that their efforts are timely and pro- OntTax. Also, inconsistency in collection practices gressive, and that all tools available have been used 3.11 affects the score assigned to accounts. For instance, to recover the most money possible. Supervisory the risk-scoring methodology assigns points to oversight of collection activities consists of two Section an account every time a notice is sent out. From main procedures: VFM our review of accounts, we noted a fair amount of Semi-annually, collection managers review •

• 3 inconsistency in the number of notices sent by col- a sample of each collector’s active files. lectors. As a result, more points had been assigned Managers also review specific accounts that

to some accounts simply because the collector sent collectors have deemed uncollectible to Chapter more notices; the points assigned did not neces- ensure that all collection efforts have been sarily reflect the risk associated with the account. exhausted, or where approval is required for We also noted several significant fluctuations in measures such as initiating payment arrange- the risk scores without a significant change in the ments beyond the collector’s authority or circumstances of the account that would explain initiating and enforcing legal actions. the difference. • The Branch’s Performance Analysis Unit (PAU) and the Ministry’s Internal Audit RECOMMENDATION 3 Division review accounts that are submitted for write-off to ensure that the amounts are To ensure the effectiveness of its risk-ranking correct, that policies and procedures were methodology for prioritizing collection efforts, followed, and that all collection efforts have the Ministry of Finance’s Collections Branch been exhausted. should formally assess this methodology to We noted that the managers’ semi-annual determine whether it is ranking accounts for reviews identified specific issues such as the action appropriately and consistently. The timeliness of collection actions, incomplete docu- Branch should then develop guidelines to mentation, and failure to use all available tools encourage collectors to use the risk scoring to in the collection effort. Although the managers prioritize their work. informed us that they discussed these issues with 270 2012 Annual Report of the Office of the Auditor General of Ontario

the respective collectors, we found no evidence of • ICS, which at the time of our audit was used any required action or follow-up to ensure that col- to administer more than 50% of the Branch’s lectors were making the required changes. We also accounts-receivable inventory, can readily noted that there is no overall analysis and related produce only pre-programmed “canned” feedback on any systemic concerns identified reports. Supplemental reports for basic through the managers’ semi-annual reviews or the information, such as requests for listings of PAU review of accounts submitted for write-off. accounts with payment plans and legal actions in place, take an inordinate amount of time RECOMMENDATION 4 to produce. Such reports can be produced only through special requests or “tickets” to To ensure that collection efforts are appropriate, extract the required data from the system—a timely and in compliance with established pro- costly and time-consuming process. Although cedures, the Ministry of Finance’s Collections the Ministry plans to decommission ICS once Branch should ensure that collectors document most of the CT accounts are resolved, the sys- any follow-up action taken in resolving issues tem’s inability to produce information useful identified during reviews of their work. The for collections activities will pose problems in Branch should also identify any systemic con- the meantime. cerns, as well as best practices, from its ongoing Although the reporting of key information reviews of active files as well as accounts that • on accounts within the OntTax system is are submitted for write-off. substantially better than within ICS, there 3.11 are still some limitations. Specifically, the MINISTRY RESPONSE amount of the assets on which liens and war- Section The Branch has since introduced a requirement rants have been registered prior to the 2008 VFM for management to perform a follow-up review implementation of OntTax and that have not •

3 with collectors to ensure that collectors make been renewed or amended since are reported the required changes recommended by manage- at $0 even though information exists within

Chapter ment. In addition, the Branch will implement the system on the actual amount of the assets a more formalized process to regularly identify on which individual liens and warrants have any systemic concerns and best practices, and been registered. Therefore, the Ministry will work closely with the Business Enhance- cannot determine the portion of the tax- ment Unit and Performance Analysis Unit to receivable inventory that is secured by liens or improve, communicate and monitor the imple- warrants for these accounts. As well, payment mentation of best practices. plans often do not cover the total amount of the debt owing, even though OntTax reports the total amount of the debt as being covered by the plan. Accordingly, for these situations Management Reports OntTax does not provide an accurate picture In order to oversee collection activities effectively, of the tax-receivable inventory that is secured managers should have access to sufficient and by payment plans. OntTax also does not timely operational and performance informa- provide details such as the number, amount tion. However, we found that reports produced and frequency of instalments associated with by the Ministry’s information systems did not individual payment plans. adequately support the oversight of the collection function. Specifically: Tax Collection 271

whether the resulting measure was meaningful RECOMMENDATION 5 given the following: In light of the fact that the OntTax system • It tracks and reports only on results for RST will continue to support the collection and and other statutes that are administered in the administration of the remaining tax statutes, as OntTax system—less than half of the Branch’s well as the collection of the province’s non-tax accounts-receivable inventory. Therefore, amounts owing if the Branch’s role is expanded, the measure does not provide a complete and the Ministry of Finance’s Collections Branch accurate representation of the effectiveness of should work with ministry systems staff to the Branch’s overall collection effort. ensure that the system reports provide com- • The number of RST referrals continues plete, accurate and up-to-date information on to decrease substantially given the recent debtors’ accounts. transfer of the administration of RST to the CRA. This decrease prevents the Branch MINISTRY RESPONSE from making any meaningful comparison of actual performance with the benchmark. The Branch’s Performance Analysis Unit and For instance, for the fiscal year 2011/12, the Business Enhancement Unit will continue to Branch reported that the number of accounts work closely with ministry systems staff to resolved as a percentage of new referrals was ensure that the system reports are complete 158%. The Branch significantly exceeded its and accurate and will address any concerns in a target of 75%, but this could have been due

timely manner. 3.11 to a significant decrease in new delinquent accounts rather than a significant increase in Section the collection of existing accounts. VFM Performance Measures Individual collector performance is measured •

3 primarily by the number of clearances (defined as The overall mandate of the Ministry’s Collections the full recovery of debt). The clearance is credited Branch is to ensure efficient and effective collec-

to the last collector assigned to the account with- Chapter tion of outstanding tax liabilities. The Branch does out considering which actions by which collector provide some information on its results for a given actually led to the recovery of the debt. This also year—for example, amounts collected and number gives no credit to collectors who were successful of accounts cleared or written off, among other in collecting at least part of the outstanding debt. things—and compares the results to those of the The timeliness of collection activities is measured prior year. However, prior to the 2010/11 fiscal by whether accounts with balances greater than year, the Branch had not set meaningful bench- $100,000, which make up about 80% of the marks aligned with its mandate against which it Branch’s total receivables, are subject to some sort could evaluate performance, either for the Branch of action every 90 days. However, the appropriate- or for individual collectors. In the 2010/11 fiscal ness or effectiveness of the collections action taken year, the Branch established an overall bench- is not routinely assessed. mark performance measure: the total number of Our research on effective collection practices accounts resolved as a percentage of new accounts in the private sector indicated that dollar recov- received. Based on the prior year’s result and other ery rate is the key consideration in grading the assumptions, the Branch set a 75% benchmark for performance of agencies responsible for collec- this measure up to the 2013/14 fiscal year. While tions, and that different parameters can be used we recognize the Branch’s attempt to establish to calculate the rate. For instance, recovery rates an overall performance measure, we questioned 272 2012 Annual Report of the Office of the Auditor General of Ontario

can be calculated on the entire portfolio of taxes MINISTRY RESPONSE receivable, on closed accounts, on accounts by age, or by region and district office. Benchmarks can be The Branch has recently hired a consulting set for the different recoveries and then compared firm to assist in developing a balanced per- to the actual rates. In addition, reporting on the formance measures scorecard for the Branch reasons why accounts are closed or written off can and for its individual collectors. The Branch provide valuable insight on the effectiveness of will build on the consultation work provided internal processes. by identifying data elements available from ministry systems and establishing performance RECOMMENDATION 6 measures and benchmarks. The Branch will also attempt to benchmark itself against similar To enable it to better track the effectiveness of organizations and will be discussing this with its collection efforts, the Ministry of Finance’s its inter-jurisdictional partners in September Collections Branch should have more clearly 2012. The initial review is to be completed by defined benchmarks and performance measures March 31, 2013. for collection, both for the Branch itself and for individual collectors. The outcomes should be tracked, evaluated against established benchmarks, and reported periodically. 3.11

Section

VFM

•

Chapter Chapter 3 Ministry of Training, Colleges and Universities Section 3.12 University Undergraduate Teaching Quality

generally required to report on financial and other Background matters to the Ministry of Training, Colleges and Universities (Ministry). Also, in addition to tuition fees, funds from student services, donations and Ontario’s 20 publicly assisted universities offer other miscellaneous revenue, universities depend graduate and undergraduate degree programs in a

on a significant amount of government funding. 3.12 wide variety of fields. In 2010/11, these universities Excluding government funding for student assist- enrolled the equivalent of about 390,000 full-time

ance, research and capital grants, the Ministry Section students, excluding about 44,000 foreign and other provided approximately $3.3 billion in operating VFM students taking courses not eligible for provincial •

funding to Ontario’s universities in 2010/11. 3 assistance. These universities employed approxi- The Ministry’s vision for post-secondary mately 15,000 full-time faculty members. Faculty education is to build the province’s competitive

include tenure-stream staff, who have both teach- Chapter advantage and enhance its quality of life by having ing and research responsibilities; teaching staff, the world’s most educated people and the most who generally have no research responsibilities; highly skilled workforce. Since the Ministry expects and part-time sessional instructors, who are under that 70% of all new jobs will require education and contract to teach one or more courses. training beyond the high school level, its goal is Most Ontario universities were established to have 70% of the people of Ontario attain post- or continued by acts of the provincial legislature secondary credentials by 2020. that set up their governing structures. University University professors typically have two main governance is often a shared responsibility between responsibilities—teaching students and conducting the Board and the Senate. The Board is generally research in their field of expertise. At the under- responsible for the university’s corporate side, graduate level, instruction is more oriented to the including management of property, revenues, classroom teaching of specific courses, while at the expenditures and other business affairs. The Sen- graduate level it tends to be done on more of a one- ate is responsible for academic matters such as to-one basis or in small group settings. Teaching determining the courses of study, setting admission students contributes significantly to their learning, standards, and awarding diplomas and degrees. and ineffective teaching can have an adverse impact Although historically Ontario universities on learning. Consequently, high-quality instruction have enjoyed a great deal of autonomy, they are

273 274 2012 Annual Report of the Office of the Auditor General of Ontario

can help the Ministry, the universities and Ontario’s in charge of a professional faculty such as engineer- students achieve their goals. ing or law, or an academic faculty such as human- ities or social sciences). In addition, we met with a number of students at each university we visited, researched practices in other jurisdictions, and Audit Objective and Scope engaged an adviser who is recognized as a leading scholar in research on student development and The objective of this audit was to assess the extent teaching assessment. to which the Ministry and selected Ontario uni- We did not rely on the Ministry’s or universi- versities support, assess and periodically report ties’ internal audit departments to reduce the meaningful performance information on the quality extent of our audit work, because they had not of instruction provided to undergraduate students. conducted any recent audit work on university Senior management at the Ministry and selected teaching quality. universities reviewed and agreed to our audit objective and associated criteria. Our audit work was primarily conducted at Brock University, the University of Ontario Insti- Summary tute of Technology, the University of Toronto, and the Ministry of Training, Colleges and Universities. From the Ministry’s perspective, a university’s most This work focused mainly on students at the under- important mandate is that it does a good job of 3.12 graduate level, because undergraduates make up teaching its students and preparing them for the the majority of university students. We also met future workforce. We believe students, their parents Section with the Council of Ontario Universities (COU), and the public would agree. VFM which represents the interests of Ontario’s publicly The administrators (deans and faculty or depart- •

3 assisted universities; the Ontario Confederation of ment heads) we spoke to at the universities visited University Faculty Associations, which represents noted that assessing undergraduate teaching qual-

Chapter the faculty associations of Ontario universities; ity in universities is complex and not easily quantifi- and the Ontario Undergraduate Student Alliance, able. Nevertheless, most felt that relevant measures which represents the interests of undergraduate could be developed to give insight into teaching university students. We also spoke to representa- quality. Although both the Ministry and the univer- tives of the Higher Education Quality Council of sities we visited were not formally assessing and Ontario, an Ontario government agency set up to reporting on teaching quality and effectiveness on generate evidence-based research to assist in the a regular basis, we observed that information con- improvement of the post-secondary education cerning teaching performance that may be useful system, and to the Ontario Universities Council in some capacity was available at all three universi- on Quality Assurance (OUCQA). The OUCQA was ties. For instance, all Ontario universities, including established by the COU to oversee quality assur- the ones we visited, have instituted a good process ance processes for all academic programs offered whereby undergraduate students are generally by Ontario’s publicly assisted universities. given the opportunity to complete evaluations In conducting our audit work we reviewed toward the end of the courses they take. relevant legislation, policies and procedures, and However, our review indicated that little aggre- met with appropriate staff of the Ministry and the gate analysis of the student evaluations is done at universities we visited, including faculty members, the universities we visited, and only about one- department heads and deans (generally, a dean is quarter of Ontario’s universities indicated that they University Undergraduate Teaching Quality 275 make the summarized results of these evaluations assistance to improve their teaching skills. available to students to assist them in their course Also, none of the universities visited required selection decisions. providing written performance appraisals It should be acknowledged that, notwithstand- to sessional instructors, even though they ing any issues regarding teaching quality, a recent accounted for between 10% and 24% of survey of Ontario university students indicates that full-time-equivalent staff. 78% of senior-year students were satisfied with • We reviewed a sample of tenure and promo- their educational experience, and all three of the tion decisions at the three universities visited universities we visited had processes in place to and noted that the documentation available varying degrees to improve teaching quality, such on teaching quality was generally positive. as establishing teaching centres and considering However, at one university we were able to teaching performance in tenure and promotion review documentation relating to 2011 only, decisions. Nevertheless, some of our key observa- because documentation for previous decisions tions on teaching quality at the universities we had been destroyed in accordance with the visited and from available data at other universities university’s collective agreement. At another suggest a need to better ensure that teaching qual- university, one individual who was granted ity is valued, encouraged and rewarded. tenure was rated well below average in stu- These observations include the following: dent course evaluations, and administration • Only one of the universities we visited peri- had consistently suggested improvement in odically aggregated undergraduate student teaching performance. This university did not 3.12 course evaluation data, although one of the have the option to defer tenure decisions until other universities informed us that it was performance improved, because a negative Section in the process of establishing a new course tenure decision leads to termination accord- VFM evaluation framework and planned to aggre- ing to its agreement with faculty. •

3 gate course evaluation results and analyze • According to aggregate undergraduate stu- them once the system was fully implemented. dent course evaluation results at one of the

The third university we visited was not aggre- universities visited, the university’s Faculty of Chapter gating and reporting the results of student Education consistently outperformed all other evaluations, nor did it plan to. In fact, we faculties by a significant margin in overall were advised that a decision to defer a faculty ratings of teacher effectiveness. We were member’s tenure was reversed on appeal in told that virtually all members of this faculty, part because this university’s administrators unlike those of other faculties, had formal did not have aggregate student evaluation training in how to teach. Ontario universities data to demonstrate that the results of this in general do not require faculty members to professor’s student evaluations were below have formal training in teaching. At all three their expectations. universities visited, performance appraisals • A number of faculty we spoke to felt that revealed examples where, to varying degrees, their annual performance appraisals did not there was room for improvement in teaching provide them with appropriate feedback on effectiveness, but we did not see evidence teaching performance. We noted examples that these instructors had been provided with where student evaluations had been critical specific guidance or sought assistance from of teaching performance but there was no the universities’ teaching and learning cen- evidence that specific guidance was provided tres. In fact, at two of the universities visited, or that the faculty members had sought records at the centres indicated that faculty 276 2012 Annual Report of the Office of the Auditor General of Ontario

attendance at teaching workshops averaged less than one hour per instructor per year. Detailed Audit Observations • While ministry funding to Ontario universities has increased significantly over the past PROCEDURES TO ASSESS decade, according to the Council of Ontario TEACHING QUALITY Universities (COU), Ontario universities receive less government funding per student Although the administrators we spoke to at the than universities in any other Canadian universities we visited (such as deans and faculty province. A recent COU study also noted that department heads) noted that assessing under- Ontario’s student-to-faculty ratio is the high- graduate teaching quality is complex and not easily est in Canada. In one university visited, 75% quantifiable, most felt that relevant measures could of first-year classroom spaces in several facul- be developed to give insight into teaching quality. ties were in classes with over 100 students. Administrators also noted that no one indicator Research indicates that large classes can pose was sufficient in itself to assess teaching quality. challenges to providing effective instruction, Although the universities we visited were not for- which can impact student learning. In a recent mally assessing and reporting on teaching quality survey, faculty across the province noted that and effectiveness on a regular basis, we observed their workload had increased over the last five that information on teaching performance that may years, resulting in their reducing one-on-one be useful in some capacity—such as student course time with students outside the classroom, evaluations, performance appraisals of instructors, 3.12 giving more multiple-choice tests and giving student surveys and program reviews—was avail- fewer writing assignments. able at all three universities. Section • The Ministry is making progress toward VFM achieving its goal of having 70% of Ontario’s • Student Course Evaluations 3 population hold post-secondary credentials by 2020. It reported in 2011 that 64% of Ontar- Student course evaluations are questionnaires that

Chapter ians aged 25 to 64 had post-secondary cre- require a written or selected response on a wide dentials, compared to 56% in 2002. However, range of topics such as course content and the qual- we found that two years after graduation only ity of instruction. Although the process differed at 65% of graduates surveyed were employed the three universities we visited, questionnaires full-time in a job that was related to the are typically given to students two to three weeks skills acquired in their studies. The Ontario before the final course exam, and results are not Undergraduate Student Alliance indicated to provided to instructors until after the final exam is us that students would find such information marked and student grades are submitted. In gen- on graduate employment outcomes beneficial eral, policies surrounding the student evaluation in choosing their university and program process were designed to ensure that the instructor of study. does not influence the responses and that the responses are provided anonymously. Research on student evaluations reveals that some faculty members question the ability of students to properly assess instructors, but other research indicates that most have a positive view of student course evaluations. Even though research shows that students cannot accurately assess an University Undergraduate Teaching Quality 277 instructor’s knowledge or competency in the disci- whereas sessional instructors on average pline, a number of studies conclude that student received lower scores (1.03). This university evaluations are valid, reliable and useful when relies heavily on the use of sessional staff, who examined thoroughly. In addition, these studies accounted for 24% of its full-time-equivalent indicate that students can very accurately assess staff and were responsible for teaching the effectiveness of in-class teaching, including approximately 40% of its courses. This overall presentation, clarity, the instructor’s level of organ- student assessment was consistent with the ization, the course workload and their own overall opinions of the administrators we spoke to, learning experience. who indicated that, partly because sessional We reviewed the student evaluation process staff often have competing employment com- at the three universities and were informed that mitments, they generally provided instruction most undergraduate courses were being evaluated. that was of lower quality. Exceptions were small classes and one university One of the other universities visited informed that did not always evaluate courses taught by us that it was in the process of establishing a new part-time sessional staff. Only one of the three online course evaluation framework with core universities used a common questionnaire for all questions common to all faculties and departments. undergraduate courses. The other two universities As well, this university planned to aggregate the did not have a common student evaluation ques- results and undertake analysis of the data once the tionnaire to be used across different faculties and, system was fully implemented. At this university, in one case, student evaluation questionnaires even the vast majority of faculty received a teaching rat- 3.12 differed across departments within a single faculty. ing of at least “good” in the student course evalua- The university that used a common question- tions we reviewed. Section naire had also taken the initiative of developing We could not review student course evaluations VFM an online system where data could be aggregated at the third university because its collective agree- •

3 and compared at the faculty and university levels. ment with faculty stipulates evaluations to be the Our analysis of this data revealed that, over the last property of the professor, and we were not given

five years, the average instructor score was 1.22 on access to them. The collective agreement stipulates Chapter a scale of –2 to +2, indicating that students were that student evaluations can be used by administra- generally satisfied with the quality of instruction tion in the tenure and promotion process, and for they had received. Further analysis of this data annual appraisals. However, a score from student revealed the following: evaluations was recorded in about half the activity • There were significant differences in pro- reports we reviewed (faculty members submit these fessors’ scores between faculties, with the reports to administrators for annual appraisal pur- average scores over the last five years ranging poses). Most of these scores were positive, although from 1.03 to 1.46. However, the university we could not verify their accuracy, and in many had not formally investigated reasons for dif- cases it was unclear whether the score represented ferences between faculties in order to identify an overall average of different questions concern- and share best practices that could improve ing teaching performance, or whether it reflected teaching quality at lower-performing faculties. the answers to a single question. We observed an • We also noted differences in the performance instance at this university where the decision to of different types of instructors. For example, defer a faculty member’s tenure, based on concerns over the last three years, tenure-stream and about teaching, had been reversed on appeal. Uni- teaching-stream faculty were rated almost versity administrators informed us that this was due identically (1.23 and 1.25) by students, in part to the fact that they did not have aggregate 278 2012 Annual Report of the Office of the Auditor General of Ontario

student evaluation data to demonstrate that the in place to annually assess the teaching perform- results of this professor’s student evaluations were ance of tenure-stream and teaching-stream faculty. below their expectations. Although the three institutions used different We found that only one of the three universities processes, all three included a requirement for gave students access to the summarized results of faculty members to submit an activity report to course evaluations, and this university informed us the administration detailing, among other things, that only half of its students could easily access the their teaching activities. This activity report could results. This university also informed us that once include information on courses taught, supervision its new framework is implemented, students would of students, course development, awards and have access to the summarized course evaluation honours, improvements in teaching, and profes- results unless the instructor elects to opt out of sional development. We were informed that these the process. The students we spoke to at all three activity reports and other relevant information are universities, as well as representatives from the considered along with student course evaluations Ontario Undergraduate Student Alliance (OUSA), in the appraisal of the faculty members’ overall and felt that students would find the results of student teaching performance. The primary source of data course evaluations useful for making decisions available to assess in-class teaching performance regarding future course selection. As well, OUSA at the three universities we visited was student representatives and some students we spoke to course evaluations, because in almost all cases noted that when results are not available, students these were the only documented assessments of tend to not take the evaluation process as seriously. classroom instruction. 3.12 This was because they feel that the results are not At one of the universities visited, administrators actually used by the university or the professor. gave the vast majority of faculty selected for review Section We contacted all 20 publicly funded universities in an overall rating indicating satisfactory teaching VFM Ontario and were informed that they all provide performance. At the second university, although •

3 most students the opportunity to complete course the faculty members we selected had received evaluations. However, just five indicated that the an annual appraisal, about half the professors

Chapter results of these evaluations are generally made evaluated did not receive a quantitative score or available to students. a qualitative rating tied to specific performance While administrators at two of the three uni- criteria. At the third university, the vast majority of versities generally felt that the results of student faculty were appraised as being in compliance with course evaluations could be used along with other their collective agreement—which, according to indicators to provide insight into teaching quality, the agreement, implies satisfactory teaching per- none of the three universities formally used such formance. The administration at this university was evaluations in aggregate to periodically analyze generally of the opinion that recognizing differing and report on teaching quality at the level of the levels of performance would be useful in facilitating university as a whole or at the faculties we focused improvements in teaching. However, under the colour work on. lective agreement, there was no requirement to provide faculty members with a performance appraisal unless a performance-related concern arose. Annual Performance Appraisals of Faculty We reviewed a sample of faculty performance The annual appraisal of faculty can assist faculty appraisals at the three universities visited and members in their career development and identify noted the following: weaknesses in performance to facilitate improve- • In a few instances at one university, student ment. The three universities we visited had policies evaluations suggested that there may be room University Undergraduate Teaching Quality 279

for improvement in teaching performance, an overall score qualitatively associated with but there was no evidence in the appraisal good performance). or in the file that specific guidance had been At the three universities we visited, a number of provided to the faculty members or that any professors told us that student course evaluations action had been taken by the faculty members factored too highly in the assessment of teaching to facilitate improvement. and that their performance appraisals did not • Another university’s institution-wide student provide them with appropriate feedback on their evaluation results indicated that its students teaching. Some professors at one university sug- were not fully satisfied with the performance gested that other evaluation techniques, such as of approximately 20% of full-time faculty periodic peer reviews by other professors, could be members. Yet, despite the suggestion in used to augment student course evaluations of in- student evaluations that there was room for class instruction. However, none of the universities improvement, in some instances we saw no visited used these other evaluation techniques to do evidence that the faculty member either had so on a regular basis. been directed to seek or had sought help. We also observed instances where the administra- Performance Appraisals of tion’s appraisal of satisfactory performance Sessional Instructors was not in line with the results of student course evaluations. In one such case, a faculty As a general rule, none of the universities visited member’s student evaluation results were well provided written annual performance appraisals 3.12 below the average, and the administrator had to their sessional instructors. Sessional instructors recommended that this individual seek help at the three universities accounted for between Section with teaching skills, but, nevertheless, this 10% and 24% of full-time-equivalent faculty. At VFM faculty member’s teaching performance was the university that had the largest proportion of •

3 appraised as eight out of 10. sessional staff, many of them had been employed at • At the third university, student comments the university for several years.

recorded on course evaluations were not The sessional instructors we sampled generally Chapter generally reviewed, and the administration received favourable evaluations from students at often evaluated only the one question on the two of the three universities visited. At the third student evaluations that asks about overall university, we observed several instances where ses- performance. This was despite the avail- sional instructors received lower course evaluation ability of student responses to a number of scores than the faculty average, and there was no potentially informative questions, including evidence that university administrators instructed whether the instructor presented material these faculty members to seek assistance. In one in an organized and well-planned manner, case a sessional instructor received a contract to explained concepts clearly with appropriate teach full time after consistently receiving some of use of examples, and communicated enthusi- the poorest student evaluations in three successive asm and interest in the course materials. In years. While we were informed that the administra- our review of a sample of student evaluations, tion had referred this individual to the university’s in some cases it was not evident whether the teaching and learning centre, our review of the administration had provided feedback to fac- centre’s attendance log did not indicate attendance ulty members whose teaching in specific areas at any workshops by this instructor during the pre- likely had room for improvement (although vious two years, and there was no documentation the vast majority of faculty members received for any other professional development work done at the centre. 280 2012 Annual Report of the Office of the Auditor General of Ontario

Because two of the three universities did not department levels than at the university level to provide annual performance appraisals to their ses- identify specific areas that require further scrutiny sional instructors, and one did so only occasionally, or action. documentation concerning the teaching perform- Overall, NSSE survey results indicate that ance of sessional staff was generally limited to the students were generally satisfied with their educa- results of student course evaluations. This was of tional experience. For example, in 2011, between concern, as research suggests that no one source of 71% and 86% of senior-year students at the three information is sufficient to assess teaching quality universities rated their educational experience as and that student course evaluations are just one of either good or excellent. The Ontario average was the factors to consider. As well, one university had 78%. In comparison, American peer institutions no requirement to administer student evaluations had an average of 84%. if the course was taught by a sessional instructor. All three universities had also participated in the Consequently, for some sessional staff, there 2010 Canadian Graduate and Professional Student may be nothing on file to indicate the quality of Survey, in which students were asked, among other their teaching. things, to comment directly on graduate-level teaching quality. Although none of the universities visited had formally used this survey to identify and Student Surveys address weaknesses, we observed that in aggregate All three of the universities we visited participated all three performed close to the Ontario average in in national student surveys that gather information questions that directly concern teaching quality. For 3.12 to gain insight into the students’ undergraduate and example, for all Ontario universities, an average of graduate school experiences. The National Survey 87% of students surveyed rated the overall quality Section of Student Engagement (NSSE) was the most-used of graduate-level teaching as good to excellent, and VFM survey and applicable to the widest cross-section of all three universities we visited received similar •

3 students. The NSSE is described as the leading tool results. At the undergraduate level, two of the three used to measure student engagement in universi- universities we visited participated in the Canadian

Chapter ties, and there is significant literature associating University Survey Consortium’s 2009 survey and student engagement with learning outcomes. While again performed close to the average with respect the NSSE survey asks a lot of informative questions, to teaching quality. Overall, 90% of students sur- those responsible for reviewing and analyzing the veyed were generally satisfied with the quality of results at the universities we visited generally felt teaching they received, and the two universities that the results of the NSSE could be used only scored 88% and 93%, respectively. to a limited degree to assess teaching quality. Consequently, they were not using the results for Program Reviews such purposes. Nevertheless, we found that the NSSE solicits In 2010, the Council of Ontario Universities (COU) information that can inform universities of areas established the Ontario Universities Council on where improvements can be made. To illustrate, at Quality Assurance (OUCQA). The OUCQA was two of the universities we visited, between 25% and established to oversee quality assurance processes 31% of first-year students selected reducing class for all academic programs offered by Ontario’s size as an item the university needs to address to publicly assisted universities and to bring the improve their learning experience. Our discussions quality assurance processes that had been in exist- with one of the survey’s creators identified that ence for many years under a common framework. NSSE results are more useful at the faculty and The OUCQA is also responsible for auditing each University Undergraduate Teaching Quality 281 university’s quality assurance processes and approving outcomes. This university had used responses ing new graduate and undergraduate programs. from student surveys to illustrate that learning We were informed that all universities adhere to outcomes had been achieved. However, student this quality assurance framework and that ministry perceptions of their learning outcomes may not funding for new programs is conditional on the be indicative of actual student achievement. For programs’ approval by the OUCQA. example, although the students surveyed were The emphasis of the quality assurance frame- confident of their writing ability, faculty noted their work is on evaluating the overall quality of pro- concerns with the quality of student essays and grams offered by universities. As part of this process, writing skills. Representatives at this university universities are required to self-assess their existing indicated that they were still working on ways to programs once every eight years against a set of measure student learning outcomes, and some sug- criteria that includes an evaluation of the program’s gested that standardized tests of key competencies curriculum, teaching and assessment methods, and could assist in this measurement. human and financial resources. As well, universities Program reviews, supplemented by independent are required to assess whether student learning audits of the review process, can provide valuable outcomes have been achieved. Learning outcomes feedback about program quality, and we believe include not only the acquisition of specific know- that with further refinement they could also pro- ledge in a given field, but also general competencies vide useful feedback on student learning outcomes. such as the ability to think critically, communicate effectively and work well with others. RECOMMENDATION 1 3.12 Program reviews offer the possibility of provid- To help ensure that administrators and students ing at least indirect feedback on teaching quality,

have sufficient information to make informed Section since teaching contributes significantly to student decisions, and that all faculty members receive VFM learning outcomes. However, discussions with •

the necessary feedback to maintain or enhance 3 university staff and our examination of program teaching quality, universities should: reviews completed under the new quality assur- consider means to aggregate student course

ance framework at two of the universities visited • Chapter evaluation information at the university, revealed that measuring student learning outcomes faculty and department levels so that admin- continues to pose challenges for universities, a view istrators can identify best practices and areas also shared by representatives we spoke to from the requiring attention; OUCQA and the COU. Recognizing this, the COU develop a core set of student course evalua- developed a Teaching and Learning Task Force in • tion questions to be used throughout the 2011 to identify and promote practices to improve university to facilitate comparison of student teaching and learning outcomes. We were informed evaluation results; that the Task Force was currently in the process provide students with the summarized of surveying university administrators to identify • results of student course evaluations to how learning outcomes are being assessed and to assist them in making informed decisions on gather opinions on how to improve the assessment course selection; and of learning outcomes. ensure that faculty, including sessional The program reviews at one of the universities • faculty, periodically receive constructive identified that it had taken steps to move beyond feedback on their teaching effectiveness, and the presumption that offering appropriate courses encourage faculty to undertake any neces- with appropriate learning objectives would ultim- sary professional development. ately lead to students meeting the intended learn- 282 2012 Annual Report of the Office of the Auditor General of Ontario

university. Tenure-stream faculty are the trad- SUMMARY OF UNIVERSITIES’ itional means by which universities are staffed and RESPONSES represent the largest group of full-time-equivalent The universities generally agreed that both faculty at all three universities visited. Their university administration and students need responsibilities generally consist of 40% teach- sufficient information to make informed deci- ing, 40% research and 20% other service (such sions and that faculty should receive sufficient as participating on committees and performing feedback to enhance their teaching skills as other administrative duties). At the universities appropriate. One university notes that it is visited, faculty members who achieved tenure were already well-advanced in the implementation generally awarded the rank of associate professor. of the recommendations, which it expects will Once faculty members receive tenure, it is very strengthen both the quality of post-secondary difficult for the university to terminate them for education and its accountability. performance concerns, although, as in any job, Another university stated that it is already their further progress depends on their ongoing in the process of implementing an online course performance. Summaries of tenure applications evaluation framework that utilizes common over the last three years revealed that the three university-wide questions that will be answered universities granted tenure to 95%, 96% and 100% by all students to ensure consistency and enable of the applicants, respectively. We were advised aggregation and comparability. In addition, that competition for tenure-stream positions results once implemented, the new system will further in numerous well-qualified candidates applying for 3.12 enhance student access to the results of course tenure positions, and therefore high success rates evaluations and will facilitate constructive, are to be expected. Section comparative feedback for instructors and During their probationary period, tenure-stream VFM administrators. This university is also commit- faculty members must demonstrate sufficient com- •

3 ted to ensuring that all sessional instructors petence in both teaching and research. At the three periodically receive constructive feedback on universities visited, the probationary period was

Chapter their teaching effectiveness. five to six years. Two of the universities also had The third university noted that it was interim review processes during the probationary interested in developing a core set of student period to assess and provide feedback on the faculty course evaluation questions that could be used member’s progress. At the conclusion of the proba- throughout the university. As well, the univer- tionary period, faculty members at all three university understands the use and value of aggregate sities are to prepare documentation with respect to data and providing students with the results teaching and research, among other achievements, of such evaluations. However, to accomplish in support of their application for tenure. this, the university will need to work with While the method of reviewing applications its faculty association through the collective for tenure and promotion differed at the three bargaining process. universities visited, all three included the review of tenure and promotion applications by multiple committees, and approval by the university’s academic vice-president and/or president. Information TENURE AND PROMOTION OF FACULTY at the disposal of these committees concerning Faculty members referred to as tenured have met a teaching included a teaching dossier that typically university’s probationary requirements pertaining contained a description of the individual’s teach- to teaching, research and other service to the ing philosophy, course development work, sample University Undergraduate Teaching Quality 283 assignments, letters from students, letters from were positive. However, we could not check the colleagues, and relevant honours and awards. The actual student evaluations to verify the accuracy of summarized results of student evaluations were such information. also typically available to the committees. At this same university, although peer evalua- Our review of the tenure and promotion tions were solicited for tenure and promotion deci- processes at all three universities revealed that, sions, generally such evaluations did not include an while it was evident what information on teaching in-class evaluation of the prospective candidate’s performance was being provided for the commit- instruction as was done at one of the other universi- tees’ consideration, it was unclear to what degree ties visited. The information that we could review at this information factored into the committees’ this university did not identify significant concerns decisions. We also noted that even though all three about teaching quality in tenure and promotion universities appraise full-time faculty members’ decisions—although, as previously noted, one performance annually, the appraisals were not used decision to defer tenure was overturned on appeal to inform tenure and promotion decisions at one because the administration did not have sufficient university, and at the other two universities it was information to demonstrate that the faculty mem- unclear whether they were used. ber’s teaching performance was significantly below We reviewed a sample of tenure and promotion average. At this university we were able to review cases at the three universities in detail. At one uni- documentation relating only to the 2011 tenure and versity, while it was unclear to what degree differ- promotion decisions. This was due to the collective ent sources of information factored into tenure and agreement, which required all materials relating to 3.12 promotion decisions, the documentation submitted the tenure and promotion process to be destroyed on teaching quality was generally positive and sup- immediately after a decision is reached unless an Section ported committee decisions. We noted from this appeal or grievance is lodged. Therefore, the univer- VFM university’s record of tenure decisions over the last sity could not demonstrate that a sound process had •

3 three years that 94% of faculty had achieved tenure been followed in reaching any previous decisions. on the basis of a combination of excellent research The tenure and promotion decisions that we

and competent teaching, and only one faculty mem- reviewed at the remaining university included Chapter ber had received tenure on the basis of excellent examples where faculty members who received teaching and competent research. While research is student evaluation scores below the average for their important to the quality of education provided by faculty were granted tenure and/or promotion to a university and to its reputation, faculty seeking full professor. In each of these cases, the administra- tenure might not be sufficiently motivated to pur- tion noted that the faculty member’s research was sue excellence in their teaching if they believe such highly rated. These cases included one where, in the results to be indicative of the relative value placed two years leading up to tenure, students rated the on teaching in tenure decisions. faculty member’s performance well below average. At another university, we were told that student The administration had on several occasions sug- course evaluations were heavily relied on for tenure gested the faculty member seek out opportunities to and promotion decisions. However, these were not improve teaching performance. We observed that, available for our review because of restrictions in unlike one of the other universities we visited, this the university’s collective agreement. While we did university’s agreement with its faculty did not give not have access to actual student evaluation results, it the option to defer decisions on tenure once the a score was generally recorded in the documenta- application was under review. A negative decision tion made available to us. In the decisions that were leads to the faculty member’s termination. This available for our review, the majority of evaluations was of concern, as we noted a number of examples 284 2012 Annual Report of the Office of the Auditor General of Ontario

where, after the granting of tenure or promotion, TRAINING AND PROFESSIONAL student evaluation results declined or did not DEVELOPMENT improve substantially. In some circumstances, Ontario universities in general do not require fac- allowing the deferral of tenure decisions could ulty members to have formal training in teaching give candidates for tenure an incentive to improve like elementary and secondary school teachers, their teaching and the additional time to take who are required, at a minimum, to take a one- formal training. year program of teacher education. In addition, the three universities we visited did not require RECOMMENDATION 2 faculty members to undertake periodic professional To help ensure that tenure and promotion development related to teaching. Nevertheless, decisions and the underlying documentation most administrators and faculty members we appropriately reflect the relative import- spoke to at all three universities felt that some ance of a professor’s teaching ability, the formal training was beneficial to being an effective universities should: university teacher. • ensure that all relevant information on Research we reviewed on other jurisdic- teaching performance is made available to tions identified that initial training of university tenure and promotion committees and that instructors is now common in several countries, all documentation supporting their recom- including the United Kingdom. Some institutions mendations is retained for an appropriate in these countries can require 120 to 500 hours

3.12 period of time; and of teacher training. In some cases, the completion

• explore means to ensure that tenure and proof this training factors into tenure and promotion

Section motion processes clearly reflect the relative decisions. As well, a study done at 22 universities

importance teaching ability has with respect in eight countries found that students judged that VFM

•

to such decisions. professors who had received training in teaching 3 had improved in areas such as enthusiasm, organ- SUMMARY OF UNIVERSITIES’ izational ability and rapport with students. Chapter RESPONSES According to aggregate student course evaluation results at one of the universities we visited, its The universities supported this recommenda- Faculty of Education consistently outperformed all tion and indicated they strongly value their pro- other faculties by a significant margin in terms of fessors’ teaching abilities. One university added overall professor ratings. Although the university that it will review its policies and processes to had not formally identified the reasons for this explore means to provide as much information difference, we were informed that virtually all as possible with regard to teaching and other members of this faculty had formal training in how duties and responsibilities in order to further to teach. strengthen existing tenure and promotion At all three universities visited, our review of processes. Another university noted that its cur- faculty performance appraisals revealed examples rent policies address this recommendation, and where there was room for improvement in teaching it will endeavour to strengthen these policies performance, but we did not see evidence that the even further. As well, the university noted that administration provided specific guidance to faculty it will enhance the communication and training or that faculty subsequently sought out assistance. for academic administrators and members of We noted that, while one university had a good tenure and promotion committees and put in initiative to identify and review the bottom 10% place more specific guidelines for the retention of faculty members based on student evaluation of related documentation. University Undergraduate Teaching Quality 285 results, the review and any recommended actions We noted that the teaching and learning centres were informal. Consequently, it was unclear who at all three universities offered comprehensive the administration determined needed assistance training to students serving in a teaching capacity, and whether recommendations to obtain assistance including programs that provide a teaching cer- were made. tificate upon completion. These programs varied We observed that the three universities visited in length, with the longest certificate programs at had teaching and learning centres to support, pro- each university requiring participation in eight to mote and advance the quality of teaching at their 16 workshops over approximately one to two years. institutions. We also noted that among the services Similar programs were generally not in place for provided at all three of these teaching and learning faculty members at the universities, although one centres were one-on-one consultations and a var- university informed us that it had recently opened iety of teaching workshops open to all faculty mem- its certificate program to all faculty members, bers. The faculty members we spoke to at all three and another offered a program that awarded a universities generally had a positive opinion of the certificate to faculty who had completed at least six quality of services offered by these teaching and teaching workshops. learning centres. However, the available attendance records at the centres (attendance records are not RECOMMENDATION 3 maintained for all services provided) indicate that To help ensure that all faculty members provide average attendance in teaching workshops per effective classroom instruction, universities faculty member was less than two hours annually at

should work with faculty to encourage greater 3.12 one of the three universities and, at the other two participation in professional development universities, less than one hour per year.

activities and implement procedures to ensure Section All three universities offered orientation to that faculty who would benefit from additional VFM newly hired faculty that included topics related to •

teacher training are formally encouraged to 3 teaching. We observed that at one of the universi- participate in these activities. ties, in addition to a one-day general orientation

provided to all new employees, orientation aimed Chapter specifically at new faculty was just four hours in SUMMARY OF UNIVERSITIES’ length. Attendance records indicate that it was RESPONSES attended by less than 25% of new hires in 2011. We The universities generally concurred with this observed that the other two universities offered far recommendation and noted they will continue to more extensive orientation, providing program- emphasize the importance of ongoing develop- ming over a full week that covered topics such as ment of teaching expertise for all instructors how to manage the classroom, engage students, through their teaching and learning centres. design assignments, use technology in the class- One university further noted that it would put in room, and plan and maintain courses. At one of place a mechanism by which those instructors these universities, the orientation sessions were who require additional teaching support are open to all instructors, including sessional staff, but identified and encouraged to engage in suitable we were unable to accurately estimate attendance. professional development. Another university At the other university, in which attendance was agreed with the importance of effective class- mandatory, we observed that attendance exceeded room instruction and will work with its faculty 80% of new full-time staff. However, this orienta- association and faculty to pursue strategies to tion was not open to sessional instructors; a separ- encourage greater participation in teaching- ate one-day voluntary orientation was provided to specific professional development, particularly them, attended by 60% of new sessional staff. 286 2012 Annual Report of the Office of the Auditor General of Ontario

the growth in the number of full-time faculty, the for those faculty members who stand to benefit student-to-faculty ratio has increased. According from such training. In support of its commitment to a recent sector publication, the average salary to strong teaching and learning, this university of Ontario’s full-time faculty is 6% higher than the created a new position—Vice Provost of Teach- average for all Canadian provinces and the highest ing and Learning—and enhanced the role of its in Canada. Centre for Pedagogical Innovation in order to Estimates of the ratio of full-time-equivalent highlight the importance of pedagogy, teach- students to full-time faculty in Ontario suggest that ing and learning. This university also noted it increased from 17 students per faculty member that it would review its processes to further in 1988 to 25 students per faculty member in 2008. encourage staff to undertake any necessary A recent estimate prepared by the COU noted that, professional development. in 2008/09, if all other instructional and academic staff were included, the student-to-faculty ratio would still be about 20 students per faculty mem- FACULTY COST AND WORKLOAD ber, the highest of all Canadian provinces. An increase in the average number of students Total operating revenue for Ontario universities per faculty member typically results in an increase amounted to almost $7.5 billion in the 2010/11 in average class size. While some studies indicate fiscal year, with the Ministry’s operating grants that factors such as the skill of the instructor and accounting for a significant portion of this amount. the teaching methods used can compensate for

3.12 Over the last 10 years, ministry assistance to larger classes, many studies on the impact of class universities has almost doubled, increasing from size suggest that large classes pose challenges to

Section $1.7 billion (2000/01) to $3.3 billion (2010/11). providing effective instruction, which may in turn After adjusting for the effects of inflation, this VFM impact student learning outcomes. Such challenges •

amounts to a 57% increase, which compares to 3 include less student interaction with faculty, and a 54% increase in enrolment, from 253,000 to difficulties in maintaining the students’ attention 390,000 students. While ministry funding to

Chapter and motivation. In a recent Ontario Confederation Ontario universities has increased significantly of University Faculty Associations survey of faculty over the past decade, according to COU analysis members at Ontario universities, 63% noted that Ontario universities receive less government fund- average class size had increased in the last five ing per student than universities in any other Can- years, and most respondents expressed concern adian province. Even when tuition is considered, over the increase. Those surveyed noted that the ministry and COU analyses show that Ontario uni- top three changes they had made to their teaching versities rank at or near the bottom of all Canadian approach in response to an increased workload universities with respect to revenue received per were to reduce one-on-one time with students full-time-equivalent student. outside the classroom, to give more multiple-choice Expenditures on the salaries of faculty holding tests and to give fewer writing assignments. an academic rank, such as tenure- and teaching- Data we saw at two of the universities we stream staff, increased from $1.05 billion to visited demonstrated that their undergraduate $2.1 billion between 2000/01 and 2010/11, an class sizes had increased. Although administrators increase of about 60% over and above inflation. at one of these universities had concerns about Over this same period, the number of full-time the accuracy of their data, the other university faculty increased by 25%, from about 12,000 to calculated that average class size had increased 15,000. Since student enrolment has outpaced by 20% over the last 10 years. The third university University Undergraduate Teaching Quality 287 had not calculated average class size but did have the last 20 years has also been due to a significant class-size distribution data for select faculties. For increase in the number of graduate students these faculties, more than 75% of available first- at some Ontario universities over this period. year classroom spaces were in classes of over 100 Professors who supervise graduate-level students students. In addition, we observed that, although typically spend less time teaching specific courses the full-time-equivalent student enrolment at this and more time providing one-on-one guidance or university increased by 50% between 1998 and teaching small groups. 2010, the number of full-time faculty had remained The teaching load for tenure-stream faculty is essentially unchanged. also significantly affected by their research respon- Research suggests that even though larger sibilities. As previously noted, tenure-stream faculty class sizes have increased faculty workloads for are typically expected to devote about 40% of their each course taught, the number of courses taught time to research. We noted that all three universi- by tenure-stream faculty has declined in the last ties we visited also employed full-time or contract 20 years from a norm of 3 full courses in previ- faculty considered as teaching staff, whose primary ous years. At all three universities we visited, responsibility is to teach and who accounted for the expected teaching load was generally 2 full between about 5% and 20% of full-time-equivalent courses per year for tenure-stream faculty. The faculty. The expected teaching load for teaching- workload for a typical course consists of three stream faculty ranged from 3+ to 4 full courses per hours of classroom instruction per week for eight year. As well, with information supplied to us by the months, as well as time spent on course prepara- universities, we calculated that the average salary 3.12 tion, office hours to address student questions of teaching-stream faculty for the three universi- and time to mark tests. We were able to obtain ties ranged from $74,000 to $100,000 annually, Section sufficient information from some of the faculties whereas the average tenure-stream faculty salary VFM or departments that we focused on to gain some ranged from $118,000 to $135,000. •

3 insight into actual teaching loads. At two of the At the two universities we visited that employed universities we visited, the actual teaching load in the highest percentage of teaching-stream faculty,

the departments reviewed that had such informa- those we spoke to generally felt that using such staff Chapter tion was 1.4 courses per faculty member per year was a good way to address resource constraints. (although teaching-load calculations do not neces- However, none of the three universities visited had sarily include the supervision of graduate students, undertaken any documented analysis to determine which, for some professors, can be a significant whether they were using the right proportion of responsibility). At the remaining university, teaching-stream staff. As well, concern was raised administrators estimated that the annual teach- at all three universities that having too many ing load for tenure-stream faculty throughout the teaching-stream faculty on staff can affect the uni- university averaged between 1.5 and 1.75 courses versity’s ability to conduct leading-edge research, per year. Although the expected teaching load at which is generally acknowledged as being a second- all three universities was generally 2 full courses ary, if not the primary, mandate of a university. This per year, there were a number of reasons why the could eventually have an impact on teaching quality actual teaching load differed, including that all if faculty do not keep up to date. three universities offered tenure-stream faculty The available performance and promotion the opportunity to take a one-year paid leave from information at two of the universities suggests the university for every six years worked, primarily that the quality of teaching provided by teaching- to undertake focused research. As well, we were stream faculty is as effective as that provided by advised that the decline in course workloads over tenure-stream faculty. A recent study on teaching 288 2012 Annual Report of the Office of the Auditor General of Ontario

staff concluded that a reasonable inference can be RECOMMENDATION 4 made that there is a positive relationship between the presence of teaching faculty and the quality of To enhance their understanding of the impact the student learning experience. At one university, that use of various teaching resources has teaching-stream faculty had to undergo a proba- on teaching quality and student outcomes, tionary period similar to tenure-stream faculty universities should: during which they were required to demonstrate • assess the impact of class size on teaching that they were excellent teachers. Conversely, quality and study how best to address the at this same university, few faculty in the tenure challenges posed by large classes; and stream had received tenure on the basis of excellent • weigh the impact of using teaching teaching; the vast majority of tenure awards were and sessional faculty and the extent to based on a combination of excellent research and which they can best be used to address competent teaching. resource constraints. Sessional instructors also provide an opportunity for universities to address resource constraints. SUMMARY OF UNIVERSITIES’ At all three universities visited, sessional staff were RESPONSES generally paid less than $15,000 per full course The universities agreed that teaching quality can taught. Figures provided at the three universities have an effect on student outcomes. One univer- we visited show that sessional faculty accounted for sity noted that it will undertake an assessment between 10% and 24% of the full-time-equivalent

3.12 of the impact of class size on teaching quality staff. However, sessional faculty are responsible and of how best to address any challenges for a larger proportion of the teaching at all three

Section posed by large classes. As well, the university universities. For example, we calculated that at the would continue to weigh carefully the impact VFM university where sessional faculty accounted for •

of sessional faculty and the ways in which they 3 about 10% of full-time-equivalent staff, they were can be fairly and effectively used to help the responsible for teaching 25% of the courses. university’s students. Another university stated

Chapter The administrators we spoke to at all three that it would continue to use existing research universities visited indicated that there are cir- and data in this area to explore means to meas- cumstances where sessional faculty may be the ure the impact of various teaching resources on most qualified to teach a specific course. However, teaching quality and learning outcomes and will using these faculty is often not an appropriate take action where opportunities are found to way to address resource constraints. Among the improve student outcomes. reasons cited are that sessional faculty often have other employment commitments that take priority and they are usually not as available to students outside the classroom. At the one university that PERFORMANCE MEASUREMENT AND had institution-wide data, our analysis of student REPORTING BY THE MINISTRY evaluation results identified that although students In May 2011, the Ministry announced a new strategy were generally satisfied with the performance for post-secondary education. The Ministry’s ultim- of sessional faculty, teaching-stream and tenure- ate goal is for 70% of Ontario’s population to hold stream faculty on average tended to perform better post-secondary education credentials by 2020 that than their sessional counterparts. will prepare them for the workforce of the future. University Undergraduate Teaching Quality 289

The Ministry expects that 70% of new jobs will as well as on the Ministry’s ability to achieve its require education and training beyond high school. goal to adequately prepare Ontario’s workforce for In 2011, the Ministry reported that 64% of the future. Ontarians aged 25 to 64 had post-secondary Although the Ministry currently does not credentials, a notable increase from 56% in 2002. directly assess teaching quality in Ontario universi- To further increase the number of post-secondary ties, it collects information that can assist in the graduates, the province has pursued a growth assessment of student outcomes, which are in part strategy that has resulted in a more than 50% dependent on teaching quality. For example, the increase in the number of university students over Ministry surveys university graduates and publishes the last 10 years, from approximately 250,000 full- graduate employment rates by program type and time-equivalent students in 2000/01 to 390,000 in institution for the periods six months and two years 2010/11. In addition, as part of its May 2011 plan, after graduation. Although it is not published, a sig- the Ministry has committed to funding an addi- nificant amount of additional graduate employment tional 40,000 university spaces by 2015/16. information is also collected from these surveys, Working with students, faculty and post- such as whether graduates are employed full time secondary institutions to identify and measure the and the annual salary of graduates. essential elements of teaching excellence to ensure Many university staff and administrators we that teaching is improved is another key goal of spoke to felt that graduate employment outcomes the Ministry’s strategy. While the Ministry has can be used in combination with other indicators committed significant financial resources to support to provide insight into teaching quality. In addition, 3.12 increased enrolment in Ontario’s universities, a the Ontario Undergraduate Student Alliance as well significant portion of faculty surveyed recently by as the students we spoke to at the universities we Section the Ontario Confederation of University Faculty visited said that students want additional informa- VFM Associations believed that the quality of under- tion on graduate employment outcomes to assist •

3 graduate education had declined over the previous them with choosing a university and program of five years. The Higher Education Quality Council study that could help them get a good job when

of Ontario (HEQCO) and the Commission on the they finish university. Chapter Reform of Ontario’s Public Service (Drummond The latest survey of 2008 graduates highlighted report) have expressed similar concerns that the that more than 90% of graduates are employed two quality of education at Ontario universities has years after graduation. However, our calculations, diminished, for reasons including increased class shown in Figure 1, indicate that a significantly sizes and the use of more sessional instructors. The smaller percentage are employed full-time in a job quality and effectiveness of teaching can have a that is somewhat or closely related to the skills considerable impact on student learning outcomes acquired in their studies. Employment outside one’s

Figure 1: Employment Situation of Ontario’s 2008 University Graduation Class (%) Source of data: Ministry of Training, Colleges and Universities

Universities Visited All Ontario Graduates Two Years after Graduation A B C Universities Employed 91 95 94 94 Employed full-time* 77 81 77 78 Employed full-time in a job related to skills acquired in university* 61 73 62 65

* unpublished (calculated) 290 2012 Annual Report of the Office of the Auditor General of Ontario

original field of study may very well be rewarding in the process of developing a set of performance for many graduates; nevertheless, undergraduate indicators including measures to monitor, evaluate students in the process of considering their career and improve the quality of education in Ontario’s options would still find it beneficial to know what post-secondary education system. the actual employment outcomes for graduates are. As well, by our calculations, only 59% of graduates RECOMMENDATION 5 surveyed across Ontario indicated that they were To assist students in making informed deci- employed in a full-time job that required at least sions on university and program selection some university education. and to help achieve its goal of adequately Other jurisdictions publish information on gradu- preparing Ontario students for the future work- ate outcomes that is significantly more comprehen- force, the Ministry of Training, Colleges and sive than that published in Ontario. In particular, Universities should: students in British Columbia are surveyed about collect and make public sufficient informa- their status two and five years after graduation, and • tion on student outcomes, including infor- the results are reported by survey year, discipline mation on graduate employment outcomes and institution. The surveys’ goals are to address and students’ satisfaction with the quality of university accountability, gather relevant data for their education; and program evaluation and planning, and provide stu- work with the university sector to support dents with information to help them make informed • the development of meaningful measures decisions. In addition to employment outcomes,

3.12 for student learning outcomes as a way to the B.C. reports include graduates’ opinions on maintain teaching quality. the usefulness of skills acquired in the program of Section study to their functioning in their jobs, the quality MINISTRY RESPONSE VFM of instruction and learning outcomes. For example, •

3 graduates are asked their views on how their univer- The Ministry supports the Auditor General’s rec- sity program helped develop skills such as written ommendation and agrees that assessing teach-

Chapter and oral communication, reading and comprehen- ing quality in the university sector is not easily sion, group collaboration and critical analysis. One quantifiable and is difficult to measure. of the universities we visited stated that it has agreed The Ministry currently publishes student to participate in a new survey of graduates that is outcome data, including graduation and modelled after the B.C. survey. employment rates for university graduates, and In 2005, the Ministry created the Higher it is committed to making public additional Education Quality Council of Ontario (HEQCO) information that it has collected on graduate to generate evidence-based research to assist in employment outcomes. improving the post-secondary education system. As well, the Ministry is currently working Since HEQCO was established, it has published with universities to implement an Ontario many research papers that have a bearing on teach- Education Number that will improve collec- ing quality. Topics covered include the approaches tion of student-level data, and the Ministry taken to deal with the challenges of large classes, will consider utilizing performance-reporting the use of teaching-stream faculty to address mechanisms already in place with universities increased enrolment, and the validity of student to collect additional information relating to course evaluations as an effective measure of teach- teaching quality. ing quality. At the time of our audit, HEQCO was University Undergraduate Teaching Quality 291

As it is increasingly recognized that student SUMMARY OF UNIVERSITIES’ learning outcomes may be an effective way RESPONSES to measure teaching quality, the Ministry has The universities agree that meaningful informa- made student learning outcomes a key focus of tion would benefit students in making informed its future direction and has recently started to decisions on university and program selection, engage sector stakeholders, including universi- and they will work with the Ministry and ties, on learning outcomes. In addition, the the university sector to develop meaningful Higher Education Quality Council of Ontario measures for student learning outcomes. One (HEQCO) supports this focus through several university noted that it already collects informa- projects aimed at defining and understanding tion on teaching quality and student outcomes student learning outcomes, including co- and will continue to improve on these efforts. ordinating Ontario’s participation in an OECD In 2012/13, the university will be implementing feasibility study, and evaluating the use of a Graduate Outcomes Survey for graduates of learning-outcome measurement tools in pilot undergraduate programs who are five years universities. As well, HEQCO is working with post-graduation. The survey will provide valu- the post-secondary education sector on system able feedback on the status, experiences and performance measures, including indicators perspectives of graduates as they become estab- of quality. lished in their post-graduation careers. 3.12

Section

VFM

•

Chapter Chapter 3 Ministry of Children and Youth Services Section 3.13 Youth Justice Services Program

on probation, 60 days for youths in open custody, Background 40 days for youths in secure custody, four days for youths in open detention and seven days for youths in secure detention. The Ministry of Children and Youth Services (Min- At the time of our audit, the Ministry was deliv- istry) is responsible for providing community and

3.13 ering the program through four regional offices custodial programs and services to Ontario youths that co-ordinated and managed Youth Justice Ser- aged 12–17 who have been charged with a crime

Section vices across Ontario through 64 probation offices, and are awaiting trial or have been found guilty by almost 500 community-based programs and servi- VFM the court. The Ministry also provides some services •

ces, 45 open custody/detention facilities operated 3 for youths at risk of committing a crime. The goals by transfer-payment agencies, and 20 secure cus- of the Youth Justice Services program are to reduce tody/detention facilities, six of which are operated

Chapter reoffending, contribute to community safety, and by the Ministry and the rest of which are operated prevent youth crime through rehabilitative pro- by transfer-payment agencies. The facilities had a gramming, holding youth accountable and creating total capacity of approximately 1,000 beds. opportunities for youths at risk. In the 2011/12 fiscal year, the Ministry spent Less than 2% of youths in Ontario are involved $370 million on the Youth Justice Services program, with the youth justice system. Males account for including $168 million in transfer payments to 75% of all cases. approximately 200 community-based agencies. The During the 2011/12 fiscal year, the average daily federal government contributed $67 million toward population in Ontario’s youth justice system was these costs under various cost-sharing agreements. about 9,200, comprising 8,600 under community supervision (that is, where regular reporting to a probation officer is generally required) and 600 in a youth custody/detention facility—specifically, 200 Audit Objective and Scope in open custody/detention (that is, a type of group home in the community) and 400 in secure cus- The objective of our audit was to assess whether tody/detention (that is, a residential setting where the Ministry of Children and Youth Services (Min- youth are restricted from leaving). The median istry) had adequate oversight and management length of time in the system was one year for youths

292 Youth Justice Services Program 293 procedures in place over the Youth Justice Services requested and obtained an Order-in-Council issued program to ensure that: by the Lieutenant Governor of Ontario to access • legislative and judicial requirements, as well youth files for the purpose of conducting our audit as program policies and procedures, are being in accordance with the Auditor General Act. At the complied with; start of this audit, we confirmed with ministry rep- • financial and human esourcesr are being man- resentatives that this Order-in-Council was still in aged cost-effectively; and effect to allow us access to youth records. However, • the Ministry measures and reports on the this Order-in-Council does not grant us access to the effectiveness of its services and programs. medical, psychological and psychiatric reports on Senior ministry management reviewed and youths resulting from court-ordered assessments. agreed to our audit objective and associated audit These reports are made available to probation offi- criteria. The scope of our audit included a review cers and can provide them with very useful informa- and analysis of case files and other relevant files tion to support decisions on how best to rehabilitate and administrative policies and procedures, as well youths placed under their supervision and to protect as discussions with appropriate staff at the Min- the public. Because we did not have access to these istry’s head office and at the three regional offices reports, we did not assess the extent to which the and four probation offices that we visited. We also information in them was taken into consideration visited a mix of open and secure youth custody/ with respect to the specific programs and services detention facilities operated directly by the Ministry offered to the youths who required court-ordered or by transfer-payment agencies. In addition, we assessments of their rehabilitation needs. 3.13 employed a number of computer-assisted audit techniques to analyze youth records in relation to Section specific aspects of case management.

Summary VFM We also met with staff of the Provincial •

3 Advocate for Children and Youth to obtain their perspective on the program, and we reviewed their Within the past 10 years, the Youth Justice Services

latest reports on practices at two custody/detention program has undergone significant transition stem- Chapter facilities. We did not conduct any detailed testing ming from changes in federal legislation and the at facilities because both the Provincial Advocate shift in philosophy from a more incarceration-based for Children and Youth and the Ministry’s internal approach to a community-based rehabilitation audit team had conducted reviews at some facilities approach. Our best-practice research indicated that over the past few years. the trend in youth justice systems in other jurisdictions has also been to adopt a more community- based focus. This transition included, among other LEGISLATIVE SCOPE LIMITATION things, a decrease in the number of custody beds, In order to protect the privacy of youths, the federal an increase in community-based programs and ser- Youth Criminal Justice Act restricts access to youth vices, and the separation of youths in custody from justice records to persons specified under the Act adults in custody. or persons designated by the Governor General of Between the 2005/06 and 2010/11 fiscal years, Canada or the Lieutenant Governor of the appropri- total program expenditures have increased by 25% ate province. The Act does not name the Auditor to 30% while the total estimated number of youths General of Ontario as one of the people with access served by the program has increased by only 5%. to youth justice records. As a result, in 2006, when As well, ministry operating costs have grown at a conducting an audit of Children’s Aid Societies, we much faster rate than funding to transfer-payment 294 2012 Annual Report of the Office of the Auditor General of Ontario

agencies, even though the agencies have had to $3,012. The average daily cost per youth at increase the number of programs and services agency-operated secure-custody facilities provided due to the focus on community-based ranged from $475 to $1,642, while those rehabilitation. for ministry-operated facilities ranged from Our more significant observations were as $1,001 to $1,483. follows: • Despite low utilization rates and probable • The growth in direct operating costs is due overstaffing in Ministry-operated secure facili- to a large extent to an increase in employee ties, in the 2010/11 fiscal year the Ministry costs. Over the five-year period ending incurred $3.9 million in overtime costs for 2010/11, all youth justice program areas, all youth services officers and an additional except for probation offices, saw a substantial $11.7 million to supplement youth services increase in the number of full-time employees. officers with contract staff. Staff working in Ministry-operated secure • A good initiative has been the introduction facilities account for more than 60% of all of the “single-case-management” model, full-time ministry staff involved with the where an attempt is made to have youths Youth Justice Services program. Although report to the same probation officer while in the average daily youth population in these the system and if they re-enter the system. facilities decreased by 37% from 2006/07 to As well, in our review of case files, we noted 2010/11, front-line staff levels have moved numerous instances where the knowledge in the opposite direction, with the number of and experience of probation officers was put 3.13 full-time youth services officers increasing by to good use in managing the needs of the 50%. In an attempt to better understand these youths. However, many of the required risk Section trends, the Ministry has hired consultants in assessments and identified rehabilitation VFM the past few years to review cost disparities needs were not being documented, and many •

3 in open custody/detention facilities, and to court-ordered conditions were either not review staff scheduling practices in Ministry- being complied with or we could not deter-

Chapter operated facilities. mine whether they had been complied with • In the 2010/11 fiscal year, the average bed due to a lack of documentation or because utilization rate for open and secure custody/ the condition was unverifiable. detention facilities was about 50%. Over the • The caseload for probation officers dropped years, the Ministry has made attempts to from 33 youths per officer in the 2005/06 improve the utilization rates by reducing the fiscal year to 26 per officer in 2010/11. While number of beds available in the system, either there is no written policy or guideline with through facility closures or by funding fewer respect to caseloads, probation managers told beds in existing facilities. However, even with us that the historical benchmark has been these efforts, the Ministry projects that, as a around 30–35 cases per officer. result of its community-based rehabilitation • We found little or no correlation between the philosophy, the overall utilization rate will target for the number of youths to receive still be only 58% by the 2012/13 fiscal year. community-based programs or services and • The average daily cost per youth in custody/ the amount of annual funding approved for detention facilities varies significantly by service providers. We also found that the facility. For example, in 2011, the average actual number of youths served in prior years daily cost per youth for agency-operated had little impact on approved funding for the open-custody facilities ranged from $331 to following year. Youth Justice Services Program 295

The Ministry aims to deliver evidence-based • OVERALL MINISTRY RESPONSE rehabilitation programs to youths, and it therefore collects academic research on The Ministry appreciates the work of the Aud- the most effective treatment models and itor General and his staff and welcomes their approaches, which it calls the “What Works input on how we can further improve youth Literature.” However, although this is a good justice services in Ontario. initiative, we found no documentation or Since the creation of the Ministry of Children other evidence to show that programs and and Youth Services in 2004, we have trans- services available to youths were actually formed our service delivery model to better evidence-based or aligned with the literature. meet the needs of youth in, or at risk of, conflict The Ministry assumes that transfer-payment with the law. Over the past eight years, the Min- agencies use the What Works Literature to istry has successfully repositioned youth justice develop evidence-based programs and servi- services from a predominantly custody-focused ces. But the Ministry’s regional staff whom we system to one that is more community-based. spoke with and who were responsible for con- Transformation required integration and tracting with service providers and evaluating realignment of two previously separate and the programs and services offered were gener- distinct systems and disentanglement of ser- ally not aware of the What Works Literature. vices between ministries to build a dedicated As well, given that most programs and servi- youth justice system that would respond to the ces are community-based, we expected that principles, provisions and sentencing options of 3.13 ministry staff would do site visits to assess the the Youth Criminal Justice Act (Act). It is import- programs being offered but saw no evidence ant to note that the Act was introduced by the Section that they did so. We were told that regional federal government to achieve a policy shift for VFM program staff conduct informal evaluations youth justice in Canada. The intent of this shift •

3 with input from probation officers. was to reduce the use of custody; increase the • Ministry recidivism statistics exclude more use of community-based alternatives; maintain

than 80% of the youths who have come youth justice separate from the adult system; Chapter into contact with the Youth Justice Services and focus on diversion, rehabilitation, reintegra- program. The recidivism rate for youths with tion and addressing the underlying causes of community-based sentences who are tracked offending behaviour by youth. has remained relatively stable over the past The new service delivery model is a con- five years, at 35%, while that for youths with tinuum of community and custodial services custody sentences who are tracked has gotten that range from prevention/diversion programs slightly better, decreasing to 59%. to providing custody and detention. The Min- • In 2010, the Ministry began to pilot exit istry uses the significant research about “What surveys to collect feedback from youths upon Works” to develop programs designed to reduce their release from custody/detention facili- reoffending, address the needs and risks of spe- ties and upon completion of their probation cial populations such as Aboriginal youth and period. The Ministry expects to implement youth with mental health issues, and provide both types of exit surveys as a province-wide gender-specific programming. The research also initiative next year. informs the development of policies, tools and probation case-management service delivery. 296 2012 Annual Report of the Office of the Auditor General of Ontario

principles of this new Act was that a young person The Ministry’s policies and processes for in custody or detention must be held separate and monitoring compliance and for use of resources apart from adults in custody or detention. This have been guided by: caused the Ontario government to begin planning legislative requirements; • for a transition of youths from adult institutions the shifts from custody to community experi- • to facilities designated as youth-only. In addition, enced under the Act; the Act indicated that the appropriate level of the responsibility to address the often • custody for a young person is the one that is the complex needs of youth in order to improve least intrusive, after taking into consideration the outcomes for them and reduce reoffending; seriousness of the offence, the youth’s needs and and circumstances, and the safety of the young person the need to maintain the safety of youth, • and society. staff and the community. On August 1, 2003, the legal and financial A Strategic Plan is in place to guide work responsibility for Phase 1 youths was transferred over the next two to five years in order to from the Ministry of Community and Social Servi- strengthen and review the gains made during ces to the Ministry of Community Safety and Cor- transformation. The Ministry is committed to rectional Services. On April 1, 2004, responsibility taking action in response to the Auditor Gen- for all youths was again transferred to the newly eral’s recommendations, which will enhance formed Ministry of Children and Youth Services, in youth justice services in Ontario. order to clearly separate the youth justice system 3.13 from the adult system. The transformation of the youth justice system Section from 2003 to 2009 involved: VFM Detailed Audit Observations the transfer of almost 1,400 staff from the •

• 3 Ministry of Community Safety and Correc- tional Services to the Ministry of Children and TRANSFORMATION OF THE YOUTH

Chapter Youth Services; JUSTICE SYSTEM • the net reduction of 478 beds in secure Prior to April 1, 2003, the federal Young Offend- custody/detention facilities (achieved by ers Act governed the prosecution and sentencing closing 429 beds in 11 adult correctional of youths for criminal offences across Canada. facilities, opening 252 beds in five newly built Among other things, it outlined the requirements youth-only facilities and further eliminating regarding custody and supervision. During this 301 beds due to the decline in the number of time, Ontario offenders under age 18 were served secure-custody court judgments); by two ministries: the Ministry of Community and • the reduction of 550 beds in open custody/ Social Services provided services to youths aged detention facilities; and 12 to 15 (referred to as Phase 1), and the Ministry • a 50% increase in the number of community of Community Safety and Correctional Services programs and services. provided services to youths aged 16 to 17 (referred Some services—such as training of custodial to as Phase 2). In the latter case, youths in custody staff, nutritional services, information technology were often accommodated in designated units and statistical services—continue to be provided by within adult correctional institutions. On April 1, the Ministry of Community Safety and Correctional 2003, the Young Offenders Act was replaced by the Services under a service-level agreement. Youth Criminal Justice Act (Act). One of the major Youth Justice Services Program 297

With the introduction of the Youth Criminal The Ministry provides funding to support more Justice Act in 2003, the trend has been a decrease in than 30 youth-mental-health court-worker pos- the use of custody and an increase in community- itions. Extrajudicial sanctions are either directly based programs to rehabilitate offenders under managed by the Ministry through its probation the age of 18. Police and Crown prosecutors are offices or administered by transfer-payment agen- diverting from the court process youths who have cies contracted by the Ministry. The probation committed the least serious offences; for those officer or the transfer-payment agency is respon- youths who go through the court process, custody sible for selecting what action is to be taken by the sentences tend to be reserved for the most serious youths serving the sanctions, overseeing comple- and repeat offenders. tion of these actions and informing the Crown attorney when each case is successfully completed. If neither the police nor the Crown divert the THE YOUTH JUSTICE SYSTEM PROCESS youth from the criminal justice system, the youth, When a youth is accused of committing an offence, represented by legal counsel, is taken before a the police will investigate the incident to determine judge who will decide whether to hold the youth in whether to lay charges. The police may choose not detention until trial. If the court decides the youth to charge the youth if the offence is non-violent in is to be held until trial, the Ministry will use criteria nature and the youth has no prior convictions. In outlined in the Child and Family Services Act to such circumstances, the police may use other meas- decide whether to place the youth in open deten- ures to divert the youth from the criminal justice tion or secure detention. The actual facility where 3.13 system. Other measures can take many forms; for the youth is placed is based on proximity to home, example, the officer may decide to take no further gender and any specific risks the Ministry identifies Section actions against the youth, give the youth a verbal during first contact with the youth, such as possible VFM warning or refer the youth to programs in the gang affiliations. The length of stay in detention •

3 community. Although the Ministry funds some of can range from a few days to more than a year, these community-based programs, it does not have depending on a number of factors, including the

any involvement in managing the cases of youths severity of the charges and when the trial ends. Chapter diverted by police. When requested by the court, the Ministry is If the youth is charged, the Crown prosecutor required to provide assistance during court pro- also has an opportunity to divert the youth from ceedings, such as preparing pre-sentence reports the criminal justice system through one of the fol- for the court and co-ordinating other assessments lowing methods: regarding the youth’s suitability for more intensive • referral to a Youth Justice Committee, whose support, rehabilitation and supervision. The Min- role is, among other things, to find services istry also pays for medical and psychiatric assess- and people in the community to provide ments requested by the courts. supervision and mentoring; If the youth is convicted, the court determines • referral to a youth-mental-health court worker the type and length of sentence as well as any con- who can connect youths to community-based ditions applied to the sentence. The court imposes mental health programs; and one of two types of sentences, which are managed • extrajudicial sanctions, which usually require by the Ministry: the youth to perform one of the following • Custody—The youth is placed in an open- or actions: apologize or make restitution to the secure-custody facility for a specified period victim, write an essay about the offending of time. These are the same facilities used behaviour, donate money to a charity, perform to hold youths in detention until trial. All community service, or attend counselling. 298 2012 Annual Report of the Office of the Auditor General of Ontario

custody terms are followed by a period of budgets, and conducting annual inspections of community supervision equal to one half of custody/detention facilities. Probation offices the custody period. provide supervision and case-management services • Community supervision—The youth is to most youths in Ontario convicted of a crime. returned to the community to serve the The Ministry contracts with transfer-payment sentence, which can be expressed in terms of agencies to deliver all other services, including all days as in the case of a probation order, or in community-based programs and services, and to terms of hours as in the case of a community- operate the 14 remaining secure custody/detention service order. Community supervision facilities and all 45 open custody/detention facili- typically includes regular reporting to a pro- ties. The transfer-payment agencies, the majority of bation officer and may require counselling or which are not-for-profit organizations, have been rehabilitation. providing services to the youth justice program for In general, every sentenced youth is assigned to a number of years. a probation officer to manage the youth’s case while in the system. The probation officer is responsible PROGRAM EXPENDITURES for identifying the youth’s needs, assessing his or her risk to reoffend, and developing a case- As a result of the shift in emphasis to a more management plan to meet those needs and mitigate community-based response for dealing with those risks. Based on the plan goals, the probation youths in conflict with the law, the number of officer connects the youth with programs and ser- youths in community-based programs almost 3.13 vices in the community, such as anger management doubled between 2005/06 and 2010/11. Over counselling. Youths who are sentenced to custody the same period, the number of youths placed by Section also are assisted and monitored by youth services court order in custody and detention dropped by VFM officers who work in the facilities. These officers are 26%. However, contrary to what one might expect, •

3 involved with the probation officers in developing this shift toward lower-cost community services case-management plans for youths in facilities, and has not brought about a decrease in total program

Chapter assist the youths in carrying out the plan’s program- costs. In fact, program costs from the 2005/06 to ming and objectives while in custody. the 2010/11 fiscal years have increased for each Programs and services to youth in conflict with type of major service activity (that is, prevention, the law are provided through the Ministry’s Youth diversion, agency-delivered community-based Justice Services Division. As of March 31, 2012, the services and programs, probation, reintegration/ Division, with a staff of more than 1,700 people, rehabilitation, and custody/detention). comprised a corporate office (with 117 staff), four Over the past several years, expenditures for regional offices (with 69 staff), 64 probation offices the Youth Justice Services program have steadily (with 512 staff) and seven secure custody/deten- increased, even though the estimated number tion facilities (with 1,039 staff). Corporate office of youths served has remained relatively stable, provides leadership and strategic direction in the as shown in Figure 1. Between the 2005/06 and development of policy, procedures and program- 2010/11 fiscal years, the total estimated number ming, and is responsible for, among other things, of youths in community and custodial programs staff training, investigations and financial planning. increased by only 5%, from about 29,000 to about Regional offices co-ordinate and manage youth 30,500, while total program expenditures increased justice services across the province, including con- by more than 30%, from $273 million to $362 mil- tracting with and funding transfer-payment agen- lion. (The Ministry informed us that the $273 mil- cies, monitoring expenditures against approved lion in spending for 2005/06 is understated Youth Justice Services Program 299

in 2010/11. Over this period, all Youth Justice Ser- Figure 1: Total Program Expenditures and Estimated vices program areas, except for probation offices, Number of Youths Served, 2005/06–2010/11 saw a substantial increase in the number of full- Source of data: Ministry of Children and Youth Services time employees, as shown in Figure 2. Staff work- 40,000 400 ing in Ministry-operated secure facilities account 35,000 350 for more than 60% of all full-time Ministry staff 30,000 300 involved with the Youth Justice Services program. 25,000 250 Although the average daily youth population in uths Served

Yo 20,000 200 these facilities decreased by 37% from 2006/07 to

15,000 Estimated number of youths served* 150 m Costs ($ million) 2010/11, front-line staff levels have moved in the Total program cost opposite direction with the number of full-time

Number of 10,000 100 Prog ra 5,000 50 youth services officers increasing by 50%. Most of this increase occurred in 2008 and 2009 when 0 0 three newly built, Ministry-operated facilities

2005/06 2006/07 2007/08 2008/09 2009/10 2010/11 became operational. In the 2010/11 fiscal year, the Ministry operated two-thirds of all secure beds * The Ministry tracks the number of services provided but not the number of individual youths being served. As youths often receive multiple services, in the system, yet received three-quarters of total counting services overstates the number served. We therefore estimated the number of youths served for each year by counting individuals in funding for secure facilities. The Ministry acknow- prevention programs, in diversion programs, assigned to probation officers ledged to us that the facilities it operates directly and placed into custody/detention facilities. are likely overstaffed at the present time and has 3.13 because it did not include the costs incurred by the started to take action that it expects will reduce Ministry of Community Safety and Correctional staffing levels. Section Services for operating youth custody/detention VFM units within adult facilities, the costs of which were • Utilization Rates 3 not available. In 2009/10, after the units were closed down, the Ministry took on the cost of all Custody and detention services provide residential

youth facilities, for which it received an annual care and 24-hour supervision. Understandably, this Chapter budget increase of $18 million. While the Ministry is the most costly service activity. The rise in the had no data for the years before 2009/10, if an cost of custody/detention facilities coupled with adjustment was made for this, program expenditures would still have increased 25%–30% since Figure 2: Full-time-equivalent Ministry Staff Working 2005/06.) During the same period, federal funding for the Youth Justice Services Program for the Ministry’s program provided under various Source of data: Ministry of Children and Youth Services cost-sharing agreements remained relatively steady Increase at approximately $65 million per year. Location/Staff Type 2005/06 2010/11 (%) Between the 2005/06 and 2010/11 fiscal years, Corporate office 75 114 53 ministry operating costs grew proportionately more Regional offices 55 69 25 than funding to transfer-payment agencies (47% Probation officers 378 384 2 and 19%, respectively), even though the number Other probation office 113 127 13 of Ministry-funded programs and services offered staff by transfer-payment agencies grew by almost 40%. Youth services officers 472 675 43 Other ministry-operated The largest increase in ministry direct operating 283 415 47 secure-facility staff expenditures was employee costs, which went up Total 1,376 1,784 30 57%, from $94 million in 2005/06 to $148 million 300 2012 Annual Report of the Office of the Auditor General of Ontario

a decrease in the number of youths admitted to utilization rates by reducing the number of beds such facilities has led to lower utilization rates and available in the system, either through facility higher per diem rates. closures or by funding fewer beds in facilities run “Utilization rate” is the term used to describe primarily by transfer-payment agencies. However, the percentage of occupied beds in a custody/ the rate of bed reductions has not kept pace with detention facility over a specified period of time, the decreased need, particularly in secure facilities. usually a year. At the start of our audit, the Ministry For example, from the 2006/07 to 2010/11 fiscal informed us that the optimal utilization rate for years, the average daily resident count in secure both the open and secure systems was 80%. By and open custody/detention facilities has decreased the end of our audit, the Ministry had revised this by 26% and 9%, respectively. In comparison, the target to 70%. We researched utilization rates Ministry has decreased the number of beds avail- elsewhere and were able to find only two other able in secure facilities by 18% and in open facilities jurisdictions that have established target utilization by 22%. As a result, the utilization rate in secure rates—the state of Virginia has established a target facilities has gotten worse, while it has improved in utilization rate of 80%–85% for its youth centres, open facilities. We noted that for secure facilities, and Wales in the United Kingdom has set a target most of the decrease in the average daily resident utilization rate of 93%. count has occurred in Ministry-operated facilities, Figure 3 summarizes the average daily resident yet bed capacity has been reduced proportionately count, daily capacity and utilization rates for secure more in agency-operated facilities. and open facilities for the past five years. For the During our audit, the Ministry undertook 3.13 2010/11 fiscal year, the average utilization rate for another round of facility closures and bed reduc- open and secure facilities was about 50%. Secure tions to try to improve utilization rates. In February Section custody/detention facilities operated by the Min- and March 2012, the Ministry closed one Ministry- VFM istry had a lower utilization rate (49%) than those operated secure custody/detention facility, reduced •

3 operated by transfer-payment agencies (60%). beds in two other Ministry-operated secure facilities With the shift away from incarceration to more and closed four agency-operated open facilities.

Chapter community-based methods of dealing with youths Aside from custody/detention units that were in conflict with the law, the average daily resident closed because they were located in adult cor- count has decreased as expected. Over the years, rectional facilities, by the time we completed our the Ministry has made attempts to improve the fieldwork, the Ministry had removed in total 442

Figure 3: Average Daily Capacity and Actual Occupancy at Youth Custody/Detention Facilities, 2006/07–2010/11 Source of data: Ministry of Children and Youth Services

Secure Facilities Open Facilities All Custody/Detention Facilities Daily Daily Daily Resident Daily Utilization Resident Daily Utilization Resident Daily Utilization Fiscal Year Count Capacity Rate (%) Count Capacity Rate (%) Count Capacity Rate (%) 2006/07 525 883 60 270 616 44 795 1,499 53 2007/08 520 785 66 288 612 47 808 1,397 58 2008/09 478 751 64 234 525 45 713 1,276 56 2009/10 440 724 61 257 478 54 697 1,202 58 2010/11 388 727 53 244 479 51 632 1,205 52 Change (%) (26) (18) (9) (22) (20) (20) Youth Justice Services Program 301 secure beds from the system by closing five facilities Figure 4: Average Daily Cost per Youth at Custody/ and downsizing eight others, and had removed 593 Detention Facilities, 2006/07–2010/11 ($) open beds from the system by closing 51 facilities Source of data: Ministry of Children and Youth Services and downsizing 11 others. The Ministry advised us 1,400 that, in assessing closures and downsizing, it must take into consideration not only the average daily 1,200 resident count but also special circumstances where 1,000 more than the average number of beds is required. This may be one reason why the Ministry projects 800 that in the 2012/13 fiscal year, the overall utiliza- 600 tion rate will improve to only 58%, well below its 400 targeted utilization rate. Ministry-operated secure facilities 200 Agency-operated secure facilities Agency-operated open facilities* Per Diem Costs 0

“Per diem” is the term used to describe the cost 2006/07 2007/08 2008/09 2009/10 2010/11 per day per youth in a custody/detention facility. * There are no Ministry-operated open facilities. Over the past few years, average per diem costs capacity rather than actual utilization) were largely have been consistently higher for Ministry-operated due to the different funding approaches of the for- facilities than for their agency-operated counter-

mer “Phase 1” and “Phase 2” systems. Specifically, 3.13 parts. Figure 4 shows how the average per diem most former Phase 1 facilities (for youths aged 12 costs have increased for every type of facility from

to 15) had per diem costs that were greater than the Section the 2006/07 to the 2010/11 fiscal year, but espe- provincial average, while most former Phase 2 facili- VFM cially for secure-custody facilities directly operated •

ties (for youths aged 16 and 17) had per diem costs 3 by the Ministry, where average per diem costs have that were less than the provincial average. Other increased by 86%. In contrast, average per diem factors that contributed to disparities included

costs increased by only 4% for agency-operated Chapter whether workers were unionized, facility capacity, secure-custody facilities and by 30% for open- residents’ gender, and whether the facility offered custody facilities. gender-specific programming. The consulting firm We also noted that per diem costs vary identified inconsistent funding practices and large significantly even for similar types of facilities. variances among facilities, and recommended For example, in 2011, per diem costs for agency- a standardized approach. The consulting firm operated open-custody facilities ranged from $331 developed a funding tool to enable more compar- to $3,012. Per diems at agency-operated secure- able funding among service providers. However, as custody facilities ranged from $475 to $1,642, noted by the consulting firm, the tool does not take while those at Ministry-operated facilities ranged into consideration the quality or scope of services from $1,001 to $1,483. being provided by the various facilities when ana- In 2008, the Ministry hired a consulting firm to lyzing the appropriate funding. In response to the conduct a cost analysis of open custody/detention consultant’s report, the Ministry increased funding facilities in order to identify the factors contribut- in the 2009/10 fiscal year for those open custody/ ing to significant cost differences across the system detention facilities whose per diem costs at full cap- and to provide a funding method to support fair acity were below the provincial average, and did not funding decisions. The consulting firm reported that change funding for facilities whose per diem costs at disparities in per diem costs (calculated based on full capacity were above the provincial average. 302 2012 Annual Report of the Office of the Auditor General of Ontario

RECOMMENDATION 1 costs to identify the impact of underutiliza- tion and confirm budget reasonableness. The To help ensure that spending for secure and Ministry will conduct more detailed review of open custody/detention facilities in the Youth those facilities where the per diems significantly Justice Services program is commensurate with exceed the provincial average and make adjust- the need for services, the Ministry of Children ments where warranted. As well, the Ministry and Youth Services should: will conduct a jurisdictional scan of comparable take additional steps to improve utilization • service providers to compare the reasonableness rates by reducing bed capacity in signifi- of the Ministry’s per diem costs. cantly underused facilities; and • review all facilities’ per diem costs for reasonableness and reduce funding for those whose per diem costs significantly exceed FACTORS INFLUENCING EMPLOYEE the provincial average, keeping in mind the COSTS AT MINISTRY-OPERATED quality and scope of services provided by FACILITIES each facility. Despite the low utilization rates and probable overstaffing in Ministry-operated secure facilities, the MINISTRY RESPONSE Ministry is also using more contract staff and incur- The Ministry agrees that program expenditures ring overtime costs. In the 2010/11 fiscal year, the

3.13 for secure and open custody/detention facili- Ministry incurred $3.9 million in overtime for all

ties must be commensurate with the need for youth services officers, and an additional $11.7 mil-

Section services. Utilization in the youth justice sector lion to supplement youth services officers with

is determined by the degree to which courts contract staff. According to an external review of VFM

•

make orders for detention and sentence youths staffing costs at directly operated facilities in 2011, 3 to custody. costs of contract staff and overtime were incurred The Ministry regularly monitors bed because of absenteeism, medical accommoda- Chapter utilization, taking into account fundamental tions, training, vacancies, scheduling practices and service principles such as providing services deployment practices. We reviewed some of these as close to a youth’s home community as pos- causes to determine if the Ministry had taken any sible, gender-dedicated services and dedicated mitigating actions to reduce the associated costs. services for Aboriginal youth in conflict with the law. In 2012/13, the Ministry will under- Sick Leave take further review of the open and secure custody/detention system to determine if other In 2008, youth services officers at custody/deten- capacity reductions are possible, while respect- tion facilities took an average of 21 sick days, about ing requirements related to maintaining safe two times the provincial average sick leave for gov- environments, meeting legislative requirements ernment employees for that year. In our last audit for security, meeting the unique needs of youth of adult correctional facilities, we found that this and maintaining capacity for potential increases issue is also common among corrections officers at as a result of the 2012 federal amendments to those facilities. To reduce absenteeism across youth the Youth Criminal Justice Act. custody/detention facilities and adult correctional In 2012/13, the Ministry will start a review facilities, the Ontario government implemented the of open and secure custody/detention per diem following initiatives: Youth Justice Services Program 303

• In August 2009, it introduced the Attendance stated that although the initiative has good Support and Management Pilot Program intentions, as it currently works it rewards for youth services officers, youth probation both those misusing and those not abusing officers, corrections officers, adult probation sick-time credits. The review also indicated officers and their managers. After a certain that there seemed to be a sense of entitlement number of absences, a meeting is triggered among staff at many of the Ministry-operated with the employee to discuss his or her facilities with regard to the use of sick time. attendance record and offer assistance such as information about the Employee Assistance Medical Accommodations Program. A progress report issued by the Min- istry of Government Services in September When an employee is temporarily or permanently 2011 provided results in aggregate. We calcu- unable to perform the essential duties of his or her lated that 45% of youth services officers and job because of a medical condition or disability, 19% of youth probation officers were required the Ministry is required by a government-wide to participate in the program between August directive to accommodate the worker in another 2009 and June 2011. The Ministry did not position that does not demand the same physical have information on the number of employees abilities. For example, a youth services officer hired who successfully exited the program, but the as a front-line worker with youths in a facility who Ministry of Government Services reported can no longer perform his or her duties might be that for the adult and youth systems com- put into an administrative position. We found that 3.13 bined, two-thirds of employees who entered in the 2010/11 fiscal year, 6.5% of youth services the program successfully reduced their sick officers in Ministry-operated facilities were on long- Section days to below seven days for one year. term permanent accommodation. One facility had VFM The government negotiated a bonus payout to as much as 17.5% of full-time youth services officers •

• 3 all youth services officers and corrections offi- in such temporary or permanent accommodation cers beginning March 2009 if their combined positions. This occurred more frequently at facili-

provincial sick time average was less than or ties that have been in operation for some time. Chapter equal to the targets set out in the Ontario Pub- lic Service Employees Union Collective Agree- Staff Scheduling Practices ment, which in 2011 was 20 days. Since this initiative was introduced, actual sick days for Custody/detention facilities are staffed based on youth services officers dropped from 21 to 14 100% capacity. Although contract staff are to be in the past three years. The Ministry has paid used on an as-needed basis, we understand that out a total of $1.5 million in bonuses from many Ministry-operated facilities fill vacancies this initiative. At our request, the Ministry of with contract staff without evaluating the needs of Government Services estimated the cumula- the site on that particular day—that is, taking into tive net savings to be $4.5 million, but we consideration the occupancy level. This leads to could not confirm whether this estimate was overstaffing, since on average only half the beds in reasonable because of conflicting information a facility are occupied. on sick days obtained from the Ministry of Furthermore, we noted that even if all the beds Children and Youth Services and the Ministry were filled, the facilities might still be overstaffed. of Government Services. Under the Child and Family Services Act (Act), all The external review of employee costs at custody/detention facilities are required to main- Ministry-operated custody/detention facilities tain a minimum staff-to-youth-in-custody ratio of 304 2012 Annual Report of the Office of the Auditor General of Ontario

1:8—that is, one youth services officer for every immediate future the youth is likely to cause serious eight youths in a facility. In this context, “staff” property damage or serious bodily harm to another refers to youth services officers only and excludes person, and no less restrictive method of managing facility management, administrative staff and other the youth’s behaviour is practical. A youth services staff with specialized duties such as kitchen staff. officer who is assigned to watch a youth in secure Although operators of custody/detention facilities isolation is not available to supervise others in the prepare staffing schedules, the Ministry does not facility. We found that Ministry-operated facilities monitor staff-to-youth ratios even for the facilities make use of secure isolation rooms more often than it operates directly. During one of our visits to a agency-operated facilities. For example, in 2011 Ministry-operated facility, we noted the facility was more than 90% of the placements in secure isola- routinely overstaffed in comparison to the legislation occurred in a Ministry-operated facility, and tion, as the staff-to-bed ratio in the residential units the average length of stay in secure isolation was 25 was 1:3 at peak hours (4 p.m.– 9 p.m.) and 1:5 at hours in a Ministry-operated facility compared to non-peak hours (overnight). We were also told at two hours in an agency-operated facility. this facility that there were occasions when some The Ministry indicated that one of the reasons youths who were excessively disruptive received secure isolation was used more in Ministry-operated one-to-one and two-to-one supervision to de- facilities is that suspected gang members are typ- escalate situations. These staffing situations would ically accommodated in ministry facilities and that not be captured in the ratio above. these youths are more frequently engaged in behav- At one of the facilities we visited, we observed iour that creates an imminent risk to others, which 3.13 other staff scheduling practices that result in high is one of the criteria for using secure isolation. But employee costs: we noted that the percentage of youths residing in Section • Both the infirmary and the area with the isola- secure facilities who were suspected of belonging to VFM tion rooms are staffed with at least one youth gangs as of December 31, 2011, was comparable— •

3 services officer at all times, whether or not that is, 15% of youths in Ministry-operated facilities they are occupied. compared to 12% of youths in agency-operated

Chapter • Youths who are suicidal require constant facilities. Furthermore, we noted that when one two-on-one supervision. Ideally, such youths Ministry-operated facility, which routinely used should be in a facility that specializes in men- secure isolation, was closed in March 2012 and its tal health issues or in a psychiatric hospital. youths were sent to an agency-operated facility, the Currently only one facility in the youth justice agency-operated facility reported no use of the isola- system is accredited as a children’s mental tion room in the following month. health centre, and it often does not have the capacity to meet the demand for its services. RECOMMENDATION 2 For example, we noted that in 2011, about 265 To help reduce employee operating costs, youths were identified as suicidal, and only particularly at Ministry-operated facilities, the 18 spent any length of time in this specialized Ministry of Children and Youth Services should: facility. • staff custody/detention facilities on the basis of expected utilization and not on the basis Use of Secure Isolation of full capacity, and use contract staff to fill vacancies only after evaluating the short- Provisions under the Act allow a secure custody/ term staffing needs of the site; detention facility to place a youth in secure isolation if that youth’s conduct indicates that in the Youth Justice Services Program 305

• reassess whether the bonus payouts from the Ministry of Government Services will reassess sick-day program are proving to be a cost- the program, the policy and the incentive pay- effective strategy in reducing absenteeism; ments as it prepares for collective bargaining • investigate high rates of long-term perma- with OPSEU. The Ministry of Children and nent medical accommodation and, where Youth Services will work with the Ministry of appropriate, implement measures to reduce Government Services in this regard. those rates; and The Ontario Public Service has recently • identify behaviour-management techniques initiated a disability management review pro- other than secure isolation that have been cess with the goal of implementing best prac- used successfully by agency-operated tices to enhance employment accommodation facilities to prevent or manage undesirable and return-to-work performance and outcomes. behaviour. A component of the review involved completing an assessment of current practices and perform- MINISTRY RESPONSE ance relating to disability management policy and workplace resources, disability prevention, The Ministry acknowledges the importance of and early intervention and return-to-work pro- effective staffing practices at its directly operated cesses. As a result of this review, an action plan facilities. Staffing in custody/detention facilities will be developed to implement enterprise-wide must be done to meet the minimum staff ratio program improvements. The Ministry will work under the Child and Family Services Act; the

with the Ministry of Government Services in 3.13 requirement for 24/7 intake; ability to maintain this regard. the buildings and provide key services (for

We acknowledge the Auditor’s concern relat- Section example, programming, education, health, clin- ing to the use of secure isolation. The Ministry is VFM ical, case management) regardless of how many •

undertaking a review of the use of secure isola- 3 youth are present; as well as safety and security tion in directly operated facilities to measure requirements. The Ministry is in the process of operational compliance with legislation, poli-

implementing the recommendations resulting Chapter cies and procedures. The Ministry has in place from the review of staff deployment and sched- training, a relationship custody approach, tools, uling practices undertaken in 2011. In addition, policies and procedures relating to behaviour- a scheduling application is under development, management techniques and the management which will result in more consistent and effective of aggressive behaviour by youth. Throughout staff scheduling practices, leading to decreased 2012–14, the Ministry will undertake reviews staff costs. The Ministry will continue to evaluate of directly operated youth centres using the capacity and take action where possible, while Correctional Program Assessment Inventory, addressing operational requirements, meeting which, among other things, is used to review the complex needs of youth in the facilities, and the types of behaviour-management strategies having due regard for the safety for youth, staff used, whether the strategies are aligned with and the community. the research and whether there are documented Over three years, the government has procedures for applying the strategies. In made substantial and measurable progress addition, the Ministry will identify behaviour- toward reducing absenteeism and its associated management techniques that have been used by costs, and sustaining this improvement across agency-operated facilities to determine whether the Ontario Public Service Employees Union their use is appropriate for its own facilities. (OPSEU) Correctional Bargaining Unit. The 306 2012 Annual Report of the Office of the Auditor General of Ontario

CASE MANAGEMENT oration with the youth and other stakeholders—the youth’s parents, for example, or the primary youth Case management is the cornerstone to helping services officer if the youth is in custody. The plan a youth become a law-abiding citizen, and there- details how the risks identified in the assessment are fore is core to contributing to community safety to be addressed, and typically specifies one or more and prevention of youth crime/reoffending. The programs that the youth must attend, how often the Ministry’s objective in case management is to take youth must report to the probation officer, and how maximum advantage of the time that a young per- to comply with the conditions of the court order. son is in the youth justice system, using that time If a youth is in custody, facilities must also to change his or her mindset and/or other aspects prepare a case-management reintegration plan of his or her situation that could reduce the likeli- for him or her within 30 days of admission, and hood of reoffending. update it 90 days after admission, six months after The time frame in which the youth is under the admission and every six months thereafter. The care of the Ministry and its service providers is lim- reintegration plan is intended to complement the ited. In the 2011/12 fiscal year, the median length case-management plan, and details what action of time a youth was in the system was one year for should be taken during the custody phase. a youth on probation, 40 days for a youth in secure We reviewed a sample of case files closed in custody, 60 days for a youth in open custody, seven 2011 for a variety of sentence types. Overall, we days for a youth in secure detention and four days found the system to be flexible and that the know- for a youth in open detention. Therefore, making ledge and experience of probation officers was

3.13 the most of this limited time is critical to realizing being put to good use. For example, although the the changes that the Ministry intends case manage- risk/needs assessment automatically calculates a

Section ment to achieve. risk score for the youth, we found instances where VFM the probation officer overrode the score on the •

3 Assessing Youth and Planning Services basis of personal observations and knowledge of the particular rehabilitation needs of the youth. The probation officer conducts a risk/needs assess-

Chapter However, we did identify some concerns with ment to assess the youth’s risk of reoffending and the risk/needs assessment process and case- determine the level of supervision needed. The pro- management planning: bation officer considers the youth’s history, lifestyle One-third of the risk/needs assessments and and personal circumstances. As well, we were told • case-management plans we reviewed were that the probation officer considers information completed late, on average by almost 60 days, contained in psychological and psychiatric reports while another third were never done, even resulting from court-ordered assessments. A youth though they are required by policy given the is rated as high, medium or low risk in eight areas: length of the youths’ sentences. Overall, only prior offences; family circumstances/parenting; 60% of the files reviewed contained all the education; peer relations; substance abuse; leisure required risk/needs assessments. and recreation; personality; and attitudes/orienta- Most case-management plans were incom- tion. The results of the assessment are then used to • plete. They failed to include all required items, decide on a supervision level and to develop a case- such as specific timelines for monitoring and management plan. referrals, or they did not indicate the manner In practice, the risk/needs assessment and the in which compliance with court-ordered con- case-management plan are one consolidated docu- ditions would be monitored. ment that the probation officer completes in collab- Youth Justice Services Program 307

• Although most case-management plans charges stemming from the terms of the court order included goals to address at least some of the he or she has breached, and taking no action. youth’s risk factors to reoffend, areas assessed The Ministry has a “single-case-management” as high risk were often not addressed. Overall, policy, in which a youth remains under the super- more than half of the files we reviewed for vision of the same probation officer for the duration youths with completed case-management of his or her court order, and any youth who comes plans did not have goals for at least one of back into the system is reassigned to that same pro- their high-risk factors. More than 20% of bation officer so long as the reassignment continues cases did not have goals for any of their high- to serve the youth’s best interest. This relationship- risk factors. building model is meant to emphasize the quality of • We found that reintegration plans were gener- interaction between the probation officer and the ally prepared on a timely basis. Reintegration youth; the importance of creating stable, consistent plans are meant to outline specific goals for care; and the value of developing individualized, the youth to work on while in custody. How- meaningful case-management plans and interven- ever, the goals in some plans were vague—for tions. We felt that this was a good policy but noted example, “youth should better himself by that the Ministry does not monitor whether its attending program or counselling as rec- single-case-management policy is being followed, ommended by clinical staff and probation particularly for youths who re-enter the system. In officer,” rather than specifying the particular addition, for the youths in our sample, which rep- in-house rehabilitation service or program resents a single continuous period of supervision, 3.13 that would benefit the youth. more than 40% were under the supervision of more than one probation officer, while 6% saw at least Section five probation officers during their period of super- Monitoring Efforts by Probation Officers VFM vision. In most cases, we were unable to determine •

3 A probation officer is the first line of contact for the reasons for the reassignments because they most sentenced youths throughout the time the were not documented in the file.

youths are in the care of the Ministry. Probation Our review of the work of probation officers Chapter officers are responsible for monitoring the youth’s found the following: progress in meeting the goals set out in the case- • For a significant number of files, there was no management plan and making referrals to programs evidence that probation officers had made a and services based on the conditions set out in the sufficient effort to assist youths to meet at least court order, the case-management plan, and other some of their established goals. For example, needs of the youth identified during the supervision where a youth’s goal was to abstain from period. Probation officers must also monitor compli- substance abuse, the probation officer made ance with all conditions set out in the court order no referral to a substance-abuse support group in accordance with the Youth Criminal Justice Act. or counselling. In another case, the youth When a youth does not comply, probation officers had a goal to complete four credits at school, are required under ministry policy to consider but there was no evidence in the file that the appropriate enforcement options and to document probation officer had worked with the youth their decision. Enforcement options can include on the issue of school attendance. In both issuing a warning, increasing the level of super- examples, the youths did not meet their goals. vision, asking the court to change the conditions of • Youths are required to meet regularly with the court order, bringing the youth back to court on their probation officers, but in almost half of the case files we looked at, we found youths 308 2012 Annual Report of the Office of the Auditor General of Ontario

had not reported to their probation officers as sanctions. We found that there are no Ministry frequently as required given their risk level. guidelines on the type of sanction to be used This was due to probation officers failing to for the type of offence or type of history—for schedule appointments as frequently as they example, if the youth is a repeat offender. should have, or to follow up on appointments Consequently, sanctions employed varied by that the youths missed. case and by probation officer. For example, • Thirty percent of the community-service for the same offence one youth was assigned orders were not complied with on a timely community-service hours while another only basis. As well, we could not verify compliance had to write a letter of apology. Although these for an additional 7% of community-service might well have been appropriate sanctions for orders because there was no documentation each youth given the individual circumstances, in the case file. supervisory staff have no way of evaluating the • Less than 40% of conditions in the files appropriateness of the sanction because none we examined were complied with. For the of the files sampled contained the underlying remainder, almost 20% were not met and 40% rationale for the sanction chosen. could not be verified, either because there was a lack of documentation in the file or because Probation Officer Caseloads the conditions set out in the orders were unverifiable—for example, “keep the peace,” Each probation officer takes responsibility for a “don’t possess a firearm,” or “don’t associate certain number of youths and manages them dur- 3.13 with person X.” ing their stay in the youth justice system. Although • When a probation officer is aware that a there is no documented caseload standard for Section court order has been breached, it is his or her probation officers, probation managers we met VFM responsibility to take appropriate action. We with indicated that a benchmark of 30–35 cases per •

3 would expect this action to be in the form of probation officer was considered reasonable. an escalation of consequences, depending on According to the Ministry’s analysis, the case-

Chapter the circumstances, starting with a warning load for probation officers dropped from 33 youths and ending by bringing the youth back to per officer in the 2005/06 fiscal year to 26 youths court on charges of breaching the terms of the per officer in 2010/11. The Ministry told us that it court order. However, in more than a quarter responded to this decrease by shifting the super- of the cases sampled where the court order vision of community-service orders from transfer- was breached, the probation officer did not payment agencies to probation officers in 2011. take timely and/or appropriate action through progressive enforcement. The Ministry indi- Ensuring Quality of Case Management cated that the courts discourage probation officers from bringing forward breaches that Probation managers are required to review case might not be considered serious in nature, files annually to monitor probation officers’ compli- and that the courts expect probation officers ance with standards, policy and procedures. The to find ways of enforcing conditions without Ministry has developed a checklist—with more resorting to returning the youth to court. than 100 standards on case-file documentation • For youths who have committed crimes that and monitoring—for probation managers to use to are not serious in nature, the Crown may assist them with this oversight responsibility. We divert them from the court system if the attempted to evaluate the case-management mon- youths agree to participate in extrajudicial itoring process at four probation offices in the three Youth Justice Services Program 309 regions we visited, but one probation office did not complete all required risk/needs assess- maintain any documentation for the three years • ments, case-management plans and casewe chose to examine, and another office could not management reintegration plans on a timely locate any of the reviews for the 2010/11 fiscal year. basis; In general, we found a variety of practices being ensure that case-management plans have used across offices, as follows: • specific goals and recommended programs Ministry policy requires at least five active • and services to assist youth in addressing files and two closed files to be reviewed per all high-risk areas identified and any court- probation officer annually. However, over ordered conditions; the past three years, we found that generally clearly document in the case files whether or only three case-file reviews per probation • not youths have complied with court-ordered office were being done, and in some cases as conditions and community-service require- few as one. This represents about 12% of the ments and, if they have not, what efforts were average probation officer caseload of 26. We made by the probation officer to rectify this; could not confirm that every probation officer develop guidelines or policies about what was reviewed because probation offices did • types of extrajudicial sanctions are appropri- not maintain lists of probation officers who ate to use and when; and worked in that office for the years we tested. ensure that the required case-file reviews are There was inconsistency among probation • • being done consistently across all probation offices on how files were selected for review.

offices and determine whether there are any 3.13 For example, in one region probation officers systemic issues warranting additional guid- were allowed to select most of their files them-

ance or training. Section selves for managers to review. In the other VFM two regions, probation managers selected files • MINISTRY RESPONSE 3 randomly from the case register. • Each case file is reviewed against more than The Ministry is pleased that the Auditor

100 standards. The probation manager judges acknowledged the knowledge and experience of Chapter whether each standard has been met, partially probation officers, and the actions already taken met or not met. For a sample of case-file to address the recommendations. The Ministry reviews conducted in the 2011/12 fiscal year launched a Probation Strategy in May 2012 that that we examined, we noted that almost two- will strengthen probation services, account- thirds met at least 80% of applicable standards. ability and compliance. In addition, the Ministry • None of the regions we visited had analyzed will review its processes and implement new the results of case-file reviews to identify which procedures where necessary to: standards probation officers consistently had • ensure that files are fully documented and the most difficulty complying with, so that this clearly demonstrate that appropriate action information could be used in staff training. was taken on a timely basis; • support quality assurance and effective use RECOMMENDATION 3 of Risk/Need Assessments (RNAs) by probation officers; and To help ensure that case-management efforts ensure that all required risk/need assess- result in youths obtaining the services and pro- • ments, case-management plans and grams needed for rehabilitation, the Ministry of case-management reintegration plans are Children and Youth Services should: completed on a timely basis. 310 2012 Annual Report of the Office of the Auditor General of Ontario

programs and services are grouped into six major In order to track the number of outstanding service activities, and each offers categories of pro- RNAs, the Ministry is developing a database grams and services. For example, community-based report that identifies “future due” and “overdue” programs/services include attendance centres and RNAs so that timely action can be taken by pro- managing youth with community-service orders. bation managers. Reintegration and rehabilitation includes employ- The Ministry agrees that it is important ment- and life-skills training, anger management for the case-management plans to address all counselling, substance abuse counselling and hous- high-risk areas and will review the practices ing support. The Ministry contracts with almost and training that support probation officers to 200 transfer-payment agencies to offer almost increase their capacity to engage youth to set 500 services and programs across the province. and achieve goals on their own. The type of programming and services youths will The Ministry will review its processes and receive is determined by their risk/needs assess- implement new procedures to ensure that files ment, and sometimes is listed on their court order. are fully documented and clearly demonstrate Transfer-payment agencies are funded on a that appropriate actions were taken on a timely historical basis. One-time funding adjustments are basis, particularly in respect of court-ordered permitted for non-recurring expenses. Permanent conditions. adjustments are permitted when there is a change The Ministry will review and revise, if in the services or programs provided. necessary, its existing policies for extrajudicial

3.13 sanctions to ensure they support selection of

appropriate extrajudicial sanctions. Effectiveness of Agency Programs and

Section The Ministry recognizes the importance of

Services completing case-file reviews consistently. In VFM The Ministry aims to deliver evidence-based pro- •

October 2011, the Ministry released an updated 3 grams to youths in conflict with the law in order to Case Management Compliance Review Tool and rehabilitate them. To this end, it collects academic Managers’ Guide, which articulates expectations

Chapter research on the most effective treatment models and provides guidance on conducting case-file and approaches, which it calls the “What Works reviews. In 2013/14, the Ministry will initiate Literature.” However, there was no evidence to show a review of the case-file review process and that the programs and services available to youths will specifically look at accountability and con- actually were selected based on the best practices sistency in case-file reviews, as recommended outlined in the literature. The Ministry’s corpor- by the Auditor. The Ministry will also assess ate office told us that it assumes transfer-payment whether there are any systemic issues that war- agencies use the What Works Literature to develop rant additional guidance or training. evidence-based programs and services. But the Min- istry’s regional staff to whom we spoke, who were responsible for contracting with service providers PROGRAMS AND SERVICES and evaluating the programs and services offered, were generally not aware of the What Works Litera- The Ministry contracts with transfer-payment agen- ture. Consequently, no one ascertains whether any cies to deliver a variety of programs and services, of the programs and services developed by transfer- primarily directed to youths who come into conflict payment agencies align with the best treatment with the law, although they also deliver some approaches according to the research. preventive programming. As seen in Figure 5, these Youth Justice Services Program 311

Figure 5: Funding and Service Activity, 2010/11 Source of data: Ministry of Children and Youth Services

Youth Served/ Funding Admissions Programs/Services Service Activity ($ million) % # % # % Prevention 2.9 1 3,637 8 51 10 Diversion 6.5 2 10,386 21 94 19 Community-based programs/services 24.8 9 9,011 18 112 23 Reintegration/rehabilitation 12.5 4 9,395 19 102 21 Probation 50.1 18 8,673 18 66a 13 Custody/detention 191.2 66 7,927b 16 70c 14 Total 288.0 100 49,029d 100 495 100

a. Includes 64 Ministry-operated and two agency-operated probation offices. b. Includes 6,819 admitted to detention and 1,108 admitted to custody. c. Includes seven Ministry-operated and 63 agency-operated facilities. d. Some youths have been counted more than once if they received services from more than one service provider.

The Ministry does not have a master list of all of as were “education advocacy,” “school attendance” the youth justice programs and services offered by and “school support.” We noted that one agency the approximately 200 transfer-payment agencies just grouped all services under the umbrella term

across the province. Services are detailed in indi- “youth support program,” providing little informa- 3.13 vidual contracts with agencies, but these details are tion on the actual services it offered. It was difficult

not catalogued in regional inventories, either. As a to determine from the contracts specifically what Section result, probation officers, who are responsible for rehabilitation service and/or program was actually VFM

• connecting youths with programs and services that being offered by each agency. 3 best meet their assessed needs and risk level, must rely on their own experience and on informal dis- RECOMMENDATION 4 cussions with other probation officers and contacts Chapter To ensure that effective programs and services in the community to determine which programs are offered to youths no matter where they live and services are available in their region. Although in Ontario, the Ministry of Children and Youth this type of informal interaction among colleagues Services should: and contacts is undoubtedly valuable, without good ascertain that the services and programs information on the specific programs and services • contracted for actually align with best- available in each region, there is a risk of inequities practice youth rehabilitation research; and across regions and a risk that youths might not be establish and maintain a master list of connected with the services and programs that best • regional programs and services that uses meet their needs. consistent terminology and make this infor- Comparing and evaluating services and pro- mation available to all probation officers. gramming was also complicated by inconsistent terminology in the Ministry’s contracts with MINISTRY RESPONSE transfer-payment agencies. Some used different wording for what seemed to be the same service. For The Ministry agrees that effective and evidence- example, “job readiness,” “job preparedness” and informed programs and services must be offered “employment search” were used interchangeably, throughout Ontario. The Ministry’s service 312 2012 Annual Report of the Office of the Auditor General of Ontario

the agencies’ target for number of youths to be continuum, which provides a range of commun- served. We also found agencies with similar ity and custodial programs and services across service-level targets for the same service the province, was guided by the “What Works” being funded at different levels. For example, literature to promote evidence-based program- in 2011/12, the Ministry approved a budget ming responsive to the needs of youths and to of $9,000 for one agency and $32,000 for contribute to community safety. This approach another to each manage community-service aligns with the Youth Criminal Justice Act focus orders for 25 youths. In another example, on addressing the causes of offending behaviour the Ministry approved a budget of $107,000 and promoting rehabilitation and reintegration. for one agency and $165,000 for another to A framework for program evaluation is each manage 300 youths serving extrajudicial under development to confirm the Ministry’s sanctions. As a result, we found significant expectations for services and determine variations in actual program costs per youth, whether the various programs adhere to the as shown in Figure 6. intended design elements. An agency’s performance has little impact on The Ministry has completed an inventory of • the funding it will receive the following year. programs for its directly operated youth centres For example, in the 2010/11 fiscal year more and is currently undertaking an inventory of than half of the programs and services pro- programs in open and secure transfer-payment vided in the community by agencies did not facilities. An inventory of programs and ser- meet their targets for the number of youths

3.13 vices offered in community agencies will be served, but almost 70% of these received developed over the coming year and will be the same funding amount or more for the

Section made available for use by all service providers, following year. The Ministry told us that the including probation officers. VFM agencies have no control over the number of •

3 youths served, and given the fact that agencies have mostly fixed costs, reducing funding

Chapter Funding and Monitoring of Programs and would likely mean staff layoffs. Services Offered in the Community • Budgets are adjusted throughout the fiscal year, some as late as March 31, resulting in The Ministry enters into annual service contracts little variance between the approved budgeted with each of its transfer-payment agencies for ser- amounts and actual expenditures at year-end. vices offered to the Youth Justice Services program. Most funding adjustments happen in the Among other things, the contract outlines a descrip- fourth quarter. During the 2010/11 fiscal year, tion of the services to be provided, the amount of budgets were adjusted during the year for annual funding, and service targets to be achieved. We identified the following with respect to the Figure 6: Actual Costs per Youth for a Sample of Ministry’s funding and monitoring of third-party Community-based Programs and Services, agencies: 2011/12 ($) We found little or no correlation between • Source of data: Ministry of Children and Youth Services service-level targets and the amount of annual funding the Ministry approved. About 60% Program/Service Lowest Highest Median of services and programs that had a funding Attendance centre 989 13,037 5,208 increase of at least 10% in the 2011/12 fiscal Anger management 256 9,704 1,437 year did not have a corresponding increase in Substance abuse 321 8,114 1,264 Youth Justice Services Program 313

half of all contracts, and 85% of the funding Program supervisory staff at each of the three adjustments were increases. We selected a regional offices we visited told us that they do sample of adjustments in 2010/11 from each not routinely ask for evidence of evaluations of the three regions we visited and noted that done. One region informed us that only one 50% of one-time adjustments had little sup- agency submitted an annual quality assur- porting documentation from the agency to ance report that detailed the evaluation it substantiate the need for additional funding. undertook, but even in this case, the agency • The Transfer Payment Accountability directive had not performed the detailed analysis it had requires ministries to establish risk criteria committed to in its contract. to assess the ability of service providers to meet service-delivery objectives. To this end, RECOMMENDATION 5 the Ministry has developed a risk-assessment To ensure that funding provided to transfer- questionnaire to be completed by the service payment agencies is commensurate with the providers. The Ministry uses the agencies’ value of services provided, the Ministry of Chil- self-assessments to determine the overall risk dren and Youth Services should: level. Of the agencies that completed assess- ensure that approved funding to agencies is ments in the 2011/12 fiscal year, other than • appropriate for the expected level of service, those providing custodial services, almost all based on levels of service achieved in the last assessed themselves as low risk, yet most did few years; not meet their service-level targets for more

compare and analyze agency costs of similar 3.13 than 50% of services and programs they were • programs across the province, and investi- contracted to provide.

gate significant variances that seem unjusti- Section A report issued by the U.S. Center for Juvenile • fied; and VFM Justice Reform in December 2010 states that •

ensure that requests for additional funding 3 rehabilitative programs, if implemented well, • are adequately supported. can reduce recidivism substantially. However,

there was no documented evidence that the Chapter MINISTRY RESPONSE Ministry or the service providers assessed the effectiveness or quality of the programs and The Ministry agrees that its transfer-payment services offered. The Ministry performs only partners in service delivery should demonstrate limited analysis on service providers. On an value for services provided. The Ministry will annual basis, the Ministry compares service- conduct a review of Youth Justice Services level targets with the actual number of youths targets, youths served and related costs over a served. We were told that regional program three-year period to assess whether funding is staff conduct informal evaluations with input appropriate and will make adjustments where from probation officers. Furthermore, aside warranted. In addition, the Ministry will conduct from visits to custody/detention facilities, a jurisdictional scan of comparable Youth Justice there was no evidence of site visits by Min- Services providers to compare the reasonable- istry staff to community-based services and ness of the Ministry’s transfer-payment costs. programs. In accordance with its contract, the Starting with the 2013/14 budget year, the service provider outlines how it will evaluate Ministry will review agencies’ budget packages each program it offers. We inquired whether and expenditure reports to investigate any the Ministry followed up with agencies to significant variances that seem unjustified, and review the nature and results of evaluation. make adjustments where appropriate. 314 2012 Annual Report of the Office of the Auditor General of Ontario

inspection process. As a result, it was difficult The Ministry will remind all staff of the to determine whether the decision to issue or obligation to provide clear documentation that review a licence was appropriate. All inspec- supports approvals for changes in funding. As tion files we reviewed indicated numerous well, the Ministry will modify the reconciliation items of non-compliance with both the Act process to require confirmation of sufficient and the manual, but no facility operator appropriate documentation for changes in had its licence suspended or revoked. One funding. licence was issued provisionally in December 2011 due to violations of the Fire Code. The Ministry indicated that it prefers to work with Ministry Oversight of Custody/Detention facilities to make the necessary improvements. Facilities • We found inconsistency in the quality of inspection reports communicated to facility Regional offices inspect all custody/detention facili- operators. Some reports quoted verbatim the ties annually to ensure they comply with the Child sections of the Act or manual that were not and Family Services Act (Act) and the Ministry’s met, whereas other reports described with Youth Justice Manual prior to renewing their more precision which aspect of a require- licence to operate. The Act sets out the minimum ment was not being met and often noted the acceptable standards for the provision of residential particular staff or resident records where the care to children, and the Ministry’s justice manual violations were identified. While two regions

3.13 includes additional ministry expectations as well made more specific recommendations for as requirements under the federal Youth Criminal corrective action, the other region’s recom-

Section Justice Act relating specifically to youths in custody/ mendations were too general to be useful; for detention facilities. We reviewed the licensing VFM example, it typically recommended just that •

process and noted that annual inspections were 3 the facility implement policies and ensure that being done. However, we noted the following with all staff were familiar with them. respect to the annual inspection process:

Chapter We found no evidence that non-compliance For each facility’s annual inspection, Ministry • • items were being addressed on a timely basis. inspectors are required to complete approxi- One regional office we visited confirmed that mately 200 pages of a checklist. They com- it doesn’t follow up on corrective action until plete another 200 pages of the checklist every the next licence-review process, while the four years as part of a more in-depth review of other two said they followed up soon after. facilities’ policies and procedures. Although However, we generally found no evidence thorough, the checklist is often repetitive on file to show that the Ministry revisited and has not been written in a way that helps the facilities or obtained verification that inspectors focus on systemic issues to gain an the agencies running them implemented its overall understanding of how well the facility recommendations. The regions considered the operates. We found that the checklist places non-compliance to have been addressed by little emphasis on the quality of the program- the agency if the agency provided a response ming and services being offered to youths to or a work plan. For a sample of facilities we reduce recidivism, and places more emphasis selected for licensing-file review, one-third on health and safety standards. of facilities had had five or more of the same There is no distinction in the severity of • violations in the previous year. non-compliance issues identified during the Youth Justice Services Program 315

The method of data collection and recording • consider requiring that people working in makes it nearly impossible for the Ministry • youth custody/detention facilities undergo a to analyze province-wide inspection data to Canadian Police Information Centre check, identify and address trends of non-compliance including vulnerable-sector screening, every or even to monitor a facility’s performance five years and not only at the time of initial from one year to the next. For instance, two hiring. regions entered results into the system for each section of the checklist reviewed and MINISTRY RESPONSE for each sample tested, while the third region aggregated responses from the samples tested In 2011, the Ministry completed a review of the and entered one set of results into the system licensing checklist and reduced it by more than for each section of the checklist. 60%, from approximately 1,500 lines to about • People working in custody/detention facilities 500. The revised checklist is to be rolled out in are required to get Canadian Police Informa- fall 2012. In addition to the licensing-review tion Centre (CPIC) clearance, including process, the Ministry will be using other means vulnerable-sector screening, when they are to review the quality of services. For example, first hired. There is no further requirement as part of its Data Strategy, in 2012/13 the for them to have another CPIC check. We Ministry will initiate a review of all the Youth noted that almost 40% of employees working Justice Services description schedules, which in agency-operated facilities in two of the define service expectations and form part of the 3.13 regions we visited were hired at least five transfer-payment agency’s contract. Further- years ago, and more than 20% were hired more, a framework for program evaluation is Section at least 10 years ago. It is not uncommon for under development to determine whether the VFM public-sector organizations requiring CPIC various programs adhere to the intended design •

3 checks on hiring to require updates every five elements that reflect the “What Works” litera- years or so. ture for evidence-based programming that is

responsive to the needs of youth. Chapter RECOMMENDATION 6 As a result of the review of the licensing checklist, the Ministry is now requiring full To ensure that the annual facility inspection and use of the automated licensing checklist for its licensing process results in a safe and secure Youth Justice licensing activities. In addition, living environment with effective services and the Ministry is building a software application programs for youth residents, the Ministry of that will allow division-wide monitoring and Children and Youth Services should: tracking of trends and identification of where revise the inspection checklist to eliminate • corrective action is warranted. As well, in duplication and place more emphasis on the April 2012, the Youth Justice Services Division quality of programming and services being approved a Data Management Framework that offered; will result in more consistency in all data activ- work toward obtaining more consistency • ities for the Division. in data collection and recording and in In September 2012, the Ministry provided reporting inspection findings; training to managers and licensing staff to where significant compliance issues are • ensure a clear understanding of legislative and noted, ensure that appropriate and timely ministry requirements in regard to the licensing follow-up is done; and reviews. These requirements include response to 316 2012 Annual Report of the Office of the Auditor General of Ontario

non-compliance and the legislative support for noted that two other Canadian provinces track the Ministry to provide a period of time for the youth recidivism rates, but neither has published agency to meet all requirements. The software results for comparison. application being developed will allow the Min- The Ministry defines recidivism as a return to istry to monitor the timeliness and effectiveness provincial youth justice supervision or adult cor- of follow-up work done. rectional supervision, within two years, on a new The Ministry will review best practices conviction that occurs following the completion of and consider requiring more frequent Can- a youth community disposition (that is, a proba- adian Police Information Centre checks and tion order only) or following the completion of vulnerable-sector screening for people working a youth custody order of six months or more. In with youths in custody/detention facilities. simpler terms, only youths who have been found Currently, the Ministry’s Code of Conduct for guilty through the court process and have served Youth Justice Services requires staff to report a sentence longer than six months are tracked for any changes to their original security checks. recidivism. Although we agree that recidivism is an important performance measure, and the definition is similar to that used for adults by the Ministry of PERFORMANCE MEASUREMENT AND Community Safety and Correctional Services, the REPORTING Ministry does not actually track recidivism for more

3.13 The Ministry has stated that it intends to improve than 80% of the youths who have come into contact

outcomes for youths through programs and services with the Youth Justice Services program. Groups

Section that are responsive to the risks, needs and strengths excluded from the calculation for the 2010/11 fiscal

of youths. Its Youth Justice Services Program has a year are all youths held in detention prior to trial, VFM

•

number of important objectives: all youths diverted from court through extrajudicial 3 • reducing reoffending; sanctions, more than 90% of youths sentenced to preventing youth crime; custody and approximately two-thirds of youths

Chapter • • increasing community safety; and sentenced to community supervision. The Ministry • holding youths accountable and creating informed us that it excludes these groups because opportunities for at-risk youths through studies indicate the Ministry cannot influence a rehabilitative programming. person’s behaviour in less than six months. However, the Ministry has only one perform- We believe that maintaining data on the ance measure for the program—the number of reoffend rates for the 80% of youths not being youths who reoffend as a percentage of all youths tracked would still prove useful. For example, based tracked, also referred to as recidivism. The target on the extrajudicial files we sampled in our case-file recidivism rate changes every year, and is equal to review, we noted that one-quarter of youths serv- the actual rate achieved in the prior year. Recid- ing extrajudicial sanctions had been in a diversion ivism rates reported by the Ministry have been program previously. Some indication of the effect- relatively stable over the past five years for youths iveness of the diversion program can be obtained with community-based sentences and have gotten by analyzing the number of youths in the program slightly better for youths with custody sentences. who reoffended and received either extrajudicial For the 2010/11 fiscal year, the recidivism rate was sanctions again or a court-ordered sentence. 35% for youths with community-based sentences, Our other observations regarding performance and 59% for youths with custody sentences. We measurement and reporting included the following: Youth Justice Services Program 317

• The Ministry does not track the programs or we were informed that the Ministry hopes to imple- services attended by each youth in its care. ment both types of exit surveys on a province-wide As a result, it cannot correlate youths who basis next year. do not reoffend to the specific programs or services they received, which might indicate RECOMMENDATION 7 the effectiveness of the programs and services To enable it to evaluate and report on the effect- over time. This might also provide some indi- iveness of the Youth Justice Services program, cation of the effectiveness of the 200 transfer- the Ministry of Children and Youth Services payment agencies that provide such services. should expand the measure for recidivism so The Ministry does not monitor the rate of com- • that it captures most of the youths in the pro- pliance with court orders, which may be used gram to better enable it to assess which services, in assessing whether the program is meeting programs and delivery agencies seem to be the its objective of holding youths accountable. most successful over time. At the time of our audit, the Ministry was in the process of developing additional outcome-based MINISTRY RESPONSE performance measures through an evaluation of current data to identify what additional data would The Ministry agrees that not all youths are cap- be needed to measure more than just recidivism. tured in its measure for recidivism. The Ministry In addition, in 2010, the Ministry began a pilot will work with partners, including the research program of exit surveys for youths upon their community, to review options for reporting on 3.13 release from custody/detention facilities in two of expanded measures for recidivism, since cur- the regions we visited. The surveys request feed- rently there is no consistent, national definition Section back from youths on programming, safety, basic of youth justice recidivism. VFM needs such as food and clothing, case management As part of its Data Strategy, the Ministry has •

3 and reintegration planning, and overall experi- developed additional Youth Justice outcome ence at the facility. In the region we didn’t visit, measures that are broader than recidivism and

the Ministry informed us that it also began a pilot provide a fuller understanding of the impact Chapter program of exit interviews with parents and youths that programs and services have on all youths. upon completion of their probation period to collect Tools to track and report on these outcomes are feedback about their probation service experience. under development. In discussing this good initiative with the Ministry, Chapter 4 318 mittee’s recommendations in the applicable section applicablesection recommendations inthe mittee’s was Com- held,we ofthe have includedasummary on PublicAccounts (Committee). Where ahearing Standing issuedby Committee the held andreports management. by timeasreported recommendations sincethat hasbeentaken to address ofactionthat our status of our2010 the anddescribes AnnualReport value-for-money oninChapter auditsreported recommendations. actions taken by respect management to our with related responses, we follow of status uponthe we rec years after publish the inChapter Tworeports 3ofourAnnualReport. wewhich includewhenwe audit publishthese response recommendation, to a written each broader publicsector toizations inthe provide Crown agenciesask ministries, ofthe andorgan- tions inourvalue-for-money and auditreports toIt isourpractice make specific recommenda- Chapter 4 Chapter For a number of these audits, hearings wereFor audits,hearings also anumberofthese Chapter onthe 4provides somebackground Value-for-money Audits on2010Follow-up ommendations and

3 that the Committee raised during the hearing and in andin hearing the Committee during raised the that being taken by audited issues entitiesto address the actionis isto helpensure that additional reporting chapter.of this Ourobjective inproviding this any Legislature. Chapter to subsequent the report when future auditsshouldbeconsidered. future auditsandmay impact ourassessmentof onin will bemore fully andreported examined effectively. Thecorrectiveactionstaken orplanned havecorrective actionsdescribed beenimplemented we cannot provide the ahighlevel that ofassurance work. this Thisisnotwith anaudit,andaccordingly, auditors organization’s alsoassisted internal the Inafew documentation. cases, selected supporting management andreviewies anddiscussionswith of 40%ofthem. forreported more than our recom 2010, isbeingmadetoward progress implementing recommendations weclose to 90%ofthe madein activitiesmore fully. Committee’s the describes Our follow-up work ofinquir consists primarily We for are pleasedto that beableto report mendations, wit h substantial progress progress h substantial

6 6 machines (at the timeofour2010 (atthe machines audit,itdirectly table games andslot both and five casinoswith ing 14 have that only slot atracetracks machines and meeting andconvention space. ashotels, limits andamenitiessuch entertainment, more gaming offer options, higher wageringwhich one smallercasinoandfour large casinos,” “resort sector day-to-day operators to run at operations facilities andfive private- casinos).Italsocontracts operated 22gaming venues, including17 each relationship to responsibilities andanarm’s-length by two Crown agencies, have which different overseen isprimarily Casino gaming inOntario Background • • Chapter 4 Chapter OLG directly operates 19 ario Lottery and Gaming Corporation andGamingCorporation Lottery ario

other: 2009/10 fiscal year). (27 racetracks facilitiesathorse slot machine 24 casinosand Ontario’s sector partners, private- directlyand operates, orwith either “operator,”(OLG), asthe builds,manages gaming inspect gaming facilities,andto enforce amandate towith regulate, licenseand “regulator,” (Commission)isthe Ontario The Ont The AlcoholandGamingCommissionof 4.01 Section

casinos and slot machine facilities in the facilitiesinthe casinos andslot machine

legislation. Follow-up to VFMSection3.01, to Follow-up Regulation Casino Gaming Alcohol andGamingCommissionofOntario

gaming venues, includ

slot - several Commission’s oversight areas where the America. oversight inNorth mechanisms framework stronger oneofthe for casinosoffers regulatory Ontario’s alsoindicated that experts, andadvicefrom external jurisdictions, other wegaming test labthat into hired. Our research by anindependentaccreditedconclusion confirmed ofgaming fairoperation equipment,ensure a the gaming enforcement procedures were adequate to The Commission’s gaming equipment test laband and procedures inplaceto expectations. meet these Commissionhadadequatethe systems, policies revenues ($3.4 generatedoperations $3.3 more than run games are the andthat and properly supervised, casinoemployeeswant that assurance are honest orcraps, asblackjack who play games, such table actually regulated pay minimums;those outthe slot facilitycustomersto slot expect machines fairlyfacilities to berun andhonestly. Casinoand erated by slot machines. 2009/10). 85%ofallrevenues More than are gen- of $700 costs ($2.5 2009/10) $2.6 andincurred However, we alsonoted inour2010 audit In our2010, we AnnualReport concludedthat In the 2011/12In the fiscal year, OLG casino gaming The general casinosandslot publicexpects

fairly.

million for the provincemillion for the ($900

billion in2009/10), for anet profit

billion at the timeofourauditin billion atthe 2010 Annual Report Annual 2010

billion in operating billion inoperating

billion in

million in 319

Chapter 4 • Follow-up Section 4.01 320 2012 Annual Report of the Office of the Auditor General of Ontario

procedures and gaming transparency could be We made a number of recommendations for enhanced, as follows: improvement and received commitments from the • Slot machine patrons are very interested Commission that it would take action to address in the actual payout ratio and in whether our concerns. these payout percentages vary depending on the machine type and denomination (for example, a one-dollar or a penny machine). Some U.S. jurisdictions such as Nevada pro- Status of Actions Taken on vide this information, but Ontario did not. Recommendations • We noted that patrons would find it difficult to locate information on the maximum At the time of our follow-up, the Commission had prize payout on certain slot machines—an taken action on several of the recommendations important disclosure should the machine we made in 2010; however, others are taking more malfunction and erroneously award a multi- time to implement. The Commission is in the pro- million-dollar jackpot, as occurred twice in cess of implementing a new approach to regulating the two years prior to our audit. In addition, gaming that aims to target higher-risk areas and the Commission did not require casinos to set standards while increasing operational flex- to post the odds of winning a jackpot on ibility for gaming operators by allowing them to slot machines. develop their own control activities and business In the 2008/09 fiscal year, commission • processes. The Commission has completed a ser- inspectors at three of four gaming facilities ies of risk assessments related to casino gaming could not meet their goal of inspecting every facilities and expects to begin introducing new slot machine once a year. In addition, the standards and requirements by 2013. As a result of Commission’s gaming audit and compli- the Commission’s development of a new regulatory ance inspectors were behind schedule in approach, which is an ongoing and evolving pro- verifying that gaming facilities complied with cess, a few of our recommendations have yet to be approval requirements and their own internal substantially addressed. control manuals. The current status of action taken on each of our In determining registration eligibility for sup- 4.01 • recommendations is as follows. pliers and gaming employees, the Commission had no policy for dealing with conflicts of Section

interest involving related employees work- CONTROLS OVER GAMES ing in the same casino. It relied, instead, on Recommendation 1 Follow-up casino and slot facility operators to deal with • To provide more useful information to slot 4 these situations. machine patrons and better communicate its role In a related issue, estimates in the 2009/10 in ensuring the integrity of gaming in Ontario,

Chapter fiscal year were that Ontario residents spent about the Alcohol and Gaming Commission of Ontario $400 million annually on foreign-based Internet (Commission) should: gaming websites. Foreign gaming operators do not make public the minimum 85% slot machine share their revenues with the province, and the • payout percentage, a range of actual payouts, Commission has no mandate to regulate Internet and the Commission’s role in overseeing this, gaming. To help address this, British Columbia and similar to the public disclosures made in Nevada Quebec now offer Internet gaming, and, at the time and New Jersey; and of our 2010 audit, OLG indicated that it planned to introduce Internet gaming in 2012. Casino Gaming Regulation 321

• review its standards and approval processes for inspection. We were advised that the Commission new and existing slot machines to ensure that does not have a policy to physically inspect 100% of the maximum prize payouts and odds of win- the devices annually. Instead, using the risk-based ning are clearly disclosed or readily obtainable approach, it carries out targeted inspections of on each machine. machines identified as high priority. These include To enhance its already strong controls over elec- new machines, those that have been converted or tronic gaming equipment, the Commission should: changed in any way that might affect the integ- • assess the reasons for its Electronic Gaming rity of the game, and those being removed from Branch not meeting its goal of inspecting all service. In addition, the Commission conducts slot machines annually and, using a risk-based risk-based random inspections on installed gaming approach, assess the implications of this but equipment. We were informed that the Commis- also the need for an annual 100% inspection sion is up to date on its risk-based inspections of practice; and gaming equipment. • regularly audit its inventory controls over The Commission has also updated its control security seals intended to prevent tampering procedures over security seals to include new pro- with electronic gaming equipment to ensure cedures at every gaming facility. At the end of each that proper accounting is in place and that month, an electronic gaming inspector physically unaccounted-for seals are immediately detected counts all unused seals, and a regional manager and investigated. reviews a report that identifies all currently applied In addition, to ensure consideration of key risk fac- seals and highlights any suspicious numbers for tors relating to table games, the Commission should further investigation. In addition, the Commis- reassess its approval requirements for surveillance sion updated the Slot History System database in plans, including minimum surveillance staff levels at July 2011 to keep a historical record of all seals gaming facilities. To ensure that gaming operators’ entered into the system so as to be able to reconcile staff who work in key risk areas, such as table game them. We visited one slot facility and found that dealers and surveillance staff, have sufficient training, new inventory controls over seals were in place as the Commission should consider whether it should required. require casino staff to meet predefined standards of The Commission has initiated a number of pilot training and competency. projects in the development of the standards-based 4.01

approach to gaming regulation, including a regula- Status tory review of surveillance plan requirements. Section The Commission advised us at the time of our While the standards and requirements being follow-up that it was still reviewing current policies developed do not specifically address surveillance on casino gaming facilities. It also said that the new Follow-up staffing levels or casino staff training and compe- • standards and requirements that casino operators 4 tency, the Commission did change existing require- must meet would begin being implemented in ments so that performance reporting of employees 2013, although a date for establishing a standard

now includes their adherence to required proced- Chapter for communicating maximum prize payouts and ures. We were also informed that risks related to odds of winning to slot players had not been set. the competency of table game dealers have been In the meantime, information on the minimum identified, with further relevant standards and 85% expected payout percentage is now currently requirements to be developed as part of the Com- available to the public on the Commission and mission’s review of policies over gaming facilities OLG websites. taking place at the time of our follow-up. The Commission also developed risk-evaluation criteria for identifying slot machines that require 322 2012 Annual Report of the Office of the Auditor General of Ontario

GAMING AUDIT AND COMPLIANCE Commission advised us it would take new operators into account in developing its new risk-based audit Recommendation 2 approach, with implementation planned for 2013. Given that Ontario’s gaming industry is mature and In addition, the Commission has been there is a high level of gaming facility compliance with developing an integrated audit model in conjunc- its regulatory requirements, the Alcohol and Gaming tion with OLG’s internal auditors to enhance Commission of Ontario (Commission) should develop audit efficiency and effectiveness and to minimize comprehensive control-risk frameworks that would overlap of the overall audit function between the allow gaming facilities to be assessed individually for two agencies. We were informed that, because of risk. Such a framework would allow the Commission shifting government and OLG priorities, the time to cost-effectively focus more of its regulatory over- frame for completion of this integrated model had sight on higher-risk facilities and less on lower-risk not been finalized. ones and yet still achieve a prudent level of oversight. In developing these frameworks, the Commission should also assess the reasons for and the potential GAMING SUPPLIER AND impact of its audit and compliance staff not achiev- EMPLOYEE REGISTRATION ing the targeted number of audits and inspections of Recommendation 3 gaming facilities. To ensure that registration and renewal processes Status meet adequate standards for timely completion and The Commission has implemented a three-phase consistent quality, the Alcohol and Gaming Commis- risk-based approach for conducting audits of sion of Ontario should: gaming facilities. • complete its risk-based assessment for stream- First, it developed an industry-wide risk assess- lining procedures, and establish benchmarks ment in November 2009 for all gaming sites, and management tracking reports for registra- leading to the creation of a risk profile for each tion and renewal processing times; and individual site. Based on the risk profiles, more fre- • establish a policy defining what could constitute quent audit cycles were established for higher-risk potential conflict-of-interest situations involving sites. We were informed that the risk profiles are gaming assistants and what situations could also updated on a quarterly basis using the latest prove problematic. 4.01

available information. Status Second, various operations within a specific Section The Commission implemented a new three-stage gaming site with a greater risk of non-compliance risk-based process in September 2011 to streamline were identified to allow the Commission to focus the registration and renewals process for gaming Follow-up its resources not only on sites determined to be • suppliers and employees. The first stage establishes 4 higher risk, but more specifically on the internal basic eligibility and determines whether further operations, such as surveillance, table games, and investigation is required using a scorecard to

Chapter cashiering, of each site deemed to be higher risk. assess risk. If an applicant’s level of risk is scored The Commission informed us that in June 2012, high, the Investigations and Enforcement Branch it implemented the third phase of the updated conducts a more rigorous inquiry, collecting more audit approach, which involves determining the in-depth information through interviews and/or a key controls to be selected for testing. In light of full background review. The final stage involves a the government’s 2012 Budget, which announced decision about whether to issue the registration or the government’s intention to explore the further renewal. We were informed that the Commission privatization of OLG casino facility operations, the will undertake a comprehensive assessment of the Casino Gaming Regulation 323 new streamlined process after it has been in place sultations and a scan of best practices across various for one year. Canadian and foreign jurisdictions, Responsible The Commission also developed the Casino Gaming Standards are currently being developed Gaming Performance Measures Dashboard as a that will also address self-exclusion programs. For performance management tracking and bench- example, the standards will require gaming oper- marking tool. The Dashboard is used to report on ators to offer a voluntary self-exclusion program, the overall performance of the Commission against terminate the OLG accounts of self-excluded per- key indicators, including registration and renewal sons, remove them from mailing lists, and withhold processing times. Benchmarks for the performance all incentives and promotions for OLG products and indicators have been established, and when targets services during the period of self-exclusion. The are not met, management investigates the causes Commission expects the new standards to begin and makes operational adjustments as required. In being implemented in 2013, although a date for a addition, monthly reports using information from standard on self-exclusion programs has not been the Dashboard—such as the number of licences and set. In the meantime, we were informed that all sites registrations issued as well as average turnaround currently operate a self-exclusion program that is times for processing—are prepared for the Com- largely consistent with the standards and require- mission’s Board of Directors and also distributed to ments that will ultimately be put in place. senior management. We were advised that along with the develop- The Commission has yet to establish any policies ment of standards, self-exclusion programs will be covering potential conflicts of interest involving examined on an ongoing basis using appropriate gaming assistants. We were advised that these regulatory assurance activities such as audits standards and requirements are being developed and inspections. as part of the Commission’s overall move toward a standards-based approach to regulation. PERFORMANCE REPORTING

SELF-EXCLUSION PROGRAM Recommendation 5 In order to provide the public, including gaming Recommendation 4 facilities’ patrons, with meaningful information on To ensure that gaming facilities adequately deal its regulatory activities, the Alcohol and Gaming 4.01 with patrons who may have a problem with or an Commission of Ontario should research other gam- addiction to gambling and those who participate in ing jurisdictions’ best practices in public reporting, Section a self-exclusion program, the Alcohol and Gaming and expand the information published in its annual Commission of Ontario should develop minimum report and website to ensure that it provides informa- Follow-up standards, policies, and procedures related to self- tion of use to gaming patrons and to the public with •

4 exclusion for use in Ontario’s gaming facilities. respect to its key regulatory activities and results, as It should also implement a process of periodic- well as performance information that demonstrates

ally reviewing gaming facilities’ compliance with the Ontario gaming industry’s competitiveness Chapter these requirements. and integrity.

Status Status The Commission advised us that it consulted with Although no research was undertaken by the Com- key stakeholders regarding various elements of self- mission on best practices in public reporting in exclusion and whether those elements should be other jurisdictions, the Commission has developed included in policies and programs approved by the a Casino Gaming Performance Measures Dashboard Board. We were informed that as a result of the con- that it now uses to report on its overall oversight of 324 2012 Annual Report of the Office of the Auditor General of Ontario

the industry. Benchmark data from the Dashboard assess best practices in regulation, technology and is used in the Commission’s annual report, which oversight and governance as they relate to Internet was expanded in the 2010/11 fiscal year to include gaming. The review identified jurisdictions where information such as the number and results of com- knowledge-sharing and collaboration should be pliance inspections, testing of electronic gaming explored because their established regulations align equipment and systems, and number and results of with the Commission’s own regulatory direction occurrences investigated by casino enforcement. for Internet gaming. The Commission also released We were informed that other key communication a request for proposals for consulting services to vehicles such as the Commission’s website will be develop a regulatory assurance model for Inter- reviewed to enhance the information that is avail- net gaming. The request closed in June 2011. A able to stakeholders. resulting contract was executed in September 2011, and work on the model began at that time. As of July 2012, the OLG was in the process of selecting a OTHER MATTER provider of Internet gaming as a result of its request Internet Gaming for proposals completed in February 2012. We were advised that since the Commission Recommendation 6 lacks the legal power and mandate to regulate Although the Alcohol and Gaming Commission of foreign operators who conduct Internet gaming in Ontario (Commission) does not have a mandate Ontario, it had not developed strategies or propos- to regulate Internet gaming, there are proactive als at the time of our follow-up to regulate and tax measures the Commission could take to protect the these operators, or charge them fees. interests of Ontarians in this area until such time as a We were informed that while the Commission decision is made as to whether Internet gaming should has formed no new formal alliances with other be regulated. Given the estimated nearly $400 million provinces to address Internet gaming, the Com- that Ontarians gamble each year with unregulated mission remains in regular contact with other Can- foreign Internet gaming operators that do not pay fees adian jurisdictions and regulatory bodies through or taxes to Ontario, and the recent decision that the the Regulatory Review Committee and other Ontario Lottery and Gaming Corporation will offer organizations such as the Canadian Gaming Regu- Internet gaming in 2012, the Commission should: lators Association and the North American Gaming 4.01 conduct research into regulatory, technological, • Regulators Association. The Commission also and oversight best practices used in other juris- advised us that it is in the process of entering into Section dictions over Internet gaming available in their memorandums of understanding with other inter- respective jurisdictions; nationally based gaming regulators, including those develop strategies for possible action that can Follow-up • involved in Internet gaming, to share information • effectively regulate and tax or charge fees on for- 4 on applicants and on regulatory processes and eign operators doing Internet gaming business standards. For example, the Commission signed a in Ontario; and

Chapter memorandum of understanding in March 2012 with consider forming alliances with other provinces • the Gambling Commission in the United Kingdom, and the federal government to address Internet which regulates Internet gaming, to allow sharing gaming, as is currently being done by some of information and documentation for eligibility other international jurisdictions. assessments of applicants, licensees or registrants, Status and for providing joint inspections, investigations, In November 2011, the Commission conducted a and other compliance and regulatory assurance survey of regulatory bodies around the world to activities with respect to gaming. processes for the safeprocesses for andtimely the discharge of hadimplementedto they assesswhether effective homes. term-care accessto,and arrange homecare andlong- both The CCACsfrom hospital. alsoassesseligibilityfor, initiatives discharge to ofpatients facilitate the Care Access Centres (CCACs) have introduced Term andCommunity hospitals Care (Ministry), care. hospital requiring accessto bedsby hospital block others necessary Further, patientsstaying longer inhospital than post-discharge carecare. costshospital more than acquired asC.difficile.Aswell, infection such ofgetting risk ahospital- includingthe health, for patient risks stays carry hospital necessary becauselonger-thandetermination isimportant timetoappropriate discharge apatient.This care home. orinalong-term- asbathing) such care services athome(for example, andpersonal- either nursing patientsrequired20% ofthese post-discharge care, last five ofthe andabout years, patients ineach 1 discharged more hospitals than Ontario Background Background Chapter 4 Chapter In 2010, we conducted work hospitals atthree andLong- ofHealth Ministry As aresult, the medically physicians Hospital determine the 4.02 Section Follow-up to VFMSection3.02, to Follow-up Patients ofHospital Discharge Ministry ofHealthandLong-TermMinistry Care

million in someareas. For example, time we atthe reported couldbeimproved hospitals processes atallthree to improve patientflow. However, we notedthat systems other and were process of changing inthe discharge processes welling their insomeareas St. Thomas. Toronto; andSt. in Hospital Thomas–ElginGeneral Trillium Centre; in St. Hospital Health Michael’s Credit Valley Credit Valley site ofthe and Hospital Credit Valley inMississauga, now Hospital the we hospitals patients. Thethree visited were the were ready to bedischarged. care, wereand long-term available whenpatients includinghomecare community-based services, that was specifically accountable for ensuring Networks hospitals, CCACs orthe (LHINS),the ever, Integration LocalHealth no one,includingthe 16% How oftotal hospitals. patient-days inOntario previous fivethe time representedthe and at years patients were increased by hospitalized 75% over of care, orALC). Thetotal ALC numberofdays that care (alsoknown aswaiting for analternate level post-discharge beds becauseofdelays inarranging who were ready to bedischarged waited inhospital province-wide, 50,000 that, more than • We concluded that the hospitals wereWe hospitals the manag- concludedthat Among our other significant observations: significantobservations: Among ourother

meetings ondischarge planning activities team multidisciplinary quick Although 2010 Annual Report Annual 2010

patients - 325

Chapter 4 • Follow-up Section 4.02 326 2012 Annual Report of the Office of the Auditor General of Ontario

were held at all three hospitals, physicians because, usually, these facilities were older attended these meetings at only one hospi- or far from family. tal, and CCAC representatives attended most We made a number of recommendations for meetings at only one other hospital. improvement and received commitments from the • A ministry expert panel recommended that Ministry and the three hospitals that they would hospital physicians prepare a discharge sum- take action to address our concerns. mary to communicate patient information, such as follow-up appointments, pending test results and medication changes, to subsequent health-care providers. Although dis- Status of Actions Taken on charge summaries were generally prepared, Recommendations one hospital was consistently very late with them. At all three hospitals, a recommended In the spring and summer of 2012, the Ministry reconciliation of medications on admis- and the three hospitals we visited in 2010 provided sion and those on discharge was often not us with information on the current status of the prepared, increasing the risk of subsequent recommendations made in our 2010 Annual Report. medication errors. According to this information, significant progress At the hospitals we visited, less than 10% of • has been made in implementing more than half the total discharges to long-term-care, complex recommendations. While some progress has been continuing care, and rehabilitation facili- made on all of the others, the extent of progress ties occurred on weekends even though the varies among the hospitals. Fully addressing certain patient could have been discharged then, recommendations (by, for example, implementing because many of these facilities would not a standardized process for moving patients from accept the patient until the next business day. hospital to long-term-care homes) will take more Wait times in hospital for ALC patients • time. The current status of the actions taken by the varied significantly across the province. For Ministry and hospitals is summarized after each example, 90% of discharged ALC patients recommendation. in the North West LHIN were placed within 27 days of being designated ALC, compared 4.02

to 97 days in the North East LHIN. PLANNING FOR IN-PATIENT DISCHARGE There were minimal guidelines on how Section • Recommendation 1 long it should take from hospital referral to To provide sufficient time for a patient’s family and patient placement in a long-term-care home. other caregivers to prepare for patients’ post-discharge Follow-up Of ALC patients waiting province-wide, 90% •

needs, hospitals should ensure that: 4 were placed in long-term-care homes within key discharge information, such as the patient’s 128 days, with 50% placed within 30 days. • estimated discharge date and discharge destina-

Chapter At the one CCAC that tracked this data, • tion, is established and documented for every long-term-care homes rejected between 25% patient by the time of admission or shortly and 33% of applications because patients thereafter, and revised if the patient’s condition required too much care or exhibited behav- warrants a change in the discharge date; ioural problems. Accepted applicants were quick round-table discussions regarding often just added to a lengthy wait-list. On • patients’ readiness for discharge are attended by the other hand, patients often did not want key decision-makers from the multidisciplinary to go to homes with short or no wait-lists Discharge of Hospital Patients 327

team, such as the patient’s physician, who is the estimated discharge date to their patients, and responsible for discharging the patient, and if some units used whiteboards, it expected to have the patient is going to a long-term-care home or hospital-wide communication of discharge dates, requires home-care services, by a representative using whiteboards, in place by fall 2012. of the Community Care Access Centre; and the estimated discharge date and discharge • ARRANGING POST-DISCHARGE CARE plans are communicated to patients and their families by using visual displays, such as white- Arranging for Home-care Services boards in patient rooms, as recommended by the and Equipment Flo Collaborative. Recommendation 2 Status To better ensure that any required home-care services At the time of our follow-up, two of the hospitals are available when eligible patients are ready to said key information, such as estimated discharge be discharged, hospitals, in conjunction with their date and destination, is generally established Community Care Access Centres (CCACs) and Local for every patient at the time of admission or Health Integration Networks (LHINs), should develop shortly thereafter. One of the two indicated it was time frames that are standardized within each LHIN upgrading its system to make it easier to document that provide adequate advance notice of the date such the expected discharge date after admission. The services will be needed and keep the CCAC apprised of third hospital said it was implementing strategies any changes to the required commencement of home- to establish target dates for patient discharge but care services. added that such estimates are complex, as patients Status are often required to wait for test results and All three hospitals noted that they had started to consultations with specialists before the estimated take steps to address this recommendation. While discharge date can be determined. all three hospitals indicated that they had not yet One of the hospitals said that physicians and implemented standardized time frames within their CCAC representatives generally attend quick round- LHINs to provide advance notice to their CCACs of table discussions about patients’ readiness for the date patients would need home-care services, discharge. A second indicated that while physicians two said their LHINs were developing a consistent 4.02 generally attend or can be consulted during such discharge process for hospitals within each LHIN. discussions, CCAC representatives attend meetings One hospital indicated that it was using a deci- Section only at high-volume in-patient units. The third said sion guide developed in conjunction with its CCAC CCAC representatives attend discharge discussions and in use at the time of our 2010 audit. The guide twice a week, but attend fewer of the daily quick Follow-up specifies timelines required to establish home-care • meetings because of resource constraints. This 4 services and helps ensure that appropriate home- hospital also launched a pilot project in one unit in care services are in place prior to a patient’s dis- May 2012 that was expected to improve attendance

charge from acute care. This hospital also said that Chapter by physicians at the daily quick meetings. it understood that an electronic Resource Matching Two of the hospitals indicated at the time of our and Referral System had been implemented across follow-up that in addition to verbal communication its LHIN, enabling hospitals to provide advance by staff, they were using whiteboards in patient notice directly to the CCAC of the date patient rooms to advise patients and their families of the services will be needed, as well as any subsequent estimated date and plans for discharge. The third changes in the date or services needed. The hospital said that while physicians verbally communicated further noted that it has been working to increase 328 2012 Annual Report of the Office of the Auditor General of Ontario

its community referrals through the Community Status Navigation and Access Program, which uses a cen- At the time of our follow-up, the Ministry said tralized telephone number to connect patients to it had begun to review information quarterly on community support services. patients waiting in hospital for a long-term-care Another hospital noted that implementation of home and discussing the status with the LHINs. The this recommendation depended on other organiza- Ministry observed that these reviews, among other tions, and that it was working with its CCAC to things, promote the development, dissemination create new processes, standards and expectations and adoption of best practices in hospital discharge around timely discharge of patients requiring and long-term-care home placement. As well, the home-care services. The hospital expected that Ministry noted that the LHINs are in the process standardized time frames for patient referrals to the of standardizing the referral management process CCAC would be implemented over the next year. province-wide for moving patients from hospitals to The third hospital noted that its CCAC was usu- long-term-care homes. The Ministry expected that ally involved with patients soon after admission and standardized referral forms would be developed therefore was generally aware of the anticipated by March 2013. Further, as part of the initiative date of discharge and whether or not home-care to standardize the referral process, one LHIN has services would be required. In May 2012, this hospi- implemented an electronic Resource Matching and tal also implemented a system to electronically refer Referral System, which enables the referral and patients to its CCAC for assessment. The hospital matching of patients to the earliest available appro- noted that this system will enable it to better track priate long-term-care home. Another LHIN has how much advance notice it is providing the CCAC implemented a paper-based Resource Matching and about patient discharge dates and the home-care Referral System. The Ministry expected the other services they require. LHINs would pilot resource matching and referral systems during the 2013/14 fiscal year that were expected to shorten the process for placement into Arranging for Long-term Care long-term-care homes. Recommendation 3 The Ministry planned to collaborate with the To improve the process for admitting hospitalized LHINs and the CCACs by fall 2012 to identify the patients to a long-term-care home, the Ministry, steps in the hospital-to-long-term-care home place- 4.02

working in conjunction with the Local Health Inte- ment process for which benchmarks or guidelines gration Networks (LHINs), Community Care Access would likely contribute to reductions in the overall Section

Centres (CCACs), long-term-care homes, and hospi- time to placement. The Ministry also said that tals, should determine the best approach to placing obtaining data from the Resource Matching and Follow-up a patient in a long-term-care home and establish Referral System on actual time frames would be •

4 benchmark standards for completing each stage in the first step in creating benchmark guidelines and this process, such as determining patient eligibility, standards for completing each stage in the process

Chapter completing applications to long-term-care homes, and of placing a hospital patient in a long-term-care the long-term-care homes’ processing of patient appli- home. Once target time frames are in place, the cations. The Ministry should also consider whether Ministry expected LHINs to be responsible for LHINs should be made accountable for monitoring monitoring the CCACs’ performance and reporting adherence to the target time frames. periodically to the Ministry. Two of the hospitals commented that the Home First/Wait at Home program has helped reduce the number of patients waiting in hospital for Discharge of Hospital Patients 329 placement in a long-term-care home. One said it family physicians in the community and began was developing a process to ensure that all options using it in spring 2012. for patients are exhausted before they wait in hos- Another hospital said at the time of our follow- pital for a long-term-care bed; the other said that up that 48% of its discharge summaries were made it asks the families of people waiting in hospital available to a patient’s family physician within for long-term care to choose five homes within 48 hours of the patient’s discharge. This hospital’s five to seven days, and encourages them to select target for the 2012/13 fiscal year was to have homes that have either an available bed or a short 75% of all discharge summaries available within wait-list. This hospital further noted that although 48 hours. The other two hospitals required com- the Long-Term Care Homes Act requires homes pleted discharge summaries within 14 days, but one to accept or reject a patient’s application within was working toward having them available within a five days, it has found that homes usually do not 48-hour deadline. respond within that time frame. With respect to a medication reconciliation template, one hospital indicated that its physicians currently complete an electronic discharge sum- COMMUNICATING INFORMATION TO mary that includes information and instructions on SUBSEQUENT HEALTH-CARE PROVIDERS a patient’s previous and new medications. Another Recommendation 4 hospital expected to have a paper-based medica- To ensure that medical information essential for the tion reconciliation system fully implemented by continuity and quality of patient care is communi- July 2012. This hospital also planned to implement cated in a timely manner to subsequent health-care an electronic medication reconciliation system, but providers, hospitals should: said it was too early to determine the likely timing. • require discharge summaries to be completed The third hospital noted that physicians now com- for all patients in accordance with the Guide plete a medication reconciliation template and that to Better Physician Documentation developed the hospital faxes each patient’s new medication by the Ministry’s Physician Documentation regimen to his or her family physician and pharmacy. Expert Panel; The Ministry indicated at the time of our • establish a target time frame, such as a max- follow-up that about 1,000 physicians had direct imum of 10 days, for completing discharge electronic access to hospital discharge summaries. 4.02

summaries and forwarding them to the patient’s Furthermore, eHealth Ontario, in conjunction with family physician or other subsequent health- the LHINs, was introducing systems province-wide Section

care providers; and to enable physicians who use provincially certified • consider the use of a medication reconcilia- electronic medical records to directly access hospi- Follow-up tion template to be completed for each patient tal discharge summaries by 2015. •

4 detailing any changes between the medications the patient was taking on admission and the HOSPITAL BED AVAILABILITY

medications that the patient will be taking Chapter post-discharge. Recommendation 5 To help reduce the time admitted hospital patients Status wait for a bed: The three hospitals said they were completing dis- hospitals should review the times and days of charge summaries for all patients. One noted that it • the week patients are admitted and discharged, had developed a standardized discharge instruction and arrange patient discharges to allow suffi- sheet based on best practices and feedback from cient time for beds to be prepared in advance for 330 2012 Annual Report of the Office of the Auditor General of Ontario

new admissions, especially for patients arriving information on the real-time status of beds, but that at known peak admission times; and it must use a separate system to let housekeeping • larger hospitals should assess the costs and know about beds that need to be cleaned. In this benefits of implementing a bed management regard, the hospital noted that it had created an system that provides “live” information on the action group to focus on reducing the amount of status of hospital beds, including which beds are time that beds are vacant. occupied, awaiting cleaning, and available for the next patient, as well as the reasons for delays PATIENTS WAITING IN HOSPITAL FOR in placing admitted patients in available beds. POST-DISCHARGE CARE Status Recommendation 6 At the time of our original audit, one hospital had To ensure that patients receive the care they need in adjusted the timing of patient discharges to better the location best for the patient: match admissions, and reviewed the days of the hospitals, in conjunction with their Local Health week and times that patients are admitted and • Integration Networks (LHINs), should educate discharged. It said it was still discharging about all patients and their families on the fact that, a quarter of its patients before 11 a.m. However, for patients whose condition has stabilized this hospital also noted that various factors affect and who no longer need acute care (especially bed availability, including increases in the number older patients), hospitals are not a safe or of emergency-department patients and higher appropriate place to wait for post-discharge care demand for specialized services, such as psychi- (for example, because of the risk of getting a atric and respiratory isolation beds. At the time hospital-acquired infection such as C. difficile); of our follow-up, another hospital said it had also the Ministry, in conjunction with the LHINs, analyzed admission and discharge data to better • should assess the costs and benefits of increasing understand patterns of patient activity, and had the level of post-discharge services that can com- aligned services to meet projected bed demand mence on weekends to better enable hospitals to where possible. The third hospital said it monitors safely discharge patients on weekends; and patient discharges before 11 a.m. and before 2 p.m., the Ministry, in conjunction with the LHINs, and has introduced a target of having an emergency • hospitals, and Community Care Access Centres, 4.02 patient in a hospital bed within 90 minutes of the should give increased consideration to options decision to admit him or her. such as more appropriate places for patients to Section At the time of our follow-up, all three hospitals safely wait for placement in an alternate-care indicated that they had implemented initiatives facility such as a long-term-care home; or to help reduce the time that admitted hospital Follow-up increased supportive-housing arrangements •

patients wait for a bed. In this regard, one of the 4 to enable patients to continue to live more hospitals said its bed management system provides independently. “live” information on the status of hospital beds at

Chapter Further, to help hospitals better manage their all times, including whether empty beds are await- patients who are waiting for post-discharge care, the ing cleaning or are available for the next patient. Ministry should: Another hospital indicated that its initiatives further clarify how alternate-level-of-care (ALC) included a system that tracks the status of beds • wait times should be measured so that ALC wait from the time one patient leaves a bed until a new times are being consistently reported to the Min- one occupies it. The third hospital noted that its istry’s Wait Time Strategy; and electronic bed management system provides some Discharge of Hospital Patients 331

• publicly report the time ALC patients wait home they want is quite long. Two of the hospitals, in hospital before being discharged into a therefore, encouraged patients waiting in hospital community-based setting. to choose five long-term-care homes, and one of these strongly encouraged these patients to choose Status at least two homes that either have beds available At the time of our follow-up, two of the hospitals or have a short wait-list. said they had processes to educate patients and The Ministry also indicated that hospitals their families about the risks of a patient waiting are provided with ongoing clarification on how in hospital for other care when the patient’s condi- alternate-level-of-care (ALC) wait times should tion has stabilized and he or she no longer requires be measured so that these times are reported in a acute care. The third hospital indicated that while it consistent way to the Ministry’s Wait Time Strategy. does not have a standardized process, some hospi- At the time of our follow-up, 114 hospitals were tal staff do discuss this issue with patients. submitting ALC data. In addition, the Ministry has The Ministry said it commissioned a compre- made this ALC information available in real time to hensive study in May 2011 of the costs and benefits hospitals, LHINs and CCACs since September 2011. of increasing post-discharge services on weekends Given the other initiatives to address ALC wait to facilitate a higher percentage of weekend dis- times, the Ministry noted that public reporting charges from hospitals. The Ministry expects the of the time ALC patients wait in hospital was not study to be completed by December 2012 and plans considered necessary at the time of our follow-up. to share it with the LHINs, hospitals and other However, the Ministry said it would monitor the stakeholders to determine how best to proceed. need for public reporting on an ongoing basis. One hospital observed that weekend discharges continue to be a problem because most long-term- care homes and rehabilitation facilities do not PERFORMANCE MEASUREMENT accept weekend admissions, which delays patients’ Recommendation 7 discharge from hospital. To help evaluate the patient discharge process, hospi- The Ministry noted that while it does not have tals should: information on the number of new spaces created in conjunction with their Local Health Integra- for supported housing and assisted living since • tion Networks (LHINs) and Community Care 4.02 March 31, 2010, it has supported the creation or Access Centres, develop measures for monitoring expansion of 28 supported-housing and assisted- and reporting on the effectiveness and safety of Section living services since that time, serving 20% more hospital processes for discharging patients, and people than before. The Ministry said it has also compare results among hospitals to help identify implemented initiatives, including new geriatric Follow-up areas for improvement or best practices that can • services and enhanced services for the frail elderly, 4 be shared with other hospitals; and to help seniors live longer in their own homes. The regularly report key discharge performance Ministry also funded an expansion of the Home •

indicators to senior management and the board Chapter First program, which provides services that help of directors. people wait at home until a bed in a long-term- As well, to help monitor, on a province-wide and care home becomes available. All three hospitals regional basis, unplanned returns to hospital for the supported the Home First program and one com- same or related conditions, the Ministry, in conjunc- mented that while legislation permits patients tion with the LHINs, hospitals, and the Canadian who must wait in hospital to choose up to five Institute for Health Information, should track post- long-term-care homes, it is permissible for them to discharge emergency-department visits as well as make only one choice even if the wait time for the 332 2012 Annual Report of the Office of the Auditor General of Ontario

readmissions to any hospital that occur within a few at the time of our 2010 audit, only one hospital was days (or otherwise established reasonable time frame) doing this. after a patient is discharged from a hospital. The Ministry noted that it does not regularly track emergency-department visits by recently Status discharged patients for the same or a related At the time of our follow-up, all three hospitals condition, although it periodically reviews cases indicated that they were monitoring and reporting of people who for any reason visit an emergency on certain hospital discharge processes. One said department within seven days of discharge it compares its readmission rates for patients with from the same hospital. The Ministry also tracks certain diagnoses to those of other hospitals across unplanned readmissions to the same or different Canada. Another said it compares key performance hospitals within 30 days of discharge for all per- measures, including readmission rates, to peer sons in defined age groups with certain medical hospitals in its LHIN. The third said that while it conditions (for example, people 45 or older with does not perform regular comparisons to other hos- congestive heart failure). The Ministry noted that pitals, its LHIN compares its percentage of patients most unplanned readmissions occur through the discharged by 11 a.m. and 2 p.m. each day to the emergency department of the same hospital from percentages discharged from other hospitals as part which the person was discharged. The Ministry of its Pay-for-Results program. also indicated that it meets with the LHINs quar- All three hospitals indicated that they were terly to review hospital performance, including reporting key discharge performance indicators to readmission rates. senior management and their boards of directors; 4.02

Section

Follow-up

•

Chapter well, management neededto work to instill aculture andcommunicationssystems. tion technology As case-management process, andimprove- itsinforma more aggressive enforcement action,enhanceits in our2010 last We timewe program. audited this concluded payments. Wesupport hadasimilarconclusion the its mandate ofcollecting unpaid andspousal child Office was still not the successfulinachieving that While acknowledging our2010 this, auditfound to avoidlengths required makingtheir payments. obligations, many goto great support others their canbedifficult,andwhilemany willinglyport meet failed to pay support. spousalorchild partners former becausetheir often assistance, orders enforced Officealsocollectsocial by the whohave manyvulnerable; support ofthose their recipients onatimely basis. payments intended toand remit support their obligations—aggressively ifnecessary— support Office (Office),whosejobitis toenforce family- the automatically filedwith Family Responsibility related to divorce proceedings orseparation are andspousalsupport orders for child All court Background Chapter 4 Chapter Enforcing court orders for spousal and child sup- ordersforEnforcing spousalandchild court clientsare amongsociety’sThe Office’s most 4.03 Section

that the Office must take Officemust the that Annual Report Follow-up to VFMSection3.03, to Follow-up Office Family Responsibility Ministry ofCommunityandSocialServices Ministry of our other observations at that timeincluded: atthat observations of ourother of achievement to neededchanges. Some make the • • • • •

or ongoing cases, the Office took almost or ongoingcases,the ofoutstanding ofalmost one-third tatus toll-free Office’s callcentre olumes atthe Officeassigned responsibility hough the months from the issue of the court order. court issueofthe from the months delay was the sevenstraight into arrears, action. For newly registered went casesthat before enforcement its first taking arrears casewent timethe into from the four months had not beenread atall. had beenread butnot they acted upon, orthat notes the was that “open,” indicatingeither onemonth— specific actiononacasewithin notes—intendedbring-forward to trigger callwas hungupbefore answered.ers the did,oneinseven that call- Ofthose through. were nearly sohighthat 80%ofcallsnever got have assignedofficer. direct accessto their payersincluding that andrecipients didnot continued to have significantshortcomings, officer,this case-ownershipservices model casetofor anindividualenforcement each awaiting payments. support and canresult inunduehardship onrecipients more difficult much to enforce cases inarrears delays make Such orders for family support. necessary, andinregistering completed court The Officew F The s Call v Alt 2010 Annual Report Annual 2010 as slow infollowing up,where 333

Chapter 4 • Follow-up Section 4.03 334 2012 Annual Report of the Office of the Auditor General of Ontario

• The Office acted in only one in four or one • the results of its review of a pilot project in in five cases each year to, for example, take which a clerk employed by the Ministry of enforcement action, update case information, the Attorney General had been loaned to the or track down delinquent payers. Office to process documents passing between • The Office had no ualityq control process certain court districts and the Office and so or effective managerial oversight to assess reduce backlogs, the impact of the project on whether enforcement staff have made reason- the Office’s enforcement of family-support able efforts to collect outstanding amounts. court orders, and whether the project will be • The Office could not provide us with a expanded to other Ontario court districts; detailed listing by individual account that • recent monthly statistics on calls to the call added up to $1.6 billion, which was the figure centre (calls answered, abandoned and provided to us as the total outstanding arrears blocked, broken down by local versus 1-800 as of December 31, 2009. calls) and the Office’s efforts to report them • The statistical information supplied monthly quarterly on its website; to the Ministry of Community and Social • recent monthly statistics on office staff absen- Services did not provide a useful summary of teeism and attendance trends by branch; the Office’s successes and failures in collecting • the Office’s progress in obtaining suggestions outstanding support payments or in achieving for improvement from MPP constituency its other key operational objectives. office caseworkers; • Security weaknesses in the Office’s informa- • the current status of outstanding bring- tion technology system put sensitive personal forward notes (notes requiring follow-up client information at risk of unauthorized action on a case to be taken within a month’s access. time), the date by which the Office is to finish • On a positive note, accounting controls cover- revising its policies and procedures for the ing payments from support payers and the proper use of these notes, and the Office’s subsequent disbursement to intended recipi- plan to routinely check and ensure that staff ents were generally satisfactory, and most are issuing the notes appropriately; support payments received were disbursed to • how other jurisdictions approach the enforce- clients within 48 hours of receipt. ment of support orders to payers whose 4.03

We made a number of recommendations for financial circumstances have temporarily and/ improvement and received commitments from or unexpectedly declined; Section

the Office that it would take action to address our • any steps the Office will take to determine concerns. whether enforcement officers are taking the Follow-up most effective and appropriate enforcement •

4 actions, including using its new management STANDING COMMITTEE ON PUBLIC system technology to proactively flag recom- ACCOUNTS

Chapter mended enforcement actions; The Standing Committee on Public Accounts held a • the Office’s progress in negotiating with the hearing on this audit in March 2011. In May 2011, Ministry of Health and Long-Term Care to the Committee tabled a report in the Legislature have OHIP provide it with up-to-date payer resulting from this hearing. The report contained contact information to help the Office enforce 16 recommendations and requested that the Office support orders; report back to the Committee with respect to the • the best option for a caseload management following: model as determined by the Office’s review Family Responsibility Office 335

of caseload management models in other jurisdictions; Status of Actions Taken on • whether the Office is using its capability of Recommendations determining at the end of each month if a payer has made his or her support payment, On the basis of information provided by the Office, and if it is, whether the Office is sending a we concluded that it had made some progress letter informing payers who have not paid on all of our recommendations, with significant that they are in arrears and have 15 days to progress being made on several of them. Signifi- respond or face further enforcement action, cant improvements have been made in registering and if it is not, its reasons for not doing so; support obligations and in introducing a case- the results of the Office’s analysis of support • management client-service model, and further payments in arrears, including the amount improvements are expected once implementation the Office believes is recoverable and whether of the new case-management technology system is other jurisdictions can “write off” amounts completed. Efforts to reduce payments in arrears deemed unrecoverable; and suspense account balances and to improve highlights of the Office’s expected September • performance measurement are ongoing. 2011 report on operational performance The status of action taken on each of our recom- measures and its progress on instilling a more mendations was as follows. results-oriented culture in its workforce; • whether the implementation of the Office’s new case-management system is on schedule REGISTRATION OF SUPPORT for spring 2012 and the timeline for the sys- OBLIGATIONS FOR ENFORCEMENT tem’s key phases; Recommendation 1 whether the case-management system will be • To maximize the likelihood of successfully collecting able to search other ministries’ databases— support obligations, and to help minimize hardships within the bounds of privacy laws—for contact for recipients awaiting their support payments, the information on support payers in arrears; and Family Responsibility Office should: the Office’s strategy for training its workforce • work proactively with family courts in Ontario on each phase of the case-management sys- • 4.03 to encourage them to provide complete and tem and its plans for evaluating whether the accurate information on a more timely basis system meets its business needs. Section so that family-support obligations can be regis- The Committee also recommended that the Min- tered and enforced more promptly; and ister of Community and Social Services request early register and begin to administer new cases Follow-up in the next Parliament that a government motion be • • requiring no additional information within the 4 introduced to establish a Select Committee under Office’s internal target of 30 days of receipt of Standing Order 112(a) to undertake a comprehen- the court order.

sive and comparative review of the Office. Chapter The Office formally responded to the Committee Status in September 2011. A number of issues raised by the In our 2010 Annual Report we noted that the Office Committee were similar to our observations. Where received court orders, on average, 48 days after the Committee’s recommendations are similar to they had been issued by the courts, and there were ours, this follow-up includes the recent actions no ongoing initiatives to encourage the courts to reported by the Office to address the concerns raised forward all support orders or separation agree- by both the Committee and our 2010 audit. ments in a timely manner. We also found that on 336 2012 Annual Report of the Office of the Auditor General of Ontario

average it took an additional 104 days for com- support orders, dealing with licence suspensions pleted court orders to be registered in the Office’s and reinstatements, changing domestic contracts, information system. enforcing support outside Ontario and other tech- In its response to our report, the Office indicated nical matters; and success stories about helping that it had initiated direct outreach and was provid- clients obtain needed support. At the time of our ing quarterly bulletins to the judiciary in an effort follow-up, six bulletins had been issued. to improve the information exchange between the With respect to registrations, the Office courts and the Office, and that it had refined the informed us that it had conducted a blitz in Janu- process it used for address verification to enable ary 2012 and successfully eliminated its backlog of more timely registrations. It also indicated that it case registrations. The Office’s latest registration was initiating two pilot projects, one to provide backlog report indicated that although another courts with real-time electronic access to its data- backlog developed in April and May 2012, it was base to expedite court decision-making on support fully addressed in June 2012. As of August 2012, arrears, and one to place a dedicated court clerk in the Office’s backlog was minimal. its Office to speed up document flow. The Office also engaged a consulting firm in The two pilot projects were initiated in Septem- March 2011 to review its registration processes ber and October 2010, respectively. In the first, a and help develop new policies and procedures for high-volume court was given access to the Office’s monitoring and following up on registration issues. database in an attempt to expedite judicial decision- The firm made a number of recommendations, and making by eliminating the need to adjourn cases the Office informed us that it had implemented until more financial information was obtained. several key ones, including streamlining the routing The results were somewhat disappointing, as the of court orders, conducting trace-and-locate efforts Office’s database was accessed only seven times to ensure the accuracy of client information, and by the court over the course of the 22-month pilot. calling new clients to let them know what to expect In the second project, a court clerk was accom- from the Office’s support programs. modated in the Office with ongoing access to the Office’s database. The Office informed us that this CASE-MANAGEMENT MODELS project was much more successful, with the clerk often being able to process documents in substan-

4.03 Recommendation 2

tially less time because it was no longer necessary Given the lack of effectiveness of the current case- to move between the Office and the courts. Many ownership model in improving the ability of the Section

documents that typically took months to process Family Responsibility Office to collect unpaid support were done within 24 to 48 hours. Although this pro- obligations, the Office should examine processes used Follow-up ject was originally planned to last only five months, in other jurisdictions to determine what best practices •

4 the Office informed us that it had expanded it to might be applicable to Ontario. include all main issuing courts in Ontario except Status

Chapter the Superior Court of Justice in Toronto and the In our 2010 Annual Report we concluded that the Ontario Court of Appeal. Office’s case-ownership model had not been effect- In October 2010 the Office began to distribute a ive in improving the collection of unpaid support quarterly bulletin for the legal community, clients payments, for several reasons. Among these was and other stakeholders to improve communication that Ontario payers and recipients did not have and information exchange with the courts. The direct access to their assigned enforcement service bulletins contain details on new Office initiatives officer, and that each officer had been assigned on of possible interest to the courts; tips on drafting average a very formidable 1,377 cases. Family Responsibility Office 337

In its response to our 2010 report, the Office Status indicated that it was moving to a proactive In our 2010 Annual Report we noted that the Office case-management model whereby a dedicated was not regularly monitoring its call centre and caseworker would be assigned for the life of each therefore had little information with respect to the client’s case and that clients would have easier number of calls that were not getting through and access to their caseworkers and experience fewer the nature of the calls. A 2008 study found that 80% blocked calls. of calls were not getting through the Office’s queuing At the time of our follow-up, the Office informed system, and one in seven callers who did get through us that it had completed an inter-jurisdictional sur- eventually hung up before being answered. vey of best practices to support payment collection The Office agreed with our recommendations, in February 2011. Twenty jurisdictions responded and in its response to our report it indicated that it to the survey, and information gathered from it had just implemented a new telephone system in was to be assessed as future business improvements June 2010 that provided managers with informa- were considered. The Office also informed us that tion to refine the scheduling of call-centre staff and it had established electronic payment transmission to monitor the number of calls not getting through. linkages with the provinces of British Columbia, The Office also noted in its response that it had Newfoundland, Prince Edward Island, Manitoba developed new customer-service standards that and Nova Scotia, and was working to set up similar would provide benchmarks to measure progress mechanisms with the remaining provinces. and future performance. The Office also informed us that it had imple- At the time of our follow-up, the Office informed mented a new case-management/client-service us that it had implemented new call-centre model in November 2011, which was based on its reporting and monitoring processes in December 2008 pilot project. The new model provides clients 2010, which incorporated performance indicators with direct access to the officer responsible for their addressing wait-time management, call-taking case. The Office reported that it has been able to standards, absenteeism and schedule adherence. It eliminate its call blockage problem since the model also informed us that it had implemented customer was implemented, and call wait times have been service standards and a feedback process for online reduced from an average of eight minutes to an clients in January 2011. In February 2011, the average of less than two minutes, while the number Office used its telephone system to collect informa- 4.03 of calls handled has increased from approximately tion on call blockage rates, the analysis of which 48,000 per month to 80,000 per month. indicated a correlation between wait times and Section

call blockage rates. The Office reported that it was able to reduce call blockage rates by 50% after this CALL-CENTRE OPERATIONS Follow-up review. However, the implementation of the new •

4 Recommendation 3 case-management model in November 2011 moved Since the call centre remains the primary means by the Office away from a call-centre business model

which clients communicate with the Family Respon- and it has since been able to eliminate the problem Chapter sibility Office, the Office should review its call-centre of blocked calls altogether. operations and take the steps necessary to ensure that The Office also informed us that it began provid- all calls are answered within a reasonable time. It ing a new customer service training program for its should also track and report the results of its efforts to staff in March 2012. improve call-centre operations. 338 2012 Annual Report of the Office of the Auditor General of Ontario

BRING-FORWARD NOTES SUPPORT-ENFORCEMENT ACTION Recommendation 4 Recommendation 5 To help ensure that the Family Responsibility Office To help it collect arrears more effectively, the deals with such issues as client inquiries and enforce- Family Responsibility Office should ensure that ment actions appropriately and on a more timely enforcement staff: basis, management should monitor whether enforce- • initiate enforcement actions for both ongoing ment services officers review their bring-forward and newly registered cases on a more timely notes, conduct the necessary follow-up work, and clear basis; and up these notes on a timely and appropriate basis. • document why specific enforcement steps were, or were not, taken, and concentrate on those Status steps that are apt to be more successful in par- In our 2010 Annual Report we noted that there were ticular circumstances. approximately 91,000 outstanding bring-forward The Office should also establish a quality control notes awaiting resolution by enforcement officers, process and effective managerial oversight to assess and that the number of bring-forward notes for a whether reasonable efforts have been made to collect sample of these officers ranged from 123 to 1,358 arrears. If it is determined that reasonable efforts per officer. Bring-forward notes are intended to have not been made, it should take corrective action. trigger specific action on a case within one month. Locating payers is often the most challenging issue, We also noted that the status of one-third of these so the Office should also discuss with the Ministry of notes was “open,” meaning they either had not Health and Long-Term Care the current restriction on been read or had not been acted upon, and that access to payer addresses from the OHIP database. despite an Office target of addressing bring-forward notes within 30 days, about half of the notes had Status been outstanding for more than 90 days. In our 2010 Annual Report we noted that approxi- In its response to our report, the Office informed mately two-thirds of all support-payers were either us it would conduct a blitz in the fall of 2010 to in non-compliance or only in partial compliance address outstanding bring-forward notes, and from with their support obligations and that enforce- that point on staff training and new performance ment actions were often neither timely nor effect- measures would be developed to ensure that bring- ive. We found that it took on average almost four 4.03

forward notes were managed properly and followed months after a case went into arrears before officers up in a timely fashion. took their first enforcement action, and seven Section

At the time of our follow-up, the Office months for newly registered cases for which no informed us that it had completed its blitz and child or spousal payments had ever been made. We Follow-up had taken action on all outstanding bring-forward also noted long gaps between enforcement actions, •

4 notes. It had also introduced a new policy and ranging from six months to five years (averaging performance measurement mechanism to monitor two years) and that only 20% to 25% of the Office’s

Chapter and manage bring-forward notes, and all Office total cases were worked on in any given year. staff and managers had been trained on the new The Office agreed with our recommendations, policy in June 2011. The Office said that it planned and in its response to our 2010 report indicated that to eliminate the use of bring-forward notes after its it was updating its policies and procedures to make new case-management system was implemented, enforcement actions more consistent and effective. replacing it with a process of system-generated and It also planned to implement new case-manage- manually created service requests to streamline the ment technology in 2012 to enable more proactive follow-up process. enforcement actions, and was working with the Family Responsibility Office 339 federal government, law enforcement agencies and • establish reasonable criteria and benchmarks other provincial ministries to obtain new tools and setting out what is a manageable caseload, and databases for locating defaulting payers. staff its enforcement activity accordingly; and At the time of our follow-up, the Office informed • regularly monitor and assess the productivity us that it had, with the assistance of an external con- and effectiveness of its enforcement staff, both sulting firm, reviewed its processes for default hear- individually and collectively, in responding ings and warrants of committal in 2009 and then to inquiries, taking timely and appropriate implemented changes to both processes to improve enforcement actions, and collecting outstanding quality control and oversight. It further informed support obligations. us that in November 2010 it had begun discussions Status with the federal government and law enforcement In our 2010 Annual Report, we noted that enforce- organizations to gain access to new trace-and-locate ment officers were each handling 1,377 cases on tools, such as the Canadian Police Information Cen- average, while enforcement staff in two other large tre database. Although this request was eventually provinces operated with average caseloads of 446 denied, in December 2010 the Office of the Registrar and 312, respectively. We further noted that the General agreed that the Family Responsibility Office Office had never established standards for what a could access its database to obtain reports of death reasonable case load should be, and had no mon- registrations and name changes for help with locat- itoring system in place for management to assess ing clients and verifying deaths. enforcement staff productivity. The Office indicated that it had completed a In its response to our report, the Office informed report profiling its arrears in February 2011, and in us that efforts were underway to establish a new April 2011 had initiated a project with the Ministry caseload-management model for staff, and that the of Health and Long-Term Care (MOHLTC) aimed at work would be completed by the end of 2010. In obtaining access to its Registered Persons Database addition, the new case-management system planned containing the address information of Ontario to be in place by the end of 2012 was expected to health-card holders. The Office was granted access equip staff with better tools, such as automated to make unlimited manual address-related quer- reminders, and allow management to better monitor ies of the database, through a dedicated resource the effectiveness of enforcement actions. at MOHLTC. In May 2011 the Office was granted 4.03 At the time of our follow-up, the Office informed further access to the databases of the Municipal us that it had begun tracking productivity meas- Property Assessment Corporation and the Personal Section ures of enforcement staff in February 2011. These Property Security Registry. Finally, in August 2011 measures included how quickly calls were being the Office’s seven trace-and-locate specialists were answered, the timeliness and appropriateness of Follow-up granted unlimited access to the Registered Persons •

enforcement actions and the appropriate use of 4 Database. Office management informed us that bring-forward notes. this tool has been found to be one of the most use-

ful for locating payers, and has led to a significant Chapter improvement in the overall tracing success rate. SUPPORT PAYMENTS IN ARREARS Recommendation 7 CASELOADS To enable it to concentrate its efforts on those accounts most likely to yield results and to objectively Recommendation 6 measure the effectiveness over time of its enforcement To help improve the administration of its enforcement activities, the Family Responsibility Office needs to program, the Family Responsibility Office should: obtain better data on support payments in arrears. 340 2012 Annual Report of the Office of the Auditor General of Ontario

Status payments to intended recipients on a timely basis, it In our 2010 Annual Report, we found that the should strengthen its internal controls by: total amount of support payments in arrears as of • more diligently following up on and clearing December 31, 2009, was approximately $1.6 bil- items in the identified, unidentified, and miscel- lion—up 23% since our previous audit in 2003. The laneous suspense accounts; and Office had minimal information on this balance, • adequately documenting the basis on which and could not provide us with the individual bal- funds have been released from suspense ances that accumulated to that total, or which por- accounts, along with evidence of managerial tion of this balance was deemed uncollectible. review and approval of the release of such funds. The Office agreed with our recommendation, and The Office should also develop the computerized in its response to our 2010 report indicated that it capability to calculate interest on support payments had developed a number of performance measures in arrears. to strengthen its collection function. These measures Status included the cost of collecting support payments, In our 2010 Annual Report, we noted that value of arrears owed, number of enforcement accounting controls over payments received and actions by type, and disbursement rates. subsequently disbursed to recipients were gener- At the time of our follow-up, the Ministry of ally satisfactory. However, we did find that some Community and Social Services informed us that it support payments were in “suspense” accounts had conducted a cross-jurisdictional scan of arrears awaiting resolution for various reasons, and that best practices in November 2010. After this, in these accounts were not adequately controlled. For December 2010, the Ministry launched a project to example, we found that the Office failed to follow develop the capacity to gather improved data about up on or clear almost three-quarters of the items arrears, client behaviour and enforcement effective- in one such account, totalling $2.9 million, within ness. The project had four deliverables: a profile the required 90 days of receipt, and that the aver- of arrears, an assessment of arrears collectability, age age of the balances in this account was more an assessment of the effectiveness of the Office’s than nine months. Another suspense account that enforcement actions with respect to collection and held $2.1 million at the time of our audit contained compliance, and a guideline on how to better target balances with an average age of over three years. cases to generate support payments. The profile of 4.03 A third suspense account found not being properly arrears was completed in February 2011, and in Sep- addressed contained $7.2 million. We also found tember 2011 the project team developed operational Section that the investigations and decisions to release measures related to arrears, including the cost of funds from these suspense accounts were often not collecting support payments, the number of enforce- adequately documented or approved. Follow-up ment actions by type, and disbursement rates. •

In its response to our report, the Office indicated 4 The Office further informed us that it had begun that it had assigned staff resources to follow up on updating policies and procedures related to manag- its suspense account balances, and was planning to

Chapter ing and categorizing arrears in March 2012. The revise its financial policies to incorporate perform- Office planned to complete this project in late 2012. ance time frames in an effort to get payments to clients more quickly. PAYMENT PROCESSING At the time of our follow-up, the Office informed us that it had realigned the accounting functions Recommendation 8 in July 2010 and added several financial staff to While the Family Responsibility Office is generally enhance the segregation of duties, increase its successful in processing and getting most support Family Responsibility Office 341 analytical capacity and better monitor the client of areas where management information would support payment process. In October 2010 the be useful, such as the time required to disburse Office implemented new procedures for sorting funds to intended recipients, the timeliness of vari- and analyzing suspense account transactions and ous enforcement actions and the length of time for documenting the reasons for holding or releas- accounts have been in arrears. ing funds in these accounts. Further, it developed In its response to our report, the Office indicated new reports to allow daily monitoring of suspense that it had established a performance measurement account balances. framework and was working to develop operational The Office also informed us that it had imple- measures for items such as the cost of collecting mented a user-log policy for bring-forward notes support payments and the time needed to respond in March 2011, which included expected time to a changed support order. frames for action on the notes and for payment At the time of our follow-up, the Office informed processing. In the fall of 2011 the Office initiated a us that it had completed several performance trace-and-locate project for a sample of transactions measurement projects in 2010. The first was the in the miscellaneous suspense account and was implementation of an “Executive Dashboard” pro- able to close out some of the accounts and release viding management with an overview of key Office payments to clients. The Office reported that, as of performance indicators and measures showing March 2012, it had reduced the suspense accounts whether the Office was achieving internal targets by $2.2 million (44%) and the miscellaneous in areas such as customer service, operations and account by $1.4 million (19%) relative to the financial strength. In November 2010 the Office December 2009 balances. implemented operational performance measure- With respect to interest on support payments ment at the branch level to provide managers and in arrears, the Office informed us that it does not directors with information to better manage their have the legislative authority to calculate interest, staff and business processes. In December 2010 it but that it does pursue interest where a court order established a protocol for incorporating its perform- includes an interest payment provision, when ance measures into the new case-management sys- claimed by the recipient. tem so that this information would still be available when the system went live. The Office further informed us that it had 4.03 PERFORMANCE MEASURES recruited a senior manager of program effective- Recommendation 9 ness and quality assurance in October 2011 to bring Section

To help assess whether the Family Responsibility greater focus and expertise to the areas of target- Office is meeting its stated objectives, and to help iden- setting and performance measurement. In January Follow-up tify in a timely manner those areas needing improve- 2012 the Office finalized a quality assurance frame- •

4 ment, the Office needs to define its key operational work incorporating the performance measures that indicators, establish realistic targets, and measure had been developed.

and report on its success in meeting such targets. The Office indicated that it would continue its Chapter work to further define and measure additional pro- Status gram performance indicators in the areas of collect- In our 2010 Annual Report, we concluded that ability, enforcement effectiveness and case profiling. the Office did not have sufficient information to enable it to properly assess its success in meeting its operational objectives, or for identifying areas in need of improvement. We suggested a number 342 2012 Annual Report of the Office of the Auditor General of Ontario

COMPUTER SYSTEMS legacy systems, including penetration testing on its firewalls and servers and the initiation of a project Recommendation 10 to investigate enterprise file-transfer processes to Pending development and implementation of a new improve controls over information exchanges with IT system, the Family Responsibility Office should external organizations. It also affirmed that it was strengthen security requirements and processes for on track to deliver a new case-management solu- its existing IT operations, including the Maintenance tion in 2012. Enforcement Computerized Assistance system, to help At the time of our follow-up, the Office informed better protect sensitive client information. us that it had established a quarterly password Status review process to better control and monitor system In our 2010 Annual Report, we concluded that the access, and in December 2010 it had its firewalls Office’s main business software was out of date and and services located in the Kingston data centre did not adequately support the administration of tested by OPS Corporate Security. The Office fur- the Office, and a project to develop a new computer ther informed us that it had expanded the use of its system had been discontinued after $21 million had electronic file transfer service to securely exchange been spent on it. At the time of our audit the Office information with a number of external organiza- had been developing another new system, with an tions in March 2011. We were also informed that expected implementation date of April 2012 at a the new case-management solution is close to projected cost of $49.4 million. completion and is expected to go live late in 2012. In response to our report, the Office indicated it had taken steps to mitigate risks related to its 4.03

Section

Follow-up

•

Chapter consistently province. across the Ourwork included were provided services care needsand ensure that were mechanisms whether inplaceto meet home Term Care (Ministry). andLong- ofHealth Ministry toare accountable the NetworksIntegration (LHINs).TheLHINs,inturn, province’s 14 to oneof the reports LocalHealth 2009/10 fiscal year—a 6%increase. clients, compared to about600,000clientsinthe In 2011/12, to 637,700 provided services Ontario fiscal year to $2.1 increased 10.5%, from $1.9 years, homecare fundinghas past three In the admissionsto homes. long-term-care authorize daily living.CCACs with asassistance also such andhomemakingservices, well support aspersonal physiotherapy asnursing, andsocialwork,such as ity andapprove provision ofprofessional services, ownindependently aspossibleintheir homes. elderly disabilitiesto live peopleand with as homes.Homecare alsoassistslong-term-care frail, might needto beadmittedsupports, to or hospitals and services these who,without ces to Ontarians providers to- provide service homecare servi with Community Care Access Centres (CCACs) contract Background Background Chapter 4 Chapter In 2010, we conducted anauditto assess There are 14 ofwhich each CCACs inOntario, The CCACs assesspotential clientsfor eligibil- 4.04 Section

billion in the 2011/12billion inthe fiscal year. Follow-up to VFM Section 3.04, VFMSection3.04, to Follow-up Home CareServices Ministry ofHealthandLong-TermMinistry Care

billion in the 2009/10billion inthe Among oursignificantfindings: still remained. homecare program ofthe 2004) identifiedinourprevious audits(in cerns 1998 and they receive. the services satisfied with homecare clientswereindicated that generally and independentCCAC clientsatisfactionsurveys had increased audit, substantially sinceour2004 homes. We alsonoted homecare funding that life benefits by allowingtheir people to remain in cost savings both andquality-of- offers services enhancinghomecare hadrecognized that Ministry of ouraudit. CCAC),Brant 11 andwe other the surveyed aspart CCACCentral andHamilton Niagara Haldimand 14 ofthe visits to three East CCACs CCAC, (South • • However, we maincon- noted someofthe that We acknowledged inour2010 the auditthat lthough ministry policyrequired CCACs ministry tolthough widely homecare fundingvaried er capita individuals assessed as high-risk oraboveindividuals assessedashigh-risk only those sothat itsservices had prioritized CCACs three weconstraints, oneofthe visited where clientslived, asa result offunding ner to ensure equitable accessnomatter inaconsistentadminister man programs locations. tion size,age andgender ofclients,orrural takes localneedsaspopula - into accountsuch needs, oreven onamore general basisthat basis ofspecificclient been allocated onthe inequities. Total funding to CCACs hadnot 14among the CCACs, resulting infunding A P 2010 Annual Report Annual 2010 - 343

Chapter 4 • Follow-up Section 4.04 344 2012 Annual Report of the Office of the Auditor General of Ontario

would be eligible for personal support servi- Ministry and the CCACs that they would take action ces, such as bathing, changing clothes, and to address our concerns. assistance with toileting. Clients assessed as moderate-risk in this CCAC were deemed not eligible, while they would have been eligible to receive home care services in the other two Status of Actions Taken on CCACs we visited. Recommendations • Eleven of the 14 CCACs have a wait list for various home care services. The other three According to information we received from the CCACs said that they had virtually no wait Ministry and the three CCACs we visited for our lists at all. This is another indicator of a pos- 2010 audit work, some progress has been made in sibly inequitable distribution of resources addressing all of the recommendations we made among the 14 CCACs. in our 2010 Annual Report. Most will require more In the absence of standard service guidelines, • time to be substantially implemented. For example, each CCAC developed its own guidelines CCACs expect to have new Standards of Care to bet- for frequency and duration of services. As a ter manage caseloads by March 31, 2013. As well, result, the recommended time allocated for efforts to improve the way funding is allocated to each task and the recommended frequency CCACs and the way CCACs pay service providers, of visits varied, indicating that the level of to better reflect client needs and in accordance service for people with similar needs may vary with outcomes, are being phased in. The status of from one CCAC to another. the actions taken is summarized following each Although CCACs had made progress in imple- • recommendation. menting a standardized initial client-care assessment tool, these assessments were often not done on a timely basis. FUNDING OF HOME CARE SERVICES Only one of the CCACs we visited conducted • Recommendation 1 routine, proactive visits to its service providers To help ensure that people with similar needs living in to monitor the quality of services delivered. different areas of the province receive similar levels of CCACs expressed concern with not being able 4.04 • home care service, the Ministry of Health and Long- to procure services from external service pro- Term Care, in conjunction with the LHINs, should allo- viders competitively. The Ministry had asked Section cate funds to CCACs primarily on the basis of assessed them to suspend the competitive procurement needs of each local community, using, for instance, the process on three occasions since 2002, and, proposed Health Based Allocation Model. Follow-up at the time of our 2010 audit, the process •

4 was still suspended. This has contributed to Status significant differences in rates paid to service The Ministry informed us that it is undergoing

Chapter providers for similar services. Health System Funding Reform to move away • The 14 CCACs have made good progress from global funding and toward patient-based in implementing an updated case manage- funding. Funding is to be allocated to the CCACs ment information system to provide useful using the Health Based Allocation Model (HBAM), information to help measure and improve which estimates expected expenses at the CCAC performance. level based on demographic, clinical and financial We made a number of recommendations for information. Also, funding for some procedures improvement and received commitments from the would be allocated under another component Home Care Services 345 of patient-based funding called quality-based Status procedures, where health-care providers receive According to the Ministry and the three CCACs, funding for the number of patients they treat for the new Client Care Model under the Integrated certain types of procedures on a price-by-volume Client Care Project (ICCP) is redesigning the way basis, using standard rates for each procedure. health-care services are provided. The Client Care The Ministry informed us that it has modified Model uses a population-based case management HBAM to take into account the specific costs of approach developed by the Ontario Association each CCAC and to include the components of the of Community Care Access Centres. Clients are Resident Assessment Instrument – Home Care tool, categorized into different populations and sub- such as measures of health status, to group CCAC populations based on factors such as health condi- clients. It used the revised HBAM in December 2011 tions, degree of independence and risk of acute to allocate $27 million in base funding to the LHINs episodes. The model enables case managers to to begin addressing some of the historical funding specialize in specific client populations, co-ordinate inequities among the 14 CCACs. It also applied client care across the entire health system and HBAM in the 2012/13 fiscal year to redistribute monitor the impact of the care provided. approximately 10% of CCAC base funding while The three CCACs informed us that, also as part ensuring that system stability was maintained and of ICCP, they are working to implement Standards access to services preserved. The Ministry also of Care, case manager–client caseload guidelines informed us that in the 2012/13 fiscal year it would outlining the role and expectations for client servi- use HBAM to determine a portion of the new LHIN ces staff by client population and sub-population. base-funding allocation for the community sector The three CCACs informed us that all 14 CCACs announced by the government in that year. were categorizing clients according to consistent The Ministry said that it would continue to work definitions of populations and sub-populations to with the CCACs to further refine the funding model enable caseload realignment among case managers. for future years. As part of this process, the Ministry The standards in place at the time of this follow-up plans to increase the portion of CCAC base funding had been approved in July 2011 by all CCACs. They allocated under Health System Funding Reform are to evolve as future sub-populations are identi- until CCACs are receiving 70% of their allocations fied and as implementation occurs. CCACs antici- by patient-based funding, by the 2014/15 fiscal year. pated complete implementation of the standards by 4.04

The Ministry also indicated that it would provide March 31, 2013. supports to inform and assist CCACs and LHINs with At the time of our follow-up, the Client Care Section the transition to the new funding approach. Model was being applied to four areas of care: individuals requiring wound care; individuals requiring Follow-up palliative care; frail seniors; and medically fragile • DELIVERY OF HOME CARE SERVICES 4 children. The model focuses on high costs and/ Case Management Caseloads or volumes and the potential to improve care

and either reduce the cost of care or increase the Chapter Recommendation 2 amount of care provided for the same cost. To ensure that case managers are deployed optimally The Ministry informed us that it was testing a and to encourage equitable service levels across the new wound care model using outcome-based path- province, the Ministry of Health and Long-Term Care ways and outcome-based payments at four CCACs, should work with LHINs and the Ontario Association with a goal to apply the new model at all CCACs of Community Care Access Centres to establish case and for all types of wounds. Five palliative care test- manager–client caseload guidelines. ing sites have also been launched, and the CCACs 346 2012 Annual Report of the Office of the Auditor General of Ontario

are to begin initial research for outcome-based care System (CHRIS) to support compliance with the pathways for frail seniors and medically fragile chil- provincially endorsed Standards of Care at the dren in April 2013. Participating CCACs and service individual client level. This should also enable providers are expected to be identified at that time. better monitoring of client assessments and re- According to the Ministry, all work completed as assessments. The improved system will remind case part of the ICCP is being assessed by external evalu- managers when assessments are due by client care ators to identify the best policy options for health model and population type according to the Stan- care and health-care funding and incorporate them dards of Care. The enhancements to CHRIS began into the system. to go live in October 2012. The three CCACs indicated that they have been monitoring reassessment standards through Admission to Services or Wait-lists, various means while they waited for the CHRIS Service Levels, Monitoring Home Care enhancements to be fully implemented. The first Services Provided, Client Reassessment for CCAC has been conducting manual audits of the Continued Services files of clients who have been identified as not hav- Recommendation 3 ing had an assessment when planned, to ensure To help ensure that an appropriate and consistent that the reasons for this are valid and documented. level of service is provided to home care clients, Com- The second CCAC has been tracking the frequency munity Care Access Centres should: at which standards are met at the caseload level • monitor case manager adherence to the and the organizational level to help pinpoint prob- established timelines for both the initial client lem areas. The third CCAC’s case managers were in assessment and the periodic client reassessments the process of setting timelines for the frequency and, where such timelines are not met, ensure at which reassessments should occur, according that case managers document the reasons in the to type of client population. It had completed its applicable client files; review of caseload files by July 2012 and expected • enhance external provider oversight to better to adjust the frequency of its reassessments to ensure that the expected and paid-for levels of match the new standards of client care for CCACs service are being provided to home care clients; by the end of November 2012. and The three CCACs informed us that they have 4.04

• regularly review both client complaints and adopted an audit framework for the oversight client events to identify any systemic areas of contracted service providers. The framework Section

requiring further follow-up. includes a process for risk evaluation to pinpoint To promote equitable funding and service levels areas that provincial audit activity should focus on. Follow-up across the province, the Ministry of Health and In 2011 an audit was conducted at three CCACs •

4 Long-Term Care, in partnership with the LHINs, on the identification, reporting, management and should consider incorporating summary data from quality-improvement processes related to missed

Chapter the standardized Resident Assessment Instrument to visits. Audit processes for assessing compliance and assist in developing a more client-needs-based funding quality of services were to be further developed by model and to encourage the CCACs to adopt consist- a provincial Missed Visit Working Group through- ent criteria for prioritizing the differing levels of home out 2012. care services. According to the three CCACs, the Ontario Asso- ciation of Community Care Access Centres main- Status tains a province-wide reporting site that captures According to the three CCACs, work was underway data that can be used to assess service provider to enhance the Client Health Related Information Home Care Services 347 performance. All three CCACs indicated that they • formally evaluate the expected cost savings from regularly review performance data with their ser- allowing CCACs to procure home care services on vice providers and discuss areas where the quality a competitive basis, keeping in mind the poten- of service could be improved. They also indicated tial impact on clients and service levels; and that work was underway to further develop the per- • in the meantime, conduct a review of service- formance measures to better support the monitor- provider rates by type of service across Ontario ing of client outcomes; their target was to introduce to determine whether the significant rate varia- public reporting through Health Quality Ontario at tions are warranted in relation to the actual cost the service provider level by the 2013/14 fiscal year. of providing the service. All three CCACs informed us that they track Status complaints and events and monitor these for areas The Ministry informed us that government direction requiring follow-up. The 14 CCACs developed a for CCACs to proceed with competitive procurement provincial common events framework to standard- was still pending at the time of our follow-up, thus ize the way in which client complaints and certain the CCACs had not yet been able to start procuring key events are tracked and managed across their home care services on a competitive basis. The sectors. The three CCACs informed us that 13 of the Ministry also informed us that the CCACs are work- CCACs had finished aligning their data capture with ing with their providers to ensure that the providers the new events framework, and that the remaining are aware of performance expectations, are focused CCAC would align its data in the future. on continuous quality improvement as part of their As noted earlier, the Ministry is moving toward core business, and have appropriate tools, training patient-based funding and has been working with and information to support quality delivery. the CCACs to enhance the Health Based Allocation The Ministry indicated that a key design element Model (HBAM), which uses demographic, clinical of its Integrated Client Care Project (ICCP) was the and financial information to estimate expected development of an alternative payment process. expenses at the CCAC level. The Ministry informed Following the new process, payment to contracted us that part of the enhancements to HBAM included home care service providers would be based on incorporating relevant components of the Resident their achievement of specific client outcomes using Assessment Instrument – Home Care to allocate evidence-based, best-practice care pathways. At the funding for long-stay clients. This includes data 4.04 time of our follow-up, this alternative payment pro- such as measures of health status, which can cess was being tested at two wound care sites. First, be used to group clients and related costs. Also, Section wound care milestones are identified (for example, as noted earlier, CCACs have been working on a wound must be 30% healed in 30 days), and then developing and implementing Standards of Care, payment to the provider is to be based on achieving Follow-up which would establish consistent criteria for priori- •

these milestones. The CCACs noted that the work on 4 tizing levels of home care services. the alternative payment process would inform the Ministry’s patient-based funding initiative as well as Acquisition of Services from Contractors the contracting approach in the future. According to Chapter the Ministry, patient-based funding addresses rate Recommendation 4 variations, as payment is based on what it refers to To ensure that home care services are procured from as evidence-based best-practice care pathways as external providers in a cost-effective manner, the Min- opposed to units of discrete service. istry of Health and Long-Term Care should work with The Ministry informed us that ICCP has also LHINs and the Ontario Association of Community developed the necessary changes to the electronic Care Access Centres to: 348 2012 Annual Report of the Office of the Auditor General of Ontario

information system to accommodate a new type mendations from the CCAC sector. The last informa- of billing. Instead of units of home care service tion session, held in March 2012, was to help the (volume of service), billing will be done by service Ministry and the CCAC sector to obtain a greater bundles tied to client outcomes and established understanding of the issues around data accuracy, payment amounts. The billing changes were to be consistency and outliers as well as the processes rolled out for wound-care populations at half of the used to address these issues. The Ministry indicated CCACs in October 2012, with full implementation that all 14 CCACs participated in the session. to take place by April 2013. According to the Min- According to the Ministry, LHIN finance and per- istry, this billing approach will be applied to pallia- formance teams are using the CCAC data for com- tive care around April 2013 and to other categories parative purposes and performance monitoring. For of care as the care pathways for them and related example, the financial reviews included utilization payment amounts are determined. or cost comparisons by functional centre and com- Building on the work conducted under the parative information about how and where each ICCP, the CCACs launched an initiative in April CCAC spent its funds compared to other CCACs. 2012 referred to as Quality and Value in Home According to the Ministry, LHINs monitor indi- Care. Under this initiative, CCACs are working with vidual CCAC activity on a monthly and quarterly service providers and their respective associations basis, using dashboards to compare performance to to review contract requirements. The reviews are expected results. LHINs also monitor CCAC activ- to encompass integration of care; outcome-based ity against other CCACs to review and compare methods of reimbursement; service provider per- performance. Comparisons are made in areas such formance indicators; contract management that is as case management full-time equivalents; nursing based on achieving client outcomes; market share volumes, personal-support volumes and home- allocation; geographic boundaries; and volumes of making volumes; therapy volumes and expendi- service. tures; and school services utilization. The three CCACs informed us that they report individually to their LHINs on indicators required DATA MANAGEMENT AND ANALYSIS under their accountability agreements, as well Recommendation 5 as any additional indicators that LHINs require To reap the full benefit of the recent improvements individual CCACs to regularly report on, based on 4.04

to the case management information system, the the priorities in their geographic area. CCACs also Ministry of Health and Long-Term Care, working with indicated they are also often asked to provide addi- Section

the LHINs, should review the summary-level data tional, ad hoc reports to LHINs and to the Ministry. on a province-wide and regional basis as a means of Various home care quality indicators for long-stay Follow-up enhancing its oversight of the home care services cur- clients—such as community wait times, cognitive •

4 rently being provided. functions and client satisfaction with home care services—are reported through Health Quality Status

Chapter Ontario’s website. The Ministry informed us that it has conducted annual standard education sessions based on recom- times. While the Ministry and the hospitals had hospitals andthe Ministry times. Whilethe impactgreater onemergency crowding andwait maying hospitalization well have hadaneven able in-patientbedsfor emergency patientsrequir However, our2010 ofavail lack auditfound that emergency asunderstaffed departments. such administrativeminor ailmentsandhospital issues wait timesare walk-indepartment patientswith causes ofwhatcanbelengthyemergency- 2009/10). cost ofapproximately $1.1 visits (5.4 were about5.9 and treatment. Inthe specialists, with consultations services, laboratory and/orphysiciannurse assessment,diagnostic and according urgencyillnessorinjury), to the oftheir patients process ofprioritizing is,the triage (that andaccurateloading ofambulancepatients,quick asprompt ofinterrelated- variety elements,such off patient care in emergencydepend on a departments by of ambulance.Thequality andefficientdelivery or inperson either to patientswhoarrive injuries ofillnessesand treatment for a broad spectrum provide emergencyHospital medical departments Background Chapter 4 Chapter The public suspects that the mainunderlying the The publicsuspectsthat 4.05 Section

million in2009/10) ata inOntario,

million emergency-department

2011/12 Follow-up onVFMSection3.05, Follow-up Departments Emergency Hospital Ministry ofHealthandLong-TermMinistry Care

billion ($960

fiscal year,there

million in - - were asfollows: , someofourmore significantobservations Report conditions.Inour more serious with or met provincial targets, especially for patients times hadnot yet shown significantimprovement timeofouraudit,emergency-department waitthe been actively attempting problem at to address the • • • • t the three hospitals we hospitals timeli three visited, the t the ovincially, only 10% to 15% patients ofthe patient flow.the of three-quarters than More affected emergency-and diagnostic services ness ofaccessingspecialist consultations before beingseenby orphysicians. nurses triage waited after six hours for more than patients timelines, andsometimes these recommended seen by the physicians within emergentwith andurgent conditionswere illnessesorinjuries. patients’ severity ofthe triaged, underestimating the majority were the under- Ofthese, incorrect. assigned by were triagenurses found to be CTAS educators, the nurse levels originally anhourto betriaged. more than wehospitals visited, somepatientswaited Yet emergencythe department. atallthree 10triaged within to 15 patientsbe guidelines recommend that A Pr In abouthalfoft The CanadianT 2010 Annual Report Annual 2010 riage andAcuity Scale(CTAS) he triagefiles reassessed by

minutes of arrival at minutes ofarrival 2010 Annual - 349

Chapter 4 • Follow-up Section 4.05 350 2012 Annual Report of the Office of the Auditor General of Ontario

hospitals that responded to our province-wide survey of 40 hospitals indicated that limited Status of Actions Taken on hours and types of specialists and diagnostic Recommendations services available on-site were key barriers to efficient patient flow. According to information provided to us by the At the time of our audit, emergency- • Ministry and the hospitals, good progress has been department patients admitted to in-patient made in implementing most of the recommenda- units across the province spent on average tions we made in our 2010 Annual Report. Efforts about 10 hours waiting for in-patient beds. to identify reasons for excessive wait times and Some waited 26 hours or more. Delays in to reduce them have resulted in some improve- transferring patients from emergency departments in patient flow, and these efforts continue. ments frequently occurred because empty New legislative requirements have expanded the beds had not been identified or cleaned on a reporting that hospitals do on the quality of care timely basis. they provide. However, the hospitals indicated Two of the three hospitals we visited had dif- • that ensuring that there are sufficient nurses and ficulty finding staff to fill nursing schedules, physicians to adequately staff their emergency especially for night shifts, and on weekends departments remains a challenge. The Ministry and and holidays. A number of emergency- the hospitals indicated that it will therefore take department nurses worked significant additional time to fully address some of our recom- amounts of overtime or took extra shifts, mendations. The current status of the actions taken leading to additional costs and increasing the on each of our recommendations is as follows. risk of burnout. • Paramedics often had to stay in emergency departments for extended periods of time ONTARIO’S WAIT TIME STRATEGY FOR to care for patients waiting for emergency- EMERGENCY DEPARTMENTS department beds or until emergency- Recommendation 1 department nurses could accept them. To ensure that emergency departments are operating Province-wide, about half of emergency- • in the most effective way to provide high-quality department visits were made by patients with 4.05 emergency care as quickly as possible to all patients: less urgent needs who could have been sup- hospitals should identify causes of delays in ported by alternatives such as walk-in clinics, • Section patient flow and examine ways of reducing wait family doctors, and urgent-care centres. times in emergency departments accordingly; We made a number of recommendations for the Ministry of Health and Long-Term Care Follow-up improvement and received commitments from the • • should work with the LHINs and with hospitals 4 Ministry and the three hospitals we visited that to identify and disseminate best practices from they would take action to address our concerns. Ontario and other jurisdictions; and Chapter • the Ministry should provide funding to hospitals in a timely manner to enable hospitals to have adequate time to implement the funded initiatives cost-effectively.

Status All three hospitals we visited informed us that they have taken actions to identify causes of delays Hospital Emergency Departments 351 in patient flow and examine ways of reducing improvement of about 26% as compared to wait times in emergency departments. One of the 14 hours in April 2008, but still above the hospitals has developed strategies that include eight-hour target. promoting the use of its urgent care centre to the In order to ensure that hospitals receive funding community as an alternative to the emergency to implement the initiatives in a timely manner, the department and reviewing discharge patterns in Ministry has created a Transfer Payment Operating the in-patient units. Another hospital has created and Capital Funding Packages Roadmap to stream- a Process Improvement Office with a focus on the line its funding processes and increase operational root-cause analysis of delays. The third hospital has efficiencies. The Roadmap is an interactive refer- introduced a Medical Assessment and Consultation ence document that shows hospitals how to prepare Unit for rapid assessment, treatment and admission a Transfer Payment Funding Package by providing of emergency-department patients. connections to the necessary templates, documents The Ministry indicated that it has expanded and information. and continued the Pay-for-Results program and the Emergency Department Process Improvement TRIAGE PROCESS Program (ED PIP) in the 2010/11 and 2011/12 fiscal years. The ED PIP trains staff on best practices Recommendation 2 and supports hospitals in funding the implementa- To ensure that triaging is done appropriately and tion of best practices and local solutions to improve consistently within the recommended time frame: patient flow and reduce wait times. • hospitals should conduct periodic audits to • As part of the ED PIP, staff teams across monitor the quality and accuracy of triage and 81 hospitals were trained in the use of Lean identify areas for improvements; methodology (a methodology focused on • hospitals should consider performing a quick process speed, efficiency and elimination of “pre-triage” on patients who cannot be tri- waste) and engaged in follow-up activities aged immediately upon arrival at emergency to improve patient flow in their emergency departments; departments and in-patient units by eliminat- • the Ministry of Health and Long-Term Care ing duplicative or unnecessary steps. The should work with the LHINs and with hospitals Ministry told us that it has also developed a to assess whether the reported length of stay at 4.05

comprehensive toolkit and web resources for emergency departments should include the time hospital use that will be maintained by Health that patients wait for triage; and Section

Quality Ontario (formerly the Health Quality • the Ministry should work with the Emergency Council of Ontario). Medical Services (EMS) to provide updated train- Follow-up The Pay-for-Results program, covering 74 hos- ing for paramedics to ensure that hospitals and •

• 4 pitals, continues to have a positive impact paramedics are using consistent triage practices. on reducing the wait times at emergency Status

departments for patients with minor condi- Chapter All three hospitals we visited stated that they have tions. Ninety percent of these patients were conducted triage audits on a routine basis and treated within 4.2 hours, almost achieving received positive results. For example, one hospital the four-hour target, according to the most informed us that its triage audits have consistently recent data in June 2012. For patients with demonstrated a level of completeness and reliability complex conditions, in June 2012 the longest of about 80%, which is a significant improvement that 90% of them could possibly spend in over the 44% reported in our 2010 audit. In order emergency departments was 10.3 hours, an 352 2012 Annual Report of the Office of the Auditor General of Ontario

to reduce the burden of auditing triage records and ASSESSMENT AND TREATMENT to improve the completeness and reliability of tri- Recommendation 3 age, one of the hospitals implemented an electronic To ensure that patients receive timely assessment triage system in February 2012 that includes and treatment and an appropriate level of care at automated prompts and mandatory fields in triage emergency departments: records. All three hospitals we visited in 2010 hospitals should work with the respective LHINs have either put “pre-triage” in place or made other • to develop, document, and implement proced- improvements in their triage process. For example, ures for monitoring and reassessing the status one hospital has introduced a process called “walk of patients in the time interval between triage the line,” which requires the triage nurse to walk and treatment in accordance with their assigned through the waiting area to take patients at the triage level; and greatest risk to triage first, and another hospital has the Ministry of Health and Long-Term Care renovated the design of its emergency department • should encourage hospitals to track critical to ensure that staff can see all patients in the wait- quality-of-care measures with respect to the most ing area. serious time-sensitive illnesses commonly seen in Regarding the time from arrival to triage, the emergency departments and consider the applic- Ministry has obtained advice from Cancer Care ability of protocols or best-practice guidelines for Ontario (CCO), which has managed emergency- those illnesses on a system-wide basis. department data through the National Ambulatory Care Reporting System (NACRS). CCO identified Status technical limitations in collecting accurate patient Regarding the procedures for monitoring the status arrival times at emergency departments. In of patients in the time interval between triage and August 2011, CCO conducted an analysis and found treatment, only one of the hospitals we visited that monitoring and reducing the time from patient indicated that it has a reassessment triage nurse in arrival to triage would have a negligible effect on place to reassess patients waiting to be seen on a the total length of stay in emergency departments. regular basis and to alert physicians immediately if Therefore, the Ministry will continue to focus on a patient’s status changes. The other two hospitals capturing the wait time after triage or registration, have relied on staff to follow their existing guide- whichever is performed first. lines in reassessing the condition of patients. 4.05

To ensure that hospitals and paramedics The Ministry indicated that the following actions use consistent triage practices, the Ministry has have been taken subsequent to our audit to ensure Section

developed a Pre-hospital Canadian Triage and that patients receive a timely, high-quality and Acuity Scale (CTAS) Paramedic Guide, which will appropriate level of care at emergency departments: Follow-up support paramedics in assigning, communicating Through the Excellent Care for All Act, 2010, •

• 4 and documenting the appropriate CTAS levels. At the Ministry has required every public the time of our follow-up, the Ministry expected hospital to establish a Quality of Care Com-

Chapter that the guide would be released at the end of mittee that reports on issues related to the August 2012 and would be posted on the Ministry’s hospital’s quality of care. All public hospitals website and distributed to all municipalities are required to submit their annual quality (municipalities have the primary responsibility for improvement plans to Health Quality Ontario dispatching land ambulances). and to make the plans available to the public. • As part of the Pay-for-Results program, the Ministry has continued to require all hospitals Hospital Emergency Departments 353

with 30,000 or more annual emergency- provided to the Ministry, hospitals and LHINs to department visits to submit information on assist hospitals in assessing the impact of specialist quality-of-care indicators regularly to their consultations on the total time patients spent in the Quality of Care Committees. emergency departments. As well, the Ministry and • The Ministry has held regular meetings with CCO are currently evaluating the feasibility of and the CEO at each LHIN to discuss wait times and technical requirements for collecting wait-time data quality of care at the hospitals in each LHIN. for MRI and CT scans for emergency patients. The Ministry informed us that preliminary data would be available by the summer of 2014. CO-ORDINATION WITH OTHER HOSPITAL DEPARTMENTS PATIENT DEPARTURE FROM THE Recommendation 4 EMERGENCY DEPARTMENT To better allow hospitals to assess the impact that timely specialist consultation and diagnostic services Recommendation 5 have on patient care, especially for high-acuity To ensure that vacant in-patient beds are identified, patients, hospitals should track targeted and actual cleaned, and made available on a timely basis to wait times for specialist consultation and diagnostic admitted patients waiting in emergency departments: services for emergency patients, so that the impact of • hospitals should have an effective process in these wait times on providing timely and appropriate place to identify vacant beds and communicate patient care can be periodically assessed. their availability between in-patient units and emergency departments; and Status the Ministry of Health and Long-Term All three hospitals we visited informed us that they • Care should work with the LHINs and with have tracked wait times for specialist consultation hospitals to identify and disseminate best and diagnostic services for emergency patients. practices that enable hospitals to reduce They have also taken actions to reduce wait times. unnecessarily long stays of admitted patients in One of the hospitals has increased its ultrasound emergency departments. hours to serve the needs of emergency patients, completed a study to identify areas for improve- Status 4.05 ment and developed a work plan to improve those All three hospitals we visited have put processes in areas accordingly. Another hospital’s emergency place to ensure that vacant beds are being identified Section department has worked with its diagnostic imaging on a timely basis and to improve communication of department to identify waste and redundancies in bed availability between in-patient units and emer- the process of obtaining diagnostic services. The gency departments. One of the hospitals has been Follow-up

• third hospital has implemented a Model for Special- able to capture the times at which beds are cleaned. 4 ist Consultation to ensure that high-acuity patients Another hospital has a central bed allocation team from its emergency department have prompt to manage an automated system that provides a diagnosis and treatment plans. real-time report to identify vacant beds that are Chapter According to the Ministry, through the Emer- ready to receive patients from the emergency gency Room/National Ambulatory Care Reporting department. The third hospital is in the beginning System (ER/NACRS) Initiative led by Cancer Care phase of implementing a new technology that will Ontario (CCO), emergency departments have been provide real-time direct communication across the collecting data relating to specialist consultation hospital of information including discharges and since April 2011. Monthly reports have been changeover of beds, and the need for cleaning. 354 2012 Annual Report of the Office of the Auditor General of Ontario

The Ministry informed us that the Emergency availability, and regional distribution of Department Process Improvement Program emergency physicians across the province in (ED PIP) and the Pay-for-Results program order to develop a sustainable human resources have focused on reducing unnecessarily long strategy that will ultimately eliminate the use stays of admitted patients in emergency of agency physicians. departments. Specifically: Status The key focus of the ED PIP is to improve • Two of the hospitals we visited indicated that patient flow through engaging housekeepers, overtime has continued to be an issue because nurses and porters to develop a streamlined recruiting and retaining skilled emergency- process and to reduce the overall bed- department nurses have continued to be a challenge. turnaround time (“bed-empty time”). The All three hospitals we visited have implemented Ministry has engaged the Institute for Clinical some measures to reduce overtime costs. Two of the Evaluative Sciences (ICES) to evaluate the hospitals have reviewed staff schedules as well as effectiveness of the ED PIP. According to trends of maternity leaves, sick days, vacations and the preliminary results of the evaluation, overtime to ensure appropriate staffing patterns the hospitals that participated in the ED PIP and to keep ahead of possible vacancies, thereby have improved their emergency-department reducing the need for overtime. Another hospital has wait times. The final report is expected in made reducing overtime a corporate goal, and its March 2013. emergency department has had some success over As part of the Pay-for-Results program, • the past two years in reducing overtime hours as a the Ministry has provided 24 emergency percentage of total worked hours from 6.6% in the departments with dedicated funding of about 2008/09 fiscal year to 5.8% in 2010/11. It is also in $26 million in the 2011/12 fiscal year to the process of finalizing a recruitment and retention create and operate 212 short-stay in-patient strategy for implementation in 2012/13. beds in order to place admitted patients from The Ministry informed us that it has introduced emergency departments quickly in in-patient several initiatives to ensure that an adequate units. The results from May 2012 showed number of physicians are available in emergency that the time patients waited in emergency departments. For example: departments for transfer to in-patient units 4.05 The Ministry offered the Summer Incentive was 21 hours, a reduction of 11% from the • for Designated Emergency Departments 23.6 hours patients waited in May 2011. Section in 2011 to provide additional emergency- physician coverage and to keep emergency STAFFING departments open during the challenging Follow-up

• summer months. Up to $2.1 million was 4 Recommendation 6 provided to help keep 97 emergency depart- To ensure that emergency departments are operating ments open during the summer of 2011. The

Chapter cost-effectively with adequate nurses and physicians: incentive is being continued in 2012. hospitals should deal with chronic overtime by • The Ministry has provided one-time funding setting targets for reducing overtime costs to • of about $2.5 million over the three fiscal acceptable levels and implementing effective years 2011/12 through 2013/14 for a pilot measures for achieving these targets; and program, called the Supplemental Emergency the Ministry of Health and Long-Term Care • Medicine Experience (SEME) program, to should work with the LHINs and with hospitals provide family physicians with three months to conduct studies to assess the requirements, Hospital Emergency Departments 355

of full-time, remunerated continuing educa- Status tion and training in emergency departments. The Ministry informed us that the Pay-for-Results The University of Toronto has implemented program has included a benchmark for ambulance the SEME program to enhance emergency- offload time of 30 minutes 90% of the time. The medicine skills for up to 20 family physicians 74 hospitals that have participated in the Pay-for- a year and to improve coverage in small rural Results program are required to submit ambulance emergency departments. offload data and to demonstrate their progress • The Ministry, in collaboration with the toward meeting the 30-minute target. Ontario College of Family Physicians, has The current challenge, according to the Ministry, offered an Emergency Medicine Primer is the lack of a provincial definition of Ambulance for Family Physicians to reintegrate family Transfer of Care. To address this, the Ministry has physicians into emergency departments. been working with the hospitals, EMS and CCO This course has been offered in locations on data collection by using the current National across Ontario as an update course for family Ambulatory Care Reporting System definition physicians who are currently working in an for Ambulance Transfer of Care as a provincial emergency department and as a refresher definition for both hospitals and EMS. course for family physicians who have worked The Ministry introduced the Offload Nurse in emergency departments before. Program in 2008/09 by providing $4.5 million to 14 municipalities experiencing offload delays, creating 42 offload nurse positions. In the 2010/11 IMPACT OF EMERGENCY-DEPARTMENT fiscal year, the Ministry expanded the program WAIT TIMES ON AMBULANCE to 16 municipalities and increased the funding to EMERGENCY MEDICAL SERVICES (EMS) $9.6 million, providing about 191,000 additional Recommendation 7 nursing hours. In 2011/12, the Ministry further To ensure the efficient use of the ambulance expanded the program to 18 municipalities with Emergency Medical Services (EMS) and to enhance total funding of $11.7 million to provide about co-ordination between EMS providers and emergency 225,400 additional nursing hours. According to departments, the Ministry of Health and Long-Term the evaluation by the Ministry, the municipalities Care should: have reported improvements in ambulance offload 4.05

• determine whether the recommendation in delays, as the Offload Nurse Program has freed up the 2005 expert panel’s report on ambulance six fully staffed ambulances per day to respond to Section

effectiveness of a benchmark ambulance offload new urgent calls in a timely manner. time of 30 minutes 90% of the time should be All three hospitals we visited informed us Follow-up accepted as a province-wide target; that they have been working closely with their •

4 • work with hospitals, EMS providers, and Cancer respective LHINs and other hospitals in monitoring Care Ontario to improve the validity and reli- ambulance offload time. They have also compared

ability of ambulance offload data and to ensure their results with peer hospitals within their Chapter that such data are standardized, consistent, and regions and across the province. As well, they have comparable; and continued to collaborate with the Ministry and the • work with hospitals and EMS providers to evalu- EMS in their regions to evaluate the effectiveness of ate on a province-wide basis the effectiveness of the Offload Nurse Program in reducing ambulance the Offload Nurse Program in reducing offload offload time. delays and improving patient flow within emergency departments. 356 2012 Annual Report of the Office of the Auditor General of Ontario

PERFORMANCE MONITORING • The Ministry issued a directive that as of October 1, 2011, all public hospitals are Recommendation 8 required to report all critical incidents To ensure that emergency departments are provid- related to medication and IV fluids through ing high-quality emergency care to all patients, the National System of Incident Reporting hospitals should: (NSIR) within 30 days following the disclo- promote a culture of patient safety by using a • sure of the critical incident to the Ministry’s non-punitive and “lesson-learned” approach to Medical Advisory Committee, the hospital ensure that adverse events are reported and sum- administrator and/or the patient. marized for analysis and corrective actions; and The 2011 Better Tomorrow for Ontario Act follow up with patients who have been triaged • • has created an exemption under the Freedom as having serious medical conditions but who of Information and Protection of Privacy Act have left emergency departments without being that would give hospitals the discretion to seen by doctors or having completed treatment. refuse to disclose records in cases where Status people providing the information have All three hospitals we visited advised us of actions an expectation of confidentiality. The taken to promote a culture of patient safety. One exemption, effective since January 1, 2012, of the hospitals has a newly implemented Critical encourages hospital staff to engage in full, Incident Process. Another hospital introduced an free and frank discussions on quality-of-care automated incident-reporting system called SAFE issues without fear that their comments will in June 2010 and has a separate review process be publicly revealed. for critical incidents. The third hospital has imple- All three hospitals we visited have procedures in mented a Good Catch Campaign and an online place to deal with patients who have left emergency incident-reporting process. departments without having being seen by doctors To protect and promote a culture of quality or having completed treatment. Two of the hospi- improvement in hospitals, the Ministry also tals have required their nurses or the manager of informed us that the government has made several the emergency department to follow up by phoning legislative changes. For example: those patients. As well, all three hospitals have According to the Excellent Care for All Act, shown improvements, with fewer patients leaving 4.05 • 2010, all hospitals are required to develop their emergency departments without being seen, annual quality improvement plans that as a result of the ED PIP and the Pay-for-Results Section

include critical incident data. Hospital program. One hospital has reduced its percentage boards are required to ensure that hospital of patients who leave without being seen from 4% Follow-up administrators have established a system for to 0.4%; another hospital has reduced the percent- •

4 analyzing critical incidents and developing age from 6.1% to 3%; and the third hospital has plans to avoid or reduce the risk of further reduced the percentage from 5.4% to 0.5%.

Chapter similar incidents. As of January 1, 2011, Regulation 965 of the • ALTERNATIVES TO EMERGENCY- Public Hospitals Act was amended to ensure DEPARTMENT SERVICES that hospital administrators provide aggregated critical incident data to their hospitals’ Recommendation 9 Quality of Care Committees at least two To ensure that the needs of patients are met times per year. appropriately, the Ministry of Health and Long-Term Care should: Hospital Emergency Departments 357

• work with hospitals to conduct further research can search for health-service providers in their on the impact of low-acuity patients on community as an alternative to the hospital emergency services and on what province-wide emergency department. All 20 kiosks were initiatives can be undertaken to encourage implemented by August 2011, and users had people to seek the right treatment from the right made over 220,000 searches as of April 2012. medical provider; and • In the 2011/12 fiscal year, the Ministry • assess and promote the availability and provided $3.5 million as base funding for 42 public awareness of health-care alternatives full-time nursing positions for the Nurse-Led to emergency departments on a regional basis, Outreach Teams (NLOTs), which provide care including walk-in clinics, urgent care centres, directly to long-term-care-home residents family physicians, and other community-based to avoid unnecessary visits to emergency supports, to optimize the right care in the right departments. Supplemental funding of about environment. $1.5 million was provided to support the creation of 30 additional full-time nursing Status positions to augment existing NLOTs in The Ministry informed us that it has continued its selected LHINs. efforts to increase public awareness of alternative At the time of our follow-up, ministry data services to emergency departments in order to showed that these efforts had reduced the encourage people to seek the right treatment from volume of low-acuity patients visiting emergency the right health-service provider. For example: departments as a percentage of total emergency- In November 2010, the Ministry re-launched • department visits from 45% in 2009/10 to 41% in the Your Health Care Options website, which 2011/12. includes an online registry for Health Care Apart from the Ministry’s efforts, all three hos- Connect to help any Ontarian who is without pitals we visited have also taken actions. One of the a family doctor to find one, and searchable hospitals informed us that it opened an urgent care tools to obtain information on wait times centre in April 2011 and has worked with its public for surgeries and diagnostic services. From relations staff to educate the community regarding May 2011 to August 2011, the Ministry the appropriate use of the emergency department launched a campaign for the Your Health Care and the urgent care centre. Two of the hospitals 4.05 Options website through television, radio, have worked with their stakeholders, including online advertising and materials distributed to LHINs, peer hospitals and long-term-care homes, Section health-care providers. to promote the use of alternative services such as yIn Ma 2011, the Ministry deployed 20 kiosks • out-patient clinics in order to reduce emergency- at selected emergency departments to provide Follow-up department visits and hospital admissions. • users with access to the Your Health Care 4 Options website and a directory where they Chapter Chapter 4 Ministry of Training, Colleges and Universities Section 4.06 Infrastructure Asset Management at Colleges

Follow-up to VFM Section 3.06, 2010 Annual Report

Our 2010 audit focused on the adequacy of Background endeavours by the Ministry and selected colleges to maintain college assets in a good state of repair. Along with our work at the Ministry, we visited Over the decade from the 2000/01 to 2009/10 the following colleges: Algonquin and La Cité in fiscal years, the Ministry of Training, Colleges and Ottawa; Confederation in Thunder Bay; and George Universities (Ministry) provided Ontario’s 24 col- Brown and Humber in Toronto. We also contacted leges of applied arts and technology with facility six other colleges to obtain their input on specific renewal funding totalling $13.3 million a year, issues and met with various stakeholders, includ- supplemented by periodic additional allocations ing Colleges Ontario and the Council of Ontario for renewals totalling $270 million. The combined Universities. annual and additional funding amounted to Our work indicated that although colleges have $403 million over the 10-year period. benefitted from the new-facility capital funding, In addition to funding facility maintenance, the ongoing funding for maintenance of existing facili- Ministry also provides capital grants to enhance 4.06 ties had not been sufficient to maintain the aging and expand the colleges’ capital needs. In recent college infrastructure, and the backlog of deferred years, the Ministry has provided this funding Section maintenance was increasing. Some of our more primarily for new facilities space so that colleges significant observations were: can accept more students. In 2009, the federal The Ministry was in the process of imple- Follow-up government initiated the Knowledge Infrastructure • •

menting a long-term capital planning process 4 Program (KIP), a two-year infrastructure program but did not have a formal plan in place at the for Canadian colleges and universities. time of our audit for overseeing the colleges’

Chapter The federal and provincial governments together investment in infrastructure. provided capital grants to colleges for new facili- Many colleges had not maintained adequate ties totalling $300.5 million between the 2006/07 • asset management systems to facilitate and 2009/10 fiscal years to create local short-term effective capital planning and performance employment and increase the number of students reporting on the condition and use of their that colleges could accommodate. Facilities space capital infrastructure. includes classrooms, laboratories, cafeterias, offices, libraries, and other such required areas.

358 Infrastructure Asset Management at Colleges 359

• As of April 2010, the deferred maintenance backlog—the cost to perform all needed main- Status of Actions Taken on tenance and repairs—exceeded $500 million Recommendations and had been increasing annually. More than $70 million in capital repairs were in the According to information provided by the Ministry critical category and needed to be dealt with and the five colleges we visited as part of our 2010 in the next year. audit, some progress has been made on imple- As of April 2010, about half of the college • menting the recommendations from our 2010 system’s infrastructure assets were likely in Annual Report, with substantial progress on a few. poor condition when rated according to a rec- The Ministry informed us that it has improved its ognized industry standard that measures the capital-project selection process and is continuing state of infrastructure. to develop its capital planning process for colleges. Applying the funding guideline of 1.5% to • In addition, as a prerequisite for infrastructure 2.5% of asset replacement cost outlined by the funding, every college will be required to develop U.S.-based Association of Higher Education an asset management plan that will outline the Facilities Officers, annual ministry funding condition of its existing assets and the institution’s to all colleges over the 2006/07–2009/10 plan for addressing its renewal needs. To this fiscal years would have needed to be in the end, the Ministry has initiated projects to create $80 million to $135 million range. However, a comprehensive college facility space inventory actual capital renewal funding has remained and establish utilization standards, and complete a at $13.3 million annually for many years; system-wide assessment of the condition of college even with the periodic additional funding of facilities across the province. Establishing space $270 million, the total barely adds up to half standards and assessing the condition of facilities of this guideline amount. are key first steps in developing a sustainable long- Administrators at all of the colleges we visited • term asset management plan, including priorities indicated they had to supplement ministry for addressing deferred maintenance. As this work renewal funds with operating funds to help is expected to be completed in two years, deferred address their most urgent priorities or run the maintenance will remain a significant issue in the risk of assets deteriorating prematurely. 4.06 foreseeable future. • Until very recently, ministry funding decisions often lacked transparency and consistent Section

criteria to evaluate funding requests, and CAPITAL PLANNING there was insufficient documentation to dem- Recommendation 1 Follow-up onstrate compliance with eligibility criteria. • To help ensure that capital infrastructure grants are 4 We made a number of recommendations for allocated on the basis of clearly identified needs and improvement and received commitments from province-wide priorities, the Ministry of Training, the Ministry and the colleges that they would take Chapter Colleges and Universities needs to continue developing action to address our concerns. a formal long-term capital planning process using current and reliable information obtained from the colleges and make funding decisions based on more predictable, rigorous, and clear criteria. 360 2012 Annual Report of the Office of the Auditor General of Ontario

Status • introduce a satellite campus policy to help The Ministry informed us that it is continuing manage growth in the system and give prior- to develop its capital planning process, and in ity to areas where rapid growth is expected; May 2010 sent out a memo to all colleges asking • develop a more comprehensive funding them to identify their capital planning priorities. policy for major capital projects, including The memo requested that each college provide procurement approaches and a framework an update of its inventory of capital projects and for determining the appropriate provincial major strategic infrastructure initiatives, includ- share; and ing asset management strategies and a ranking of • emphasize asset management planning and project proposals. Ministry priorities for projects facilities renewal and repurposing. included criteria such as the capacity for growth As a prerequisite for infrastructure funding, in high-demand areas, a contribution to long-term every college and university will be required to economic development, and demonstrated linkages develop an asset management plan that outlines with provincial priorities in post-secondary educa- the condition of its existing assets and its plan for tion, including growth in the college sector and the addressing the renewal needs identified. When provision of quality education. seeking funding for expansion, institutions will The Ministry has developed a project-selection need to provide a clear rationale to opt for new framework to assess all project proposals received construction instead of renewing or repurposing from the colleges with an approach focused on existing space. assessing the alignment of the proposals with its In collaboration with the Ministry of Infrastruc- own priorities and those of the Ministry of Infra- ture, the Ministry intends to ensure that institutions structure. In June 2011, the Ministry selected 10 comply with the new requirements set out in the capital projects at colleges and committed funding provincial 10-year infrastructure plan, including of $245 million with an estimated incremental developing and publishing detailed institutional enrolment of more than 9,200 students. asset management plans as a prerequisite to receiv- In May 2011, the government announced “Put- ing provincial capital funding. The Ministry has ting Students First: Ontario’s Plan for Postsecond- also initiated projects to create a comprehensive ary Education.” The plan is designed to support college facility space inventory, develop space stan- enrolment growth by providing space and operat- dards and assess the condition of facilities across 4.06

ing funding for 60,000 more students at colleges the province. and universities over the next five years. The plan The colleges informed us that the Ministry had Section

also emphasizes strategic and well-planned growth asked them to submit updates to their proposed cap- in the post-secondary education sector. ital projects in 2008 and 2010, but there had been Follow-up In June 2011, the Ministry of Infrastructure no further requests since then. One college noted •

4 released Ontario’s long-term infrastructure plan, that these requests represent an excellent founda- Building Together: Jobs and Prosperity for Ontarians, tion for a possible province-wide capital planning

Chapter which outlines the government’s plans for the next process. However, the same college also expressed 10 years. The plan notes that Ontario will work with concern that the process needed to be ongoing colleges and other post-secondary institutions to: in order to provide current and reliable planning • ensure that infrastructure investments information. Another college noted that the Min- respond to demand, align with the aims of istry promised to continue consulting with colleges “Putting Students First” and support the regarding asset management planning and that it goal of a 70% attainment rate for post- subsequently had a number of meetings with min- secondary education; istry staff regarding growth and facilities planning. Infrastructure Asset Management at Colleges 361

FACILITIES RENEWAL AND are key first steps to developing a sustainable long- MAINTENANCE term asset management plan. Ontario colleges currently maintain their space Recommendation 2 inventory data in a variety of formats. Further- To preserve the taxpayer’s investment in the college more, there is no common definition for different infrastructure and maintain these assets in good categories of college student space, nor are there condition so that colleges can provide an adequate standards for the amount of space required for each learning environment, the Ministry of Training, Col- category. To address this issue, the Ministry and leges and Universities should continue to work with Colleges Ontario (the advocacy organization for the Ontario colleges to: province’s colleges of applied arts and technology) ensure that the asset management information • signed a transfer-payment agreement in October system is regularly and consistently maintained 2011 to work on the development of system-wide to enable both the Ministry and colleges to make space standards and utilization benchmarks, and to informed decisions based on current, accurate, complete a system-wide space inventory. and complete information; and In November 2011, Colleges Ontario engaged develop strategies, targets, and timelines to • a consultant to lead the work to develop a space address the deferred maintenance backlog. inventory, utilization and planning framework for Status Ontario’s 24 colleges. Key deliverables stipulated in The Ministry’s facilities renewal program is the agreement include completion of a system-wide intended to assist post-secondary institutions in space inventory by September 2012; implementa- addressing ongoing maintenance, repairs and tion of a common space inventory database; and renovations of existing facilities. Recurring annual development of standards, benchmarks and a facilities renewal funding had been $13.3 million reporting system for space utilization. On Febru- over the previous 10 years and was provided ary 27, 2012, the Ministry received a progress to supplement the colleges’ own annual capital report on the work accomplished to date, which maintenance programs. Ministry funding for largely focused on the development of a common capital maintenance decreased in 2011/12 and space inventory database. At the time of our follow- is not expected to increase, as we were informed up, the Ministry was expecting another progress that funding for facilities renewal was budgeted at report on the space inventory and database by 4.06

$8.7 million for the 2012/13 fiscal year. Estimates September 15, 2012. Colleges Ontario was planning provided by the colleges indicate that components to deliver its final report by January 31, 2013. Section of the 10 approved major capital projects (totalling In April 2012, the Ministry received approval $245 million) are projected to contribute an addi- to procure a vendor to refine and standardize the Follow-up tional $14 million to address the colleges’ current existing facility condition assessment systems in •

4 deferred maintenance backlog. use by Ontario’s colleges. To ensure that the results The Ministry and colleges are working to of the procurement meet the needs of all decision-

develop an inventory of college spaces that support makers, a working group with representatives Chapter student enrolment, as well as to develop related from the Ministry, the colleges, the Ministry of standards, consistently assess facility conditions Infrastructure and Colleges Ontario was set up to across the college sector and determine the appro- provide advice. The project is expected to ensure priate priorities for addressing deferred mainten- consistent standards in key areas such as replace- ance given available funding. Establishing space ment values and unit costs, and to undertake standards and assessing the condition of facilities system-wide facility condition assessments with related data updates. The project will be funded by 362 2012 Annual Report of the Office of the Auditor General of Ontario

the Ministry and is expected to be completed within • maintain adequate documentation throughout two years. At that time, consideration will be given the process to demonstrate that the program is to initiating a cycle of updates to the assessment for transparent, fair, and achieves value for money, up to 20% of facilities per year. as well as college and ministry objectives. Establishing space standards and assessing Status the condition of facilities provide the information In May 2010, the Ministry requested that colleges needed to make good investment decisions. Once identify their capital planning priorities and pro- better information on the demand for space and vide an update to their inventory of capital projects the condition of existing space is available, updated and major strategic infrastructure initiatives, estimates on the funding required to meet identi- including institutions’ asset management strat- fied needs can be prepared. egies and a ranking of their project proposals. To Several of the colleges advised us that they were assess all project proposals received, the Ministry actively participating in the Ministry’s assessment developed a project selection framework with an of space utilization and/or facility condition. One approach focused on assessing the alignment of college noted that consistently updating the space the proposals with overall provincial infrastructure inventory and utilization data for all colleges was priorities and initiatives, as well as their fit with an excellent step toward effective asset manage- the strategic priorities of Ontario’s post-secondary ment, and it anticipated that all colleges would be education sector. up to date by the end of March 2013. The college Project assessment criteria included the esti- also noted that, since no system-wide strategies mated total and provincial cost (including cost per appeared to be emerging and there was no new each new student enrolled); the type of project money and even less facilities renewal funding, it (for example, renewal, expansion); the impact on is continuing to develop its own strategies to cope capacity growth in underserved regions; align- with its deferred maintenance backlog. This often ment with key government initiatives, such as the results in addressing only the most urgent needs. province’s growth plans; and links to key sectors outlined in Ontario’s innovation agenda (such MAJOR CAPITAL PROJECT MANAGEMENT as health services, science and technology, and skilled trades).

4.06 Recommendation 3 These project assessment criteria were included To help ensure that new construction and major in the evaluation templates that ministry staff com- renovations efficiently and cost-effectively achieve Section pleted in order to evaluate all the project proposals both college capacity goals and ministry economic received from the institutions. After the initial objectives, the Ministry of Training, Colleges and project assessment, ministry staff met with repre- Follow-up Universities should: •

sentatives from the colleges to review their project 4 implement fair and transparent procedures, • proposals and their alignment with institutional similar to those developed for the Knowledge priorities and ministry capital planning objectives.

Chapter Infrastructure Program, for its project proposal, Following these meetings, the Ministry identified evaluation, and selection process; 10 capital projects for funding. enter into an agreement with each college to • The Ministry entered into transfer payment indicate the Ministry’s and college’s respective agreements with all institutions whose infra- responsibilities for completing the project and structure projects were selected, with the excep- the necessary reporting requirements; tion of three large and complex infrastructure advance funds to colleges as the work pro- • projects approved for Alternative Financing and gresses; and Infrastructure Asset Management at Colleges 363

Procurement (AFP), a construction delivery ibility, the Ministry has not issued a request to col- model developed and managed by Infrastructure leges for updated or new proposals since 2010. Ontario. The Ministry will enter into transfer pay- With respect to submissions, project selection, ment agreements for the AFP projects once the funding agreements, cash flow and monitoring, projects are at the appropriate stage. No funding one of the colleges responded that there has been will be flowed to these three institutions until a noticeable increase in rigour in the requirements agreements are in place. The Ministry’s transfer related to the documentation and reporting for its payment agreements stipulate that each institution funded capital projects. The college also noted that is required to comply with the Procurement Direc- these projects still necessitated a significant amount tive that came into effect on April 1, 2011, under of effort to solicit support for approvals, and it the Broader Public Sector Accountability Act, 2010. would like to see a process that would help provide The agreements were reviewed and approved by more predictable funding decisions. the Ministry of the Attorney General to ensure they protect, to an appropriate degree, the interests of MEASURING AND REPORTING ON the province. PROGRAM EFFECTIVENESS The Ministry’s project oversight includes monitoring progress on the projects and ensuring Recommendation 4 compliance with the government’s Transfer Pay- To help ensure that all stakeholders have a good ment Accountability Directive and with the Cash understanding of the condition of the province’s college Management Directive. The latter directive requires infrastructure assets, the Ministry of Training, Colleges payments be made at or close to the time money is and Universities and the colleges should continue to needed by the recipient. As part of the process, in develop and report long-term performance indicators addition to the pre-existing annual and close-out on the management and condition of their facilities. reporting requirements, the Ministry has imple- Status mented a quarterly reporting cycle that captures The Ministry is engaged in ongoing discussions expenditure reporting and provides details that with the sector on the data requirements for the enable the Ministry to monitor project progress asset management framework, including defining and provide transfer payments as the construction the most suitable asset management performance phase progresses. 4.06 indicators. The outcomes from the space standards The Ministry also introduced a new requirement and facility condition assessment projects, to be for institutions to provide a cost consultant’s report Section completed over the next two years, will guide the reviewing the projected costs of the projects. In development of performance indicators. addition to the pre-existing requirement that each In December 2011, the Ministry established Follow-up institution provide a copy of the Certificate of Sub- •

the Deferred Maintenance Working Group with 4 stantial Performance in accordance with subsection representation from the Ministry of Infrastructure, 2(1) of the Construction Lien Act upon completion Colleges Ontario and various colleges. The Work-

of the project, colleges are now required to provide Chapter ing Group had a general discussion of the capital- a commissioning agent’s report to confirm that the related decision-making processes of colleges, building systems work as they are intended to. the Ministry and the Ministry of Infrastructure. The Ministry intends to refine its existing project The aim was to determine what sorts of data were selection and project oversight to reflect the long- required to improve decision-making at all levels. A term capital planning objectives and provincial second meeting focused on the facilities condition fiscal realities. Given limited capital funding flex- assessments, specific pieces of data that would 364 2012 Annual Report of the Office of the Auditor General of Ontario

need to be collected and the mechanics of collecting such data. Before the facilities condition assessments begin, a meeting will be held with the working group and the consultant engaged to refine and standardize the existing facilities condition assessment systems in use by Ontario’s colleges. Data requirements will be finalized at this meeting and performance indicators discussed. The practical experience of the successful consultant will be combined with the reporting needs of colleges and the provincial government to arrive at a final suite of indicators. While the working group is currently focused on implementing the facility condition assessment project, the focus will shift to how best to use the resulting data when the project is complete. Several colleges indicated that the two major initiatives in progress, the space standards and facility condition assessment projects, will contribute to the development of performance indicators and reporting mechanisms to ensure consistent and comparable reporting for both the condition and utilization of college assets. 4.06

Section

Follow-up

•

Chapter on three of these programs, which accounted for which programs, ofthese on three levelsprovince three ofgovernment. by the spendingacross the $8billionininfrastructure than would programs leadto fulltake-up, more the with federal government. the Itwas estimatedwith that province by the itselfandfunded50/50 undertaken dollar.another Aswell, anumberofprojects were and provincial governments would commit each committed to anapproved federal project, the First Nations andnot-for-profit organizations— eligiblerecipients—municipalities, the dollar that completed by31, March 2011. besubstantially and hadrequirements they that two have those years, beenbuiltwithin otherwise targeted would projects that not construction-ready 2009and2010.jobs throughout Theseprograms projects andcreate infrastructure to support grams global economiccrisis. the of to economyeffects stimulate the andcombat to businessesandindividualsasmeasures grants investments,included infrastructure relief and tax EconomicAction Plan,which announced the federal government 2009,the In January Background Chapter 4 Chapter In 2010, we focused conducted anauditthat were forThese programs every designedsothat pro- The Planalsoincludedseveral short-term 4.07 Section Follow-up to VFMSection3.07, to Follow-up Spending StimulusInfrastructure Ministry ofInfrastructure Ministry provincial short-term infrastructure commitment: infrastructure provincial short-term about $3.9 CC, while the Ministry ofTourism Ministry CC, whilethe andCulture Food administered andRural ISFandBCF- Affairs On aday-to-day ofAgriculture, Ministry basis, the oversight andnegotiating fundingarrangements. responsible for leadministry MEIwas the grams. pro- ment were three ofthe responsible for delivery federal govern andthe - Energy andInfrastructure) ture of (MEI—now ministries dividedinto the provincial cost-shared projects. been committed to approximately 2,300federal– $3.9 31,jects. AsofMarch 2010, about$3.1 as the deadlinefor completion substantial as the ofpro- set 31, March 2009,they inspring programs 2011, two Whenthe governmentsstructure. unveiled the plants, whileRINC helpedbuildrecreational- infra buildings andwater andwastewater processing asmunicipal facilitiessuch alongwith trails, and ofroads, bridges,parks construction ported • • • The ISF and BCF-CC programs primarily sup primarily The ISFandBCF-CC programs - ofEnergy andInfrastruc Ministry The Ontario

astructure Stimulusastructure Fund (ISF); ecreational Infrastructure Canada Program CanadaProgram ecreational Infrastructure billion available under the three programs had programs three billion available underthe ponent Top-Up (BCF-CC); and (RINC). Recreational Program andOntario in Ontario Infr R Building CanadaF

billion of the total $6.9billionfederal–billion ofthe 2010 Annual Report Annual 2010 und–Communities Com

billion of the billion ofthe - - 365

Chapter 4 • Follow-up Section 4.07 366 2012 Annual Report of the Office of the Auditor General of Ontario

(now the Ministry of Tourism, Culture and Sport) been possible to conduct the necessary due administered RINC. diligence within such a tight time frame. In our 2010 Annual Report, we noted that in • Applicants were not required to prioritize order to ensure that stimulus funds would be their infrastructure needs, and none did in injected into the economy to create jobs as quickly their applications, making it more difficult to as possible, the three programs were to give prior- assess the benefits of the proposed projects ity to construction-ready projects of demonstrable so that informed funding decisions could be benefit to their communities that could be substan- made. One municipality submitted 150 appli- tially completed within two years. Priority was also cations valued at $408 million, and received to be given to those who planned to spend 50% or approvals for 15 projects worth $194 million. more of the funds by March 31, 2010, the end of From our visit to this municipality, we noted the programs’ first year. However, as of March 31, that 11 of the approved projects, valued at 2010, the end of the first year of the two-year pro- $121.7 million, were ranked at or near the grams, less than $510 million, or only about 16%, bottom of the municipality’s own priority list, of the total $3.1 billion committed by the federal while other, higher-ranked eligible projects and Ontario governments had actually been spent. were not approved. According to the job-creation model used by MEI, • We noted that technical experts were gener- the three programs we examined would create and ally not involved in assessing the applications preserve about 44,000 jobs, but given the lower even though thorough analysis by such level of actual spending during the first year of the experts would have helped assess the reason- programs, only about 7,000 jobs were estimated to ableness of project cost estimates and identify have been created or preserved during the first year those unlikely to meet the two-year comple- of the two-year program. tion deadline. With respect to the grant-application and After assessment and review by civil servants application-assessment processes, we noted that: in the appropriate ministries, the applications • MEI placed no limit on the number of applica- were submitted to the office of Ontario’s Minister tions that municipalities with populations of Energy and Infrastructure and to his federal of more than 100,000 could submit under counterpart for final review and approval. With ISF, the largest of the three infrastructure respect to this process, we noted that there was a 4.07

programs. This provided an incentive to general lack of documentation to support the deci- submit large numbers of applications in hopes sions regarding which projects were approved and Section

of getting as many approved as possible. For which were not. In some cases, ministers’ offices example, four municipalities submitted a total approved projects that civil servants had earlier Follow-up of almost 1,100 applications, accounting for deemed ineligible or about which they had flagged •

4 40% of the total applications submitted by the concerns. 421 Ontario municipalities for this program. Federal and provincial funding was to end on

Chapter • Due to the tight deadlines, the time allotted March 31, 2011, after two years. As only 16% of for the provincial review of ISF applications the committed funds had been spent after the first was in most cases just one to two days. In one year, many recipients had difficulty completing their instance, we noted that a key component of projects by this deadline. Our survey indicated that the provincial review for 56 projects worth as of May 2010, more than one-third of respondents an estimated $585 million was carried out in had to take such steps as adjusting project specifica- just four hours. In our view, it would not have tions and cost estimates in the original applications, Infrastructure Stimulus Spending 367 paying contractors overtime and sole-sourcing some 2009 programs have been substantially imple- contracts to meet the deadline. mented, in part because of the federal and provin- We made a number of recommendations for cial governments’ extension of the funding deadline improvement and received commitments from the for qualifying projects to October 31, 2011, and the Ministry of Infrastructure (Ministry) that it would further provincial extension for qualifying projects take action to address our concerns. to March 31, 2012. In December 2010 (subsequent to our audit), The Ministry agreed to take our recommen- both the federal and provincial governments dations pertaining to any future infrastructure announced a one-time extension of the March 31, programs into consideration when designing 2011, deadline until October 31, 2011, to provide capital-grant programs. The status of the action one more construction season for projects not likely taken on each of our recommendations at the time to meet the original deadline. In order to qualify of our follow-up was as follows. for this new deadline, projects had to have incurred eligible costs before March 31, 2011, and were PROGRAM ELIGIBILITY AND required to submit an application for the extension. APPLICATION ASSESSMENT According to the Ministry, nearly 800 projects were approved for this extension. In November 2011 and Program Design and the Submission of February 2012, the province exempted 21 projects Applications from the extended deadline and continued to reim- Recommendation 1 burse its share of eligible costs incurred between To help ensure that projects best suited to meeting pro- November 1, 2011, and March 31, 2012. gram objectives are funded in any future infrastructure programs, the Ministry of Infrastructure should: STANDING COMMITTEE ON PUBLIC • follow a more risk-based approach to design- ACCOUNTS ing and implementing future capital-grant programs and consider all important factors The Standing Committee on Public Accounts held a affecting program delivery, including project hearing on this audit in March 2011. In May 2011, suitability, reasonableness of timelines, and the the Committee reported to the Legislature on this capacity of and demand on ministry resources; hearing. It stated in a letter that it was satisfied with 4.07 require that applicants better demonstrate the the Ministry’s responses to its questions and with • benefits of their proposed projects, provide evi- the actions taken to date to address the Auditor Section dence that the expected benefits are achievable, General’s recommendations. The Committee did and prioritize their applications; and not make any recommendations of its own. strengthen its due-diligence process and include Follow-up

• • the use of technical experts to review high-risk 4 projects, in assessing grant applications.

Status of Actions Taken on Status Chapter Recommendations Since our 2010 audit, the province has not initiated similar infrastructure programs. However, the Min- istry indicated that it is committed to taking this The Ministry provided us with information in recommendation into consideration when design- spring 2012 on the current status of the actions ing any future capital-grant programs. Specifically, taken on our recommendations. According to this the Ministry informed us that it will expand the use information, recommendations applicable to the of risk-based program design and analysis, assess 368 2012 Annual Report of the Office of the Auditor General of Ontario

the implications of program-design decisions on infrastructure stimulus funding were required to ministry resources, and work to incorporate tech- apply for the October 31, 2011, extension and to nical expertise in the due-diligence process. The submit supporting documentation including a Ministry also committed to seeking the advice of detailed construction schedule signed by a profes- the province’s Internal Audit Division early in the sional engineer or architect. These applications development of future programs. were reviewed by the Ministry, the partner minis- For future infrastructure programs that involve tries and the federal government to assess whether partner ministries, the Ministry committed to each extension was compliant with the program providing better guidance to all staff responsible criteria. The review process allowed for follow-up for reviewing applications, to strengthen the due- with applicants to seek clarification on issues and diligence process for application assessments. The concerns identified in their applications before the Ministry also indicated that it will ensure that future decision on each extension was made. The Ministry application processes place a greater onus on appli- maintained documentation to record whether cants to demonstrate that the proposal and claims in each application for the extension was complete. their applications are valid and achievable and that In the case of incomplete applications, the reasons they address a priority need for the community. for their incomplete status were recorded so that the rationale for any denial of an extension was documented. Federal–provincial program manage- Approvals of Applications ment committees provided recommendations on Recommendation 2 extension approvals to the federal and provincial To ensure a fair and transparent project selection ministers, who had the final authority to approve process is followed for any similar programs in future, extensions. the Ministry of Infrastructure should: To improve the transparency of projects funded • address all significant concerns raised during by infrastructure stimulus spending, in December initial assessment and satisfactorily follow up 2010 the Ministry launched an enhanced version and resolve them before approving the projects; of the Revitalizing Ontario’s Infrastructure website • strengthen documentation of the rationale for that contains additional information on the pro- decisions reached throughout all stages of the jects, such as project descriptions and maps that grant-assessment and approval processes; and better indicate the location of projects. In 2011, 4.07

• consider whether providing additional infor- further enhancements were made to the website, mation would enhance transparency and which included improved project filtering and Section

be of interest to the general public and the the addition of web pages showing the benefits Legislature. provided by the province’s infrastructure projects. Follow-up The Ministry indicated that the website continues • Status 4 to be updated with information on the construction Similar to the previous recommendation, this rec- status of projects funded by stimulus spending, ommendation is also aimed at the administration

Chapter given that for some projects, payments for costs of future capital-grant programs. However, after incurred up to March 31, 2012, will continue well the March 31, 2011, deadline for the completion into the 2012/13 fiscal year. of projects under the 2009 infrastructure stimulus programs was extended to October 31, 2011, the Ministry made some efforts to implement the recommendation in the administration of the extended projects. For instance, recipients of the Infrastructure Stimulus Spending 369

PROJECT MANAGEMENT by the March 31, 2011, cut-off date for federal funding. Close to 800 projects were approved for Reporting and Monitoring the Progress of the extension. Subsequent to the extension by Projects the federal government, the Ontario government Recommendation 3 further exempted 21 projects from the extended To help ensure that funded projects are completed deadline and continued to reimburse its share on time and on budget, and to comply with funding of eligible costs incurred to March 31, 2012. The agreements, the Ministry of Infrastructure should: Ministry indicated that, as a result of these federal • ensure that recipients report project informa- and provincial measures, only 19 projects had not tion consistently and on a timely basis, and fol- been completed by March 31, 2012. According to low up on projects at risk of missing the funding the Ministry, most of these had incurred sufficient cut-off deadline; and eligible costs prior to March 31, 2012, to have the • consider raising the issue with the federal gov- full provincial contribution paid out. ernment once reliable data is available on the number and extent of projects that will not be Costs to Meet the Funding Deadline completed by the March 31, 2011, federal funding cut-off. Recommendation 4 To help ensure that funds are spent wisely, the min- Status istries of Infrastructure, Tourism and Culture, and In our 2010 Annual Report, we noted delays in Agriculture, Food and Rural Affairs should work reporting the status of 17% of the ISF municipal with any recipients experiencing significant delays on projects and 40% of the RINC projects. The status their projects to evaluate the options and solutions of the RINC projects, in particular, was difficult to best suited to meet stimulus-program objectives and determine reliably due to issues with RINC’s infor- ensure value for money in completing the projects. mation system. In June 2010, subsequent to our audit fieldwork, we were informed that progress Status reports had been submitted for 95% of all projects. The Ministry indicated that following our 2010 However, we noted a variety of interpretations by audit, partner ministries engaged on an ongoing recipients as to what constitutes “progress” on a basis with recipients of stimulus spending, includ- 4.07 project. Some defined it as estimated work done, ing conducting site visits to assess and validate while others used actual dollars spent and engin- project progress. The partner ministries regularly Section eering assessments. The analysis we did in 2010 reported issues they noted to the Ministry and the of progress based on actual spending by recipients federal government. They paid special attention revealed that only 16% of the committed funds had to projects identified as delayed or otherwise at Follow-up

• been spent. This lagged significantly behind the risk of not being completed by the new October 31, 4 province’s initial expectation of equal spending in 2011, deadline or the March 31, 2012, provincial each of the program’s two years and left a signifi- deadline. The provincial ministries and the federal cant amount of the funds originally committed still government collaborated to assess options and Chapter to be spent in the 2010/11 fiscal year. identify solutions for projects that needed support. As noted earlier, in December 2010, the federal This included working with recipients of stimulus government announced that it was extending the funds to identify options such as changing the scope funding deadline for qualifying projects funded of their projects so that they could be completed by by stimulus spending to October 31, 2011, since the funding deadline. these projects were at risk of not being completed 370 2012 Annual Report of the Office of the Auditor General of Ontario

The Ministry also indicated that, ultimately, smaller NPOs using stimulus funding, the Ministry funding recipients were accountable under the did indicate that a small sample of the statements binding stimulus funding agreement for the proof expenditures submitted by the NPOs was curement, management and delivery of their own reviewed by an accounting firm to assess whether projects by the funding deadline. Follow-up letters the statements were prepared in accordance with approved by the Ministry, the partner ministries the terms of the funding agreement. The Ministry and the federal government were issued when war- also indicated that it has been working with the ranted to ensure that recipients were aware of the Ontario Internal Audit Division (OIAD) to ensure current situation related to their projects and the that project and program close-out protocols are potential consequences that could arise if they did providing appropriate monitoring and risk mitiga- not take the required action. tion to achieve successful close-out of projects. As a result of the above actions by the Ministry, The Ministry noted that the binding funding the partner ministries and the federal government, agreements with recipients give the province the the number of projects considered at risk of not right to audit projects to determine whether funds meeting the funding deadline steadily declined, have been spent wisely. The province’s right to and the vast majority of them were completed by audit extends beyond the completion of the project. March 31, 2012. The Ministry indicated that the government will continue to use the risk-assessment model to determine if projects warrant an audit upon completion. Financial and Claims Administration To this end, following up on a recommendation pro- Recommendation 5 vided by the OIAD, the Ministry issued an Assistant To ensure that funds are spent wisely and for the pur- Deputy Minister–level memo to partner ministries pose intended, the Ministry of Infrastructure should on April 3, 2012, about these ministries’ ongoing work with the Internal Audit Division to develop accountability responsibilities as they relate to the appropriate monitoring and audit coverage of fund stimulus programs. One such responsibility is to recipients according to assessed risk. retain proper and accurate project and program records for six years following March 31, 2012, Status to support any audits that may be undertaken. In our 2010 Annual Report, this recommendation The Ministry stated that it will continue to seek 4.07 was predominantly aimed at the better monitoring advice from the OIAD as the government continues of about 150 not-for-profit organizations (NPOs) to wind down the stimulus programs and in the Section that at the time of our audit had received approxi- design and implementation of future programs. mately $360 million in stimulus funding. Since these NPOs were typically smaller organizations Follow-up

• that ranged from multi-purpose community organ- MEASURING AND REPORTING ON 4 izations and recreational centres to special-purpose PROGRAM EFFECTIVENESS organizations like sports clubs, they might lack the

Chapter Recommendation 6 project-management expertise and accountability To better enable the public and legislators to evaluate structures of larger organizations. Furthermore, the the effectiveness of these stimulus programs, the Min- provincial ministries involved with this program istry of Infrastructure should: had no previous experience in dealing with these provide timely and accurate information on the smaller organizations. • progress of these projects; and Although since our 2010 audit it has not specific- ensure that the methodology used to calculate ally audited any of the projects procured by these • the impact of stimulus funds on employment is Infrastructure Stimulus Spending 371

adjusted as needed to reflect the actual flow of funds into the economy and the impact on the job market.

Status As noted earlier, to provide more timely and accurate information on the progress of projects funded by stimulus spending, shortly after our 2010 audit the Ministry added new features and additional information on the stimulus projects on its Revital- izing Ontario’s Infrastructure website. The Ministry also indicated that it uses job multipliers provided by the Ministry of Finance (MOF), which MOF annually updates on the basis of the latest data from Statistics Canada. These updated multipliers, according to the Ministry, help ensure that the most current information about the structure of the Ontario economy is used in job calculations. Lastly, the Ministry stated that in view of the winding down of the stimulus programs, it continues to work with its partner ministries and the federal government to identify the lessons learned from the stimulus programs so that future programs can be made more effective and accountable. 4.07

Section

Follow-up

•

Chapter Chapter 4 Section 4.08 Municipal Property Assessment Corporation

Follow-up to VFM Section 3.08, 2010 Annual Report

Board, the independent tribunal that hears appeals Background from people who believe their properties have been incorrectly assessed or classified. To get an indication of whether the Corpora- The determination of the market value of a prop- tion’s mass-appraisal system achieved this object- erty is critical because it ultimately determines how ive, in our 2010 audit we compared the sale prices much property tax an owner must pay. In Ontario, of 11,500 properties identified as having been sold this tax is calculated by multiplying a property’s at arm’s length in 2007 and 2008 to their assessed assessed market value by the tax rate of the prop- market value as of January 1, 2008. We found erty’s municipality. that in 1,400 of these cases, or one in eight, the On December 31, 1998, the province transferred assessed value differed from the sale price by more the responsibility for determining the assessed than 20%. In many cases, the difference between value for properties to the Ontario Property Assess- assessed market value and actual selling price ment Corporation, later renamed the Municipal was substantial. Property Assessment Corporation (Corporation). 4.08 At the time of our audit, the Corporation The primary responsibility of the Corporation’s acknowledged that some individual property 1,600 employees is to prepare an annual assess- Section assessments might not have reflected the current ment roll for each local municipality that identifies or fair-market property-value range as indicated by each property and its assessed market value. In an arm’s-length sale price. These variations most Follow-up the 2011 calendar year, Corporation expenditures •

often occurred because it did not have up-to-date 4 totalled $190.4 million ($185.5 million in 2009), property data from a property inspection. As a most of which was funded by the province’s 444 result, some property owners may have been over-

Chapter municipalities. or under-assessed, and therefore paid more or less From the perspective of a property owner, it than their fair share. While the Corporation did get is reasonable to expect that each property will be it right for the vast majority of properties, this was assessed within a range that is reasonably close of little solace to property owners who were over- to its fair market value—the most likely sale price assessed relative to neighbouring properties, and between a willing buyer and seller. At the time of therefore paid more than their fair share of tax. our 2010 audit, this was also the position of the More frequent property inspections and timely Corporation and Ontario’s Assessment Review investigations of sales values that vary significantly

372 Municipal Property Assessment Corporation 373 from assessed values should reduce the differences Corporation that it would take action to address between assessed values and sale prices. Neverthe- our concerns. less, our discussions with the Association of Muni- cipalities of Ontario indicated that municipalities STANDING COMMITTEE ON were generally satisfied with the assessment-roll PUBLIC ACCOUNTS information the Corporation provided. In our 2010 Annual Report, we identified a num- The Standing Committee on Public Accounts held a ber of areas where improvement was needed with hearing on this audit in March 2011. In May 2011, respect to the Corporation’s collection of informa- the Committee tabled a report in the Legislature tion essential for accurate and consistent property- resulting from this hearing. The report contained tax assessments. The most significant of these areas five recommendations and requested that the Cor- were the following: poration report back to the Committee with respect • In the 1,400 cases in which we found the to the following: sale price differed by more than 20% from • the Corporation’s most recent data on the assessed value, the Corporation had not significant variances between properties’ sale investigated the reasons for these differences prices and assessed values, including when or made any adjustments to the assessed value and how the Corporation investigated the of these properties where warranted. variances and what adjustments were made • We found almost 18,000 building permits to assessments; with a total value of about $5.1 billion • the status of the Corporation’s efforts to as of December 31, 2009, for which the obtain information on building-permit work Corporation had failed to inspect the cor- completed and occupancy/completion status responding properties within the three-year reports from municipalities, including the statutory period for reassessing property number of municipalities that are submitting and improvements. this information and an indication of whether • Although the Corporation’s target is to municipalities are reporting in a standardized inspect each property in the province at way with enough detail and whether the Cor- least once every 12 years, under the actual poration would be publicizing on its website inspection cycle it would take at least 18 years those municipalities with more than 5,000 4.08

to achieve this, assuming current staffing properties that: levels and no further growth in the number of have not submitted any information on Section

• residential properties. building-permit work completed or any • The Corporation began work on a new com- occupancy/completion status reports; Follow-up puter system in 2000, but the system was not submitted building-permit information

• •

4 yet fully functional, and costs incurred at the more than six months after work was time of our audit exceeded $50 million com- completed; and

pared to an original budget of $18.3 million. • submitted occupancy/completion status Chapter • While the Corporation had established rea- information more than three months after sonable requirements for acquiring goods and the occupancy permit was issued; services, it often did not comply with good • the results of the Corporation’s managerial business practices, including its own manda- file reviews of Requests for Reconsideration tory policies and procedures. decisions since the audit; We made a number of recommendations for • the Corporation’s progress in meeting a improvement and received commitments from the 12-year inspection cycle, including the average 374 2012 Annual Report of the Office of the Auditor General of Ontario

daily number of inspections per inspector tracts are in place, and evaluations of contractors’ (and the length of the ensuing inspection performance are documented. It also advised us cycle), how the Corporation is ensuring that that it now follows the government’s Travel, Meal the risk of under- or over-assessments is the and Hospitality Expenses Directive. The Corpora- main driver of inspections, the effect of tools tion will need additional time to fully address some like hand-held computing devices, satellite of our other recommendations, such as inspecting imagery and phone surveys in shortening the and reassessing all properties for which a building inspection cycle, and measures taken to ensure permit has been issued before statutory limits on high-quality field-office review of inspection collecting additional taxes expire. The Corporation files; and will also need additional time to implement its • what oversight procedures the Corporation 12-year residential property inspection plan. The has to ensure that its staff are complying with status of the action taken on each recommendation its revised policies on procurement and on at the time of our follow-up was as follows. travel, meals and hospitality. The Corporation formally responded to the ASSESSED VALUES OF Committee in September 2011. A number of RESIDENTIAL PROPERTIES issues raised by the Committee were similar to our observations. Where the Committee’s recommen- Recommendation 1 dations are similar to ours, this follow-up includes To help ensure that individual properties are assessed the recent actions reported by the Corporation to in accordance with the Assessment Act at the amount address the concerns raised by both the Committee that a willing buyer would pay to a willing seller, the and our 2010 audit. Municipal Property Assessment Corporation should: • formally establish a threshold above which differences between a property’s sale price and its assessed market value must be investigated Status of Actions Taken on within a reasonable period of time; and Recommendations • where warranted, adjust the property’s assessed market value accordingly. 4.08 According to the information we received from Status the Corporation, significant progress has been The Corporation informed us that it used sale Section made in addressing most of our recommendations, prices from 2009, 2010 and 2011 to develop values with some progress being made on the others. For for its January 1, 2012, assessment update. In instance, the Corporation has established a 25% 2011, the Corporation established a threshold of Follow-up

• threshold, above which differences between a 25%, together with assessment-to-sales analysis 4 property’s sale price and its assessed market value from each market area, to flag sales that should be are to be investigated within a 12-month period. investigated. Depending on the difference between Chapter The Corporation has also expanded its use of its the previous assessed value and the current market information management systems to document value, the Corporation would conduct either an on- property inspections and Requests for Reconsidera- site inspection or a desktop review of the property. tion. It also conducts managerial reviews on these A desktop review, which is an alternative method electronic files. In addition, the Corporation has to an on-site visit, consists primarily of reviewing revised and strengthened its procurement policies digital images of the property and requesting sup- to ensure that goods and services are acquired porting documentation from the property owner. appropriately, written and properly authorized con- The Corporation aimed to conduct all targeted Municipal Property Assessment Corporation 375 investigations within 12 months from the date it Status obtains sale information. The Corporation indicated that it has developed an The Corporation advised us that out of the extensive communications strategy and introduced approximately 1,118,000 sales that occurred various initiatives to encourage all municipalities between 2009 and 2011, it had identified about to provide building-permit information via a 534,000 properties for investigations as they standardized electronic process (in the absence of met the 25% threshold and a predetermined any legislative requirement to do so). It has also assessment-to-sales ratio. It had conducted inves- developed internal standardized reports, provided tigations on approximately 382,000 of them as of to its managers every month, that contain key March 1, 2012. About 215,000 (or 56%) of these building-permit information, such as the age of the investigations resulted in a change to the property’s permit, the actual or expected completion date for assessed value. Approximately 191,000 (or 89%) the project and the value of the permit issued. were adjusted upward in value while the remaining The Corporation advised us that the number of 24,000 (11%) were adjusted downward. In total, municipalities in compliance with the standardized the exercise resulted in value changes of some process had more than doubled, from 69 to 154, $16.6 billion. The Corporation planned to complete over the four-month period from November 2011 the remaining approximately 152,000 investiga- to February 2012. Although the 154 municipalities tions by the end of 2012. represent only 37% of the 414 municipalities that The Corporation also informed us that it was in issue permits, these municipalities accounted for the process of updating its sales investigation pro- close to 126,000 (or 68%) of the approximately cedures, originally established in 2011. It planned 185,000 permits received by the Corporation for to incorporate a risk-analysis model and develop 2010. The remaining 32% of permits (59,000) were additional alternatives to the procedures to ensure submitted by 260 municipalities but were not pro- that sales investigations are completed on a more vided in compliance with the standardized process. timely basis. The Corporation’s goal was to have at least 80% of permits submitted in the standardized format. Once this 80% target is reached, the Corporation is BUILDING PERMITS to examine the costs versus the benefits of trying to raise this percentage further, as opposed to manag-

Recommendation 2 4.08

To help ensure that inspections of properties for which ing the remaining paper permits manually. a building permit has been issued are completed on a With respect to further inspection and reassess- Section timely basis so that retroactive assessments and tax ment of properties with a building permit issued, can be levied as soon as possible and certainly before the Corporation informed us that it used the Follow-up statutory limits expire, the Municipal Property Assess- building-permit information received from muni- •

4 ment Corporation should: cipalities to prioritize certain properties for value • ask all municipalities in the province to provide assessment before statutory limits expire. The

the Corporation with formal notification when Corporation reported that this permit prioritization Chapter the work with respect to a building permit has helped it to reduce the assessment amounts that been completed; and had passed the statutory limits from 15.7% in 2010 • inspect and reassess the market value of all such to 10% in 2011. The Corporation also indicated that properties before statutory limits on collecting the number of permits worth more than $10,000 additional tax expire. each that had been issued more than three years ago had been reduced from almost 18,000 as of December 31, 2009, to approximately 10,500 376 2012 Annual Report of the Office of the Auditor General of Ontario

permits as of March 31, 2012. The total value of and further revised in January 2011 to incorporate these older permits had also been reduced, from the new managerial review process. The Corpora- approximately $5.1 billion to about $2.3 billion over tion also implemented an automated document the same period. management system in 2010 to manage all RfRs. The Corporation further advised us that the Process controls were incorporated into the docu- average time to process an assessment change ment management system to improve documenta- resulting from building-permit activity had been tion quality and for ease of management review. reduced from 10.8 months in 2010 to 9.6 months Because all documentation and correspondence in 2011 and that the building-permit activity had received from property owners is scanned and resulted in over $28.4 billion in increased assess- stored electronically, paper documents are no ments in 2011. longer required. In addition, the Corporation indicated that, As of March 1, 2012, the Corporation had as of March 31, 2012, there were about 243,000 processed more than 18,500 RfRs received in 2011 building permits to be reviewed, including those for that tax year. The Corporation indicated that issued in prior years. Of the 243,000 permits, about manager reviews had been conducted for all of the 156,000 (or 64%) had been identified as ready for 2011 RfRs it received, including those that resulted reassessment and prioritized for completion within in no change in assessment value. As a result of legislated timelines. The construction work for the the managerial reviews, almost 2,700 (or 14.4%) remaining approximately 87,000 (or 36%) building of the 18,500 RfRs were flagged for further action. permits had not yet been completed. Of these, approximately 60% had errors relating to property valuation or classification. The remaining 40% had inadequate supporting documentation, REQUESTS FOR RECONSIDERATION AND but there was no impact on the property value or ASSESSMENT REVIEW BOARD APPEALS owner. The Corporation informed us that all of Recommendation 3 these RfRs were further reviewed to ensure they To help ensure that the merits of Requests for Recon- had been accurately processed prior to issuing the sideration (RfRs) are properly assessed, and that the final assessment for the property. adjustments to the property’s assessed market value The Corporation informed us that it will use are adequately supported, the Municipal Property the 14.4% rejection rate to determine the appropri- 4.08

Assessment Corporation should: ate level of sampling for future managerial file establish mandatory requirements for con- reviews of RfRs. Section • ducting and documenting RfRs; and on a sample basis, conduct and document • INSPECTIONS Follow-up managerial file reviews of all RfRs, including •

4 those that result in no assessment changes, to Property Inspection Cycle ensure compliance with suggested requirements Recommendation 4

Chapter for conducting an RfR. To help ensure that the property information in its Status database is as complete and up to date as possible, The Corporation advised us that it has established and that it has reliable information with respect to and implemented new Requests for Reconsidera- inspections completed, the Municipal Property Assess- tion (RfRs) Procedures, which identify the manda- ment Corporation should: tory requirements for conducting and documenting • require that each regional office select annually RfRs. These were implemented in October 2009 at least some properties for an inspection based Municipal Property Assessment Corporation 377

on the assessed risk of under- or over-assessment The Corporation informed us that, when changes with a view to working toward meeting its identified by self-reporting property owners cannot 12-year inspection cycle; and be validated through the use of digital imagery, the • maintain accurate and meaningful information validation is completed by way of an on-site review with respect to the number and type of inspec- of the property. tions completed (for example, sales investiga- In addition, the Corporation informed us that tions, building permits, and new constructions). to facilitate the tracking of accurate and meaningful information regarding the number and type Status of inspections completed, it added new property The Corporation has established a 12-Year Property review reason codes and method codes in March Review Plan and identified the number of proper- 2012 to improve its information systems. The ties that must be reviewed each year in order to new codes are intended to allow for more precise inspect all Ontario residential properties over a information that will better reflect the nature and 12-year period. The Corporation informed us that outcome of the inspections. it has prioritized this listing to ensure properties deemed to be at risk of under- or over- assessment are inspected. Some of the at-risk properties were Inspector Workloads, Quality of identified based on such factors as local area know- Inspections Performed ledge, historical inquiries, appeals and RfR activ- Recommendation 5 ities, and the average time since the last inspection. To ensure that inspections are conducted efficiently The workload listing has been distributed to man- and are adequately completed and documented, and agers at each regional office bi-weekly since June support the changes to a property’s assessed value, the 2011. As of July 2012, the Corporation had com- Municipal Property Assessment Corporation should: pleted a total of approximately 206,000 property regularly monitor and assess the productivity reviews for the year: approximately 111,000 were • of inspectors with respect to both the quality cyclical (targeted) inspections and approximately and average number of inspections being done 95,000 related to building permits, sales investiga- each day; tions, RfRs or appeals. ensure that files are documented in compliance The Corporation indicated that the number of • with acceptable standards and clearly dem- 4.08 residential properties that have not been inspected onstrate what work was completed and what or verified in more than 12 years had been reduced assessment changes were made as a result; and Section from over 1.5 million, as reported in our 2010 oversee the success of each regional office in Annual Report, to about 1.3 million as of July 1, • meeting the 12-year inspection-cycle target. 2012. Under its 12-Year Property Review Plan, the Follow-up

• Corporation intends to gradually reduce the gap Status 4 and eventually eliminate it by 2021. The Corporation informed us that it has developed As was also reported in our 2010 Annual and, since September 2011, distributed monthly Report, not all the Corporation’s inspections are inspection reports that summarize work activities Chapter conducted by an on-site visit. As mentioned earlier, such as average inspections per day, total inspec- the Corporation may validate property informa- tions completed and total work time. The reports are tion through a desktop review depending on the intended to assist regional managers in their mon- assessed risk. The desktop review may include itoring of inspector productivity and ensure that sending out sales questionnaires to property documentation standards are met; however, formal owners to obtain data for verification purposes. policies and procedures on the new documentation 378 2012 Annual Report of the Office of the Auditor General of Ontario

standards had not yet been established. The Cor- December 2012, inspection targets for 2012 were poration planned to formalize the new requirements reduced from the previous year by about 220,000 by early 2013. (40%), to 319,000 inspections. The Corporation also informed us that, effective We reported in 2010 that after the Ombudsman’s June 2011, inspection results, such as inspection or 2006 report recommended that the Corporation verification dates, comments, outcomes and value review its staffing needs, the total number of inspect- changes (if any), are entered and stored in the ors at the Corporation peaked at approximately 320 Integrated Property System. Manual files comple- in 2007; however, it had steadily dropped since then menting the system are no longer mandatory. to about 230 as of April 2010. Although the Corpora- Moreover, the Corporation indicated that property tion has increased its number of inspectors since, reviews were deemed to be completed only when and had some 250 and 260 inspectors on staff as of the appropriate information had been updated in December 2010 and 2011, respectively, it still has the system. fewer staff than it did in 2007. With respect to meeting the 12-year provincial In order to complete the 12-Year Property inspection-cycle target, the Corporation indicated Review Plan using available resources, the Corpor- that it had established a 12-Year Property Review ation has introduced several initiatives to improve Plan (see Figure 1) and an annual work plan at a efficiency. For example, by the end of 2012, the field-office level, and it monitored progress on a Corporation planned to complete an upgrade of monthly basis. The Corporation informed us that AboutMyPropertyTM, an on-line service property in 2011 it inspected approximately 543,000 proper- owners can use to validate their property infor- ties, or about 4,000 more than targeted. However, mation electronically. The Corporation believed there are fluctuations in the inspection targets over this service would be used by significantly more the 12-year cycle related to such factors as existing property owners than in the past because of an workload, competing demands and lack of resour- upgrade making it much more accessible and user- ces. For example, because of the legislative require- friendly. Moreover, the Corporation has replaced ment for delivering the 2013 Assessment Roll by traditional map books with GPS navigational devices and cellphones. As a result, its property Figure 1: 12-Year Residential Property Review Cycle inspectors have been able to locate properties Source of data: Municipal Property Assessment Corporation more quickly and can get in touch with the office 4.08

at any time for further information. # of Residential

Section Year Property Reviews

2010 446,565 Quality Control for Inspections Completed 2011 538,500 Follow-up 2012 319,000 Recommendation 6 •

4 2013 261,929 To enhance the effectiveness of the current quality 2014 545,052 control function, the Municipal Property Assessment

Chapter 2015 566,245 Corporation should: 2016 366,701 • ensure that supervisory reviews of inspection 2017 314,315 files are properly completed and adequately 2018 576,245 documented as required; and 2019 580,908 • include in its review process some inspection files 2020 419,087 that did not result in a change to a property’s 2021 314,315 assessed value. Total 5,248,862 Municipal Property Assessment Corporation 379

Status The Corporation informed us that it was con- In July 2011, the Corporation introduced a Property tinuing to develop improvements to its Integrated Review Process Control Team to investigate oppor- Property System. At the time of our follow-up, it tunities for streamlining and automating the prop- was developing and testing some system changes erty review process, to promote a consistent and and working to add some automatic control func- more effective approach to management oversight, tions to improve the efficiency and accuracy of and to facilitate the assignment and reporting of reviews. It planned to implement these changes by random property reviews for each field office. the end of 2012. Subsequently, in August 2011, the Corporation established standardized procedures to enable EXPENDITURES proper supervisory review of inspections. For example, the team developed a new form that Establishing the Need for Goods and is used to electronically document reviews on Services, Acquisition Process for Goods selected inspections on a bi-weekly basis. The Cor- and Services poration informed us that property reviews were Recommendation 7 being done based on both the information in its To ensure that goods and services are acquired only Integrated Property System and on-site visits. The when necessary and are the most appropriate in the results of each review are then to be documented circumstances, the Municipal Property Assessment on the new form and distributed to the responsible Corporation (Corporation) should comply with its own management team for appropriate action and procurement policy and ensure that each acquisition is: follow-up, if necessary. justified based on clear business requirements; The Corporation informed us that about 44,600 • the most appropriate option to satisfy the busi- (or 8%) of the property reviews for 2011 were • ness requirement under the circumstances; and found to have problems, either because of non- supported by a properly authorized purchase compliance with standard operating procedures or • requisition that provides evidence of the author- because they were deemed to be duplicate reviews ization to proceed. of the same properties. Although a property may To ensure that all vendors are treated fairly and have had to be reviewed more than once due to equitably and that it obtains value for money spent, multiple work activities, for the purpose of contrib- 4.08 the Corporation should also: uting to the Corporation’s 12-Year Property Review acquire goods and services competitively in Plan, the property was counted only once as a • Section compliance with its own requirements and those unique property review. of the Ministry of Finance; and The Corporation indicated that the samples prepare and maintain, for each transaction, Follow-up selected for the review process included both • • adequate documentation to demonstrate why 4 properties where the initial inspection resulted in a the successful vendor was selected. change to either the property record or the assess-

ment, and properties where the inspection resulted Status Chapter in no changes. In 2011, about 2,600 properties were In its response to our 2010 Annual Report, the sampled, of which approximately 1,500 (or 60%) Corporation indicated that it had implemented a were properties where the initial inspection had new Procurement Policy in 2009 to conform to the resulted in no change. As of December 31, 2011, the province’s procurement directive. The Corporation reviews of these 2,600 properties had resulted in has further revised the policy, and received board absolute value changes of 0.3% of their total value. approval of the changes in September 2011. The 380 2012 Annual Report of the Office of the Auditor General of Ontario

new policy requires the justification for each acqui- • all such agreements are approved by individuals sition to be based on clear and documented busi- with the authority to do so; ness requirements. It also requires the use of the • supplier invoices contain sufficient detail so that Integrated Resources Information System (IRIS) for the reasonableness of amounts billed and paid requisitions over $10,000. These purchases cannot can be assessed; and be processed without proper electronic approval in • it assesses and adequately documents the quali- accordance with a Delegation of Authority Frame- fications and performance of suppliers of goods work built into the system. and services. The new Procurement Policy also requires goods Status and services to be acquired competitively, with a The Corporation advised us that it had established few exceptions. Goods and non-consulting services written service agreements with all of its suppliers valued over $100,000 must be procured through a and contractors in the fall of 2009. These agree- Request for Proposal (RFP) process, and all RFPs ments include terms and conditions such as ceiling must be publicly posted. For acquisition of consult- price, expected deliverables, associated time frames ing services valued at $100,000 or more, an open and the circumstances under which price increases competitive procurement process generally must would be permitted. be used. A minimum of three vendors are required As mentioned in the previous section, the Cor- to submit a written proposal for consulting servi- poration’s new Procurement Policy requires that ces valued less than $100,000. The Corporation purchase requisitions be recorded in IRIS for each informed us that it would use the government’s supplier or contractor. The purchase requisition vendors of record where appropriate. requires electronic approval consistent with the The Corporation informed us that its Procure- Delegation of Authority. Once approved, a purchase ment Unit must maintain a full record of the pro- order and a service agreement are created. Accord- cess used to demonstrate and support the vendor ing to the Corporation, all existing service agree- selected for each RFP. For purchases that are over ments are signed by the accountable manager, as $10,000 but not tendered, the supporting docu- per the Delegation of Authority Framework. ments must be included with the purchase requisi- With regard to payments to contractors, the tion in the IRIS system. Corporation informed us that, effective Janu- The Corporation advised us that its internal audit 4.08 ary 2010, accountable managers are required to service unit was planning a compliance review of document the time spent on each project by each the Procurement Policy by the end of 2012. Section contractor, and provide this information to the Pro- ject Management Office. The Project Management Contractual Agreements Office is then to match it to contractor invoices in Follow-up

•

the IRIS system, ensuring there is sufficient sup- 4 Recommendation 8 port before forwarding the invoices to the account- The Municipal Property Assessment Corpora- ing department for payment.

Chapter tion should adhere to good business practices by The Corporation also informed us that at the ensuring that: end of each contract, managers must also review it enters into appropriate written agreements • contractor performance, and these evaluations are with all of its of suppliers of goods and services stored in the IRIS system. The Corporation had and that these written agreements include all established a standard template to evaluate con- the normally expected terms and conditions, tractor performance, but was still in the process of such as ceiling prices, expected deliverables, and developing a similar template to evaluate supplier associated time frames; performance at the time of our follow-up. Municipal Property Assessment Corporation 381

The Corporation’s internal audit unit was Status planning a compliance review of the Procurement The Corporation advised us that it had fully Policy, which is to include a review of its new ser- implemented the government’s Travel, Meal and vice agreement practices, by the end of 2012. Hospitality Expenses Directive as of January 2011 and provided the directive to all managers to make sure they were aware of the changes and new Travel, Meals and Hospitality requirements. Recommendation 9 The Corporation informed us that, in January The Municipal Property Assessment Corporation 2010, it added control checks to the IRIS system to (Corporation) should consult with the Ministry of flag possible duplicate expenses for investigation. Finance to determine whether it is the Ministry’s In October and November 2011, the Corporation’s intention to have the Corporation comply with the Finance Branch reviewed about 31% of the total spirit and intent of the government’s own directive expenses claimed for these two periods and found for the reimbursement of travel, meal, and hospital- that seven meal claims, worth about $110, were in ity expenses. As well, the Corporation needs to adopt fact duplicated claims. The claims were corrected more rigour in enforcing its travel, meal, and hospi- prior to expense reimbursement. The branch did tality policies. not identify any subsequent non-compliance and informed us that it planned to continue with compliance reviews on at least 5% of the total expenses claimed on a weekly basis. 4.08

Section

Follow-up

•

Chapter Chapter 4 Ministry of the Environment Section 4.09 Non-hazardous Waste Disposal and Diversion

Follow-up to VFM Section 3.09, 2010 Annual Report

and regulations and for enforcing compliance with Background these legislative requirements. The Ministry is also responsible for approving new municipal and private-sector waste management sites and facilities Non-hazardous waste includes non-recyclable and and for ensuring that these operations comply with recyclable materials generated by households and legislative requirements. In Ontario, the manage- by businesses and organizations in the industrial, ment of non-hazardous waste is governed primarily commercial and institutional (IC&I) sector. At the by the Environmental Protection Act (EPA), the time of our 2010 audit, approximately 12.5 million Environmental Assessment Act (EAA), and the Waste tonnes of non-hazardous waste were being gener- Diversion Act, 2002 (WDA). ated in Ontario annually. The IC&I sector generated Under the WDA, the provincial government has about 60% of this waste, and households—the established an arm’s-length organization, governed residential sector—generated 40%. Non-hazardous by a board of directors, called Waste Diversion waste is managed in two main ways: by disposal Ontario (WDO). The key responsibility of WDO (usually in a landfill or incineration) or by diversion 4.09 is to develop, implement and operate diversion (for example, recycling). About two-thirds of the programs for certain wastes, as designated by the province’s disposed waste is deposited in landfills Section Minister of the Environment, and to monitor the in Ontario, with the rest shipped to landfills in the effectiveness and efficiency of those programs. It United States. does this in conjunction with an Industry Fund- Follow-up Municipal governments are generally respon- •

ing Organization (IFO) consisting of industry 4 sible for managing waste generated by the resi- “stewards”—brand owners and first importers of dential sector. The IC&I sector and most multi-unit products that generate the waste. At the time of our

Chapter residential buildings are responsible for managing 2010 audit, three IFOs had also been established: the waste they produce and typically use private- Stewardship Ontario, for blue box waste and muni- sector companies to transport the waste to landfills cipal hazardous or special waste; Ontario Electronic or recycling facilities. Stewardship, for waste electrical and electronic The Ontario government, primarily through equipment; and Ontario Tire Stewardship, for the Ministry of the Environment (Ministry), is used tires. responsible for setting standards for the manage- In 2004, the government set a goal of diverting ment of non-hazardous waste through legislation 60% of Ontario’s waste from being disposed in

382 Non-hazardous Waste Disposal and Diversion 383 landfills by the end of 2008. At the time of our 2010 survey indicated that the funding they audit, the combined diversion rate of waste gener- received under the cost-sharing formula ated by the residential and IC&I sectors was about from industry “stewards” to offset some of 24%. This diversion rate ranked Ontario as sixth the costs they incurred for running the blue among the provinces. Waste diversion in the resi- box program was not sufficient. dential sector, at about 40%, had increased fairly • Landfill capacity that is available to a muni- substantially since 2002, but this increase had been cipality. When waste is collected less often offset by a drop in the IC&I sector’s diversion rate. and bag limits are imposed, residents typ- We noted that many of the issues that the ically divert more waste. For example, one government identified in 2004 as keys to achieving municipality indicated that by collecting 60% waste diversion by the end of 2008 had yet to recyclable materials weekly and disposable be successfully addressed. Our specific observations waste every two weeks while imposing a were as follows: bag limit, it was able to increase its diver- • Municipalities (generally responsible for sion rate by about 20%. But the responses managing residential waste) and households to our survey indicated that municipalities were making progress in diverting waste away that have sufficient landfill capacity are less from landfills. However, although their overall likely to limit the frequency of waste collec- diversion rate for residential waste was about tion and impose a bag limit on residents. 40%, we found that individual municipalities’ • Residents’ preferences. Municipal councils diversion rates reported to us varied signifi- are well aware that their constituents want cantly, from about 20% to more than 60%. a higher level of waste pickup service and This was mainly due to differences in the fre- no bag limits regardless of the impact on quency and quantity of disposable waste col- waste diversion. lection, differences in the blue box recyclable • The IC&I sector generated approximately 60% materials that were collected and the fact that of the waste in Ontario, but managed to divert only some municipalities had implemented only about 12% of its waste. Regulations organic-waste composting programs. Other under the Environmental Protection Act require differences in municipalities’ waste manage- large generators to conduct a Waste Audit, ment practices included the following: prepare a Waste Reduction Work Plan, and 4.09

• Whether a municipality can market its blue implement programs to source-separate waste box and organic recyclable waste. Munici- for reuse or recycling. However, the Ministry Section

palities compete with each other and with had little assurance that the regulations were the private sector for markets for recyclable being complied with for the following reasons: Follow-up waste. The larger municipalities, which can The Ministry did not have adequate infor-

• •

4 generate significant volumes, are more suc- mation on either the number of businesses cessful at securing markets than the smaller or organizations to which the regulation

municipalities and therefore can encourage applied or which segments of the IC&I sec- Chapter greater recycling. tor generated the largest amounts of waste • Cost. On average, municipalities reported so that it might target them for inspection. that the cost of diverting a tonne of blue • In half of the inspection files we reviewed, box recyclable materials was about 40% there was no evidence that the ministry higher than the cost of disposing of a tonne inspector had reviewed either the Waste of waste in a landfill. More than half of Audit or the Waste Reduction Work Plan. the municipalities that responded to our 384 2012 Annual Report of the Office of the Auditor General of Ontario

• The inspections did not assess the extent STANDING COMMITTEE ON to which the IC&I-sector businesses were PUBLIC ACCOUNTS separating recyclable waste or whether the The Standing Committee on Public Accounts held waste that had been source-separated was a hearing on this audit in February 2011. In May actually being processed for recycling. 2011, the Committee tabled a report in the Legis- Organic waste generated by both the residen- • lature resulting from this hearing. The report con- tial and IC&I sectors represented almost one- tained nine recommendations, requesting that the third of the total waste generated in Ontario, Ministry report back to the Committee with respect but there was no province-wide organic waste to the following: diversion program or target, despite the when the Ministry estimates it will reach its Ministry’s having considered establishing a • goal of 60% diversion of Ontario’s waste from program as early as 2002. landfills and whether any additional tools are One in five municipalities that responded • required for this; to our survey felt that they had insufficient the Ministry’s strategy for increasing the rate landfill disposal capacity for their residential • of waste diversion in the IC&I sector, includ- waste. As well, the existing capacity was ing a diversion target and timeline for the expected to be filled more quickly once export sector, ways that the Ministry will motivate of residential waste to Michigan largely ended businesses and organizations that are not after 2010 and an additional 1 million tonnes regulated under the Environmental Protection of this waste previously shipped to that state Act to improve their diversion rates, and how began being deposited in Ontario landfills the Ministry will monitor the businesses annually. Opening new landfills within muni- and organizations; cipalities is not always a viable option, both the Ministry’s plans for obtaining adequate because they are costly and because residents • information on the number of businesses and do not support new landfills. organizations to which the waste diversion The Ministry inspected landfills and non- • regulations apply and on which of these are hazardous waste management sites, facilities the largest waste generators; and systems against the conditions of their the Ministry’s 2011 protocols for field inspect- certificates of approval. But we noted that • 4.09 ors, including whether inspections are risk- many of these certificates did not reflect based and target the largest waste generators, changes in standards. Also, in our review Section how the Ministry tracks whether businesses of inspection files, we found that numerous and organizations have implemented their examples of non-compliance with the certifi- Waste Audits and Waste Reduction Work Follow-up cates’ conditions had been noted, but many of •

Plans, and how the Ministry determines 4 these were not being followed up on a timely whether businesses and organizations are basis to ensure that the required changes source-separating waste and whether the

Chapter were made. source-separated waste is being recycled; We made a number of recommendations for how the Ministry’s new guidelines will improvement and received commitments from • increase the diversion of organic waste; the Ministry that it would take action to address the Ministry’s public consultations on and our concerns. • review of the Waste Diversion Act, 2002; • how the Ministry will improve its oversight of Waste Diversion Ontario (WDO) and whether Non-hazardous Waste Disposal and Diversion 385

it will consider having a senior ministry repre- This and recent ministry inspection results suggest sentative sit on WDO’s board of directors; and that much remains to be done to increase the waste • the average time the Ministry takes to approve diversion rate in the IC&I sector. The status of a certificate-of-approval application under actions taken on each of our recommendations at its new approvals system and whether it has the time of our follow-up was as follows. a standard for how long it takes to review an application. WASTE DIVERSION The Committee also requested that WDO report back on how it monitors whether its diver- Residential-sector Waste sion programs are meeting targets, what steps it Recommendation 1 takes to address setbacks in meeting targets, and To further increase diversion of waste in the resi- how it assesses the waste-diversion information it dential sector, and as part of its current review of receives from municipalities and Industry Funding the Waste Diversion Act, 2002, the Ministry of the Organizations. Environment should work with municipalities, indus- The Ministry formally responded to the Com- try “stewards,” and other stakeholders to: mittee in October 2012. A number of issues raised increase the availability of reliable and sustain- by the Committee were similar to our observations. • able markets for recyclable and organic waste; Where the Committee’s recommendations were increase capacity within the province to process similar to ours, this follow-up includes the recent • recyclable materials and organic waste; and actions reported by the Ministry to address the review the current funding formula for the blue concerns raised by both the Committee and our • box program to ensure that it achieves its object- 2010 audit. ive of municipalities and “stewards” equally sharing costs.

Status Status of Action Taken At the time of our follow-up, the Ministry indicated on Recommendations that in 2011, Stewardship Ontario had undertaken a market development program that identified new opportunities to improve the recycling chain and 4.09 According to information provided to us by the extract more value from recyclable materials in Ministry, some progress has been made in address- Ontario. Specifically, the Ministry informed us of Section ing many of the recommendations we made in our the following: 2010 Annual Report. For example, governance at In spring 2011, Stewardship Ontario issued Waste Diversion Ontario has been strengthened • Follow-up a Request for Expressions of Interest (RFEI) • and Ontario’s compost framework has been 4 aimed at identifying new companies with updated to encourage more composting. However, innovative approaches to recycling. More than several recommendations will require more time to

60 submissions were received from companies Chapter be addressed fully. In particular, our concerns with interested in forming commercial relation- regard to diversion of waste in the IC&I sector have ships with Stewardship Ontario. As part of yet to be substantially addressed. In this regard, this initiative, Stewardship Ontario invested we note that according to Statistics Canada, the $500,000 in a company with a view toward waste diversion rate in the IC&I sector in 2008 (the commercializing new plastics recycling latest year for which information was available) technology and, according to the Ministry, was 12.7%, up only slightly from 12% in 2006. 386 2012 Annual Report of the Office of the Auditor General of Ontario

was in active discussions with more than 15 • conduct research into successful practices used other companies. in other provinces and European countries • Stewardship Ontario provided a loan to an to divert IC&I-sector waste from landfills. In existing plastics recycling company to enable assessing which practices might be transferable it to relocate and further expand its facility. to Ontario, the Ministry will need to balance • Stewardship Ontario initiated a review of the environmental benefits with the economic recycling of fibre packaging and the poten- challenges currently being faced by the tial to expand collection and recycling of business community. these materials. Status As prescribed in the WDA, the blue box funding At the time of our follow-up, the Ministry indicated formula requires industry to fund 50% of the net that it was continuing to consider ways of obtaining cost of the municipal blue box program. As noted appropriate information necessary to support earlier, more than half of the municipalities that diversion policies and programs in this sector, and responded to the survey we conducted as part of conducting further jurisdictional research as part of our 2010 audit indicated that the funding they its ongoing policy development work. received under the cost-sharing formula to offset The Ministry had also supported the October some of the cost they incurred for running the blue 2011 launch of the 3RCertified Waste Diversion box program was not sufficient. At the time of our Certification Program by the Recycling Council follow-up, the Ministry indicated to us that any of Ontario (RCO), a not-for-profit organization changes to the funding allocation model are pro- involved in policy, education and work surround- posed through WDO’s Municipal Industry Program ing waste generation and diversion. This program Committee, on which municipalities and Steward- gives businesses and institutions the opportunity to ship Ontario are equally represented, and that a become accredited with the 3RCertified standard review of the funding formula was under way. by demonstrating conformance to a set of criteria, verified by an onsite evaluation by the RCO. Com- Industrial, Commercial, and Institutional pliance with the Ministry’s regulations for the man- (IC&I) Sector Waste agement of non-hazardous waste is also one of the requirements to achieve base certification under

4.09 Recommendation 2 the program. In order to increase waste diversion in the IC&I sector, At the time our report was being finalized, the the Ministry of the Environment should: Section RCO announced that four organizations in the gather information on the amount and type of • province had achieved 3RCertification. waste generated by small and medium-sized Follow-up businesses and organizations that are not regu- •

4 lated under the Environmental Protection Act Compliance in the IC&I Sector, Scope (EPA) and consider what actions could be taken of Inspections in the IC&I Sector and

Chapter to reduce the amount of waste that is currently Enforcement of Other EPA Regulations going to landfills; Recommendation 3 require those large entities that are regulated • To improve waste diversion in the IC&I sector, the under the EPA to publicly report their waste Ministry of the Environment should: diversion rates. The Ministry should then, as gather data on the number of businesses to which part of its inspection work, assess the accuracy • the waste diversion regulations apply and on of the rates reported; and which of these are the largest waste generators Non-hazardous Waste Disposal and Diversion 387

to assist both its inspection activities and policy The Ministry also indicated that, in the last two decisions, and ensure that businesses are aware years, it had conducted outreach efforts tailored to of the requirements of the regulations; individual sub-sectors. For instance, the Ministry • increase the scope of its inspections to include had worked with 11 school boards and with the an assessment of the extent to which businesses Ontario Hospital Association to customize Waste have implemented their Waste Audits and Audit and Waste Reduction Work Plan guides to Waste Reduction Work Plans and whether there help the multiple facilities in each sub-sector meet has been any increase in the amount of waste their regulatory requirements. diverted; and In the 2011/12 fiscal year, the Ministry • verify during inspections and document whether conducted a “re-sweep” of a select number of waste management companies are operating previously inspected facilities to assess the impact under a valid certificate of approval. of inspection efforts on, for example, the extent to If the Ministry plans to continue not to enforce which the facilities were preparing Waste Audits its regulation that requires large manufacturers, and Waste Reduction Plans and source-separating packagers, and importers to implement a packaging required wastes. Out of 104 selected sites, 17 had reduction plan and its regulation that requires all car- changed operations or ownership or had closed. Of bonated soft drinks to be sold in refillable containers, the remaining 87, only 21 passed the re-inspection. it should consider revoking these regulations. The Ministry informed us that as of August 2012, 65 of the 66 sites that had failed the re-inspection Status had been brought into compliance. There was an At the time of our follow-up, the Ministry indicated Order issued against the remaining site with a that, as an overall measure aimed at increasing compliance date of August 31, 2012. The Ministry waste diversion, it was continuing to work with also indicated that based on the results of the all its partners, including businesses and organ- re-inspections, recommendations for program izations and the Canadian Council of Ministers improvements would be made. of the Environment, to reduce the amount of A number of the inspection files of waste packaging created. management companies that we reviewed as part In our 2010 Annual Report, we noted that the of our 2010 audit did not contain evidence that Ministry did not have adequate information on the the inspector had checked that the company was 4.09 number of businesses and organizations to which operating under a valid Ministry-issued certificate the EPA’s waste diversion regulations applied. This of approval. The Ministry informed us that chan- Section observation prompted us to recommend to the Min- ges implemented to its inspection tracking and istry to gather such information and to ensure that reporting system now ensure that whether waste businesses and organizations are aware of the regu- Follow-up management companies are operating under a • latory requirements. At the time of our follow-up 4 valid certificate of approval is consistently docu- the Ministry provided us with the sources of infor- mented within the system. mation it uses to identify businesses and organiza-

Lastly, the Ministry indicated to us that, as part Chapter tions covered by the EPA regulations. The Ministry of its initiative to reduce unnecessary or obsolete indicated that, based on these sources, it now had regulatory requirements, it was still considering an estimate of the number of facilities to which the whether the regulation that requires large manu- waste diversion regulations apply. The Ministry facturers, packagers and importers to implement a also informed us that over the last two years it packaging reduction plan and the regulation that had begun working with 14 businesses (with a requires all carbonated soft drinks to be sold in combined total of more than 550 retail locations) to refillable containers should be revoked. develop company-wide waste diversion programs. 388 2012 Annual Report of the Office of the Auditor General of Ontario

Organic Waste • ensure that the waste diversion information submitted by municipalities and the Industry Recommendation 4 Funding Organizations (IFOs) is objectively To increase overall waste diversion in Ontario, the assessed, including the impact on this informa- Ministry of the Environment should work with tion of unregistered collectors that do not submit municipalities, businesses and organizations, and waste diversion data; and private-sector waste management companies to phase reconsider its policy of allowing collectors of in over time a province-wide organic waste diversion • designated wastes the option of whether or not program for both the residential and IC&I sectors. to register with an IFO. As part of implementing the program, the Ministry, Where retailers are charging a specific “eco fee,” in conjunction with these stakeholders, will need to the Ministry should also reconsider whether they ensure that there is sufficient capacity to process the should be required to disclose the amount of the fee on additional organic waste and that a sustainable mar- the customer receipt. ket exists for the processed waste. Status Status On October 25, 2010, after the completion of The Ministry indicated that it had completed our 2010 audit, the Minister of the Environment consultations with the public and industry on a advised WDO of the need to amend its board struc- proposed update to Ontario’s compost framework, ture so that it: including appropriate standards for compost, reflected the knowledge and expertise environmental protection measures and other • required to oversee waste diversion programs tools to support a sustainable market and process- under the Waste Diversion Act, 2002 (WDA); ing capacity for organic waste. At the time of our avoided real, potential or apparent conflicts follow-up, the Ministry had released an update to • of interest between WDO board members and the framework that included changes to compost the programs they oversee; and quality categories and standards, and best practices included at least one board member selected for the siting, design, operation and maintenance • from a consumer-focused organization to of composting facilities. These changes were to take ensure that a consumer perspective would be effect January 1, 2013. brought to the WDO board. 4.09 In a letter dated February 9, 2012, the Minister Waste Diversion Ontario stated that he had accepted in principle the pro- Section posal submitted by WDO’s board on May 27, 2011, Recommendation 5 with regard to a new governance structure. WDO To enhance accountability for the achievement of transitioned to the new board in April 2012. Follow-up diversion targets for wastes specifically designated • In the February 9 letter, the Minister also 4 under the Waste Diversion Act, 2002, and to ensure directed WDO to undertake a detailed review of all that the reporting of the diversion results against the diversion program budgets and to regularly monitor

Chapter targets to the Minister is complete and reasonably expenditures to ensure that there are realistic and accurate, the Ministry of the Environment should: cost-effective plans to achieve IFO performance review the operating agreement to ensure that it • targets. In this regard, WDO is required to report contains sufficient accountability provisions to quarterly to the Ministry. The Minister also required require Waste Diversion Ontario to provide an WDO to implement an effective monitoring pro- action plan when waste diversion targets are not gram to review IFOs’ annual program performance being met; and the achievement of plan targets, to promptly Non-hazardous Waste Disposal and Diversion 389 take corrective action on performance issues and to WASTE DISPOSAL report quarterly on these matters to the Ministry. Recommendation 6 At the time of our follow-up, the Ministry informed To increase Ontario’s capacity to dispose waste, the us that, in addition to the existing requirement for Ministry of the Environment should take a leader- audited financial statements, WDO had begun to ship role in working with municipalities and other implement independent third-party verification stakeholders to research and adopt alternative waste of environmental performance and standard- disposal technologies such as the thermal treatment ized reporting for all waste diversion programs. facilities that are in use in other jurisdictions. However, the Ministry also informed us that since the WDA doesn’t compel individuals or companies Status who collect waste to share information on their At the time of our follow-up, the Ministry indicated activities, the third-party verification is limited to that the province’s priority was still to divert as those companies that participate in the diversion much waste as possible from disposal through the programs. The audits of the programs for blue box 3Rs (reduce, reuse and recycle). Generating energy and municipal hazardous or special waste and for from waste is an option that can be considered to waste electrical and electronic equipment had been help manage the residual waste that remains after completed and were being reviewed by WDO before diversion. In this regard, the Ministry informed being made public. The audit of the used-tires us that regulations that came into effect in 2007 program had been deferred by WDO pending an had streamlined the approvals process for pilot evaluation of the performance audits undertaken and demonstration energy-from-waste facilities. for the other waste diversion programs. The Ministry indicated that it had approved under At the time of our follow-up, no action had been these regulations a number of pilot and demonstra- taken on our recommendation to the Ministry to tion facilities incorporating energy-from-waste reconsider its practice of allowing collectors of technologies. For example, in July 2011, a cer- designated wastes the option of whether or not to tificate of approval was issued for the York and register with an IFO. The Ministry informed us that Durham Regions’ energy-from-waste facility. This it would consider this recommendation as it works facility is to have the capacity to receive 140,000 toward making the province’s waste diversion tonnes of residual waste annually and the ability framework more effective. to generate up to 20 megawatts of energy. Also, 4.09

The Ministry also informed us at the time of our an approval was issued in October 2011 for the follow-up that, to investigate whether retailers are permanent operation of a demonstration energy- Section charging incorrect or misleading eco fees, environ- from-waste facility in Ottawa. mental compliance officers have been conducting Follow-up site visits since November 2010 as a follow-up to • Monitoring of Waste Disposal Sites and 4 consumer hotline calls and undertaking mystery Waste Management Systems shopping activities. The Ministry indicated that it

had conducted 1,303 mystery shopping activities Certificates of Approval Chapter as of early January 2012 and that 252 (19%) of the Recommendation 7 sites involved had been referred to the Ministry’s To better facilitate compliance with certificates of Investigation and Enforcement Branch to assess approval for non-hazardous waste management whether there was evidence of a violation that sites, facilities, and systems, the Ministry of the needed to be referred to the Ministry of Consumer Environment should: Services for prosecution under the Consumer Pro- • review its existing certificates, especially for tection Act, 2002. the larger or more environmentally risky 390 2012 Annual Report of the Office of the Auditor General of Ontario

operations, to ensure that they reflect current and reviewing the application fee to ensure that it standards and operations and revise those that reflects the cost of processing the applications. need updating; in cases where numerous amendments have • Inspections been issued to an existing certificate, consolidate the amendments into one, new certificate; Recommendation 8 • develop a standard for the time it should take To improve its monitoring of non-hazardous waste to review certificate-of-approval applications management operations for compliance with legislative for non-hazardous waste operations and requirements, the Ministry of the Environment should: review the outdated application fee it charges to • impose time frames for corrective action where ensure that it reflects the cost of processing the inspections detect cases of non-compliance, and applications; and follow up to ensure that the required remedial • collect, follow up on, and review the re- action has been taken within the required valuation of the required financial assurance, timelines; and especially for the larger operators, on a • ensure that time-sensitive materials such as timely basis. annual reports from non-hazardous waste management operations are submitted and reviewed Status on a timely basis. At the time of our follow-up, the Ministry indicated that it had completed a review of the Environ- Status mental Compliance Approvals (ECAs, formerly At the time of our follow-up, the Ministry had certificates of approval) for 32 larger landfill sites implemented updated monitoring procedures, that together received more than 85% of the waste including a time frame for finalizing actions of 45 destined for landfill in Ontario. The review, accord- to 60 days from sign-off of inspections that detect ing to the Ministry, concluded that most approvals cases of non-compliance. The updated procedures were current and that only one site required a were communicated to staff in February 2011. minor update, which was later issued. Training sessions for environmental officers on the Also, the Ministry informed us, as it did in 2010, updated monitoring procedures were held dur- that it had revised the practice of amending ECAs. ing spring 2011. The Ministry indicated that the 4.09 A single ECA is now to be issued and is to include training also covered the Ministry’s data system the original as well as any subsequent notices functionalities that track and provide automated Section of amendment. alerts on the status of abatement actions. According The Ministry also informed us that its Environ- to the Ministry, through the combination of train- mental Approvals Access and Service Integration ing and better tracking of the status of abatement Follow-up

•

Branch had begun providing monthly reports to actions, it had improved the documentation and 4 regional directors on outstanding financial assur- follow-up of remedial actions where inspections ance matters to ensure that field staff were follow- detect non-compliance. Chapter ing up with ECA holders. The Ministry had also developed a risk-based Over the next two years, the Ministry indicated approach to address the submission and review of that it planned to continue modernizing the annual reports from waste disposal sites. Environ- approvals program in ways that would address mental officers were trained in March and April our other recommendations aimed at developing a 2011 on the new procedures for annual report standard for the time it should take to review ECA assessment. According to the Ministry, the training applications for non-hazardous waste operations provided to environmental officers, together with guidance material developed for the stakeholder Non-hazardous Waste Disposal and Diversion 391 community to improve the quality of submissions, Status had led to a more effective review process. At the time of our follow-up, the Ministry indicated that it was using a variety of waste diversion data sources to assist in evaluating waste diversion MEASURING PROGRESS IN progress, including data from Waste Diversion WASTE DIVERSION Ontario, Statistics Canada, municipalities and Recommendation 9 waste management companies, as well as informa- The Ministry of the Environment should assess the tion from the Ministry of Finance on the Ontario benefits of adopting an alternative performance Deposit Return Program. The Ministry informed indicator, such as the per capita waste disposal rate, us that it was continuing to assess the benefits of a because it is more straightforward to calculate and is range of performance indicators related to waste likely a more accurate and reliable measure of waste diversion, including the use of a per capita waste diversion in Ontario that will facilitate benchmarking disposal rate. progress relative to other jurisdictions. 4.09

Section

Follow-up

•

Chapter Chapter 4 Ministry of Health and Long-Term Care Section 4.10 Organ and Tissue Donation and Transplantation

Follow-up to VFM Section 3.10, 2010 Annual Report

ability to meet organ- and tissue-transplant needs. Background As demand exceeds availability for many organs, the willingness of families of deceased people to donate organs is critical. Since the Network’s In the 2011/12 fiscal year, almost 1,000 organs establishment, the number of deceased donors were transplanted from almost 550 donors at the has increased from 11.3 to 16.7 donors per million eight Ontario hospitals that perform transplants. As people. Nevertheless, our 2010 audit suggested of March 31, 2012, more than 1,500 Ontarians were certain changes that could be made to reduce wait waiting for organs, most of them for a kidney or a times for organs, thus saving lives and improving liver. As well as saving or enhancing lives, trans- patients’ quality of life. plants can save money. For example, each kidney Our findings included the following: transplant surgery cost about $25,000 at the time There was no periodic independent review of of our 2010 audit. The same year, however, the cost • the Network’s allocation of organs. Oversight of dialysis—a mechanical procedure carried out of transplant activities on a province-wide frequently to cleanse the blood of a person whose 4.10 basis needed to be enhanced to help ensure kidneys have failed—was about $70,000 a year. that patients were consistently prioritized on The Trillium Gift of Life Network (Network), Section wait lists and that the highest-priority patient which has a staff of about 130, was established received the first available compatible organ. in 2002 as an agency of the Ministry of Health In more than 40% of the cases we reviewed, Follow-up and Long-Term Care (Ministry) to co-ordinate the • the highest-priority patient did not get the 4 donation of organs and tissue, which includes eyes organ, and no reason for this decision was and bones. Funding to the Network and the eight documented.

Chapter hospitals for transplants in the 2009/10 fiscal year Forty hospitals generally did not refer poten- was about $100 million. Our 2010 audit assessed • tial donors to the Network even though they whether there were adequate policies, procedures had the medical equipment necessary to and systems in place to meet the organ and tissue maintain organs until transplant. needs of Ontarians in an efficient and fair manner. For years, many Ontarians signed the As we reported in our 2010 Annual Report, • donation-consent part of their driver’s initiatives by the Network, the Ministry and the licence and kept it in their wallet. However, transplant hospitals had improved the province’s

392 Organ and Tissue Donation and Transplantation 393

this type of consent was not included in the the Network to verbally communicate this Ministry’s consent registry, which the Network information, increasing the risk of decisions uses to determine if a potential donor has being based on incomplete or inaccurate given consent. information. • There was a lack of consistent clinical criteria • Less than 8% of tissue needed for transplant regarding the time when hospitals should in Ontario actually came from Ontario donors refer potential donors to the Network, because of a lack of resources to recover, resulting in many referrals being made too process and store it. Instead, Ontario hospitals late or not at all. purchased tissue primarily from the United • Only 15,000 of the 4 million Ontarians who States and Quebec. still have red-and-white OHIP cards had their • One Ontario hospital performed only six consent registered with the Ministry (almost transplants in a year, and although Ontario certainly because this required mailing in does not stipulate a minimum yearly number a separate form to ServiceOntario), while of transplants to ensure a hospital remains 1.9 million people with photo OHIP cards had proficient, the U.S. minimum requirement is registered (because people are specifically generally 10. asked during the application/renewal process We made a number of recommendations for if they want to register). Consent-registration improvement and received commitments from the rates also varied significantly among regions, Network and the Ministry that they would take from less than 10% in Toronto to more than action to address our concerns. 40% in Sudbury. • Hospitals indicated that patients requiring organs were not always referred so that the transplant hospital could determine whether Status of Actions Taken on the patient was eligible to be placed on a Recommendations transplant wait list. For example, only 13% of dialysis patients were on a kidney wait list, The Network and the Ministry provided us with and rates varied from only 3% in the South information in the spring and summer of 2012 on East Local Health Integration Network (LHIN) 4.10 the current status of our recommendations. Accord- to 16% in the Champlain LHIN. ing to this information, some progress was made Instead of being allocated to the highest- Section • in implementing all of the recommendations in our priority patient province-wide, kidneys and 2010 Annual Report, with significant progress in livers generally stayed in the same region areas such as simplifying the consent registration Follow-up in which they were donated. As a result, for • process and using kidney pumps to increase the 4 example, 90% of kidney recipients received viability of kidneys from deceased donors. While the organ within four years in one area of the more hospitals are now required to report potential

province, while those in two other regions Chapter organ and tissue donors to the Network, it will take waited about nine years. Wait times by organ additional time to fully address and implement this type were generally not publicly available. and several of our other recommendations. Some Transplant hospitals did not have electronic • actions depend on the interprovincial–territorial access to donor information such as medical response to the national organ and tissue donation history and laboratory results, which is neces- and transplantation plan released in April 2011 by sary to determine organ viability for potential Canadian Blood Services. recipients. Instead, they generally relied on 394 2012 Annual Report of the Office of the Auditor General of Ontario

The current status of the actions taken by the advanced ventilator capacity. As of summer 2012, Network and the Ministry is summarized following 18 more hospitals were notifying the Network each recommendation. of potential organ donors, and another 10 were expected to start by the end of the 2012/13 fiscal year. The remaining hospitals with ventilator cap- ORGANS acity were expected to begin notifying the Network Identifying and Referring Donors in the 2013/14 fiscal year. The Network indicated that updated clinical Recommendation 1 triggers, which assist physicians in identifying when To increase the number of organs available to individ- to notify the Network of potential donors, were uals waiting for a transplant, the Trillium Gift of Life reviewed and endorsed by the Network’s Donation Network (Network) could enhance the identification Steering Committee in fall 2011. The Network of potential organ donors through such means as: noted that the revised triggers were being incor- determining whether all 61 hospitals with • porated into the policies of all hospitals required advanced ventilator capacity (necessary to to report potential organ donors to the Network. maintain the viability of organs for transplant), The Network also planned broader consultations rather than just the current 21 hospitals, should to ensure acceptance of the criteria, and indicated be required to notify the Network of potential that it may further revise the triggers after the con- organ donors, in accordance with the recom- sultations end in December 2012. mendation of the Ministry of Health and Long- The Network said it planned to review oppor- Term Care’s Organ and Tissue Transplantation tunities to integrate organ donation and end-of-life Wait Times Expert Panel; care, including ways to leverage the Emergency developing and implementing consistent, appro- • Neurosurgery Image Transfer System and CritiCall, priate clinical criteria, in conjunction with during the 2012/13 fiscal year. The Network noted hospitals, to assist physicians in knowing when that the Executive Director of CritiCall was a mem- to notify the Network of potential donors; ber of the Network’s Donation Steering Committee. using existing provincial systems, such as Criti • The Network said it worked with the Ontario Call, a referral service for critically ill patients, Medical Association and physicians, in consultation and the Emergency Neurosurgery Image Transfer with the Ministry, to identify gaps and oppor- 4.10 System, used to remotely view the computed tunities in the current physician fee schedule to tomography (CT) images that can confirm brain encourage doctors to identify and report potential Section death, to help identify potential donors; and organ donors. The Ministry determined that the working with all stakeholders—including the • physician fee schedule already includes fees cover- Ministry, hospitals, and physicians—to ensure Follow-up ing this area—but that physicians may not be aware • that there are sufficient financial incentives to 4 that it does because they seldom bill for it. The encourage more widespread identification and Ministry expected the Network to clarify with phys- reporting of potential donors.

Chapter icians when these fees can be billed and further Status indicated that it will monitor the implementation At the time of our follow-up, the Network said that of the Health System Funding Strategy for hospital the requirement to report potential organ donors funding opportunities related to the identification was being expanded to all hospitals that have and reporting of potential donors. Organ and Tissue Donation and Transplantation 395

CONSENT ServiceOntario launched a pilot project in which ServiceOntario counter staff ask people who renew Recommendation 2 a driver’s licence or obtain an Ontario photo health To help improve consent rates for potential organ card if they wish to register as potential donors. donation, the Trillium Gift of Life Network (Net- The Network and ServiceOntario were planning to work) should: expand this practice province-wide by March 2013. work with the Ministry of Health and Long- • In May 2011, the Network began to monitor Term Care, the Ministry of Transportation, the effectiveness of all staff involved in obtaining and ServiceOntario to change the system of consent from families of potential donors, and obtaining consent at the time of driver’s licence found that staff with Network training and experi- renewal to enable persons to be added to the ence were more successful at obtaining consent donor registry, because neither the Network than health-care professionals with no training in nor hospitals have access to the donor card this area. The Network’s policy endorses its staff previously sent with licence renewals that many approaching families for consent, but also says people sign and keep in their wallet; that a discussion is required between Network staff determine, in conjunction with the hospitals, • and the health-care team of a potential donor to the best approaches to increasing consent rates determine who should approach the family, and at the hospitals, especially in those areas of how best to do it. The Network provides quarterly the province where consent rates are low—for training sessions for staff, and monitors all staff example, by identifying specific individuals who involved in obtaining consent to identify areas have an aptitude for or training in successfully where additional follow-up training is necessary. requesting consent; and The Network also noted that it held a workshop in consider implementing a “mandatory ask” • March 2012 to help ensure staff have the appropri- policy, along the lines of a policy used in the ate knowledge and understanding to deal with United States, which would require that the next families and health-care professionals from diverse of kin of every potential organ donor be asked cultural backgrounds. for consent before the removal of life support. To improve consent rates, the Network’s Dona- Further, the Ministry of Health and Long-Term tion Steering Committee endorsed a “mandatory Care should simplify the process by which people ask” policy in summer 2012 whereby consent to 4.10 register consent to be an organ donor, such as by donation must be requested of all potential donors implementing an on-line consent registry similar before the withdrawal of life-sustaining therapy. Section to those available in British Columbia and other The Network also indicated that it was reviewing jurisdictions. situations where organ donations do not proceed Follow-up Status because a potential donor’s family does not support •

4 In fall 2010, the Network, in collaboration with the person’s previously registered consent deci- ServiceOntario and the ministries of Transportation sion, to see if it can devise strategies for ensuring

and Health and Long-Term Care, began mailing that decisions are fully informed while the family’s Chapter donor-registration forms with prepaid return envel- rights are still respected. opes to persons renewing their driver’s licence. The Ministry said an online donor registration The Network indicated that the response rate has system was implemented in spring 2011, and that been much higher, with 4.3% of people signing more than 44,000 people used it to register their their consent and returning the forms to Service consent between June 2011 and April 2012. Also, Ontario between December 2010 and March 2012. the Ministry, working with the Ministry of Govern- The Ministry further noted that the Network and ment Services and ServiceOntario, made online 396 2012 Annual Report of the Office of the Auditor General of Ontario

registration accessible through mobile telephone The Network indicated that its organ-specific and computing devices in April 2012. As well, the working groups identified consistent practices for Network is working on an electronic “grass roots” adding patients to transplant wait lists, and are social media campaign, whereby interested individ- working on standardized referral criteria. Once uals can encourage others to register their consent referral criteria are developed for all organs, the to be organ and tissue donors. Network plans to develop a communication and implementation strategy, and expects to begin communicating with referring physicians by the end of ORGAN WAIT-LISTS the 2012/13 fiscal year. Recommendation 3 The Network noted that its organ-specific work- To enhance its management of the wait-lists for organ ing groups were developing standardized processes transplants, the Trillium Gift of Life Network (Net- to remove patients from wait lists, including work), in conjunction with transplant hospitals and documentation of the reasons for removal, and it physicians, should: expected to have the processes for all organ wait • develop target time frames for provincial lists in place by the end of the 2012/13 fiscal year. priority rating scales for organ transplants, as Once that is done, the Network said it plans to peri- recommended by the Ministry of Health and odically review on a hospital-by-hospital basis the Long-Term Care’s Organ and Tissue Trans- number of patients removed from wait lists because plantation Wait Times Expert Panel; they die or become too ill for a transplant to see if • determine the best way to communicate referral there are any ways to minimize the incidence of criteria to non-transplant physicians, so that such cases. individuals who would benefit from a transplant (including from a quality-of-life perspective) are ALLOCATION OF ORGANS added to the wait-list; and • require hospitals to enter on the Network’s Recommendation 4 system the reason for taking a patient off the To better ensure that organs are allocated in an effi- wait-list, and periodically review, by hospital, cient and equitable manner, the Trillium Gift of Life the number of patients removed from the Network (Network) should: wait-list because they die or become too ill for a in conjunction with the transplant hospitals, 4.10 • transplant, to determine whether actions can be review kidney and liver allocations, with a view taken to minimize the incidence of such cases. to having one province-wide wait-list (rather Section

than up to five regional wait-lists) for each Status organ, so that the highest-priority patient in the At the time of our follow-up, the Network said it Follow-up province, based on clinical evidence, receives the • had initiated a review of national and global prac- 4 first suitable organ available, and transplant tices for establishing target wait times for organ program sustainability is maintained; transplants. In conjunction with clinical experts,

Chapter have periodic independent reviews conducted the Network plans to use this information to help • of organ allocations, to ensure that either the determine by fall 2013 the appropriate use of highest-priority compatible patient received the wait-time information for transplant patients. The organ or there was a valid reason for allocating Network noted that the development of target wait the organ to another patient; and times also requires standardized organ-transplant provide information to the eight transplant listing criteria and organ-allocation practices. • hospitals on organs made available but not accepted by them, so that the Network and the Organ and Tissue Donation and Transplantation 397

hospitals can monitor the acceptance rates and • determine the feasibility of providing transplant determine whether any changes are needed to hospitals with simultaneous electronic access to the process for offering and accepting organs. information required to facilitate the physician’s assessment of the compatibility of the donor and Status a potential recipient, such as the donor’s labora- At the time of our follow-up, the Network indicated tory test results; that a single wait list for liver transplants was being review the costs and benefits of implementing developed and would likely be implemented by • a system capable of tracking the information the end of summer 2012. With respect to persons required to oversee the organ donation process, waiting for kidney transplants, a working group including the time taken for each stage of the was evaluating the effect of sharing kidneys among donation process from identification of the the five regions and had agreed in principal to potential donor to the time of transplant (com- move toward a single province-wide wait list. The pared against target times), and the reasons for Network expected to have in place by winter 2013 any delays; and a system that alternates between allocating one review research on current best practices with kidney locally and one provincially. • respect to the use of kidney pumps when trans- With respect to conducting independent reviews porting donated kidneys to transplant hospitals to ensure that either the highest-priority patient and track the use of such pumps. receives the organ, or a valid reason is identified Further, the Ministry of Health and Long-Term for allocating it to another patient, the Network Care should review its agreement with the air ambu- noted that a quality committee would be formed to lance provider, Ornge, and, in conjunction with the consider a system for periodically reviewing organ Network, clarify Ornge’s transportation responsibil- allocation. The Network expected this process to ities with respect to organ transplantation. be established by the end of the 2012/13 fiscal year. In the interim, the Network stated that it has Status improved its chart documentation to better identify At the time of our follow-up, the Network indicated the rationale for allocating an organ to a patient that it had conducted a competitive process and who does not have the highest priority, and is con- expected to have a more comprehensive donor- ducting internal audits of organ allocations. management system in place by the end of the 4.10 The Network expected that changes to its 2012/13 fiscal year. Once implemented, the system current database to track the number of organs will enable the sharing of donor data via email with Section accepted and declined by the eight transplant transplant hospitals, which helps assess the medical hospitals would be in place in the 2013/14 fiscal suitability of a donor organ. The Network antici- year. The Network indicated that, once available, pated that further work will be done in the 2013/14 Follow-up

• it planned to share this information with all trans- fiscal year to electronically share donor information 4 plant hospitals. with the transplant programs at subsequent stages of the organ donation process. The Network expected that the new donor Chapter EFFICIENCY OF THE ORGAN DONATION management system would also facilitate the PROCESS measurement and reporting of more indicators, Recommendation 5 including time spent on each stage of the donation To improve the efficiency of the organ donation process, from identification of a potential donor to process and avoid delays that may harm the viability the time of transplant, and reasons for any delays. of donated organs, the Trillium Gift of Life Network (Network) should: 398 2012 Annual Report of the Office of the Auditor General of Ontario

The Network said that it completed its review is recovered, processed, and stored safely and of best practices for kidney-pump use and as of efficiently; and March 31, 2012, all transplant hospitals have • consider whether specific funding should be agreed to use the pumps to preserve kidneys from provided to offset the costs incurred by hospitals deceased donors. The Network has also begun and to compensate physicians for their time tracking use of the devices, and has identified pre- with respect to tissue donation and banking. liminary measures for monitoring and evaluating Status their use, which will be discussed at future meet- At the time of our follow-up, the Network said that ings of its Kidney/Pancreas Working Group. the additional hospitals required to refer potential The Ministry stated that the Network and Ornge organ donors to the Network also have to refer air ambulance are developing a formal service potential tissue donors to the Network. As well, agreement to standardize processes for organ the Network stated that it requires reporting of all transport. The Ministry expected a draft agreement deaths occurring at designated hospitals, but that it would be ready for its review in January 2013. would consider in the 2012/13 fiscal year the implications of moving to more specific clinical triggers TISSUE for reporting potential tissue donors. The Network planned to establish a Tissue Working Group, which Recommendation 6 would begin meeting in fall 2012, to increase tissue To help ensure that there is an adequate supply of donation through effective donation, recovery and quality tissue, such as bones and eyes, to meet the banking practices. needs of Ontarians and reduce reliance on tissue With respect to obtaining consent for tissue purchased from other jurisdictions, the Trillium Gift donation, the Network tracked the consent rates for of Life Network (Network) should: its staff and hospital staff in 2011, and found that increase the number of hospitals required to • Network staff obtained an average consent rate of report potential tissue donors to the Network 50% while untrained hospital staff obtained 4%. and, in conjunction with the hospitals, develop The Network said it shared this information with more specific clinical triggers (such as age the hospitals to emphasize the importance of hav- criteria) to help hospitals determine which ing the Network approach families for consent. The patients should be referred to the Network as 4.10 Network indicated that in the 2011/12 fiscal year, it potential tissue donors; approached almost 75% of potential tissue donors, review the process of obtaining consent for tissue Section • while hospitals spoke to the rest. donation, in conjunction with the hospitals, At the time of our follow-up, the Network noted with a view to increasing consent rates; and that it was working with various Ontario tissue Follow-up reassess, in conjunction with the tissue banks, • • banks to define and implement “exclusion criteria” 4 the screening processes used to determine tissue to help identify tissue that is not viable and there- viability so that non-viable tissue is identified as fore should not be referred for tissue donation.

Chapter quickly as possible. The Network anticipates that this will streamline Further, the Ministry of Health and Long-Term workload and ensure that only relevant referrals Care, in conjunction with the Network and the tissue are made to the tissue banks. banks, should: The Ministry indicated that it was reviewing, in assess the costs and benefits of implementing • conjunction with the Network and other provinces a centralized tissue bank, which would help and territories, the April 2011 national plan of ensure that, after consent is received, tissue the Canadian Blood Services for organ and tissue Organ and Tissue Donation and Transplantation 399 donation and transplantation. The plan recom- Status mends a restructured tissue system, including At the time of our follow-up, the Ministry stated nationally centralized tissue banking. At the time that oversight of transplant activities in Ontario of our follow-up, the Ministry expected that an was a collaborative effort between the Ministry interprovincial–territorial response to this plan and the Network. In this regard, the Network would be completed by fall 2012, and therefore it established donation and transplant steering com- was not planning to otherwise assess the costs and mittees, as well as organ-specific working groups, benefits of a centralized tissue bank. to help implement an integrated structure for The Ministry said it met with the Network and effective management of organ and tissue donation physicians who recover tissue donations to discuss and transplantation in Ontario. The Ministry also reimbursements to physicians performing tissue indicated that, in conjunction with the Network, recovery. The Ministry concluded that the current it is reviewing the April 2011 national plan of the physician payment schedule included sufficient fees Canadian Blood Services for organ and tissue dona- for tissue recovery and that it, the Network and the tion and transplantation. The plan includes recom- Ontario Medical Association had to be more pro- mendations related to the oversight of donation active in making physicians aware of these fees. The and transplantation, and the Ministry expected that Ministry indicated that hospital funding for tissue an interprovincial–territorial response to this plan recovery may need to be considered in future as the would be completed by fall 2012. Health System Funding Strategy is implemented. At the time of our follow-up, the Network’s website had data on organ transplant wait lists as well as on the number of patients who received a PERFORMANCE MONITORING transplant, broken down by categories such as age Recommendation 7 and gender. The Ministry noted that the Network is To provide additional assurance that organ and tissue consulting with transplant hospitals to establish the transplantation in Ontario is meeting the needs of means and timelines for publicly reporting hospital- patients safely and efficiently, the Ministry of Health based transplant statistics, including the number of and Long-Term Care (Ministry), in conjunction transplants performed by each hospital. However, with key stakeholders, including the Trillium Gift of the Ministry indicated that public reporting of other Life Network, transplant hospitals, and transplant data, such as wait time for transplant by organ 4.10 physicians, should determine the best structure for type and life expectancy after transplant, would providing effective oversight for organ and tissue require more time because of factors that included Section transplantation in Ontario, as recommended in the need to develop consistent data definitions and the 2009 report of the Ministry’s Organ and Tissue implement information technology changes. In Follow-up Transplantation Wait Times Expert Panel. As well, this regard, the Ministry noted that the Network’s •

4 performance indicators for transplant activity in Transplant Steering Committee and its organ- Ontario—such as wait times for transplant by organ, specific working groups were developing standard

number of transplants performed by hospital and definitions for wait times that it expected would be Chapter patient survival rates by hospital—should be estab- drafted by spring 2013. lished and made publicly available. Chapter 4 Ministry of Education Section 4.11 School Safety

Follow-up to VFM Section 3.11, 2010 Annual Report

concluded that better information on the success Background of its initiatives would help the Ministry to allocate funding to the areas of greatest need. Some of our other key observations were as follows: A learning environment that is not physically and The Ministry allocated $34 million—about psychologically safe can adversely affect not only • two-thirds of its total annual school safety a student’s safety but also his or her motivation to funding—to two initiatives focused on learn. The impact of bullying, for example, can be assisting suspended, expelled, and other high- severe: victims may have to deal with such issues risk students. Most of this funding was allo- as social anxiety, loneliness, physical ailments, low cated based on total board enrolment rather self-esteem, absenteeism, diminished academic than on more targeted factors such as the performance, depression and, in extreme cases, actual number of students needing assistance. thoughts of suicide. A 2009 survey of Ontario The percentage of students that had been students in Grades 7 through 12 by the Centre for • suspended in each board ranged from 1% to Addiction and Mental Health identified that almost more than 11% of the student population, one in three students has been bullied at school and which may explain why some boards under- approximately one-quarter of students have bullied utilized their funding by as much as 70%. A 4.11 others at school. comparison of provincial and school board Our audit in 2010 focused on the adequacy of data on suspension rates to a recent anonym- Section activities undertaken by the Ministry of Education ous provincial survey of students suggests, (Ministry) and selected school boards (Durham however, that school administrators were not District School Board, Sudbury Catholic District Follow-up aware of the extent of serious safety issues • School Board and Toronto District School Board) 4 in some schools, such as the incidence of to improve the safety of Ontario schools. Our students being threatened or injured with a work indicated that a number of initiatives had

Chapter weapon. Most senior safety staff at the school been taken to address safety issues in Ontario’s boards we visited, as well as administra- schools. These initiatives included new legislation, tors at the schools we visited, said that the teacher training and targeted school safety funding. discrepancy was due to a lack of reporting by However, neither the Ministry nor the three school students, possibly because of fear of reprisals, boards or various schools we visited were collecting and that more needed to be done to facilitate sufficient information on whether these initiatives student reporting of incidents. were having an impact on student behaviour. We

400 School Safety 401

• The Ministry had established requirements across the province for Ontario’s publicly funded for school boards and schools pertaining to boards in March 2011 and again in spring 2012. the application of escalating discipline for These regional sessions focused on strategies for students who had repeatedly violated school maintaining consistency in enforcing suspensions safety policies. Despite significant differences and expulsions, sharing best practices, and dealing in suspension rates among boards and among with the challenges in designing and implementing schools we visited, neither the Ministry nor school safety programs. Also of significance is the the boards we visited had formally analyzed passing of the Accepting Schools Act, 2012 in June the differences in suspension rates to assess 2012. This Act encourages stronger actions to create whether progressive discipline policies were a safe and inclusive environment in all schools. being applied consistently by Ontario’s 72 The status of actions taken on each of our recom- publicly funded school boards. mendations is described in the following sections. • An evaluation of a program that stations police officers in schools identified an SCHOOL SAFETY INITIATIVES improvement in relationships between students and police. The majority of school Recommendation 1 administrators we interviewed indicated To ensure that school safety funding is used effectively that having an officer in the school improved to achieve program goals to improve school safety, the school safety and that expansion of such pro- Ministry of Education and, where appropriate, school grams should be considered. boards should: We made a number of recommendations for • reconsider the appropriateness of allocating, on improvement and received commitments from the the basis of enrolment, the majority of school Ministry and the school boards that they would safety funding primarily to assist suspended, take action to address our concerns. expelled, and other high-risk students, given that the ratio of such students to total enrolment may vary significantly among school boards; • for other specific program funding, ensure that Status of Actions Taken on the funds are allocated based on identified needs and follow up to verify that the funds provided 4.11 Recommendations are being spent for the intended purpose; and obtain and share information on the success Section According to information received from the Min- • of initiatives such as Student Support Leader- istry and each of the three school boards we visited ship and police officer placements in schools, during the audit, progress has been made on imple- Follow-up and determine whether a more significant co- • menting all of the recommendations in our 2010 4 ordinating role for the Ministry is appropriate Annual Report, with substantial progress made on to enhance their effectiveness. several. For example, the Ministry now monitors specific program funding through progress reports Status Chapter received from boards and by analyzing data to While the main focus of school safety funding has assess activities and how funds were spent to sup- been to provide supports for students who have been port school safety initiatives. In addition, as part expelled or given long-term suspensions, funds can of the initial investigation into the variations in be used for a variety of programs, including those suspension rates within boards and across the prov- to support at-risk students who exhibit behaviours ince, the Ministry held eight regional data sessions that might lead to suspension or expulsion. The 402 2012 Annual Report of the Office of the Auditor General of Ontario

Ministry noted that it continues to work with boards • hosting the Urban and Priority High Schools and its other partners to identify student needs and Symposium for participating schools to share fund specific initiatives to address those needs. The information on leading practices and chal- three boards we visited in 2010 informed us that lenges (the Ministry intends to produce an ministry funding was allocated to each school based annual newsletter related to this symposium); on factors such as the overall board strategy, board- • developing and distributing Promoting a supported initiatives and individual school needs. Positive School Climate, which provides an However, the Ministry acknowledged that more evidence-based list of practical ideas for Safe consistent, reliable information is required to ensure Schools Teams to consider in their efforts to that funds are appropriately allocated on the basis of develop and maintain a positive climate for student needs. In this regard, the Ministry has been students; and engaged in extensive activities designed to enhance • establishing a safe schools working group that the quality of student information. now holds regular teleconferences. The Ministry requires detailed reporting on The Ministry also worked with the Ministry of all safe schools initiatives on an ongoing basis Community Safety and Correctional Services and that outlines, in addition to financial information, police services partners to negotiate and release a program activities and outcomes. It monitors revised Provincial Model of the Police/School Board specific program funding through progress reports Protocol in January 2011. The Ministry has helped received from boards and by analyzing data to boards form partnerships with police services by assess program activities and how funds were spent providing one-time funding for joint training on the to support school safety initiatives. The Ministry is revised protocol. In addition, the Ministry of Com- examining the possibility of conducting spot audits munity Safety and Correctional Services provided to further ensure that the funds provided for these $1.68 million for a Safe Schools Grants program initiatives are spent for the purposes intended. intended to: The Ministry supports partnerships between • encourage police services, local schools and school boards and other community groups boards to collaborate on various activities; through its Student Support Leadership Initiative. • create teams of police, educators, counsellors, This initiative helps clusters of school boards and health-care professionals, parents and stu- community agencies provide supports outside the dents that will work together to help reduce 4.11

classroom that promote positive student behaviour. violence and bullying; and Focusing on the prevention of at-risk behaviours in help police officers become more active and Section • 2011/12, the clusters invited leaders of municipal engaged in school-related activities. services that offer services for children and youth, The boards we visited during our 2010 audit Follow-up such as parks and recreation and public health, provided further detail on their delivery of the •

4 to participate. These clusters submitted mid-year Student Support Leadership and other initiatives to reports noting their accomplishments and will sub- encourage continued sharing and implementation

Chapter mit final reports in fall 2012. Clusters are also com- of best practices. One board noted that, to encour- pleting a self-evaluation of their 2011/12 activities age the sharing of best practices across cluster that will help them to continue with their activities members, a number of committees were estab- once the financial support from the Student Sup- lished that met regularly throughout the school port Leadership Initiative comes to an end. year, including a collaborative practices committee Since 2010, the Ministry has gathered and and a committee to plan and hold partnership shared information on the success of school safety symposiums. Another board noted that it provided initiatives through a variety of other means, such as: release time for its Safe Schools Teams to meet School Safety 403 and analyze surveys to identify and address issues data to assess board activities and how funds were of concern. The third board initiated a Positive spent to support school safety. Space campaign where each school is to provide a In the 2010/11 school year, the Ministry began designated room or private area within the school distributing $5 million annually to fund a regionally where students can speak freely to an adult repre- based school board audit function to help ensure sentative. By June 2012, one representative from compliance with school safety legislation, policies each school was to be trained as a Positive Space and initiatives. As of January 31, 2011, audit com- representative. mittees had been established in all boards. The Ministry developed a risk assessment tool, and eight regional internal audit teams were expected MONITORING COMPLIANCE WITH to complete a risk assessment using this tool by the SCHOOL SAFETY REQUIREMENTS end of the 2011/12 school year. As of April 2012, Recommendation 2 some teams had already completed their risk To promote compliance with all school safety legisla- assessment. All school boards were expected to tion and policies designed to provide a safe learning have internal audits performed in the 2011/12 environment for Ontario students, the Ministry of school year, some of which could have been audits Education should work with school boards to: of school safety initiatives if the region’s risk assess- • monitor compliance with required school safety ment determined this area to be a priority. legislation and ministry policies; A memo was sent to all Directors of Education in • ensure that schools have functioning Safe School October 2010 reminding boards of the requirement Teams in place that include representation from to have a functioning Safe Schools Team in place all required groups; with appropriate membership, including the school • investigate significant differences in suspension principal and at least one student, parent, teacher, rates between school boards and schools to non-teaching staff member and community partner. assess whether such differences are reasonable In November 2010, the Ministry created the Pre- and to determine whether additional student mier’s Safe Schools Awards in order to increase the disciplinary guidance is necessary to ensure a profile of these teams. For the 2010/11 school year, reasonable level of consistency across the prov- 10 schools were recognized for the exceptional and ince; and innovative work they were doing to create a safe 4.11

• assess whether requiring periodic updates to and caring environment. The Ministry is developing criminal background checks for school staff, various ways to showcase Safe Schools Teams that Section

service providers, and volunteers would enhance have received a Premier’s Award, highlighting the safety of students in Ontario’s schools. their accomplishments and best practices. Also, the Follow-up Ministry noted that, through a safe schools working • Status 4 group teleconference in June 2011, boards shared a The Ministry informed us that it promotes compli- number of best practices. One of these practices is ance with school safety legislation, policies and

to require schools to report team membership to the Chapter initiatives in a variety of ways, including ongoing school board, to help ensure the establishment and discussions with school boards on reporting, proper composition of Safe Schools Teams. monitoring and data-based decision-making. With One board reported that it has added a require- ongoing dialogue and requirements to report on ment that an equity representative be added to every specific initiatives, the Ministry expects that boards Safe Schools Team to help eliminate biases and will continue to improve their practices over time. barriers that may be related to gender, race, religion, The Ministry monitors compliance by analyzing socio-economic background and other factors. For 404 2012 Annual Report of the Office of the Auditor General of Ontario

accountability reasons, the schools report the name area superintendants and school principals, and, of the team chair to the board. Also, this board in camera, to board trustees, for monitoring and advised us that it provides training for Safe Schools decision-making purposes. Teams on topics such as creating a positive school A regulation under the Education Act requires climate, its protocol for police–school board partner- school boards to collect criminal background checks ships and the use of data to help develop a safety and for all employees and, with the exception of school bullying prevention plan. bus drivers, for service providers who have direct In 2011, the Ministry completed an in-depth and regular contact with students. School bus driv- analysis of school board data by examining suspen- ers do require criminal background checks, how- sion and expulsion rates by school and by board ever, under the Highway Traffic Act. The Ministry and including such student attributes as gender, of Transportation is expected to remove the licens- infraction type, length of suspension and the num- ing requirement for school bus drivers to obtain ber of repeat suspensions. The Ministry found, for criminal background checks under the Highway example, that boys were three times more likely Traffic Act effective July 2013. Before that time, the to be suspended than girls and more than 80% of Ministry of Education plans to amend its regulation suspensions were coded as “other,” which includes under the Education Act to remove the exemption such dissimilar violations as fighting, being under of school bus drivers from obtaining criminal the influence of drugs, committing uniform infrac- background checks. In addition to the requirements tions and being late for school. under the regulations, one board noted that it now As part of the initial analysis of the variations requires criminal background checks for all volun- in suspension rates within boards and across the teers, with an annual declaration and an updated province, the Ministry invited each publicly funded criminal background check at least every five years. board to one of eight regional data sessions held Another board advised us that it was in consultation across the province in March 2011. In-depth analy- with its local police force to determine if ongoing sis of suspension and expulsion data continues updating would enhance safety for students. and subsequent regional sessions were held in spring 2012. The goals of these sessions, which the MEASURING AND REPORTING ON Ministry intends to hold annually, were to work SCHOOL SAFETY toward a more consistent application of legislation, 4.11

regulations and policies in schools and boards and, Recommendation 3 over time, reduce the differences in suspension and To help in its efforts to ensure that students are edu- Section

expulsion rates for students with similar types of cated in a safe environment, the Ministry of Educa- inappropriate behaviour. tion should work with school boards to: Follow-up Discussions at the regional sessions focused on develop measurable objectives and related •

• 4 strategies for sharing best practices and dealing performance indicators for activities intended to with the challenges in designing and implementing improve school safety, and periodically measure

Chapter programs for suspended and expelled students, progress in achieving these objectives; and on strategies for generating and maintaining • capture data on incidents of inappropriate consistency among school staff in enforcing disci- student behaviour and complaints received, in plinary procedures. For example, one board noted addition to the information currently collected that it collects suspension rates by school, by area on suspensions and expulsions, to support the and for the whole board mid-year and at the end assessment of existing initiatives and identify of the school year. This data is then provided to areas on which to focus future efforts; School Safety 405

• conduct school safety surveys to gauge the Also, in May 2011, the Ministry informed boards progress achieved in improving school safety at that they would be required to report data on the provincial and school board levels; and violent incidents to the Ministry beginning in the • review existing best practices in Ontario and 2011/12 school year. The collection of data on elsewhere that have been found to be effective in student participation in these programs and on encouraging students to report serious school incidents will form the basis for the measurement safety incidents. of some of the proposed performance indicators for the Safe Schools Strategy. Status In November 2010, the Ministry spent $3.1 mil- The Ministry engaged a consultant to develop an lion to develop and distribute resource guides to evaluation framework of indicators for its Safe promote a positive school climate. Receipt of this Schools Strategy. The consultant’s February 2011 funding was contingent upon the school’s having report to the Ministry included proposed perform- undertaken a survey to assess whether the climate ance indicators and data collection strategies. The in the school is inclusive and accepting of all pupils, model aims to establish interim accomplishments including pupils of any race, gender, creed, sexual that can indicate whether a program is mak- orientation or disability. ing progress in the desired direction, given that All three boards reported on the use of school the ultimate goal is likely to take many years to surveys. One board noted that a pilot survey was accomplish. The consultant based the proposed given to a number of schools in the spring of 2011 performance indicators on research that included and then extended to all of its schools in the fall. current practices in Ontario and other national and The boards noted that the surveys were used for international jurisdictions, as well as best practices multiple purposes, such as to identify areas of in performance measurement. concern that needed to be addressed, to support The Ministry stated that it is committed to the development of safety and bullying prevention implementing a comprehensive evaluation frame- plans and to assist in the understanding of the work to measure the effectiveness of safe schools school climate in order to improve program plan- policies and programs, which would include ning at the school level. indicators of students’ and parents’ perceptions of The Ministry was also involved in the develop- school safety. To obtain the data to assess these per- ment of legislation that requires board multi-year 4.11 ceptions, the Ministry plans to engage an outside plans to include goals that promote a positive agency to collect and analyze school survey data, school climate and bullying prevention. The legisla- Section starting in the 2012/13 school year, on students’ tion also requires each board to establish a bullying and parents’ perceptions of school safety. prevention plan that will include procedures for The Ministry noted that collecting and analyzing Follow-up reporting the incidence of bullying and the range • aggregate data at the provincial level helps it to 4 of disciplinary actions a principal may take against make legislative, policy and program decisions. The a student involved in bullying. The legislation also Ministry also noted that data collected at the board

establishes strategies for protecting a person who Chapter and school level helps boards and schools make witnesses an act of bullying, reports bullying or pro- local decisions on programs and implementation vides information during a bullying investigation. strategies that best suit their specific communities. This legislation, the Accepting Schools Act, 2012, In this regard, the Ministry informed us that it which amended the Education Act, received royal made amendments to its OnSIS database to capture assent in June 2012. more data from schools on student participation in programs for suspended and expelled students. 406 2012 Annual Report of the Office of the Auditor General of Ontario

SCHOOL SAFETY TRAINING and the analysis of various scenarios are effective methods of training staff within boards and Recommendation 4 schools. In May 2011, the Ministry also funded To build on the steps taken to date to ensure that Toronto Police Services to produce a training video school staff are adequately trained to deal with school on the revised police–school board protocol and safety issues, the Ministry of Education should work provided funding for joint training of school board with school boards to assess whether school safety and police services staff. training delegated to schools is of sufficient depth to All three boards we visited during our audit meet the needs of school staff. advised us that they had systems in place to track Status and provide professional development oppor- The Ministry informed us that, in consultation with tunities for board and school staff. One board school boards, it continues to build on the training developed a central tracking tool that staff use to that it has provided over the last several years with register for professional learning opportunities respect to safe and inclusive schools. The Ministry found within the board. A second board noted that also noted that it is working closely with boards to it provides its schools with consistent school safety meet the needs that boards identify in the areas of training packages related to various initiatives. The training and staff development. third board noted that it provides ongoing training The Ministry continues to share, and create and resources to the Safe Schools Teams in order to opportunities to share, best practices on how to support the planning that is being done with school train staff on an ongoing basis. For example, at climate surveys to extend the development of a the regional sessions it was noted that e-learning positive climate within its schools. 4.11

Section

Follow-up

•

Chapter to ensure Act. adherence to the workon the we have doneover past fiscal year the annually LegislativeAct to to report Assembly the Act aswell Auditorments inthe General asinthe Act canbefoundthe atwww.e-laws.gov.on.ca . standards. Thefulltext anditsother of advertising prohibition onpartisan Act’s comply the they with to ensure that beforerun advertisements they toGeneral review andapprove most government government. TheAct Auditorthe mandates the tive impression of critical ofany orperson group a positive impression government oranega ofthe - politicalinterests governing by party’s the fostering couldbeseenaspromoting that ment advertising interests. its partisan could beconsidered to further that on advertising ofagovernmentpriateness spendingpublicfunds legislators appro- questioned to that—as the prior Legislature—and severalin the years ofdiscussion two inDecember2005after yearseffect ofdebate and Internet, billboards andinnewspapers the radio, year toeach ontelevision, placeadvertisements advertiser, spendingtens ofmillionsdollars is a majorThe government ofOntario Canadian Advertising Review of Government Chapter 5 Chapter This chapter satisfies the legislativeThis chapter satisfiesrequire- ActThe mainintent isto ofthe prohibit govern- (Act) Act,2004 Advertising tookThe Government

magazines. Advertising Review ofGovernment later in this chapter). later inthis Factors” “Other (see the governing section party of anitem interests isto promoteofthe partisan the objective factors aprimary indetermining whether to consideradditional authority discretionary the governingitical interests party.” ofthe itemobjective pol - isto ofthe promote partisan the Auditor opinionofthe aprimary General, in the if, must “anitem meet ispartisan andstates that to have using to distributed households inOntario agovernmentmatter officeproposes that to pay ortelevision.on radio Italsoappliesto printed magazine, displayed onabillboard, orbroadcast pose to pay to have publishedinanewspaper or the Premier—pro- the Officeof Cabinet Officeand ment offices—specifically, ministries, government governThe Act appliesto- that advertisements in partisan or couldbeinterpreted asbeing,primarily Above is, must not that anything all,they contain tisements to meet ensure legislated they standards. for reviewing specifiedtypesofgovernment adver AuditorUnder Act, the isresponsible General the Review Function of the Advertising Overview WHAT FALLS UNDER THE ACT

nature. The Act alsoprovides Auditor the with General advertisement The Act outlinesstandards each - 407

Chapter 5 Chapter 5 408 impression that the ad hasbecomepartisan. impression the that if we new leave circumstances determine that the year, my although Officecan rescind an approval final. Auditor Act, alldecisionsofthe are General the Office hasseven days to render adecision. Under review.another submission, first my the Aswith rejectedsubmit arevised version item ofthe for not beused.However, government officemay the item item may doesnot standards, the the meet the may berun. item Act, andthe item standards ofthe meets the office isdeemed to havethe receivedthat notice government time,the not give this notice within seven businessdays to render its decision.Ifwe do hasbeenapproved. advertisement the notice, that receives notice, orisdeemedto have received the deputyminister, office,usually headofthat the submitted ordisseminate itemdistribute the until ment officecannot publish,display, broadcast, reviewableevery item to ourOffice.Thegovern- the Act:the are not they subjectto government that the with excluded, we have cometo amutualunderstanding excluded. orsafety, publichealth matters affecting are also vices tothose onurgent agovernment office, and provision onthe ofgoodsandser Advertisements ments andnotices publicrequired to the by law. before canrun. they be submitted to my Office for review andapproval initions are known as“reviewable” items andmust delivery. meeting Advertisements def- any ofthese ofbulk method unaddressed bulkmailoranother 2012 oftheOffice ofthe Annual Report Auditor General of Ontario • • Approval isvalidfor ofanadvertisement one If my that the Officenotifies office government The Auditor Office, General’s by regulation, has The Act requires government offices to submit Although the following the are notAlthough specifically The Act excludes from review- jobadvertise ertising; and ertising; newsletters,ochures, docu consultation publications. or similarmaterials andother ments, reports br online adv - - meets the standards of the Act. standards ofthe meets the soweished form canreview itstill itto ensure that government officemust submititinfin- used, the office. However, canbe the advertisement before Act, we governmentstandards ofthe soadvisethe government the office.Ifitappears the to meet to Act, violate we the provide to anexplanation Act. the requirements of statutory andisoutsidethe part onour Act. Apre-reviewthe voluntary isstrictly couldbedeemedobjectionableunder that material time andmoney spentto develop adscontaining appear whencompleted. Pre-reviews helplimitthe the itemit reasonably asitisintended reflects to provided orstoryboard, ad. Thiscanbeascript that offices wishingus anearlyexamine to ofan version ment meets the standards of the Act, which are: Act, which standards ofthe ment meets the - the proposed advertise Office determines whether In conductingitsreview, Auditor the General’s ADVERTISEMENTS PROPOSED FOR STANDARDS • • • If material submittedIf material for pre-review appears A pre-review isalsoavailable to government em must not include the name,voiceem must not or includethe itis em must includeastatement that em must beareasonable meansof • • • • Assembly (unless the primary target audience primary Assembly (unlessthe Legislative(cabinet) oramemberofthe Executiveimage ofamemberthe Council paid for government by ofOntario. the objectives: following oneormore ofthe achieving The it The it The it o promote Ontario, or any part of the prov ofthe oranyo promote part Ontario, o encourage ordiscourage specificsocial and publicofitsrights the o inform orproposed publicofcurrent the o inform activity or sector of Ontario’s economy.activity orsector of Ontario’s study orvisit,to promote any economic ince, asagoodplaceto live, work, invest, t t t t behaviour in the publicinterest;behaviour inthe and/or law;responsibilities underthe orservices; government policies,programs - certain undesirable ones,asfollows: undesirable certain andavoids attributes desirable it includescertain to beperceived. isgiven Consideration to whether impression conveyed adandhow by the itislikely additionalfactors relate overall these toeral, the governing interests party. ofthe partisan Ingen- objective ofanitem isto promote a primary the consider additionalfactors to determine whether above, Act allows Auditor the the to General standards specific statutory In additionto the OTHER FACTORSOTHER • • • • ems should not: h item should: is,inthe that em must not bepartisan; objectiveem must not have aprimary • • • • • • partisan interests of the governing interests party. ofthe partisan promotion objective ofthe the as aprimary Auditoropinion ofthe itcannot General, have government. ofthe orentitycritical person governing party, oranegative impression ofa of fostering apositive impression ofthe item isexempt requirement). from this casethe inwhich is located outsideOntario, It Eac The it The it ain subjectmatter relevant to govern opriately personalize (foropriately personalize instance, he audienceto distinguish between phasize factsand/orexplanations, objectively,esent information intone and monly associated with the governing party; governing party; monly the associated with dealt with in the item); inthe dealt with specificmatters responsibilities for the ment shouldhave direct andsubstantial govern- is,the ment responsibilities (that use colour enable t em pr cont inappr by attacking opponentsorcritics); by attacking other.or analysis onthe onehandandcomment,opinion fact onthe als; and ofpropos- politicalmerits the than rather language; accurately, usingunbiasedandobjective clearly factsexpressed content, with and s, logosand/orslogans com - - which casewe page.which next review We the examine isa gateway click first unlessthat page, in click, review. We first content consideronly ofthe the cited inareviewable item would beincludedinour page, first ofa or“click,” the website that agreement government in2005,we the cametosions with an ad.Followingtisement isanextension ofthe discus- Response Codeorsimilar linkage usedinanadver Act, weunder the believeawebsite, that Quick websitesAlthough are not specifically reviewable the clarification overyears. have mainareas that required ofthe description Act issilent.Whatfollowsareas where the isabrief clarify, government offices, with inco-operation government my advertising, to Officehastried review onresponsibility forSince taking of the Websites REVIEW PROTOCOLS OTHER • • • • • • esent pre-existing policies,products, asapolicyproposal h matters such orimage- er self-congratulatory entionally promote, orbeperceived as criti ridicule, ectly orindirectly attack, orm resource locatororm (URL)to of those critical of the government; ofthe critical of those views, policiesoractions cize orrebut the pr deal wit deliv int dir use aunif services or activities as if they were oractivitiesasifthey new; or services the disadvantages; benefitsand the both item providesthe of abalancedexplanation where nodecisionhasbeenmade,unless building messages; message willbecommunicated);the overallat andthe environment inwhich audienceitisaimed message, the ing ofthe isalsogivenend, consideration tim- to the interestspromoting, (to political-party this following section). Act (see“Websites”standards ofthe inthe may contentpage not that meet with the direct viewers readers, orlisteners to aweb Review ofGovernment Advertising - - 409

Chapter 5 Chapter 5 410 events as conferences such shows. andtrade Our atpublic distributed materials andother programs Government sometimes advertisements appearin office. ernment gov controland canattimesbebeyondofthe the networks content onthese changes frequentlythat references.are However, nopartisan we recognize cited there inanadtomedia channel ensure that aninitialscanofany social- our Officewillperform government that the we with anagreement reached Act issilentonthis, the social-media sites. Although presence on government’s icons pointingto the time,ourOfficefirst received ads for approval with exponentially inrecent years. Thisyear, for the into in2005.However, effect its usehasgrown Act came Social mediawas initsinfancywhenthe following ofthe criteria: three tising must besubmitted for review ifitmeets all adverthird-party and mythat Officehave agreed sometimes usedfor Thegovernment advertising. Government are fundsprovided toparties third policiesoropinionsofothers. the messages orany attacks contentcongratulatory that nameorphoto,not includeaminister’s any self- Act. Forstandards ofthe page example, must the page forthis any may content not that meet the 2012 oftheOffice ofthe Annual Report Auditor General of Ontario Event/Conference Program Advertisements Program Event/Conference Social Media Advertising Third-party • • • he government officeapprovesthe content of permis party third the he government grants third party officeprovidesthe vernment provincialthe item; and visualidentifierin official logooranother Ontario sion to usethe the item. the item; the distributing cost ofpublishing,displaying, broadcasting or fundsintended to orallofthe with pay part t t a go - - - to my Office for review. campaignsgeneric recruitment must besubmitted and,asaresult, ourinterpretation with agreed campaigns.recruitment Thegovernment has government jobs,butnot to broad-ranging generic exemption asapplying to for advertising specific tisements from review. We have interpreted this As previously noted, Act excludes the jobadver mitted for review. items must besub- andthese approach, this with Thegovernment“free” advertisement. hasagreed government indirectly the paidfor the consider that the event.has financially sponsored therefore We government free ofcharge becausethe ads are run these media.Often, to print magazines andother similar apurpose serve Actject to becausethey the these adsshouldbesub- that the view Office took fiscal year: the 2011/12 to my provided Officeduring services review ofsubmissions.Thefollowing advisers to advisers assist usinthe a numberofexternal . Instead,tising Act,2004 my Officehasengaged Adverthe the Government requirementsto fulfill of Commissionertocan appointanAdvertising help Under Auditor Auditor General the Act,the General Government Recruitment Advertisements Government Recruitment Advertisements External Advisers • afe Engle (J.D.,L.L.M.)isaToronto lawyer working industry. advertising inthe andcommunicationswhile media, advertising acquired acomprehensive in background dards Council. Before studying law, Mr. Engle StanCanada, andasChairofitsAdvertising - side legal counselfor Standards Advertising not-for-profitthe out- sectors. Healso actsas for-profit ofclientsinthe diverse group and law formunications andentertainment a marketing,specializing inadvertising, com- R - - our review past year. ofgovernment this advertising that contained 11 contained that stage of adsatapreliminary 3.2 past fiscal year the was timeduring turnaround average the Nevertheless, workother priorities. adandon complexity ofthe depending onthe time required for areview anddecisioncanvary, seven of businessdays inall cases. Thelength $50millionlast year.than ing 1,082 atotal valueofmore individualads,with been. Thiscompares to 165 submissions,compris- submitted to ourOffice for review, butshouldhave hadnot been $202,000,that includes 38ads,worth atotal valueof$34.8 sions, with itemsindividual advertising in121 finalsubmis- 2011/12 the During fiscal year, we reviewed 565 RESULTS OFOURREVIEWS 2011/12 Review Activity,Advertising • • • These advisers providedThese advisers in valuableassistance We eightpre-review alsoexamined submissions We required gave the ourdecisionwithin

business days. ra is the former director former ofoper isthe ra uimy isacommunicationsconsultant han Rose isAssociate Professor of until hisretirement in2010. and overseeing ourreview responsibilities implementing review ouradvertising function ations inmy Office.He was instrumental national politics in print andtelevision. national politicsinprint ist, editor and andproducer covering Ontario - asajournal decadesofexperience three with and governments useadvertising. way politicalparties onthe inwhich articles inCanadaandanumberof ment advertising Professor Rose abookongovern haswritten - andCanadianpolitics. in politicaladvertising interests a leadingCanadianacademicwith Political Studies atQueen’s University. Heis John Sciar Joel R Jonat

million. This - quickly it. corrected Assembly.”cil oramemberofthe TheMinistry voice Executive orimage ofamemberthe Coun says name, Act, ads“must not which includethe government.” Thiscontravenes section6(1)3ofthe includedareferencepage “McGuinty that to the approved, approved achange to contained the web we Savings Bonds,which for hadearlier Ontario campaign we Act the after foundtion ofthe that 2011 to the same subjectprior election. similarvisualsonthe strikingly released anadwith ofOntario Party Liberal the ical wait timesafter andLong-Termof Health Care campaign onmed- werevideo andtelevisionofaMinistry adsthat part average submissionswas timefor these turnaround to complete areasonable time.The within them finished items. effort makeWe nonetheless every to are Act, secondinpriority they ments ofthe require- statutory andoutsidethe on ourpart development. Becausepre-reviews are voluntary 2011/12 fiscal year, we found two inviolation: 5.6 • • We ofFinance for aviola- Ministry alsocited the We alsorescinded approval digital for three Of all the finalsubmissionsthe receivedOf allthe in

business days. adio campaign availability promoting the int and radio campaign about agricultural campaignint andradio aboutagricultural ad andresubmitted it,andwe approved it. Long-Term Care subsequently reworked the and ofHealth infants.TheMinistry newborn 2011the medicalscreening of electiononthe to prior ran commercial that ofOntario Party rejected becauseitloosely resembled aLiberal was rotavirus of free vaccinationsfor the approved it. Food resubmitted andRural itandwe Affairs ofAgriculture, Ministry campaign, the ing the governing party.”of the quickly rework After objective to “fostermary apositive impression anad must not says have that which - asapri Act,because itviolated section6(1)5ofthe risk-management was rejected programs A r A pr Review ofGovernment Advertising - - 411

Chapter 5 Chapter 5 412 ted to usfor review. They are asfollows: having been submit first ran without tisements that 2012 oftheOffice ofthe Annual Report Auditor General of Ontario • • • • We alsonoted contraventions Act—adver ofthe try ofEconomicDevelopment and try ofCommunityandSocialServi try Attorney advised General ofthe try e discovered ourexpenditure- through submitted, ourOffice would have approved it. submitting it first for review.the adbeen Had to Toronto’s communitywithout francophone available services last sixyears aboutministry anadover ithadrun the ces advisedusthat urgent-advertising process with the Ministry, the urgent-advertising process with Wepractical. have an established therefore ad onanurgent matter, itmay not always be running an the government notified to prior messages. WhileourOfficeprefers to be exclusion respect toAct ofurgent the with in ouropinion,didnot intent meet ofthe the focused onfiretion asthey prevention which, 12 didnot qualify exempthat ofthem for the - Act. Wethe reviewed adsanddetermined the exempt from review undersubsection matters regarding foresttherefore fires and ads were these abouturgent that explained submit to ourOffice for review. TheMinistry itdidnotResources 34 ran adsthat radio ofNatural Ministry the process that gathering approved by ourOffice. in question was subsequently submitted and doesnotto happenagain. ensure Thead this haveus they sinceadopted new procedures andagency advised our Office.TheMinistry elements were submitted to andapproved by a large campaign international whoseother to ourOffice for review. of Thisad was part ithadnot yetcation although beensubmitted agency inadvertently released anadfor publi- itsadvertising Innovation advisedusthat Office would have approved it. it for review. adbeensubmitted, our Hadthe Children’s Lawyer submitting first the without about anewOfficeof mailingaddress for the anadinapublicationfor itran lawyersus that W The Minis The Minis The Minis

2(5) of - - - for the government Actfor to to the amendthe include any Internet costs. We believetimehascome the chapter does not include endofthis atthe reporting for allgovernment Ourexpenditure advertising. Act intent isto ofthe bemetbe addressed ifthe that should the legislation significant loopholein increases, we believe itsexemption hasbecomea onlineadvertising government’s costs). Asthe for ontelevision advertising (excluding production itdid ment spentmore than onInternet advertising for review. 2011/12 Inthe fiscal year,the govern- hadbeensubmitted Act to ifthey of the ourOffice wouldonline campaigns that have beeninviolation past, we Inthe advertising. have seengovernment entirely online.TheAct doesnot cover online online component andsomecampaigns even run large campaigns advertising ment’s includean most marketing campaigns. govern Many- ofthe of part hasbecomeanintegral Online advertising government offices with an overview of the Act, an of governmentoverview officeswith Review Guidelines.The revised Guidelinesprovide fall,ourOffice released updated Advertising In the Internet Online Advertising MATTERSOTHER Guidelines Revised Government Advertising • try ofTourism,try Culture andSport a monthly basisofanya monthly urgent-advertising run. whereby ourOfficewillbeprovided copieson such advertising for advertising review.such inJuly 2011,Ministry ithasbeensubmitting June 2011. the Sincewe issuewith this raised Maycontraventions and inApril, occurred covered andtherefore Act.istry by These the the Min- of areOffice. Theseattractions part them to our Park submitting first without ParksHistorical andFort William Historical mediaaboutHuronia 23adsinvarious ran The Minis

advertising. advertising. ment offices. website to provide ingovern ready accessto- staff years ago.We have Guidelinesonour posted the adoptedpractices Act took seven sincethe effect Act. They alsoreflect sectionsofthe evolvingtain ments for review, andprovide oncer clarification . ing Act,2004 matter- Advertis reviewable Government underthe foron expenditures andprinted advertisements annually Legislative to report General Assembly the The Auditor Auditor General Actrequires the that ourprocesses forexplain submittingadvertise found no matters of concern inourreviewfound work. nomatters of concern Auditorof items pendingthe review. General’s We submission requirements use andprohibition onthe . Thesedealwith Act,2004 Advertising Government requirementsto the ofsections2,3,4and8the compliance respect procedures with certain formed We atselecteddocumentation ministries. alsoper supporting andtheir tising andcreative services reviewed selected payments to ofadver suppliers my expenditures, advertising reported Office the In order to completeness test the of andaccuracy tising campaigns to reported ministry. usby each Matter and Printed Advertisements Expenditures on Figure details expenditure ofadver 1contains - - - - - Review ofGovernment Advertising 413

Chapter 5 Chapter 5 414 avention—ad was published before being reviewed, then submittedand approved. avention—ad waspublishedbeforebeingreviewed,then avention—ad by wasnotsubmittedforreview, Ministry. butreported 6. 4. in2012,eported in2013. butmorecosts 3. in2011,eported in2012. butmorecosts 2. 1. 2004 Act, offices government Ontario Source ofdata: Advertising Government underthe Matter andPrinted ReviewableFigure 1:Expendituresfor Advertisements 2012 oftheOffice ofthe Annual Report Auditor General of Ontario Risk Management Programs Risk ManagementPrograms inOntario Invest Good ThingsGrowinOntario Foodland Ontario Affairs FoodandRural Agriculture, Signs Infrastructure Affairs Aboriginal Title Ministry/Campaign Full-day Kindergarten Full-day Kindergarten Full-day Kindergarten Education Your NextBigIdea Your NextBigIdea Exports Ontario Exports Ontario Ontario Invest Ontario Invest Go North Business Immigration Business Immigration Economic Development andTrade RIDE RIDE Cyber Safety Alive Arrive andCorrectional Services Community Safety Numbers Social ServiceContact Community andSocialServices Remembrance Day Order ofOntario Global ExperienceOntario Citizenship andImmigration LegalProfession Notice tothe LegalProfession Notice tothe Attorney General

Contr Violation—ad wasreviewed,butdidnotmeet t Contr R R 2 1 1 1 1 1 1 6 2 1 1 4 3 3 , April1, 2011–March 31, 2012 he required standards, then resubmittedandapproved. then he requiredstandards, Submissions # of — — — — — — — — — — 4 1 1 1 1 2 4 1 3 1 2 2 3 1 2 1 Items # of 26 23 64 22 12 13 13 27 — — — — — — — — 8 1 8 1 1 1 2 1 1 1 Agency Fees Agency 569,148 100,194 177,623 54,629 15,443 13,912 11,106 3,488 1,111 –340 528 — — — — — — — — — — — — — — — Production 109,286 236,917 18,000 Third-party Costs ($) Costs Third-party 5,892 2,038 9,686 3,400 3,146 959 72 — — — — — — — — — — — — — — — —

227,789 30,422 7 415 27, Talent 6,000 6,000 — — — — — — — — — — — — — — — — — — — — — Bulk Mail — — — — — — — — — — — — — — — — — — — — — — — — — — 4,492 Other 220 520 184 158 401 612 191 40 17 — — — — — — — — — — — — — — — — 2,954,708 953,954 108,055 322,181 –17,728 55,334 12,463 –1,941 –367 TV — — — — — — — — — — — — — — — — — 740,793 92,648 Media Costs ($) Media Costs Radio 329 — — — — — — — — — — — — — — — — — — — — — — — 5,125,369 380,823 122,330 140,582 –10,202 30,683 11,439 40,163 71,910 30,174 –5,077 2,052 1,250 1,831 1,827 7,938 3,574 Print 940 — — — — — — — — Out-of-Home* 336,034 1,474 517, 20,581 — — — — — — — — — — — — — — — — — — — — — — — Ad Value 5,500 500 248 ($) — — — — — — — — — — — — — — — — — — — — — — — † 1,009,505 Campaign 2,975,618 6,267,988 2,476,127 –10,202 –16,788 136,082 Total ($) 144,290 186,122 186,547 132,811 –3,767 60,859 73,334 31,469 22,617 6,028 2,052 2,038 1,538 1,831 1,827 –367 7,938 3,574 — a a a a Full-day Kindergarten Full-day Kindergarten Full-day Kindergarten Education Your NextBigIdea Your NextBigIdea Exports Ontario Exports Ontario Ontario Invest Ontario Invest Go North Business Immigration Business Immigration Economic Development andTrade RIDE RIDE Cyber Safety Alive Arrive andCorrectional Services Community Safety Numbers Social ServiceContact Community andSocialServices Remembrance Day Order ofOntario Global ExperienceOntario Citizenship andImmigration LegalProfession Notice tothe LegalProfession Notice tothe Attorney General Risk ManagementPrograms inOntario Invest Good ThingsGrowinOntario Foodland Ontario Affairs FoodandRural Agriculture, Signs Infrastructure Affairs Aboriginal Title Ministry/Campaign 2 1 1 1 1 1 1 6 2 1 1 4 3 3 Submissions # of — — — — — — — — — — 1 3 1 4 1 1 1 1 2 4 2 3 1 2 1 2 Items # of 26 23 64 22 12 13 13 27 — — — — — — — — 1 1 8 1 8 1 2 1 1 1 Agency Fees Agency 569,148 100,194 177,623 54,629 15,443 13,912 11,106 3,488 1,111 –340 528 — — — — — — — — — — — — — — — Production 109,286 236,917 18,000 Third-party Costs ($) Costs Third-party 5,892 2,038 9,686 3,400 3,146 959 72 — — — — — — — — — — — — — — — — 227,789 30,422 7 415 27, Talent 6,000 6,000 — — — — — — — — — — — — — — — — — — — — — Bulk Mail — — — — — — — — — — — — — — — — — — — — — — — — — — 4,492 Other 220 520 184 158 401 612 191 40 17 — — — — — — — — — — — — — — — — alue denotes the value of an ad space provided to government offices at no cost, often where the government hasprovidedfundingforarelatedevent. the government where often officesatnocost, valueofanadspaceprovidedto government alue denotesthe a. † *

Negative tot Ad V Out-of-Home adver 2,954,708 953,954 108,055 322,181 –17,728 55,334 12,463 –1,941 al duetomediacreditsbeingapplied. –367 TV — — — — — — — — — — — — — — — — — tising includes, for example, billboards and transit posters. tising includes,forexample,billboardsand transit 740,793 92,648 Media Costs ($) Media Costs Radio 329 — — — — — — — — — — — — — — — — — — — — — — — 5,125,369 380,823 122,330 140,582 –10,202 30,683 11,439 40,163 71,910 30,174 –5,077 2,052 1,250 1,831 1,827 7,938 3,574 Print 940 — — — — — — — — Out-of-Home* 336,034 1,474 517, 20,581 — — — — — — — — — — — — — — — — — — — — — — — Ad Value Review ofGovernment Advertising 5,500 500 248 ($) — — — — — — — — — — — — — — — — — — — — — — — † 1,009,505 Campaign 2,975,618 6,267,988 2,476,127 –10,202 –16,788 136,082 Total ($) 144,290 186,122 186,547 132,811 –3,767 60,859 73,334 31,469 22,617 2,052 2,038 6,028 1,538 1,831 1,827 –367 7,938 3,574 — a a a a 415

Chapter 5 Chapter 5 416 eported in2011,eported in2012—approval butmorecosts withdrawn. 7. 5. in2012,eported in2013. butmorecosts 4. in2011,eported in2012. butmorecosts 2. 1. 2012 oftheOffice ofthe Annual Report Auditor General of Ontario Postal Strike Postal Licence PlateStickerRenewal Government Services Savings Bonds Ontario Budget Ontario Children’s ActivityTax Credit Finance DrinkingWaterProtecting Ontario’s Environment Plan Energy Long-term Energy Title Ministry/Campaign Employment Standards Employment Standards Safe atWork Ontario Labour World JuniorBaseball Diabetes andSport Health Promotion ofLifeNetwork Trillium Gift Stroke Warning Signs Seasonal Flu Seasonal Flu Session Public Information MedsCheck forDiabetesPatients VaccineImmunization –Rotavirus Healthy SmilesOntario CareOptions Health CareOptions Health CareOptions Health Diabetes CancerScreeningExpansion Breast Health andLong-Term Care Offices ServiceOntario Offices ServiceOntario Awareness Brand ServiceOntario ServiceOntario

Changes tot R Violation—ad wasreviewed,butdidnotmeett R R 1 he website after approval violated the Act, then corrected by corrected Ministry. Act,then approval violatedthe he websiteafter 1 1 1 1 5 2 2 1 1 7 1 1 1 4 he required standards, then resubmittedandapproved. then he requiredstandards, Submissions # of — — — — — — — — — — — 4 2 4 1 2 1 1 4 1 3 3 1 3 2 1 1 3 Items # of 25 28 32 32 24 18 11 — — — — — — — — — — — 2 1 4 2 4 7 2 3 1 1 Agency Fees Agency 403,260 165,012 79,231 43,478 4,250 6,525 4,720 2,756 5,270 7,235 350 — — — — — — — — — — — — — — — — — Production 312,430 491,638 493,875 9,874 197, 83,295 25,241 Third-party Costs ($) Costs Third-party 13,775 77,863 14,137 8,590 2,300 1,010 — — — — — — — — — — — — — — — — 121,260 82,628 46,287 92,414 Talent 9,625 9,046 1,000 — — — — — — — — — — — — — — — — — — — — — Bulk Mail — — — — — — — — — — — — — — — — — — — — — — — — — — — — 10,465 3,252 4,000 3,708 1,961 3,102 Other 3,115 3,417 567 771 50 60 18 — — — — — — — — — — — — — — — 1,583,001 1,569,587 1,244,106 1,351,051 488,492 791,971 TV — — — — — — — — — — — — — — — — — — — — — — 1,218,136 112,022 375,880 157,326 Media Costs ($) Media Costs –2,379 –2,077 5,622 9,865 Radio — — — — — — — — — — — — — — — — — — — — 264,334 860,062 764,796 361,306 196,912 671,932 141,082 93,932 20,769 –7,260 8,742 5,416 ,511 7, Print 17 — — — — — — — — — — — — — — Out-of-Home* 1,677,192 186,614 271,873 6,037 — — — — — — — — — — — — — — — — — — — — — — — — Ad Value 2,400 3,250 1,195 100 ($) — — — — — — — — — — — — — — — — — — — — — — — — † 1,706,628 2,619,006 1,583,001 2,639,775 3,419,458 Campaign 1,872,918 400,056 Total ($) 869,612 398,912 262,257 118,030 842,677 186,614 141,082 –2,379 35,893 –7,260 94,703 52,547 23,415 2,400 5,622 9,921 5,476 1,195 100 17 — a a Employment Standards Employment Standards Safe atWork Ontario Labour World JuniorBaseball Diabetes andSport Health Promotion ofLifeNetwork Trillium Gift Stroke Warning Signs Seasonal Flu Seasonal Flu Session Public Information MedsCheck forDiabetesPatients VaccineImmunization –Rotavirus Healthy SmilesOntario CareOptions Health CareOptions Health CareOptions Health Diabetes CancerScreeningExpansion Breast Health andLong-Term Care Offices ServiceOntario Offices ServiceOntario Awareness Brand ServiceOntario ServiceOntario Strike Postal Licence PlateStickerRenewal Government Services Savings Bonds Ontario Budget Ontario Children’s ActivityTax Credit Finance DrinkingWaterProtecting Ontario’s Environment Plan Energy Long-term Energy Title Ministry/Campaign 1 1 1 1 1 5 2 2 1 1 7 1 1 1 4 Submissions # of — — — — — — — — — — — 4 2 4 1 2 1 1 4 1 3 3 1 3 1 3 2 1 Items # of 25 28 32 32 24 18 11 — — — — — — — — — — — 2 1 4 2 4 7 2 1 3 1 Agency Fees Agency 403,260 165,012 79,231 43,478 4,250 6,525 4,720 2,756 5,270 7,235 350 — — — — — — — — — — — — — — — — — Production 312,430 491,638 493,875 9,874 197, 83,295 25,241 Third-party Costs ($) Costs Third-party 13,775 77,863 14,137 8,590 2,300 1,010 — — — — — — — — — — — — — — — — 121,260 82,628 46,287 92,414 Talent 9,625 9,046 1,000 — — — — — — — — — — — — — — — — — — — — — Bulk Mail — — — — — — — — — — — — — — — — — — — — — — — — — — — — 10,465 3,252 4,000 3,708 1,961 3,102 Other 3,115 3,417 567 771 50 60 18 — — — — — — — — — — — — — — — alue denotes the value of an ad space provided to government offices at no cost, often where the government hasprovidedfundingforarelatedevent. the government where often officesatnocost, valueofanad spaceprovidedtogovernment alue denotesthe a. † *

Negative tot Ad V Out-of-Home adver 1,583,001 1,569,587 1,244,106 1,351,051 488,492 791,971 al duetomediacreditsbeingapplied. TV — — — — — — — — — — — — — — — — — — — — — — tising includes, for example, billboards and transit posters. tising includes,forexample,billboardsandtransit 1,218,136 112,022 375,880 157,326 Media Costs ($) Media Costs –2,379 –2,077 5,622 9,865 Radio — — — — — — — — — — — — — — — — — — — — 264,334 860,062 764,796 361,306 196,912 671,932 141,082 93,932 20,769 –7,260 8,742 5,416 ,511 7, Print 17 — — — — — — — — — — — — — — Out-of-Home* 1,677,192 186,614 271,873 6,037 — — — — — — — — — — — — — — — — — — — — — — — — Ad Value Review ofGovernment Advertising 2,400 3,250 1,195 100 ($) — — — — — — — — — — — — — — — — — — — — — — — — † 1,706,628 2,619,006 1,583,001 2,639,775 3,419,458 Campaign 1,872,918 400,056 Total ($) 869,612 398,912 262,257 118,030 842,677 186,614 141,082 –2,379 35,893 –7,260 94,703 52,547 23,415 2,400 5,622 9,921 5,476 1,195 100 17 — a a 417

Chapter 5 Chapter 5 418 avention—ad by wasnotsubmittedforreview, Ministry. butreported in2012,eported in2013. butmorecosts 3. in2011,eported in2012. butmorecosts 2. 1. 2012 oftheOffice ofthe Annual Report Auditor General of Ontario Shoreline Property Notices Shoreline Property CardRenewal Outdoors Parks Ontario Parks Ontario Parks Ontario Contest Art Fish WildfirePrevention FireSmart WildfirePrevention FireSmart BearWise Members CommitteeSeeksNew Advisory Natural Resources Title Ministry/Campaign Total Veterans LicencePlates Graphic Drive Safe Transportation Study inOntario Public Education Awareness and Post-secondary Employment Ontario Training, CollegesandUniversities Parks Huronia Historical Parks Huronia Historical Park William Historical Fort Park William Historical Fort Tourism, Culture andSport Tax Credits Revenue Fund Heritage Ontario Northern Credit Energy Ontario Northern Development, MinesandForestry Northern Wildfire Prevention Wildlife Management Species atRisk

Contr R R 1 2 1 2 1 3 3 3 2 1 1 1 Submissions 121 # of 11 17 — — — — — — — — — 1 2 1 2 1 2 8 1 1 1 1 2 1 2 Items 565 # of 20 12 16 16 14 31 — — — — — — 1 2 1 2 1 3 8 1 3 2 1 2 7 Agency Fees Agency 1,778,408 101,479 8,000 — — — — — — — — — — — — — — — — — — — — — — — Production 2,220,285 91,283 Third-party Costs ($) Costs Third-party 2,615 5,154 7,920 200 169 770 375 375 — — — — — — — — — — — — — — — — 714,869 48,254 Talent 6,729 — — — — — — — — — — — — — — — — — — — — — — — Bulk Mail 300,560 300,560 — — — — — — — — — — — — — — — — — — — — — — — — 64,780 18,634 4,080 Other 745 — — — — — — — — — — — — — — — — — — — — — — 13,146,014 1,144,300 535,933 35,342 15,580 TV –8 — — — — — — — — — — — — — — — — — — — — 2,947,272 102,113 20,330 78,024 29,918 Media Costs ($) Media Costs 8,722 Radio — — — — — — — — — — — — — — — — — — — — 10,379,658 704,151 –10,374 42,569 40,600 84,538 32,739 29,758 15,228 10,465 7 70 87 67, 5,000 1,469 1,506 1,359 4,515 1,108 Print — — — — — — — — — Out-of-Home* 3,107,489 20,650 64,113 6,921 — — — — — — — — — — — — — — — — — — — — — — Ad Value 106,815 50,000 12,860 14,560 6,024 8,350 1,200 628 ($) — — — — — — — — — — — — — — — — — — † 34,766,150 2,408,661 Campaign –10,382 Total ($) 561,897 102,113 105,574 116,273 111,189 88,200 30,039 30,386 50,000 13,235 78,024 16,428 14,729 40,975 5,000 6,024 8,550 1,506 1,359 4,515 1,108 — — a Total Veterans LicencePlates Graphic Drive Safe Transportation Study inOntario Public Education Awareness and Post-secondary Employment Ontario Training, CollegesandUniversities Parks Huronia Historical Parks Huronia Historical Park William Historical Fort Park William Historical Fort Tourism, Culture andSport Tax Credits Revenue Fund Heritage Ontario Northern Credit Energy Ontario Northern Development, MinesandForestry Northern Wildfire Prevention Wildlife Management Species atRisk Notices Shoreline Property CardRenewal Outdoors Parks Ontario Parks Ontario Parks Ontario Contest Art Fish WildfirePrevention FireSmart WildfirePrevention FireSmart BearWise Members CommitteeSeeksNew Advisory Natural Resources Title Ministry/Campaign 1 2 1 2 1 3 3 3 2 1 1 1 Submissions 121 # of 11 17 — — — — — — — — — 1 1 2 1 2 1 2 8 1 1 1 1 2 2 Items 565 # of 20 12 16 16 14 31 — — — — — — 3 1 2 1 2 1 3 8 1 1 2 2 7 Agency Fees Agency 1,778,408 101,479 8,000 — — — — — — — — — — — — — — — — — — — — — — — Production 2,220,285 91,283 Third-party Costs ($) Costs Third-party 2,615 5,154 7,920 200 169 770 375 375 — — — — — — — — — — — — — — — — 714,869 48,254 Talent 6,729 — — — — — — — — — — — — — — — — — — — — — — — Bulk Mail 300,560 300,560 — — — — — — — — — — — — — — — — — — — — — — — — 64,780 18,634 4,080 Other 745 — — — — — — — — — — — — — — — — — — — — — — alue denotes the value of an ad space provided to government offices at no cost, often where the government hasprovidedfundingforarelatedevent. the government where often officesatnocost, valueofanadspaceprovidedtogovernment alue denotesthe a. † *

Negative tot Ad V Out-of-Home adver 13,146,014 1,144,300 535,933 35,342 15,580 al duetomediacreditsbeingapplied. TV –8 — — — — — — — — — — — — — — — — — — — — tising includes, for example, billboards and transit posters. tising includes,forexample,billboardsandtransit 2,947,272 102,113 20,330 78,024 29,918 Media Costs ($) Media Costs 8,722 Radio — — — — — — — — — — — — — — — — — — — — 10,379,658 704,151 –10,374 42,569 40,600 84,538 32,739 29,758 15,228 10,465 7 70 87 67, 5,000 1,469 1,506 1,359 4,515 1,108 Print — — — — — — — — — Out-of-Home* 3,107,489 20,650 64,113 6,921 — — — — — — — — — — — — — — — — — — — — — — Ad Value 106,815 50,000 12,860 14,560 Review ofGovernment Advertising 6,024 8,350 1,200 628 ($) — — — — — — — — — — — — — — — — — — † 34,766,150 2,408,661 Campaign –10,382 Total ($) 561,897 102,113 105,574 116,273 111,189 88,200 30,039 30,386 50,000 13,235 78,024 16,428 14,729 40,975 5,000 6,024 8,550 1,506 1,359 4,515 1,108 — — a 419

Chapter 5 Chapter 6 420 2012, following the members: with bly, Committee 23, was appointed the onFebruary Assem- Standing Ordersofthe the accordance with 2012. inFebruary membership Committee’s the In on government,ity Liberal was reached agreement ing ageneral electionto itsdissolution. sessionimmediately openingofitsfirst the follow Parliament, ofthe duration from forestablished the vote only to break atie.TheCommittee isnormally are entitledto vote Chairmay onmotions, whilethe Legislative Assembly. except Chair Allmembers the the in representation that party’s reflects given party from any Legislature. Thenumberofmembers the Accounts (Committee) are appointed by amotion of Standing Committee ofthe onPublic Members Committee of theComposition Appointment and Chapter 6 Chapter David Zimmer, Liberal Jagmeet Singh,New Democrat Liz Sandals,Liberal Ouellette, Progressive Conservative Jerry Liberal Reza Moridi, Phil McNeely, Liberal France Gélinas,New Democrat Toby Vice-chair, Barrett, Progressive Miller,Norm Chair, Progressive Conservative Following October 2011 the electionofaminor Conservative on PublicAccounts The StandingCommittee - - was prorogued onOctober 15, 2012. fall2012 formembers the session.TheLegislature had not yet committee beenpassedappointingthe was prepared, amotion report timethis At the Committee 23motion, was dissolved. the February special assignmentonitsbehalf. a PublicAccounts ter orundertake inrespect ofthe Auditor anyrequest the examine General mat that Auditor Committee General mayof the Act,the also Legislative Assembly.the Under sections16 and17 andrecommendations to presents itsobservations and orourspecial reports in ourAnnualReport year relating to the matters raisedthroughout Committee typically holdsanumberofhearings objectives. The their inachieving ness ofprograms broader-public-sector effective andthe operations economythe andefficiencyofgovernment and tive Assembly onanumberofissues,including Legisla to the assessesandreports examines, becomeavailable.mittee asthey TheCommittee to have Com referredto beenpermanently the are PublicAccounts,deemed which and onthe Auditor from the General mendations onreports opinionsandrecom Houseitsobservations, to the The Committee isempowered to review andreport Role of theCommittee On September 10, 2012, the inaccordance with - - - - - ters are rarely asked to are attend. Onceitshearings than policymatters, minis- tiverather andfinancial administra operational, - dealswith Annual Report questions from committee Becauseour members. andrespond to auditee(s) hearings to appearatthe the seniorofficialsfromCommittee typically asks audit.The subjectofthe was the public sector that istry, Crown agency broader ororganization inthe the min- to ourfindingsand recommendations from responses sectionandthe applicablereport on the researcher, Committee Committee’s the the briefs Committee to review. for sectionsfromthe ourAnnualReport or other audits selectsthree normally politicalparties three for review.Annual Report For ofthe example, each value-for-money from the reports chapter ofour Ittypically focuses onthem. conducts hearings and orourspecialreports from ourAnnualReport Legislative Assembly proceedings. ofgovernment andother debates,report speeches ings are official recorded inHansard,verbatim the Allpubliccommittee proceed- ofitsreports. aration prep agenda- Committee’s orthe the dealing with meetings public except are opento the for those All House,atany timeofitschoosing. other the approvaltive the of Assembly issitting,and,with The Committee may meet weekly Legisla- whenthe issuedby ourOffice. reports PublicAccounts andany special Ontario’s Report, relating itsreviews to ourAnnual andhearings with all committee meetings Committee to assist the Auditor attenderal Act,the andseniorstaff General section16In accordance with Auditor Gen- ofthe Operations andCommittee Procedures WITH THECOMMITTEE AUDITOR GENERAL’S ADVISORY ROLE At each hearing, the Auditor the alongwith General, hearing, each At The Committee identifiesmatters ofinterest mendations madeinourreports. andrecom raised - concerns totaking address the are Committee update onwhatactionsthey the were auditees that notthose selected for hearings Legislativeand recommendations to Assembly. the completed, Committee provides the itscomments results of this workresults ofthis in2013. project. We planto onthe issueaSpecialReport cancelledMississaugataxpayers gas plant ofthe Powertion/Eastern Limited cost regardingto the Power andGreenfield South - Authority Corpora Power between contract Ontario the examine the passed arelated motion requestingourOffice that final meeting onSeptember 5,2012,the Committee Power Ontario Inits contracts. certain Authority examine tee requestingourOffice members that or related to amotionthe commit- filed by oneof ing meetings were administrative innature either witnesses were calledandquestioned. Theremain- , andincludedanumberofsessionswhere Services Ornge AirAmbulanceandRelated special report, meetings were related2012 March to ourOffice’s 2012–September 2012 Almost allofthese period. February The Committee the met 20timesduring a stipulatedthese OurOffice timeframe. reviews provide Committee responses within Clerk the with istry, agency orbroader-public-sector organization Min- dations andrequestmanagement ofthe that tee oronlineatwww.ontla.on.ca. Commit- Clerkpublicly ofthe the available through andletters allcommitteeOnce tabled, are reports comments andrecommendations. Committee’s the itsmeetings Committee andinclude by during the gathered information the and letters summarize Legislative Assembly.for inthe tabling Thesereports andlettersThe Committee onitswork issuesreports MEETINGS HELD REPORTS OF THE COMMITTEE REPORTS The Clerk of the CommitteeThe Clerk alsorequests ofthe that Committee reports typicallyCommittee includerecommen- reports The Standing Committee on PublicAccounts 421

Chapter 6 Chapter 6 422 Mazza at the hearings on our Ornge special report. specialreport. onourOrnge hearings Mazza atthe WarrantSpeaker’s ofDr. appearance for the Chris requesting Legislature, both two a the items with Septemberthrough 2012, Committee didtable the from February period the time,butduring as ofthat tember 2012. No hadbeenissued committee reports Committee was dissolvedongoing whenthe inSep- were hearings still , andthese and Related Services Orngerelating AirAmbulance to ourspecialreport, Committee focused almost exclusively onmatters area. operational tion orauditsofthat into inany consideration subsequent follow-up sec- recommendations andresponses, andwe take them 2012 oftheOffice ofthe Annual Report Auditor General of Ontario Since its appointment in February 2012,Since itsappointmentinFebruary the to Nunavut andwas heldinIqaluitfrom August 19 interest. Legislative Auditors to discuss issuesofmutual CanadianCouncilof the annual conference with mittees from across Canada.CCPAC holdsajoint publicaccountscom- provincial andterritorial tees (CCPAC) consists ofdelegates from federal, The CanadianCouncilofPublicAccounts Commit- ACCOUNTS COMMITTEES ACCOUNTS COUNCIL OF PUBLIC CANADIAN

21, 2012. The 33rd annualconference was hosted by released onJune28,2011. 26, April wasmost issuedon recent pre-election report fiscal over Thegovernment’s years. three next the the province of financialperformance tions for the - onitsexpecta pre-election report government’s also required to reasonableness ofthe review the gov.on.ca. actscanbefoundfunction). Both atwww.e-laws. review advertising Office’s for more details onthe proposed government (seeChapter advertising 5 ornotdeciding whether to approve typesof certain Auditor isresponsible General for reviewing and , the Act,2004 Advertising Government under the Auditor General ofthe Act.Inaddition, authority money spent. taxpayers, for the onbehalfofOntario obtain, valuethey spendpublicfundsandfor the they for recipients accountable how and grant prudently government,Assembly itsadministrators holdthe the Legislative Officehelps the Bydoingthis, them. on and financialaudits reviews and reporting by conductingvalue-for-moneycitizens ofOntario the Legislativethe Assembly and (Office) serves the AuditorThe Officeof ofOntario General Chapter 7 Chapter In anelectionyear, Auditor the is General the under The work Officeisperformed ofthe

2011, results ofourreview were and the Auditor General of Ontario General Auditor The

Office broad criteria to be considered in this assessment: to beconsidered inthis broad criteria value-for-money following auditsofthe entities: to conduct the authority TheOfficehas services. relative public andthe cost-effectiveness ofthese beingdelivered level toto the assessthe ofservices money auditsdelve underlying into the operations value-for-auditors typicallynumbers, focus onthe oractivities.Andwhile administers itsprograms we entitythat audit)manages“auditee” (the and money auditsare assessmentsofhow well agiven value-for-money value-for- auditing.TheOffice’s Office’s work two-thirdsAbout ofthe relates to ANNUAL REPORT VALUE-FOR-MONEYIN THE AUDITS Overview General

• • • • • The Auditor Generalthe Act(Act) identifies of y should be spent with dueregard fory shouldbespentwith broader publicsector ganizations inthe and own-controlled corporations; own agencies; government ministries; ario economy. services). mental-health dren’s aidsocieties provide andagencies that - boards, homes,chil long-term-care school universities,hospitals, communitycolleges, receive (forthat government example, grants Mone or Cr Cr Ont

the

423

Chapter 7 Chapter 7 424 money auditsis: work is typically includedinourvalue-forthat compliance(discussed inalater section).Other into value-for-money both auditsand“attest” audits pliance” auditwork butisgenerally incorporated istechnically “com- risks controlling against these instances where the following whereinstances the was observed: value-for-moneyfor each audit. or organization beingaudited are alsodeveloped ministry, particular program ofthe operations the that relate directly Moreto specificcriteria them. havecriteria not on beenmet, heorshereport value-for-money three where instances the observes 2012 oftheOffice ofthe Annual Report Auditor General of Ontario • • • • • • • Assessing the extent to which the auditee was extent the to which Assessing the The Act also requires that the AuditorThe Acton alsorequires the report that AuditorThe Act requires ifthe General that, y was expended other than for the pur for the than y was other expended ecords were not orthe maintained ccounts were not properly kept orpublic procedures shouldbeinplacetoopriate dueregard fory shouldbespentwith forming the tests andprocedures the weforming deem • • • poses for which itwasposes for appropriated. which requirements. key these legislationand authority with management auditee’s hascomplied the that to reasonable assurance obtain necessary responsible for and administering; management auditee’s is the that as those andactivitiesaswell programs auditee’s the auditee govern or the that authorities and the ficient to: and procedures appliedwererules not suf- money was not fully accounted for. programs. effectiveness of onthe measure andreport efficiency. per identifying t Mone Essential r A Appr Mone e that expenditures were expenditures e that madeonly assessment,collection effectivelyheck the eguard andcontrol publicproperty; ensur c saf as authorized. and proper allocationofrevenue; or he key provisions inlegislation - of the processes we below.of the useare described quality,the andvalueofourwork. integrity Some dards require we that have processes for ensuring Institute- Accountants. Thesestan ofChartered professional Canadian by standards established the value-for-money work, we follow relevant the estimates.expenditure reviewstive oflegislationand sessions andthrough legisla- questions during policiesthrough ernment Assembly continually monitors andchallenges gov foraccountable policymatters. TheLegislative governmentLegislative holdsthe Assembly that ofgovernment policy. merits on the Rather, itisthe note, however, indoingsowe that donot comment ing government to policydecisions.Itisimportant how well management andexecut isadministering - ourvalue-for-moneycould say that auditsfocus on result ofgovernment policydecisions.Thus,we higher-risk audits,we as: considerfactors such lated value-for-money To criteria. help uschoose legis- anauditee isnot three isthat meeting the risk year,tion to auditeach we considerhow the great corporations. agencies), aswell asseveral large Crown-controlled (Children’s Aidsocieties andsocialservice services boards, universities(school andcolleges) andsocial providers), education service and mental-health homes, long-term-care (hospitals, including health covered awiderange oftopics inseveral sectors, to allow entities,ourauditshave us to auditthese and diverse. Sinceourmandate in2004 expanded activitiesare sonumerous andtheir ber ofthem num- agreat are samecycle such because there the public sector andCrown-controlledon corporations vals. We broader donot auditorganizations inthe activities atapproximately five- to seven-year inter and programs The Officeauditsmajor ministry Selecting What to AuditSelecting What to In planning, performing and reporting onour andreporting In planning,performing andactivitiesareGovernment the programs In selecting what program, activityororganiza- In selectingwhatprogram, - - work inourauditreport. audit resultsbut rely ofinternal onandpresent the cases, we scopeofouraudit donot diminishthe scopeto Inother avoidaudit’s duplicationofeffort. work,on that we may defer anauditorchange our auditors have completed orplanned.Depending tion process iswhatwork internal auditee’s the auditjustify itscosts. ducting the regarding- ourauditobjective andmake observa we aconclusion sothat canreach and criteria, and procedures to address ourauditobjective audit. planning stage ofthe oractivityatthe program ment responsible for the we seniormanage develop- the are discussedwith able laws, authorities. regulations andother fully auditsorreviews; appliedinother andapplic- success- policies andprocedures; applicable criteria own management’s andservices; similar programs delivering bodiesorjurisdictions other of experts; sively asrecognized bodies sources such researching ing effectively. involves Developing criteria exten - - shouldbeinplaceandoperat and procedures that cover key that the systems, policies audit criteria what we want to achieve. We develop then suitable When we beginanaudit,we set anobjective for Assurance Levels Assurance and Audit Objectives, Audit Criteria Setting • • • • • • Another factorAnother we selec- take into accountinthe We benefitsofcon- the alsoconsiderwhether The next stepThe next isdesigningandconductingtests To suitability, ensure their criteria further the he significance of the issuesanauditmight he significanceof the auditee’s changesecent significant in he complexity auditee’s anddiversity ofthe activityororgan program, he impact ofthe he total revenues involved; orexpenditures he results ofprevious auditsandrelated ization on the public; ization onthe operations; and operations; operations; follow-ups; identify. t r t t t t - ate level of assurance, obtained primarily through through ate level primarily obtained ofassurance, anaudit.Areview provides than amoder rather Act. responsibilities underthe to wefulfillour decisions that deemnecessary inCabinet contained submissionsor information However, all Officecanaccessvirtually other the do not ofCabinet. deliberations seekaccessto the ofCabinet privilege,we principle respect for the ofourduties. Outof performance to the necessary have accessto andrecords allrelevant information ment provides, Act we underthe are entitledto andadvice. assistance expert obtaining because we ahighly are area, examining technical independentsources; and,where necessary with facts tems, procedures confirming andtransactions; itprovides;information andtesting- examining sys management by andanalyzing interviewing the Specifically, anauditlevel isobtained ofassurance usingourregular auditprocedures.can obtain highest reasonable we level that ofassurance for ourwork to provide isatan“auditlevel”—the example, ininterpreting information. ofourwork—foron professional judgmentinmuch itisconclusive,persuasive andwe than must rely available for concludingonourobjective ismore evidence ofthe lem willnever Also,much arise. systems, sowe prob- cannot the guarantee that mightbe ableto circumventor staff control such auditee management problem from occurring—but was workingthat effectively to prevent aparticular control system inplacefor aprocess orprocedure example, auditee we hada the may concludethat For to factors alsocontribute this. matters. Other ourauditworkance” that identifiesallsignificant provide whatiscalledan“absolute level ofassur We hasitslimitations. information cannot therefore our work isexplained. auditobjective scopeof isstated the and the which a sectionentitled“Audit Objective andScope,” in has auditreport tions andrecommendations. Each Infrequently, a review Officewillperform the manage- that information respect toWith the we that assurance plan For reasons, the allthese Conducting tests andprocedures to gather The Officeofthe Auditor General ofOntario - - 425

Chapter 7 Chapter 7 426 The Auditor General finalizes the draft report (on report The Auditorthe draft finalizes General report. draft into the discussed andincorporated responses are to ourrecommendations, andthese managementThe auditee’s provides written seniormanagement. auditee’s the discussed with prepared and isthen work. report of the Adraft results preliminary management onthe isbriefed conclusionofon-site work,munication. At the to review andensure openlinesofcom- progress audit orreview, management meet with ourstaff focus ofourwork the ingeneral During terms. andthe objectivement to and criteria discussthe manage meet- Early with process, ourstaff inthe value-for-money the throughout auditorreview. seniormanagement auditee’s the municate with from ourOfficecom- tions andconclusions,staff To ofourobserva factualaccuracy helpensure the oversight practices. estimates jurisdictions’ mates other andresearched approve province’s Esti annualexpenditure - the year, we reviewed process usedto review the and actions beingtaken liability. to reduce that This Committee hasshown anongoinginterest inthe Standing Committee onPublicAccounts, andthe was well Boardand Insurance that received by the Workplace unfundedliabilityofthe of the Safety sector.electricity In2009,we conducted areview oversightenergy initiatives andregulatory ofthe our related value-for-money auditsofrenewable complemented stranded debt,which city sector’s audit year, we- conducted electri areview ofthe 2011 anaudit.Inthe review than more appropriate oractivitymake a program nature ofthe to the or activityto beaudited factors relating orother General Actdoesnot allow program for acertain has prohibitive costs orisunnecessary, Auditor the for example, providing ahigherlevel ofassurance ures We andtransactions. reviews when, perform limited andtesting examination ofsystems, proced - managementses ofinformation provides; andonly management; anddiscussionswith inquiries analy 2012 oftheOffice ofthe Annual Report Auditor General of Ontario Communicating with Management - - Annual Report. ispublishedinthe report the which ization, after agency,the organ- orgrant-recipient corporation deputyminister orheadof the will bebased)with Chapter AnnualReport the 3sectionofthe which special report, which we 2012. which didinMarch special report, Legislative Assembly inthe asa report we the table auditee. TheMinister requested oncompletion that the with the due toreport challenges infinalizing delayedAmbulance andRelated Services—was using ourrisk-based Air selectioncriteria—Ornge on completion time. andmadepublicatthat Legislature inthe willbetabled specialreport the that requester’s agreement has beento the obtain request undersection16 or17, practice ournormal duties. other conflicts with workdelay conductingthe ifheorshebelieves it Auditorassignment, the candecline or General duties. Inaddition,ifaminister requests an such to take precedence over Auditor the other General’s ment. However, specialassignmentsare not these Auditor aspecialassign the undertake General that Accounts Crown oraminister ofthe may require tive Assembly, Standing Committee the onPublic PublicAccounts. Underthe section17, Legisla- the onany andreport mattermust examine respecting lic Accounts may Auditor resolve the General that Under section16, Standing Committee the onPub- additionalspecialwork. toGeneral undertake AnnualReport. not untilthe bedeferred Auditor opinionofthe should General, inthe that, Legislative Assemblythe atany time,onany matter Officemay makeIn addition,the to aspecial report Legislative to Assembly.its inanAnnualReport the onitsaud- As required Office reports Act, by the the SPECIAL REPORTS SPECIAL REPORTS In 2011, we auditsthat hadinitiated oneofthe In recent years, whenwe have received aspecial Two Auditor the Act authorize sections ofthe - 9(1), Crown.of the Specifically,the Act [insubsections province accountsofmanythe agencies andthe year,Every we financial statements of audit the intended for. andnotexpenditures; spendingmoney onwhatitis and properly allocatingrevenue; unauthorized deficientprocedures for assessing,collecting erty; keeping; inadequate safeguarding ofpublicprop- unaccounted-for ofrecord publicmoney; lack relating to improperlyaging risks kept accounts; work. Specifically, the controls for man- we assess audit work into attest incorporated audit isoften overview ofvalue-for-moneythe audits,compliance accepted Asmentionedin accountingprinciples). Canadiangenerallypolicies (inmost cases,with accounting certain complies with is fairandthat andfinancialpositionina operations that way auditee’s cial statementsonthe present information finan the hisorheropiniononwhether expresses the auditor audits, financial statements. Insuch Attest ofanauditee’s auditsare examinations The Auditees AUDITSATTEST • • • • he Auditor financial state auditthe General he Auditor accountsand auditthe General

not audited by auditor; another Crown agencies are ofthe ments ofthose that ments of the corporation and a copy of the andacopy ofthe corporation ments ofthe audited acopyGeneral financial state- ofthe deliver Auditor tocontrolled corporations the Auditorresults to the and General; Auditor their tion ofthe andreport General direc- auditsunderthe their Crown perform agencies of the auditors ofcertain to bethe orotherwise; trust Consolidated Revenue Fund, heldin whether province’s ofthe part public money forming receipt of records anddisbursement ofthe public accountingfir public accountingfir t t (2), and(3)]requires that: ms auditingCrown- are appointed ms that - - audited directly during the 2011/12audited the directly during audityear. publicaccountingfirms that trolled corporations Crown agencies Crown-conlist ofthe the andthe - behalf. Exhibit1, 2andExhibit2,respectively, Part Office’s agenciesto onthe auditanumberofthese publicaccountingfirms with contracts currently 2011/12 the audited during audityear. TheOffice Annual Report. Legislature, we attention ofthe includeitinour the that is sosignificant we feel itshouldbebrought to agency’s auditcommittee Ifamatter ifoneexists. the isdiscussedwith which prepares afinal report, auditormanagement the responds to itinwriting, agency’s iscleared senior andthe report draft the that discussion.After the toresults of reflect seniormanagement.isrevised Thereport with anddiscussesit findings report prepares adraft management must make improvements, auditor the Treasury ofthe Board. Secretary associatedminister ministry, ofthe aswell asto the related agencythe deputy financial statements to andofthe reports of ourindependentauditor’s agency.sible for the OurOfficealsoprovides copies agency’s minister(s)to board respon the - andthe Auditor responsibilities are the reporting General’s Agency stipulates legislationnormally report. that agencies andCrown-controlledinthis corporations province’s consolidated financial statements. Bill Assent onNovember of Thepurpose 30,2004. Statute received AmendmentAct,which Law Royal sage, onNovember ofBill18, 22,2004, Audit the pas- The Auditor the General Actcameaboutwith OTHER STIPULATIONS OF THE STIPULATIONS OTHER GENERAL ACT GENERAL Exhibit 1, were agencies 1lists that the Part Where anagency attest auditnotes areas where We results ofattest donot auditsof discussthe attestChapter auditofthe year’s 2discussesthis

18 was amendments to to the make certain contained inamanagementcontained letter). recommendations to management (typically ofitsfindingsand report accounting firm’s The Officeofthe Auditor General ofOntario

AUDITOR AUDITOR 427

Chapter 7 Chapter 7 428 in the Legislative free to Assemblyin the fulfill andisthus politicalparties government andthe from the tance dis- Auditor an arm’s-length the maintains General Legislativecause by Assembly. the Consequently, non-renewable andcanbedismissedonly term, for responsibilities objectivelyreporting andfairly. Office to fulfillitsauditingand enablesthe that tion. Thisindependenceisanessentialsafeguard government anditsadministra- independent ofthe Officeare The Auditor ofthe andstaff General Legislativetion to Assembly. the mittee. madeitsrecommenda - Thecommittee then selection com- major sittingonthe politicalparties three representativesconducted, from the with anopencompetitiveGeneral, selectionprocess was last twoChapter selectionofthe Auditors 6).Inthe Committee, see made (for onthe more information opposition—be consulted appointmentis before the Legislative Assembly, official isamemberofthe StandingAccounts—who, Ordersofthe underthe Standing Committee Chairofthe onPublic the that Legislativeby Assembly. the TheAct alsorequires bly,” appointee must beapproved the meaningthat Assem- address ofthe appointment ismade“onthe Executive Cabinet).advice ofthe The Council(the AuditorGovernor appointsthe onthe General Lieutenant tenant is,the Governor inCouncil—that the Legislative Assemblythe Lieu- an officerof by AuditorUnder Act, the isappointed General the as audit mandate. seventhyear ofourexpanded the marks Report receive This2012 government grants. Annual broader publicsectororganizations that inthe value-for-money Office’s of the auditmandate to inBill18amendment contained expansion was the Legislative Assembly. the serve Themost significant Office to abilityofthe Audit Actto enhancethe 2012 oftheOffice ofthe Annual Report Auditor General of Ontario Independence General Appointment of Auditor The Auditor isappointed General to a10-year, at the end of this chapter. endofthis at the year forcussion ofexpenditures are presented the Assembly. Theaudited statements andrelated dis- Legislativeand subsequently inthe must betabled Officeare submittedthe Boardstatements ofthe to audited financial andthe accountants, chartered 2011/12 fiscal year have beenaudited of by afirm expendituresthe relating Office’s to Act,by the the Legislativelaid before Assembly. the Asrequired approves budget, Office’s issubsequently which the administrativegovernment’s process—reviews and legislative isindependentofthe committee that pressure. legislated political mandate Office’s without the competence and integrity in their work.competence intheir and integrity higheststrive to standards ofbehaviour, achieve the conduct themselves inaprofessional mannerandto for employees Office,whohave ofthe aduty to philosophy, regarding conduct andrules principles The Codeisintended to beageneral statement of dards andensure aprofessional work environment. to highprofessionalencourage maintain stan staff The OfficehasaCodeofProfessional Conduct to confidentiality. ensuring further thus auditee, cannotthe beaccessedfrom ourOffice, ofanauditfrom course the during mation obtained and auditworking includeallinfor which papers, reports tion andProtection Act,ourdraft ofPrivacy exemptour Officeis the Freedom- from ofInforma Assembly orany ofitscommittees. Aswell, because eral Act,donot have to Legislative belaidbefore the accordingpapers, to section19 Auditor Gen- ofthe working Itshouldbenoted these ing papers. that ofourauditworkconsidered to part beanintegral that are andfindings reports auditreports draft activities,we ofourreporting prepare course In the CONFIDENTIALITY OF WORKING PAPERS CONFIDENTIALITY CODE OF PROFESSIONAL CONDUCT CODE OF PROFESSIONAL The Board of Internal Economy—an all-party Economy—anThe Board ofInternal all-party - - - It brings togetherIt brings legislative auditors andmembers Canadian CouncilofPublic Accounts Committees. annualconference ofthe the been heldjointly with has,for anumber ofyears, This annualgathering inIqaluit,from August 19(CCOLA) to 21, 2012. CanadianCouncilofLegislative Auditorsof the This year, Nunavut annualmeeting 40th hosted the Senior Management Committee. Reviewment Advertising Office’s make upthe Resources and ofCommunicationsandGovern- Managers Directors,ofHuman andthe the eral, (seeFigure auditstaff 1). ous other teams are anumberofauditManagersthe andvari Directors androunding out Assisting the portfolio. assigned the auditswithin and isresponsible for the headedbytion, are aDirector, each whooversees organiza- own government’s ministry based onthe of auditactivity. are loosely which Theportfolios, areas various entities andto inthe foster expertise framework attempts that to alignrelated audit The Officeisorganized intoteams—a portfolio fivethereafter. andevery years hiring on check andundergodeclaration apolicesecurity required to complete anannualconflict-of-interest conflict-of-interest situations.Allemployees are sure requirements steps to andthe betaken to avoid itees. TheCodealsoprovides guidanceondisclo- Legislative Assembly,the publicandouraud- the responsibilities Office’s to the describes and further Legislative Auditors Legislative Auditors Canadian Council of ganization Personnel Office The Code explains why these expectations exist exist whyexpectations The Codeexplains these The Auditor DeputyAuditor the General, Gen-

and

- exchange with the Auditor the exchange ofTasmania. General with ina staff visitors. TheOfficealsoparticipated delegations from Chinahave most frequent beenthe last few Inthe years, countries. gations from other several Officemet dele- with the report, ered by this audityear cov the During them. with experiences and to share ourvalue-for-money audit andother Office roles andresponsibilities ofthe to discussthe to visitors anddelegations meet from abroad with receives Officeperiodically requestsauditing, the As anacknowledged leaderinvalue-for-money ideas andexchanging information. andprovides for sharing ausefulforum territories, federal provinces governmentfrom the and the and Standing Committeesof the onPublicAccounts tion of Aboriginal students, the Ontario Diabetes Ontario students, the tion ofAboriginal including educa- toalso ofimportance Ontarians, tion, we ofgovernment avariety examined activities quality andteaching viders, atuniversities. Inaddi- pro- justice service home placementprocess, youth long-term-care facilities,the by independenthealth provided planimplementation, transit rapid services Metrolinx’s cancerscreening programs, Ontario’s provincially fundedagencies asCancerCare such public sector aspectsofanumber of examined year ourvalue-for-money broader auditsinthe province’s annualExpenditure Estimates. the This and reviewed process for Legislature’s reviewing the 17 AirAmbulanceandRelated onOrnge Services, undersection money audits,issuedaspecialreport Office.In fortotal, the we conducted 12 value-for- The 2011/12 fiscal successful year was another year International Visitors International Visitors Office byResults the Produced The Officeofthe Auditor General ofOntario

This

Year

- 429

Chapter 7 Chapter 7 430 *Staff below manager level shift between portfolios to address seasonal financial statement auditworkloadpressures. statement toaddressseasonalfinancial between portfolios belowmanagerlevelshift *Staff 30,2012 September Figure 1:OfficeOrganization, 2012 oftheOffice ofthe Annual Report Auditor General of Ontario Deputy AuditorGeneral Gary Peall Gary Financial StatementAuditPo Va Jim McCarter Health Megan Sim Romano Mary Fletcher Kandy To Wa John McDowell,Director Crown Agencies (1),Finance Crown Agencies and Revenu Community andSocialServices, Auditor Genera nt hug OscarRodriguez Helen Chow Cheung Anita Ariane Chan Denise Young, Manager Manager Gigi Yip, Rudolph Chiu,Director naGefn JanetWa Michael Okulicz Inna Guelfand Ti Sandy Chan,Manager Vanna Gotsis,Director no Bove m Chatzidimos lue-for-money AuditPo lter Allan,Manager e l Alla Volodina Lisa Li Zhenya Stekovic rf olios andStaf rt folios and n Quality AssuranceandSpecialProjects Human Resources Shariq Saeed Snowden,Manager Kristin Paul Amodeo,Manager Shayna Whiteford Annemarie Wiebe,Manager St Re andar becca Yo f* St af f* ds andResearch iia iirv LindeQiu Jennifer Fung Dimitrov Dimitar Sally Chang Kevin Aro Manager Naomi Herberg, We Susan Klein,Director Margaret Chen Zachary Thomas Zachary Ts Cynthia DeSousa Constantino Chen Margaret Izabela Beben Teresa Carello,Manager Bell,Director Laura Health andLong-term-careProviders Crown Agencies (2) Crown Agencies Education andTr Michael Katsevman Dora Ulisse Dora EllenTepelenas Michael Katsevman Rumi Janmohamed Jaffer Zahra Johan Boer Ta Emanuel Tsikritsis, Manager Director Fitzmaurice, Gerard sipovich, Manager ra ndy Cumbo,Manager Bhst NinaKhant Beheshti aining Ti Veronica Ho Ingrid Goh Ro Mark Smith Pratheeskar Mythili ff ger Munroe any Ya u o Operations Communications andGovernmentAdver Administration Information Technology Christine Wu Christine Persaud Shanta Sohani Myers Maureen Bissonnette Peter Lee Shams Ali Randoja Tiina Shirley McGibbon Mariana Green Pedias,Manager Christine We agrtLm Michael Ya Lam Margaret Alfred Kiang Tanmay Grewal Kiran Ra Vivian Sin,Manager Ro Fraser Rick MacNeil,Manager Vince Mazzone,Director Justice andRegulatory Nick St Kim Cho,Manager Gus Chagani,Director Georgegiana TanudjajaGeorgegiana Allen Fung Bill Pelow, Director acaDSua Shreya Shah Li-LianKoh Ka Marcia DeSouza Amerski Bartosz arnHna JingWang Ho Kristy Lauren Hanna Environment andNaturalResources Public Accounts hetGl AliceNowak shmeet Gill trina Exalta ndy Ng Gpa BrianWanchuk Gupta av gers ropoulos, Manager , Manager in AlexanderTr cion Celia Ye Ro Ruchir Patel byn Wilson tising Review ung rm uong olinsky Prepared by the Office of the Auditor General ofOntario the AuditorGeneral Prepared by Officeof the Figure 2:Five-year ComparisonofSpending(AccrualBasis)($000) chapter outline endofthis ments reproduced atthe The following discussionandourfinancial state- advisers. andexpert ofouragent auditors,that staff contract hard work aswell as anddedicationofourstaff, would clearly not have the beenpossiblewithout discussed inChapter 5. further , as Act,2004 Advertising Government under the conducting ourfinancial-statement audits. for standardsand assurance andmethodology fully implement significant revisions to accounting continuing ourinvestment to intraining success- our key financial-statement auditdeadlineswhile 40Crownmore than agencies. We again met allof cussed inChapter 2),aswell statements of asthe dis- consolidated financial statements (further tion, we province’s are responsible for auditingthe Provincial PoliceOntario services. Attorney andaspectsofthe General, ofthe Ministry of overdue Law Divisionofthe Criminal taxes, the Strategy, collection the CleanProgram, Drive the Professional and other services Professional andother Salaries andbenefits Actual expenses budget Approved Returned province* to Total Other Travel andcommunications Rent * Financial Accountability assets, deferred leaseinducementsandemployeeassets, deferred futurebenefitaccruals). These amountsaretypicallyslightlydif The results produced by the Office this year Office The results produced by the We successfully met ourreview responsibilities earlier AttestAs mentionedinthe Audits sec- ferent than the excess of appropriation over expenses as a result of non-cash expenses (such as amortization of capital ofcapital excessofappropriationoverexpenses asaresultofnon-cashexpenses(suchamortization the ferent than 2007/08 15,308 14,002 1,608 9,999 1,525 1,033 1,048 397 budget over andexpenditures last five the years. 2011/12 fiscal year. early pre-election report review inthe ofthe the some value-for-money resources to helpcomplete slightly from 2011, mostly becausewe diverted responsibilities. Thesepercentages changed only ing 3%was devoted two other to statutory the province andsome40ofitsagencies. Theremain- the annualfinancial statements of auditsofthe the lic Accounts. 30%was About devoted to completing of most interest Standing to Committee the onPub- value-for-moneyperform is activitythat audits, the almost two-thirds ofourresources were usedto 10that financial statements andindicates to the responsibilities. Thisbreakdown isprovided innote 2011 was) andourgovernment review advertising inanelectionyear,pre-election report which government’s reasonableness ofthe review the chapter,this Auditor the isrequired General to 2011 earlier (asdescribed in pre-election report statement government’s audits,ourreview ofthe is responsible for: value-for-money audits,financial four by mainactivitiesourOffice our expenses the we have presentedtimeabreakdown first for the of standards these ing standards. Inaccordance with public-sectorprepared inaccordanceaccount- with year. Thisyear ourfinancial statements have been the financial results for Office’s the 2011/12 fiscal 2008/09 Figure 2provides acomparison ofourapproved 14,534 16,245 10,279 1,561 1,096 1,051 1,776 332 The Officeofthe Auditor General ofOntario 2009/10 14,853 16,224 10,862 1,498 1,489 1,069 1,073 360 2010/11 16,224 15,163 11,228 1,222 1,036 1,491 1,071 337 2011/12 16,224 15,241 11,039 1,667 1,016 1,216 303 997 431

Chapter 7 Chapter 7 432 competitive, we must pay ournewly qualified staff us.To year andremained with the nation during be professional accountingdesig - their earned trainees to employ fewer student asmany trainees, ofour to fillin. professionalcontract services We continue ourbenefitcosts andourneed increased both for to retirees year, this costs, whilelowering salary As well, leaves parental andseverance payments by staff. delays inreplacing anddeparting retiring additional responsibilities were offset more than abilityto take demonstrating on the staff those year andfor the accounting designationsduring professional by their whoobtained trainees earned year,again this any increases dueto promotions the previous benefit costs rose 5.2%from year. year while costs declined3.2%this Our salary follows. challenges we andsomeofthe expenses are facing province faces. the tightfiscalcircumstances upgiven the staff sector. Inaddition, we have to beenreluctant fully private professionalstion increases for inthe such ranges have simply not kept compensa- pacewith Toronto jobmarket—our salary public-service the competitiveing qualified in professional staff andretainhistorically- facedchallenges inhiring our abilityto comeinunderbudget we isthat have funds totalling $5.3 mandate unspent while returning on ourOffice’s last four years, yet we have successfully delivered under budget. Ourbudget hasbeenfrozen over the (2.1% in2010/11) andwere again significantly tions have beenrelatively inrecent stable years. remainder.comprised most ofthe Thesepropor andrent whileprofessional services andother staff, 2010/11) andbenefitcosts relatedfor our to salary 72%spending andshows more (74% than that in Figure major 3presents the components ofour 2012 oftheOffice ofthe Annual Report Auditor General of Ontario SALARIES AND BENEFITS AND BENEFITS SALARIES With the legislated freeze on salary ranges legislated freeze onsalary the With changes inour A more detailed discussion ofthe Overall, increased just ourexpenses 0.5%

million. Themainreason for - in the first five first in the years following qualification. rise fairly for qualified Salaries accountants quickly were they considerably paidastrainees. more than experienced staff retire over staff few next experienced years. the auditswillonly increase asmore ofourmostthese andenhancing ourcapacitytotaining perform aspossible.Thechallenge ofmain asmuch staff wedemands that usehighly qualified,experienced positions. Thegrowing complexity ofouraudits Figure 4shows, we still have anumberofunfilled as professional Thisisone reason that, accountants. scalesfor more experienced salary public-sector levels. We quickly andbroader- fallbehindprivate- andbenefitsare competitiveour salaries these at atmore juniorlevels,to given beprimarily that continues Ourhiring economic uncertainties. hasremained fairlyaccountants robust despite market continueasthe departures for professional 102, down from 104 previous year. inthe Staff yearing over declinedto ofthis about course the arose. Asaresult, ouraverageopportunities staff as butmore juniorstaff experienced and hiring replacement seniorstaff delaying ofretiring the departed, upwhen staff cautious aboutstaffing tinuing needfor we cost containment, remained Prepared by the Office of the Auditor General ofOntario the AuditorGeneral Prepared by Officeof the 2011/12 Figure 3:Spendingby MajorExpenditureCategory, able to the salary ranges of similar positions in the ranges ofsimilarpositionsinthe salary able to the benefits (72.4%)benefits Salaries and (10.9%) services other Professional and With the economic uncertainty and the con andthe economicuncertainty the With Under the Act, our salary levels mustUnder becompar Act, oursalary the Re nt (6.7%) Other (8.0%) Other communications (2.0%) Tr av el and - - - - sionals to meet ourlegislated responsibilities infrastructure. IT Office’s to enhancethe IT consultingexpertise of fill-insas contract requiring the purchase well as last year dueto anincrease leaves inparental total costs were Such expenditures. higherthan expendituresignificant area atalmost of 11% of and mostrepresent ournext firms CA and contract professionals contract includeboth These services our financial statements. 2011in the calendaryear are disclosedinNote 8to ofourhighest-paid staff Thesalaries accountants. competitors for professional are ourprimary which ($150,400), firms CA ing for professional service and, most importantly, 26%lower work those than not-for-profit inthe those than sector ($129,900) ingovernment ($111,200)for CAs were 14% lower Accountants publishedin2011, average salaries CanadianInstitute by ofChartered the survey sectors offer.private According most recent to the not-for-profit the both that the salaries and the with government. Theseranges remain uncompetitive ofOntario the AuditorGeneral Prepared by Officeof the 2005/06–2011/12Figure 4:Staffing, 120 10 10 110 115 80 85 90 95 70 PROFESSIONAL AND OTHER SERVICES AND OTHER PROFESSIONAL 75 0 5 We continueto have to rely profes oncontract - 2005/06 105 16 89

2006/07 115 99 16

2007/08 1 1171 117 1 110 110 77

2008/09

2009/10 Actual average FTEs positions Unfilled 10 313 13 7117 17 41 2010/11 04

2011/12 117 10 15 2 - the market for such services as contracts expire. ascontracts marketthe services for such standards.ing andassurance We continueto test required to implement ongoingchanges to account- additionalhours andthe staff payies they their the highersalar firms we becauseof workCA with approved complement of117 staff. no longer realistically upto to our afford staff Further, four years after ofbudget freezes we can several yearswith ofpost-qualifying experience. levels,petitive particularly for salary professionals our approved fullcomplement given ouruncom- downturn ithasremained difficult for us to reach are requested. economic Also,even the during ifsignificantin-yeardisruption cuts to ourbudget itprovidestimes, inthat andless more flexibility economic uncertain to particularly staffing, during asfor leaves, parental approach such isaprudent to needs, fill using moretemporary staff contract province.agencies andthe We alsobelieve that the financial-statement auditsof Crownfinalizing given more complex work andtightdeadlinesfor we significantly have more travel beenincurring Student SuccessInitiatives. Ingeneral, however, for Disabilities,and People with Services portive extensive travel: Forest Management Sup- Program, travel usual.Last year, than audits required three money auditswe selected generally required less 10%more than from last year. Thevalue-for- Our travel andcommunications costs declined fall of2011. leaserenewal commencinginthe terms ofthe part Office aggressively negotiated a rent reduction as becausethe total spendingandwilldeclinefurther Accommodation costs declinedasapercentage of costs,in buildingoperating particularly utilities. last year,less than to owing adecline primarily Our costs for accommodationwere again slightly TRAVEL ANDCOMMUNICATIONS RENT We continueto costs incurhighercontract for The Officeofthe Auditor General ofOntario - 433

Chapter 7 Chapter 7 434 equipment amortization wereequipment amortization largely offsetting. salary,General’s transfer payment and the to CCAF year. Auditor for Smallerchanges inexpenses the reviewvolume declinedthis their ofads requiring , asthe Act,2004 Advertising Government the required to administer services advisory in expert These increases were by reductions offset partially totransition frameworks. new financial reporting agencies we andthe auditpreparestaff for the Training increased $35,000to expenses helpour implementation ofelectronic working papers. licencesaswe and auditsoftware phaseinthe licences for additionalITsupport ment, primarily also spent$70,000more onsuppliesandequip- Finances we, which oninJune2011. reported We 2011review ofthe onOntario’s Pre-Election Report we hiredist expertise to complete ourstatutory year, andspecial- dueto staff contract primarily costs were Such expenses. 13.5% last higherthan and equipment maintenance, andstatutory training supplies costs includeasset amortization, Other broader-public-sector organizations. ofourmandate expansion tocosts audit sincethe 2012 oftheOffice ofthe Annual Report Auditor General of Ontario OTHER FINANCIAL STATEMENTS FINANCIAL The Officeofthe Auditor General ofOntario 435

Chapter 7 Chapter 7 436 2012 oftheOffice ofthe Annual Report Auditor General of Ontario The Officeofthe Auditor General ofOntario 437

Chapter 7 Chapter 7 438 2012 oftheOffice ofthe Annual Report Auditor General of Ontario See accompanying notes tofinancial statements. deficit,Accumulated of end year deficitAccumulated Net Less Excess of Revenue Expenses 2011 31, March 2012and 31, March Ended Year the For Deficit Accumulated and of Operations Statement Office of the

operations operations : returned Province tothe Consolidated RevenueConsolidated Fund Statutory expenses: Transfer payment: CCAF and equipment Supplies Training development and andTravel communication Amortization of capital assets Professional and other services Office rent Employee benefits (Note Salaries wages and

appropriation Total expensesTotal (Note

deficiency , asrestated

Auditor General of Ontario of General Auditor

over expenses

Statutory services Government Advertising Act Auditor General Act

s 2& s -

FCVI Inc.

[ Note , beginning, ofyear

– s

9 Voted appropriation

)

& 10 & 3 ( B)

) ]

(Note 2)

[ Note 3

( B) ]

(Unaudited

16,224,100 1 – 1,7 1,062,400 2,041,200 9,755,400 6 Note 13) , 222,700 377,500 3 418,800 150,000 2 Budget 73 78 30,000 14 2 4 20 ,000 ,600 ,500 , 1 1 00 — $ 2

16,224,100 15, ( ( 2, 2, 1,666,589 1, 2,085,050 8,953,56 246,575 208,311 165,152 303,072 324,489 561,021 547 997,433 983,508 187,582 016,280 240,592 ( 13,925 72,989 Actual 10,942 20 ,096 1 $ 1 2

) ) )

16,2 15,1 ( ( 2, 2 1,490,944 1,035,522 1,9 9,245,160 1, 1, (Note 12) ( ,386 1 229,147 136,574 130,700 337,301 339,316 5 108,434 221,911 0 60 61,775 Actual 47 65,060 82 61 2 62 4, 20 ,639 ,096 ,457 ,895 ,272 ,828 1 1 00 $ 1 ) ) )

See accompanying notes tofinancial statements. Cash Cas Increase (decrease) in cash and cash equivalents transactionsFinancing transactionsInvesting Capital transactions operatinCash by provided non Changes in O (outflow)Net inflow ofcash following tothe related activities: 2011 31, March 2012and 31, March Ended Year the For Cash of Flows Statement Ontario of General Auditor the of Office

perating h Purchase of inducement deferredleasein Increase Decrease accountsin payable accrued and liabilities (increase)Decrease harmonizedin sales taxes recoverable inducement leasein receivable Increase (increase)Decrease duein from Consolidated Revenue Fund obligation employeeAccrued benefits Amortization of capital ass Net operations deficiency and cash equivalents cash and equivalents cash and transactions - cash working capital tangible

capital assets g transactions , , ofyear end , ofyearbeginning

ets

The Officeofthe Auditor General ofOntario (322 (188,122) (209,475) ( (287,475) 416,564 494,564 184 324,489 290,695 500,170 207,089 308,799 (92,250) ( 13,925 6,323 20 ,225) ,000 1 — — $ 2 ) )

(300,677) (390,609) (128,927) (16 500,170 370,802 129,368 430,045 185,368 704,904 2 339,316 44 66,000 0 20 ,639) ,677 1 — — — — $ 1

439

Chapter 7 Chapter 7 440 2012 oftheOffice ofthe Annual Report Auditor General of Ontario deficit at March 31, 2011 at March 31, 2011 de 2011 31, March at expensebenefits decreased by $5,000 for fiscal 2010/2011 The resultingThe periods. expense related to accumulated unused render services to the Office in forreturn the benefits. An adjustment was made to recognize a liability and an recognized for post for recognized the under tocompensated related absences benefits employee for future obligation an not Office record the did Previously, One statements for which statements Canadian public sector accounting (“PSA”) standards. These financial statements the are first financial has adopted (“Office”) of Ontario Auditor General the of Office the 2012fiscal year, the with Commencing Conversion2. to Public Sect As required by the Fiscal Transparencyand Accountability Act, 2004 repo General Auditor the Acts, both Under whether they meet the standards required by the Act. oftypes printedadvertising, matter or reviewable messages proposed by government offices to determine the under Additionally, organizations. sector public broader in and operations government in money value and accountability promotes ofGeneral so,Auditor the of Office the the doing In Crown. agencies that receive government grants, ofand the fairness of financialthe statements of Provthe sector public broader the in of institutions programs, government of audits independent conducts General Auditor of Act Auditor the General provisions the with accordance In Nature1. Operations of 2011 31, March 2012and 31, March Ended Year the For Statements Financial to Notes Ontario of General Auditor the of Office report

adjustment ed

on the 20 the on nraet cuuae eii rmrcriguue C rdt (0,0) (209,000) (204,000) Accumulated deficit as restated, beginning of year Increase toaccumulated deficit recording from unused MCO credits Accumulated deficit as originally reported, beginning of year Management Compensation Option increase

to previously reported figures resulted figures reported to previously 11 - employment benefits and compensated absences earned

creased creased theOffice has applied Canadian publicsector accountingstandards. Pre

to to Government Advertising Act,20 Government Advertising - account

and the comparative accumulated deficit is presented below. ispresented below. deficit accumulated comparative the and Election Report prepared by the Ministry of Finance. to

$204,000 s payable and accrued liabilities or Accounting Standards MC . rts directly to the Legislative Assembly. Legislative the to directly rts n to PSA standards on the accum the on standards PSA to n conversio the of impact The O credit (“ MC s and beenhas applied retroactively with restatement of prior O”

04, the Auditor General is required to review specified ). PSA standards require that a liability and an expensebe standards an PSAthat and liability a require ).

from the adoptio the from

and a and and various other statutes and authorities, the the authorities, and statutes other various and

ccounts payableccounts and recorded liabilities accrued at April 1, 2010 was $209,000. Employee , the Auditor General General Auditor the 2011 June in

n of these accounting standards. standards. these of accounting n

in periodthe in which employees

(2,547,096) (2,343,096)

2011

ince and numerousince and

(2,386,457) (2,177,457) 2010 also also - $ ulated ulated for

- accounting, including the capitalization and amortization of capital capital of assets amortization and capitalization the including accounting, cashmodified basis, an excess or deficiency of revenue overexpenses arises from the application of accrual capital assets is recorded on straight the isrecorded assets capital c Tangible (C) TANGIBLE CAPITAL ASSETS appropriations. inducements • Office’sThe financial assets and financial liabilities are accounted for as follows: FINANCIAL(D) INSTRUMENTS principles financialThe statements beenhave prepared in accordance with Canadian genera 3. Significant Accounting Policies 2011 31, March 2012and 31, March Ended Year the For Statements Financial to Notes Ontario of General Auditor the of Office

are returnedto appropriations Unspent of Province the Ontario. from voted appropriations annual through isfunded Office The VOTED(B) APPROPRIATIONS year that the events giving rise to the expense occur and resources consumed.are financialThese statements are accounted for on an basisaccrual whereby expenses are rec ACCRUAL(A) BASIS follows: • • • arising from its financial instruments due to their nature. It is management’s opinion that the Office is not exposed to any interest rate, currency, liquidity or credit risk

fair value. Cash cashand equivalents subjectis to an insignificant risk Due fromFundrecorded cost.Revenue is Consolidated at Due amount. be paid has not been made as it is not expected that there would be significant a fromdifference the recorded actually will benefits these when about on assumptions based actuarial estimate 31,2012. A value fair March Accrued employee benefits obligation is recorded costat based on the entitlements earned by employees up to cost. at recorded are liabilities accrued payable and Accounts

apital assets are record are apital assets established by the Canadian Public Sector Accounting Board Accounting Sector Public Canadian bythe established

and the recognition of employee benefit costs earned to date but that will be funded from future be future from thatfunded to but will date earned benefit costs employee of recognition the and

the Province’s Consolidated Revenue Fund each year. As the voted appropriation

Leasehold improvements Furniture Computer software Computer hardware

tangible tangible of Amortization amortization. cost accumulated less ed athistorical and fixtures - line methodline over the estimated useful lives of the follows:assets as

The remaining of term the lease 5 years 3 years 3 years

of change in value so carrying value approximates

The Officeofthe Auditor General ofOntario . The.significant accountingpolicies are as , the deferral and ofamortization lease

lly accepted accounting accounting accepted lly

ognized in the fiscal fiscal the in ognized is

prepared

on a a on 441

Chapter 7 Chapter 7 442 2012 oftheOffice ofthe Annual Report Auditor General of Ontario Accumulated amortizationAccumulated Cost 4. from differ could financialthese statements. results Actual are reviewed annually to reflect new information as it becomes available. Measurement uncertainty exists in Estimates basedare on the best information available theat time of preparation of financial the statements and obligation. Items therequiring of use significant estimates include: useful life of capital assets and accrued employee benefits statem financial the of date management to make estimates and assumptions that the affect reported amounts of assets and liabilities theat requires standards sector accounting public Canadian with accordance in statements financial of preparation The (E) MEASUREMENT UNCERTAINTY Significant3. Accounting Policies (Continued) 2011 31, March 2012and 31, March Ended Year the For Statements Financial to Notes Ontario of General Auditor the of Office Net Book Value, March 2012 31, Closing balance, March 31, 2012 31, balance, MarchClosing Opening balance, April 1, 2011 2012 31, balance, MarchClosing Opening balance, April 1, 2011 Tangible Tangible

Disposals Am Disposals Additions ortization

Capital Assets

ents, and the reported amounts of revenue and expenses during the reporting period.

Computer hardware (153,590) (153,590) 283,522 403,848 159,472 397,966 687,370 243,826 597,134 $

Computer software 142,490 210,495 100,245 147,169 352,985 340,833 (36,919) (36,919) 49,072 these estimates.

and Fixtures and Furniture (176,288) (176,288) 152,204 294,185 211,914 378,491

59,710 34,307 9,711 $

Improvements Leasehold 108,257 241,566 211,101 349,823 11 235,868 30,465 3,955 — — $

1,008,113 1,050,421 1,602,092 1,552,325 (366,797) (366,797) 593,979 324,489 416,564 Total $

follows: benefitsfor accounted are as These employees that earnedhave these ben future forliability benefits earned by the Office’s employees is included thein estimated forliability provincialall Act General Although the Office’s employees notare members of the Ontario Public Service, provisionsunder in the Auditor 6. Obligation for Future Employee Benefits legislat arrangements and standard commercial terms. Accrued vacation, salaries and benefits recordedare based on employment third with transactions business to normal largely relates payable Accounts Accounts5. Payable and Accrued Liabilities Net Book Value, March 31, amortizationAccumulated Cost 4. 2011 31, March 2012and 31, March Ended Year the For Statements Financial to Notes Ontario of General Auditor the of Office Closing balance, March 31, balance, MarchClosing Opening balance, April 1, 31, balance, MarchClosing Opening balance, April 1, Tangible Tangible

Disposals Amortization Disposals Additions severanceAccrued benefits and salaries Accrued Accounts payable , the Office’s employees entitledare to the same benefits as Ontario Public Service employees. The

Capital Assets

, vacation and MCO creditsvacation , and

2011 ed entitlementsed 2010 2010 2011 2011

(Continued) efits and is recognized in the Province’s consolidatedProvince’sefitsthe recognizedfinancialstatements. is in and

Computer . hardware 155,730 337,310 597,134 10 585,915 199,168 397,966

(95,074) (95,074) 6,293

Computer

software March 31,2012 140,165 340,832 180,559 230,634 193,663 147,169 (70,361) (70,361)

77,365

1,647,041 $ 983,000 409,284 254,757

The Officeofthe Auditor General ofOntario $

and Fixtures and

Furniture - March 31, party vendors and is subject to to subject is and vendors party 237,499 378,491 364,666 294,185 56,686 13,825 84,306 (restated –(restated 1,739,291 1,061,000 — — $ 369,301 308,990 N

ote 2) ote 2

011 Improvements $

Leasehold

161,566 235,868 235,868 211,101 49,535 24,767 April 1,2010 (restated –(restated 2,129,900 1,005,000 — — — $ 773,293 351,607 N

ote 2) ote

1,552,325 1,417,083 1,050,421 (165,435) (165,435) 339,316 876,5 300,677 501,904 $

Total 40 $

443

Chapter 7 Chapter 7 444 2012 oftheOffice ofthe Annual Report Auditor General of Ontario included in accounts payable and accrued liabi accrued payable and in accounts included reflected in the accrued employee benefits obligation, less any amounts payable within one year, which are be employee in thesein financialstatements. These costs for the year amounted to $274,000 (2011 – employeesby recognizedare by the Province earnedwhen by eligible employees, costs these are also recognized Although the costs of any legislated, severance compensated absences ACCRUED(B) EMPLOYE Deficit. Accumulated thefor as follows: of remainingleaseis termcommitment on 31, October 2021 expires which premises torent lease Office operating an has The s Commitment 7. acc and Services Government of Ministry Ontario othernon of cost The OTHER(C) NON-PENSION POST $ . Office Office’s the of The required annual or obligations payments not assets are valuations funding of actuarial surpluses any viable, funds arefinancially the pension that orensuring unfunded liabilities arising from statutory sponsor of the PSPF, determines the Office’s paymentsannual to the fund. As sponsor the sole the is which Ontario, of Province The provincial agencies. many ofand Province the for employees plan Office’sThe employees participate in the Public Service Pension Fund (PSPF) which is a defined benefit pension PENSION(A) BENEFITS 6. Obligation for Future Employee Benefits (Continued) 2011 31, March 2012and 31, March Ended Year the For Statements Financial to Notes Ontario of General Auditor the of Office 719,119

(20 accountspayable and accrued liabilities Accrued Less: severance,creditsliability for vacation andMCO Total 11 nefits in the Statement of Operations Accumulated and Deficit. total The forliability these costs is - $

732,873 included in employee are benefits), expense in the Statement of Operations and employee benefits obligationemployee benefits Due one within year and included in

- - post pension E BENEFITSOBLIGATIO -EMPLOYMENT BENEFITS retirement benefits is determined and funded on ongoingan basis by the 201 201 201 201 201 201 7 6 5 4 3 2 – – – – – – 1 1 1 1 1 1 8 7 6 5 4 3

N and beyond lities, as follows: as lities, ordingly notis included in these financial statements.

2, 508,800 501 4 48 483 412

and unused vacation en vacation unused and 95 8 2,172,000 3,155,000

, , , , , 3 9 4 0 0 8,0 1,061 983,000 00 00 00 00 00 $ 2012

(restated (restated . The minimum The rental . $225,000) and are included

is is responsible for responsible 1,988,000 3 – titlements earned ,

049 N ote 2) ote 2011 ,000 ,000 $

earning Section 3(5) of this Act requires disclosure public paid Ontario of to benefits all and salary the Public8. Sector Salary Disclosure Act, 1996 2011 31, March 2012and 31, March Ended Year the For Statements Financial to Notes Ontario of General Auditor the of Office

an annual salary in excess of $100,000 of excess in salary annual an Wiebe, AnnemarieWiebe, ChristinePedias, Young, Denise Tsikritsis, Emanuel Stavropoulos, Nick Rogers, Fraser MacNeil, Richard Her Cumbo, Wendy Chan, Sandy Carello, Teresa Amodeo, Paul WalterAllan, Pelow, William John McDowell, Maz SusanKlein, VannaGotsis, Fitzmaurice, Gerard Chiu, Rudolph Chagani, Bordne, Walter Be Gary Peall, McCarter, Jim Name ll, Laurall, b zone, Vince erg, Naomi

Gus

Manager, Human Resources Government Advertising Review Manager, Corporate Communications and Audit Manager Audit M Audit Manager Audit Manager Audit Manager Audit Manager Audit Manager Audit Manage Audit Manager Audit Manager Audit Manager Director Director Director Director Director Director Director Director Director Director Deputy Auditor General Auditor General Position anager

. This disclosure for thefor 201 This disclosure . r

The Officeofthe Auditor General ofOntario 1 1 calendar year is as follows: as is year calendar 106,218 131,000 131,000 114,606 131,000 114,606 167,076 219,397 105,986 100,658 105,986 105,986 105,986 105,986 105,986 105,986 105,986 105,986 105,986 100,542 105,986 107,644 131,000 131,000 131,000 Salary $

- sector employees Benefits Taxable 4,871 160 196 196 171 196 171 250 1 145 1 159 159 1 1 1 1 159 159 169 159 162 196 196 196 5 59 5 5 5 5 $ 9 9 9 9 9

445

Chapter 7 Chapter 7 446 2012 oftheOffice ofthe Annual Report Auditor General of Ontario is as follows: as is V in reported expenses total of reconciliation acquisition rather than being capitalized and amortized over their useful lives. accrued employee future benefit costs the to for submitted approval of Estimates the preparation followed for policies the with accounting consistent basis Office’sThe Statement of Expenses presented in Volume 1 of the Public Accounts of Ontario was prepared on a Reconciliation9. to Public Accounts V 2011 31, March 2012and 31, March Ended Year the For Statements Financial to Notes Ontario of General Auditor the of Office

Board of Internal Economy, under which purchases of computers and software are expensed in the year of of year the in expensed software are and computers of purchases under which Economy, Board Internal of

Total expensesTotal per audited financial statements expensesTotal amortization ofdeferred lease induceme change accruedin future employee benefit costs amortization ofcapital assets purchase of capital assets per Public Accounts Volume 1 and d and eferred lease inducementeferred

olume 1 to total the expenses reported in these financial statements

olume 1Basis Presentationof

recognize 15, 15, ( 416,564 ( 1 324,489 240,592 240,093 13,426 06 20 ,000

499 Volume 1 also excludes the d thesed in financialstatements. 1 $ 2 ) )

– – (restated

15,162,828 15,002,189 ( 300,677 160,639 122,000 339,316 N ote 20 1 — 2) $ 1

)

A most travel costs and professional services, are allocated to that activity based on actual billings. on based actual that to activity allocated are services, costs professional and travel most costs that could not otherwise be identified with specifica activity. Expenses incurred for only one activity, such as activity as recorded by staff in the Office’s time accounting system, including timeadministrative and overh each to charged the on hours primarily based four activities main Office’s tothe been allocated have Expenses Expenses10. Activity by 2011 31, March 2012and 31, March Ended Year the For Statements Financial to Notes Ontario of General Auditor the of Office

Financial Statement audits auditsand special for money Value Government Advertising Pre Election Report Financial Statement audits and special for money Value Government Advertising Pre Election Report

audits

% %

11,228,054 11,038, Salaries & Salaries & 3,222,451 7,837,182 ,5,3 14534 467 4,825,709 24,657 1,445,314 3,355,738 7,417,946 Benefits Benefits 3,3 3,3 7,1 245,788 76,517 34,534 134,737 121,425 143,502 33,684 74.0 72.4 611

Operating Operating Expenses Operating Operating Expenses 3,756,882 2,233,192 3,532,133 1,414,654 2,076,615 31,595 46,781 Other Other

Other Other 6,330 24.7 23.3 2012 2011

The Officeofthe Auditor General ofOntario Expenses Expenses Statutory Statutory 402,641 284,827 445,099 131,756 265,415 18,382 22,915 23,271 2.7 2.9

15,240,592 15,162,828 10,19 9,916,553 4,660,020 176,291 322,039 58,396 Total 8,624 Total 100 100

100.0 100.0 31.7 6 30.7 67.3 ead ead 1.1 2.1 5 1.6 % % .1 .4

447

Chapter 7 Chapter 7 448 2012 oftheOffice ofthe Annual Report Auditor General of Ontario beenaudited. not has and Budgeted figures were Budgeted13. Figures presentation. current year’s tothe to conform reclassified been have figures comparative Certain Comparative12. Figures straight expenserent on a accommodation tofuture applied be to $322,225 of inducement a lease negotiated Office the premises, foroffice its the of arrangements part lease As Deferred Inducemen Lease 11. 2011 31, March 2012and 31, March Ended Year the For Statements Financial to Notes Ontario of General Auditor the of Office

approved by the Board of Internal Economy. It is presented for information purposes only only purposes for information presented bythe It is of Economy. Board Internal approved -

line basisline over the 10

costs. This deferred lease inducement is being amortized as a reductionof abeing amortizedlease inducementis as costs.This deferred t and Receivable - year period lease thatcommenced 1, 2011 November

. es in parentheses indicate fiscal periods endingona indicate fiscalperiods es inparentheses * Ontario Electricity Financial Electricity Corporation Ontario Educational CommunicationsAuthority Ontario Development Corporation Ontario CleanWaterOntario Agency (December31)* the Ombudsman Office of andPrivacy the Information Office of the Environmental Commissioner Office of the Children’s LawyerOffice of the Assembly Office of Fund Heritage Corporation Ontario Northern Livestock Financial Protection Board, Fund for Liquor Control Board ofOntario Legal AidOntario Investor Securities EducationFund, Ontario FinancialGrain Protection Board, Funds for CommissionofOntario Financial Services Election Fees andExpenses, Chief Electoral Officer, Centennial Centre ofScienceandTechnology Cancer Care Ontario Algonquin Forestry Authority Agricorp gencies whose accounts are audited General by theAuditor 1.

date other than March 31. March date than other Dat

A Commissioner Livestock Producers Commission Canola Soybeans, Corn, ofGrain Wheatand Producers Exhibit 1 Election Finances Act Finances Election Agencies oftheCrown Agencies Election Act Election Public Guardian andTrustee Province for the of Provincial Judges Pension Fund, Provincial Judges Provincial Advocate for Children andYouth Arts CouncilforProvince the ofOntario Pension BenefitsGuarantee Fund, Financial Commission Securities Ontario Commission Racing Ontario (December 31)* PlaceCorporation Ontario Transportation Northland Commission Ontario Mortgage andHousingCorporation Ontario MediaDevelopment Corporation Ontario Investor Immigrant Ontario Corporation Trust Heritage Ontario FoodOntario Terminal Board Financing Authority Ontario Energy Board Ontario Workplace Safety Board andInsurance St. Lawrence Parks Commission Niagara Parks Commission(October 31)* Motor Vehicle Accident ClaimsFund General the Auditor gencies whose accounts are audited by under anotherthedirection auditor of 2.

A Ontario Pension Board CommissionofOntario Services (December 31)* 449

Exhibit 1 Exhibit 2 450 es in parentheses indicate fiscal periods endingona indicate fiscalperiods es inparentheses * Higher EducationQualityCouncil ofOntario MarketingHealthForceOntario andRecruitment Hamilton Niagara LocalHealth HaldimandBrant Hamilton Niagara Community HaldimandBrant Foundation at Queen’s University atKingston Network St.Integration Erie ClairLocalHealth St.Erie ClairCommunityCare Access Centre Ontario eHealth Education QualityandAccountability Office ImprovingEcho: Women’s inOntario Health ofOntario Corporation Deposit Insurance Network Integration Champlain LocalHealth Champlain CommunityCare Access Centre WestCentral Network Integration LocalHealth WestCentral CommunityCare Access Centre Network Integration LocalHealth Central Network Integration EastCentral LocalHealth EastCentral CommunityCare Access Centre CommunityCareCentral Access Centre UniversityBrock Foundation Board ofFuneral Services Institute Research ofOntario Agricultural andotherrelated documents papers working audit reports, to General Auditor with full access byGeneral, theAuditor audited by other than an auditor the whose accounts areCorporations

date other than March 31. March date than other Dat Agency NetworkIntegration Care Access Centre (December 31)* Exhibit 2 Corporations Crown-controlled Ontario PowerOntario Inc.(December 31)* Generation PowerOntario (December31)* Authority Pension Board (December31)*Ontario andGamingCorporation Lottery Ontario andLandsCorporation Infrastructure Ontario Quality Council Health Ontario French-languageOntario Educational Growth Corporation Capital Ontario Agency Protection for Health Ontario and West NetworkNorth Integration LocalHealth WestNorth CommunityCare Access Centre Integration SimcoeMuskoka LocalHealth North SimcoeMuskoka CommunityCare Access North Network Integration East LocalHealth North East CommunityCare Access Centre North AssessmentCorporation Municipal Property Integration Mississauga Halton LocalHealth Mississauga Halton CommunityCare Access Centre Metropolitan Toronto Convention Centre Metrolinx Collection CanadianArt McMichael McMaster University Foundation System OperatorIndependent Electricity Hydro OneInc.(December31)* Centre Human RightsLegal Support Communications Authority Promotion Network Centre Network Corporation (December 31)* Toronto CommunityCare Central Access Centre WestSouth Network Integration LocalHealth WestSouth CommunityCare Access Centre Network Integration East LocalHealth South East CommunityCare Access CentreSouth Science North Royal Museum Ontario Owen SoundTransportation Company Limited Ottawa Convention Centre Corporation TrilliumOntario Foundation TourismOntario Marketing Partnership Corporation Waterloo Wellington Integration LocalHealth Waterloo Wellington CommunityCare Access Centre Waterfront Regeneration Trust Agency Walkerton CleanWater Centre University ofOttawa Foundation Trillium ofLife Network Gift Trent University Foundation Toronto Waterfront Revitalization Corporation Toronto IslandsResidential CommunityTrust Toronto Network Integration LocalHealth Central Network Corporation Exhibit 2 451

Exhibit 2 Exhibit 3 452 ized and the amount expended. Theseare outlined amountexpended. ized andthe order, date ofeach the stating amountauthor the payments inexcess ofappropriations, authorize Treasury allordersofthe Boardreport madeto AuditorAct, the isrequired General to annually Under subsection12(2)(e) Auditor General ofthe Agriculture, Food and Rural Affairs Agriculture, FoodandRural Aboriginal Affairs Ministry Attorney General Attorney Cabinet Office Exhibit 3 Treasury Board Orders Board Treasury - Aug 11, 2011 Jul 15, 2011 Jul 14, 2011 Jun 16, 2011 Mar 22,2012 Feb 15, 2012 Jun 16, 2011 Jun 16, 2011 May 20,2011 May 19, 2011 Date ofOrder Mar 13, 2012 Feb 15, 2012 Dec 14, 2011 Aug 17, 2011 Jun 16, 2011 Apr 14, 2011 Mar 22,2012 Mar 22,2012 Jul 29,2011 at the voteat the anditem level. expenditures such summarizes level, schedule this detail sub-vote by creating accountsatthe anditem related expenditures ordersinmore totrack these may following ministries in the Although table. Authorized ($) Authorized 48,968,000 48,372,600 23,028,600 31,707,500 13,402,800 11,000,000 20,100,000 21,801,400 11,172,100 1,000,000 4,000,000 3,000,000 4,000,000 2,700,000 3,139,400 5,718,600 3,497,700 800,000 286,500 300,000 500,000 601,000 Expended ($) 28,508,432 20,656,524 7 517,124 17, 21,797,092 3,139,400 8,875,024 4,000,000 5,718,600 4,875,024 286,500 300,000 406,240 — — — — — — — — — — Children andYouth Services Ministry Environment Economic DevelopmentandTrade Community andSocialServices Citizenship andImmigration Finance Energy Education Services Community SafetyandCorrectional Mar 14, 2012 Aug 19, 2011 Aug 11, 2011 Jul 14, 2011 Jun 16, 2011 May 19, 2011 Date ofOrder Mar 22,2012 Jul 14, 2011 Jul 14, 2011 May 19, 2011 Apr 19, 2012 Apr 19, 2012 Apr 11, 2012 Mar 19, 2012 Aug 22,2011 Mar 30,2012 Mar 22,2012 Aug 11, 2011 May 17, 2011 Mar 22,2012 Feb 13, 2012 Jan 29,2012 Aug 9,2011 Aug 15, 2012 Aug 9,2011 Mar 22,2012 Mar 19, 2012 Mar 19, 2012 Dec 8,2011 Aug 11, 2011 May 19, 2011 Mar 22,2012 Authorized ($) Authorized 190,000,000 193,874,600 20,646,700 33,340,400 24,592,300 78,123,700 30,129,800 29,900,000 12,000,000 13,965,400 15,000,000 15,172,900 74,393,700 13,608,100 6,978,700 4,892,100 4,950,800 4,250,000 3,000,000 2,000,000 3,228,700 3,730,000 1,360,000 1,545,000 1,800,000 1,600,000 1,556,200 6,746,500 2,218,800 8,190,000 3,675,000 3,874,600 1,710,200 229,800 266,600 809,900 700,000 642,100 687,500 60,800 51,500 Expended ($) 170,240,051 166,365,451 12,073,864 17,622,007 21,676,651 71,406,605 74,111,760 13,787,865 17,392,207 7,952,253 1,234,189 2,601,805 5,808,966 2,182,459 2,705,155 1,234,189 3,874,600 1,419,858 7,952,253 8,937,305 1,967,622 Exhibit 3 229,800 266,600 659,962 863,214 634,183 863,214 687,500 — — — — — — — — — — — — — 453

Exhibit 3 Exhibit 3 454 2012 oftheOffice ofthe Annual Report Auditor General of Ontario Infrastructure PromotionandSport Health Services Government Ministry Health andLong-TermHealth Care Northern Development,MinesandForestry Northern ResourcesNatural andHousing Municipal Affairs Labour Jun 16, 2011 Mar 19, 2012 Aug 9,2011 Aug 9,2011 Mar 5,2012 Mar 2,2012 Feb 15, 2012 Aug 11, 2011 Aug 9,2011 Jul 14, 2011 Jun 16, 2011 Date ofOrder Apr 19, 2012 Mar 22,2012 Mar 19, 2012 Feb 15, 2012 Feb 6,2012 Oct 31, 2011 Aug 11, 2011 Aug 17, 2011 Mar 22,2012 Mar 22,2012 Dec 8,2011 Aug 11, 2011 Jul 14, 2011 Apr 14, 2011 Apr 19, 2012 Aug 11, 2011 Aug 11, 2011 Aug 11, 2011 Apr 19, 2012 Mar 22,2012 Aug 22,2011 Aug 11, 2011 Feb 7, 2012 Apr 19, 2012 1,898,670,600 Authorized ($) Authorized 1,791,017,500 186,256,400 111,503,900 175,000,000 40,642,000 37,629,800 36,440,000 29,629,800 52,000,000 13,900,000 90,744,200 21,000,000 20,413,900 8,050,400 9,768,000 3,000,000 5,000,000 3,325,200 2,000,000 8,290,500 2,000,000 4,500,000 2,000,000 5,000,000 2,650,000 1,324,400 1,000,000 1,724,000 3,512,800 3,015,000 3,150,000 6,106,400 1,105,600 752,000 550,000 238,000 450,000 792,000 753,000 821,400 67,400 1,831,225,961 1,744,746,316 Expended ($) 146,189,077 138,091,753 20,644,554 27,779,006 20,644,554 13,900,000 77,509,270 12,691,079 5,656,059 9,528,808 3,325,200 2,000,000 5,000,000 1,000,000 5,824,478 3,145,897 2,776,101 6,097,324 550,000 288,662 602,853 287,814 821,400 752,707 585,074 670,797 — — — — — — — — — — — — — — Training, CollegesandUniversities Tourism andCulture Revenue Research andInnovation Ministry Total Orders Board Treasury Transportation Jan 29,2012 Dec 8,2011 Aug 19, 2011 Aug 11, 2011 Aug 11, 2011 Jul 14, 2011 Apr 13, 2012 Mar 22,2012 Mar 19, 2012 Aug 17, 2011 Aug 11, 2011 Aug 11, 2011 Aug 11, 2011 Aug 11, 2011 Aug 11, 2011 Jul 14, 2011 Jul 14, 2011 Jul 14, 2011 Jul 14, 2011 Feb 15, 2012 Feb 13, 2012 Jan 29,2012 Nov 22,2011 Aug 11, 2011 Date ofOrder Feb 15, 2012 Mar 19, 2012 Feb 15, 2012 Mar 22,2012 Mar 19, 2012 Feb 15 2012 Aug 17, 2011 Mar 22,2012 Mar 22,2012 Mar 22,2012 Mar 22,2012 3,116,632,700 Authorized ($) Authorized 110,136,700 45,000,000 16,634,000 71,625,900 53,000,000 26,000,000 42,163,700 34,894,700 13,682,600 10,866,000 10,000,000 6,000,000 8,000,000 2,000,000 2,500,000 8,000,000 2,000,000 4,000,000 3,000,000 2,500,000 3,500,000 6,000,000 6,500,000 8,549,000 2,600,000 2,500,000 1,000,000 1,984,000 2,182,600 2,100,000 8,418,900 5,164,800 1,101,700 200,000 380,000 400,000 585,000 500,000 451,000 2,542,034,424 Expended ($) 26,848,559 44,997,983 32,578,383 27,481,969 10,000,000 9,680,393 6,000,000 2,500,000 8,000,000 2,000,000 3,000,000 6,000,000 2,500,000 2,400,477 7,168,166 1,155,210 8,789,742 5,164,800 2,400,477 1,935,317 2,192,105 1,101,700 7,872,916 Exhibit 3 200,000 380,000 400,000 868,562 — — — — — — — — — — — — 455

Exhibit 3