Report No. 5, September 2010
Inquiry into an Evaluation of the Effectiveness of the Performance Management Systems (PMS) audit mandate
Legislative Assembly of Queensland
Public Accounts and Public Works Committee
Inquiry into an Evaluation of the Effectiveness of the Performance Management Systems (PMS) audit mandate
September 2010
Report No. 5
PUBLIC ACCOUNTS COMMITTEE 52 nd Parliament
Chair Mr Ken Hayward MP Member for Kallangur Deputy Chair Mr Vaughan Johnson MP Member for Gregory Members Mr John English MP Member for Redlands Mr David Gibson MP Member for Gympie Ms Grace Grace MP Member for Brisbane Central (from 30 October 2007) Mr John-Paul Langbroek MP Member for Surfers Paradise Ms Carolyn Male MP Member for Glass House (to 30 October 2007) Mr Peter Lawlor MP Member for Southport
PUBLIC ACCOUNTS AND PUBLIC WORKS COMMITTEE 53rd Parliament
Chair Mr Wayne Wendt MP Member for Ipswich West Deputy Chair Mr Ian Rickuss MP Member for Lockyer Members Mr Michael Crandon MP Member for Coomera Ms Dianne Farmer MP Member for Bulimba Mr Ray Hopper MP Member Condamine Ms Mary-Anne O’Neill MP Member for Kallangur Mrs Christine Smith MP Member for Burleigh
Secretariat Research Director Ms Deborah Jeffrey Principal Research Officer Mrs Helen Bogiatzis Executive Assistant Mrs Marilyn Freeman
Contacting the committee Copies of this report and other committee publications are available on the Internet via the Queensland Parliament home page www.parliament.qld.gov.au or by contacting the secretariat at: Public Accounts and Public Works Committee Telephone: 07 3406 7576 Parliament House Fax: 07 3406 7500 George Street Email: [email protected] Brisbane Q 4000
Contents
Glossary of terms and abbreviations ...... iv Role of the Public Accounts and Public Works Committee ...... v Chair’s foreword ...... vii 1. Introduction...... 1 1.1 Recommendations in this report ...... 1 1.2 2004 Strategic review of QAO...... 1 1.3 Use of PMS audit mandate by QAO ...... 2 1.4 Committee’s actions since last strategic review...... 4 1.5 Conduct of the inquiry ...... 5 1.6 2010 Strategic review of QAO...... 5 2. Role of PMS audit mandate...... 6 2.1 What is a Performance Management Systems audit?...... 6 2.2 Does the PMS audit mandate fulfil its function in the overall accountability process for the Parliament? ...... 8 2.3 What improvements has the mandate brought to the accountability systems?...... 10 2.4 Disadvantages of the PMS audit mandate...... 12 Committee comments ...... 13 3. Is the concept of PMS auditing understood by both auditor and auditee?...... 14 Committee comments ...... 17 Recommendation 1...... 17 4. Is the PMS audit mandate being used effectively by QAO? ...... 17 Committee comments ...... 19 5. How does QAO assess if individual PMS audits have been effective?...... 19 6. How could the effectiveness of the PMS audit mandate be improved? ...... 21 Committee comments ...... 25 Recommendation 2...... 26 7. Should the Auditor-General’s mandate be extended to include Performance Audits?.....26 7.1 What is a performance audit? ...... 26 7.2 How do performance audits differ from PMS audits? ...... 27 7.3 Previous consideration of extension of the Auditor-General’s mandate to include performance audit ...... 28 7.4 Arguments in favour of extension of the Auditor-General’s mandate ...... 34 7.5 Arguments against extension of the Auditor-General’s mandate ...... 37 7.6 Resource implications ...... 39 7.7 2010 Strategic Review recommendations – performance audit mandate ...... 40
i
Committee comments ...... 42 Recommendation 3...... 43 Recommendation 4...... 44 7.8 2010 Strategic Review recommendations – Strategic Audit Plan...... 44 Committee comments ...... 45 Recommendation 5...... 46 Recommendation 6...... 46 Appendix 1 – PMS audit reports tabled by QAO during the review period ...... 47 Appendix 2 – Copy of Terms of Reference...... 50 Appendix 3 – Submissions received...... 51 Appendix 4 – Public hearing held 18 September 2009 ...... 52 Appendix 5 – Comparison of Auditor-General’s mandate in Australian jurisdictions ...... 53
ii
iii
Glossary of terms and abbreviations
AAASB Australian Auditing and Assurance Standards Board ACAG Australasian Council of Auditors-General ANAO Australian National Audit Office DETA Former Department of Education, Training and the Arts DEIR Former Department of Employment and Industrial Relations DES Former Department of Emergency Services DIP Department of Infrastructure and Planning DLGSR Former Department of Local Government, Sport and Recreation DoCS Former Department of Child Safety DPIF Former Department of Primary Industries and Fisheries DPW Department of Public Works DTRDI Former Department of Tourism, Regional Development and Industry EARC Electoral and Administrative Review Commission EPA Former Environmental Protection Agency FA&A Act Financial Administration and Audit Act 1977 . This act was replaced by the Financial Accountability Act 2009 and the Auditor-General Act 2009 which commenced on 1 July 2009. GBE Government Business Enterprise GOC Government Owned Corporation ICAA Institute of Chartered Accountants in Australia MOG Machinery of government NSWAGO New South Wales Auditor-General’s Office NZ New Zealand PAC Public Accounts Committee (52 nd Parliament) PCEAR Parliamentary Committee for Electoral and Administrative Review PAPWC Public Accounts and Public Works Committee (53 rd Parliament) PMS Performance management systems PSC Public Service Commission PWC Public Works Committee QAO Queensland Audit Office SDPC Service Delivery and Performance Commission. The SDPC was abolished, as of 1 July 2008, with its role included within the Public Service Commission. The committee Public Accounts Committee/Public Accounts and Public Works Committee UK United Kingdom VAGO Victorian Auditor-General’s Office
iv
Role of the Public Accounts and Public Works Committee
The Public Accounts and Public Works Committee is a statutory committee of the Queensland Parliament, established by the Parliament of Queensland Act 2001.
The committee’s area of responsibility as described in section 95 of the Act is:
(a) the assessment of the integrity, economy, efficiency and effectiveness of government financial management by
(i) examining government financial documents; and (ii) considering the annual and other reports of the auditor-general. (b) works (public works) undertaken by an entity that is a constructing authority for the works if the committee decides to consider the works. (c) Any major GOC works if the committee decides to consider the works.
In deciding whether to consider public works, the committee may have regard to:
(a) the stated purpose of the works and the apparent suitability of the works for the purpose; and
(b) the necessity for, and the advisability of, the works; and
(c) value for money achieved, or likely to be achieved, by the works; and
(d) revenue produced by, and recurrent costs of, the works or estimates of revenue and costs for the works; and
(e) the present and prospective public value of the works, including, for example, consideration of the impact of the works on the community, economy and environment; and
(f) procurement methods for the works; and
(g) the balance of public and private sector involvement in the works; and
(h) the performance of –
(i) the constructing authority for the works; and
(ii) the consultants and contractors for the works;
with particular regard to the time taken for finishing the works and the cost and quality of the works; and
(i) the actual suitability of the works in meeting the needs and in achieving the stated purpose of the works.
v
Government financial documents are defined in section 79 of the Act as:
a document tabled in the Legislative Assembly under the Financial Accountability Act 2009
the annual financial statements and annual reports of a government owned corporation
a document that would be a government financial document if it had been tabled in the Legislative Assembly as required by law.
The term does not include estimates of receipts for the proposed expenditure under an Annual Appropriation Act.
The committee may also refer issues within its area of responsibility to the Auditor-General for consideration.
An entity is a constructing authority for works if the entity is the State or a department. An entity is also a constructing authority for works if:
the entity is established under an Act, or under State or local government authorisation, for a public, State or local government purpose; and
the works are funded from the consolidated fund or the proceeds of a financial arrangement within the meaning of the Statutory Bodies Financial Arrangements Act 1982 .
A GOC is a constructing authority for works if the works are undertaken specifically or substantially for a community service obligation of the GOC or for major GOC works referred to the committee by the Assembly.
An entity (a commercial entity) is a constructing authority for works if, under an agreement for the works:
(a) the state or another entity representing the State -
(i) has, or will or may have, a financial liability or interest; or
(ii) has granted, or will or may grant land, or an interest in land or another right, privilege, monopoly, concession, franchise or interest; or
(iii) has contributed, or will or may contribute, resources of any kind; and
(b) the works have become, or will or may become, the absolute property of the State or another entity representing the State.
Major GOC works means works (other than public works) undertaken as part of a major infrastructure investment outlined in a GOC’s statement of corporate intent.
If the committee makes a recommendation in a report that a minister should take particular action about an issue, the minister must table a response within three months after the report is tabled.
vi
Chair’s foreword
This inquiry was commenced by the Public Accounts Committee of the previous Parliament, which sought public submissions on the inquiry terms of reference. The Public Accounts and Public Works Committee of 53 rd Parliament resolved to complete the inquiry and held a public hearing at Parliament House on 18 September 2009.
During the course of the inquiry the 2010 Strategic Review of the Queensland Audit Office was completed and the committee had regard for the recommendations made in that report in its consideration of the issues.
Whilst this inquiry was an examination of the Auditor-General’s performance management systems audit mandate, it became clear early in the inquiry that there was a strong undercurrent of opinion that the Auditor-General’s mandate should be expanded to encompass performance auditing. The committee heard strong evidence to support this suggestion.
The committee was ever mindful in its deliberations of the importance of the Auditor-General’s independence as this is an essential facet of the Westminster system of accountability. The committee considers that an extension of the Auditor-General mandate can be achieved without affecting that independence.
The committee has made six recommendations to government as follows:
1. The committee recommends that the Department of Premier and Cabinet, in its role as central agency, ensure that the QAO is supported in its efforts to ensure that agencies are made aware of both the extent of the Auditor-General’s mandate, and what is expected of agencies in respect of their compliance requirements under that mandate.
2. The committee recommends that the Department of Premier and Cabinet, in its role as central agency, implement action to ensure that all audit recommendations, including PMS/Performance audit recommendations, are considered by agency audit committees.
3. The committee recommends that the Auditor-General Act 2009 be amended to allow for provision of the following:
mandate to allow for the conduct of a performance audit of government agencies with the exception of government owned corporations;
a performance audit of a government owned corporation may be conducted at the request of the Parliament, the responsible minister, the Treasurer, or the Public Accounts and Public Works Committee;
Auditor-General not to question the merits of government policy.
the Auditor-General to have regard for the audit priorities of the Parliament when deciding upon the program of performance audits.
vii
4. The committee recommends that sufficient additional funding be provided to QAO to enable the strategic use of subject experts in all phases of performance audits.
5. The committee recommends that the Auditor-General Act 2009 be amended to allow for the provision of the following:
include a requirement for the QAO to prepare a three year Strategic Audit Plan for performance audits and to update the plan each year;
the Auditor-General consult with the PAPWC and consider their feedback, prior to finalisation of the Strategic Plan;
the Auditor-General consult with potential audit clients and affected parties and consider their feedback, prior to finalisation of the Strategic Plan;
following the passage of appropriation, the Auditor-General finalise the Strategic Audit Plan and publish it on the QAO website;
in undertaking performance audits, the Auditor-General to take into consideration performance management standards and guidance issued by government.
6. The committee recommends that sufficient additional funding be provided to QAO to enable preparation of the Strategic Plan.
It should be noted that the Legislative Assembly passed the Financial Accountability Bill and the Auditor-General Bill on 19 May 2009. The bills were assented to on 28 May 2009 and commenced on 1 July 2009 as the Financial Accountability Act 2009 and the Auditor-General Act 2009 . These Acts replace the Financial Administration and Audit Act 1977 (FA&A Act).
On behalf of both committees, I would like to thank all those organisations that took the time to provide submissions, the Auditor-General and the management and staff of the organisations who met with us during the course of the inquiry. The committee would also like to thank the Strategic Reviewers, Mr Graham Carpenter and Mr Mark Gray, for their thorough analysis and constructive recommendations.
Finally, I would like to thank the other Members of the committee for their continuing hard work and dedication in ensuring that there is a thorough examination of all of the issues brought before the committee.
Wayne Wendt MP Chair
viii
ix
Public Accounts and Public Works Committee Evaluation of Auditor-General’s PMS Audit Mandate
1. Introduction
1.1 Recommendations in this report
1. The recommendations in this report are addressed to the Premier and Minister for the Arts, as the responsible minister 1.
1.2 2004 Strategic review of QAO
2. The 2004 strategic review of the Queensland Audit Office (QAO) was tabled in the Legislative Assembly on 6 October 2004. Included in the review recommendations and conclusions was the need for the Public Accounts Committee (PAC) to examine more formally, either publicly or in camera, the Auditor-General’s reports to the Parliament and to undertake detailed scrutiny of performance management systems (PMS) audit reports.2 The committee supported the review comments. 3
3. In relation to the role of the PAC the reviewers made the following observations 4:
Review conclusions Reference: C.4 there would be benefit in an enhanced level of examination of the work of the Auditor-General and the QAO by the Parliament having regard to the significant powers and responsibilities vested in the Office of the Auditor-General by the legislation this enhanced scrutiny can best be undertaken by the PAC on behalf of the Parliament utilising existing available processes the high standard and standing of the Office of the Auditor-General would be further enhanced by greater involvement by the Parliament and the PAC areas where the PAC could play a more significant role include – = the annual PMS audit plan = proposed audit and other standards = emerging areas of audit interest = communication and media protocols = considering areas identified by the Auditor-General as being of future concern and interest = examining more formally, either publicly or in camera, reports to the Parliament by the Auditor-General the increased role of the PAC should not be such that it would compromise the independence of the Auditor-General. Review recommendations Reference: C.4 11. this is ultimately a matter for the Parliament and the PAC
4. The committee agreed with the reviewers that there was scope for it to play a more significant role in the areas outlined in their conclusions.
1 Parliament of Queensland Act 2001 (Qld), s.107 ministerial response to committee report 2 Anderson, R and Smerdon, H, Report on the Strategic Review of the Queensland Audit Office, October 2004: 43 3 PAC, Report No.69 Review of the Report of the Strategic Review of the Queensland Audit Office, June 2005: 6, 9 4 Anderson, R and Smerdon, H, Report on the Strategic Review of the Queensland Audit Office, October 2004: 43-44
1 Public Accounts and Public Works Committee Evaluation of Auditor-General’s PMS Audit Mandate
1.3 Use of PMS audit mandate by QAO
5. The 2004 Strategic Reviewers, Mr Henry Smerdon and Mr Richard Anderson, considered the issue of PMS audits and performance audits as part of their review. They advised that QAO had undertaken 27 reviews either wholly or partly under the PMS mandate. They considered that some of these reviews could have been categorised as other types of audits. They were concerned as to whether the PMS mandate was being addressed as Parliament originally intended. 5
6. They found that while some good work had been done on a sector-wide basis, there has not been the level of activity they had expected at the individual agency level. They found that resources available to the PMS audit function have been limited and have been diverted to other special audit activities. 6
7. The reviewers considered that the mandate had not proceeded with the level of commitment that may have been envisaged by the original legislators. Some of the reasons identified included:
the concept of PMS auditing is somewhat unique and not widely used in other jurisdictions. This has limited the capacity of the QAO to draw on the experience of others to assist with implementation; resources allocated to PMS audits have been limited and not fully or consistently applied, with diversion to other special audits a common occurrence. It would have been preferable for the Parliament to provide sufficient resources to the QAO to support its activities in this area; the Auditor-General has expressed publicly a strong view that QAO needs the wider performance audit mandate which could lead to the perception within the QAO and externally that the PMS audit function is not valued; the Auditor-General may be have been overly cautious and conservative in bringing PMS auditing to the public sector – and this is understandable given the potential for resistance within agencies and the need for an educative process initially; Parliament itself has probably not stated as clearly as it might, what is expected to be achieved with PMS audits …7
8. The reviewers’ observations of the performance audit mandate in other jurisdictions were that there was a significant commitment of resources to it. However, the capacity to undertake performance audits is limited. They found that the focus tended to be on specific programs rather than on overall evaluations of the effectiveness of management systems and consequently there was not a broad coverage of management processes but a concentration in depth of a limited number of specific budgetary allocations or expected policy outcomes. 8
9. The reviewers considered that PMS audits, properly implemented, with a balance of sector wide and agency specific initiatives, can cover a greater range of activities in a shorter space of time and make a positive contribution to improving overall management culture in the public sector. 9
5 Anderson, R and Smerdon, H, Report on the Strategic Review of the Queensland Audit Office, October 2004: 23 6 Anderson, R and Smerdon, H, Report on the Strategic Review of the Queensland Audit Office, October 2004: 24 7 Anderson, R and Smerdon, H, Report on the Strategic Review of the Queensland Audit Office, October 2004: 24 8 Anderson, R and Smerdon, H, Report on the Strategic Review of the Queensland Audit Office, October 2004: 25 9 Anderson, R and Smerdon, H, Report on the Strategic Review of the Queensland Audit Office, October 2004: 26
2 Public Accounts and Public Works Committee Evaluation of Auditor-General’s PMS Audit Mandate
10. The 2004 Strategic Reviewers stated that whilst they recognised, in principle, that a case could be made for performance auditing, they were not of a mind, in the absence of satisfactory PMS audit experience, to go once again down the path of recommending for QAO a performance audit mandate that has been rejected on at least two previous occasions. 10 They considered that they did not have sufficient data or confidence that the PMS audit mandate had been sufficiently well examined to be able to form a judgement that they should recommend that the Parliament’s wishes, expressed on two separate occasions in the past, should be overturned. In their opinion, a proper assessment and evaluation of the relative merits of PMS audits needed to be undertaken based on sufficient data to enable a well-founded conclusion to be drawn. 11
11. The reviewers provided the following conclusions and recommendations regarding the Auditor- General’s mandate 12 :
Review conclusions
C.1.2 PMS audits have made a limited but nevertheless valuable contribution to overall public sector management and performance the PMS audit mandate has not been as fully and actively exploited by the QAO as Parliament might have originally envisaged too few resources have been applied to the PMS audit task and even these limited resources have too often been diverted to other special audit tasks the PAC could and should play a greater role in the overall PMS audit process the value of the PMS audit function needs to be better evaluated, which can only occur when more PMS audits have been undertaken at the agency level PMS audits need to be better focused and completed in tighter timeframes to maximise their value performance auditing is a valuable and effective tool for targeted independent evaluations for Parliament. But it is not necessarily as valuable for overall management purposes in an environment where primary responsibility for performance is unambiguously with Ministers, Directors-General and CEOs the PMS audit mandate should remain in place pending a more detailed evaluation based on a three year plan of targeted PMS audits D.2.2 additional resources are required for the PMS audit function to be undertaken as Parliament intended PMS audit resources should not be diverted to other functions. Review recommendations
C.1.2 1. QAO, in consultation with the PAC, develop a three year plan to undertake at least 20 targeted PMS audits across the public sector, with each audit taking no more than six months 2. the PAC undertake detailed scrutiny of PMS audit reports when completed 3. the PAC evaluate the value of PMS audits at the end of this three year period and report on the effectiveness of the PMS audit mandate in terms of Parliament’s previously expressed position 4. the current PMS audit mandate remain in place pending the outcome of this process 5. in the event that the PMS audit mandate is not delivering what Parliament intended, a performance audit mandate be further considered 6. an appropriate level of resources be provided to the QAO to enable the plan to be fully implemented D.2.2 7. the resources allocated to the PMS audit function not be called upon to undertake other tasks such as special audits except in exceptional circumstances 8. additional funding be provided for PMS audits
10 Anderson, R and Smerdon, H, Report on the Strategic Review of the Queensland Audit Office, October 2004: 24 11 Anderson, R and Smerdon, H, Report on the Strategic Review of the Queensland Audit Office, October 2004: 46 12 Anderson, R and Smerdon, H, Report on the Strategic Review of the Queensland Audit Office, October 2004: 27, 52
3 Public Accounts and Public Works Committee Evaluation of Auditor-General’s PMS Audit Mandate
12. In June 2005, the PAC tabled its report examining the strategic review. The committee agreed with the reviewers’ recommendations with the exception of recommendation 1 which was subject to eliminating the suggested targets. The committee was of the view that the reviewers’ intention in specifying the number of audits and maximum timeframe for completion was to ensure that a sufficient body of work was available for the committee to assess the value of the PMS audit mandate. The committee considered that the Auditor-General should be allowed to retain flexibility. To do otherwise may have been seen as compromising the Auditor-General’s independence. 13 The government endorsed the committee’s recommendations. 14
1.4 Committee’s actions since last strategic review
13. In reviewing the Auditor-General’s reports, the committee assesses the nature of the findings and issues in terms of their impact on the integrity, economy, efficiency and effectiveness of government financial management together with the explanations and undertakings advanced by management of the audited agencies.
14. Over the past three years the Auditor-General has tabled 22 PMS audit reports (listed in Appendix 1). Copies of the Auditor-General’s reports are available on QAO’s website. The committee has examined all of these reports. It has conducted separate inquiries, hearings and tabled reports regarding the following:
Committee Report no. and date Relates to Auditor-General’ Report tabled
PAC Report No 71, tabled Report No 3 for 2005 – Results of PMS audits of Output Performance Reporting 25 May 2006 (tabled 19 September 2005);
Report No 5 for 2005 – Results of PMS audits of Output Performance Reporting Phase 2 (tabled 25 May 2006)
PAC Report No 73, tabled Report No 5 for 2006 – Results of PMS Audit of Capital Works at Departments of 19 April 2007 Corrective Services, Education, Health and Housing (tabled 1 November 2006)
PAC Report No 74, tabled Report No 6 for 2006 – Results of PMS Audit of Workforce Planning at the 19 April 2007 Department of Education, Training and the Arts and the Department of Health (tabled 13 November 2006)
PAC Report No 76, tabled Report No 2 for 2006 – Results of PMS Audit of Government Owned Corporations’ 21 August 2007 Performance Reporting (tabled 5 June 2006)
PAC Report No 79, tabled Report No 4 for 2007 – Are departmental output performance measures, relevant, 28 August 2008 appropriate and a fair representation of performance achievements? (tabled 7 August 2007)
PAC Report No 81, tabled Report No 7 for 2007 – Addressing Skills Shortages in Queensland (tabled 4 December 2008 13 November 2007)
PAPWC Report No 2, tabled Report No 2 for 2007 – Results of PMS Audits of Management of Funding to Non- 17 September 2009 Government Organisations (tabled 22 May 2007)
PAPWC Report No 3, tabled Examined all Auditor-General’s reports tabled in 2009. 11 February 2010
13 PAC, Report No.69 Review of the Report of the Strategic Review of the Queensland Audit Office, June 2005: 6 14 Queensland Government, Government Response to the PAC Report No 69 – Review of the Report of the Strategic Review of the Queensland Audit Office , September 2005: 1
4 Public Accounts and Public Works Committee Evaluation of Auditor-General’s PMS Audit Mandate
15. Copies of the committee’s reports are available on its website at: http://www.parliament.qld.gov.au/view/committees/PAPWC.asp?SubArea=reports
16. During the course of its inquiries the committee also collected evidence regarding the effectiveness of these PMS audits.
1.5 Conduct of the inquiry
17. The PAC (52 nd Parliament) resolved to conduct an inquiry into an evaluation of the effectiveness of the PMS audit mandate in terms of the Parliament’s previously expressed position.
18. The inquiry terms of reference are contained in Appendix 2. The committee wrote to a variety of interested groups and organisations advising them of the inquiry and inviting them to make a submission. The closing date for submissions was 18 August 2008.
19. In response to its terms of reference, the committee received 23 submissions. A list of those who made submissions is contained in Appendix 3. Copies of the submissions have been tabled and are available from the committee secretariat and on the committee’s website. In response to the submissions, the committee wrote to a number of submitters seeking additional information.
20. In February 2009 the 52 nd Parliament was dissolved for the State Election to be held on 21 March 2009. Subsequent to the election, the government implemented a number of machinery of government (MOG) changes which resulted in significant changes to the departmental structure. Submissions for this inquiry were received prior to the MOG changes and as such the previous departments have been quoted in this report where required.
21. The PAC and the Public Works Committee (PWC) of the 53 rd Parliament were appointed on 23 April 2009. On 19 May 2009, the Legislative Assembly passed the Parliament of Queensland Amendment Bill which merged the PAC and the PWC to form the PAPWC. This bill was assented to and commenced on 28 May 2009.
22. The PAPWC resolved to complete the inquiry. The committee held a public hearing at Parliament House in Brisbane on Friday 18 September 2009. The names of witnesses are detailed in Appendix 4. A copy of the hearing transcript is available on the committee’s website.
23. This report draws on the views presented at the hearing together with the submissions and other information sources.
1.6 2010 Strategic review of QAO
24. On 9 October 2009, the Governor approved the appointment of Mr Graham Carpenter and Mr Mark Gray to conduct the Strategic Review of the Queensland Audit Office, following consultation with the Public Accounts and Public Works Committee (PAPWC) and the Auditor- General of Queensland, in accordance with s68 of the Auditor-General Act 2009 .15
15 Queensland Government Gazette no 46, 9 October 2009: 432
5 Public Accounts and Public Works Committee Evaluation of Auditor-General’s PMS Audit Mandate
25. The terms of reference for this review included consideration of comparative models, practices and procedures used by offices in other jurisdictions equivalent to the QAO. The 2010 Strategic Review Report was tabled by the Premier on 10 March 2010. Under s70(7) of the Auditor- General Act the report, once tabled in the Parliament by the Minister, is referred to the PAPWC. 16
26. A copy of the strategic review report is available on the Parliament’s tabled papers database at: http://www.parliament.qld.gov.au/view/legislativeAssembly/tabledPapers/tabledpapers.asp
27. On 22 March 2010 the committee resolved to formally review the Report of the 2010 Strategic Review of the Queensland Audit Office to consider the recommendations made and comment on other findings where appropriate.
28. This report focuses solely on the recommendations made in respect of the PMS audit mandate. The committee will report separately on the other issues examined in the report.
2. Role of PMS audit mandate
2.1 What is a Performance Management Systems audit?
29. The Auditor-General Act 2009 17 provides the Auditor-General with the authority to conduct PMS audits. The aim of this type of audit is to determine whether an entity has systems in place to enable management to assess if the entity’s objectives are being achieved economically, efficiently and effectively. The audit may be conducted as a separate audit or as part of another audit (including an audit of another public sector entity).
30. PMS audits are:
…an independent examination of whether an entity or part of an entity’s activities have performance management systems in place to enable management to assess whether its objectives are being achieved economically, efficiently and effectively. 18
31. PMS audits have been part of the audit mandate since 1993 and are carried out in accordance with the Guidelines for the Conduct of Performance Management Systems Audit issued by the QAO.
32. PMS audits are discretionary non-chargeable audits and therefore must be provided for by appropriation. Subsequent to the 2004 Strategic Review, the Auditor-General took steps to ramp up the operation of the PMS audit section and with the support of the PAC increased the funding available to conduct PMS audits.
16 Auditor-General Act 2009 (s70(7)) 17 Auditor-General Act 2009 (s38) 18 Queensland Audit Office, Guidelines for the Conduct of Audits of Performance Management Systems , July 2002: 2
6 Public Accounts and Public Works Committee Evaluation of Auditor-General’s PMS Audit Mandate
33. A key outcome of PMS auditing is the identification of improvements to financial and non- financial systems to enhance public sector accountability and performance. It is the responsibility of management to implement and maintain the appropriate systems. 19 PMS audits are intended to add value to the quality of public administration and help entities discharge their governance obligations. 20
34. The Auditor-General confirmed that the PMS audit mandate was introduced to enable him to review and report to the Parliament on how well management is monitoring and reporting on the entity’s performance. The Auditor-General reviews the extent to which management systems are in place and are working properly. In this way, entities retain responsibility for their performance management while being accountable by Parliament through regular external reporting processes as well as periodic scrutiny by external audit in relation to their performance systems. 21
35. Queensland Treasury advised the committee that the role of the PMS audit mandate is to provide Parliament with independent assurance that agencies have both financial and non-financial systems in place to record data necessary to report on agency performance and that the measures agencies have chosen to assess achievement of their stated objectives are relevant to the purpose. 22
36. The Public Service Commission advised that from a systemic perspective the PMS audit mandate focuses on establishing the veracity and dependability of performance management systems and examines the tools and methods used to measure performance and assesses agencies’ ability to inform performance analysis and decision making. 23
37. In October 2006, the Parliament passed legislation extending the Auditor-General’s mandate to undertake an assessment of the relevance of the published measures used by government agencies. The Deputy Premier, Treasurer and Minister for Infrastructure noted, in her second reading speech, that the extended mandate will ensure that appropriate independent scrutiny of the performance of government agencies contributes to improved service delivery to the community and will complement the operations of the Service Delivery and Performance Commission, established to assess how well government services are meeting community needs. 24
38. The extension to the Auditor-General mandate came into effect on 1 January 2007 and may include:
a review of the public sector entity’s performance measures; and
may state whether, in the Auditor-General’s opinion, the performance measures –
(i) are relevant and otherwise appropriate, having regard to their purpose; and
(ii) fairly represent the public sector entity’s performance. 25
19 Queensland Audit Office, Guidelines for the Conduct of Audits of Performance Management Systems , July 2002: i 20 Queensland Audit Office, Guidelines for the Conduct of Audits of Performance Management Systems , July 2002: 2 21 Submission 13: 1 22 Submission 15: 1 23 Submission 16: 2 24 Queensland Legislative Assembly, Hon AM Bligh, Deputy Premier, Treasurer and Minister for Infrastructure, Second Reading Speech, Parliamentary Debates (Hansard) , 11 October 2006: 64 25 Auditor-General Act 2009 , s38(7)
7 Public Accounts and Public Works Committee Evaluation of Auditor-General’s PMS Audit Mandate
39. The Auditor-General advised that this change allowed him to form an opinion on whether the performance measures are relevant and otherwise appropriate, having regard to their purpose and whether they fairly represent the public sector entity’s performance. 26
40. The 2010 Strategic Reviewers concluded that, at date of their report, the Auditor-General had not directly reported on any individual public sector entity’s performance measures. 27
41. The Auditor-General advised the committee that QAO has found the extension of the mandate to be a very useful part of their audit program. They have used it in a number ways, specifically looking at external performance reporting, because in some earlier work they found that all they were able to do was to comment on the systems without saying if the performance indicator is relevant or appropriate. The combination of looking at the systems and also looking at the measures is quite powerful. QAO has also used it as an add-on for virtually all the audits where part of the thinking process and the audit process is to look at the performance measures and to see whether they think they are relevant and appropriate. 28
42. Queensland Treasury advised that the change to the audit mandate to look at the relevance of performance measures addressed an important weakness in the previous mandate as it existed. It has allowed the audit office to more properly examine areas and relevance of performance measures. 29
43. The committee noted that there are no Australian Auditing and Assurance Standards issued that cover PMS audits. In Australia, Queensland and the Northern Territory are the only jurisdictions that have preferred the PMS audit model. Other jurisdictions have elected to adopt the performance audit model. Appendix 5 contains a comparison of the mandates of other Auditors- General in Australian jurisdictions.
2.2 Does the PMS audit mandate fulfil its function in the overall accountability process for the Parliament?
44. The Auditor-General advised the committee that for the Parliament to be able to hold executive government and public sector entities accountable it requires quality information on performance. Under the current system of PMS auditing, the Parliament relies on entities themselves to report on their performance. Parliament then relies on the Auditor-General to provide independent advice on whether appropriate systems are in place to enable the entities to report on their performance. 30
26 Submission 13: 2 27 Carpenter, G and Gray, M, Report of the 2010 Strategic Review of the Queensland Audit Office, March 2010: 47 28 Mr Poole, Transcript 18 September 2009: 10 29 Mr Bradley, Transcript 18 September 2009: 10 30 Submission 13: 2
8 Public Accounts and Public Works Committee Evaluation of Auditor-General’s PMS Audit Mandate
45. The Auditor-General noted that over the last five years, QAO has completed a significant body of PMS audit activity which has covered a range of agencies and a range issues including the systems used by agencies to report their performance achievements externally, and planning systems for capital works, workforce planning and biosecurity and rural fire incidents. He advised that this extensive audit program has fully explored the boundaries of the current audit mandate and has generated many useful recommendations for improvement across the public sector. 31
46. The Auditor-General advised that the PMS audit function has generated many recommendations and has led to improved management practices and controls in public sector entities. 32
47. Queensland Treasury confirmed that the PMS audits have covered quite an extensive range of issues across government and that each audit is usually different from others and brings forward different recommendations and learnings for agencies. They also confirmed that there have been some important recommendations that they have accepted in terms of improving the framework within which agencies provide performance information and have included the role of central agencies.33
48. Queensland Treasury considers that the audit office has probably been quite effective in using the mandate as it currently exists and it has been explored reasonably well. They consider that while it has had to try and put it back within the context of the mandate, QAO has nevertheless been able to make strong recommendations on improvements across a range of areas. 34
49. They also advised that the outcome of recent audits indicates that the PMS audit mandate is effective in achieving its objective as QAO is providing a valuable role in preventing complacency with regard to performance management. 35
50. The 2010 Strategic Reviewers concluded that PMS audits undertaken by QAO are acceptable in terms of the quality of their content, but that QAO should continually strive to improve that quality. They considered that PMS audits provide valuable information to Parliament on the quality of performance management systems and they are an important tool for Accountable Officers and CEOs to guide improvements in public administration. 36
51. The committee sought input from agencies, subject to PMS audits over the review period, regarding whether they benefited from being subject to the PMS audit process.
52. The Department of Emergency Services (DES) advised that they use the audit reports as an opportunity to benchmark their activities against other agencies and enhance performance management internally. They did advise, however, that they believe that on the whole, the public sector needs to be more effective in how it implements audit findings to enhance overall accountability and achieve the audit mandate’s function. 37
31 Mr Poole, Transcript 18 September 2009: 2 32 Submission 13: 2 33 Mr Bradley, Transcript 18 September 2009: 4-5 34 Mr Bradley, Transcript 18 September 2009: 11 35 Submission 15: 1 36 Carpenter, G and Gray, M, Report of the 2010 Strategic Review of the Queensland Audit Office, March 2010: 51 37 Submission 7: 1
9 Public Accounts and Public Works Committee Evaluation of Auditor-General’s PMS Audit Mandate
53. The Department of Employment and Industrial Relations (DEIR) advised that undergoing the PMS audit process provided the department with an opportunity to review its performance management framework and systems and implement best practice improvements. 38
54. The Department of Infrastructure and Planning (DIP) advised that it is their view that QAO, operating under the current mandate, has been very successful in efficiently conducting PMS audits and identifying for agencies, including central agencies, specific management frameworks necessary for improvement. 39
55. Queensland Health advised the committee that any system that allows external review of an agency’s efficiency and leads to improvement is welcomed by Queensland Health. They advised that they have been through several audits through the PMS audit process and found that it has brought an opportunity to improve systems and led to changes. 40
56. The Department of Transport and Main Roads (DTMR) confirmed that they appreciate and value the opportunity to learn that the PMS audit process present. 41 They advised that one of the key things they got out of the process was having a third party come in and test the premise of their policy positions and implementation plans. 42
2.3 What improvements has the mandate brought to the accountability systems?
57. The Auditor-General advised that in his reports to Parliament he comments on whether public sector entities have appropriate performance management systems to record and use the information they need to manage well. These reports include recommendations for improvement. He may later perform a follow up audit and report on the progress accomplished by the entity since the original audit was conducted. 43
58. In the Auditor-General’s opinion, by independently reporting, providing these recommendations and following up on their implementation, PMS audits have strengthened accountability systems within entities. The Auditor-General advised that follow-up audits have generally shown a genuine attempt by entities to implement recommendations. 44
59. The Auditor-General also noted that his reports to Parliament provide examples of better practice, rather than just reporting on poor performance management systems or measures. He advised that this establishes credibility of the reports across the sector, encouraging more information sharing and providing practical guidance. 45
38 Submission 14: 1 39 Mrs Schaefer, Transcript 18 September 2009: 2 40 Mr Mehan, Transcript 18 September 2009: 3 41 Mr Stewart, Transcript 18 September 2009: 3 42 Mr Stewart, Transcript 18 September 2009: 5 43 Submission 13: 2 44 Submission 13: 3 45 Submission 13: 6
10 Public Accounts and Public Works Committee Evaluation of Auditor-General’s PMS Audit Mandate
60. He advised the committee that he believes that the audits that QAO have undertaken have provided a good insight into the range of performance management systems that agencies have, both within and across agencies. 46
61. The Auditor-General’s opinion is supported by the responses from the agencies reviewed.
62. DES advised the committee that the audits they have been involved in have provided the department with information to support improvements in its accountability systems. PMS audits identify opportunities for improvement to be factored into the department’s forward policy program and complement business improvement, risk management and internal audit findings. The department has also used the audit reports as resources to support an extensive review of the department’s performance measures and subsequent amendments to those measures. 47
63. The Department of Public Works (DPW) advised that the capacity of QAO to undertake PMS audits has seen a positive addition to the State’s accountability regime. It has enabled QAO to examine a number of important issues which could not have been covered by the more traditional financial and compliance audits. 48
64. Queensland Treasury advised that they support the work undertaken to date by the QAO and believe that the focus of external audit in reporting results of PMS audits to Parliament is placing greater emphasis by agencies on this important aspect of management responsibilities. 49
65. The Department of Tourism, Regional Development and Industry (DTRDI) advised that they have been involved in one PMS audit and experienced some improvements as a result of the implementations of the recommendations made. 50
66. The Auditor-General provided an example of how individual PMS audits have improved the accountability systems. He advised the committee that in 2007-08 only 9% of departmental outputs, reported as part of the budget papers, had a clear and measurable objective. By the 2009- 10 budget papers, that had increased to 51% of the departmental objectives having a clear and measurable output objective attached. He considered that this significant improvement had come through the work that QAO had done. 51
67. Queensland Health expressed the view that PMS audits are audits where both parties have the intent to improve performance and improve the outcome for the public. It is the attitude and conversations that allow agencies to work through the issues to get the best result. The attitude is one of cooperation and collaboration and the outcomes reflect that. 52
46 Mr Poole, Transcript 18 September 2009: 3 47 Submission 7: 2 48 Submission 18: 1 49 Submission 15: 3 50 Submission 1: 2 51 Mr Poole, Transcript 18 September 2009: 3 52 Mr Mehan, Transcript 18 September 2009: 6
11 Public Accounts and Public Works Committee Evaluation of Auditor-General’s PMS Audit Mandate
68. As an example of the benefits that have flowed to their organisation, they advised that complicated and complex agencies tend to have complicated and complex ways of looking at their own performance. When an external agency comes in with specific questions like ‘how do you do planning?’ or ‘how do you plan patient flow?’ it forces agencies to try to answer that question in a simple way. It forces the agency to look at its own systems to ensure that they are able to communicate to public more clearly about its measures. They found that QAO assisted in this process by bringing constructive dialogue. 53
69. Queensland Health advised that what they found most beneficial was the third party fresh eyes perspective, particularly when that perspective is to represent accountability back to the public. 54
2.4 Disadvantages of the PMS audit mandate
70. The Auditor-General advised that QAO is constrained by its current mandate when it comes to reporting. He advised that the audit activity for both performance audits and PMS audits is the same. But when it comes to reporting the PMS audit mandate forces them to tie back to what was found in the systems and focus on the systems. This can lead to a deficiency because QAO is not forced to draw a conclusion about whether any deficiencies in the systems have resulted in poor performance. Performance can be ignored, irrespective of whether that performance is good, bad or indifferent. 55
71. He advised that a feature of many of the critical performance failures for entities both in the private and public sectors in recent years has not been the absence of good systems or controls but usually where management has ignored or avoided the proper application of the control systems. The Auditor-General stated that it is difficult under the current mandate to report on the manner in which an agency is using its systems to effectively achieve its objectives. 56
72. Under the current mandate, QAO would examine whether an agency has appropriate frameworks and systems for measuring their performance. If QAO finds that the systems are there but are not being used or where there is some breakdown in those systems, under the PMS mandate there is some difficulty as to how that is reported. He advised that if they were undertaking a financial and compliance audit and a systems breakdown results in some deficiency within the preparation of the financial statements they are able to talk about that as that goes to the heart of the audit opinion. But in areas of non-financial performance, a breakdown in those systems, under the current mandate, they have great difficulty drawing some conclusions. 57
53 Mr Mehan, Transcript 18 September 2009: 6 54 Mr Mehan, Transcript 18 September 2009: 6 55 Mr Poole, Transcript 18 September 2009: 10 56 Correspondence from Mr G Poole, Auditor-General of Queensland to the Public Accounts Committee dated 6 February 2009: 8 57 Mr Poole, Transcript 18 September 2009: 13
12 Public Accounts and Public Works Committee Evaluation of Auditor-General’s PMS Audit Mandate
73. The Auditor-General highlighted that he can report that an agency has a beautiful set of systems that produce a beautiful set of numbers, but whether they are using that and whether that is impacting on the performance of the agency is another matter. He advised that under the expanded mandate he can also comment on whether the performance measures are relevant but he cannot talk about what is the end performance result for the agency. He advised that if there is a breakdown somewhere, he then has to use some creativity when he is writing the report as to how he can highlight this. He advised that this sometimes turns out to be very subtle and most people miss it. 58
74. Queensland Treasury commented that they usually have very good dialogue with the audit office when reviews are done. The Auditor-General is usually able to point out the subtleties in the reports to them where they have particular concerns or issues. 59
Committee comments
75. In 2004 the Strategic Reviewers found that QAO had not pursued PMS audit mandate with the vigour they expected and recommended that a greater body of work was required to allow for a full evaluation of the mandate. The committee considers that the Auditor-General and QAO have thoroughly explored the PMS audit mandate over the past five years.
76. In reflecting upon the PMS audits undertaken over the past five years, in comparison to prior PMS audits, it is clear that there was probably a lack of understanding of what could be achieved by a more extensive exploration of the mandate.
77. The committee understands the notion behind the establishment of the PMS audit mandate in 1992 was to place the emphasis of performance reporting on agencies. The government at the time considered that if agencies were to be held accountable for their performance, then they must have the freedom to undertake their own internal reviews.
78. The committee has found that despite the effort agencies put into internal performance reviews, the independent perspective provided by QAO has proven to afford enormous benefits to agencies. The committee considers that the extension to the Auditor-General’s mandate to include comment on entities performance measures has added substantially to this.
79. The committee, however, is concerned that gaps remain in the accountability system and that the Auditor-General must resort to means outside his reports to Parliament in order to identify shortfalls in performance. Whilst the committee is encouraged that central agencies are committed to engaging with the Auditor-General in identifying particular concerns or issues, the Auditor-General’s primary client is the Parliament. The Auditor-General should not be restricted in the information he can provide to the Parliament.
58 Mr Poole, Transcript 18 September 2009: 13 59 Mr Bradley, Transcript 18 September 2009: 13
13 Public Accounts and Public Works Committee Evaluation of Auditor-General’s PMS Audit Mandate
80. The committee considers that this gap in accountability can work to both the benefit and the detriment of agencies. Agencies that have poor systems in place may in fact have excellent performance, however, this is not reported to Parliament.
3. Is the concept of PMS auditing understood by both auditor and auditee?
81. The Auditor-General advised that conduct of PMS audits has created a reasonable understanding of the concept across the Queensland public sector, particularly within those entities which have been subject to audit over recent years. 60
82. He advised that upon commencing a PMS audit, an explanation is provided to the entity’s management on the nature of a PMS audit, the process to be followed and how the information will be reported. Every report includes a section describing what a PMS audit is and the audit approach taken by QAO. 61
83. The Auditor-General also advised that to develop a wider understanding of the concept, QAO has conducted educational activities including presentations at appropriate forums and client information sessions. 62
84. The Auditor-General comments were supported by some agencies’ in their submissions.
85. Queensland Transport advised the committee that auditors ensure that the auditee understands the objectives including determining whether the performance management systems enable the department to assess whether its objectives are being achieved economically, efficiently and effectively. 63 QAO also has put in place a number of mechanisms, such as regular information sessions about audits conducted, and publishing the reports and other information on its website to promote understanding of the PMS audit mandate. 64
86. The Department of Local Government, Sport and Recreation (DLGSR) advised the committee that evidence that PMS auditing is understood is reflected in the quality of the recommendations made by QAO and the extent to which departments have accepted and actioned those recommendations. 65
87. DEIR commented that QAO practice guides and previously published audit reports provided the department with a detailed understanding of the purpose of the audit, what would be examined and the standards expected. 66
60 Submission 13: 3 61 Submission 13: 4 62 Submission 13: 4 63 Submission 1: 2 64 Submission 5: 2 65 Submission 2: 3 66 Submission 14: 2
14 Public Accounts and Public Works Committee Evaluation of Auditor-General’s PMS Audit Mandate
88. DPW advised the committee that the concept of PMS auditing is well documented in QAO publications including the Guidelines for the Conduct of Audits of Performance Management Systems. Each PMS audit report also clearly outlines the PMS audit approach, including the legislative basis for the audit, audit objective, audit scope and coverage and audit methodology. 67
89. The Queensland Police Service (QPS) advised that their experience was that the approach taken by QAO was constructive and collaborative, involving a comprehensive briefing at the commencement and finalisation of the audit and regular progress reports throughout the audits duration. 68
90. DIP advised that based on the scope, content and the manner in which the PMS audits by QAO have been conducted in recent years, it is evident that agencies have increased their understanding of the concept of PMS audits and the reasoning behind undertaking them. Consequently, the outcomes of these audits have proven very successful in raising the overall level of compliance with public sector performance management best practices and the quality of systems used. 69
91. DIP further advised that more recently the recommendations from PMS audits have resulted in central agencies undertaking fresh reviews of the risk management and managing for outcomes frameworks. Future improvements to these will further raise the level of performance management across government and provide greater assurance that government priorities will be achieved efficiently and effectively through the delivery of programs, products and services by agencies. 70
92. Queensland Treasury advised that they also offer a range of information sessions which are provided through their financial management branch.71
93. However, the 2010 Strategic reviewers found, during their interviews with various stakeholders, that there is a need for a greater awareness and understanding of the role and purpose for PMS audits amongst audit clients. They recommended that opportunities be taken by the Auditor- General and senior staff of the QAO to educate audit clients on the mandate for, and benefits arising from, PMS audits. 72
94. The Queensland Water Commission advised that they consider that the effectiveness of the PMS audit mandate may be improved by giving the process a greater profile within organisations. The effectiveness of the PMS audit mandate maybe enhanced by more widely communicating how the PMS audit information is to be used and who will use it. Although a QAO brochure on PMS audits was provided to the Commission, if the process and outcomes were communicated more widely, it may assist in developing a proactive organisational culture, rather than reactive responses. Further they advised that, while the QAO criterion for the PMS assessment provides a framework for organisational PMS, these are not widely known. 73
67 Submission 18: 1 68 Submission 17: 1 69 Submission 21: 3 70 Submission 21: 3 71 Mr Bradley, Transcript 18 September 2009: 4 72 Carpenter, G and Gray, M, Report of the 2010 Strategic Review of the Queensland Audit Office, March 2010: 51-52 73 Submission 20: 1
15 Public Accounts and Public Works Committee Evaluation of Auditor-General’s PMS Audit Mandate
95. A number of agencies commented on some of the misconceptions surrounding the scope of the PMS audit mandate.
96. Queensland Transport advised that there is still some misconception about the scope of the PMS audit mandate. They advised that there is a view that the mandate is the same as the ANAO performance audit mandate, which has wider scope and provides for review or examination or any aspect of the operations of an agency. 74
97. The Department of Mines and Energy noted that whilst the majority of auditors and auditees understand the purpose of PMS audits, the extent to which the Auditor-General may delve, report and comment on entities’ performance management systems remains contentious. 75
98. DES advised that while the concept of PMS auditing is gaining greater recognition for its role in organisational performance and improvement, there are still opportunities to enhance understanding of how these audits fit within the accountability framework for the public sector. 76
99. The committee found, during the course of this inquiry, however, that a number of agencies mistakenly confused the concept of PMS auditing with the PMS audits the Auditor-General had undertaken in the area of output performance measurement and reporting.
100. The Auditor-General advised the committee that the series of PMS audits undertaken initially by QAO were focused around the external reporting by departments. He advised that these were selected because the system of financial management requires that Accountable Officers are accountable for their performance. He felt that it was an appropriate that QAO test the systems that agencies’ were using to report that performance. This series really focused on the performance reporting. 77 He advised that part of the confusion identified by the committee may have arisen because of this. He told the committee that he would not be surprised to believe that agencies therefore equated the PMS audit mandate with output performance. 78
101. He advised the committee, however, that the bulk of the audits since the initial work in 2005 and 2006 had been broader and he considered that agencies are now more focused on and he would expect agencies to understand the mandate better now. 79
102. The committee asked the Auditor-General whether he had ever been challenged by departments about their authority in respect of the PMS audit mandate. The Auditor-General advised that there has not been any direct challenge to the mandate. However, he did advise that there have been questions raised about audit coverage where they were able to assure the agencies involved that the direction the audit was heading was totally within the mandate. 80 The Auditor-General advised that QAO is very deliberate in the way it scopes the audits to make sure that they are focusing on the systems and that it falls within the mandate. 81
74 Submission 5: 2 75 Submission 11: 1 76 Submission 7: 2 77 Mr Poole, Transcript 18 September 2009: 3 78 Mr Poole, Transcript 18 September 2009: 4 79 Mr Poole, Transcript 18 September 2009: 4 80 Mr Poole, Transcript 18 September 2009: 7 81 Mr Poole, Transcript 18 September 2009: 8
16 Public Accounts and Public Works Committee Evaluation of Auditor-General’s PMS Audit Mandate
Committee comments
103. The committee considers that the Auditor-General and QAO have taken active steps to try to ensure that agencies understand the PMS auditing concept and what is to be expected during the audit process. The committee considers that the Auditor-General should continue in his efforts in this regard.
104. The committee was initially concerned about the confusion between the general PMS audit mandate and the output performance reporting audits undertaken under the auspices of the PMS audit mandate identified in the early submissions to this inquiry. However, it is now satisfied that this confusion has been mitigated by the expansion of the topics considered by the Auditor- General in later audits.
Recommendation 1
The committee recommends that the Department of Premier and Cabinet, in its role as central agency, ensure that the QAO is supported in its efforts to ensure that agencies are made aware of both extent of the Auditor-General’s mandate, and what is expected of agencies in respect of their compliance requirements under that mandate.
4. Is the PMS audit mandate being used effectively by QAO?
105. The committee considered the issue of the effective use of the PMS audit mandate by QAO to some extent in section 2 of this report. However, agencies raised a number of other issues which the committee considered deserving of further consideration. These issues included:
Timeliness of completion of audits;
Reporting of issues identified in audits;
Scoping of audits
Audit time frames
106. One of the issues raised by Queensland Health was a perceived lack of timeliness in completion of audits compared with the initial times advised by QAO. They advised that QAO provided different times at the opening interview. 82
107. The Auditor-General advised that QAO has focused on ensuring the audit scope allows for timely reporting to Parliament so that the findings and recommendations retain their relevance. 83
82 Submission 4: 1 83 Submission 13: 6
17 Public Accounts and Public Works Committee Evaluation of Auditor-General’s PMS Audit Mandate
108. DES advised that the benefit of PMS audits is that they can go beyond compliance auditing into the realm of “are we delivering appropriate services” rather than “are our services delivered in line with regulation/policy?”. They advised that the recent refocus of the government on core service delivery and the economical and effective achievement of organisational objectives, increases the value of PMS audits particularly in the service delivery review area. They stated that, subject to QAO’s ability to maintain PMS audits at the strategic level, PMS audits will enhance government’s ability to ensure departments are planning and delivering services to meet future needs. 84
109. The Auditor-General advised that strategic planning activities are used to maximise the benefits of the PMS audit mandate through identifying and prioritising important issues, trends and emerging risks relevant to public sector activities. 85
110. Queensland Transport advised that in some instances, the results of some audits have included simple technical compliance issues and while not detailed in the published report, they felt that these have detracted from the overall quality of the audit outcome. They considered that simple compliance issues identified during PMS audits should be reported in the context of, and consistent with, the audit objective. 86
111. DIP advised that QAO operating under the current mandate has been very successful in efficiently conducting PMS audits and identifying for agencies necessary areas for improvement, including improvements to performance management frameworks. 87 However, they consider that it may be more appropriate for the mandate to overtly state the ability of QAO to investigate any public entity either individually or as part of a group or program audit, across government. 88 DIP advised that they use a range of measures, in addition to the audit office’s audits, to check efficiency and effectiveness of departmental programs. They suggested that in some cases it may be better for QAO to rely more on those other processes rather than just trying to cover and spread themselves across some technical areas. 89
112. The Auditor-General explained that when QAO scope out an audit and undertake an audit they do review what other review activity has occurred in a particular area. However, QAO still needs to observe what is going on and, to the extent that they are able to rely on it and to say that it is working well, they would do so. QAO needs to assess in their opinion the validity and depth and quality of work and report on that accordingly. 90
113. The 2004 Strategic Reviewers considered that the PMS audit mandate enables wider coverage of activities in a shorter space of time than that covered by traditional performance audits. 91 The committee asked the Auditor-General to comment on this. He advised that in a number of jurisdictions some performance audits take a considerable amount of time and that the 2004 Strategic Reviewers recommended that QAO try to work within a six month time frame. He confirmed that generally over the review period QAO has managed to do that.
84 Submission 7: 2 85 Submission 13: 6 86 Submission 5: 2 87 Submission 21: 1 88 Submission 21: 3 89 Mrs Schaefer, Transcript 18 September 2009: 9 90 Mr Poole, Transcript 18 September 2009: 9 91 Anderson, R and Smerdon, H, Report on the Strategic Review of the Queensland Audit Office, October 2004: 26
18 Public Accounts and Public Works Committee Evaluation of Auditor-General’s PMS Audit Mandate
114. The average time frame from the start of a PMS audit to tabling of the report reduced from 6.3 months in 2006-07 to 5.9 months in 2007-08. 92 In 2008-09 the time elapsed was 6.8 months meeting QAO’s target of less then 7 months. 93 This period includes the 21 days provided to agencies to respond to the report.
115. The Auditor-General advised that he considers that the 6 to 7 month time frame is tight but is achievable. He considers that a period of less than this would have to have a very narrowly targeted audit scope which he did not consider would be useful. 94
Committee comments
116. The committee considers that the Auditor-General should be allowed flexibility in the time frame in which audits are completed. That said, it also encourages the Auditor-General to consider the timeliness of reporting when scoping audits to ensure that the information remains relevant to Parliament.
117. The committee also encourages the QAO to be conscious of advising of accurate timeframes in its initial discussions with agency management and of ensuring agencies are advised in a timely manner should it be necessary to alter these timeframes once an audit has commenced.
118. It also encourages the Auditor-General to ensure that agencies understand the nature and purpose of the scope and objectives of the audit.
5. How does QAO assess if individual PMS audits have been effective?
119. The Auditor-General advised that QAO assesses the effectiveness of individual audits by examining feed back received from clients on the audit process and the consequential actions by entities to implement audit recommendations and actions taken by other external stakeholders such as the committee. 95 After the initial audit, QAO writes to the agency checking on the progress of the implementation of the audit recommendations. 96
120. QAO regularly survey clients to ensure QAO is aware of any issues or opportunities for improvement. QAO has found that the vast majority of client responses either agreed or strongly agreed with every aspect of the survey relating to the audit process. 97
92 Submission 13: 6 93 Queensland Audit Office, Annual Report 2008-09 , October 2009: 41 94 Mr Poole, Transcript 18 September 2009: 12 95 Submission 13: 7 96 Submission 1: 3 97 Queensland Audit Office, Annual Report 2008-09 , October 2009: 42
19 Public Accounts and Public Works Committee Evaluation of Auditor-General’s PMS Audit Mandate
121. Each year a sample of PMS audit files are selected for internal review. Live reviews are performed prior to the audits being finalised. Audit files are assessed against the ACAG Governance Audit Standards Checklist for Review of Individual Performance Audit Engagements and Standard on Assurance Engagements ASAE 3500 Performance Engagements . Results of the three PMS audits subject to review in 2008-09 concluded that files demonstrated a generally acceptable level of quality. 98
122. QAO seeks comments from the auditee about audit findings, in addition to agreement with the audit recommendations. They also conduct follow up reviews on some audits to assess what action has been taken to implement improvements identified during the audit. 99
123. The Department of Education, Training and the Arts (DETA) advised that they have been involved in the ORIMA survey, QAO’s internal evaluation conducted within two weeks of the audit and follow-up monitoring 18 months to 2 years after the initial audit. 100
124. ORIMA Research Pty Ltd is a company specialising in performing market, social and organisational performance research. The company has developed a client survey benchmarking process on behalf of Australian state audit offices. As part of this process ORIMA Research conducts annual surveys of audit clients on behalf of audit offices, including QAO. 101
125. DES advised that they have also been involved in some of the activities QAO uses to assess if the audits have been effective including an external evaluation process and follow-up audits. 102
126. DEIR advised that following the audit of their agency, they and QAO agreed an action plan to address the recommendations. 103 They advised that QAO audit staff provided an effective consulting service, usually in the form of exchange of ideas on how well proposed objectives, measures, measurement systems and new policy met QAO’s better practice by suggesting possible improvements to processes. 104
127. Queensland Treasury advised that they would consider a PMS audit effective if it raised an issue for an agency that the agency had not already been aware of, that the agency conceded was a significant issue and the agency agreed to address. They also advised that the PMS audit mandate is effective if it assists agencies to improve reliability and relevance of performance reporting and assists Parliament in better assessing the performance of public sector agencies. 105
128. PSC advised that their experiences of the former Service Delivery and Performance Commission (SDPC) point to the need for a more robust monitoring and reporting process for major decisions made by Cabinet and the government. 106
98 Queensland Audit Office, Annual Report 2008-09 , October 2009: 42 99 Submission 5: 2 100 Submission 8: 2 101 http://www.orima.com/index.html [14 January 2010] 102 Submission 7: 3 103 Submission 14: 2 104 Submission 14: 3 105 Submission 15: 3 106 Submission 16: 5
20 Public Accounts and Public Works Committee Evaluation of Auditor-General’s PMS Audit Mandate
6. How could the effectiveness of the PMS audit mandate be improved?
129. The committee asked the Auditor-General where the existing mandate could be improved. He advised that he considers that the existing mandate is working quite well and that QAO has taken it to the boundaries of the mandate. He advised that he did not think there is anything extra that they can do. 107
130. Queensland Treasury advised that, from their observations of recent audits, the audit office has not been particularly constrained in what it looks at in terms of its reviews and they have addressed pretty comprehensively issues raised through their audit process. 108
131. The issue of the Auditor-General using subject matter experts in audits of professional, technical or specialist fields was raised by a number of agencies. The committee was advised that current audit office staff have been recruited from a variety of places and have built up expertise in some areas, but they cannot have the expertise to review all of the processes within government. It was suggested that these experts could be sourced from third party service providers, such as large accountancy firms, from within the agency under review or from an equivalent agency in other jurisdictions. 109
132. EPA raised the issue of the quality of the auditor undertaking PMS audits. They were concerned over the ability of an auditor to have the appropriate skills and abilities to understand the diverse and sometimes technical nature of the agency-specific information being examined in an audit. 110 They considered that the expertise that exists within the EPA, including in Internal Audit, would be sufficient to undertake performance audits. 111
133. The Department of Mines and Energy also raised a similar issue. They noted their concern about whether auditors have the resources available to identify with the system subject to audit review, as some systems and business processes will involve engineering and scientific concepts with which the auditor may not be familiar. 112
134. DTMR suggested that one of the things that QAO should do in the future is utilise independent experts from around various organisations, whether they are internal to government or external from the private sector. 113
107 Mr Poole, Transcript 18 September 2009: 10 108 Mr Bradley, Transcript 18 September 2009: 10 109 Submission 23: 1 110 Submission 10: 1 111 Submission 10: 2 112 Submission 11: 1 113 Mr Stewart, Transcript 18 September 2009: 5
21 Public Accounts and Public Works Committee Evaluation of Auditor-General’s PMS Audit Mandate
135. The Auditor-General advised that as a result of the QAO Strategic Review in 2004 PMS audit staff numbers have increased from 9 to 15. These additional staff have been recruited from a variety of disciplines. As a result PMS staff have a broad range of skills, qualifications and experience. The Auditor-General did note, however, that recruitment was impacted by the workforce skills shortage over the last 3 years. 114
136. In the course of their review, the 2010 Strategic Reviewers heard from stakeholders that there was a need for the involvement of external independent subject matter experts during the entire audit process, rather than being at selective points. They found that in interstate audit offices experts tend to have greater involvement over the entire course of their performance audits. In Queensland, the approach has been to involve a subject expert to undertake a review of the audit report whilst at an advanced stage, such as at the final draft stage. 115
137. They found that one of the major challenges for QAO is to ensure that the team of internal staff together with external experts/advisors have the necessary depth of knowledge and understanding of the service area for a PMS audit. The found that there is a need to give further attention to the way in which independent objective subject experts are to involved with PMS audits, and especially the stage at which they are to be involved. Also, there is a need to ensure that such experts are genuinely independent and objective, and do not have particular biases in relation to the subject matter. 116 They recommended that the Auditor-General give consideration to wider involvement of subject matter experts in the planning, audit field work and reporting for PMS audits. 117
138. Agencies also highlighted a number of other areas where they believe the Auditor-General’s mandate can be improved.
139. DTMR advised that the current mandate focuses on the system as opposed to the actual performance and they feel that this needs to be addressed. They consider that going forward the focus needs to be on how agencies are performing in delivering on an outcome. 118
140. DTRDI advised the committee that they were of the view that the PMS audit mandate would be more effective it there was also an examination of specific costs and benefits to agencies through the implementation of the best practice performance management systems. 119
114 Submission 13: 5-6 115 Carpenter, G and Gray, M, Report of the 2010 Strategic Review of the Queensland Audit Office, March 2010: 50 116 Carpenter, G and Gray, M, Report of the 2010 Strategic Review of the Queensland Audit Office, March 2010: 51 117 Carpenter, G and Gray, M, Report of the 2010 Strategic Review of the Queensland Audit Office, March 2010: 52 118 Mr Stewart, Transcript 18 September 2009: 10 119 Submission 1: 3
22 Public Accounts and Public Works Committee Evaluation of Auditor-General’s PMS Audit Mandate
141. The Department of Primary Industries and Fisheries (DPIF) suggested that the scope of PMS audits could be broadened to include an assessment of the effectiveness of an agency’s performance management framework and assessing whether the agency’s framework supports:
the ability to demonstrate improvements to industry and community outcomes;
greater public access to data, to support performance evaluation; and
fostering innovation and learning through improvements to public policy. 120
142. DES advised that whilst PMS audits focus on aspects of economy, efficiency and effectiveness, service delivery in government includes a community service obligation. They consider that explicit awareness of how this community obligation reflects in PMS auditing would be beneficial. 121
143. DES also advised that the effectiveness of the audits could be improved through more effective coordination of the audit process and tightened protocols in terms of how the audit is engaged. 122
144. DIP advised that agencies would benefit by understanding more about the consideration given to the determination of annual PMS audit programs by the QAO. They suggested that the current mandate might be amended to specify the number and categories of PMS audits to be undertaken each year. In identification of specific categories, consideration could be given to including ‘project evaluation’ audits within the mandate. 123
145. DIP advised that a minor point of concern that they have observed is that, unlike the SDPC which had published the frameworks and assessment criteria rationale for the reviews and the forward work program, there has been less transparency in the method QAO uses to select and execute its PMS audit program. 124
146. DIP advised that they are responsible for Special Purpose Vehicles (SPVs) which are delivering certain key projects within the SEQ Infrastructure Plan and Program. QAO has conducted and charged for ‘project evaluation’ audits. In the department’s view these audits were very similar to PMS audits which are undertaken without charge to the public sector entity. Of concern to the department is the cost impact which results from a determination that both types of audit are conducted under separate sections of the Act. DIP consider that the current mandate for PMS audits should be amended to address this discrepancy by specifically including ‘project evaluation’ audits. 125
120 Submission 6: 1 121 Submission 7: 2 122 Submission 7: 3 123 Submission 21: 1 124 Submission 21: 3 125 Submission 21: 4
23 Public Accounts and Public Works Committee Evaluation of Auditor-General’s PMS Audit Mandate
147. The Auditor-General advised that his auditing mandate is quite broad, stretching from audits of financial statements through to the PMS audit mandate. In the middle is the area of compliance. He has taken the view that a large part of the compliance activities, such as is being suggested by DIP, does fit within what QAO has determined is the financial and compliance audit mandate because financial statement audits merge into a compliance activity. There is a continuum of audit activity and it is a matter of judgement as to what fits into which box. He understands the funding of the audit activity is different and that is why he is fairly careful to make sure of the demarcation, as far as he is able. The PMS audits look at systems. The other activity is around do they comply with the requirements of legislation and do they comply with the accepted norms of financial management. That is the demarcation that QAO applies. 126
148. DIP have also suggested that QAO’s mandate should also include responsibility for auditing and updating whole-of-government framework guidelines and practices. This has been suggested on the basis that QAO, after initially identifying problems with agencies’ meeting framework compliance standards as set by central agencies, they have taken the step of publishing more comprehensive guidelines, including practical steps to ensure recommendations could be implemented effectively, in an endeavour to increase understanding and compliance by agencies. 127
149. Queensland Treasury considered that whilst agencies need to develop the skill base and the underpinning process and arrangements for producing good quality performance information, QAO also needs to keep building its skill base to undertake their work. Their work requires some judgement in terms of measuring and assessing performance. It is important to have good dialogue and understanding between agencies and QAO so that QAO understands the operations of the agencies but also the agencies understand the need to improve their own performance systems. 128
150. DPC advised the committee that, in its capacity as a central agency, it has some processes in place that look at whether departments have complied with government policy objectives but the systems are not as detailed or as comprehensive to cover all audit recommendations and implementation of them. They consider that there are two mechanisms that are in place which could be strengthened. The first is that there is an onus on departments themselves to implement audit findings and the second is the follow-up by QAO on its own findings. 129 DTMR and DIP both agreed with this assessment. 130
151. During the course of their review the 2010 Strategic Reviewers interviewed various stakeholders. They received a number of comments on the process for undertaking PMS audits. These comments included the following:
A concern that the scope and work plans for the audit were not settled at commencement of the audit task, and that changes in the objectives and scope of the audit occurred after commencement, sometimes without sufficient notification of such to the entities involved.
126 Mr Poole, Transcript 18 September 2009: 9 127 Submission 21: 4 128 Mr Bradley, Transcript 18 September 2009: 5 129 Mr Philip, Transcript 18 September 2009: 7 130 Mr Stewart and Mrs Schaefer, Transcript 18 September 2009: 7
24 Public Accounts and Public Works Committee Evaluation of Auditor-General’s PMS Audit Mandate
An absence of preliminary discussions with the entities concerning the nature and purpose of the PMS audit.
Instances of factual inaccuracies in drafts of a report at an advanced stage of the audit which required significant changes to key elements of the report.
A concern from some entities such as GOCs and universities as to whether they should be subject to PMS audits, given that they are subject to a range of other accountability and performance reporting requirements. 131
152. Having regard to this feedback the 2010 Strategic Reviewers suggested that there are benefits for QAO in having initial discussions with relevant CEOs prior to finalising the scope and objectives of PMS audits. 132
Committee comments
153. The committee has identified the issue of the use of experts and the skills of audit staff as a major issue for agencies. Whilst the Auditor-General has developed a diverse team within the PMS audit division, given the size of the division it follows that they cannot be experts in everything.
154. The committee agrees with the 2010 Strategic Reviewers assessment that the need to ensure that internal staff together with external advisors/experts having the necessary depth of knowledge and understanding remains a major challenge for QAO. The committee encourages the Auditor- General in the strategic use of subject use throughout the course of PMS audits.
155. Whilst the committee sympathises with DIP in their bid to reduce the audit costs to their agency, by including ‘project evaluation costs’ within the PMS audit mandate, ultimately the cost will be factored into the overall cost of audit services to government. Accountability is not without cost, however, it is essential in ensuring that valuable public resources are not wasted.
156. The independence of the Auditor-General is of paramount importance to the committee. It therefore disagrees with the suggestion that the Auditor-General’s mandate be amended to specify the number and categories of PMS audits to be undertaken each year. The Auditor-General needs to remain in a position where issues arising suddenly can be examined without fear or favour. This issue will be examined further in section 8 of this report.
157. The committee considers that there is a need for agencies to strengthen the mechanisms in place covering follow-up of audit recommendations. The committee heard that agencies have good systems in place to follow-up financial and compliance audit recommendations via audit committees. The committee can see no reason why this should not include PMS audit recommendations.
131 Carpenter, G and Gray, M, Report of the 2010 Strategic Review of the Queensland Audit Office, March 2010: 50-51 132 Carpenter, G and Gray, M, Report of the 2010 Strategic Review of the Queensland Audit Office, March 2010: 51
25 Public Accounts and Public Works Committee Evaluation of Auditor-General’s PMS Audit Mandate
Recommendation 2
The committee recommends that the Department of Premier and Cabinet, in its role as central agency, implement action to ensure that all audit recommendations, including PMS/Performance audit recommendations, are considered by agency audit committees.
7. Should the Auditor-General’s mandate be extended to include Performance Audits?
158. A number of submissions to the committee’s inquiry answered the question of how the effectiveness of the PMS audit mandate could be improved with arguments in favour of altering the Auditor-General’s mandate to conduct performance audits. The Auditor-General also advocated for an extension of the audit mandate to a performance audit mandate.
159. The 2010 Strategic Reviewers’ terms of reference included consideration of comparative models, practices and procedures used by offices in other jurisdictions equivalent to the QAO. They concentrated on the approaches adopted at the Commonwealth level by the ANAO, in New South Wales by the NSWAGO and in Victoria by the VAGO. All of these offices have an audit mandate for performance auditing. 133 They too recommended that the Auditor-General’s mandate be broadened to enable the Auditor-General to undertake performance audits. 134
7.1 What is a performance audit?
160. A performance audit is:
…an independent, objective and systematic assessment of public sector entities’ programs, resources, information systems, performance measures, monitoring systems and legal and policy compliance. 135
161. Performance audits, which are also referred to efficiency and/or effectiveness audits, evaluate whether an organisation is effectively meeting its objectives and using its resources economically and efficiently. 136 Performance audits involve the evaluation of the implementation of specific government programs, policies, projects and activities. They also examine how well administrative support systems operate. Audits can include consideration of economy (minimising cost); efficiency (maximising the ratio of outputs to inputs); effectiveness (the extent to which intended outcomes were achieved); and legislative and policy compliance. 137
133 Carpenter, G and Gray, M, Report of the 2010 Strategic Review of the Queensland Audit Office, March 2010: 107 134 Carpenter, G and Gray, M, Report of the 2010 Strategic Review of the Queensland Audit Office, March 2010: 121 135 ANAO, Performance Auditing in the Australian National Audit Office , April 2008: 3 136 http://download.audit.vic.gov.au/abt_performance_auditing.html [10 December 2009] 137 ANAO, Performance Auditing in the Australian National Audit Office , April 2008: 3
26 Public Accounts and Public Works Committee Evaluation of Auditor-General’s PMS Audit Mandate
162. From July 2002 until 31 December 2009, Performance Audits were prepared in line with Australian Auditing Standard AUS 806 – Performance Auditing. This standard has been replaced, effective from 1 January 2009, by Standard on Assurance Engagements ASAE 3500 – Performance Engagements. This Standard establishes mandatory requirements and provides explanatory guidance for conducting and reporting on performance engagements. 138
163. The standard defines a performance engagement to mean:
…a performance audit or a performance review of all or a part of the activities of an entity (or entities) to assess economy, efficiency or effectiveness. It includes a “performance audit engagement” or a” performance review engagement” directed to assess:
(i) the adequacy of an internal control structure or specific internal controls, in particular those intended to safeguard assets and to ensure due regard for economy, efficiency and effectiveness;
(ii) the extent to which resources have been managed economically or efficiently; and
(iii) the extent to which activities have been effective .139
164. The standard also notes that the terms ‘performance audit’ and ‘performance review’ are predominantly applied in the public sector. In the private sector these audits and reviews are commonly referred to as ‘operational audits’ and ‘operational reviews’. 140
165. Australasian Council of Auditors-General (ACAG) advised the committee that the Australian Auditing and Assurance Standards Board (AAASB) performance auditing standard was developed with significant input from ACAG. They also advised that although the legislative basis differs between jurisdictions, a co-operative performance auditing training program has been developed under the auspices of ACAG. 141
7.2 How do performance audits differ from PMS audits?
166. The Auditor-General commented that performance auditing is more widely recognised and understood then PMS auditing as it is supported by a greater body of research. He noted that the main difference between the two types of audits exists at the reporting stage of the audit. Reporting on PMS audit precludes direct observation about whether an entity is delivering its program economically, efficiently and effectively and is phrased using terms such as ‘systems’ and ‘frameworks’. 142
138 Australian Auditing and Assurance Standards Board, Standard on Assurance Engagements ASAE 3500 – Performance Engagements , July 2008: 5 139 Australian Auditing and Assurance Standards Board, Standard on Assurance Engagements ASAE 3500 – Performance Engagements , July 2008: 13 140 Australian Auditing and Assurance Standards Board, Standard on Assurance Engagements ASAE 3500 – Performance Engagements , July 2008: 14 141 Submission 12: 6 142 Submission 13: 4
27 Public Accounts and Public Works Committee Evaluation of Auditor-General’s PMS Audit Mandate
167. Messrs Anderson and Smerdon noted in the 2004 Strategic Review of QAO that PMS audit and performance audit are often characterized as being at opposite ends of the spectrum when in fact in many ways there is a significant degree of complementarity. 143
168. ACAG noted in their submission that there are a number of important similarities between the mandates. These include:
ensuring governments are accountable to Parliament;
independent reporting on agencies;
seeking to improve economy and efficiency in the management of public resources; and
holding agencies to account for their performance. 144
169. ACAG also advised that performance audits enable Auditors-General to address the issue of whether public sector entities are achieving value for money. They report on whether public sector entities meet the expectations of relevant stakeholders and comply with all the relevant legislation and policy requirements. As in-depth independent assessments they use all relevant data to assess economy, efficiency and effectiveness in the management of resources. A performance audit will then make recommendations for improving economy, efficiency and effectiveness of public sector entities and for the delivery of government programs. 145
170. They advised that the scope of a PMS audit is narrower. It does not comment on how to improve economy, efficiency and effectiveness. It determines whether the performance management systems in a public sector entity are good enough to enable the entity itself to assess if its objectives are being achieved economically, efficiently and effectively. 146
171. The Auditor-General advised that while there are a number of similarities between the two styles of audits, there are some significant differences. When scoping an audit a performance audit does not require that QAO look specifically at systems as does the current PMS audit mandate. There are further differences around the types of findings that can be reported and there are some differences around the scope of recommendations. 147
7.3 Previous consideration of extension of the Auditor-General’s mandate to include performance audit
172. The extension of the Auditor-General’s mandate, to include performance audits, has been considered a number of times since the introduction of the FA&A Act in 1977.
143 Anderson, R and Smerdon, H, Report on the Strategic Review of the Queensland Audit Office, October 2004: 24 144 Submission 12: 2 145 Submission 12: 3 146 Submission 12: 3 147 Mr Poole, Transcript 18 September 2009: 11
28 Public Accounts and Public Works Committee Evaluation of Auditor-General’s PMS Audit Mandate
173. Commissioner Tony Fitzgerald noted in his Report of the Commission of Inquiry into Possible Illegal Activities and Associated Police Misconduct in 1989 that:
…the Auditor-General had not concentrated upon the discovery of errors or improprieties in the general decision-making process of the departments and instrumentalities which have been audited. The primary concern of the Auditor-General has been to monitor and improve internal financial systems and controls. Limited attention has been paid to specific items of expenditure.
This is partly because it has been thought that any other course would be impractical and too expensive. It was also considered that all financial systems were vulnerable to error and impropriety, and that the effectiveness of the Auditor-General’s staff was less in relation to senior officials who had greater authority.
These self-imposed limitations should be reviewed if the Auditor-General is to become an effective check on the abuse of public money. The Auditor-General’s office should also be given resources to the extent necessary for the role to be fulfilled. 148
174. In September 1991, the Electoral and Administrative Review Commission (EARC) published its Report on Review of Public Sector Auditing in Queensland . EARC heard evidence arguing both for and against the introduction of performance or efficiency audits. The commission noted that whilst performance auditing was an accepted part of the external audit function in many public sector financial systems, there was a reluctance on the part of previous Queensland administrations to provide the Auditor-General with clear legislative responsibility and resources to undertaken performance audits which stemmed from the philosophy that placed primary responsibility for performance review on the management of departments and statutory bodies. 149
175. The commission acknowledged that the Goss government had reinforced the responsibility of departments and statutory bodies for the effective management of their assigned resources by further developing program budgeting and management and introducing a requirement for annual performance reviews and comprehensive cyclical program evaluations with these evaluations being the responsibility of accountable officers. The commission, however, did not agree that these reviews and evaluations replace the need for performance auditing by the Auditor- General. 150
176. The commission cited a number of reasons for this view. The most compelling of these reasons was that performance auditing provides a form of program review that is totally independent of program management and of executive government generally. They argued that an Auditor- General review brings a degree of professional objectivity, impartiality and independence to the examination of organisational efficiency which may be difficult to match within the organisation concerned. 151
148 Commission of Inquiry into Possible Illegal Activities and Associated Police Misconduct, Report of a Commission of Inquiry into Possible Illegal Activities and Associated Police Misconduct , July 1989: 135 149 EARC, Report on Review of Public Sector Auditing in Queensland , September 1991: 78 150 EARC, Report on Review of Public Sector Auditing in Queensland , September 1991: 79 151 EARC, Report on Review of Public Sector Auditing in Queensland , September 1991: 80
29 Public Accounts and Public Works Committee Evaluation of Auditor-General’s PMS Audit Mandate
177. They also considered that having an Auditor-General able to review government efficiency and report independently to Parliament on matters arising from the review is an important and necessary check and balance in the overall system of public administration and has the potential to benefit everyone – the Parliament in its role of oversee of executive performance, the community, which an be made aware of issues which affect it as client and taxpayer, and executive government, which has opportunity to obtain improved insights into the management of particular activities and the level of performance and practice expected by the Parliament. 152
178. EARC also commented on arguments that the Auditor-General’s independence could be called into question if they were to challenge or debate the wisdom of policy decisions through means of performance reports. The commission found that while Auditors-General must be careful to avoid suggestions of partiality, and to avoid intrusion into areas for which they may have no professional responsibility or expertise, there may be times when it is appropriate or necessary for auditors to draw attention to government policy which appears to have a negative impact on operational efficiency. The commission felt that in these circumstances, the Auditor-General should have the freedom to draw attention to the existence of the policy and its apparent effects without being drawn into debate on the policy’s inherent merits. 153
179. EARC recommended that:
(a) the Auditor-General should have authority under the Financial Administration and Audit Act to undertake performance audits into all:
(i) departments;
(ii) statutory bodies;
(iii) bodies associated with a department or statutory body;
(iv) local authorities including the Brisbane City Council; and
(v) any other person or body whose accounts are required by law to be audited by the Auditor-General, including any future GBEs established;
(b) the Act should provide that a performance audit may be of all or any of the activities of the body to determine whether it is carrying out those activities effectively, economically and efficiently and in compliance with all applicable laws;
(c) while the Act should permit the Auditor-General to examine whether government activities are being carried out effectively as well as economically and efficiently, the Auditor- General should not, as a general rule, undertake evaluations of programs that would require the Auditor-General to examine the appropriateness of strategic program goals. However, there may be circumstances where the Auditor-General is required to draw the attention of Parliament to particular government policies which appear to have a negative impact on operational efficiency. Legislative provisions should not restrict the Auditor- General from making this type of observation;
152 EARC, Report on Review of Public Sector Auditing in Queensland , September 1991: 79 153 EARC, Report on Review of Public Sector Auditing in Queensland , September 1991: 86
30 Public Accounts and Public Works Committee Evaluation of Auditor-General’s PMS Audit Mandate
(d) the Act should provide that the Auditor-General has, in addition to his or her other powers under the Act, at all reasonable times full and free access to all records and other documents in the possession of any auditable body that are reasonably necessary for the purpose of undertaking a performance audit;
(e) the Auditor-General should have freedom to determine the priority to be given to performance audits of particular government activities. The Commission does not recommend that any particular cyclical approach be adopted by the Auditor-General or that such an approach be required under legislation;
(f) where the Auditor-General undertakes a performance audit, the Auditor-General should, as part of the audit, examine the integrity of relevant performance review and evaluation systems established for the program or activity under review and report on this examination as part of the performance audit report;
(g) the Auditor-General should not be required to give an audit opinion on performance indicators; and
(h) from time to time, the Auditor-General should undertake a performance audit into the integrity of program review and evaluation systems across government or across particular types of agencies of the kind undertaken by the ANAO .154
180. The Parliamentary Committee for Electoral and Administrative Review (PCEAR) considered EARC’s report and recommendations after calling for further public submissions. The committee, reporting in December 1991, endorsed EARC’s recommendations and the provisions of the draft bill in respect to performance auditing. 155
181. In November 1992, the government tabled its response to the EARC report. The government advised that they proposed to expand the audit role of the Auditor-General to provide a clear audit mandate in relation to the performance of public sector entities in Queensland, however it did not accept the recommendation that Queensland follow the approach to performance auditing adopted by the Commonwealth and some other states. 156
182. The response articulates the government’s position as follows:
As Queensland’s Public Finance Standards require departments and statutory bodies to institute a system of performance evaluation and review and to carry out regular position assessments, the Government considers that an alternative approach to the issue would better reflect the public sector financial management framework in Queensland. Such an approach, “systems-based performance auditing”, is derived from models adopted in other countries.
Under the systems-based performance auditing, the auditor is not required to adopt the management role of reviewing and reporting on an organisation’s performance, but instead reviews how well management itself is doing this job. In this way, the auditor’s role – to assess whether performance management systems are in place and are working properly – is clearly differentiated from management’s role.
154 EARC, Report on Review of Public Sector Auditing in Queensland , September 1991: 337-338 155 Parliamentary Committee for Electoral and Administrative Review, Public Sector Auditing , December 1991: 5 156 Queensland Government, Queensland Government Response to the Electoral and Administrative Review Commission Report on Review of Public Sector Auditing in Queensland , November 1992: 14
31 Public Accounts and Public Works Committee Evaluation of Auditor-General’s PMS Audit Mandate
This approach reflects the Government’s view, which underlies its policies on program management, corporatisation and commercialisation, that responsibility for establishing standards of performance, for determining who is responsible for achieving those standards and for reporting the results rests squarely with the managers of public sector entities. The Auditor-General’s role is to audit and report on the existence and adequacy of performance management systems and practices .157
183. In November 1992, the government introduced the Audit Legislation Amendment Bill . The Minister noted, when introducing the bill, the government proposed major amendments to the FA&A Act designed to give effect to the EARC-based reforms. 158 The Minister advised the Parliament that EARC considered that the Auditor-General should be required to review and report on program performance, however, this recommendation did not take sufficient account of the existing responsibilities of departments and statutory bodies to review and report on their performance and that Auditor-General can best add value to this process by auditing the management systems adopted by agencies. The Minister advised that the Auditor-General can scrutinise the performance management systems of any public sector entity and in this way, agencies retain their responsibility for performance management, but are held accountable to Parliament through regular external reporting processes as well as periodic scrutiny by external audit. 159
184. In August 1997, Mr Tom Sheridan, provided his report on the first strategic review of QAO. Mr Sheridan, a former Auditor-General for South Australia, noted the reasons advanced for restricting the Auditor-General’s role to the audit of performance management systems and strongly supported the view that it is the responsibility of management of an entity to ensure that the management practices and performance of the entity are efficient, economic and effective. 160 At the time of the strategic review, the Auditor-General had had the PMS audit mandate for approximately 3 years.
185. However, he argued that to restrict the Auditor-General’s role in this area and for Parliament to rely solely on an entity’s own assessment of its performance overlooks two important factors:
An independent assurance that adequate systems are in place to assess whether an entity’s objectives are being achieved economically, efficiently and effectively is not guarantee, by itself, that actual performance measures up to that criteria; and
An assurance on actual performance by an entity through its responsible Minister to the Parliament, leaves open to question the impartiality of that assurance. In making this point Mr Sheridan stressed that the integrity of the persons providing that assurance was not questioned, but rather the recognition of an understandable degree of self-interest that can exist in the nature of the reporting framework itself. 161
157 Queensland Government, Queensland Government Response to the Electoral and Administrative Review Commission Report on Review of Public Sector Auditing in Queensland , November 1992: 14 158 Queensland Legislative Assembly, Hon KE De Lacy, Treasurer, Second Reading Speech, Parliamentary Debates (Hansard) , 11 November 1992: 466 159 Queensland Legislative Assembly, Hon KE De Lacy, Treasurer, Second Reading Speech, Parliamentary Debates (Hansard) , 11 November 1992: 467 160 Sheridan, TA, Report of the Strategic Review of the Queensland Audit Office , July 1997: 24 161 Sheridan, TA, Report of the Strategic Review of the Queensland Audit Office, July 1997: 24
32 Public Accounts and Public Works Committee Evaluation of Auditor-General’s PMS Audit Mandate
186. Mr Sheridan considered that the restriction denies Parliament and the community a level of independent advice and information to which it is entitled. He noted that his experience is that, properly managed, performance audit can make a significant contribution to effective public sector management and administration and to effective resource use. He advised that performance audit should be seen as a logical extension of the financial and compliance audit. He also considered that Auditors-General can undertake performance audits without transgressing on the area of government policy objectives as the mandate is to review and report on the economy, efficiency and effectiveness of the implementation and delivery of government policy objectives but not appropriateness (or otherwise) of government policy objectives. 162
187. In April 1998, the PAC reported on its findings after considering the 1997 QAO Strategic Review. That committee found that the majority of the submissions it received agreed in principle to the broadening the Auditor-General’s mandate on the basis that it provides greater accountability within the public sector. 163
188. Whilst the committee agreed that the carrying out of operations in an efficient, effective and economical manner was the responsibility of management, they considered that a performance audit did not relieve management of such responsibilities but provided an objective and independent assessment of the information reported to Parliament. 164
189. The committee found that performance audit reports provide information that can be used by executive government to make better informed decisions and can also act as a deterrent to sloppy management. 165
190. The committee considered that a major drawback in any argument for performance audit centred on the potential encroachment of commenting on government policy issues by the Auditor- General. It found that the distinction between commenting on policy and commenting on its implementation is sometimes a grey area and has been a source of conflict in other jurisdictions. 166 The committee recommended, as a safeguard, that the Auditor-General brief the PAC on all proposed performance audits on a consultative basis. 167
191. The committee recommended that:
the FA&A Act be amended to permit the Auditor-General to conduct performance audits that examine the economy, efficiency and effectiveness of the operations of public sector entities.
the performance audit mandate extend to all public sector entities for which the Auditor-General is Auditor-General.
the FA&A Act be amended to preclude the Auditor-General from commenting on government policy objectives.
162 Sheridan, TA, Report of the Strategic Review of the Queensland Audit Office , July 1997: 25 163 PAC, Report No 44 – Review of the Report of the Strategic Review of the Queensland Audit Office , April 1998: 5 164 PAC, Report No 44 – Review of the Report of the Strategic Review of the Queensland Audit Office , April 1998: 6 165 PAC, Report No 44 – Review of the Report of the Strategic Review of the Queensland Audit Office , April 1998: 7 166 PAC, Report No 44 – Review of the Report of the Strategic Review of the Queensland Audit Office , April 1998: 7 167 PAC, Report No 44 – Review of the Report of the Strategic Review of the Queensland Audit Office , April 1998: 8
33 Public Accounts and Public Works Committee Evaluation of Auditor-General’s PMS Audit Mandate
The Auditor-General exercise discretion as to the degree of emphasis to be given to economy, efficiency and effectiveness in the conduct of a particular performance audit.
The conduct of any performance audit be reported to Parliament in accordance with existing provisions in the FA&A Act.
The Auditor-General consult with the PAC prior to undertaking a performance audit.
Performance audits be undertaken by a specialised unit within QAO.
The Auditor-General is appropriately resourced to meet the additional services that will be required under a performance audit mandate. Funding is to be incorporated in the QAO annual budget allocation. There should be no charge to auditees for a performance audit.
QAO liaise with internal audit to minimise audit duplication. 168
192. In December 1999, the government tabled its response to the committee’s recommendations relating to performance auditing. It did not accept the recommendations. It considered that improvement of the economy, efficiency and effectiveness of the public sector is essentially a management function and that performance auditing should be used as a management tool to improve efficiency rather than an accountability measure. 169
193. The response also noted the government’s concern that potential problems could arise with the Auditor-General being drawn into the partisan political and policy debate. They did not consider that the legislated safeguards proposed by the PAC would be sufficient to prevent the potential involvement of the Auditor-General in the policy debate. 170
194. The government also advised that it had recently introduced accrual output budgeting under the Managing for Outcomes (MFO) Framework and it was concerned that the extension of the Auditor-General’s mandate to examine economy, efficiency and effectiveness against the outcomes approved in the budget may introduce more change in a system that is currently heavily committed to implementing major new accounting strategies and the skills of articulating outcomes and performance indicators were still being developed. 171
7.4 Arguments in favour of extension of the Auditor-General’s mandate
195. The Auditor-General advocated that a reporting regime allowing him to directly provide an opinion on the economy, efficiency and effectiveness of an entity’s operations would provide a greater understanding and enhancement of accountability requirements across the public sector. It would provide Parliament with an additional level of assurance on the performance of entities in delivering the outputs required by government. 172
168 PAC, Report No 44 – Review of the Report of the Strategic Review of the Queensland Audit Office , April 1998: 13 – 14 169 Hon P Beattie MLA, Premier , Response to the PAC Report on the Review of the Report on the Strategic Review of the Queensland Audit Office , 24 December 1999: 3 170 Hon P Beattie MLA, Premier , Response to the PAC Report on the Review of the Report on the Strategic Review of the Queensland Audit Office , 24 December 1999: 4 171 Hon P Beattie MLA, Premier , Response to the PAC Report on the Review of the Report on the Strategic Review of the Queensland Audit Office , 24 December 1999: 5 172 Submission 13: 2
34 Public Accounts and Public Works Committee Evaluation of Auditor-General’s PMS Audit Mandate
196. The Auditor-General advised the committee that he considered that it is time for the Queensland Parliament to consider a change in the mandate to reflect the performance audit approach used now in most parts of Australia and internationally. He advised that the conclusion in other jurisdictions is that this is an enhancement to the systems of accountability as it allows the audit report to focus directly on the performance achieved by the agency rather than indirectly through an examination of just its reporting systems. 173
197. The Auditor-General advised that the advantages that a performance audit mandate would bring to the Parliament includes that it would receive an independent assurance around a broader range of things that are more focused on what the community is looking for. He advised that he believes the community is looking for the outcomes from government activity and assurance that those outcomes are being achieved efficiently, effectively and with due economy. He considers that simply focusing on the systems does not achieve that. Whilst he considers that performance reporting by agencies is improving, without third-party assurance the system is deficient. 174
198. The Auditor-General advised that he considers that the Parliament receives from him independent assurance on all the financial statements but with all the performance data and the non-financial performance data there is currently no independent assurance. 175
199. He further advised that whilst PMS audits are not unique to Queensland, there are not too many jurisdictions that have it. There is a far larger body of expertise, understanding and research on auditing standards and so forth around performance audit activity. He said that the performance audit activity is better known and understood by agencies so therefore moving in that way would enable QAO to more easily do the work.176
200. ACAG highlighted that the advantage of performance auditing is that it parallels the move in public sector management to focus more on outcomes, performance indicators and transparency in public administration. They consider that the broader performance audit mandate offers a number of benefits to improving public sector accountability and performance including broader topics, scope, coverage, impact and recommendations. 177
201. ACAG advised that there is a clearer link between process and outcomes with a performance audit compared with a PMS audit. Performance audits typically review and report on programs that contribute to government outcomes and comment on the actual impact of process deficiencies. They provide qualitative as well as quantitative comment such as analysing the causes of failures or performance gaps. A PMS audit looks at and reports on the status of systems and does not comment on the link between systems and outcomes. 178
173 Mr Poole, Transcript 18 September 2009: 2 174 Mr Poole, Transcript 18 September 2009: 11 175 Mr Poole, Transcript 18 September 2009: 11 176 Mr Poole, Transcript 18 September 2009: 14 177 Submission 12: 3-4 178 Submission 12: 5
35 Public Accounts and Public Works Committee Evaluation of Auditor-General’s PMS Audit Mandate
202. ACAG considers that performance audit reports improve the transparency of government operations through reporting publicly to Parliament. Parliament and the community can look for confirmation that selected public sector programs, activities or operations are being conducted efficiently, effectively and economically. Agency officials responsible for the implementation of government policy and the provision of services are held accountable for those services. 179
203. ACAG advised that performance audits in many jurisdictions are an accepted and important component of the landscape of public administration. Performance audits are a ready source of public information to aid the understanding of public administration and provide stimulus for agencies to improve their performance. 180
204. The Auditor-General argued that having to frame findings in the final PMS audit report in the context of systems and frameworks can result in an overly technical report which may be less useful to the Parliament, the entity itself and the community. Performance audit reports can deal specifically about achievement of the economy, efficiency and effectiveness of an entity’s operations, providing greater focus on the achievement of outputs. This would complement any heightened requirements by government for clearer specification of outputs and for consistent achievement of those outputs. He advised that a performance audit report can directly comment on the cause of the problem and recommendations can be made about resolution, enabling a better understanding of the issues being raised. 181
205. The Auditor-General advised that as his mandate currently stands, the audit may identify performance issues which he considers are significant and may wish to inform the Parliament. In some cases, the cause of the issue may be able to be related to a weakness in the entity’s performance management systems or frameworks. In these instances, the report to Parliament must focus on the system weakness rather than the performance deficiency, thus resulting in a more technical style of report. However, if the performance issue cannot be directly attributed to the performance management systems and frameworks, the Auditor-General is put in a position of not reporting because of the restrictions of the PMS audit mandate or considering the ability, if any, to report under the financial and compliance audit mandate. 182
206. DPW argued that given that it is an essential function of public audit to provide an assurance to Parliament and the community regarding the exercise of public accountability obligations, a decision to expand the QAO PMS audit mandate to a full performance audit mandate could be perceived as a most useful improvement in the State’s accountability regime. Such a performance audit mandate would not involve a review of government policy decisions but rather examine actions taken in the course of agency administration and can encompass a review of information leading to policy decisions. Under the Westminster system, particularly within its application in the Australian context, an Auditor-General traditionally does not comment on the merits or efficacy of political or government policy decisions. 183
179 Submission 12: 5 180 Submission 12: 6 181 Submission 13: 4 182 Submission 13: 12 183 Submission 18: 2
36 Public Accounts and Public Works Committee Evaluation of Auditor-General’s PMS Audit Mandate
207. The Institute of Chartered Accountants in Australia (ICAA) advised the committee that they believe that the alignment of the QAO mandate with that of the many other audit offices with regard to performance auditing has much to recommend it. The reason for making this suggestion is that the concept of performance auditing is now well established and users of these reports have, in ICAA’s view, a reasonably clear understanding of what performance auditing does and does not encompass. The extent of understanding and acceptance of matters addressed under the PMS auditing mandate is not as well established and understood and as a consequence the benefits to be gained from such engagements are likely to be more limited. In ICAA’s view, the public interest would be better served by alignment with other jurisdictions in respect of performance auditing. 184
208. It has also been suggested to the committee that it would simplify the whole understanding and negotiation process for determining the audit scope and objective and the reporting process to use the term performance audit and its common meaning rather than artificially defining the PMS mandate. The amount of effort required to research and conduct the audit would be similar, however, the real difference, compared to what the Auditor-General is currently able to report, is that he is precluded from commenting on actual performance. One risk of this is that he can identify an efficient and effective system of measuring performance but can’t publicly comment on whether or not the right thing is being measured or whether or not the desired outputs and outcomes are being achieved. 185
209. The Auditor-General confirmed this by citing an example from a minister. He advised that while discussing a draft report the minister commented that the report seemed to be focused on the performance management systems and did not take into account the fact that in his view the department’s performance was good. He advised the minister that the mandate required him to look at the systems irrespective of their view of performance. 186
210. The Auditor-General confirmed that as part of the current mandate there are some constraints around reporting, where they try to report on what the systems are but form some judgements as to how material any breakdown in those systems are going to be and what is that impact on performance. But because of the restrictions of the mandate the Auditor-General can’t make that connection explicitly within the report and therefore they have some discussions around the systems rather than performance which, in his view, is a bit of a problem with the current mandate. 187
7.5 Arguments against extension of the Auditor-General’s mandate
211. Queensland Treasury argued that QAO has focused on PMS audits per se only in the past few years and given the complex nature of and associated difficulty of assessing public sector service delivery outcomes from performance measures, it is too early to properly assess the effectiveness of the current PMS audit mandate. They considered that further research would need to be undertaken to determine the effectiveness of the outcomes achieved from the previous audits. 188
184 Submission 19: 1 185 Submission 23: 1 186 Mr Poole, Transcript 18 September 2009: 7 187 Mr Poole, Transcript 18 September 2009: 7 188 Submission 15: 3
37 Public Accounts and Public Works Committee Evaluation of Auditor-General’s PMS Audit Mandate
212. PSC considers that the role of the PMS audit is different to the function of assessing performance and should be valued in its own right. 189 The Commission advised that given the past utilisation of PMS audit reports by the former SDPC, they consider PMS audits to value-add to the performance management scheme applying to the Queensland public sector, fulfil a critical and vital function in the public sector accountability framework and should continue to be a mandated activity of the Auditor-General. 190
213. PSC further advised that the Public Service Act 2008 (s37) established the following mandate in relation to the operation of the PSC from 1 July 2008.191 Report on effectiveness and efficiency (1) The Minister may, by signed notice, refer to the commission any matter relating to the effectiveness or efficiency of a public service office for the commission to review and report to the Minister about (a commission review). (2) The reference may be for a single commission review for 2 or more public service offices. (3) Each chief executive or head of a public service office the subject of the reference must give the commission the help it reasonably requires to conduct the review. (4) The Minister must table the report in the Legislative Assembly. 192
214. The Commission advised that these legislative provisions establish a performance auditing mandate, to be exercised by them, at the direction of the Minister, with respect to departments and the police service. It does not provide a mandate in respect of public sector offices, statutory authorities or GOCs. 193
215. They consider that the need to extend QAO’s mandate to conduct more holistic performance audits of departments will in part be informed by the government’s deliberations on the future conduct of performance management reviews under its public sector reform agenda. 194
216. PSC also advised that they consider that, should the Auditor-General PMS audit mandate be extended, then they would support the establishment of an evaluation framework to be applied to the PMS audit program. 195
217. The QPS argued that there is no need to expand the PMS audit mandate to include provision for the conduct of full performance audit on the basis that they have in place robust performance management systems to enable the optimal performance of the organisation and these systems are monitored on an ongoing basis through the internal Operational Performance Reviews conducted in each police district across Queensland and other internal and whole-of-government accountability processes. 196
189 Submission 16: 2 190 Submission 16: 3 191 Submission 16: 3 192 Public Service Act 2008 , s37 193 Submission 16: 3 194 Submission 16: 3 195 Submission 16: 5 196 Submission 17: 2
38 Public Accounts and Public Works Committee Evaluation of Auditor-General’s PMS Audit Mandate
218. Queensland Treasury highlighted its concern that performance audits require issues of judgement and it brings into question areas of government policy. They noted that in other jurisdictions they have limitations on their audit mandates so that it doesn’t bring into account challenging whether policy is appropriate or not appropriate. 197
7.6 Resource implications
219. The committee asked the Auditor-General whether there would be any resource implications for QAO should the mandate be extended to a performance audit mandate. He advised that over the past 5 years the number of staff dedicated to the PMS audit activity has increased significantly. He advised that QAO now has a breath of expertise within the group coming from a range of disciplines. 198
220. He further advised that the in a number of audits they have been able to draw on expertise outside of the organisation by way of experts who can provide some reference for them. He advised that QAO certainly use the expertise of the agencies to understand the particular issues. They undertake the audits through a performance engagement framework under auditing standards. He believes that this provides QAO with the capacity to ask the right questions and to draw the right conclusions, but then to rely, where necessary, on some technical experts outside the organisation to help with understanding. 199
221. If the mandate was to be expanded, the major impact would be that QAO would do slightly different audits. The mandate changes would change the scope of the audits without necessarily having to have additional resources. He advised that the current staff are well equipped to undertake performance audits. Additional staff would only enable QAO to undertake more audits. 200
222. The 2010 Strategic Reviewers also commented on the issue of the level of skills and expertise within QAO to undertake performance audits. They encouraged the Auditor-General to continue to build the internal skills base of QAO. In addition, they suggested that consideration should be given to a greater use of external expertise, from either as part of the audit team or as an advisor for the audit. They considered that it is desirable that there is involvement from subject experts from initial scoping and planning of the audit through to review of the final report to Parliament. In selecting external expertise, the Auditor-General needs to consider the background and experience of the expert and to avoid any conflicts of interest, or other biases or prejudices which may impact on the audit. 201
197 Mr Bradley, Transcript 18 September 2009: 11-12 198 Mr Poole, Transcript 18 September 2009: 13 199 Mr Poole, Transcript 18 September 2009: 13 200 Mr Poole, Transcript 18 September 2009: 14 201 Carpenter, G and Gray, M, Report of the 2010 Strategic Review of the Queensland Audit Office, March 2010: 120
39 Public Accounts and Public Works Committee Evaluation of Auditor-General’s PMS Audit Mandate
223. In his response to the 2010 Strategic Review, included in the report, the Auditor-General indicated that if the Parliament approves the amendment to the legislation to provide for performance audits, his office is well-equipped and ready to undertake the activity. 202 He advised that over the past five years, staff recruitment and the experience gained in conducting PMS audits have ensured that QAO staff stand ready to implement the expended mandate, should it be approved. He also agreed that in line with the recommendations and in keeping with contemporary audit practice in other jurisdictions, QAO will use independent experts for these audits to assist internal staff. 203
224. The Auditor-General advised that the significant cost of subject experts for all phases of performance audits is likely to add significant costs for those audits. 204
7.7 2010 Strategic Review recommendations – performance audit mandate
225. The 2010 Strategic Reviewers noted from their comparative analysis of audit models in other jurisdictions that there are a number key differences in approach as to the nature and extent of the audit mandate in those jurisdictions. The primary difference in the legislative mandate between Queensland and other jurisdictions relates to the power to undertake performance audit generally. Other jurisdictions have the power to audit performance management and to report generally on performance. 205
226. They advised that in their view, QAO has responded positively to the challenge issued in the 2004 Strategic Review and has expanded both the number and range of PMS audits and it is timely to revisit the issue of the scope of the mandate. 206
227. The 2010 Strategic Reviewers consider that there is a strong case for a broadening of the Auditor- General’s mandate, to include auditing on performance generally, on the following grounds 207 :
Parliament, as the Auditor-General’s client, should have its auditor provide independent reports on performance of public sector entities and on government programs/activities. This would enable an objective assessment as to whether public resources are being used with appropriate economy and efficiency, and are delivering government programs and activities effectively and in compliance with appropriate laws and regulations.
It is the performance of public sector entities in delivering government services and programs which should be of primary importance, rather than just their performance management systems per se. A focus on systems rather than performance can produce some perverse and misleading outcomes. For example, while an entity’s systems may be good, it does not follow necessarily that the performance will be good. Conversely, there may be instances where an entity’s systems are poor, but nevertheless overall performance may well be sound.
202 Carpenter, G and Gray, M, Report of the 2010 Strategic Review of the Queensland Audit Office, March 2010: 168 203 Carpenter, G and Gray, M, Report of the 2010 Strategic Review of the Queensland Audit Office, March 2010: 170 204 Carpenter, G and Gray, M, Report of the 2010 Strategic Review of the Queensland Audit Office, March 2010: 174 205 Carpenter, G and Gray, M, Report of the 2010 Strategic Review of the Queensland Audit Office, March 2010: 118 206 Carpenter, G and Gray, M, Report of the 2010 Strategic Review of the Queensland Audit Office, March 2010: 119 207 Carpenter, G and Gray, M, Report of the 2010 Strategic Review of the Queensland Audit Office, March 2010: 119
40 Public Accounts and Public Works Committee Evaluation of Auditor-General’s PMS Audit Mandate
There is a fundamental principle in terms of integrity and accountability as to whether the performance of public sector entities, as outlined in reports to Parliament (including in Annual Reports), has been subject to external independent scrutiny.
Performance audits provide a constructive opportunity for the QAO to work closely and co-operatively with entities to improve their performance. In this way, performance audits can be a powerful catalyst for desirable change, and provide a different and fresh perspective on issues which may not be apparent to those inside an entity.
Performance auditing is well established, and has proven to work effectively in all other jurisdictions in Australia, apart from the Northern Territory. It is generally accepted as best practice, and is widely applied in other advanced nations, including the UK, USA, Canada, Denmark, Norway, Sweden, Portugal, Austria and New Zealand. 208
228. The Reviewers noted that there have been some arguments advanced against broadening of the mandate, as follows:
The existing mandate allows for the Auditor-General to audit and report on performance management systems and to also report both on the relevance of the performance measures as well as the fairness with which that information is represented. This is not far short of a full performance mandate in any case.
The extension of the power of the Auditor-General in 2007 to cover performance measures has not been in place for long enough to assess its benefits.
There is a concern from some audit clients as to whether the QAO has the appropriate level of skills and expertise to undertake performance audits (over and above the skills and expertise required for PMS audits). 209
229. The 2010 Strategic Reviewers found that there was a diversity of views expressed on the scope of the mandate for performance auditing. Overall, there was strong and broad-based support for an extension of the mandate, on the basis that it would strengthen public administration and accountability in Queensland. They concluded that it is now time for QAO’s audit mandate to be expanded to full performance audits. 210
230. The 2010 Strategic Reviewers made the following recommendation in respect of the performance audit mandate:
RN.16(i) That the Auditor-General Act 2009 be amended to broaden the audit mandate to enable the Auditor-General to undertake performance audits, such amendment to be generally consistent with legislation applying in other Australian jurisdictions. 211
208 Carpenter, G and Gray, M, Report of the 2010 Strategic Review of the Queensland Audit Office, March 2010: 119 209 Carpenter, G and Gray, M, Report of the 2010 Strategic Review of the Queensland Audit Office, March 2010: 120 210 Carpenter, G and Gray, M, Report of the 2010 Strategic Review of the Queensland Audit Office, March 2010: 120 211 Carpenter, G and Gray, M, Report of the 2010 Strategic Review of the Queensland Audit Office, March 2010: 121
41 Public Accounts and Public Works Committee Evaluation of Auditor-General’s PMS Audit Mandate
Committee comments
231. As the Premier noted in her response to the Integrity and Accountability in Queensland discussion paper, government has a duty to continually adjust to meet the challenges never imagined 20 years ago. 212 These challenges include the management of performance by government agencies.
232. The committee has found that the Auditor-General is currently restricted in his ability to report to Parliament unreservedly on the results of his PMS audits due to the fact that he must report in a manner limited to an examination of the systems and frameworks in place. The legislation, as it is currently framed, can lead to less the optimum results in terms of disclosure to the Parliament. It can also, at its worst, lead to overly technical or misleading results. The committee has heard evidence that agencies can have sound systems and poor performance or poor systems and sound performance which currently cannot be clearly articulated in reports to Parliament.
233. The Auditor-General has provided an extensive body of work in terms of PMS audit reports published over the past 5 years. The committee considers that he has extensively tested the boundaries of the existing mandate. Whilst it has been acknowledged that this has been beneficial to agencies, the committee believes that more can be achieved with an expanded mandate.
234. The committee considers that both the Parliament and agency management would in fact benefit from the provision of independent oversight on agency performance. The committee acknowledges that it is management’s responsibility to review and report on their agencies’ performance. However, this does not prevent QAO from bringing a fresh and independent assessment of that performance. The provision of this mandate to the Auditor-General also does not prevent oversight agencies within government, including the PSC, from conducting there own assessment of agency performance specific to their own requirements.
235. The committee has heard from the Auditor-General that he is convinced that QAO has the capacity to undertake the work involved in an extended mandate. The Auditor-General would be assisted by the fact that accounting standards covering performance audit already exist and there is extensive expertise already available which is skilled in providing performance audit services.
236. The committee notes the comments made by the 2010 Strategic Reviewers in respect of the use of subject matter experts during the PMS audit process referred to in section 6 of this report. The committee agrees that one of the major challenges facing QAO in the future will be to ensure that the team involved in the PMS/Performance audit area has the necessary depth of knowledge.
237. Whilst the Auditor-General is confident that QAO has the capacity to expand the mandate he needs to ensure there is judicious use of external experts/advisers. The committee agrees that it makes sense that subject area experts, including secondments from within departments, be utilised over the entire course of the audit. Whilst this may increase the cost of the audits, it will ensure that the audit arrives at more accurate conclusions which will be of benefit both to Parliament, QAO and the agency being audited. However, the committee is conscious that any increase in funding must reflect a demonstrable benefit and it encourages the Auditor-General to be strategic in his use of experts as funding is not infinite.
212 Queensland Government, Response to Integrity and Accountability in Queensland , November 2009: 5
42 Public Accounts and Public Works Committee Evaluation of Auditor-General’s PMS Audit Mandate
238. It is on this basis that committee therefore considers that the Auditor-General’s mandate should be broadened to include performance auditing. This recommendation is, however, prefaced with a number of other considerations.
239. The committee considers that, at least initially, the performance audit mandate should only apply to government departments and that the government owned corporations sector remain subject to the existing PMS audit mandate. The committee believes that any move to the performance audit mandate within this sector needs to be extensively examined with the commercial nature of these bodies given due consideration.
240. This is recommendation is consistent with the practice in the Commonwealth where government business enterprises are generally not subject to performance audits by the Australian National Audit Office. The Commonwealth legislation includes provision for the responsible Minister, the Finance Minister or the Joint Committee of Public Accounts and Audit to be able to request the Auditor-General to conduct a performance audit of a government business enterprise. The committee considers that this provision should be included in the Queensland legislation.
241. The committee is ever conscious of the need for the Auditor-General’s independence to be maintained. This independence can be jeopardised if the Auditor-General were to be drawn into the partisan political debate. It is for this reason that the committee considers that restrictions on the Auditor-General’s ability to question the merits of government policy need to be included in the legislation.
242. The committee considers that the Auditor-General should have regard for the priorities of the Parliament when deciding upon the program of performance audits to be conducted. This issue is examined further in section 7.8 of this report.
Recommendation 3
The committee recommends that the Auditor-General Act 2009 be amended to allow for provision of the following:
mandate to allow for the conduct of a performance audit of government agencies with the exception of government owned corporations;
a performance audit of a government owned corporation may be conducted at the request of the Parliament, the responsible minister, the Treasurer, or the Public Accounts and Public Works Committee;
Auditor-General not to question the merits of government policy;
the Auditor-General to have regard for the audit priorities of the Parliament when deciding upon the program of performance audits.
43 Public Accounts and Public Works Committee Evaluation of Auditor-General’s PMS Audit Mandate
Recommendation 4
The committee recommends that sufficient additional funding be provided to QAO to enable the strategic use of subject experts in all phases of performance audits.
7.8 2010 Strategic Review recommendations – Strategic Audit Plan
243. In Queensland, there is regular consultation between the Auditor-General and the PAPWC on audit matters, including some discussion on the program of proposed PMS audits. QAO does not publish a formal plan for PMS audits. However, the 2010 Strategic Reviewers consider that the process is more informal and less structured than is desirable. They believe that there would be benefit in an enhanced process that is more transparent and accountable than is currently the case and that provides for greater input from stakeholders earlier in the planning process. 213
244. The 2010 Strategic Reviewers noted that other jurisdictions have a more formalised process for audit offices to consult with the relevant Parliamentary Committees on the planning of audits, including performance audits. In some jurisdictions there are legislative requirements to publish an annual audit plan. In other jurisdictions, such as the ANAO, there are no legislative requirements however the audit office chooses to publish its annual plan for performance audits. The relevant Parliamentary Committee has a greater role in the budget process for the respective audit offices and consultation on the audit program is normally undertaken in that context. 214
245. The 2010 Strategic Reviewers consider that there would be benefits in the development of a rolling three-year Strategic Audit Plan by the QAO to provide better guidance to the Parliament, other stakeholders and the public on potential topics or subjects for future audits, whether that be PMS or performance audits. The plan would encompass indicative audit topics for the future audits over a three year period, with a more specific and definite scoping of topics for the first year. It is recognised that there needs to be a degree of flexibility built into such a plan, to accommodate changing circumstances and priorities, which may cause the Auditor-General to vary the plan. 215
246. In developing a Strategic Audit Plan, they reviewers considered that QAO should first consult with the PAPWC, as well as potential audit clients and other affected parties, and consider their feedback, before finalising the plan. It is important that responsibility for the content of the plan, and for implementation, rests with the Auditor-General. In the public interest, the Strategic Audit Plan should be published on the QAO’s website. 216
247. They did not see a need for a formal process, such as that adopted in Victoria, however, they considered it desirable that there is a statutory requirement for the Auditor-General to publish a three-year Strategic Audit Plan. 217
213 Carpenter, G and Gray, M, Report of the 2010 Strategic Review of the Queensland Audit Office, March 2010: 107-108 214 Carpenter, G and Gray, M, Report of the 2010 Strategic Review of the Queensland Audit Office, March 2010: 107 215 Carpenter, G and Gray, M, Report of the 2010 Strategic Review of the Queensland Audit Office, March 2010: 108 216 Carpenter, G and Gray, M, Report of the 2010 Strategic Review of the Queensland Audit Office, March 2010: 108 217 Carpenter, G and Gray, M, Report of the 2010 Strategic Review of the Queensland Audit Office, March 2010: 108
44 Public Accounts and Public Works Committee Evaluation of Auditor-General’s PMS Audit Mandate
248. The 2010 Strategic Reviewers made the following recommendation in respect of a Strategic Audit Plan:
RN.16(ii) That the Auditor-General Act 2009 be amended to include a requirement for the QAO to prepare a three-year Strategic Audit Plan for Performance Audits, and to update the Plan each year.
RN.16(iii) That the Auditor-General and the Chair of the PAPWC agree on a protocol for consultation with the PAPWC on the draft Strategic Audit Plan prior to the commencement of each financial year.
RN.16(iv) That the Auditor-General consult with potential audit clients and other affected parties, and consider their feedback, prior to finalisation of the Strategic Audit Plan.
RN.16(v) That, following the passage of appropriation, the Auditor-General finalise the Strategic Audit Plan, and publish it on the QAO website.
RN.16(vi) That the Auditor-General Act 2009 be amended to require the Auditor-General, in undertaking performance audits, to take into consideration performance management standards and guidance issued by the government. 218
249. It should be noted that Auditor-General indicated in his response to the review report that the review report did not attempt to quantify the cost of implementing any of its recommendations. He advised that a preliminary assessment indicates that not all of the proposed initiatives that flow from the review report recommendations are likely to be cost neutral. He advised that the experience in other jurisdictions is that development of a three year strategic plan for performance audits involves a significant level of staff resources. He estimated this cost to be between $1m and 1.5m annually. 219
250. The committee has examined the actions adopted in other jurisdictions in respect of their strategic audit plans. These are included in the table in Appendix 5.
Committee comments
251. The committee considers that there is a need for the Auditor-General to be more transparent in the preparation of his audit plan. The preparation and publication of a Strategic Audit Plan will assist both stakeholders and the public on potential subjects for future audits. The committee is cognisant that there is a need for this plan to remain flexible to allow the Auditor-General to adjust to change priorities.
252. The committee agrees with the recommendations made by the 2010 Strategic Reviewers in respect of the preparation of a Strategic Audit plan, with the exception of the proposed protocol between the Auditor-General and the PAPWC.
218 Carpenter, G and Gray, M, Report of the 2010 Strategic Review of the Queensland Audit Office, March 2010: 121 219 Carpenter, G and Gray, M, Report of the 2010 Strategic Review of the Queensland Audit Office, March 2010: 174
45 Public Accounts and Public Works Committee Evaluation of Auditor-General’s PMS Audit Mandate
253. Whilst the committee agrees with the spirit of this recommendation, the committee is not convinced that there is a need for the Auditor-General and the Chair of the PAPWC to agree on a formal protocol for consultation with the committee on the draft Strategic Audit Plan prior to the commencement of each financial year. The committee believes that both the Auditor-General and the committee should be allowed flexibility in how the consultation process occurs. The committee considers that it should retain the option to come to its own agreed arrangements with the Auditor-General.
254. The committee considers that the requirement that the Auditor-General have regard to the audit priorities of the Parliament can be satisfied by a consultation process with the PAPWC. The committee envisages that it will seek feed back from other Parliamentary Committees, as occurs in the Commonwealth, as part of its consultation process to ensure a broad base analysis.
255. Even though the committee would be more involved in the consultation phase of the preparation of the Strategic Audit Plan, ultimately the Auditor-General must be responsible for the plan and its contents in order not to compromise the independence of the office.
Recommendation 5
The committee recommends that the Auditor-General Act 2009 be amended to allow for the provision of the following:
include a requirement for the QAO to prepare a three year Strategic Audit Plan for performance audits and to update the plan each year;
the Auditor-General consult with the PAPWC and consider their feedback, prior to finalisation of the Strategic Plan;
the Auditor-General consult with potential audit clients and affected parties, and consider their feedback, prior to finalisation of the Strategic Plan;
following the passage of appropriation, the Auditor-General finalise the Strategic Audit Plan and publish it on the QAO website;
in undertaking performance audits, the Auditor-General to take into consideration performance management standards and guidance issued by government.
Recommendation 6
The committee recommends that sufficient additional funding be provided to QAO to enable preparation of the Strategic Plan.
46
Appendix 1 – PMS audit reports tabled by QAO during the review period
Report Name Date tabled Agencies Examined
Report No 3 for 2005 – Results of 19 September 2005 Department of Emergency Services PMS audits of Output Performance Department of Natural Resources and Reporting Mines Department of Transport Disability Services Queensland
Report No 5 for 2005 – Results of 1 December 2005 Department of Education and the Arts PMS audits of Output Performance Department of Employment and Training Reporting – Phase 2 Department of Local Government, Planning, Sport and Recreation Department of Main Roads Department of Police Department of Primary Industries and Fisheries Department of State Development, Trade and Innovation
Report No 2 for 2006 – Results of 5 June 2006 Ergon Energy Corporation Ltd PMS audits of Government Owned Mackay Port Authority Corporations Performance Port of Brisbane Corporation Reporting Powerlink Queensland Queensland Rail Stanwell Corporation Ltd
Report No 5 for 2006 – Results of 1 November 2006 Department of Corrective Services PMS audits of capital works at Department of Education and the Arts Departments of Corrective Services, Department of Health Education, Health and Housing Department of Housing
Report No 6 for 2006 – Results of 13 November 2006 Department of Education, Training and PMS audit of Workforce Planning at the Arts the Department of Education, Department of Health Training and the Arts and the Department of Health
Report No 8 for 2006 – Results of 28 November 2006 Department of Corrective Services PMS audit of the Management of Department of Education and the Arts Departmental Fees and Charges Department of Emergency Services Department of Primary Industries and Fisheries Department of Transport
Report No 2 for 2007 – Results of 22 May 2007 Department of Child Safety PMS audits of Management of Department of Communities Funding to Non-Government Disability Services Queensland Organisations
Report No 4 for 2007 – Are 7 August 2007 Department of Employment and Industrial departmental output performance Relations measures, relevant, appropriate and Department of Health a fair representation of performance Department of Justice and Attorney- achievement? General Environmental Protection Agency
47 Public Accounts and Public Works Committee Evaluation of Auditor-General’s PMS Audit Mandate
Report Name Date tabled Agencies Examined
Report No 6 for 2007 – Beyond 26 September 2007 Department of Communities Agency Risk Department of Education, Training and the Arts Department of Housing Department of Infrastructure Department of Mines and Energy Department of State Development Department of Police Department of Transport Treasury Department
Report No 7 for 2007 – Addressing 13 November 2007 Department of Education, Training and Skills Shortages in Queensland the Arts Department of Employment and Industrial Relations Department of Tourism, Regional Development and Industry
Report No 8 for 2007 – South East 14 November 2007 Queensland Water Commission Queensland – Towards a Short- Term Water Balance
Report No 1 for 2008 – Enhancing 17 April 2008 Department of Communities Accountability through Annual Department of Corrective Services Reporting – a PMS audit Department of Education, Training and the Arts Department of Emergency Services Department of Health Department of Infrastructure Department of Premier and Cabinet Department of Transport Environmental Protection Agency Treasury Department
Report No 3 for 2008 – 15 May 2008 Department of Emergency Services Management of Rural Fire Services in Queensland – a PMS audit
Report No 5 for 2008 – Protecting 26 August 2008 Department of Primary Industries and Queensland’s primary industries Fisheries and environment from pests and Biosecurity Queensland disease – a PMS audit
Report No 6 for 2008 – Follow-up 7 October 2008 Department of Education, Training and audit of Workforce Planning at the Arts Departments of Education, Training Department of Health and the Arts and Health, Public Service Commission incorporating their responses to an Department of Employment and Industrial ageing workforce – a PMS audit Relations
48
Report Name Date tabled Agencies Examined
Report No 7 for 2008 – 28 October 2008 Brisbane City Council Administration of Grants and Cassowary Coast Regional Council Funding to Community Gladstone Regional Council Organisations by Local Government Gold Coast City Council in Queensland Goondiwindi Regional Council Ipswich City Council Logan City Council Longreach Regional Council Mt Isa City Council Redland City Council Townsville City Council Winton Shire Council
Report No 8 for 2008 – Follow-up of 12 November 2008 Department of Corrective Services selected audits tabled in 2006 – a Department of Education and the Arts PMS audit Department of Health Department of Housing Department of Emergency Services Department of Primary Industries and Fisheries Department of Transport
Report No 2 for 2009 – Health 9 June 2009 Department of Health service planning for the future
Report No 3 for 2009 – Transport 23 June 2009 Department of Transport network management and urban Department of Main Roads congestion in South East Department of Infrastructure and Planning Queensland Brisbane City Council TransLink Transit Authority
Report No 5 for 2009 – 29 July 2009 Department of Health Management of patient flow through Queensland hospitals
Report No 6 for 2009 – Providing 18 August 2009 Treasury Department the information required to make Department of Environment and Resource good regulation – a PMS audit Management Department of Justice and Attorney- General Department of Employment, Economic Development and Innovation
Report No 7 for 2009 – Follow-up 27 October 2009 Treasury Department on government owned corporation Ergon Energy Corporation Ltd and budget sector performance Queensland Rail measurement and reporting
49 Public Accounts and Public Works Committee Evaluation of Auditor-General’s PMS Audit Mandate
Appendix 2 – Copy of Terms of Reference
Inquiry into an evaluation of the effectiveness of the performance management systems (PMS) audit mandate in terms of the Parliament’s previously expressed position
Terms of reference
1. What is the role of the PMS audit mandate? Does it fulfil its function in the overall accountability process for the Parliament? What improvements has the mandate brought to the accountability systems?
2. Is the concept of PMS auditing understood by both auditor and auditee?
3. Is the PMS audit mandate being used effectively by QAO?
4. How does QAO assess if individual PMS audits have been effective?
5. How could the effectiveness of the PMS audit mandate be improved?
50
Appendix 3 – Submissions received
1. Department of Tourism, Regional Development and Industry
2. Department of Local Government, Sport and Recreation
3. Minister for Child Safety and Women
4. Queensland Health
5. Queensland Transport
6. Department of Primary Industries and Fisheries
7. Department of Emergency Services
8. Department of Education, Training and the Arts
9. Department of Justice and Attorney-General
10. Environmental Protection Agency
11. Department of Mines and Energy
12. Australasian Council of Auditors-General
13. Queensland Audit Office
14. Department of Employment and Industrial Relations
15. Queensland Treasury
16. Public Service Commission
17. Queensland Police Service
18. Department of Public Works
19. Institute of Chartered Accountants in Australia
20. Queensland Water Commission
21. Department of Infrastructure and Planning
22. Department of Premier and Cabinet
23. Confidential
51 Public Accounts and Public Works Committee Evaluation of Auditor-General’s PMS Audit Mandate
Appendix 4 – Public hearing held 18 September 2009
Name Title Organisation Mr Glenn Poole Auditor-General of Queensland Queensland Audit Office Ms Terry Campbell Assistant Auditor-General Queensland Audit Office Dr Pradeep Philip Associate Director-General Department of Premier and Cabinet Mr Michael Tennant Executive Director, Performance Department of Premier and Cabinet and Delivery Office Mr Gerard Bradley Under Treasurer Queensland Treasury Ms Sue Highland Director, Financial Management Queensland Treasury Branch Ms Kathy Schaefer Deputy Director-General, Strategy Department of Infrastructure and and Governance Planning Mr Gary White Deputy Director-General, Local Department of Infrastructure and Government and Planning Planning Mr David Stewart Director-General Department of Transport and Main Roads Mr Bruce Ollason Acting Deputy Director-General Department of Transport and Main Roads Mr Robert McDonald Senior Director, Assurance and Risk Queensland Health Advisory Services Mr Terry Mehan Deputy Director-General, Queensland Health Performance and Accountability Division
52 Public Accounts and Public Works Committee Evaluation of Auditor-General’s PMS Audit Mandate
Appendix 5 – Comparison of Auditor-General’s mandate in Australian jurisdictions
Qld Cwlth NSW Vic Tas WA SA NT ACT Act Auditor-General Auditor-General Public Finance Audit Act 1994; Audit Act 2008 Auditor Public Audit Act Auditor- Act 2009 Act 1997 & Audit Act Constitution General Act Finance and General Act Financial Public Accounts 1983 Act 1975; 2006 Audit Act 1987 1996 Accountability and Audit Parliamentary Financial Act 2009 Committee Act Committees Management 1951 Act 2003 Act 2006 Mandate PMS Performance Performance Performance Performance Performance Performance PMS Performance Terms of AG may conduct AG may at any AG may Perf audits AG is to AG may at any When AG may AG can mandate an audit of PMS time conduct a conduct an determine perform the time carry out conducting an conduct an conduct a of a public performance audit audit of all or whether an audits in an examination audit of the audit of PMS performance sector entity. which will review any particular authority’s manner as AG or investigation accounts of to determine if audit, being a Objective or examine any activity of an objectives, thinks fit as to efficiency each public PMS enable review or includes aspect of the authority to operations or having regard and authority the the agency to examination of deciding operations of determine activities of the to character & effectiveness. AG may assess any aspect of whether the person or body whether the whole or any effectiveness examine the whether its the operations PMS enable the subject to the authority is part of Vic of internal efficiency & objectives are of the person, entity to assess audit. carrying out public sector, control & economy with being body or thing. whether its those activities are being internal audit which a public achieved objectives are effectively & achieved entity or authority uses economically, being achieved doing so effectively, audited its resources. efficiently & economically, economically economically, subsidiary effectively. efficiently & & efficiently & efficiently & in entity. AG effectively. in compliance compliance may take into with all with all account any relevant laws. relevant Acts. other matter that affects efficiency, effectiveness or economy Report on N/A Not specified in Specified in Specified in Not specified Not specified N/A. N/A Not specified government Act but by Act - AG not to Act – AG is not in Act but by in Act but by Efficiency & in Act but by policy convention do not question the to question the convention do convention do economy convention do question merit of merits of policy merits of policy not question not question mandate only. not question govt policy. objectives. objectives. merit govt merit govt merit govt policy. policy. policy.
53 Public Accounts and Public Works Committee Evaluation of Auditor-General’s PMS Audit Mandate
Qld Cwlth NSW Vic Tas WA SA NT ACT Auditing Act specifies AG Set by AG by AG required to Act specifies AG is to Act specifies No specific No specific No specific Standards must prepare a notice in Gazette. have regard to that Standards perform the that the AG is mention in the mention in the mention in Act. report to Parl recognised made by the audits in to carry out an Act. Act. setting out professional Auditing & manner as AG audit in such standards to be standards & Assurance thinks fit manner as the applied to practices. Standards having regard AG thinks fit conduct of Board must be to Aust having regard audits & applied, as Auditing & to Auditing & selection, appropriate, in Assurance Assurance engagement & performance of Standards. Standards quality control of functions & work of contract exercise of auditors & state powers in extent to which relation to perf standards are in audits. accordance with auditing standards made by professional or statutory bodies. Audit No formal AG must have AG may have AG must Refer Audit AG must have No formal No formal No formal Priorities involvement regard to audit regard to confer with and Plan regard to audit involvement involvement involvement priorities of Parl whether there have regard to priorities of determined by the has been any any audit Parl JCPAA and any wastage of priorities determined by reports made by public determined by either House of JCPAA. Refer resources or PAEC Parl, PAC (LA) Audit Plan. any lack of or Estimates & probity or Financial financial Operations prudence in Committee mgt or appln of (LC). public resources Policy objectives not to be questioned.
54 Public Accounts and Public Works Committee Evaluation of Auditor-General’s PMS Audit Mandate
Qld Cwlth NSW Vic Tas WA SA NT ACT Audit Plan No formal JCPAA involved No formal Before beg of By 31/3 must No formal No formal No formal Published on involvement in consultation on involvement. each financial prepare involvement involvement involvement AG’s website the AG’s draft AG discusses year AG must annual plan of work program for his plan for the prepare annual proposed work following financial year with the plan of program & year, then PAC and proposed work submit PAC. referred to AG for invites input program & PAC must consideration in when he is submit to consider & preparing final starting an PAEC. PAEC may comment work program audit. must return it on it. PAC (subject to a basic Audit Plan with any must return screening process published on comments to with any to make sure the AG’s AG. After comments to topics suggested website. passage of AG by 31/5. are appropriate). annual AG must Work program for appropriation finalise after following financial acts, AG must considering year is then complete plan comments of published on after PAC. AG ANAO’s website. considering must indicate PAEC’s in annual plan comments & nature of any indicate nature changes of any changes suggested by suggested by PAC that AG PAEC not has not adopted. adopted. AG must Before begin present annual of financial plan to PAEC year AG must & transmit to present annual each House of plan to PAC & Parl for tabling. to President & Must be Speaker published on No person is AG’s website. to direct AG in relation to content of plan.
55 Public Accounts and Public Works Committee Evaluation of Auditor-General’s PMS Audit Mandate
56